From Mohammad.Rezaei at gs.com Mon Jun 1 02:42:51 2015 From: Mohammad.Rezaei at gs.com (Rezaei, Mohammad A.) Date: Sun, 31 May 2015 22:42:51 -0400 Subject: JEP 132: More-prompt finalization In-Reply-To: <556B61B0.5030708@gmail.com> References: <55674C6E.8050509@gmail.com> <5567CC5E.7000102@oracle.com> <8DBC7345-09D8-458F-9017-298D1E620F86@gmail.com> <6882C9A35DFB9B4FA2779F7BF5B9757D20761169A2@GSCMAMP06EX.firmwide.corp.gs.com> <556B61B0.5030708@gmail.com> Message-ID: <6882C9A35DFB9B4FA2779F7BF5B9757D20761169E3@GSCMAMP06EX.firmwide.corp.gs.com> The problems start with the ReferenceQueue object and the heavy synchronization in it. Consider structures like j.u.WeakHashMap that need to expunge entries. If such structures are made somewhat concurrent, the lock in ReferenceQueue starts to become a serious problem: - In structures that are concurrent at the entry level (like jdk 8's ConcurrentHashMap), if methods like get() or put() try to expunge, the lock in ReferenceQueue renders the structures non-concurrent. - In structures that are multi-reader-single-writer locked, read methods cannot expunge (because they have to promote to a writer), but they can't even check the queue, because that turns the multi-reader structure into a synchronized one. - In addition to expunge calls contending on the ReferenceQueue lock, ReferenceHandler thread can also contend on the same lock. - There is no fast clear() method on ReferenceQueue. That would be quite useful on a resize event. The above issues forced us to have a dedicated thread that does periodic expunging of References. This works ok under light load, but can fall behind under heavy load. Thanks Moh @Moh Can you elaborate some more on what twists were necessary or what problems you had? From staffan.larsen at oracle.com Mon Jun 1 07:18:51 2015 From: staffan.larsen at oracle.com (Staffan Larsen) Date: Mon, 1 Jun 2015 09:18:51 +0200 Subject: RFR(M, v10): JDK-8059036 : Implement Diagnostic Commands for heap and finalizerinfo In-Reply-To: <556AF3CD.50900@oracle.com> References: <554894E9.8020908@oracle.com> <1AF219C7-1055-4D97-BB46-2FA390C4135D@oracle.com> <5548D65C.8040304@gmail.com> <554A8CEE.2090009@oracle.com> <55523418.5010708@oracle.com> <55527935.5010408@gmail.com> <5553B552.9060900@oracle.com> <555453BA.2070205@gmail.com> <55567748.6020304@oracle.com> <55567A68.9020802@oracle.com> <5556F34B.5050701@oracle.com> <55573A31.7050306@gmail.com> <55573C86.8030807@gmail.com> <5559D85B.2050500@oracle.com> <6FB8DF19-1D67-4702-9642-4C630765D2D1@oracle.com> <555C2EEB.1050505@oracle.com> <3941B430-FE6C-4AF0-A875-76D24411C654@oracle.com> <5564802A.2010403@oracle.com> <5565738A.4040109@gmail.com> <7EB0BE71-1608-4047-B71D-1A9EAA9083BB@oracle.com> <556727A2.9060209@gmail.com> <55675908.6030406@oracle.com> <556AF3CD.50900@oracle.com> Message-ID: <37227441-B016-4A9A-B9CA-BE100E992C9A@oracle.com> Dmitry, Instead of hardcoding the field offsets, you can use InstanceKlass::find_field and fieldDescriptor::offset to find the offset at runtime. Thanks, /Staffan > On 31 maj 2015, at 13:43, Dmitry Samersoff wrote: > > Everyone, > > Please take a look at new version of the fix. > > http://cr.openjdk.java.net/~dsamersoff/JDK-8059036/webrev.10/ > > Changes (to previous version) are in > Finalizer.java and diagnosticCommand.cpp > > This version copy data from Map.Entry<> to array of > FinalizerHistogramEntry instances then, > VM prints content of this array. > > -Dmitry > > > On 2015-05-28 21:06, Mandy Chung wrote: >> >> On 05/28/2015 07:35 AM, Peter Levart wrote: >>> Hi Mandy, >>> >>> On 05/27/2015 03:32 PM, Mandy Chung wrote: >>>> Taking it further - is it simpler to return String[] of all >>>> classnames including the duplicated ones and have the VM do the >>>> count? Are you concerned with the size of the String[]? >>> >>> Yes, the histogram is much smaller than the list of all instances. >>> There can be millions of instances waiting in finalizer queue, but >>> only a few distinct classes. >> >> Do you happen to know what libraries are the major contributors to these >> millions of finalizers? >> >> It has been strongly recommended to avoid finalizers (see Effective Java >> Item 7). I'm not surprised that existing code is still using >> finalizers while we should really encourage them to migrate away from it. >> >>> What could be done in Java to simplify things in native code but still >>> not format the output is to convert the array of Map.Entry(s) into an >>> Object[] array of alternating {String, int[], String, int[], .... } >>> >>> Would this simplify native code for the price of a little extra work >>> in Java? The sizes of old and new arrays are not big (# of distinct >>> classes of finalizable objects in the queue). >> >> I also prefer writing in Java and writing less native code (much >> simplified). It's an interface that we have to agree upon and keep it >> simple. Having the returned Object[] as alternate String and int[] is a >> reasonable compromise. >> >> ReferenceQueue.java - you can move @SuppressWarning from the method to >> just the field variable "rn" >> @SuppressWarnings("unchecked") >> Reference rn = r.next; >> >> Finalizer.java >> It's better to rename printFinalizationQueue as it inspects the >> finalizer reference queue (maybe getFinalizerHistogram?). Can you move >> this method to the end of this class and add the comment saying that >> this is invoked by VM for jcmd -finalizerinfo support and @return to >> describe the returned content. I also think you can remove >> @SuppressWarnings for this method. >> >> Mandy > > > -- > Dmitry Samersoff > Oracle Java development team, Saint Petersburg, Russia > * I would love to change the world, but they won't give me the sources. From aleksey.shipilev at oracle.com Mon Jun 1 08:31:09 2015 From: aleksey.shipilev at oracle.com (Aleksey Shipilev) Date: Mon, 01 Jun 2015 11:31:09 +0300 Subject: JEP 254: Compact Strings In-Reply-To: References: <20150514230513.0363C62D05@eggemoggin.niobe.net> Message-ID: <556C184D.6020107@oracle.com> (getting back to this) Hi Jeremy, On 05/16/2015 03:34 AM, Jeremy Manson wrote: > So, I'm pretty dubious, mostly because of the risks mentioned in the JEP. > If you need a flag-check and two code paths for every String method, that's > going to make the String class more slow and bloated, and make it very > difficult for the JIT compiler to do its job inlining and intrinsifying > calls to String methods. Yes, we know that's a potential problem, e.g. outlined here: http://cr.openjdk.java.net/~shade/density/equals.txt The hope is that the string coder check would be amortized by the substantial performance improvement with the ubiquitous Latin1 (optimized) operations. Also, getting a few code generation quirks kicked out may further offset the perceived performance costs of doing this (you can do such a trick every so often, but not all the time). > The proposed change here has the potential of doing the opposite with most > String operations - trading off less GC overhead for more mutator cost. > But String operations are a pretty big chunk of CPU time, on the whole. The thing is, many mutator ops on Strings are also improved, because the data become more easily cacheable and/or require less steps to complete (think vectorization that takes 2x less instructions). > Does anyone really have a sense of how to make this kind of decision? The > JEP seems mostly to be hoping that other organizations will do the testing > for you. It is not true that JEP hopes to have other organizations to do testing for it. The JEP tries to illuminate that this is a performance-sensitive change, so early testing and feedback is very appreciated. So, if you have the String-intensive workloads in your org, can you try and run the prototype JDK against it? Our early runs on our workloads of interest show the appealing improvements. That is, the decision to integrate this is not done yet, as we don't have the complete performance picture and/or fully-tested prototype. In other words, there are quite a few blank spots to fill out. Your data may be the part of that picture when we decide to integrate in JDK 9. > (I agree that it is worth doing some experimentation in this area, but I > wanted to say this early, because if I could reach back in time and tell > you *not* to make the substring change, I would. We seriously considered > simply backing it out locally, but it would have been a lot of effort for > us to maintain that kind of patch, and we didn't want our performance > tradeoffs to be that much different from the stock JDK's.) This is your golden ticket: if you come back with concrete data in your hands saying that the particular tradeoff the JEP made is not sensible for your applications, it would be considered in the decision to integrate. But, it should be a real data and/or contrived benchmark simulating the real-world scenario, not just theoretical appeals -- we know how misguided those can get. Thanks, -Aleksey From Ulf.Zibis at CoSoCo.de Mon Jun 1 08:32:52 2015 From: Ulf.Zibis at CoSoCo.de (Ulf Zibis) Date: Mon, 01 Jun 2015 10:32:52 +0200 Subject: 8058779: Faster implementation of String.replace(CharSequence, CharSequence) In-Reply-To: <556B30EA.5050801@oracle.com> References: <556231BE.9050703@oracle.com> <5564FD92.2050408@oracle.com> <556503D8.6000303@oracle.com> <5565E2D6.2070806@oracle.com> <5565E667.4050601@oracle.com> <5565F988.7080209@oracle.com> <55660180.8030003@oracle.com> <556605E9.3010302@oracle.com> <55660805.1030007@oracle.com> <55661E7B.2010405@oracle.com> <556624BA.1030703@oracle.com> <556625E5.5040806@oracle.com> <556A6FB3.3060705@oracle.com> <556AC9C8.3070503@univ-mlv.fr> <556B2EBB.2080503@oracle.com> <556B30EA.5050801@oracle.com> Message-ID: <556C18B4.1020408@CoSoCo.de> Hi, Am 31.05.2015 um 18:03 schrieb Ivan Gerasimov: > > On 31.05.2015 18:54, Ivan Gerasimov wrote: >> >> I fixed the code and added the case to the regression test in the new webrev. >> > Which is right here: > http://cr.openjdk.java.net/~igerasim/8058779/05/webrev/ Shoudn't the user be informed via javadoc about the risk of an OutOfMemoryError, just from illegal arguments of this method? Example: this.value.length() = 100 target.value.length() = 200 replacement.value.length() = 50 results in: newLenHint = -500 --> OutOfMemoryError In other words, is this a good reason to throw such an Error? Little nit: Indentation for line continuation should be 8 spaces. -Ulf From paul.sandoz at oracle.com Mon Jun 1 08:33:57 2015 From: paul.sandoz at oracle.com (Paul Sandoz) Date: Mon, 1 Jun 2015 10:33:57 +0200 Subject: 8058779: Faster implementation of String.replace(CharSequence, CharSequence) In-Reply-To: <556B30EA.5050801@oracle.com> References: <556231BE.9050703@oracle.com> <5564FD92.2050408@oracle.com> <556503D8.6000303@oracle.com> <5565E2D6.2070806@oracle.com> <5565E667.4050601@oracle.com> <5565F988.7080209@oracle.com> <55660180.8030003@oracle.com> <556605E9.3010302@oracle.com> <55660805.1030007@oracle.com> <55661E7B.2010405@oracle.com> <556624BA.1030703@oracle.com> <556625E5.5040806@oracle.com> <556A6FB3.3060705@oracle.com> <556AC9C8.3070503@univ-mlv.fr> <556B2EBB.2080503@oracle.com> <556B30EA.5050801@oracle.com> Message-ID: <7E00911B-3B67-4A6F-AB73-428AD0781D21@oracle.com> On May 31, 2015, at 6:03 PM, Ivan Gerasimov wrote: >> > Which is right here: > http://cr.openjdk.java.net/~igerasim/8058779/05/webrev/ > Much better. For the test can you use RandomFactory recently added to the test library? Paul. From aleksey.shipilev at oracle.com Mon Jun 1 08:50:06 2015 From: aleksey.shipilev at oracle.com (Aleksey Shipilev) Date: Mon, 01 Jun 2015 11:50:06 +0300 Subject: JEP 254: Compact Strings In-Reply-To: References: <20150514230513.0363C62D05@eggemoggin.niobe.net> <6882C9A35DFB9B4FA2779F7BF5B9757D20761165B8@GSCMAMP06EX.firmwide.corp.gs.com> Message-ID: <556C1CBE.8070401@oracle.com> On 05/18/2015 05:35 PM, Vitaly Davidovich wrote: > This part is a bit unclear for the proposed changes. While it's true that > single byte encoding will be denser than two byte, most string ops end up > walking the backing store linearly; prefetch (either implicit h/w or > software-assisted) could hide the memory access latency. It will still pollute the caches though, and generally incur more instructions to be executed (e.g. think about the vectorized scan of the char[] array -- the compressed version will take 2x less instructions). > Personally, what I'd like to see is fusing storage of String with its > backing data, irrespective of encoding (i.e. removing the indirection to > fetch the char[] or byte[]). This is not the target for this JEP, and the groundwork for String-char[] fusion is handled elsewhere (I put my hopes at Valhalla that will explore the exact path to add the "exotic" object shapes into the runtime). String-char[] fusion neither conflicts with the Compact String optimization, nor provides the alternative. Removing the "excess" headers from backing char[] array would solve the "static" overhead in Strings, while the String compaction would further compact the backing storage. Thanks, -Aleksey. From peter.levart at gmail.com Mon Jun 1 09:04:11 2015 From: peter.levart at gmail.com (Peter Levart) Date: Mon, 01 Jun 2015 11:04:11 +0200 Subject: JEP 132: More-prompt finalization In-Reply-To: <6882C9A35DFB9B4FA2779F7BF5B9757D20761169E3@GSCMAMP06EX.firmwide.corp.gs.com> References: <55674C6E.8050509@gmail.com> <5567CC5E.7000102@oracle.com> <8DBC7345-09D8-458F-9017-298D1E620F86@gmail.com> <6882C9A35DFB9B4FA2779F7BF5B9757D20761169A2@GSCMAMP06EX.firmwide.corp.gs.com> <556B61B0.5030708@gmail.com> <6882C9A35DFB9B4FA2779F7BF5B9757D20761169E3@GSCMAMP06EX.firmwide.corp.gs.com> Message-ID: <556C200B.3030603@gmail.com> Hi Moh, On 06/01/2015 04:42 AM, Rezaei, Mohammad A. wrote: > > The problems start with the ReferenceQueue object and the heavy > synchronization in it. Consider structures like j.u.WeakHashMap that > need to expunge entries. If such structures are made somewhat > concurrent, the lock in ReferenceQueue starts to become a serious problem: > > -In structures that are concurrent at the entry level (like jdk 8?s > ConcurrentHashMap), if methods like get() or put() try to expunge, the > lock in ReferenceQueue renders the structures non-concurrent. > The presented prototype changes the implementation of ReferenceQueue. It doesn't use a lock any more for enqueu-ing when there are no waiters and never for poll-ing. ReferenceQueue.poll() is a single volatile read when queue is empty and a read followed by CAS when it is not (with retries when contended). If there is a desire and new API could be extended, the method like the following: Iterator> pollChunk(int maxChunkSize); ...could return a chunk of enqueued references (or an empty iterator) so that the reduced number of CAS instructions per de-queued reference could be traded for the greater probability of retries because of contention. > -In structures that are multi-reader-single-writer locked, read > methods cannot expunge (because they have to promote to a writer), but > they can?t even check the queue, because that turns the multi-reader > structure into a synchronized one. > By checking you mean poll() which also de-queues a reference if available? What do you do when this happens. Would you need a peek() method maybe? > -In addition to expunge calls contending on the ReferenceQueue lock, > ReferenceHandler thread can also contend on the same lock. > That's right. And in the presented prototype, this is minimized by allowing ReferenceHandler thread to enqueue a chunk of pre-prepared references in one go, minimizing the need to frequently notify any possible waiters via a lock.notifyAll(). > -There is no fast clear() method on ReferenceQueue. That would be > quite useful on a resize event. > This would be easy to implement if new API could be added. > The above issues forced us to have a dedicated thread that does > periodic expunging of References. This works ok under light load, but > can fall behind under heavy load. > Because of the overhead/bottleneck of the reference processing I assume. It would be great if you could check whether the prototype in webrev.02 improves your situation. It should be simple. Just compile the sources and prepend the resulting classes to the bootclasspath of the JDK. Regards, Peter > Thanks > > Moh > > @Moh > > Can you elaborate some more on what twists were necessary or what > problems you had? > > From forax at univ-mlv.fr Mon Jun 1 08:57:02 2015 From: forax at univ-mlv.fr (=?ISO-8859-1?Q?R=E9mi_Forax?=) Date: Mon, 01 Jun 2015 08:57:02 +0000 Subject: 8058779: Faster implementation of String.replace(CharSequence, CharSequence) In-Reply-To: <556B2EBB.2080503@oracle.com> References: <556231BE.9050703@oracle.com> <5564FD92.2050408@oracle.com> <556503D8.6000303@oracle.com> <5565E2D6.2070806@oracle.com> <5565E667.4050601@oracle.com> <5565F988.7080209@oracle.com> <55660180.8030003@oracle.com> <556605E9.3010302@oracle.com> <55660805.1030007@oracle.com> <55661E7B.2010405@oracle.com> <556624BA.1030703@oracle.com> <556625E5.5040806@oracle.com> <556A6FB3.3060705@oracle.com> <556AC9C8.3070503@univ-mlv.fr> <556B2EBB.2080503@oracle.com> Message-ID: <44788D53-E059-45CE-BB1B-956C06FA272B@univ-mlv.fr> Hi Ivan, Le 31 mai 2015 17:54:35 CEST, Ivan Gerasimov a ?crit : >Hi Remi! > >On 31.05.2015 11:43, Remi Forax wrote: >> I agree with the others the code is more readable. >> >> There is a slightly difference between the current behavior and the >> behavior of the proposed code, >> "foo".replace("bar", null) >> should throw a NPE because replacement is null but the proposed code >> will return "foo" >> because replacement.toString() is called after the short-circuit test > >> (j < 0). >> >Yes, you're right, thanks for catching it! >And the regression test should have caught that, but I only had >"a".replace("a", null) there, which passed. > >I fixed the code and added the case to the regression test in the new >webrev. > >> so the first part of the code should be: >> String starget = target.toString(); // implict nullcheck >> String srepl = replacement.toString(); // implicit nullcheck >> int j = indexOf(starget); >> if (j < 0) { >> return this; >> } >> ... >> >> also 'i' can be initialized just before the 'do', there is no point >to >> initialize it before. >> >> To finish, the two 'final' are useless here but i suppose it's a >> matter of style :) >> >I moved declaration of i just to save a line. I don't think it >decreased performance. I don't think so too. It decrease readability, when you read the code of the loop you have to scroll up to find the initialization of 'i' like in plain old good C. > >Declaring the 'value' variable final was suggested by Martin, and I >think it is reasonable (see >http://mail.openjdk.java.net/pipermail/core-libs-dev/2015-March/032601.html). >The other variable is final just for symmetry :) Either i read the code in an IDE and local variables and fields do not have the same color or i read diff and in that case fields are usually not visible. For me, final on local variable is just noise. But as i said it's a matter of taste. > >Sincerely yours, >Ivan cheers, R?mi > > >> cheers, >> R?mi >> >> >> On 05/31/2015 04:19 AM, Ivan Gerasimov wrote: >>> Hi everyone! >>> >>> Here's another webrev, in which replace() is implemented with >>> StringBuilder. >>> On my benchmark it is almost as fast as the version backed with >>> arrays, but this variant is much shorter. >>> >>> Credits to Sherman for combining the general algorithm with the case > >>> of empty target. >>> >>> Comments, further suggestions are welcome! >>> >>> BUGURL: https://bugs.openjdk.java.net/browse/JDK-8058779 >>> WEBREV: http://cr.openjdk.java.net/~igerasim/8058779/04/webrev/ >>> >>> Sincerely yours, >>> Ivan >>> >> >> >> From peter.levart at gmail.com Mon Jun 1 12:09:19 2015 From: peter.levart at gmail.com (Peter Levart) Date: Mon, 01 Jun 2015 14:09:19 +0200 Subject: 8058779: Faster implementation of String.replace(CharSequence, CharSequence) In-Reply-To: <556C18B4.1020408@CoSoCo.de> References: <556231BE.9050703@oracle.com> <5564FD92.2050408@oracle.com> <556503D8.6000303@oracle.com> <5565E2D6.2070806@oracle.com> <5565E667.4050601@oracle.com> <5565F988.7080209@oracle.com> <55660180.8030003@oracle.com> <556605E9.3010302@oracle.com> <55660805.1030007@oracle.com> <55661E7B.2010405@oracle.com> <556624BA.1030703@oracle.com> <556625E5.5040806@oracle.com> <556A6FB3.3060705@oracle.com> <556AC9C8.3070503@univ-mlv.fr> <556B2EBB.2080503@oracle.com> <556B30EA.5050801@oracle.com> <556C18B4.1020408@CoSoCo.de> Message-ID: <556C4B6F.8070201@gmail.com> On 06/01/2015 10:32 AM, Ulf Zibis wrote: > Hi, > > Am 31.05.2015 um 18:03 schrieb Ivan Gerasimov: >> >> On 31.05.2015 18:54, Ivan Gerasimov wrote: >>> >>> I fixed the code and added the case to the regression test in the >>> new webrev. >>> >> Which is right here: >> http://cr.openjdk.java.net/~igerasim/8058779/05/webrev/ > > Shoudn't the user be informed via javadoc about the risk of an > OutOfMemoryError, just from illegal arguments of this method? > Example: > this.value.length() = 100 > target.value.length() = 200 > replacement.value.length() = 50 > results in: ...no replacements made, since target.value.length() > this.value.length() and 1st indexOf returns -1... if 1st indexOf returns >= 0, then this.value.length() >= target.value.length() and the only way to get negative newLenHint is via an overflow indicating that a String with length() > Integer.MAX_VALUE would be required to hold the result, which is impossible. We don't have a special StringTooBigException. OOME is the best approximation here. IllegalArgumentException is another option, but seems to be to general in this case. What happens when regexp based replace is fed with such huge strings? I had to set the max. heap size to 14Gbytes to get the answer that was not an OOME caused by not enough heap space: char[] chars = new char[1 + (1 << 30)]; chars[0] = 'a'; String big = new String(chars); big.replace("a", big); Exception in thread "main" java.lang.OutOfMemoryError at java.lang.AbstractStringBuilder.expandCapacity(AbstractStringBuilder.java:137) at java.lang.AbstractStringBuilder.ensureCapacityInternal(AbstractStringBuilder.java:124) at java.lang.AbstractStringBuilder.append(AbstractStringBuilder.java:517) at java.lang.StringBuilder.append(StringBuilder.java:175) at java.util.regex.Matcher.appendTail(Matcher.java:1122) at java.util.regex.Matcher.replaceAll(Matcher.java:1169) at TestReplace.replace(TestReplace.java:11) at TestReplace.main(TestReplace.java:18) at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method) at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62) at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) at java.lang.reflect.Method.invoke(Method.java:502) at com.intellij.rt.execution.application.AppMain.main(AppMain.java:140) Regards, Peter > newLenHint = -500 --> OutOfMemoryError > In other words, is this a good reason to throw such an Error? > > Little nit: > Indentation for line continuation should be 8 spaces. > > -Ulf > From vitalyd at gmail.com Mon Jun 1 12:54:54 2015 From: vitalyd at gmail.com (Vitaly Davidovich) Date: Mon, 1 Jun 2015 08:54:54 -0400 Subject: JEP 254: Compact Strings In-Reply-To: <556C1CBE.8070401@oracle.com> References: <20150514230513.0363C62D05@eggemoggin.niobe.net> <6882C9A35DFB9B4FA2779F7BF5B9757D20761165B8@GSCMAMP06EX.firmwide.corp.gs.com> <556C1CBE.8070401@oracle.com> Message-ID: Hi Aleksey, While it's true that the denser format will require fewer cachelines, my experience is that most strings are smaller than a single cacheline worth of storage, maybe two lines in some cases; there's just a ton of them in the heap. So the heap footprint should be substantially reduced, but I'm not sure the cache pollution will be significantly reduced. There's currently no vectorization of char[] scanning (or any vectorization other than memcpy for that matter) - are you referring to the recent Intel contributions here or there's a plan to further improve vectorization in time for this JEP? Just curious. I agree that string fusion is separate from this change, and we've discussed this before. It just seems to me like that's a bigger perf problem today since even tiny/small strings (very common, IME) incur the indirection and bloat overhead, so would have liked to see that addressed first. If you're saying that's fully on valhalla's plate, ok, but I haven't seen anything proposed there yet. Thanks sent from my phone On Jun 1, 2015 4:50 AM, "Aleksey Shipilev" wrote: > On 05/18/2015 05:35 PM, Vitaly Davidovich wrote: > > This part is a bit unclear for the proposed changes. While it's true > that > > single byte encoding will be denser than two byte, most string ops end up > > walking the backing store linearly; prefetch (either implicit h/w or > > software-assisted) could hide the memory access latency. > > It will still pollute the caches though, and generally incur more > instructions to be executed (e.g. think about the vectorized scan of the > char[] array -- the compressed version will take 2x less instructions). > > > > Personally, what I'd like to see is fusing storage of String with its > > backing data, irrespective of encoding (i.e. removing the indirection to > > fetch the char[] or byte[]). > > This is not the target for this JEP, and the groundwork for > String-char[] fusion is handled elsewhere (I put my hopes at Valhalla > that will explore the exact path to add the "exotic" object shapes into > the runtime). > > String-char[] fusion neither conflicts with the Compact String > optimization, nor provides the alternative. Removing the "excess" > headers from backing char[] array would solve the "static" overhead in > Strings, while the String compaction would further compact the backing > storage. > > Thanks, > -Aleksey. > > > From aleksey.shipilev at oracle.com Mon Jun 1 13:31:07 2015 From: aleksey.shipilev at oracle.com (Aleksey Shipilev) Date: Mon, 01 Jun 2015 16:31:07 +0300 Subject: JEP 254: Compact Strings In-Reply-To: References: <20150514230513.0363C62D05@eggemoggin.niobe.net> <6882C9A35DFB9B4FA2779F7BF5B9757D20761165B8@GSCMAMP06EX.firmwide.corp.gs.com> <556C1CBE.8070401@oracle.com> Message-ID: <556C5E9B.9070302@oracle.com> On 06/01/2015 03:54 PM, Vitaly Davidovich wrote: > While it's true that the denser format will require fewer cachelines, my > experience is that most strings are smaller than a single cacheline > worth of storage, maybe two lines in some cases; there's just a ton of > them in the heap. So the heap footprint should be substantially > reduced, but I'm not sure the cache pollution will be significantly reduced. This calculation assumes object allocations are granular to the cache lines. They are not: if String takes less space within the cache line, it allows *more* object data to be squeezed there. In other words, with compact Strings, the entire dataset can take less cache lines, thus improving performance. > There's currently no vectorization of char[] scanning (or any > vectorization other than memcpy for that matter) - are you referring to > the recent Intel contributions here or there's a plan to further improve > vectorization in time for this JEP? Just curious. String methods are intensely intrinsified (and vectorized in those implementations). String::equals, String::compareTo, and some encoding/decoding come to mind. I really, really invite you to read the collateral materials from the JEP, where we explored quite a few performance characteristics already. Thanks, -Aleksey. From Roger.Riggs at Oracle.com Mon Jun 1 13:32:50 2015 From: Roger.Riggs at Oracle.com (Roger Riggs) Date: Mon, 01 Jun 2015 09:32:50 -0400 Subject: RFR 9: 8081565 : javac lint warnings in jdk testlibrary Message-ID: <556C5F02.7000103@Oracle.com> Please review test library changes to remove javac lint warnings. And 1 test that incorrectly used try-with-resources. Webrev: http://cr.openjdk.java.net/~rriggs/webrev-testlint-8081565/ Issue: https://bugs.openjdk.java.net/browse/JDK-8081565 Thanks, Roger From vitalyd at gmail.com Mon Jun 1 13:55:45 2015 From: vitalyd at gmail.com (Vitaly Davidovich) Date: Mon, 1 Jun 2015 09:55:45 -0400 Subject: JEP 254: Compact Strings In-Reply-To: <556C5E9B.9070302@oracle.com> References: <20150514230513.0363C62D05@eggemoggin.niobe.net> <6882C9A35DFB9B4FA2779F7BF5B9757D20761165B8@GSCMAMP06EX.firmwide.corp.gs.com> <556C1CBE.8070401@oracle.com> <556C5E9B.9070302@oracle.com> Message-ID: My calculation doesn't assume cacheline granularity; I'm looking at strictly the strings. What's allocated next to/around them is completely arbitrary, circumstantial, uncontrollable to a large extent, and often not repeatable. If you're claiming that some second or even third order locality effects will be measurable, I don't know how :). I'm sure there will be some as theoretically it's possible, but it'll be hard to demonstrate that on anything other than specially crafted microbenchmarks. Ok, you're talking about some string intrinsics and not general char[] being vectorized - fair enough. sent from my phone On Jun 1, 2015 9:31 AM, "Aleksey Shipilev" wrote: > On 06/01/2015 03:54 PM, Vitaly Davidovich wrote: > > While it's true that the denser format will require fewer cachelines, my > > experience is that most strings are smaller than a single cacheline > > worth of storage, maybe two lines in some cases; there's just a ton of > > them in the heap. So the heap footprint should be substantially > > reduced, but I'm not sure the cache pollution will be significantly > reduced. > > This calculation assumes object allocations are granular to the cache > lines. They are not: if String takes less space within the cache line, > it allows *more* object data to be squeezed there. In other words, with > compact Strings, the entire dataset can take less cache lines, thus > improving performance. > > > > There's currently no vectorization of char[] scanning (or any > > vectorization other than memcpy for that matter) - are you referring to > > the recent Intel contributions here or there's a plan to further improve > > vectorization in time for this JEP? Just curious. > > String methods are intensely intrinsified (and vectorized in those > implementations). String::equals, String::compareTo, and some > encoding/decoding come to mind. > > I really, really invite you to read the collateral materials from the > JEP, where we explored quite a few performance characteristics already. > > > Thanks, > -Aleksey. > > > From Roger.Riggs at Oracle.com Mon Jun 1 13:57:05 2015 From: Roger.Riggs at Oracle.com (Roger Riggs) Date: Mon, 01 Jun 2015 09:57:05 -0400 Subject: RFR 9: 8081566: java/lang/ProcessHandle/InfoTest.java failed on case sensitive command Message-ID: <556C64B1.7050804@Oracle.com> Please review this testbug fix to correctly check the reported command name. On Windows the command filename may differ only by case and should be checked to see if they are the same file, not just the same string. webrev: http://cr.openjdk.java.net/~rriggs/webrev-infotest-8081566/ Issue: https://bugs.openjdk.java.net/browse/JDK-8081566 Thanks, Roger From lance.andersen at oracle.com Mon Jun 1 14:09:10 2015 From: lance.andersen at oracle.com (Lance Andersen) Date: Mon, 1 Jun 2015 10:09:10 -0400 Subject: RFR 9: 8081566: java/lang/ProcessHandle/InfoTest.java failed on case sensitive command In-Reply-To: <556C64B1.7050804@Oracle.com> References: <556C64B1.7050804@Oracle.com> Message-ID: <7A319F9B-12CC-447F-8B76-01A4942313A5@oracle.com> Hi Roger, The changes all seem reasonable Best Lance On Jun 1, 2015, at 9:57 AM, Roger Riggs wrote: > Please review this testbug fix to correctly check the reported command name. > On Windows the command filename may differ only by case > and should be checked to see if they are the same file, not just the same string. > > webrev: > http://cr.openjdk.java.net/~rriggs/webrev-infotest-8081566/ > > Issue: > https://bugs.openjdk.java.net/browse/JDK-8081566 > > Thanks, Roger > Lance Andersen| Principal Member of Technical Staff | +1.781.442.2037 Oracle Java Engineering 1 Network Drive Burlington, MA 01803 Lance.Andersen at oracle.com From lance.andersen at oracle.com Mon Jun 1 14:10:08 2015 From: lance.andersen at oracle.com (Lance Andersen) Date: Mon, 1 Jun 2015 10:10:08 -0400 Subject: RFR 9: 8081565 : javac lint warnings in jdk testlibrary In-Reply-To: <556C5F02.7000103@Oracle.com> References: <556C5F02.7000103@Oracle.com> Message-ID: <5F4CFC4A-1C5E-4F76-B0C5-FD99241E12EB@oracle.com> Hi Roger, These changes also look fine Best Lance On Jun 1, 2015, at 9:32 AM, Roger Riggs wrote: > Please review test library changes to remove javac lint warnings. > And 1 test that incorrectly used try-with-resources. > > Webrev: > http://cr.openjdk.java.net/~rriggs/webrev-testlint-8081565/ > > Issue: > https://bugs.openjdk.java.net/browse/JDK-8081565 > > Thanks, Roger > > > > Lance Andersen| Principal Member of Technical Staff | +1.781.442.2037 Oracle Java Engineering 1 Network Drive Burlington, MA 01803 Lance.Andersen at oracle.com From Alan.Bateman at oracle.com Mon Jun 1 14:17:31 2015 From: Alan.Bateman at oracle.com (Alan Bateman) Date: Mon, 01 Jun 2015 15:17:31 +0100 Subject: RFR 9: 8081566: java/lang/ProcessHandle/InfoTest.java failed on case sensitive command In-Reply-To: <556C64B1.7050804@Oracle.com> References: <556C64B1.7050804@Oracle.com> Message-ID: <556C697B.1000707@oracle.com> On 01/06/2015 14:57, Roger Riggs wrote: > Please review this testbug fix to correctly check the reported command > name. > On Windows the command filename may differ only by case > and should be checked to see if they are the same file, not just the > same string. File path comparison is case insensitive on Windows so equals would work if the path Strings were obtaining from actual file paths. This doesn't seem to be case here so using isSameFile make sense. -Alan. From Mohammad.Rezaei at gs.com Mon Jun 1 15:03:49 2015 From: Mohammad.Rezaei at gs.com (Rezaei, Mohammad A.) Date: Mon, 1 Jun 2015 11:03:49 -0400 Subject: JEP 132: More-prompt finalization In-Reply-To: <556C200B.3030603@gmail.com> References: <55674C6E.8050509@gmail.com> <5567CC5E.7000102@oracle.com> <8DBC7345-09D8-458F-9017-298D1E620F86@gmail.com> <6882C9A35DFB9B4FA2779F7BF5B9757D20761169A2@GSCMAMP06EX.firmwide.corp.gs.com> <556B61B0.5030708@gmail.com> <6882C9A35DFB9B4FA2779F7BF5B9757D20761169E3@GSCMAMP06EX.firmwide.corp.gs.com> <556C200B.3030603@gmail.com> Message-ID: <6882C9A35DFB9B4FA2779F7BF5B9757D20761169FE@GSCMAMP06EX.firmwide.corp.gs.com> I read the code in the prototype and I liked what I saw (even excited!), which is the reason I spoke up. On the API side, I was mulling over what the addition of bulk methods and even just a size() method could do to help and it struck me: the JDK has come a long way since 1.0 and there are now fully fledged, well thought out queue implementations. Is there a reason ReferenceQueue can't be upgraded to be one of those (api and/or implementation wise)? I hope given all the reasoning that's gone into the canonical queue implementations there wouldn't be a lot of reason to debate their usefulness. But just as an example, let's consider having access to a fast, roughly accurate ReferenceQueue.size() method for the hash structures use case. A hash structure has to find the freed instances and remove them. If size() returns a large portion of the total entries, it would be better to re-hash the structure and bulk clear the references instead. Unfortunately, I don't have a great benchmark, so I can't run the prototype quickly. I'll have to put some time aside to figure out a good way to represent the situation before I can come back with meaningful numbers. Thanks Moh From: Peter Levart [mailto:peter.levart at gmail.com] Sent: Monday, June 01, 2015 5:04 AM To: Rezaei, Mohammad A. [Tech] Cc: 'hotspot-gc-dev at openjdk.java.net openjdk.java.net'; 'core-libs-dev at openjdk.java.net' Subject: Re: JEP 132: More-prompt finalization Hi Moh, On 06/01/2015 04:42 AM, Rezaei, Mohammad A. wrote: The problems start with the ReferenceQueue object and the heavy synchronization in it. Consider structures like j.u.WeakHashMap that need to expunge entries. If such structures are made somewhat concurrent, the lock in ReferenceQueue starts to become a serious problem: - In structures that are concurrent at the entry level (like jdk 8's ConcurrentHashMap), if methods like get() or put() try to expunge, the lock in ReferenceQueue renders the structures non-concurrent. The presented prototype changes the implementation of ReferenceQueue. It doesn't use a lock any more for enqueu-ing when there are no waiters and never for poll-ing. ReferenceQueue.poll() is a single volatile read when queue is empty and a read followed by CAS when it is not (with retries when contended). If there is a desire and new API could be extended, the method like the following: Iterator> pollChunk(int maxChunkSize); ...could return a chunk of enqueued references (or an empty iterator) so that the reduced number of CAS instructions per de-queued reference could be traded for the greater probability of retries because of contention. - In structures that are multi-reader-single-writer locked, read methods cannot expunge (because they have to promote to a writer), but they can't even check the queue, because that turns the multi-reader structure into a synchronized one. By checking you mean poll() which also de-queues a reference if available? What do you do when this happens. Would you need a peek() method maybe? - In addition to expunge calls contending on the ReferenceQueue lock, ReferenceHandler thread can also contend on the same lock. That's right. And in the presented prototype, this is minimized by allowing ReferenceHandler thread to enqueue a chunk of pre-prepared references in one go, minimizing the need to frequently notify any possible waiters via a lock.notifyAll(). - There is no fast clear() method on ReferenceQueue. That would be quite useful on a resize event. This would be easy to implement if new API could be added. The above issues forced us to have a dedicated thread that does periodic expunging of References. This works ok under light load, but can fall behind under heavy load. Because of the overhead/bottleneck of the reference processing I assume. It would be great if you could check whether the prototype in webrev.02 improves your situation. It should be simple. Just compile the sources and prepend the resulting classes to the bootclasspath of the JDK. Regards, Peter Thanks Moh @Moh Can you elaborate some more on what twists were necessary or what problems you had? From konstantin.shefov at oracle.com Mon Jun 1 16:22:30 2015 From: konstantin.shefov at oracle.com (Konstantin Shefov) Date: Mon, 01 Jun 2015 19:22:30 +0300 Subject: [8u-dev] Review request : JDK-8062904: TEST_BUG: Tests java/lang/invoke/LFCaching fail when run with -Xcomp option In-Reply-To: <55685264.20001@oracle.com> References: <5565D9BB.60003@oracle.com> <5565F6D5.7030900@oracle.com> <5566D9D8.4090401@oracle.com> <55675CCB.8010506@oracle.com> <5568281B.7040108@oracle.com> <55685264.20001@oracle.com> Message-ID: <556C86C6.2000409@oracle.com> Hi Vladmir It seems we have only this bug that manifests the problem. As I understand, this is a product issue, not test. -Konstantin On 29.05.2015 14:49, Vladimir Ivanov wrote: >> What do you mean by ignore code cache overflow? Do you mean I should fix >> the test to ignore these errors or should I leave this test unfixed >> because it is a product related issue? > The former. How reliable the test is if it ignores > NoSuchMethodException & VirtualMachineError? Are there other > manifestations of the problem? > > Best regards, > Vladimir Ivanov > >> On 28.05.2015 21:22, Vladimir Ivanov wrote: >>> Got it, thanks. >>> >>> Can we ignore errors caused by code cache overflow for now? >>> >>> Best regards, >>> Vladimir Ivanov >>> >>> On 5/28/15 12:03 PM, Konstantin Shefov wrote: >>>> Vladimir, >>>> >>>> This fix is not for timeout issue, this fix is for >>>> "java.lang.VirtualMachineError: out of space in CodeCache for >>>> adapters". >>>> >>>> Timeout issue is other bug and should be filed separately. >>>> I do not know why SQE added RULES with timeout to this bug. >>>> >>>> By the way, if -Xcomp is set on JDK 8u, test works if not more than >>>> one >>>> iteration is allowed. The same thing was for JDK 9 until >>>> JDK-8046809 had >>>> been fixed. >>>> >>>> -Konstantin >>>> >>>> On 27.05.2015 19:54, Vladimir Ivanov wrote: >>>>> Have you tried to reduce iteration granularity? >>>>> >>>>> Probably, checking execution duration on every test case is more >>>>> robust. >>>>> >>>>> Best regards, >>>>> Vladimir Ivanov >>>>> >>>>> On 5/27/15 5:50 PM, Konstantin Shefov wrote: >>>>>> Hello, >>>>>> >>>>>> Please review the test bug fix >>>>>> https://bugs.openjdk.java.net/browse/JDK-8062904 >>>>>> Webrev is http://cr.openjdk.java.net/~kshefov/8062904/webrev.01/ >>>>>> Test fails only against JDK 8u and passes against JDK 9. >>>>>> >>>>>> Thanks >>>>>> >>>>>> -Konstantin >>>> >> From ivan.gerasimov at oracle.com Mon Jun 1 18:53:26 2015 From: ivan.gerasimov at oracle.com (Ivan Gerasimov) Date: Mon, 01 Jun 2015 21:53:26 +0300 Subject: 8058779: Faster implementation of String.replace(CharSequence, CharSequence) In-Reply-To: <7E00911B-3B67-4A6F-AB73-428AD0781D21@oracle.com> References: <556231BE.9050703@oracle.com> <5564FD92.2050408@oracle.com> <556503D8.6000303@oracle.com> <5565E2D6.2070806@oracle.com> <5565E667.4050601@oracle.com> <5565F988.7080209@oracle.com> <55660180.8030003@oracle.com> <556605E9.3010302@oracle.com> <55660805.1030007@oracle.com> <55661E7B.2010405@oracle.com> <556624BA.1030703@oracle.com> <556625E5.5040806@oracle.com> <556A6FB3.3060705@oracle.com> <556AC9C8.3070503@univ-mlv.fr> <556B2EBB.2080503@oracle.com> <556B30EA.5050801@oracle.com> <7E00911B-3B67-4A6F-AB73-428AD0781D21@oracle.com> Message-ID: <556CAA26.2050303@oracle.com> On 01.06.2015 11:33, Paul Sandoz wrote: > On May 31, 2015, at 6:03 PM, Ivan Gerasimov wrote: >> Which is right here: >> http://cr.openjdk.java.net/~igerasim/8058779/05/webrev/ >> > Much better. > > For the test can you use RandomFactory recently added to the test library? Sure. Here the updated webrev with this change and a few other minor changes. http://cr.openjdk.java.net/~igerasim/8058779/06/webrev/ The changes are: - move declaration of i below, - indent .append(), - use RandomFactory in the test, - extend number of test cases with null input. Do you think it's ready to go? Sincerely yours, Ivan > Paul. > > > From xueming.shen at oracle.com Mon Jun 1 19:10:32 2015 From: xueming.shen at oracle.com (Xueming Shen) Date: Mon, 01 Jun 2015 12:10:32 -0700 Subject: 8058779: Faster implementation of String.replace(CharSequence, CharSequence) In-Reply-To: <556CAA26.2050303@oracle.com> References: <556231BE.9050703@oracle.com> <5564FD92.2050408@oracle.com> <556503D8.6000303@oracle.com> <5565E2D6.2070806@oracle.com> <5565E667.4050601@oracle.com> <5565F988.7080209@oracle.com> <55660180.8030003@oracle.com> <556605E9.3010302@oracle.com> <55660805.1030007@oracle.com> <55661E7B.2010405@oracle.com> <556624BA.1030703@oracle.com> <556625E5.5040806@oracle.com> <556A6FB3.3060705@oracle.com> <556AC9C8.3070503@univ-mlv.fr> <556B2EBB.2080503@oracle.com> <556B30EA.5050801@oracle.com> <7E00911B-3B67-4A6F-AB73-428AD0781D21@oracle.com> <556CAA26.2050303@oracle.com> Message-ID: <556CAE28.5030801@oracle.com> Ivan, The code looks fine for me. Just wonder what's the motivation of using the newStringUnsafe() in the test case. Simply to save the char[] copy to speed up the test? I don't think we really care about it here, right? -Sherman On 06/01/2015 11:53 AM, Ivan Gerasimov wrote: > > > On 01.06.2015 11:33, Paul Sandoz wrote: >> On May 31, 2015, at 6:03 PM, Ivan Gerasimov wrote: >>> Which is right here: >>> http://cr.openjdk.java.net/~igerasim/8058779/05/webrev/ >>> >> Much better. >> >> For the test can you use RandomFactory recently added to the test library? > > Sure. > Here the updated webrev with this change and a few other minor changes. > http://cr.openjdk.java.net/~igerasim/8058779/06/webrev/ > > The changes are: > - move declaration of i below, > - indent .append(), > - use RandomFactory in the test, > - extend number of test cases with null input. > > Do you think it's ready to go? > > Sincerely yours, > Ivan > >> Paul. >> >> >> > From Roger.Riggs at Oracle.com Mon Jun 1 21:38:41 2015 From: Roger.Riggs at Oracle.com (Roger Riggs) Date: Mon, 01 Jun 2015 17:38:41 -0400 Subject: RFR 9: 8081567 : java/lang/ProcessHandle/InfoTest.java failed Cannot run program "whoami" Message-ID: <556CD0E1.2080102@Oracle.com> Please review a change to report the user/owner in ProcessHandle.info to have the same form for the owner identification as provided by java.nio.Files. On Windows it includes the domain with the user name. The test is updated to remove a dependency on the OS command whoami and instead compare the users process with the owner of a file created by the process improving portability. Webrev: http://cr.openjdk.java.net/~rriggs/webrev-whoami-8081567/ Issue: https://bugs.openjdk.java.net/browse/JDK-8081567 Thanks, Roger From joe.darcy at oracle.com Mon Jun 1 22:16:27 2015 From: joe.darcy at oracle.com (joe darcy) Date: Mon, 01 Jun 2015 15:16:27 -0700 Subject: JDK 9 RFR of JDK-8075551: Add tiered testing definitions to the jaxp repo Message-ID: <556CD9BB.6040204@oracle.com> Hello, Please review these changes to regularize the jaxp regression testing infrastructure with the JDK tiered testing policy. [1] JDK-8075551: Add tiered testing definitions to the jaxp repo http://cr.openjdk.java.net/~darcy/8075551.0/ In brief, testing tiers are defined (with an empty tier 1 for jaxp) and an empty problem list is added. When analogous changes are made in all the repositories, it will be possible to use a jtreg command like jtreg -exclude:ProblemList.txt [... other options...] :tier$N for any repo. Thanks, -Joe [1] http://mail.openjdk.java.net/pipermail/jdk9-dev/2015-March/001991.html From huizhe.wang at oracle.com Tue Jun 2 00:15:08 2015 From: huizhe.wang at oracle.com (huizhe wang) Date: Mon, 01 Jun 2015 17:15:08 -0700 Subject: JDK 9 RFR of JDK-8075551: Add tiered testing definitions to the jaxp repo In-Reply-To: <556CD9BB.6040204@oracle.com> References: <556CD9BB.6040204@oracle.com> Message-ID: <556CF58C.1030906@oracle.com> On 6/1/2015 3:16 PM, joe darcy wrote: > Hello, > > Please review these changes to regularize the jaxp regression testing > infrastructure with the JDK tiered testing policy. [1] > > JDK-8075551: Add tiered testing definitions to the jaxp repo > http://cr.openjdk.java.net/~darcy/8075551.0/ > > In brief, testing tiers are defined (with an empty tier 1 for jaxp) > and an empty problem list is added. The change looks good to me. > > When analogous changes are made in all the repositories, it will be > possible to use a jtreg command like > > jtreg -exclude:ProblemList.txt [... other options...] :tier$N How does it work in JPRT, something like "-testset core:tier$2"? Thanks, Joe > > for any repo. > > Thanks, > > -Joe > > [1] > http://mail.openjdk.java.net/pipermail/jdk9-dev/2015-March/001991.html From alejandro.murillo at oracle.com Tue Jun 2 00:45:05 2015 From: alejandro.murillo at oracle.com (Alejandro E Murillo) Date: Mon, 01 Jun 2015 18:45:05 -0600 Subject: JDK 9 RFR of JDK-8075551: Add tiered testing definitions to the jaxp repo In-Reply-To: <556CF58C.1030906@oracle.com> References: <556CD9BB.6040204@oracle.com> <556CF58C.1030906@oracle.com> Message-ID: <556CFC91.4080208@oracle.com> On 6/1/2015 6:15 PM, huizhe wang wrote: > > On 6/1/2015 3:16 PM, joe darcy wrote: >> Hello, >> >> Please review these changes to regularize the jaxp regression testing >> infrastructure with the JDK tiered testing policy. [1] >> >> JDK-8075551: Add tiered testing definitions to the jaxp repo >> http://cr.openjdk.java.net/~darcy/8075551.0/ >> >> In brief, testing tiers are defined (with an empty tier 1 for jaxp) >> and an empty problem list is added. > > The change looks good to me. > >> >> When analogous changes are made in all the repositories, it will be >> possible to use a jtreg command like >> >> jtreg -exclude:ProblemList.txt [... other options...] :tier$N > > How does it work in JPRT, something like "-testset core:tier$2"? I'm also very interested on an answer to this thanks Alejandro > > Thanks, > Joe > >> >> for any repo. >> >> Thanks, >> >> -Joe >> >> [1] >> http://mail.openjdk.java.net/pipermail/jdk9-dev/2015-March/001991.html > -- Alejandro From jeremymanson at google.com Tue Jun 2 07:30:21 2015 From: jeremymanson at google.com (Jeremy Manson) Date: Tue, 2 Jun 2015 00:30:21 -0700 Subject: JEP 254: Compact Strings In-Reply-To: <556C184D.6020107@oracle.com> References: <20150514230513.0363C62D05@eggemoggin.niobe.net> <556C184D.6020107@oracle.com> Message-ID: TL;DR: In principle, we'd love to do more early testing of Hotspot / JDK features, but our benchmarks are, honestly, not all that great. We end up having to test against live services, which makes this sort of thing really hard. More info than you need: There are two real problems here: 1) To do apples-to-apples comparisons, we have to make sure that *our* patches all work with whatever version of Hotspot we're testing. 2) Pulling down a new JDK9 - even an official release - usually means that there are a lot of instabilities, half-finished work, and inefficiencies, so we can't really run tests very well against it. That's not a knock on Hotspot developers; the only way to know about some of these problems is to run the JDK in infrastructure like ours. ( An example of something that hit us hard that no one else would notice: http://hg.openjdk.java.net/jdk9/hs/hotspot/rev/5ba37c4c0578 ) It took us months to forward port all of our patches to JDK8, and we've spent the last six months getting it to the point that we're comfortable enough to ship to our users (just in time for JDK7 EOL!). That's required disabling tiered compilation, heavily tweaking code cache flushing (which is still causing us CPU regressions), rewriting various parts of the metaspace to behave more efficiently, and fixing various incompatibilities with our internal patches. That's completely apart from the dozens of backwards incompatibilities introduced in JDK8 that triggered a very, very, very large code cleanup effort, from the new hash iteration order to the unicode update to the fact that if you call flush on a closed BufferedOutputStream it now throws an exception. (We actually ended up randomizing our hash iteration order, which helps us guard against broken code, is slightly more secure, and means that we never get bitten by that as part of an upgrade again.) In short, upgrading is in no sense cheap for us, and trying out new features is hard. We usually restrict ourselves to using new features that can be more-or-less cleanly patched to the version of the JDK we're using and hidden behind a flag. This is an important enough change that we might be able to make some effort, but we'll have to see how it goes. Jeremy On Mon, Jun 1, 2015 at 1:31 AM, Aleksey Shipilev < aleksey.shipilev at oracle.com> wrote: > (getting back to this) > > Hi Jeremy, > > On 05/16/2015 03:34 AM, Jeremy Manson wrote: > > So, I'm pretty dubious, mostly because of the risks mentioned in the JEP. > > If you need a flag-check and two code paths for every String method, > that's > > going to make the String class more slow and bloated, and make it very > > difficult for the JIT compiler to do its job inlining and intrinsifying > > calls to String methods. > > Yes, we know that's a potential problem, e.g. outlined here: > http://cr.openjdk.java.net/~shade/density/equals.txt > > The hope is that the string coder check would be amortized by the > substantial performance improvement with the ubiquitous Latin1 > (optimized) operations. Also, getting a few code generation quirks > kicked out may further offset the perceived performance costs of doing > this (you can do such a trick every so often, but not all the time). > > > > The proposed change here has the potential of doing the opposite with > most > > String operations - trading off less GC overhead for more mutator cost. > > But String operations are a pretty big chunk of CPU time, on the whole. > > The thing is, many mutator ops on Strings are also improved, because the > data become more easily cacheable and/or require less steps to complete > (think vectorization that takes 2x less instructions). > > > > Does anyone really have a sense of how to make this kind of decision? > The > > JEP seems mostly to be hoping that other organizations will do the > testing > > for you. > > It is not true that JEP hopes to have other organizations to do testing > for it. The JEP tries to illuminate that this is a performance-sensitive > change, so early testing and feedback is very appreciated. So, if you > have the String-intensive workloads in your org, can you try and run the > prototype JDK against it? Our early runs on our workloads of interest > show the appealing improvements. > > That is, the decision to integrate this is not done yet, as we don't > have the complete performance picture and/or fully-tested prototype. In > other words, there are quite a few blank spots to fill out. Your data > may be the part of that picture when we decide to integrate in JDK 9. > > > > (I agree that it is worth doing some experimentation in this area, but I > > wanted to say this early, because if I could reach back in time and tell > > you *not* to make the substring change, I would. We seriously considered > > simply backing it out locally, but it would have been a lot of effort for > > us to maintain that kind of patch, and we didn't want our performance > > tradeoffs to be that much different from the stock JDK's.) > > This is your golden ticket: if you come back with concrete data in your > hands saying that the particular tradeoff the JEP made is not sensible > for your applications, it would be considered in the decision to > integrate. But, it should be a real data and/or contrived benchmark > simulating the real-world scenario, not just theoretical appeals -- we > know how misguided those can get. > > > Thanks, > -Aleksey > > From chris.plummer at oracle.com Tue Jun 2 07:36:29 2015 From: chris.plummer at oracle.com (Chris Plummer) Date: Tue, 02 Jun 2015 00:36:29 -0700 Subject: [9] RFR (M) 8054386: Allow Java debugging when CDS is enabled In-Reply-To: <555B47EA.3000900@oracle.com> References: <555B47EA.3000900@oracle.com> Message-ID: <556D5CFD.1020709@oracle.com> [Adding core-libs-dev at openjdk.java.net since this update includes changes to jdk/test library code] Please review the updated webrev: Webrev: http://cr.openjdk.java.net/~cjplummer/8054386/webrev.02/ Bug: https://bugs.openjdk.java.net/browse/JDK-8054386 There were concerns about the new hotspot tests referencing jdk tests. One concern was that if the jdk tests change, they could break the hotspot tests, and this might initially go undetected. The other concern is that if the jdk and hotspot tests are placed in separate test bundles, then it would not be possible to run the hotspot tests. Because of these concerns, I moved the tests that were in hotspot/test/runtime/CDSJDITests to instead be in jdk/test/com/sun/jdi/CDSJDITests. There was a slight renaming of the tests in the process. Also, I had to update the jdk version of ProcessTool.java to include the createJavaProcessBuilder() variant that is in the hotspot version of ProcessTool.java. Lastly, in CDSJITTest.java I changed: OutputAnalyzer output = new OutputAnalyzer(pb.start()); to instead be: OutputAnalyzer output = ProcessTools.executeProcess(pb); I had to do this since the jdk version of the OutputAnalyzer constructor is not public. The 1st version is what is commonly used in hostspot tests, and the 2nd version is what is commonly used in jdk tests. I decided to adopt the jdk way rather than make the OutputAnalyzer constructors public, although this will probably happen eventually when the two versions are unified. thanks, Chris On 5/19/15 7:25 AM, Chris Plummer wrote: > Hi, > > Please review the following changes for allowing java debugging when > CDS is enabled. > > Webrev:http://cr.openjdk.java.net/~cjplummer/8054386/webrev.01/ > Bug:https://bugs.openjdk.java.net/browse/JDK-8054386 > > The VM changes are simple. I removed the check that prevents debugging > with CDS enabled, and added logic that will map the CDS archive RW > when debugging is enabled. > > The tests are a bit more complex. There are a bunch of existing JDI > tests for testing debugging support. Rather than start from scratch or > clone them, I instead just wrote wrapper tests that put the relevant > JDI test classes in the archive, and then invoke the JDI test. I did > this for 3 of the JDI tests. If you feel there are others that would > be good candidates, I'd be happy to add them. I'm looking for ones > that would result in modification of the RO class metadata, such as > setting a breakpoint (for which I already added two tests). > > Testing done: > -Using JPRT to run the new jtreg tests on all platforms. > -Using JPRT to run all jtreg runtime tests on linux x86 and x_64. > -Regular JPRT "-testset hotspot" run > -Putting the JCK JVMTI tests in the archive and then running them. > -Putting the nsk jdb, jdwp, jvmti, and jdi tests in the archive and > then running them. > -Putting a simple test class in the archive and then setting a > breakpoint on it using jdb > > Some of the above testing resulted in the discovery of bugs that still > need to be addressed: JDK-8078644, JDK-8078730, and JDK-8079181. > > I also verified that without the change to map the archive RW, the > above testing resulted in a SEGV, which is what you would expect (and > actually want to see to prove that the testing is effective). > > thanks, > > Chris > From paul.sandoz at oracle.com Tue Jun 2 08:20:04 2015 From: paul.sandoz at oracle.com (Paul Sandoz) Date: Tue, 2 Jun 2015 10:20:04 +0200 Subject: 8058779: Faster implementation of String.replace(CharSequence, CharSequence) In-Reply-To: <556CAA26.2050303@oracle.com> References: <556231BE.9050703@oracle.com> <5564FD92.2050408@oracle.com> <556503D8.6000303@oracle.com> <5565E2D6.2070806@oracle.com> <5565E667.4050601@oracle.com> <5565F988.7080209@oracle.com> <55660180.8030003@oracle.com> <556605E9.3010302@oracle.com> <55660805.1030007@oracle.com> <55661E7B.2010405@oracle.com> <556624BA.1030703@oracle.com> <556625E5.5040806@oracle.com> <556A6FB3.3060705@oracle.com> <556AC9C8.3070503@univ-mlv.fr> <556B2EBB.2080503@oracle.com> <556B30EA.5050801@oracle.com> <7E00911B-3B67-4A6F-AB73-428AD0781D21@oracle.com> <556CAA26.2050303@oracle.com> Message-ID: <459FA37E-0A2E-4415-804A-8071AEB49FBF@oracle.com> On Jun 1, 2015, at 8:53 PM, Ivan Gerasimov wrote: > > > On 01.06.2015 11:33, Paul Sandoz wrote: >> On May 31, 2015, at 6:03 PM, Ivan Gerasimov wrote: >>> Which is right here: >>> http://cr.openjdk.java.net/~igerasim/8058779/05/webrev/ >>> >> Much better. >> >> For the test can you use RandomFactory recently added to the test library? > > Sure. > Here the updated webrev with this change and a few other minor changes. > http://cr.openjdk.java.net/~igerasim/8058779/06/webrev/ > > The changes are: > - move declaration of i below, > - indent .append(), > - use RandomFactory in the test, > - extend number of test cases with null input. > > Do you think it's ready to go? > Almost :-) Like Sherman i think you should get rid of JavaLangAccess.newStringUnsafe. 78 @Test(dataProvider="sourceTargetReplacementWithNull") You can use the "expectedExceptions" attribute instead of an explicit try/fail/catch. 79 public void testNPE(String source, String target, String replacement) { 80 try { 81 source.replace(target, replacement); 82 fail("Expected to throw NPE: " + source + ".replace(" + target + 83 "," + replacement + ")"); 84 } catch (NullPointerException npe) { 85 } 86 } You could simplify the data provider sourceTargetReplacementWithNull, there is no point testing with a null source. String.replace accepts two arguments, each can be either null or non-null. Thats 2 bits of state, so one can simply be explicit and presumably it should not matter what the non-null argument value (there are enough non-null values supplied by the other data providers): {null, null} {null, "foo"} {"foo", null} No need for another review for any of these. Paul. From ivan.gerasimov at oracle.com Tue Jun 2 08:49:20 2015 From: ivan.gerasimov at oracle.com (Ivan Gerasimov) Date: Tue, 02 Jun 2015 11:49:20 +0300 Subject: 8058779: Faster implementation of String.replace(CharSequence, CharSequence) In-Reply-To: <556CAE28.5030801@oracle.com> References: <556231BE.9050703@oracle.com> <5564FD92.2050408@oracle.com> <556503D8.6000303@oracle.com> <5565E2D6.2070806@oracle.com> <5565E667.4050601@oracle.com> <5565F988.7080209@oracle.com> <55660180.8030003@oracle.com> <556605E9.3010302@oracle.com> <55660805.1030007@oracle.com> <55661E7B.2010405@oracle.com> <556624BA.1030703@oracle.com> <556625E5.5040806@oracle.com> <556A6FB3.3060705@oracle.com> <556AC9C8.3070503@univ-mlv.fr> <556B2EBB.2080503@oracle.com> <556B30EA.5050801@oracle.com> <7E00911B-3B67-4A6F-AB73-428AD0781D21@oracle.com> <556CAA26.2050303@oracle.com> <556CAE28.5030801@oracle.com> Message-ID: <556D6E10.6030906@oracle.com> On 01.06.2015 22:10, Xueming Shen wrote: > Ivan, > > The code looks fine for me. > > Just wonder what's the motivation of using the newStringUnsafe() in > the test case. Simply > to save the char[] copy to speed up the test? I don't think we really > care about it here, > right? > Yes, right. I'll replace it with new String(char[]). Sincerely yours, Ivan > -Sherman > > On 06/01/2015 11:53 AM, Ivan Gerasimov wrote: >> >> >> On 01.06.2015 11:33, Paul Sandoz wrote: >>> On May 31, 2015, at 6:03 PM, Ivan Gerasimov >>> wrote: >>>> Which is right here: >>>> http://cr.openjdk.java.net/~igerasim/8058779/05/webrev/ >>>> >>> Much better. >>> >>> For the test can you use RandomFactory recently added to the test >>> library? >> >> Sure. >> Here the updated webrev with this change and a few other minor changes. >> http://cr.openjdk.java.net/~igerasim/8058779/06/webrev/ >> >> The changes are: >> - move declaration of i below, >> - indent .append(), >> - use RandomFactory in the test, >> - extend number of test cases with null input. >> >> Do you think it's ready to go? >> >> Sincerely yours, >> Ivan >> >>> Paul. >>> >>> >>> >> > > > From forax at univ-mlv.fr Tue Jun 2 08:54:03 2015 From: forax at univ-mlv.fr (Remi Forax) Date: Tue, 02 Jun 2015 10:54:03 +0200 Subject: 8058779: Faster implementation of String.replace(CharSequence, CharSequence) In-Reply-To: <556CAA26.2050303@oracle.com> References: <556231BE.9050703@oracle.com> <5564FD92.2050408@oracle.com> <556503D8.6000303@oracle.com> <5565E2D6.2070806@oracle.com> <5565E667.4050601@oracle.com> <5565F988.7080209@oracle.com> <55660180.8030003@oracle.com> <556605E9.3010302@oracle.com> <55660805.1030007@oracle.com> <55661E7B.2010405@oracle.com> <556624BA.1030703@oracle.com> <556625E5.5040806@oracle.com> <556A6FB3.3060705@oracle.com> <556AC9C8.3070503@univ-mlv.fr> <556B2EBB.2080503@oracle.com> <556B30EA.5050801@oracle.com> <7E00911B-3B67-4A6F-AB73-428AD0781D21@oracle.com> <556CAA26.2050303@oracle.com> Message-ID: <556D6F2B.8050107@univ-mlv.fr> Looks Ok to me, thumb up :) R?mi On 06/01/2015 08:53 PM, Ivan Gerasimov wrote: > > > On 01.06.2015 11:33, Paul Sandoz wrote: >> On May 31, 2015, at 6:03 PM, Ivan Gerasimov >> wrote: >>> Which is right here: >>> http://cr.openjdk.java.net/~igerasim/8058779/05/webrev/ >>> >> Much better. >> >> For the test can you use RandomFactory recently added to the test >> library? > > Sure. > Here the updated webrev with this change and a few other minor changes. > http://cr.openjdk.java.net/~igerasim/8058779/06/webrev/ > > The changes are: > - move declaration of i below, > - indent .append(), > - use RandomFactory in the test, > - extend number of test cases with null input. > > Do you think it's ready to go? > > Sincerely yours, > Ivan > >> Paul. >> >> >> > From ivan.gerasimov at oracle.com Tue Jun 2 08:56:42 2015 From: ivan.gerasimov at oracle.com (Ivan Gerasimov) Date: Tue, 02 Jun 2015 11:56:42 +0300 Subject: 8058779: Faster implementation of String.replace(CharSequence, CharSequence) In-Reply-To: <459FA37E-0A2E-4415-804A-8071AEB49FBF@oracle.com> References: <556231BE.9050703@oracle.com> <5564FD92.2050408@oracle.com> <556503D8.6000303@oracle.com> <5565E2D6.2070806@oracle.com> <5565E667.4050601@oracle.com> <5565F988.7080209@oracle.com> <55660180.8030003@oracle.com> <556605E9.3010302@oracle.com> <55660805.1030007@oracle.com> <55661E7B.2010405@oracle.com> <556624BA.1030703@oracle.com> <556625E5.5040806@oracle.com> <556A6FB3.3060705@oracle.com> <556AC9C8.3070503@univ-mlv.fr> <556B2EBB.2080503@oracle.com> <556B30EA.5050801@oracle.com> <7E00911B-3B67-4A6F-AB73-428AD0781D21@oracle.com> <556CAA26.2050303@oracle.com> <459FA37E-0A2E-4415-804A-8071AEB49FBF@oracle.com> Message-ID: <556D6FCA.1040609@oracle.com> On 02.06.2015 11:20, Paul Sandoz wrote: > On Jun 1, 2015, at 8:53 PM, Ivan Gerasimov wrote: > >> >> On 01.06.2015 11:33, Paul Sandoz wrote: >>> On May 31, 2015, at 6:03 PM, Ivan Gerasimov wrote: >>>> Which is right here: >>>> http://cr.openjdk.java.net/~igerasim/8058779/05/webrev/ >>>> >>> Much better. >>> >>> For the test can you use RandomFactory recently added to the test library? >> Sure. >> Here the updated webrev with this change and a few other minor changes. >> http://cr.openjdk.java.net/~igerasim/8058779/06/webrev/ >> >> The changes are: >> - move declaration of i below, >> - indent .append(), >> - use RandomFactory in the test, >> - extend number of test cases with null input. >> >> Do you think it's ready to go? >> > Almost :-) > > > Like Sherman i think you should get rid of JavaLangAccess.newStringUnsafe. > > > 78 @Test(dataProvider="sourceTargetReplacementWithNull") > > You can use the "expectedExceptions" attribute instead of an explicit try/fail/catch. > > 79 public void testNPE(String source, String target, String replacement) { > 80 try { > 81 source.replace(target, replacement); > 82 fail("Expected to throw NPE: " + source + ".replace(" + target + > 83 "," + replacement + ")"); > 84 } catch (NullPointerException npe) { > 85 } > 86 } Alright, I will use expectedExceptions here. > > You could simplify the data provider sourceTargetReplacementWithNull, there is no point testing with a null source. String.replace accepts two arguments, each can be either null or non-null. Thats 2 bits of state, so one can simply be explicit and presumably it should not matter what the non-null argument value (there are enough non-null values supplied by the other data providers): > > {null, null} > {null, "foo"} > {"foo", null} I agree there is not much value in testing the case source == null, so it can be removed. However, I would leave other test cases. Earlier in this thread R?mi spotted a bug, which had been observed in a call "bar".replace("foo", null), but not in "foo".replace("foo", null). That was my motivation to extend the number of test cases for null inputs. Sincerely yours, Ivan > No need for another review for any of these. > > Paul. > > From ivan.gerasimov at oracle.com Tue Jun 2 08:57:58 2015 From: ivan.gerasimov at oracle.com (Ivan Gerasimov) Date: Tue, 02 Jun 2015 11:57:58 +0300 Subject: 8058779: Faster implementation of String.replace(CharSequence, CharSequence) In-Reply-To: <556D6F2B.8050107@univ-mlv.fr> References: <556231BE.9050703@oracle.com> <5564FD92.2050408@oracle.com> <556503D8.6000303@oracle.com> <5565E2D6.2070806@oracle.com> <5565E667.4050601@oracle.com> <5565F988.7080209@oracle.com> <55660180.8030003@oracle.com> <556605E9.3010302@oracle.com> <55660805.1030007@oracle.com> <55661E7B.2010405@oracle.com> <556624BA.1030703@oracle.com> <556625E5.5040806@oracle.com> <556A6FB3.3060705@oracle.com> <556AC9C8.3070503@univ-mlv.fr> <556B2EBB.2080503@oracle.com> <556B30EA.5050801@oracle.com> <7E00911B-3B67-4A6F-AB73-428AD0781D21@oracle.com> <556CAA26.2050303@oracle.com> <556D6F2B.8050107@univ-mlv.fr> Message-ID: <556D7016.9090600@oracle.com> Thanks! On 02.06.2015 11:54, Remi Forax wrote: > Looks Ok to me, > thumb up :) > > R?mi > > On 06/01/2015 08:53 PM, Ivan Gerasimov wrote: >> >> >> On 01.06.2015 11:33, Paul Sandoz wrote: >>> On May 31, 2015, at 6:03 PM, Ivan Gerasimov >>> wrote: >>>> Which is right here: >>>> http://cr.openjdk.java.net/~igerasim/8058779/05/webrev/ >>>> >>> Much better. >>> >>> For the test can you use RandomFactory recently added to the test >>> library? >> >> Sure. >> Here the updated webrev with this change and a few other minor changes. >> http://cr.openjdk.java.net/~igerasim/8058779/06/webrev/ >> >> The changes are: >> - move declaration of i below, >> - indent .append(), >> - use RandomFactory in the test, >> - extend number of test cases with null input. >> >> Do you think it's ready to go? >> >> Sincerely yours, >> Ivan >> >>> Paul. >>> >>> >>> >> > > > From paul.sandoz at oracle.com Tue Jun 2 09:15:59 2015 From: paul.sandoz at oracle.com (Paul Sandoz) Date: Tue, 2 Jun 2015 11:15:59 +0200 Subject: 8058779: Faster implementation of String.replace(CharSequence, CharSequence) In-Reply-To: <556D6FCA.1040609@oracle.com> References: <556231BE.9050703@oracle.com> <5564FD92.2050408@oracle.com> <556503D8.6000303@oracle.com> <5565E2D6.2070806@oracle.com> <5565E667.4050601@oracle.com> <5565F988.7080209@oracle.com> <55660180.8030003@oracle.com> <556605E9.3010302@oracle.com> <55660805.1030007@oracle.com> <55661E7B.2010405@oracle.com> <556624BA.1030703@oracle.com> <556625E5.5040806@oracle.com> <556A6FB3.3060705@oracle.com> <556AC9C8.3070503@univ-mlv.fr> <556B2EBB.2080503@oracle.com> <556B30EA.5050801@oracle.com> <7E00911B-3B67-4A6F-AB73-428AD0781D21@oracle.com> <556CAA26.2050303@oracle.com> <459FA37E-0A2E-4415-804A-8071AEB49FBF@oracle.com> <556D6FCA.1040609@oracle.com> Message-ID: On Jun 2, 2015, at 10:56 AM, Ivan Gerasimov wrote: >> >> You could simplify the data provider sourceTargetReplacementWithNull, there is no point testing with a null source. String.replace accepts two arguments, each can be either null or non-null. Thats 2 bits of state, so one can simply be explicit and presumably it should not matter what the non-null argument value (there are enough non-null values supplied by the other data providers): >> >> {null, null} >> {null, "foo"} >> {"foo", null} > I agree there is not much value in testing the case source == null, so it can be removed. > However, I would leave other test cases. > Earlier in this thread R?mi spotted a bug, which had been observed in a call "bar".replace("foo", null), but not in "foo".replace("foo", null). > Ok, presumably for the same identity rather than value? Paul. From dmitry.samersoff at oracle.com Tue Jun 2 11:12:48 2015 From: dmitry.samersoff at oracle.com (Dmitry Samersoff) Date: Tue, 02 Jun 2015 14:12:48 +0300 Subject: RFR(M, v11): JDK-8059036 : Implement Diagnostic Commands for heap and finalizerinfo In-Reply-To: <37227441-B016-4A9A-B9CA-BE100E992C9A@oracle.com> References: <554894E9.8020908@oracle.com> <1AF219C7-1055-4D97-BB46-2FA390C4135D@oracle.com> <5548D65C.8040304@gmail.com> <554A8CEE.2090009@oracle.com> <55523418.5010708@oracle.com> <55527935.5010408@gmail.com> <5553B552.9060900@oracle.com> <555453BA.2070205@gmail.com> <55567748.6020304@oracle.com> <55567A68.9020802@oracle.com> <5556F34B.5050701@oracle.com> <55573A31.7050306@gmail.com> <55573C86.8030807@gmail.com> <5559D85B.2050500@oracle.com> <6FB8DF19-1D67-4702-9642-4C630765D2D1@oracle.com> <555C2EEB.1050505@oracle.com> <3941B430-FE6C-4AF0-A875-76D24411C654@oracle.com> <5564802A.2010403@oracle.com> <5565738A.4040109@gmail.com> <7EB0BE71-1608-4047-B71D-1A9EAA9083BB@oracle.com> <556727A2.9060209@gmail.com> <55675908.6030406@oracle.com> <556AF3CD.50900@oracle.com> <37227441-B016-4A9A-B9CA-BE100E992C9A@oracle.com> Message-ID: <556D8FB0.4070608@oracle.com> Staffan, > Instead of hardcoding the field offsets, you can use > InstanceKlass::find_field and fieldDescriptor::offset to find the > offset at runtime. Done. Please, see http://cr.openjdk.java.net/~dsamersoff/JDK-8059036/webrev.11 I put the function int get_filed_offset_by_name(Symbol*,Symbol*) to oop.inline.hpp leaving a room for further cleanup because I found couple of places in hotspot that implements mostly similar functionality. -Dmitry On 2015-06-01 10:18, Staffan Larsen wrote: > Dmitry, > > Instead of hardcoding the field offsets, you can use InstanceKlass::find_field and fieldDescriptor::offset to find the offset at runtime. > > Thanks, > /Staffan > >> On 31 maj 2015, at 13:43, Dmitry Samersoff wrote: >> >> Everyone, >> >> Please take a look at new version of the fix. >> >> http://cr.openjdk.java.net/~dsamersoff/JDK-8059036/webrev.10/ >> >> Changes (to previous version) are in >> Finalizer.java and diagnosticCommand.cpp >> >> This version copy data from Map.Entry<> to array of >> FinalizerHistogramEntry instances then, >> VM prints content of this array. >> >> -Dmitry >> >> >> On 2015-05-28 21:06, Mandy Chung wrote: >>> >>> On 05/28/2015 07:35 AM, Peter Levart wrote: >>>> Hi Mandy, >>>> >>>> On 05/27/2015 03:32 PM, Mandy Chung wrote: >>>>> Taking it further - is it simpler to return String[] of all >>>>> classnames including the duplicated ones and have the VM do the >>>>> count? Are you concerned with the size of the String[]? >>>> >>>> Yes, the histogram is much smaller than the list of all instances. >>>> There can be millions of instances waiting in finalizer queue, but >>>> only a few distinct classes. >>> >>> Do you happen to know what libraries are the major contributors to these >>> millions of finalizers? >>> >>> It has been strongly recommended to avoid finalizers (see Effective Java >>> Item 7). I'm not surprised that existing code is still using >>> finalizers while we should really encourage them to migrate away from it. >>> >>>> What could be done in Java to simplify things in native code but still >>>> not format the output is to convert the array of Map.Entry(s) into an >>>> Object[] array of alternating {String, int[], String, int[], .... } >>>> >>>> Would this simplify native code for the price of a little extra work >>>> in Java? The sizes of old and new arrays are not big (# of distinct >>>> classes of finalizable objects in the queue). >>> >>> I also prefer writing in Java and writing less native code (much >>> simplified). It's an interface that we have to agree upon and keep it >>> simple. Having the returned Object[] as alternate String and int[] is a >>> reasonable compromise. >>> >>> ReferenceQueue.java - you can move @SuppressWarning from the method to >>> just the field variable "rn" >>> @SuppressWarnings("unchecked") >>> Reference rn = r.next; >>> >>> Finalizer.java >>> It's better to rename printFinalizationQueue as it inspects the >>> finalizer reference queue (maybe getFinalizerHistogram?). Can you move >>> this method to the end of this class and add the comment saying that >>> this is invoked by VM for jcmd -finalizerinfo support and @return to >>> describe the returned content. I also think you can remove >>> @SuppressWarnings for this method. >>> >>> Mandy >> >> >> -- >> Dmitry Samersoff >> Oracle Java development team, Saint Petersburg, Russia >> * I would love to change the world, but they won't give me the sources. > -- Dmitry Samersoff Oracle Java development team, Saint Petersburg, Russia * I would love to change the world, but they won't give me the sources. From mikael.gerdin at oracle.com Tue Jun 2 12:06:36 2015 From: mikael.gerdin at oracle.com (Mikael Gerdin) Date: Tue, 02 Jun 2015 14:06:36 +0200 Subject: RFR(M, v11): JDK-8059036 : Implement Diagnostic Commands for heap and finalizerinfo In-Reply-To: <556D8FB0.4070608@oracle.com> References: <554894E9.8020908@oracle.com> <1AF219C7-1055-4D97-BB46-2FA390C4135D@oracle.com> <5548D65C.8040304@gmail.com> <554A8CEE.2090009@oracle.com> <55523418.5010708@oracle.com> <55527935.5010408@gmail.com> <5553B552.9060900@oracle.com> <555453BA.2070205@gmail.com> <55567748.6020304@oracle.com> <55567A68.9020802@oracle.com> <5556F34B.5050701@oracle.com> <55573A31.7050306@gmail.com> <55573C86.8030807@gmail.com> <5559D85B.2050500@oracle.com> <6FB8DF19-1D67-4702-9642-4C630765D2D1@oracle.com> <555C2EEB.1050505@oracle.com> <3941B430-FE6C-4AF0-A875-76D24411C654@oracle.com> <5564802A.2010403@oracle.com> <5565738A.4040109@gmail.com> <7EB0BE71-1608-4047-B71D-1A9EAA9083BB@oracle.com> <556727A2.9060209@gmail.com> <55675908.6030406@oracle.com> <556AF3CD.50900@oracle.com> <37227441-B016-4A9A-B9CA-BE100E992C9A@oracle.com> <556D8FB0.4070608@oracle.com> Message-ID: <556D9C4C.9080200@oracle.com> Hi Dmitry, On 2015-06-02 13:12, Dmitry Samersoff wrote: > Staffan, > >> Instead of hardcoding the field offsets, you can use >> InstanceKlass::find_field and fieldDescriptor::offset to find the >> offset at runtime. > > Done. Please, see > > http://cr.openjdk.java.net/~dsamersoff/JDK-8059036/webrev.11 > > I put the function int get_filed_offset_by_name(Symbol*,Symbol*) to > oop.inline.hpp leaving a room for further cleanup because I found couple > of places in hotspot that implements mostly similar functionality. Sorry for this sort of drive-by review, but I really don't think get_field_offset_by_name should be in the oop class. If anywhere it should be on InstanceKlass, and using Symbol* to identify a Klass* seems incorrect since the same symbol can refer to different classes in different class loader contexts. /Mikael > > -Dmitry > > On 2015-06-01 10:18, Staffan Larsen wrote: >> Dmitry, >> >> Instead of hardcoding the field offsets, you can use InstanceKlass::find_field and fieldDescriptor::offset to find the offset at runtime. >> >> Thanks, >> /Staffan >> >>> On 31 maj 2015, at 13:43, Dmitry Samersoff wrote: >>> >>> Everyone, >>> >>> Please take a look at new version of the fix. >>> >>> http://cr.openjdk.java.net/~dsamersoff/JDK-8059036/webrev.10/ >>> >>> Changes (to previous version) are in >>> Finalizer.java and diagnosticCommand.cpp >>> >>> This version copy data from Map.Entry<> to array of >>> FinalizerHistogramEntry instances then, >>> VM prints content of this array. >>> >>> -Dmitry >>> >>> >>> On 2015-05-28 21:06, Mandy Chung wrote: >>>> >>>> On 05/28/2015 07:35 AM, Peter Levart wrote: >>>>> Hi Mandy, >>>>> >>>>> On 05/27/2015 03:32 PM, Mandy Chung wrote: >>>>>> Taking it further - is it simpler to return String[] of all >>>>>> classnames including the duplicated ones and have the VM do the >>>>>> count? Are you concerned with the size of the String[]? >>>>> >>>>> Yes, the histogram is much smaller than the list of all instances. >>>>> There can be millions of instances waiting in finalizer queue, but >>>>> only a few distinct classes. >>>> >>>> Do you happen to know what libraries are the major contributors to these >>>> millions of finalizers? >>>> >>>> It has been strongly recommended to avoid finalizers (see Effective Java >>>> Item 7). I'm not surprised that existing code is still using >>>> finalizers while we should really encourage them to migrate away from it. >>>> >>>>> What could be done in Java to simplify things in native code but still >>>>> not format the output is to convert the array of Map.Entry(s) into an >>>>> Object[] array of alternating {String, int[], String, int[], .... } >>>>> >>>>> Would this simplify native code for the price of a little extra work >>>>> in Java? The sizes of old and new arrays are not big (# of distinct >>>>> classes of finalizable objects in the queue). >>>> >>>> I also prefer writing in Java and writing less native code (much >>>> simplified). It's an interface that we have to agree upon and keep it >>>> simple. Having the returned Object[] as alternate String and int[] is a >>>> reasonable compromise. >>>> >>>> ReferenceQueue.java - you can move @SuppressWarning from the method to >>>> just the field variable "rn" >>>> @SuppressWarnings("unchecked") >>>> Reference rn = r.next; >>>> >>>> Finalizer.java >>>> It's better to rename printFinalizationQueue as it inspects the >>>> finalizer reference queue (maybe getFinalizerHistogram?). Can you move >>>> this method to the end of this class and add the comment saying that >>>> this is invoked by VM for jcmd -finalizerinfo support and @return to >>>> describe the returned content. I also think you can remove >>>> @SuppressWarnings for this method. >>>> >>>> Mandy >>> >>> >>> -- >>> Dmitry Samersoff >>> Oracle Java development team, Saint Petersburg, Russia >>> * I would love to change the world, but they won't give me the sources. >> > > From konstantin.shefov at oracle.com Tue Jun 2 12:15:48 2015 From: konstantin.shefov at oracle.com (Konstantin Shefov) Date: Tue, 2 Jun 2015 05:15:48 -0700 (PDT) Subject: [9] Review request for 8072515: Test Task: Develop new tests for JEP 219: Datagram Transport Layer Security (DTLS) Message-ID: Hello, Please review new tests fro DTLS feature for JDK 9: bug: https://bugs.openjdk.java.net/browse/JDK-8072515 webrev: http://cr.openjdk.java.net/~kshefov/8072515/webrev.00/ Thanks -Konstantin From ivan.gerasimov at oracle.com Tue Jun 2 12:31:23 2015 From: ivan.gerasimov at oracle.com (Ivan Gerasimov) Date: Tue, 02 Jun 2015 15:31:23 +0300 Subject: 8058779: Faster implementation of String.replace(CharSequence, CharSequence) In-Reply-To: References: <556231BE.9050703@oracle.com> <5564FD92.2050408@oracle.com> <556503D8.6000303@oracle.com> <5565E2D6.2070806@oracle.com> <5565E667.4050601@oracle.com> <5565F988.7080209@oracle.com> <55660180.8030003@oracle.com> <556605E9.3010302@oracle.com> <55660805.1030007@oracle.com> <55661E7B.2010405@oracle.com> <556624BA.1030703@oracle.com> <556625E5.5040806@oracle.com> <556A6FB3.3060705@oracle.com> <556AC9C8.3070503@univ-mlv.fr> <556B2EBB.2080503@oracle.com> <556B30EA.5050801@oracle.com> <7E00911B-3B67-4A6F-AB73-428AD0781D21@oracle.com> <556CAA26.2050303@oracle.com> <459FA37E-0A2E-4415-804A-8071AEB49FBF@oracle.com> <556D6FCA.1040609@oracle.com> Message-ID: <556DA21B.90002@oracle.com> On 02.06.2015 12:15, Paul Sandoz wrote: > On Jun 2, 2015, at 10:56 AM, Ivan Gerasimov wrote: >>> You could simplify the data provider sourceTargetReplacementWithNull, there is no point testing with a null source. String.replace accepts two arguments, each can be either null or non-null. Thats 2 bits of state, so one can simply be explicit and presumably it should not matter what the non-null argument value (there are enough non-null values supplied by the other data providers): >>> >>> {null, null} >>> {null, "foo"} >>> {"foo", null} >> I agree there is not much value in testing the case source == null, so it can be removed. >> However, I would leave other test cases. >> Earlier in this thread R?mi spotted a bug, which had been observed in a call "bar".replace("foo", null), but not in "foo".replace("foo", null). >> > Ok, presumably for the same identity rather than value? For any valid substring, so "foobar".replace("bar", null) wouldn't have thrown OOM, even though it should have. Surely, the current testcases often duplicate each other, but it seems to be easier to process them all then to filter the duplicates out. Sincerely yours, Ivan > Paul. > > > > From miroslav.kos at oracle.com Tue Jun 2 12:32:12 2015 From: miroslav.kos at oracle.com (Miroslav Kos) Date: Tue, 02 Jun 2015 14:32:12 +0200 Subject: RFR [9] 8072839: JAX-B Plugability Layer: using java.util.ServiceLoader In-Reply-To: <5564F8A4.5050602@oracle.com> References: <55560D2B.3040400@oracle.com> <555C8C7B.7060801@oracle.com> <5564F8A4.5050602@oracle.com> Message-ID: <556DA24C.9080708@oracle.com> Hi Alan, Daniel, would you have some time to check the changes in this one? Thanks a lot Miran On 27/05/15 00:50, Miroslav Kos wrote: > > On 20/05/15 15:30, Daniel Fuchs wrote: >> Hi Miroslav, >> >> I haven't looked in all details, but this text in JAXBContext.java looks >> odd to me - it seems that the bullet is repeated twice (or if they >> differ - the diff must be subtle): >> >> lines 241-257 seem identical to lines 259-275 >> >> 241 *
  • >> 242 * Look for resource >> /META-INF/services/javax.xml.bind.JAXBContext using provided >> class loader. >> 243 * Methods without class loader parameter use {@code >> Thread.currentThread().getContextClassLoader()}. >> 244 * If such a resource exists, its content is assumed to be the >> provider factory class and must supply >> 245 * an implementation class containing the following method >> signatures: >> 246 * >> 247 * 
    >>  248  *
>>  249  * public static JAXBContext createContext(
>>  250  *                                      String contextPath,
>>  251  *                                      ClassLoader classLoader,
>>  252  * Map<String,Object> properties throws JAXBException
>>  253  *
>>  254  * public static JAXBContext createContext(
>>  255  *                                      Class[] classes,
>>  256  * Map<String,Object> properties ) throws JAXBException
>>  257  *
    >> 258 * >> 259 *
  • >> 260 * Look for resource >> /META-INF/services/javax.xml.bind.JAXBContext using provided >> class loader. >> 261 * Methods without class loader parameter use {@code >> Thread.currentThread().getContextClassLoader()}. >> 262 * If such a resource exists, its content is assumed to be the >> provider factory class and must supply >> 263 * an implementation class containing the following method >> signatures: >> 264 * >> 265 * 
    >>  266  *
>>  267  * public static JAXBContext createContext(
>>  268  *                                      String contextPath,
>>  269  *                                      ClassLoader classLoader,
>>  270  * Map<String,Object> properties throws JAXBException
>>  271  *
>>  272  * public static JAXBContext createContext(
>>  273  *                                      Class[] classes,
>>  274  * Map<String,Object> properties ) throws JAXBException
>>  275  *
    > Hi Daniel, > thanks for catching this - obviously incorrectly applied patch. >> >> So if I understand well - this is somewhat similar to what >> ServiceLoader does in terms of lookup, except that the >> mechanism defined here does not use an instance of the >> declared class, nor require that the class declared in >> /META-INF/services/javax.xml.bind.JAXBContext >> is a subclass of JAXBContext - but simply that it defines >> the appropriate static factory methods... > Yes, exactly. It is "ServiceLoader like" implementation from times > when ServiceLoader wasn't in jdk yet (jdk5). >> >> Maybe the fact that this step is deprecated should be noted more >> prominently in the bullet itself. > I added a note mentioning that the step is deprecated. > > Corrected version: http://cr.openjdk.java.net/~mkos/8072839/jaxws.03/ > I also came over the changed files and added new lines to make lines > shorter. > > Thanks > Miran > >> >> best regards, >> >> -- daniel >> >> On 15/05/15 17:13, Miroslav Kos wrote: >>> Hi everybody, >>> >>> this is review request for: 8072839: JAX-B Plugability Layer: using >>> java.util.ServiceLoader >>> The JAX-B API changed a little bit - proprietary ServiceLoader-like >>> code >>> has been replaced by java.util.ServiceLoader. This change is >>> required by >>> Jigsaw, old configuration way still supported. >>> >>> JBS:https://bugs.openjdk.java.net/browse/JDK-8072839 >>> webrev: http://cr.openjdk.java.net/~mkos/8072839/jaxws.02/index.html >>> CCC: http://ccc.us.oracle.com/8072839 >>> >>> Testing: JAX-WS (standalone) unit tests, no, jtreg tests available now, >>> TCK/JCK tests will require new tests. >>> >>> Thanks >>> Miran >>> >> > From paul.sandoz at oracle.com Tue Jun 2 13:13:04 2015 From: paul.sandoz at oracle.com (Paul Sandoz) Date: Tue, 2 Jun 2015 15:13:04 +0200 Subject: RFR 8071597 Add Stream dropWhile and takeWhile operations Message-ID: <79392DA7-6A8E-42AC-9E83-AA7F89D8F2A5@oracle.com> Hi, Please review this webrev that adds take/dropWhile operations to streams: http://cr.openjdk.java.net/~psandoz/jdk9/JDK-8071597-take-drop-while/webrev/ I opted to weight the documentation of the operations towards ordered streams in the first paragraph. That is what makes most sense in terms of usage and what most people will read. Thus i refer to the "longest prefix" in the first paragraph then define what that means in subsequent paragraphs for ordered and unordered streams: 482 /** 483 * Returns a stream consisting of the longest prefix of elements taken from 484 * this stream that match the given predicate. 485 * 486 *

    If this stream is ordered then the prefix is a contiguous sequence of 487 * elements of this stream. All elements of the sequence match the given 488 * predicate, the first element of the sequence is the first element 489 * (if any) of this stream, and the element (if any) immediately following 490 * the last element of the sequence does not match the given predicate. 491 * 492 *

    If this stream is unordered then the prefix is a subset of elements of 493 * this stream. All elements (if any) of the subset match the given 494 * predicate. In this case the behavior of this operation is 495 * nondeterministic; it is free to select any valid subset as the prefix. 496 * 497 *

    This is a short-circuiting 498 * stateful intermediate operation. 499 * ... 528 default Stream takeWhile(Predicate predicate) { 537 /** 538 * Returns a stream consisting of the remaining elements of this stream 539 * after dropping the longest prefix of elements that match the given 540 * predicate. 541 * 542 *

    If this stream is ordered then the prefix is a contiguous sequence of 543 * elements of this stream. All elements of the sequence match the given 544 * predicate, the first element of the sequence is the first element 545 * (if any) of this stream, and the element (if any) immediately following 546 * the last element of the sequence does not match the given predicate. 547 * 548 *

    If this stream is unordered then the prefix is a subset of elements of 549 * this stream. All elements (if any) of the subset match the given 550 * predicate. In this case the behavior of this operation is 551 * nondeterministic; it is free to select any valid subset as the prefix. 552 * ... 584 default Stream dropWhile(Predicate predicate) { After this has been reviewed i will follow up with a further issue regarding the specification of takeWhile, stateful predicates and cancellation. I avoided such specification here as it's likely to rathole :-) Basically the takeWhile operation is implemented such that one can do: long t = System.currentTimeMillis(); List pps = Stream .generate(() -> BigInteger.probablePrime(1024, ThreadLocalRandom.current())) .parallel() .takeWhile(e -> (System.currentTimeMillis() - t) < TimeUnit.SECONDS.toMillis(5)) .collect(toList()); Paul. From paul.sandoz at oracle.com Tue Jun 2 13:37:29 2015 From: paul.sandoz at oracle.com (Paul Sandoz) Date: Tue, 2 Jun 2015 15:37:29 +0200 Subject: RFR 8081678: Add Stream returning methods to classes where there currently exist only Enumeration returning methods Message-ID: <3093DBE7-AC02-4371-B7EC-12C3A15EA334@oracle.com> Hi, Please review a follow up to Stuart's Enumeration.asIterator patch that adds some Stream return methods to classes where there is only Enumeration returning methods to support traversal: http://cr.openjdk.java.net/~psandoz/jdk9/JDK-8081678-enumeration-and-stream/webrev/ I took the opportunity to sprinkle some Enumerator.asIterator impls. I originally included some methods on ClassLoader but Alan wisely warned me away from doing that as this area is likely to be impacted by modularity. There is one small area of uncertainty with NetworkInterface. Can the following method ever return null? 342 public static Enumeration getNetworkInterfaces() 343 throws SocketException { 344 NetworkInterface[] netifs = getAll(); 345 346 // specified to return null if no network interfaces 347 return netifs != null 348 ? enumerationFromArray(netifs) 349 : null; 350 } Contrary to the comment i cannot find any specification. For the stream returning method, networkInterfaces, i have specified this to return an empty stream, thus it might be good to update the enumeration returning method as well to say whether it returns null or an empty enumeration. Paul. From forax at univ-mlv.fr Tue Jun 2 13:47:38 2015 From: forax at univ-mlv.fr (Remi Forax) Date: Tue, 02 Jun 2015 15:47:38 +0200 Subject: RFR 8071597 Add Stream dropWhile and takeWhile operations In-Reply-To: <79392DA7-6A8E-42AC-9E83-AA7F89D8F2A5@oracle.com> References: <79392DA7-6A8E-42AC-9E83-AA7F89D8F2A5@oracle.com> Message-ID: <556DB3FA.5040700@univ-mlv.fr> Nice, another example of takeWhile is how to iterate over a linked list, class Entry { private final Entry next; private final Object value; Entry getNext() { return next; } Object getValue() { return value; } } Entry head = ... Stream.iterate(head, Entry::getNext) .takeWhile(Objects::nonNull) .map(Entry::getValue) .forEach(System.out::println); cheers, R?mi On 06/02/2015 03:13 PM, Paul Sandoz wrote: > Hi, > > Please review this webrev that adds take/dropWhile operations to streams: > > http://cr.openjdk.java.net/~psandoz/jdk9/JDK-8071597-take-drop-while/webrev/ > > I opted to weight the documentation of the operations towards ordered streams in the first paragraph. That is what makes most sense in terms of usage and what most people will read. Thus i refer to the "longest prefix" in the first paragraph then define what that means in subsequent paragraphs for ordered and unordered streams: > > 482 /** > 483 * Returns a stream consisting of the longest prefix of elements taken from > 484 * this stream that match the given predicate. > 485 * > 486 *

    If this stream is ordered then the prefix is a contiguous sequence of > 487 * elements of this stream. All elements of the sequence match the given > 488 * predicate, the first element of the sequence is the first element > 489 * (if any) of this stream, and the element (if any) immediately following > 490 * the last element of the sequence does not match the given predicate. > 491 * > 492 *

    If this stream is unordered then the prefix is a subset of elements of > 493 * this stream. All elements (if any) of the subset match the given > 494 * predicate. In this case the behavior of this operation is > 495 * nondeterministic; it is free to select any valid subset as the prefix. > 496 * > 497 *

    This is a short-circuiting > 498 * stateful intermediate operation. > 499 * > ... > 528 default Stream takeWhile(Predicate predicate) { > > 537 /** > 538 * Returns a stream consisting of the remaining elements of this stream > 539 * after dropping the longest prefix of elements that match the given > 540 * predicate. > 541 * > 542 *

    If this stream is ordered then the prefix is a contiguous sequence of > 543 * elements of this stream. All elements of the sequence match the given > 544 * predicate, the first element of the sequence is the first element > 545 * (if any) of this stream, and the element (if any) immediately following > 546 * the last element of the sequence does not match the given predicate. > 547 * > 548 *

    If this stream is unordered then the prefix is a subset of elements of > 549 * this stream. All elements (if any) of the subset match the given > 550 * predicate. In this case the behavior of this operation is > 551 * nondeterministic; it is free to select any valid subset as the prefix. > 552 * > ... > 584 default Stream dropWhile(Predicate predicate) { > > > After this has been reviewed i will follow up with a further issue regarding the specification of takeWhile, stateful predicates and cancellation. I avoided such specification here as it's likely to rathole :-) > > Basically the takeWhile operation is implemented such that one can do: > > long t = System.currentTimeMillis(); > List pps = Stream > .generate(() -> BigInteger.probablePrime(1024, ThreadLocalRandom.current())) > .parallel() > .takeWhile(e -> (System.currentTimeMillis() - t) < TimeUnit.SECONDS.toMillis(5)) > .collect(toList()); > > Paul. From spliterator at gmail.com Tue Jun 2 13:50:48 2015 From: spliterator at gmail.com (Stefan Zobel) Date: Tue, 2 Jun 2015 15:50:48 +0200 Subject: Fwd: RFR 8071597 Add Stream dropWhile and takeWhile operations In-Reply-To: <79392DA7-6A8E-42AC-9E83-AA7F89D8F2A5@oracle.com> References: <79392DA7-6A8E-42AC-9E83-AA7F89D8F2A5@oracle.com> Message-ID: Hi Paul, Looks good. I was wondering why the truncate method in Node.OfInt / OfLong / OfDouble did not receive the same + if (to == count()) { + spliterator.forEachRemaining(nodeBuilder); + } else { + for (int i = 0; i < size && spliterator.tryAdvance(nodeBuilder); i++) { } + } treatment as the truncate method in Node itself. Looks pretty symmetric to my untrained eyes? Thanks, Stefan From paul.sandoz at oracle.com Tue Jun 2 14:10:57 2015 From: paul.sandoz at oracle.com (Paul Sandoz) Date: Tue, 2 Jun 2015 16:10:57 +0200 Subject: RFR 8071597 Add Stream dropWhile and takeWhile operations In-Reply-To: References: <79392DA7-6A8E-42AC-9E83-AA7F89D8F2A5@oracle.com> Message-ID: <7AC6D067-0780-453E-B82E-5741141A7492@oracle.com> On Jun 2, 2015, at 3:50 PM, Stefan Zobel wrote: > Hi Paul, > > Looks good. > > I was wondering why the truncate method in Node.OfInt / OfLong / OfDouble > did not receive the same > > > + if (to == count()) { > + spliterator.forEachRemaining(nodeBuilder); > + } else { > + for (int i = 0; i < size && > spliterator.tryAdvance(nodeBuilder); i++) { } > + } > > > treatment as the truncate method in Node itself. Looks pretty symmetric to > my untrained eyes? > Indeed so, thanks for noticing. Webrev updated in place. http://cr.openjdk.java.net/~psandoz/jdk9/JDK-8071597-take-drop-while/webrev/src/java.base/share/classes/java/util/stream/Node.java.sdiff.html Paul. From kirk.pepperdine at gmail.com Mon Jun 1 09:41:44 2015 From: kirk.pepperdine at gmail.com (Kirk Pepperdine) Date: Mon, 1 Jun 2015 11:41:44 +0200 Subject: JEP 132: More-prompt finalization In-Reply-To: <556B61B0.5030708@gmail.com> References: <55674C6E.8050509@gmail.com> <5567CC5E.7000102@oracle.com> <8DBC7345-09D8-458F-9017-298D1E620F86@gmail.com> <6882C9A35DFB9B4FA2779F7BF5B9757D20761169A2@GSCMAMP06EX.firmwide.corp.gs.com> <556B61B0.5030708@gmail.com> Message-ID: Hi Peter, Interesting email. I think it is a thoughtful contribution and these are great responses to concerns and questions. I hope it receives the due consideration it deserves. Kind regards, Kirk On May 31, 2015, at 9:32 PM, Peter Levart wrote: > Hi, > > Thanks for views and opinions. I'll try to confront them in-line... > > On 05/29/2015 04:18 AM, David Holmes wrote: >> Hi Peter, >> >> I guess I'm very concerned about the premise that finalization should scale to millions of objects and be performed highly concurrently. To me that's sending the wrong message about finalization. It also isn't the most effective use of cpu resources - most people would want to do useful work on most cpu's most of the time. >> >> Cheers, >> David > > @David > > Ok, fair enough. It shouldn't be necessary to scale finalization to millions of objects and be performed concurrently. Normal programs don't need this. But there is a diagnostic command being developed at this moment that displays the finalization queue. The utility of such command, as I understand, is precisely to display when the finalization thread can not cope and Finalizer(s) accumulate. So there must be that some hypothetical programs (ab)use finalization or are buggy (deadlock) so that the queue builds up. To diagnose this, a diagnostic command is helpful. To fix it, one has to fix the code. But what if the problem is not that much about the allocation/death rate of finalizable instances then it is about the heavy code of finalize() methods of those instances. I agree that such programs have a smell and should be rewritten to not use finalization but other means of cleanup such as multiple threads removing WeakReferences from the queue for example or something completely different and not based on Reference(s). But wouldn't it be nice if one could simply set a system property for the max. number of threads processing Finalizer(s)? > > I have prepared an improved variant of the prototype that employs a single ReferenceHandler thread and adds a ForkJoinPool that by default has a single worker thread which replaces the single finalization thread. So by default, no more threads are used than currently. If one wants (s)he can increase the concurrency of finalization with a system property. > > I have also improved the benchmarks that now focus on CPU overhead when processing references at more typical rates, rather than maximum throughput. They show that all changes taken together practically half the CPU time overhead of the finalization processing. So freed CPU time can be used for more useful work. I have also benchmarked the typical asynchronous WeakReference processing scenario where one thread removes enqueued WeakReferences from the queue. Results show about 25% decrease of CPU time overhead. > > Why does the prototype reduce more overhead for finalization than WeakReference processing? The main improvement in the change is the use of multiple doubly-linked lists for registration of Finalizer(s) and the use of lock-less algorithm for the lists. The WeakReference processing benchmark also uses such lists internally to handle registration/deregistration of WeakReferences so that the impact of this part is minimal and the difference of processing overheads between original and changed JDK code more obvious. (De)registration of Finalizer(s) OTOH is part of JDK infrastructure, so the improvement to registration list(s) also shows in the results. The results of WeakReferece processing benchmark also indicate that reverting to the use of a single finalization thread that just removes Finalizer(s) from the ReferenceQueue could lower the overhead even a bit further, but then it would not be possible to leverage FJ pool to simply configure the parallelism of finalization. If parallel processing of Finalizer(s) is an undesirable feature, I could restore the single finalization thread and the CPU overhead of finalization would be reduced to about 40% of current overhead with just the changes to data structures. > > So, for the curious, here's the improved prototype: > > http://cr.openjdk.java.net/~plevart/misc/JEP132/ReferenceHandling/webrev.02/ > > And here are the improved benchmarks (with some results inline): > > http://cr.openjdk.java.net/~plevart/misc/JEP132/ReferenceHandling/refproc/ > > > The benchmark results in the ThroughputBench.java show the output of the test(s) when run with the Linux "time" command which shows the elapsed real time and the consumed user and system CPU times. I think this is relevant for measuring CPU overhead. > > So my question is: Is it or is it not desirable to have a configurable means to parallelize the finalization processing? The reduction of CPU overhead of infrastructure code should always be desirable, right? > > On 05/29/2015 05:57 AM, Kirk Pepperdine wrote: >> Hi Peter, >> >> It is a very interesting proposal but to further David?s comments, the life-cycle costs of reference objects is horrendous of which the actual process of finalizing an object is only a fraction of that total cost. Unfortunately your micro-benchmark only focuses on one aspect of that cost. In other words, it isn?t very representative of a real concern. In the real world the finalizer *must compete with mutator threads and since F-J is an ?all threads on deck? implementation, it doesn?t play well with others. It creates a ?tragedy of the commons?. That is situations where everyone behaves rationally with a common resource but to the detriment of the whole group?. In short, parallelizing (F-Jing) *everything* in an application is simply not a good idea. We do not live in an infinite compute environment which means to have to consider the impact of our actions to the entire group. > > @Kirk > > I changed the prototype to only use a single FJ thread by default (configurable with a system property). Lowering the CPU overhead of finalizer processing for 50% is also an improvement. I'm still keeping finalization FJ-pool for now because it is more scaleable and has less overhead than a solution with multiple threads removing references from the same ReferenceQueue. This happens when the FJ-pool is configured with > 1 parallelism or when user code calls Runtime.runFinalization() that translates to ForkJoinPool.awaitQuiescence() which lends the calling thread to help the poll execute the tasks. > >> This was one of the points of my recent article in Java Magazine which I wrote to try to counter some of the rhetoric I was hearing in conference about the universal benefits of being able easily parallelize streams in Java 8. Yes, I agree it?s a great feature but it must be used with discretion. Case in point. After I finished writing the article, I started running into a couple of early adopters that had swallowed the parallel message whole indiscriminately parallelizing all of their streams. As you can imagine, they were quite surprised by the results and quickly worked to de-parallelize *all* of the streams in the application. >> >> To add some ability to parallelize the handling of reference objects seems like a good idea if you are collecting large numbers of reference objects (>10,000 per GC cycle). However if you are collecting large numbers of reference objects you?re most likely doing something else wrong. IME, finalization is extremely useful but really only for a limited number of use cases and none of them (to date) have resulted in the app burning through 1000s of final objects / sec. >> >> It would be interesting to know why why you picked on this particular issue. > > Well, JEP-132 was filed by Oracle, so I thought I'll try to tackle some of it's goals. I think I at least showed that the VM part of reference handling is mostly not the performance problem (if there is a problem at all), but the Java side could be modernized a bit. > >> Kind regards, >> Kirk > > On 05/29/2015 07:20 PM, Rezaei, Mohammad A. wrote: >> For what it's worth, I fully agree with David and Kirk around finalization not necessarily needing this treatment. >> >> However, I was hoping this would have the effect of improving (non-finalizable) reference handling. We've seen serious issues in WeakReference handling and have had to write some twisted code to deal with this. > > @Moh > > Can you elaborate some more on what twists were necessary or what problems you had? > >> So I guess the question I have to Kirk and David is: do you feel a GC load of 10K WeakReferences per cycle is also "doing something else wrong"? > > If there is an elegant way to achieve your goal without using WeakReferences then it might be better to not use them. But it is also true that WeakReferences frequently lend an elegant way to solve a problem. The same goes with finalization which is sometimes even more elegant. > >> Sorry if this is going off-topic. > > You're spot on topic and thanks for your comment. > >> Thanks >> Moh >> >> > > > Regards, Peter > From volker.simonis at gmail.com Tue Jun 2 16:12:02 2015 From: volker.simonis at gmail.com (Volker Simonis) Date: Tue, 2 Jun 2015 18:12:02 +0200 Subject: RFR(M): 8081674: EmptyStackException at startup if running with extended or unsupported charset Message-ID: Hi, can I please have review (and a sponsor) for these changes: https://bugs.openjdk.java.net/browse/JDK-8081674 http://cr.openjdk.java.net/~simonis/webrevs/2015/8081674.jdk http://cr.openjdk.java.net/~simonis/webrevs/2015/8081674.hs Please notice that the fix requires synchronous changes in the jdk and the hotspot forest. The changes themselves are by far not that big as this explanation but I found the problem to be quite intricate so I tried to explain it as good as I could. I'd suggest to read the HTML-formatted explanation in the webrev although the content is equal to the one in this mail: Using an unsupported character encoding (e.g. vi_VN.TCVN on Linux) results in an immediate VM failure with jdk 8 and 9: export LANG=vi_VN.TCVN java -version Error occurred during initialization of VM java.util.EmptyStackException at java.util.Stack.peek(Stack.java:102) at java.lang.ClassLoader$NativeLibrary.getFromClass(ClassLoader.java:1751) at java.lang.ClassLoader$NativeLibrary.findBuiltinLib(Native Method) at java.lang.ClassLoader.loadLibrary0(ClassLoader.java:1862) at java.lang.ClassLoader.loadLibrary(ClassLoader.java:1835) at java.lang.Runtime.loadLibrary0(Runtime.java:870) at java.lang.System.loadLibrary(System.java:1119) at java.lang.System.initializeSystemClass(System.java:1194) This is a consequence of "8005716: Enhance JNI specification to allow support of static JNI libraries in Embedded JREs". With jdk 9 we get this error even if we're running with a supported charset which is in the ExtendedCharsets (as opposed to being in the StandardCharsets) class which is a consequence of delegating the loading of the ExtendedCharsets class to the ServiceLoader in jdk 9. export LANG=eo.iso-8859-3 output-jdk9/images/jdk/bin/java -version Error occurred during initialization of VM java.util.EmptyStackException at java.util.Stack.peek(Stack.java:102) at java.lang.ClassLoader$NativeLibrary.getFromClass(ClassLoader.java:1737) at java.lang.ClassLoader$NativeLibrary.findBuiltinLib(Native Method) at java.lang.ClassLoader.loadLibrary0(ClassLoader.java:1866) at java.lang.ClassLoader.loadLibrary(ClassLoader.java:1840) at java.lang.Runtime.loadLibrary0(Runtime.java:874) at java.lang.System.loadLibrary(System.java:1111) at java.lang.System.initializeSystemClass(System.java:1186) Here's why the exception happens for an unsupported charset (see the mixed stack trace below for the full details): java.lang.System.loadLibrary() wants to load libzip.so. It calls java.lang.Runtime.loadLibrary0() which at the very beginning calls the native method ClassLoader$NativeLibrary.findBuiltinLib() which checks if the corresponding library is already statically linked into the VM (introduced by 8005716). Java_java_lang_ClassLoader_00024NativeLibrary_findBuiltinLib(), the native implementation of findBuiltinLib() in Classloader.c calls GetStringPlatformChars() to convert the library name into the native platform encoding. GetStringPlatformChars() calls the helper function jnuEncodingSupported() to check if the platform encoding which is stored in the property "sun.jnu.encoding" is supported by Java. jnuEncodingSupported() is implemented as follows: static jboolean isJNUEncodingSupported = JNI_FALSE; static jboolean jnuEncodingSupported(JNIEnv *env) { jboolean exe; if (isJNUEncodingSupported == JNI_TRUE) { return JNI_TRUE; } isJNUEncodingSupported = (jboolean) JNU_CallStaticMethodByName ( env, &exe, "java/nio/charset/Charset", "isSupported", "(Ljava/lang/String;)Z", jnuEncoding).z; return isJNUEncodingSupported; } Once the function finds that the platform encoding is supported (by calling java.nio.charset.Charset.isSupported()) it caches this value and always returns it on following calls. However if the platform encoding is not supported, it ALWAYS calls java.nio.charset.Charset.isSupported() an every subsequent invocation. In order to call the Java method Charset.isSupported() (in JNU_CallStaticMethodByName() in file jni_util.c), we have to call jni_FindClass() to convert the symbolic class name "java.nio.charset.Charset" into a class reference. But unfortunately, jni_FindClass() (from jni.cpp in libjvm.so) has a special handling if called from java.lang.ClassLoader$NativeLibrary to ensure that JNI_OnLoad/JNI_OnUnload are executed in the correct class context: instanceKlassHandle k (THREAD, thread->security_get_caller_class(0)); if (k.not_null()) { loader = Handle(THREAD, k->class_loader()); // Special handling to make sure JNI_OnLoad and JNI_OnUnload are executed // in the correct class context. if (loader.is_null() && k->name() == vmSymbols::java_lang_ClassLoader_NativeLibrary()) { JavaValue result(T_OBJECT); JavaCalls::call_static(&result, k, vmSymbols::getFromClass_name(), vmSymbols::void_class_signature(), thread); if (HAS_PENDING_EXCEPTION) { Handle ex(thread, thread->pending_exception()); CLEAR_PENDING_EXCEPTION; THROW_HANDLE_0(ex); } So if that's the case and jni_FindClass() was reallycalled from ClassLoader$NativeLibrary, then jni_FindClass() calles ClassLoader$NativeLibrary().getFromClass() to find out the corresponding context class which is supposed to be saved there in a field of type java.util.Stack named "nativeLibraryContext": // Invoked in the VM to determine the context class in // JNI_Load/JNI_Unload static Class getFromClass() { return ClassLoader.nativeLibraryContext.peek().fromClass; } Unfortunately, "nativeLibraryContext" doesn't contain any entry at this point and the invocation of Stack.peek() will throw the exception shown before. In general, the "nativeLibraryContext" stack will be filled later on in Runtime.loadLibrary0() like this: NativeLibrary lib = new NativeLibrary(fromClass, name, isBuiltin); nativeLibraryContext.push(lib); try { lib.load(name, isBuiltin); } finally { nativeLibraryContext.pop(); } such that it always contains at least one element later when jni_FindClass() will be invoked. So in summary, the problem is that the implementors of 8005716 didn't took into account that calling ClassLoader$NativeLibrary.findBuiltinLib() may trigger a call to jni_FindClass() if we are running on a system with an unsupported character encoding. I'd suggest the following fix for this problem: Change ClassLoader$NativeLibrary().getFromClass() to return null if the stack is empty instead of throwing an exception: static Class getFromClass() { return ClassLoader.nativeLibraryContext.empty() ? null : ClassLoader.nativeLibraryContext.peek().fromClass; } Unfortunately this also requires a HotSpot change in jni_FindClass() in order to properly handle the new 'null' return value: if (k.not_null()) { loader = Handle(THREAD, k->class_loader()); // Special handling to make sure JNI_OnLoad and JNI_OnUnload are executed // in the correct class context. if (loader.is_null() && k->name() == vmSymbols::java_lang_ClassLoader_NativeLibrary()) { JavaValue result(T_OBJECT); JavaCalls::call_static(&result, k, vmSymbols::getFromClass_name(), vmSymbols::void_class_signature(), thread); if (HAS_PENDING_EXCEPTION) { Handle ex(thread, thread->pending_exception()); CLEAR_PENDING_EXCEPTION; THROW_HANDLE_0(ex); } oop mirror = (oop) result.get_jobject(); if (oopDesc::is_null(mirror)) { loader = Handle(THREAD, SystemDictionary::java_system_loader()); } else { loader = Handle(THREAD, InstanceKlass::cast(java_lang_Class::as_Klass(mirror))->class_loader()); protection_domain = Handle(THREAD, InstanceKlass::cast(java_lang_Class::as_Klass(mirror))->protection_domain()); } } } else { // We call ClassLoader.getSystemClassLoader to obtain the system class loader. loader = Handle(THREAD, SystemDictionary::java_system_loader()); } These changes are sufficient to solve the problem in Java 8. Unfortunately, that's still not enough in Java 9 because there the loading of the extended charsets has been delegated to ServiceLoader. But ServiceLoader calls ClassLoader.getBootstrapResources() which calls sun.misc.Launcher.getBootstrapClassPath(). This leads to another problem during class initialization of sun.misc.Launcher if running on an unsupported locale. The first thing done in sun.misc.Launcher. is the initialisation of the bootstrap URLClassPath in the Launcher. However this initialisation will eventually call Charset.isSupported() and if we are running on an unsupported locale this will inevitably end in another recursive call to ServiceLoader. But as explained below, ServiceLoader will query the Launcher's bootstrap URLClassPath which will be still uninitialized at that point. So we'll have to additionally guard guard against this situation on JDK 9 like this: private static Charset lookupExtendedCharset(String charsetName) { if (!sun.misc.VM.isBooted() || // see lookupViaProviders() sun.misc.Launcher.getBootstrapClassPath() == null) return null; This fixes the crashes, but still at the price of not having the extended charsets available during initialization until Launcher.getBootstrapClassPath is set up properly. This may be still a problem if the jdk is installed in a directory which contains characters specific to an extended encoding or if we have such characters in the command line arguments. Thank you and best regards, Volker *Mixed stack trace of the initial EmptyStackException for unsupported charsets described before:* Native frames: (J=compiled Java code, j=interpreted, Vv=VM code, C=native code) j java.util.Stack.peek()Ljava/lang/Object;+1 j java.lang.ClassLoader$NativeLibrary.getFromClass()Ljava/lang/Class;+3 v ~StubRoutines::call_stub V [libjvm.so+0x9d279a] JavaCalls::call_helper(JavaValue*, methodHandle*, JavaCallArguments*, Thread*)+0x6b4 V [libjvm.so+0xcad591] os::os_exception_wrapper(void (*)(JavaValue*, methodHandle*, JavaCallArguments*, Thread*), JavaValue*, methodHandle*, JavaCallArguments*, Thread*)+0x45 V [libjvm.so+0x9d20cf] JavaCalls::call(JavaValue*, methodHandle, JavaCallArguments*, Thread*)+0x8b V [libjvm.so+0x9d1d3b] JavaCalls::call_static(JavaValue*, KlassHandle, Symbol*, Symbol*, JavaCallArguments*, Thread*)+0x139 V [libjvm.so+0x9d1e3f] JavaCalls::call_static(JavaValue*, KlassHandle, Symbol*, Symbol*, Thread*)+0x9d V [libjvm.so+0x9e6588] jni_FindClass+0x428 C [libjava.so+0x20208] JNU_CallStaticMethodByName+0xff C [libjava.so+0x21cae] jnuEncodingSupported+0x61 C [libjava.so+0x22125] JNU_GetStringPlatformChars+0x125 C [libjava.so+0xedcd] Java_java_lang_ClassLoader_00024NativeLibrary_findBuiltinLib+0x8b j java.lang.ClassLoader$NativeLibrary.findBuiltinLib(Ljava/lang/String;)Ljava/lang/String;+0 j java.lang.ClassLoader.loadLibrary0(Ljava/lang/Class;Ljava/io/File;)Z+4 j java.lang.ClassLoader.loadLibrary(Ljava/lang/Class;Ljava/lang/String;Z)V+228 j java.lang.Runtime.loadLibrary0(Ljava/lang/Class;Ljava/lang/String;)V+54 j java.lang.System.loadLibrary(Ljava/lang/String;)V+7 j java.lang.System.initializeSystemClass()V+113 v ~StubRoutines::call_stub V [libjvm.so+0x9d279a] JavaCalls::call_helper(JavaValue*, methodHandle*, JavaCallArguments*, Thread*)+0x6b4 V [libjvm.so+0xcad591] os::os_exception_wrapper(void (*)(JavaValue*, methodHandle*, JavaCallArguments*, Thread*), JavaValue*, methodHandle*, JavaCallArguments*, Thread*)+0x45 V [libjvm.so+0x9d20cf] JavaCalls::call(JavaValue*, methodHandle, JavaCallArguments*, Thread*)+0x8b V [libjvm.so+0x9d1d3b] JavaCalls::call_static(JavaValue*, KlassHandle, Symbol*, Symbol*, JavaCallArguments*, Thread*)+0x139 V [libjvm.so+0x9d1e3f] JavaCalls::call_static(JavaValue*, KlassHandle, Symbol*, Symbol*, Thread*)+0x9d V [libjvm.so+0xe3cceb] call_initializeSystemClass(Thread*)+0xb0 V [libjvm.so+0xe44444] Threads::initialize_java_lang_classes(JavaThread*, Thread*)+0x21a V [libjvm.so+0xe44b12] Threads::create_vm(JavaVMInitArgs*, bool*)+0x4a6 V [libjvm.so+0xa19bd7] JNI_CreateJavaVM+0xc7 C [libjli.so+0xa520] InitializeJVM+0x154 C [libjli.so+0x8024] JavaMain+0xcc From staffan.friberg at oracle.com Tue Jun 2 17:27:20 2015 From: staffan.friberg at oracle.com (Staffan Friberg) Date: Tue, 02 Jun 2015 10:27:20 -0700 Subject: RFR 8080640: Reduce copying when reading JAR/ZIP entries In-Reply-To: <555F8E78.40404@oracle.com> References: <555A957F.5010102@oracle.com> <555CCAFB.4090805@oracle.com> <555E0C62.9090406@oracle.com> <555E1D49.9000707@oracle.com> <555F7862.9030307@oracle.com> <555F7AB8.1050202@oracle.com> <555F8E78.40404@oracle.com> Message-ID: <556DE778.1040300@oracle.com> On 05/22/2015 01:15 PM, Staffan Friberg wrote: > On 05/22/2015 11:51 AM, Xueming Shen wrote: >> On 05/22/2015 11:41 AM, Staffan Friberg wrote: >>> >>> On 05/21/2015 11:00 AM, Staffan Friberg wrote: >>>> >>>> On 05/21/2015 09:48 AM, Staffan Friberg wrote: >>>>> >>>>> On 05/20/2015 10:57 AM, Xueming Shen wrote: >>>>>> On 05/18/2015 06:44 PM, Staffan Friberg wrote: >>>>>>> Hi, >>>>>>> >>>>>>> Wanted to get reviews and feedback on this performance >>>>>>> improvement for reading from JAR/ZIP files during classloading >>>>>>> by reducing unnecessary copying and reading the entry in one go >>>>>>> instead of in small portions. This shows a significant >>>>>>> improvement when reading a single entry and for a large >>>>>>> application with 10k classes and 500+ JAR files it improved the >>>>>>> startup time by 4%. >>>>>>> >>>>>>> For more details on the background and performance results >>>>>>> please see the RFE entry. >>>>>>> >>>>>>> RFE - https://bugs.openjdk.java.net/browse/JDK-8080640 >>>>>>> WEBREV - http://cr.openjdk.java.net/~sfriberg/JDK-8080640/webrev.0 >>>>>>> >>>>>>> Cheers, >>>>>>> Staffan >>>>>> >>>>>> Hi Staffan, >>>>>> >>>>>> If I did not miss something here, from your use scenario it >>>>>> appears to me the only thing you really >>>>>> need here to help boost your performance is >>>>>> >>>>>> byte[] ZipFile.getAllBytes(ZipEntry ze); >>>>>> >>>>>> You are allocating a byte[] at use side and wrapping it with a >>>>>> ByteBuffer if the size is small enough, >>>>>> otherwise, you letting the ZipFile to allocate a big enough one >>>>>> for you. It does not look like you >>>>>> can re-use that byte[] (has to be wrapped by the >>>>>> ByteArrayInputStream and return), why do you >>>>>> need two different methods here? The logic would be much easier >>>>>> to simply let the ZipFile to allocate >>>>>> the needed buffer with appropriate size, fill the bytes and >>>>>> return, with a "OOME" if the entry size >>>>>> is bigger than 2g. >>>>>> >>>>>> The only thing we use from the input ze is its name, get the >>>>>> size/csize from the jzentry, I don't think >>>>>> jzentry.csize/size can be "unknown", they are from the "cen" table. >>>>>> >>>>>> If the real/final use of the bytes is to wrap it with a >>>>>> ByteArrayInputStream,why bother using ByteBuffer >>>>>> here? Shouldn't a direct byte[] with exactly the size of the >>>>>> entry server better. >>>>>> >>>>>> -Sherman >>>>>> >>>>> Hi Sherman, >>>>> >>>>> Thanks for the comments. I agree, was starting out with bytebuffer >>>>> because I was hoping to be able to cache things where the buffer >>>>> was being used, but since the buffer is past along further I >>>>> couldn't figure out a clean way to do it. >>>>> Will rewrite it to simply just return a buffer, and only wrap it >>>>> in the Resource class getByteBuffer. >>>>> >>>>> What would be your thought on updating the ZipFile.getInputStream >>>>> to return ByteArrayInputStream for small entries? Currently I do >>>>> that work outside in two places and moving it would potentially >>>>> speed up others reading small entries as well. >>>>> >>>>> Thanks, >>>>> Staffan >>>> Just realized that my use of ByteArrayInputStream would miss Jar >>>> verification if enabled so the way to go hear would be to add it if >>>> possible to the ZipFile.getInputStream. >>>> >>>> //Staffan >>> Hi, >>> >>> Here is an updated webrev which uses a byte[] directly and also uses >>> ByteArrayInputStream in ZipFile for small entries below 128k. >>> >> >> I'm not sure about the benefit of doing the ByteArrayInputStream in >> ZipFile.getInputStream. It has >> the consequence of changing the "expected" behavior of >> getInputStream() (instead of return an >> input stream waiting for reading, it now reads all bytes in advance), >> something we might not want >> to do in a performance tuning. Though it might be reasonable to guess >> everyone get an input stream >> is to read all bytes from it later. >> >> -Sherman >> >> >>> http://cr.openjdk.java.net/~sfriberg/JDK-8080640/webrev.1 >>> >>> //Staffan >> > Agree that it will change the behavior slightly, but as you said it is > probably expected that some one will read the stream eventually. > We could reduce the size further if that makes a difference, if the > size is below 65k we would not use more memory than the buffer > allocated for the InflaterStream today. > The total allocation would be slightly larger for deflated entries as > we would allocate a byte[] for the compressed bytes, but it would be > GC:able and not kept alive. So from a memory perspective the > difference is very limited. > > //Staffan Hi, Bumping this thread to get some more comments on the concern about changing the ZipFile.getInputStream behavior. The benefit of doing this change is that any read of small entries from ZIP and JAR files will be much faster and less resources will be held, including native resources normally held by the ZipInputStream. The behavior change that will occur is that the full entry will be read as part of creating the stream and not lazily as might be expected. However when getting a today InputStream zip file will be accessed to read information about the size of the entry, so the zip file is already touched when getting an InputStream, but not the compressed bytes. I'm fine with removing this part of the change and just push the private getBytes function and the updates to the JDK libraries to use it. Thanks, Staffan From Roger.Riggs at Oracle.com Tue Jun 2 18:33:04 2015 From: Roger.Riggs at Oracle.com (Roger Riggs) Date: Tue, 02 Jun 2015 14:33:04 -0400 Subject: RFR 9: 8067808 : java/lang/ProcessBuilder/Basic.java failed on Assertion Message-ID: <556DF6E0.2010000@Oracle.com> Please review test cleanup to make process id testing more reliable. It now spawns javaChild to report the child pid instead of parsing the output of OS specific commands. Webrev: http://cr.openjdk.java.net/~rriggs/webrev-builder-8067808/ Issue: http://cr.openjdk.java.net/~rriggs/webrev-builder-8067808/ Thanks, Roger p.s. I'm looking for a review also for: 8081567 : java/lang/ProcessHandle/InfoTest.java failed Cannot run program "whoami" From vladimir.x.ivanov at oracle.com Tue Jun 2 18:51:08 2015 From: vladimir.x.ivanov at oracle.com (Vladimir Ivanov) Date: Tue, 02 Jun 2015 21:51:08 +0300 Subject: [8u-dev] Review request : JDK-8062904: TEST_BUG: Tests java/lang/invoke/LFCaching fail when run with -Xcomp option In-Reply-To: <556C86C6.2000409@oracle.com> References: <5565D9BB.60003@oracle.com> <5565F6D5.7030900@oracle.com> <5566D9D8.4090401@oracle.com> <55675CCB.8010506@oracle.com> <5568281B.7040108@oracle.com> <55685264.20001@oracle.com> <556C86C6.2000409@oracle.com> Message-ID: <556DFB1C.7010301@oracle.com> Konstantin, > It seems we have only this bug that manifests the problem. As I > understand, this is a product issue, not test. My question was about the symptoms - how the test can fail. If the test ignores NSME & VME exceptions, will it always pass w.r.t. code cache overflows? Code cache overflow is definitely a JVM problem, but we don't plan to address it in the near future. So, either the test should be excluded or adjusted to be tolerant to code cache overflows. Best regards, Vladimir Ivanov > > -Konstantin > > On 29.05.2015 14:49, Vladimir Ivanov wrote: >>> What do you mean by ignore code cache overflow? Do you mean I should fix >>> the test to ignore these errors or should I leave this test unfixed >>> because it is a product related issue? >> The former. How reliable the test is if it ignores >> NoSuchMethodException & VirtualMachineError? Are there other >> manifestations of the problem? >> >> Best regards, >> Vladimir Ivanov >> >>> On 28.05.2015 21:22, Vladimir Ivanov wrote: >>>> Got it, thanks. >>>> >>>> Can we ignore errors caused by code cache overflow for now? >>>> >>>> Best regards, >>>> Vladimir Ivanov >>>> >>>> On 5/28/15 12:03 PM, Konstantin Shefov wrote: >>>>> Vladimir, >>>>> >>>>> This fix is not for timeout issue, this fix is for >>>>> "java.lang.VirtualMachineError: out of space in CodeCache for >>>>> adapters". >>>>> >>>>> Timeout issue is other bug and should be filed separately. >>>>> I do not know why SQE added RULES with timeout to this bug. >>>>> >>>>> By the way, if -Xcomp is set on JDK 8u, test works if not more than >>>>> one >>>>> iteration is allowed. The same thing was for JDK 9 until >>>>> JDK-8046809 had >>>>> been fixed. >>>>> >>>>> -Konstantin >>>>> >>>>> On 27.05.2015 19:54, Vladimir Ivanov wrote: >>>>>> Have you tried to reduce iteration granularity? >>>>>> >>>>>> Probably, checking execution duration on every test case is more >>>>>> robust. >>>>>> >>>>>> Best regards, >>>>>> Vladimir Ivanov >>>>>> >>>>>> On 5/27/15 5:50 PM, Konstantin Shefov wrote: >>>>>>> Hello, >>>>>>> >>>>>>> Please review the test bug fix >>>>>>> https://bugs.openjdk.java.net/browse/JDK-8062904 >>>>>>> Webrev is http://cr.openjdk.java.net/~kshefov/8062904/webrev.01/ >>>>>>> Test fails only against JDK 8u and passes against JDK 9. >>>>>>> >>>>>>> Thanks >>>>>>> >>>>>>> -Konstantin >>>>> >>> > From chris.plummer at oracle.com Tue Jun 2 18:55:47 2015 From: chris.plummer at oracle.com (Chris Plummer) Date: Tue, 02 Jun 2015 11:55:47 -0700 Subject: [9] RFR (M) 8054386: Allow Java debugging when CDS is enabled In-Reply-To: <556D5CFD.1020709@oracle.com> References: <555B47EA.3000900@oracle.com> <556D5CFD.1020709@oracle.com> Message-ID: <556DFC33.4060600@oracle.com> I'm going to have to separate out the ProcessTool.java changes into a separate changeset (and CR). In the meantime, feel free to review what I have below. The code won't be changing at all when I separate out the ProcessTool.java changes. thanks, Chris On 6/2/15 12:36 AM, Chris Plummer wrote: > [Adding core-libs-dev at openjdk.java.net since this update includes > changes to jdk/test library code] > > Please review the updated webrev: > > Webrev: http://cr.openjdk.java.net/~cjplummer/8054386/webrev.02/ > Bug: https://bugs.openjdk.java.net/browse/JDK-8054386 > > There were concerns about the new hotspot tests referencing jdk tests. > One concern was that if the jdk tests change, they could break the > hotspot tests, and this might initially go undetected. The other > concern is that if the jdk and hotspot tests are placed in separate > test bundles, then it would not be possible to run the hotspot tests. > > Because of these concerns, I moved the tests that were in > hotspot/test/runtime/CDSJDITests to instead be in > jdk/test/com/sun/jdi/CDSJDITests. There was a slight renaming of the > tests in the process. Also, I had to update the jdk version of > ProcessTool.java to include the createJavaProcessBuilder() variant > that is in the hotspot version of ProcessTool.java. > > Lastly, in CDSJITTest.java I changed: > > OutputAnalyzer output = new OutputAnalyzer(pb.start()); > > to instead be: > > OutputAnalyzer output = ProcessTools.executeProcess(pb); > > I had to do this since the jdk version of the OutputAnalyzer > constructor is not public. The 1st version is what is commonly used in > hostspot tests, and the 2nd version is what is commonly used in jdk > tests. I decided to adopt the jdk way rather than make the > OutputAnalyzer constructors public, although this will probably happen > eventually when the two versions are unified. > > thanks, > > Chris > > > On 5/19/15 7:25 AM, Chris Plummer wrote: >> Hi, >> >> Please review the following changes for allowing java debugging when >> CDS is enabled. >> >> Webrev:http://cr.openjdk.java.net/~cjplummer/8054386/webrev.01/ >> Bug:https://bugs.openjdk.java.net/browse/JDK-8054386 >> >> The VM changes are simple. I removed the check that prevents >> debugging with CDS enabled, and added logic that will map the CDS >> archive RW when debugging is enabled. >> >> The tests are a bit more complex. There are a bunch of existing JDI >> tests for testing debugging support. Rather than start from scratch >> or clone them, I instead just wrote wrapper tests that put the >> relevant JDI test classes in the archive, and then invoke the JDI >> test. I did this for 3 of the JDI tests. If you feel there are others >> that would be good candidates, I'd be happy to add them. I'm looking >> for ones that would result in modification of the RO class metadata, >> such as setting a breakpoint (for which I already added two tests). >> >> Testing done: >> -Using JPRT to run the new jtreg tests on all platforms. >> -Using JPRT to run all jtreg runtime tests on linux x86 and x_64. >> -Regular JPRT "-testset hotspot" run >> -Putting the JCK JVMTI tests in the archive and then running them. >> -Putting the nsk jdb, jdwp, jvmti, and jdi tests in the archive and >> then running them. >> -Putting a simple test class in the archive and then setting a >> breakpoint on it using jdb >> >> Some of the above testing resulted in the discovery of bugs that >> still need to be addressed: JDK-8078644, JDK-8078730, and JDK-8079181. >> >> I also verified that without the change to map the archive RW, the >> above testing resulted in a SEGV, which is what you would expect (and >> actually want to see to prove that the testing is effective). >> >> thanks, >> >> Chris >> > From chris.hegarty at oracle.com Tue Jun 2 18:58:12 2015 From: chris.hegarty at oracle.com (Chris Hegarty) Date: Tue, 2 Jun 2015 19:58:12 +0100 Subject: RFR 8071597 Add Stream dropWhile and takeWhile operations In-Reply-To: <79392DA7-6A8E-42AC-9E83-AA7F89D8F2A5@oracle.com> References: <79392DA7-6A8E-42AC-9E83-AA7F89D8F2A5@oracle.com> Message-ID: <0EA606C1-FBBB-4466-BBB7-DF89C32E0CEC@oracle.com> Very nice. I just looked over the spec, for now. * @param predicate a non-interfering, * stateless * predicate to apply elements to determine the longest * prefix of elements. Is this missing a *to*, ?? predicate to apply to elements to determine the ?" -Chris. On 2 Jun 2015, at 14:13, Paul Sandoz wrote: > Hi, > > Please review this webrev that adds take/dropWhile operations to streams: > > http://cr.openjdk.java.net/~psandoz/jdk9/JDK-8071597-take-drop-while/webrev/ > > I opted to weight the documentation of the operations towards ordered streams in the first paragraph. That is what makes most sense in terms of usage and what most people will read. Thus i refer to the "longest prefix" in the first paragraph then define what that means in subsequent paragraphs for ordered and unordered streams: > > 482 /** > 483 * Returns a stream consisting of the longest prefix of elements taken from > 484 * this stream that match the given predicate. > 485 * > 486 *

    If this stream is ordered then the prefix is a contiguous sequence of > 487 * elements of this stream. All elements of the sequence match the given > 488 * predicate, the first element of the sequence is the first element > 489 * (if any) of this stream, and the element (if any) immediately following > 490 * the last element of the sequence does not match the given predicate. > 491 * > 492 *

    If this stream is unordered then the prefix is a subset of elements of > 493 * this stream. All elements (if any) of the subset match the given > 494 * predicate. In this case the behavior of this operation is > 495 * nondeterministic; it is free to select any valid subset as the prefix. > 496 * > 497 *

    This is a short-circuiting > 498 * stateful intermediate operation. > 499 * > ... > 528 default Stream takeWhile(Predicate predicate) { > > 537 /** > 538 * Returns a stream consisting of the remaining elements of this stream > 539 * after dropping the longest prefix of elements that match the given > 540 * predicate. > 541 * > 542 *

    If this stream is ordered then the prefix is a contiguous sequence of > 543 * elements of this stream. All elements of the sequence match the given > 544 * predicate, the first element of the sequence is the first element > 545 * (if any) of this stream, and the element (if any) immediately following > 546 * the last element of the sequence does not match the given predicate. > 547 * > 548 *

    If this stream is unordered then the prefix is a subset of elements of > 549 * this stream. All elements (if any) of the subset match the given > 550 * predicate. In this case the behavior of this operation is > 551 * nondeterministic; it is free to select any valid subset as the prefix. > 552 * > ... > 584 default Stream dropWhile(Predicate predicate) { > > > After this has been reviewed i will follow up with a further issue regarding the specification of takeWhile, stateful predicates and cancellation. I avoided such specification here as it's likely to rathole :-) > > Basically the takeWhile operation is implemented such that one can do: > > long t = System.currentTimeMillis(); > List pps = Stream > .generate(() -> BigInteger.probablePrime(1024, ThreadLocalRandom.current())) > .parallel() > .takeWhile(e -> (System.currentTimeMillis() - t) < TimeUnit.SECONDS.toMillis(5)) > .collect(toList()); > > Paul. From ecki at zusammenkunft.net Tue Jun 2 20:53:19 2015 From: ecki at zusammenkunft.net (Bernd Eckenfels) Date: Tue, 2 Jun 2015 22:53:19 +0200 Subject: JEP 132: More-prompt finalization In-Reply-To: <5567CC5E.7000102@oracle.com> References: <55674C6E.8050509@gmail.com> <5567CC5E.7000102@oracle.com> Message-ID: <20150602225319.00004eb6.ecki@zusammenkunft.net> Hello, Am Fri, 29 May 2015 12:18:06 +1000 schrieb David Holmes : > I guess I'm very concerned about the premise that finalization should > scale to millions of objects and be performed highly concurrently. I would agree that it is a bad idea to design applications for such demanding workloads and use finalizer and it is not a design goal for the finalizer to support this. > To > me that's sending the wrong message about finalization. It also isn't > the most effective use of cpu resources - most people would want to > do useful work on most cpu's most of the time. There is however a problem with the finalizer in sane applications: even when you do not use any domain objects with finalize() you still have to cope with JDK classes which needs it. This includes file streams, java.util.zip, sockets (especially SSLSocket which sends close records in a blocking manor (JDK-6932592)) and third party libraries (unfortunatelly). So if the finalizer queue builds up, it is often due to bad behaving objects and the finalizer has not much to deal with it. This could be as simple as diag commands, but it could also be a multi threaded approach where no more than one class finalizer is executed at the same time. I wish we had some (more) progress in this area. Greetings Bernd From dmitry.samersoff at oracle.com Tue Jun 2 21:04:53 2015 From: dmitry.samersoff at oracle.com (Dmitry Samersoff) Date: Wed, 03 Jun 2015 00:04:53 +0300 Subject: RFR(M, v11): JDK-8059036 : Implement Diagnostic Commands for heap and finalizerinfo In-Reply-To: <556D9C4C.9080200@oracle.com> References: <554894E9.8020908@oracle.com> <1AF219C7-1055-4D97-BB46-2FA390C4135D@oracle.com> <5548D65C.8040304@gmail.com> <554A8CEE.2090009@oracle.com> <55523418.5010708@oracle.com> <55527935.5010408@gmail.com> <5553B552.9060900@oracle.com> <555453BA.2070205@gmail.com> <55567748.6020304@oracle.com> <55567A68.9020802@oracle.com> <5556F34B.5050701@oracle.com> <55573A31.7050306@gmail.com> <55573C86.8030807@gmail.com> <5559D85B.2050500@oracle.com> <6FB8DF19-1D67-4702-9642-4C630765D2D1@oracle.com> <555C2EEB.1050505@oracle.com> <3941B430-FE6C-4AF0-A875-76D24411C654@oracle.com> <5564802A.2010403@oracle.com> <5565738A.4040109@gmail.com> <7EB0BE71-1608-4047-B71D-1A9EAA9083BB@oracle.com> <556727A2.9060209@gmail.com> <55675908.6030406@oracle.com> <556AF3CD.50900@oracle.com> <37227441-B016-4A9A-B9CA-BE100E992C9A@oracle.com> <556D8FB0.4070608@oracle.com> <556D9C4C.9080200@oracle.com> Message-ID: <556E1A75.3070705@oracle.com> Mikael, The reason of placing get_filed_offset_by_name to the oop.inline is that hotspot widely duplicate this code. Symbol is used to identify a field within klass, not a klass them self so I think it's OK to have it tied to the oopDesc. -Dmitry On 2015-06-02 15:06, Mikael Gerdin wrote: > Hi Dmitry, > > On 2015-06-02 13:12, Dmitry Samersoff wrote: >> Staffan, >> >>> Instead of hardcoding the field offsets, you can use >>> InstanceKlass::find_field and fieldDescriptor::offset to find the >>> offset at runtime. >> >> Done. Please, see >> >> http://cr.openjdk.java.net/~dsamersoff/JDK-8059036/webrev.11 >> >> I put the function int get_filed_offset_by_name(Symbol*,Symbol*) to >> oop.inline.hpp leaving a room for further cleanup because I found couple >> of places in hotspot that implements mostly similar functionality. > > Sorry for this sort of drive-by review, but I really don't think > get_field_offset_by_name should be in the oop class. If anywhere it > should be on InstanceKlass, and using Symbol* to identify a Klass* seems > incorrect since the same symbol can refer to different classes in > different class loader contexts. > > /Mikael > >> >> -Dmitry >> >> On 2015-06-01 10:18, Staffan Larsen wrote: >>> Dmitry, >>> >>> Instead of hardcoding the field offsets, you can use >>> InstanceKlass::find_field and fieldDescriptor::offset to find the >>> offset at runtime. >>> >>> Thanks, >>> /Staffan >>> >>>> On 31 maj 2015, at 13:43, Dmitry Samersoff >>>> wrote: >>>> >>>> Everyone, >>>> >>>> Please take a look at new version of the fix. >>>> >>>> http://cr.openjdk.java.net/~dsamersoff/JDK-8059036/webrev.10/ >>>> >>>> Changes (to previous version) are in >>>> Finalizer.java and diagnosticCommand.cpp >>>> >>>> This version copy data from Map.Entry<> to array of >>>> FinalizerHistogramEntry instances then, >>>> VM prints content of this array. >>>> >>>> -Dmitry >>>> >>>> >>>> On 2015-05-28 21:06, Mandy Chung wrote: >>>>> >>>>> On 05/28/2015 07:35 AM, Peter Levart wrote: >>>>>> Hi Mandy, >>>>>> >>>>>> On 05/27/2015 03:32 PM, Mandy Chung wrote: >>>>>>> Taking it further - is it simpler to return String[] of all >>>>>>> classnames including the duplicated ones and have the VM do the >>>>>>> count? Are you concerned with the size of the String[]? >>>>>> >>>>>> Yes, the histogram is much smaller than the list of all instances. >>>>>> There can be millions of instances waiting in finalizer queue, but >>>>>> only a few distinct classes. >>>>> >>>>> Do you happen to know what libraries are the major contributors to >>>>> these >>>>> millions of finalizers? >>>>> >>>>> It has been strongly recommended to avoid finalizers (see Effective >>>>> Java >>>>> Item 7). I'm not surprised that existing code is still using >>>>> finalizers while we should really encourage them to migrate away >>>>> from it. >>>>> >>>>>> What could be done in Java to simplify things in native code but >>>>>> still >>>>>> not format the output is to convert the array of Map.Entry(s) into an >>>>>> Object[] array of alternating {String, int[], String, int[], .... } >>>>>> >>>>>> Would this simplify native code for the price of a little extra work >>>>>> in Java? The sizes of old and new arrays are not big (# of distinct >>>>>> classes of finalizable objects in the queue). >>>>> >>>>> I also prefer writing in Java and writing less native code (much >>>>> simplified). It's an interface that we have to agree upon and keep it >>>>> simple. Having the returned Object[] as alternate String and int[] >>>>> is a >>>>> reasonable compromise. >>>>> >>>>> ReferenceQueue.java - you can move @SuppressWarning from the method to >>>>> just the field variable "rn" >>>>> @SuppressWarnings("unchecked") >>>>> Reference rn = r.next; >>>>> >>>>> Finalizer.java >>>>> It's better to rename printFinalizationQueue as it inspects the >>>>> finalizer reference queue (maybe getFinalizerHistogram?). Can you >>>>> move >>>>> this method to the end of this class and add the comment saying that >>>>> this is invoked by VM for jcmd -finalizerinfo support and @return to >>>>> describe the returned content. I also think you can remove >>>>> @SuppressWarnings for this method. >>>>> >>>>> Mandy >>>> >>>> >>>> -- >>>> Dmitry Samersoff >>>> Oracle Java development team, Saint Petersburg, Russia >>>> * I would love to change the world, but they won't give me the sources. >>> >> >> -- Dmitry Samersoff Oracle Java development team, Saint Petersburg, Russia * I would love to change the world, but they won't give me the sources. From ivan.gerasimov at oracle.com Tue Jun 2 21:37:01 2015 From: ivan.gerasimov at oracle.com (Ivan Gerasimov) Date: Wed, 03 Jun 2015 00:37:01 +0300 Subject: RFR 9: 8081567 : java/lang/ProcessHandle/InfoTest.java failed Cannot run program "whoami" In-Reply-To: <556CD0E1.2080102@Oracle.com> References: <556CD0E1.2080102@Oracle.com> Message-ID: <556E21FD.10203@oracle.com> Hi Roger! On 02.06.2015 0:38, Roger Riggs wrote: > Please review a change to report the user/owner in ProcessHandle.info to > have the same form for the owner identification as provided by > java.nio.Files. > On Windows it includes the domain with the user name. > > The test is updated to remove a dependency on the OS command whoami > and instead compare the users process with the owner of a file created > by the process improving portability. > > Webrev: > http://cr.openjdk.java.net/~rriggs/webrev-whoami-8081567/ > 1) + if (domainLen > 0) { + wcscat(domain, L"\\"); + } else { + domain[0] = L'\0'; + } + wcscat(domain, name); Here, domainLen is either equal to whatever LookupAccountSid has set it to. In MSDN [1] it's written: On input, specifies the size, in TCHARs, of the lpReferencedDomainName buffer. If the function fails because the buffer is too small or if cchReferencedDomainName is zero, cchReferencedDomainName receives the required buffer size, including the terminating null character. I see in the comments that in fact on success this parameter receives the length of the resulting string. But should we rely on that undocumented behavior? Does it really happen that LookupAccountSidW() returns 0, setting domainLen to 0 and leaving the domain buffer uninitialized? From the published example [2] it seems that this shouldn't happen. 2) I think domainLen should be set to DWORD domainLen = sizeof(domain) - sizeof(name) - 1; Otherwise LookupAccountSid is allowed to fill up the whole buffer, so that 'name' could not be appended. 3) Indentation isn't consistent: it should be multiple of 4. Sincerely yours, Ivan [1] https://msdn.microsoft.com/en-us/library/windows/desktop/aa379166%28v=vs.85%29.aspx [2] https://msdn.microsoft.com/en-us/library/windows/desktop/aa379554(v=vs.85).aspx > Issue: > https://bugs.openjdk.java.net/browse/JDK-8081567 > > Thanks, Roger > > From mikhailo.seledtsov at oracle.com Tue Jun 2 21:39:45 2015 From: mikhailo.seledtsov at oracle.com (Mikhailo Seledtsov) Date: Tue, 02 Jun 2015 14:39:45 -0700 Subject: [9] RFR (M) 8054386: Allow Java debugging when CDS is enabled In-Reply-To: <556DFC33.4060600@oracle.com> References: <555B47EA.3000900@oracle.com> <556D5CFD.1020709@oracle.com> <556DFC33.4060600@oracle.com> Message-ID: <556E22A1.8080605@oracle.com> Changes look good to me. Misha On 6/2/2015 11:55 AM, Chris Plummer wrote: > I'm going to have to separate out the ProcessTool.java changes into a > separate changeset (and CR). In the meantime, feel free to review what > I have below. The code won't be changing at all when I separate out > the ProcessTool.java changes. > > thanks, > > Chris > > On 6/2/15 12:36 AM, Chris Plummer wrote: >> [Adding core-libs-dev at openjdk.java.net since this update includes >> changes to jdk/test library code] >> >> Please review the updated webrev: >> >> Webrev: http://cr.openjdk.java.net/~cjplummer/8054386/webrev.02/ >> Bug: https://bugs.openjdk.java.net/browse/JDK-8054386 >> >> There were concerns about the new hotspot tests referencing jdk >> tests. One concern was that if the jdk tests change, they could break >> the hotspot tests, and this might initially go undetected. The other >> concern is that if the jdk and hotspot tests are placed in separate >> test bundles, then it would not be possible to run the hotspot tests. >> >> Because of these concerns, I moved the tests that were in >> hotspot/test/runtime/CDSJDITests to instead be in >> jdk/test/com/sun/jdi/CDSJDITests. There was a slight renaming of the >> tests in the process. Also, I had to update the jdk version of >> ProcessTool.java to include the createJavaProcessBuilder() variant >> that is in the hotspot version of ProcessTool.java. >> >> Lastly, in CDSJITTest.java I changed: >> >> OutputAnalyzer output = new OutputAnalyzer(pb.start()); >> >> to instead be: >> >> OutputAnalyzer output = ProcessTools.executeProcess(pb); >> >> I had to do this since the jdk version of the OutputAnalyzer >> constructor is not public. The 1st version is what is commonly used >> in hostspot tests, and the 2nd version is what is commonly used in >> jdk tests. I decided to adopt the jdk way rather than make the >> OutputAnalyzer constructors public, although this will probably >> happen eventually when the two versions are unified. >> >> thanks, >> >> Chris >> >> >> On 5/19/15 7:25 AM, Chris Plummer wrote: >>> Hi, >>> >>> Please review the following changes for allowing java debugging when >>> CDS is enabled. >>> >>> Webrev:http://cr.openjdk.java.net/~cjplummer/8054386/webrev.01/ >>> Bug:https://bugs.openjdk.java.net/browse/JDK-8054386 >>> >>> The VM changes are simple. I removed the check that prevents >>> debugging with CDS enabled, and added logic that will map the CDS >>> archive RW when debugging is enabled. >>> >>> The tests are a bit more complex. There are a bunch of existing JDI >>> tests for testing debugging support. Rather than start from scratch >>> or clone them, I instead just wrote wrapper tests that put the >>> relevant JDI test classes in the archive, and then invoke the JDI >>> test. I did this for 3 of the JDI tests. If you feel there are >>> others that would be good candidates, I'd be happy to add them. I'm >>> looking for ones that would result in modification of the RO class >>> metadata, such as setting a breakpoint (for which I already added >>> two tests). >>> >>> Testing done: >>> -Using JPRT to run the new jtreg tests on all platforms. >>> -Using JPRT to run all jtreg runtime tests on linux x86 and x_64. >>> -Regular JPRT "-testset hotspot" run >>> -Putting the JCK JVMTI tests in the archive and then running them. >>> -Putting the nsk jdb, jdwp, jvmti, and jdi tests in the archive and >>> then running them. >>> -Putting a simple test class in the archive and then setting a >>> breakpoint on it using jdb >>> >>> Some of the above testing resulted in the discovery of bugs that >>> still need to be addressed: JDK-8078644, JDK-8078730, and JDK-8079181. >>> >>> I also verified that without the change to map the archive RW, the >>> above testing resulted in a SEGV, which is what you would expect >>> (and actually want to see to prove that the testing is effective). >>> >>> thanks, >>> >>> Chris >>> >> > From lance.andersen at oracle.com Tue Jun 2 22:06:39 2015 From: lance.andersen at oracle.com (Lance Andersen) Date: Tue, 2 Jun 2015 18:06:39 -0400 Subject: RFR 8081678: Add Stream returning methods to classes where there currently exist only Enumeration returning methods In-Reply-To: <3093DBE7-AC02-4371-B7EC-12C3A15EA334@oracle.com> References: <3093DBE7-AC02-4371-B7EC-12C3A15EA334@oracle.com> Message-ID: <06A9E918-9FBF-41D9-B594-E94ED50A2089@oracle.com> Hi Paul, All the changes seem reasonable. A couple minor suggestions - DriverManager.drivers() - I do not think we need to repeat the note from getDrivers(), otherwise, I would use {@code} vs in the new javadoc comment - DriverManagerTests.test19() - For the new test, if you could add a simple comment about the test, it would keep it consistent with the other tests, otherwise I can do it later Best Lance On Jun 2, 2015, at 9:37 AM, Paul Sandoz wrote: > Hi, > > Please review a follow up to Stuart's Enumeration.asIterator patch that adds some Stream return methods to classes where there is only Enumeration returning methods to support traversal: > > http://cr.openjdk.java.net/~psandoz/jdk9/JDK-8081678-enumeration-and-stream/webrev/ > > I took the opportunity to sprinkle some Enumerator.asIterator impls. > > I originally included some methods on ClassLoader but Alan wisely warned me away from doing that as this area is likely to be impacted by modularity. > > There is one small area of uncertainty with NetworkInterface. Can the following method ever return null? > > 342 public static Enumeration getNetworkInterfaces() > 343 throws SocketException { > 344 NetworkInterface[] netifs = getAll(); > 345 > 346 // specified to return null if no network interfaces > 347 return netifs != null > 348 ? enumerationFromArray(netifs) > 349 : null; > 350 } > > Contrary to the comment i cannot find any specification. For the stream returning method, networkInterfaces, i have specified this to return an empty stream, thus it might be good to update the enumeration returning method as well to say whether it returns null or an empty enumeration. > > Paul. Lance Andersen| Principal Member of Technical Staff | +1.781.442.2037 Oracle Java Engineering 1 Network Drive Burlington, MA 01803 Lance.Andersen at oracle.com From Roger.Riggs at Oracle.com Tue Jun 2 22:22:27 2015 From: Roger.Riggs at Oracle.com (Roger Riggs) Date: Tue, 02 Jun 2015 18:22:27 -0400 Subject: RFR 9: 8081567 : java/lang/ProcessHandle/InfoTest.java failed Cannot run program "whoami" In-Reply-To: <556E21FD.10203@oracle.com> References: <556CD0E1.2080102@Oracle.com> <556E21FD.10203@oracle.com> Message-ID: <556E2CA3.6050002@Oracle.com> Hi Ivan, Thanks for the review. Updated the webrev in place with 2 corrections. http://cr.openjdk.java.net/~rriggs/webrev-whoami-8081567 On 6/2/2015 5:37 PM, Ivan Gerasimov wrote: > Hi Roger! > > On 02.06.2015 0:38, Roger Riggs wrote: >> Please review a change to report the user/owner in ProcessHandle.info to >> have the same form for the owner identification as provided by >> java.nio.Files. >> On Windows it includes the domain with the user name. >> >> The test is updated to remove a dependency on the OS command whoami >> and instead compare the users process with the owner of a file created >> by the process improving portability. >> >> Webrev: >> http://cr.openjdk.java.net/~rriggs/webrev-whoami-8081567/ >> > > 1) > > + if (domainLen > 0) { > + wcscat(domain, L"\\"); > + } else { > + domain[0] = L'\0'; > + } > + wcscat(domain, name); > > Here, domainLen is either equal to whatever LookupAccountSid has set > it to. > > In MSDN [1] it's written: > On input, specifies the size, in TCHARs, of the lpReferencedDomainName > buffer. If the function fails because the buffer is too small or if > cchReferencedDomainName is zero, cchReferencedDomainName receives the > required buffer size, including the terminating null character. > > I see in the comments that in fact on success this parameter receives > the length of the resulting string. > But should we rely on that undocumented behavior? Returning the actual size is (one) convention of a number of Win APIs. > Does it really happen that LookupAccountSidW() returns 0, setting > domainLen to 0 and leaving the domain buffer uninitialized? > From the published example [2] it seems that this shouldn't happen. Only successful calls reach the code in question. If domainLen is returned as zero, the domain[0] is zero terminated; not counting on the API to do it. Otherwise, it returned (some) string and it is null terminated. Am I misunderstanding the case? > > 2) > I think domainLen should be set to > DWORD domainLen = sizeof(domain) - sizeof(name) - 1; > Otherwise LookupAccountSid is allowed to fill up the whole buffer, so > that 'name' could not be appended. Agree, but I think it should be -2; one for the delimiter and for the trailing \0. I could not find a maximum length of a domain or name and picked 255. > > 3) > Indentation isn't consistent: it should be multiple of 4. Thanks, helpful but not correct emacs c-mode. Roger > > Sincerely yours, > Ivan > > [1] > https://msdn.microsoft.com/en-us/library/windows/desktop/aa379166%28v=vs.85%29.aspx > [2] > https://msdn.microsoft.com/en-us/library/windows/desktop/aa379554(v=vs.85).aspx > >> Issue: >> https://bugs.openjdk.java.net/browse/JDK-8081567 >> >> Thanks, Roger >> >> > From stuart.marks at oracle.com Wed Jun 3 02:05:39 2015 From: stuart.marks at oracle.com (Stuart Marks) Date: Tue, 02 Jun 2015 19:05:39 -0700 Subject: RFR 8071597 Add Stream dropWhile and takeWhile operations In-Reply-To: <79392DA7-6A8E-42AC-9E83-AA7F89D8F2A5@oracle.com> References: <79392DA7-6A8E-42AC-9E83-AA7F89D8F2A5@oracle.com> Message-ID: <556E60F3.7060800@oracle.com> Hi Paul, Some comments on the spec. On 6/2/15 6:13 AM, Paul Sandoz wrote: > http://cr.openjdk.java.net/~psandoz/jdk9/JDK-8071597-take-drop-while/webrev/ > > I opted to weight the documentation of the operations towards ordered streams in the first paragraph. That is what makes most sense in terms of usage and what most people will read. Thus i refer to the "longest prefix" in the first paragraph then define what that means in subsequent paragraphs for ordered and unordered streams: I see what you're trying to do here. The first sentence of a doc comment is special and so it needs to describe concisely but very generally what the method is trying to do. The problem is that the first sentence is flatly contradicted by the third paragraph. If the stream is unordered "longest prefix" is completely ill-defined. "Prefix" is nonsensical for an unordered stream, but the third paragraph attempts to redefine it to fit. "Longest" doesn't apply, since any subset may be returned. Maybe the thing to do just give up on the idea of the first sentence trying to cover the semantics for both ordered and unordered streams. The problem is that something like "takeWhile" is really mostly applicable to ordered streams, so in the first paragraph just give people what they're expecting to see. Then have the second paragraph cover the case of unordered streams. Maybe something like this: ---------- If this stream is ordered, returns a stream consisting of the longest prefix of elements taken from this stream, where the elements all match the given predicate. The prefix is a contiguous sequence of elements of this stream. The first element of the sequence is the first element (if any) of this stream, and the element (if any) immediately following the last element of the sequence does not match the given predicate. If this stream is unordered, this method returns a subset of elements of this stream. All elements (if any) of the subset match the given predicate. In this case the behavior of this operation is nondeterministic; it is free to select any valid subset of elements that match the predicate. ---------- s'marks > 482 /** > 483 * Returns a stream consisting of the longest prefix of elements taken from > 484 * this stream that match the given predicate. > 485 * > 486 *

    If this stream is ordered then the prefix is a contiguous sequence of > 487 * elements of this stream. All elements of the sequence match the given > 488 * predicate, the first element of the sequence is the first element > 489 * (if any) of this stream, and the element (if any) immediately following > 490 * the last element of the sequence does not match the given predicate. > 491 * > 492 *

    If this stream is unordered then the prefix is a subset of elements of > 493 * this stream. All elements (if any) of the subset match the given > 494 * predicate. In this case the behavior of this operation is > 495 * nondeterministic; it is free to select any valid subset as the prefix. > 496 * > 497 *

    This is a short-circuiting > 498 * stateful intermediate operation. > 499 * > ... > 528 default Stream takeWhile(Predicate predicate) { > > 537 /** > 538 * Returns a stream consisting of the remaining elements of this stream > 539 * after dropping the longest prefix of elements that match the given > 540 * predicate. > 541 * > 542 *

    If this stream is ordered then the prefix is a contiguous sequence of > 543 * elements of this stream. All elements of the sequence match the given > 544 * predicate, the first element of the sequence is the first element > 545 * (if any) of this stream, and the element (if any) immediately following > 546 * the last element of the sequence does not match the given predicate. > 547 * > 548 *

    If this stream is unordered then the prefix is a subset of elements of > 549 * this stream. All elements (if any) of the subset match the given > 550 * predicate. In this case the behavior of this operation is > 551 * nondeterministic; it is free to select any valid subset as the prefix. > 552 * > ... > 584 default Stream dropWhile(Predicate predicate) { > > > After this has been reviewed i will follow up with a further issue regarding the specification of takeWhile, stateful predicates and cancellation. I avoided such specification here as it's likely to rathole :-) > > Basically the takeWhile operation is implemented such that one can do: > > long t = System.currentTimeMillis(); > List pps = Stream > .generate(() -> BigInteger.probablePrime(1024, ThreadLocalRandom.current())) > .parallel() > .takeWhile(e -> (System.currentTimeMillis() - t) < TimeUnit.SECONDS.toMillis(5)) > .collect(toList()); > > Paul. > From chris.plummer at oracle.com Wed Jun 3 03:20:03 2015 From: chris.plummer at oracle.com (Chris Plummer) Date: Tue, 02 Jun 2015 20:20:03 -0700 Subject: [9] RFR (XS) JDK-8081771: ProcessTool.createJavaProcessBuilder() needs new addTestVmAndJavaOptions argument Message-ID: <556E7263.4050601@oracle.com> Please review the following: Webrev: http://cr.openjdk.java.net/~cjplummer/8054386/webrev.02/ Bug: https://bugs.openjdk.java.net/browse/JDK-8081771 This review only concerns the changes to ProcessTool.java. The CDSJDITests and filemapp.cpp changes will be committed under CR JDK-8054386, but they rely on this change to ProcessTool.java, so both CRs will be pushed together. See the following thread for the JDK-8054386 review: http://mail.openjdk.java.net/pipermail/hotspot-runtime-dev/2015-June/014923.html thanks, Chris From amy.lu at oracle.com Wed Jun 3 03:31:50 2015 From: amy.lu at oracle.com (Amy Lu) Date: Wed, 03 Jun 2015 11:31:50 +0800 Subject: JDK 9 RFR of JDK-8081775: two lib/testlibrary tests are failing with "Error. failed to clean up files after test" with jtreg 4.1 b12 Message-ID: <556E7526.7060301@oracle.com> lib/testlibrary/OutputAnalyzerReportingTest.java lib/testlibrary/OutputAnalyzerTest.java These tests fail with jtreg4.1/b12 because jtreg4.1/b12 adds stricter checking of @library tags and the library directory defined in the test doesn't exist. Please review the fix, removed the unneeded @library. bug: https://bugs.openjdk.java.net/browse/JDK-8081775 webrev: http://cr.openjdk.java.net/~amlu/8081775/webrev.00/ --- old/test/lib/testlibrary/OutputAnalyzerReportingTest.java 2015-06-03 10:48:57.000000000 +0800 +++ new/test/lib/testlibrary/OutputAnalyzerReportingTest.java 2015-06-03 10:48:57.000000000 +0800 @@ -28,7 +28,6 @@ * @summary Test the OutputAnalyzer reporting functionality, * such as printing additional diagnostic info * (exit code, stdout, stderr, command line, etc.) - * @library /testlibrary * @modules java.management * @build jdk.testlibrary.* * @run main jdk.testlibrary.OutputAnalyzerReportingTest --- old/test/lib/testlibrary/OutputAnalyzerTest.java 2015-06-03 10:48:58.000000000 +0800 +++ new/test/lib/testlibrary/OutputAnalyzerTest.java 2015-06-03 10:48:58.000000000 +0800 @@ -25,7 +25,6 @@ /* * @test * @summary Test the OutputAnalyzer utility class - * @library /testlibrary * @modules java.management * @build jdk.testlibrary.* * @run main jdk.testlibrary.OutputAnalyzerTest Thanks, Amy From peter.levart at gmail.com Wed Jun 3 06:06:40 2015 From: peter.levart at gmail.com (Peter Levart) Date: Wed, 03 Jun 2015 08:06:40 +0200 Subject: RFR(M, v11): JDK-8059036 : Implement Diagnostic Commands for heap and finalizerinfo In-Reply-To: <556D8FB0.4070608@oracle.com> References: <554894E9.8020908@oracle.com> <1AF219C7-1055-4D97-BB46-2FA390C4135D@oracle.com> <5548D65C.8040304@gmail.com> <554A8CEE.2090009@oracle.com> <55523418.5010708@oracle.com> <55527935.5010408@gmail.com> <5553B552.9060900@oracle.com> <555453BA.2070205@gmail.com> <55567748.6020304@oracle.com> <55567A68.9020802@oracle.com> <5556F34B.5050701@oracle.com> <55573A31.7050306@gmail.com> <55573C86.8030807@gmail.com> <5559D85B.2050500@oracle.com> <6FB8DF19-1D67-4702-9642-4C630765D2D1@oracle.com> <555C2EEB.1050505@oracle.com> <3941B430-FE6C-4AF0-A875-76D24411C654@oracle.com> <5564802A.2010403@oracle.com> <5565738A.4040109@gmail.com> <7EB0BE71-1608-4047-B71D-1A9EAA9083BB@oracle.com> <556727A2.9060209@gmail.com> <55675908.6030406@oracle.com> <556AF3CD.50900@oracle.com> <37227441-B016-4A9A-B9CA-BE100E992C9A@oracle.com> <556D8FB0.4070608@oracle.com> Message-ID: <556E9970.8010704@gmail.com> Hi Dmitry, On 06/02/2015 01:12 PM, Dmitry Samersoff wrote: > Staffan, > >> Instead of hardcoding the field offsets, you can use >> InstanceKlass::find_field and fieldDescriptor::offset to find the >> offset at runtime. > Done. Please, see > > http://cr.openjdk.java.net/~dsamersoff/JDK-8059036/webrev.11 In the jdk part, now that you have a FinalizerHistogramEntry class, you can simplify the code even further: private static final class FinalizerHistogramEntry { int instanceCount; final String className; int getInstanceCount() { return instanceCount; } FinalizerHistogramEntry(String className) { this.className = className; } } static Object[] getFinalizerHistogram() { Map countMap = new HashMap<>(); queue.forEach(r -> { Object referent = r.get(); if (referent != null) { countMap.computeIfAbsent( referent.getClass().getName(), FinalizerHistogramEntry::new).instanceCount++; /* Clear stack slot containing this variable, to decrease the chances of false retention with a conservative GC */ referent = null; } }); FinalizerHistogramEntry fhe[] = countMap.values() .toArray(new FinalizerHistogramEntry[countMap.size()]); Arrays.sort(fhe, Comparator.comparingInt( FinalizerHistogramEntry::getInstanceCount).reversed()); return fhe; } ... see, no copying loop required. Also notice the access modifier used on the nested class and it's fields/method/constructor. This way the class is private and fields can be accessed from getFinalizerHistogram and lambda without compiler generating access bridges. Regards, Peter > > I put the function int get_filed_offset_by_name(Symbol*,Symbol*) to > oop.inline.hpp leaving a room for further cleanup because I found couple > of places in hotspot that implements mostly similar functionality. > > -Dmitry > > On 2015-06-01 10:18, Staffan Larsen wrote: >> Dmitry, >> >> Instead of hardcoding the field offsets, you can use InstanceKlass::find_field and fieldDescriptor::offset to find the offset at runtime. >> >> Thanks, >> /Staffan >> >>> On 31 maj 2015, at 13:43, Dmitry Samersoff wrote: >>> >>> Everyone, >>> >>> Please take a look at new version of the fix. >>> >>> http://cr.openjdk.java.net/~dsamersoff/JDK-8059036/webrev.10/ >>> >>> Changes (to previous version) are in >>> Finalizer.java and diagnosticCommand.cpp >>> >>> This version copy data from Map.Entry<> to array of >>> FinalizerHistogramEntry instances then, >>> VM prints content of this array. >>> >>> -Dmitry >>> >>> >>> On 2015-05-28 21:06, Mandy Chung wrote: >>>> On 05/28/2015 07:35 AM, Peter Levart wrote: >>>>> Hi Mandy, >>>>> >>>>> On 05/27/2015 03:32 PM, Mandy Chung wrote: >>>>>> Taking it further - is it simpler to return String[] of all >>>>>> classnames including the duplicated ones and have the VM do the >>>>>> count? Are you concerned with the size of the String[]? >>>>> Yes, the histogram is much smaller than the list of all instances. >>>>> There can be millions of instances waiting in finalizer queue, but >>>>> only a few distinct classes. >>>> Do you happen to know what libraries are the major contributors to these >>>> millions of finalizers? >>>> >>>> It has been strongly recommended to avoid finalizers (see Effective Java >>>> Item 7). I'm not surprised that existing code is still using >>>> finalizers while we should really encourage them to migrate away from it. >>>> >>>>> What could be done in Java to simplify things in native code but still >>>>> not format the output is to convert the array of Map.Entry(s) into an >>>>> Object[] array of alternating {String, int[], String, int[], .... } >>>>> >>>>> Would this simplify native code for the price of a little extra work >>>>> in Java? The sizes of old and new arrays are not big (# of distinct >>>>> classes of finalizable objects in the queue). >>>> I also prefer writing in Java and writing less native code (much >>>> simplified). It's an interface that we have to agree upon and keep it >>>> simple. Having the returned Object[] as alternate String and int[] is a >>>> reasonable compromise. >>>> >>>> ReferenceQueue.java - you can move @SuppressWarning from the method to >>>> just the field variable "rn" >>>> @SuppressWarnings("unchecked") >>>> Reference rn = r.next; >>>> >>>> Finalizer.java >>>> It's better to rename printFinalizationQueue as it inspects the >>>> finalizer reference queue (maybe getFinalizerHistogram?). Can you move >>>> this method to the end of this class and add the comment saying that >>>> this is invoked by VM for jcmd -finalizerinfo support and @return to >>>> describe the returned content. I also think you can remove >>>> @SuppressWarnings for this method. >>>> >>>> Mandy >>> >>> -- >>> Dmitry Samersoff >>> Oracle Java development team, Saint Petersburg, Russia >>> * I would love to change the world, but they won't give me the sources. > From staffan.larsen at oracle.com Wed Jun 3 07:59:31 2015 From: staffan.larsen at oracle.com (Staffan Larsen) Date: Wed, 3 Jun 2015 09:59:31 +0200 Subject: JDK 9 RFR of JDK-8081775: two lib/testlibrary tests are failing with "Error. failed to clean up files after test" with jtreg 4.1 b12 In-Reply-To: <556E7526.7060301@oracle.com> References: <556E7526.7060301@oracle.com> Message-ID: <27EED981-1D6A-40F3-B798-16FAAF993ABC@oracle.com> Looks good! Thanks, /Staffan > On 3 jun 2015, at 05:31, Amy Lu wrote: > > lib/testlibrary/OutputAnalyzerReportingTest.java > lib/testlibrary/OutputAnalyzerTest.java > > These tests fail with jtreg4.1/b12 because jtreg4.1/b12 adds stricter checking of @library tags and the library directory defined in the test doesn't exist. > > Please review the fix, removed the unneeded @library. > > bug: https://bugs.openjdk.java.net/browse/JDK-8081775 > webrev: http://cr.openjdk.java.net/~amlu/8081775/webrev.00/ > > > --- old/test/lib/testlibrary/OutputAnalyzerReportingTest.java 2015-06-03 10:48:57.000000000 +0800 > +++ new/test/lib/testlibrary/OutputAnalyzerReportingTest.java 2015-06-03 10:48:57.000000000 +0800 > @@ -28,7 +28,6 @@ > * @summary Test the OutputAnalyzer reporting functionality, > * such as printing additional diagnostic info > * (exit code, stdout, stderr, command line, etc.) > - * @library /testlibrary > * @modules java.management > * @build jdk.testlibrary.* > * @run main jdk.testlibrary.OutputAnalyzerReportingTest > --- old/test/lib/testlibrary/OutputAnalyzerTest.java 2015-06-03 10:48:58.000000000 +0800 > +++ new/test/lib/testlibrary/OutputAnalyzerTest.java 2015-06-03 10:48:58.000000000 +0800 > @@ -25,7 +25,6 @@ > /* > * @test > * @summary Test the OutputAnalyzer utility class > - * @library /testlibrary > * @modules java.management > * @build jdk.testlibrary.* > * @run main jdk.testlibrary.OutputAnalyzerTest > > > Thanks, > Amy > From serguei.spitsyn at oracle.com Wed Jun 3 08:40:49 2015 From: serguei.spitsyn at oracle.com (serguei.spitsyn at oracle.com) Date: Wed, 03 Jun 2015 01:40:49 -0700 Subject: [9] RFR (M) 8054386: Allow Java debugging when CDS is enabled In-Reply-To: <556E73AF.3020606@oracle.com> References: <555B47EA.3000900@oracle.com> <556D5CFD.1020709@oracle.com> <556DFC33.4060600@oracle.com> <556E73AF.3020606@oracle.com> Message-ID: <556EBD91.6030204@oracle.com> Chris, It looks good in general. I'd suggest to rename the folder: || test/com/sun/jdi/CDSJDITests to: test/com/sun/jdi/cds There is no need to spell "JDI" as it is already a sub-folder of the com/sun/jdi and there is no need to spell "Tests" too as it is in the test repo. Also, all the folders are normally named in the lower case. Thanks, Serguei On 6/2/15 8:25 PM, Chris Plummer wrote: > Ok, I'm going to keep this as one webrev, but I did create JDK-8081771 > for the ProcessTool.java changes: > > https://bugs.openjdk.java.net/browse/JDK-8081771 > > I will commit ProcessTool.java under JDK-8081771, and the rest of the > changes under JDK-8054386. Both will then be pushed together. I also > started a new thread for the review of JDK-8081771: > > http://mail.openjdk.java.net/pipermail/hotspot-runtime-dev/2015-June/014930.html > http://mail.openjdk.java.net/pipermail/core-libs-dev/2015-June/033892.html > > thanks, > > Chris > > On 6/2/15 11:55 AM, Chris Plummer wrote: >> I'm going to have to separate out the ProcessTool.java changes into a >> separate changeset (and CR). In the meantime, feel free to review >> what I have below. The code won't be changing at all when I separate >> out the ProcessTool.java changes. >> >> thanks, >> >> Chris >> >> On 6/2/15 12:36 AM, Chris Plummer wrote: >>> [Adding core-libs-dev at openjdk.java.net since this update includes >>> changes to jdk/test library code] >>> >>> Please review the updated webrev: >>> >>> Webrev: http://cr.openjdk.java.net/~cjplummer/8054386/webrev.02/ >>> Bug: https://bugs.openjdk.java.net/browse/JDK-8054386 >>> >>> There were concerns about the new hotspot tests referencing jdk >>> tests. One concern was that if the jdk tests change, they could >>> break the hotspot tests, and this might initially go undetected. The >>> other concern is that if the jdk and hotspot tests are placed in >>> separate test bundles, then it would not be possible to run the >>> hotspot tests. >>> >>> Because of these concerns, I moved the tests that were in >>> hotspot/test/runtime/CDSJDITests to instead be in >>> jdk/test/com/sun/jdi/CDSJDITests. There was a slight renaming of the >>> tests in the process. Also, I had to update the jdk version of >>> ProcessTool.java to include the createJavaProcessBuilder() variant >>> that is in the hotspot version of ProcessTool.java. >>> >>> Lastly, in CDSJITTest.java I changed: >>> >>> OutputAnalyzer output = new OutputAnalyzer(pb.start()); >>> >>> to instead be: >>> >>> OutputAnalyzer output = ProcessTools.executeProcess(pb); >>> >>> I had to do this since the jdk version of the OutputAnalyzer >>> constructor is not public. The 1st version is what is commonly used >>> in hostspot tests, and the 2nd version is what is commonly used in >>> jdk tests. I decided to adopt the jdk way rather than make the >>> OutputAnalyzer constructors public, although this will probably >>> happen eventually when the two versions are unified. >>> >>> thanks, >>> >>> Chris >>> >>> >>> On 5/19/15 7:25 AM, Chris Plummer wrote: >>>> Hi, >>>> >>>> Please review the following changes for allowing java debugging >>>> when CDS is enabled. >>>> >>>> Webrev:http://cr.openjdk.java.net/~cjplummer/8054386/webrev.01/ >>>> Bug:https://bugs.openjdk.java.net/browse/JDK-8054386 >>>> >>>> The VM changes are simple. I removed the check that prevents >>>> debugging with CDS enabled, and added logic that will map the CDS >>>> archive RW when debugging is enabled. >>>> >>>> The tests are a bit more complex. There are a bunch of existing JDI >>>> tests for testing debugging support. Rather than start from scratch >>>> or clone them, I instead just wrote wrapper tests that put the >>>> relevant JDI test classes in the archive, and then invoke the JDI >>>> test. I did this for 3 of the JDI tests. If you feel there are >>>> others that would be good candidates, I'd be happy to add them. I'm >>>> looking for ones that would result in modification of the RO class >>>> metadata, such as setting a breakpoint (for which I already added >>>> two tests). >>>> >>>> Testing done: >>>> -Using JPRT to run the new jtreg tests on all platforms. >>>> -Using JPRT to run all jtreg runtime tests on linux x86 and x_64. >>>> -Regular JPRT "-testset hotspot" run >>>> -Putting the JCK JVMTI tests in the archive and then running them. >>>> -Putting the nsk jdb, jdwp, jvmti, and jdi tests in the archive and >>>> then running them. >>>> -Putting a simple test class in the archive and then setting a >>>> breakpoint on it using jdb >>>> >>>> Some of the above testing resulted in the discovery of bugs that >>>> still need to be addressed: JDK-8078644, JDK-8078730, and JDK-8079181. >>>> >>>> I also verified that without the change to map the archive RW, the >>>> above testing resulted in a SEGV, which is what you would expect >>>> (and actually want to see to prove that the testing is effective). >>>> >>>> thanks, >>>> >>>> Chris >>>> >>> >> > From dmitry.samersoff at oracle.com Wed Jun 3 08:48:11 2015 From: dmitry.samersoff at oracle.com (Dmitry Samersoff) Date: Wed, 03 Jun 2015 11:48:11 +0300 Subject: RFR(M, v12): JDK-8059036 : Implement Diagnostic Commands for heap and finalizerinfo In-Reply-To: <556E9970.8010704@gmail.com> References: <554894E9.8020908@oracle.com> <1AF219C7-1055-4D97-BB46-2FA390C4135D@oracle.com> <5548D65C.8040304@gmail.com> <554A8CEE.2090009@oracle.com> <55523418.5010708@oracle.com> <55527935.5010408@gmail.com> <5553B552.9060900@oracle.com> <555453BA.2070205@gmail.com> <55567748.6020304@oracle.com> <55567A68.9020802@oracle.com> <5556F34B.5050701@oracle.com> <55573A31.7050306@gmail.com> <55573C86.8030807@gmail.com> <5559D85B.2050500@oracle.com> <6FB8DF19-1D67-4702-9642-4C630765D2D1@oracle.com> <555C2EEB.1050505@oracle.com> <3941B430-FE6C-4AF0-A875-76D24411C654@oracle.com> <5564802A.2010403@oracle.com> <5565738A.4040109@gmail.com> <7EB0BE71-1608-4047-B71D-1A9EAA9083BB@oracle.com> <556727A2.9060209@gmail.com> <55675908.6030406@oracle.com> <556AF3CD.50900@oracle.com> <37227441-B016-4A9A-B9CA-BE100E992C9A@oracle.com> <556D8FB0.4070608@oracle.com> <556E9970.8010704@gmail.com> Message-ID: <556EBF4B.6070400@oracle.com> Everyone, Updated webrev: http://cr.openjdk.java.net/~dsamersoff/JDK-8059036/webrev.12 getFinalizerHistogram and FinalizerHistogramEntry moved to separate package-private class. Hotspot code changed accordingly. getFinalizerHistogram updated to use Peter's code. -Dmitry On 2015-06-03 09:06, Peter Levart wrote: > Hi Dmitry, > > On 06/02/2015 01:12 PM, Dmitry Samersoff wrote: >> Staffan, >> >>> Instead of hardcoding the field offsets, you can use >>> InstanceKlass::find_field and fieldDescriptor::offset to find the >>> offset at runtime. >> Done. Please, see >> >> http://cr.openjdk.java.net/~dsamersoff/JDK-8059036/webrev.11 > > In the jdk part, now that you have a FinalizerHistogramEntry class, you > can simplify the code even further: > > > private static final class FinalizerHistogramEntry { > int instanceCount; > final String className; > > int getInstanceCount() { > return instanceCount; > } > > FinalizerHistogramEntry(String className) { > this.className = className; > } > } > > static Object[] getFinalizerHistogram() { > Map countMap = new HashMap<>(); > queue.forEach(r -> { > Object referent = r.get(); > if (referent != null) { > countMap.computeIfAbsent( > referent.getClass().getName(), > FinalizerHistogramEntry::new).instanceCount++; > /* Clear stack slot containing this variable, to decrease > the chances of false retention with a conservative GC */ > referent = null; > } > }); > > FinalizerHistogramEntry fhe[] = countMap.values() > .toArray(new FinalizerHistogramEntry[countMap.size()]); > Arrays.sort(fhe, > Comparator.comparingInt( > > FinalizerHistogramEntry::getInstanceCount).reversed()); > return fhe; > } > > > ... see, no copying loop required. Also notice the access modifier used > on the nested class and it's fields/method/constructor. This way the > class is private and fields can be accessed from getFinalizerHistogram > and lambda without compiler generating access bridges. > > > Regards, Peter > >> >> I put the function int get_filed_offset_by_name(Symbol*,Symbol*) to >> oop.inline.hpp leaving a room for further cleanup because I found couple >> of places in hotspot that implements mostly similar functionality. >> >> -Dmitry >> >> On 2015-06-01 10:18, Staffan Larsen wrote: >>> Dmitry, >>> >>> Instead of hardcoding the field offsets, you can use InstanceKlass::find_field and fieldDescriptor::offset to find the offset at runtime. >>> >>> Thanks, >>> /Staffan >>> >>>> On 31 maj 2015, at 13:43, Dmitry Samersoff wrote: >>>> >>>> Everyone, >>>> >>>> Please take a look at new version of the fix. >>>> >>>> http://cr.openjdk.java.net/~dsamersoff/JDK-8059036/webrev.10/ >>>> >>>> Changes (to previous version) are in >>>> Finalizer.java and diagnosticCommand.cpp >>>> >>>> This version copy data from Map.Entry<> to array of >>>> FinalizerHistogramEntry instances then, >>>> VM prints content of this array. >>>> >>>> -Dmitry >>>> >>>> >>>> On 2015-05-28 21:06, Mandy Chung wrote: >>>>> On 05/28/2015 07:35 AM, Peter Levart wrote: >>>>>> Hi Mandy, >>>>>> >>>>>> On 05/27/2015 03:32 PM, Mandy Chung wrote: >>>>>>> Taking it further - is it simpler to return String[] of all >>>>>>> classnames including the duplicated ones and have the VM do the >>>>>>> count? Are you concerned with the size of the String[]? >>>>>> Yes, the histogram is much smaller than the list of all instances. >>>>>> There can be millions of instances waiting in finalizer queue, but >>>>>> only a few distinct classes. >>>>> Do you happen to know what libraries are the major contributors to these >>>>> millions of finalizers? >>>>> >>>>> It has been strongly recommended to avoid finalizers (see Effective Java >>>>> Item 7). I'm not surprised that existing code is still using >>>>> finalizers while we should really encourage them to migrate away from it. >>>>> >>>>>> What could be done in Java to simplify things in native code but still >>>>>> not format the output is to convert the array of Map.Entry(s) into an >>>>>> Object[] array of alternating {String, int[], String, int[], .... } >>>>>> >>>>>> Would this simplify native code for the price of a little extra work >>>>>> in Java? The sizes of old and new arrays are not big (# of distinct >>>>>> classes of finalizable objects in the queue). >>>>> I also prefer writing in Java and writing less native code (much >>>>> simplified). It's an interface that we have to agree upon and keep it >>>>> simple. Having the returned Object[] as alternate String and int[] is a >>>>> reasonable compromise. >>>>> >>>>> ReferenceQueue.java - you can move @SuppressWarning from the method to >>>>> just the field variable "rn" >>>>> @SuppressWarnings("unchecked") >>>>> Reference rn = r.next; >>>>> >>>>> Finalizer.java >>>>> It's better to rename printFinalizationQueue as it inspects the >>>>> finalizer reference queue (maybe getFinalizerHistogram?). Can you move >>>>> this method to the end of this class and add the comment saying that >>>>> this is invoked by VM for jcmd -finalizerinfo support and @return to >>>>> describe the returned content. I also think you can remove >>>>> @SuppressWarnings for this method. >>>>> >>>>> Mandy >>>> >>>> -- >>>> Dmitry Samersoff >>>> Oracle Java development team, Saint Petersburg, Russia >>>> * I would love to change the world, but they won't give me the sources. >> > -- Dmitry Samersoff Oracle Java development team, Saint Petersburg, Russia * I would love to change the world, but they won't give me the sources. From mikael.gerdin at oracle.com Wed Jun 3 09:14:21 2015 From: mikael.gerdin at oracle.com (Mikael Gerdin) Date: Wed, 03 Jun 2015 11:14:21 +0200 Subject: RFR(M, v11): JDK-8059036 : Implement Diagnostic Commands for heap and finalizerinfo In-Reply-To: <556E1A75.3070705@oracle.com> References: <554894E9.8020908@oracle.com> <1AF219C7-1055-4D97-BB46-2FA390C4135D@oracle.com> <5548D65C.8040304@gmail.com> <554A8CEE.2090009@oracle.com> <55523418.5010708@oracle.com> <55527935.5010408@gmail.com> <5553B552.9060900@oracle.com> <555453BA.2070205@gmail.com> <55567748.6020304@oracle.com> <55567A68.9020802@oracle.com> <5556F34B.5050701@oracle.com> <55573A31.7050306@gmail.com> <55573C86.8030807@gmail.com> <5559D85B.2050500@oracle.com> <6FB8DF19-1D67-4702-9642-4C630765D2D1@oracle.com> <555C2EEB.1050505@oracle.com> <3941B430-FE6C-4AF0-A875-76D24411C654@oracle.com> <5564802A.2010403@oracle.com> <5565738A.4040109@gmail.com> <7EB0BE71-1608-4047-B71D-1A9EAA9083BB@oracle.com> <556727A2.9060209@gmail.com> <55675908.6030406@oracle.com> <556AF3CD.50900@oracle.com> <37227441-B016-4A9A-B9CA-BE100E992C9A@oracle.com> <556D8FB0.4070608@oracle.com> <556D9C4C.9080200@oracle.com> <556E1A75.3070705@oracle.com> Message-ID: <556EC56D.3080703@oracle.com> On 2015-06-02 23:04, Dmitry Samersoff wrote: > Mikael, > > The reason of placing get_filed_offset_by_name to the oop.inline is that > hotspot widely duplicate this code. > > Symbol is used to identify a field within klass, not a klass them self > so I think it's OK to have it tied to the oopDesc. Ok, I didn't relize that both the symbols were used for the field, sorry about that. I still strongly feel that the method should be on InstanceKlass though. /Mikael > > -Dmitry > > On 2015-06-02 15:06, Mikael Gerdin wrote: >> Hi Dmitry, >> >> On 2015-06-02 13:12, Dmitry Samersoff wrote: >>> Staffan, >>> >>>> Instead of hardcoding the field offsets, you can use >>>> InstanceKlass::find_field and fieldDescriptor::offset to find the >>>> offset at runtime. >>> >>> Done. Please, see >>> >>> http://cr.openjdk.java.net/~dsamersoff/JDK-8059036/webrev.11 >>> >>> I put the function int get_filed_offset_by_name(Symbol*,Symbol*) to >>> oop.inline.hpp leaving a room for further cleanup because I found couple >>> of places in hotspot that implements mostly similar functionality. >> >> Sorry for this sort of drive-by review, but I really don't think >> get_field_offset_by_name should be in the oop class. If anywhere it >> should be on InstanceKlass, and using Symbol* to identify a Klass* seems >> incorrect since the same symbol can refer to different classes in >> different class loader contexts. >> >> /Mikael >> >>> >>> -Dmitry >>> >>> On 2015-06-01 10:18, Staffan Larsen wrote: >>>> Dmitry, >>>> >>>> Instead of hardcoding the field offsets, you can use >>>> InstanceKlass::find_field and fieldDescriptor::offset to find the >>>> offset at runtime. >>>> >>>> Thanks, >>>> /Staffan >>>> >>>>> On 31 maj 2015, at 13:43, Dmitry Samersoff >>>>> wrote: >>>>> >>>>> Everyone, >>>>> >>>>> Please take a look at new version of the fix. >>>>> >>>>> http://cr.openjdk.java.net/~dsamersoff/JDK-8059036/webrev.10/ >>>>> >>>>> Changes (to previous version) are in >>>>> Finalizer.java and diagnosticCommand.cpp >>>>> >>>>> This version copy data from Map.Entry<> to array of >>>>> FinalizerHistogramEntry instances then, >>>>> VM prints content of this array. >>>>> >>>>> -Dmitry >>>>> >>>>> >>>>> On 2015-05-28 21:06, Mandy Chung wrote: >>>>>> >>>>>> On 05/28/2015 07:35 AM, Peter Levart wrote: >>>>>>> Hi Mandy, >>>>>>> >>>>>>> On 05/27/2015 03:32 PM, Mandy Chung wrote: >>>>>>>> Taking it further - is it simpler to return String[] of all >>>>>>>> classnames including the duplicated ones and have the VM do the >>>>>>>> count? Are you concerned with the size of the String[]? >>>>>>> >>>>>>> Yes, the histogram is much smaller than the list of all instances. >>>>>>> There can be millions of instances waiting in finalizer queue, but >>>>>>> only a few distinct classes. >>>>>> >>>>>> Do you happen to know what libraries are the major contributors to >>>>>> these >>>>>> millions of finalizers? >>>>>> >>>>>> It has been strongly recommended to avoid finalizers (see Effective >>>>>> Java >>>>>> Item 7). I'm not surprised that existing code is still using >>>>>> finalizers while we should really encourage them to migrate away >>>>>> from it. >>>>>> >>>>>>> What could be done in Java to simplify things in native code but >>>>>>> still >>>>>>> not format the output is to convert the array of Map.Entry(s) into an >>>>>>> Object[] array of alternating {String, int[], String, int[], .... } >>>>>>> >>>>>>> Would this simplify native code for the price of a little extra work >>>>>>> in Java? The sizes of old and new arrays are not big (# of distinct >>>>>>> classes of finalizable objects in the queue). >>>>>> >>>>>> I also prefer writing in Java and writing less native code (much >>>>>> simplified). It's an interface that we have to agree upon and keep it >>>>>> simple. Having the returned Object[] as alternate String and int[] >>>>>> is a >>>>>> reasonable compromise. >>>>>> >>>>>> ReferenceQueue.java - you can move @SuppressWarning from the method to >>>>>> just the field variable "rn" >>>>>> @SuppressWarnings("unchecked") >>>>>> Reference rn = r.next; >>>>>> >>>>>> Finalizer.java >>>>>> It's better to rename printFinalizationQueue as it inspects the >>>>>> finalizer reference queue (maybe getFinalizerHistogram?). Can you >>>>>> move >>>>>> this method to the end of this class and add the comment saying that >>>>>> this is invoked by VM for jcmd -finalizerinfo support and @return to >>>>>> describe the returned content. I also think you can remove >>>>>> @SuppressWarnings for this method. >>>>>> >>>>>> Mandy >>>>> >>>>> >>>>> -- >>>>> Dmitry Samersoff >>>>> Oracle Java development team, Saint Petersburg, Russia >>>>> * I would love to change the world, but they won't give me the sources. >>>> >>> >>> > > From konstantin.shefov at oracle.com Wed Jun 3 09:44:57 2015 From: konstantin.shefov at oracle.com (Konstantin Shefov) Date: Wed, 03 Jun 2015 12:44:57 +0300 Subject: [8u-dev] Review request : JDK-8062904: TEST_BUG: Tests java/lang/invoke/LFCaching fail when run with -Xcomp option In-Reply-To: <556DFB1C.7010301@oracle.com> References: <5565D9BB.60003@oracle.com> <5565F6D5.7030900@oracle.com> <5566D9D8.4090401@oracle.com> <55675CCB.8010506@oracle.com> <5568281B.7040108@oracle.com> <55685264.20001@oracle.com> <556C86C6.2000409@oracle.com> <556DFB1C.7010301@oracle.com> Message-ID: <556ECC99.2000702@oracle.com> Hi Vladimir On 02.06.2015 21:51, Vladimir Ivanov wrote: > Konstantin, > >> It seems we have only this bug that manifests the problem. As I >> understand, this is a product issue, not test. > My question was about the symptoms - how the test can fail. If the > test ignores NSME & VME exceptions, will it always pass w.r.t. code > cache overflows? > > Code cache overflow is definitely a JVM problem, but we don't plan to > address it in the near future. > > So, either the test should be excluded or adjusted to be tolerant to > code cache overflows. The test has the code cache overflow failure only when run with "-Xcomp", all other failures has been fixed by https://bugs.openjdk.java.net/browse/JDK-8058733 So my suggestion is either to exclude this test when run with -Xcomp or (better) to reduce iteration number to 1 when -Xcomp, so that no code cache overflow in this case. -Konstantin > > Best regards, > Vladimir Ivanov > >> >> -Konstantin >> >> On 29.05.2015 14:49, Vladimir Ivanov wrote: >>>> What do you mean by ignore code cache overflow? Do you mean I >>>> should fix >>>> the test to ignore these errors or should I leave this test unfixed >>>> because it is a product related issue? >>> The former. How reliable the test is if it ignores >>> NoSuchMethodException & VirtualMachineError? Are there other >>> manifestations of the problem? >>> >>> Best regards, >>> Vladimir Ivanov >>> >>>> On 28.05.2015 21:22, Vladimir Ivanov wrote: >>>>> Got it, thanks. >>>>> >>>>> Can we ignore errors caused by code cache overflow for now? >>>>> >>>>> Best regards, >>>>> Vladimir Ivanov >>>>> >>>>> On 5/28/15 12:03 PM, Konstantin Shefov wrote: >>>>>> Vladimir, >>>>>> >>>>>> This fix is not for timeout issue, this fix is for >>>>>> "java.lang.VirtualMachineError: out of space in CodeCache for >>>>>> adapters". >>>>>> >>>>>> Timeout issue is other bug and should be filed separately. >>>>>> I do not know why SQE added RULES with timeout to this bug. >>>>>> >>>>>> By the way, if -Xcomp is set on JDK 8u, test works if not more than >>>>>> one >>>>>> iteration is allowed. The same thing was for JDK 9 until >>>>>> JDK-8046809 had >>>>>> been fixed. >>>>>> >>>>>> -Konstantin >>>>>> >>>>>> On 27.05.2015 19:54, Vladimir Ivanov wrote: >>>>>>> Have you tried to reduce iteration granularity? >>>>>>> >>>>>>> Probably, checking execution duration on every test case is more >>>>>>> robust. >>>>>>> >>>>>>> Best regards, >>>>>>> Vladimir Ivanov >>>>>>> >>>>>>> On 5/27/15 5:50 PM, Konstantin Shefov wrote: >>>>>>>> Hello, >>>>>>>> >>>>>>>> Please review the test bug fix >>>>>>>> https://bugs.openjdk.java.net/browse/JDK-8062904 >>>>>>>> Webrev is http://cr.openjdk.java.net/~kshefov/8062904/webrev.01/ >>>>>>>> Test fails only against JDK 8u and passes against JDK 9. >>>>>>>> >>>>>>>> Thanks >>>>>>>> >>>>>>>> -Konstantin >>>>>> >>>> >> From ivan.gerasimov at oracle.com Wed Jun 3 09:48:45 2015 From: ivan.gerasimov at oracle.com (Ivan Gerasimov) Date: Wed, 03 Jun 2015 12:48:45 +0300 Subject: RFR 9: 8081567 : java/lang/ProcessHandle/InfoTest.java failed Cannot run program "whoami" In-Reply-To: <556E2CA3.6050002@Oracle.com> References: <556CD0E1.2080102@Oracle.com> <556E21FD.10203@oracle.com> <556E2CA3.6050002@Oracle.com> Message-ID: <556ECD7D.7020703@oracle.com> On 03.06.2015 1:22, Roger Riggs wrote: > Hi Ivan, > > Thanks for the review. > > Updated the webrev in place with 2 corrections. > http://cr.openjdk.java.net/~rriggs/webrev-whoami-8081567 > > On 6/2/2015 5:37 PM, Ivan Gerasimov wrote: >> Hi Roger! >> >> On 02.06.2015 0:38, Roger Riggs wrote: >>> Please review a change to report the user/owner in >>> ProcessHandle.info to >>> have the same form for the owner identification as provided by >>> java.nio.Files. >>> On Windows it includes the domain with the user name. >>> >>> The test is updated to remove a dependency on the OS command whoami >>> and instead compare the users process with the owner of a file created >>> by the process improving portability. >>> >>> Webrev: >>> http://cr.openjdk.java.net/~rriggs/webrev-whoami-8081567/ >>> >> >> 1) >> >> + if (domainLen > 0) { >> + wcscat(domain, L"\\"); >> + } else { >> + domain[0] = L'\0'; >> + } >> + wcscat(domain, name); >> >> Here, domainLen is either equal to whatever LookupAccountSid has set >> it to. >> >> In MSDN [1] it's written: >> On input, specifies the size, in TCHARs, of the >> lpReferencedDomainName buffer. If the function fails because the >> buffer is too small or if cchReferencedDomainName is zero, >> cchReferencedDomainName receives the required buffer size, including >> the terminating null character. >> >> I see in the comments that in fact on success this parameter receives >> the length of the resulting string. >> But should we rely on that undocumented behavior? > Returning the actual size is (one) convention of a number of Win APIs. My concern is that I don't see this convention explicitly stated, at least in the documentation for LookupAccountSidW(). On the other hand, the doc does promise that the string will be null-terminated upon successful call. Thus, I think it's safer to just remove lines 423-427 and do concatenation right away, unless you have evidence that LookupAccountSidW may fail to produce null-terminated string, contradicting the documentation. In the later case, I think, it should be stated in the comments. >> Does it really happen that LookupAccountSidW() returns 0, setting >> domainLen to 0 and leaving the domain buffer uninitialized? >> From the published example [2] it seems that this shouldn't happen. > Only successful calls reach the code in question. > If domainLen is returned as zero, the domain[0] is zero terminated; > not counting on the API to do it. > Otherwise, it returned (some) string and it is null terminated. > Am I misunderstanding the case? >> >> 2) >> I think domainLen should be set to >> DWORD domainLen = sizeof(domain) - sizeof(name) - 1; >> Otherwise LookupAccountSid is allowed to fill up the whole buffer, so >> that 'name' could not be appended. > Agree, but I think it should be -2; one for the delimiter and for the > trailing \0. I was thinking of 256 (== sizeof(domain) - sizeof(name) - 1) for the domain + \0. Then the trailing \0 is replaced with the delimiter and the remaining 256 WCHARs can be used for the name + \0; > I could not find a maximum length of a domain or name and picked 255. The example from MSDN [2] uses 256 for the name + terminating null, so I think it should be fine. To be extra-safe we could check GetLastError() for ERROR_NONE_MAPPED and in such rare cases allocate the buffers dynamically. Sincerely yours, Ivan >> >> 3) >> Indentation isn't consistent: it should be multiple of 4. > Thanks, helpful but not correct emacs c-mode. > > Roger > >> >> Sincerely yours, >> Ivan >> >> [1] >> https://msdn.microsoft.com/en-us/library/windows/desktop/aa379166%28v=vs.85%29.aspx >> [2] >> https://msdn.microsoft.com/en-us/library/windows/desktop/aa379554(v=vs.85).aspx >> >>> Issue: >>> https://bugs.openjdk.java.net/browse/JDK-8081567 >>> >>> Thanks, Roger >>> >>> >> > > From amy.lu at oracle.com Wed Jun 3 10:11:13 2015 From: amy.lu at oracle.com (Amy Lu) Date: Wed, 03 Jun 2015 18:11:13 +0800 Subject: JDK 9 RFR of JDK-8081773: sun/net/www/protocol/https/ChunkedOutputStream.java references library that doesn't exist Message-ID: <556ED2C1.5050003@oracle.com> sun/net/www/protocol/https/ChunkedOutputStream.java This test fails with jtreg4.1/b12 because jtreg4.1/b12 adds stricter checking of @library tags and the library directory defined in the test doesn't exist. Please help to review and sponsor this fix, removed the unneeded @library. bug: https://bugs.openjdk.java.net/browse/JDK-8081773 webrev: http://cr.openjdk.java.net/~amlu/8081773/webrev.00/ --- old/test/sun/net/www/protocol/https/ChunkedOutputStream.java 2015-06-03 11:48:01.000000000 +0800 +++ new/test/sun/net/www/protocol/https/ChunkedOutputStream.java 2015-06-03 11:48:01.000000000 +0800 @@ -1,5 +1,5 @@ /* - * Copyright (c) 2004, 2012, Oracle and/or its affiliates. All rights reserved. + * Copyright (c) 2004, 2015, Oracle and/or its affiliates. All rights reserved. * DO NOT ALTER OR REMOVE COPYRIGHT NOTICES OR THIS FILE HEADER. * * This code is free software; you can redistribute it and/or modify it @@ -25,7 +25,6 @@ * @test * @bug 5026745 * @modules java.base/sun.net.www - * @library ../../httpstest/ * @build TestHttpsServer HttpCallback * @run main/othervm ChunkedOutputStream * Thanks, Amy From chris.hegarty at oracle.com Wed Jun 3 10:24:08 2015 From: chris.hegarty at oracle.com (Chris Hegarty) Date: Wed, 03 Jun 2015 11:24:08 +0100 Subject: JDK 9 RFR of JDK-8081773: sun/net/www/protocol/https/ChunkedOutputStream.java references library that doesn't exist In-Reply-To: <556ED2C1.5050003@oracle.com> References: <556ED2C1.5050003@oracle.com> Message-ID: <556ED5C8.8020600@oracle.com> Reviewed. I can push this for you. -Chris. On 03/06/15 11:11, Amy Lu wrote: > sun/net/www/protocol/https/ChunkedOutputStream.java > > This test fails with jtreg4.1/b12 because jtreg4.1/b12 adds stricter > checking of @library tags and the library directory defined in the test > doesn't exist. > > Please help to review and sponsor this fix, removed the unneeded @library. > > bug: https://bugs.openjdk.java.net/browse/JDK-8081773 > webrev: http://cr.openjdk.java.net/~amlu/8081773/webrev.00/ > > --- old/test/sun/net/www/protocol/https/ChunkedOutputStream.java > 2015-06-03 11:48:01.000000000 +0800 > +++ new/test/sun/net/www/protocol/https/ChunkedOutputStream.java > 2015-06-03 11:48:01.000000000 +0800 > @@ -1,5 +1,5 @@ > /* > - * Copyright (c) 2004, 2012, Oracle and/or its affiliates. All rights > reserved. > + * Copyright (c) 2004, 2015, Oracle and/or its affiliates. All rights > reserved. > * DO NOT ALTER OR REMOVE COPYRIGHT NOTICES OR THIS FILE HEADER. > * > * This code is free software; you can redistribute it and/or modify it > @@ -25,7 +25,6 @@ > * @test > * @bug 5026745 > * @modules java.base/sun.net.www > - * @library ../../httpstest/ > * @build TestHttpsServer HttpCallback > * @run main/othervm ChunkedOutputStream > * > > Thanks, > Amy From Alan.Bateman at oracle.com Wed Jun 3 10:46:40 2015 From: Alan.Bateman at oracle.com (Alan Bateman) Date: Wed, 03 Jun 2015 11:46:40 +0100 Subject: RFR 8081678: Add Stream returning methods to classes where there currently exist only Enumeration returning methods In-Reply-To: <3093DBE7-AC02-4371-B7EC-12C3A15EA334@oracle.com> References: <3093DBE7-AC02-4371-B7EC-12C3A15EA334@oracle.com> Message-ID: <556EDB10.9060708@oracle.com> On 02/06/2015 14:37, Paul Sandoz wrote: > : > > There is one small area of uncertainty with NetworkInterface. Can the following method ever return null? > > 342 public static Enumeration getNetworkInterfaces() > 343 throws SocketException { > 344 NetworkInterface[] netifs = getAll(); > 345 > 346 // specified to return null if no network interfaces > 347 return netifs != null > 348 ? enumerationFromArray(netifs) > 349 : null; > 350 } > > Contrary to the comment i cannot find any specification. For the stream returning method, networkInterfaces, i have specified this to return an empty stream, thus it might be good to update the enumeration returning method as well to say whether it returns null or an empty enumeration. > cc'ing net-dev. AFAIK, getNetworkInterface was originally specified to return null but this was changed in Java SE 8 to always return with at least one element, the loopback interface. The underlying native getAll might still return null for exception cases (which will cause the SocketException to be thrown on return). So I think you can remove the null check. -Alan From ivan.gerasimov at oracle.com Wed Jun 3 10:58:43 2015 From: ivan.gerasimov at oracle.com (Ivan Gerasimov) Date: Wed, 03 Jun 2015 13:58:43 +0300 Subject: RFR 9: 8067808 : java/lang/ProcessBuilder/Basic.java failed on Assertion In-Reply-To: <556DF6E0.2010000@Oracle.com> References: <556DF6E0.2010000@Oracle.com> Message-ID: <556EDDE3.9070807@oracle.com> Hi Roger! Seems Okay to me. Do you think it would make sense to make the doc for Process.getPid() clearer that it guarantees to return the same pid even after calling waitFor()? This's true for current implementation, but from the OS point of view, pid does not necessarily uniquely identifies the process after it has terminated. Sincerely yours, Ivan On 02.06.2015 21:33, Roger Riggs wrote: > Please review test cleanup to make process id testing more reliable. > It now spawns javaChild to report the child pid instead of parsing > the output of OS specific commands. > > Webrev: > http://cr.openjdk.java.net/~rriggs/webrev-builder-8067808/ > > Issue: > http://cr.openjdk.java.net/~rriggs/webrev-builder-8067808/ > > Thanks, Roger > > p.s. I'm looking for a review also for: > 8081567 : java/lang/ProcessHandle/InfoTest.java failed Cannot run > program "whoami" > > > From amy.lu at oracle.com Wed Jun 3 13:10:21 2015 From: amy.lu at oracle.com (Amy Lu) Date: Wed, 03 Jun 2015 21:10:21 +0800 Subject: JDK 9 RFR of JDK-8081773: sun/net/www/protocol/https/ChunkedOutputStream.java references library that doesn't exist In-Reply-To: <556ED5C8.8020600@oracle.com> References: <556ED2C1.5050003@oracle.com> <556ED5C8.8020600@oracle.com> Message-ID: <556EFCBD.5060109@oracle.com> Thank you Chris ! /Amy On 6/3/15 6:24 PM, Chris Hegarty wrote: > Reviewed. I can push this for you. > > -Chris. > > On 03/06/15 11:11, Amy Lu wrote: >> sun/net/www/protocol/https/ChunkedOutputStream.java >> >> This test fails with jtreg4.1/b12 because jtreg4.1/b12 adds stricter >> checking of @library tags and the library directory defined in the test >> doesn't exist. >> >> Please help to review and sponsor this fix, removed the unneeded >> @library. >> >> bug: https://bugs.openjdk.java.net/browse/JDK-8081773 >> webrev: http://cr.openjdk.java.net/~amlu/8081773/webrev.00/ >> >> --- old/test/sun/net/www/protocol/https/ChunkedOutputStream.java >> 2015-06-03 11:48:01.000000000 +0800 >> +++ new/test/sun/net/www/protocol/https/ChunkedOutputStream.java >> 2015-06-03 11:48:01.000000000 +0800 >> @@ -1,5 +1,5 @@ >> /* >> - * Copyright (c) 2004, 2012, Oracle and/or its affiliates. All rights >> reserved. >> + * Copyright (c) 2004, 2015, Oracle and/or its affiliates. All rights >> reserved. >> * DO NOT ALTER OR REMOVE COPYRIGHT NOTICES OR THIS FILE HEADER. >> * >> * This code is free software; you can redistribute it and/or >> modify it >> @@ -25,7 +25,6 @@ >> * @test >> * @bug 5026745 >> * @modules java.base/sun.net.www >> - * @library ../../httpstest/ >> * @build TestHttpsServer HttpCallback >> * @run main/othervm ChunkedOutputStream >> * >> >> Thanks, >> Amy From peter.levart at gmail.com Wed Jun 3 13:45:26 2015 From: peter.levart at gmail.com (Peter Levart) Date: Wed, 03 Jun 2015 15:45:26 +0200 Subject: RFR: JDK-8079063: ZoneOffsetTransitionRule.of should throw IAE for non-zero nanoseconds In-Reply-To: <554A39AA.5000303@Oracle.com> References: <55474866.3020506@gmail.com> <554A288A.4090305@Oracle.com> <554A36BC.40004@gmail.com> <554A39AA.5000303@Oracle.com> Message-ID: <556F04F6.2090205@gmail.com> Hi Roger, Now that CCC has approved the change in spec., I have prepared the final fix: http://cr.openjdk.java.net/~plevart/jdk9-dev/ZoneOffsetTransitionRule.time/webrev.02/ I added asserts to package-private constructors as a means to catch possible future mistakes as constructors are sometimes called internally among the package classes directly. At least the constructor for ZoneOffsetTransition is. The ZoneOffsetTransitionRule constructor is not and could be made private as an alternative. What do you say? Regards, Peter On 05/06/2015 05:56 PM, Roger Riggs wrote: > Hi Peter, > > I'll run the CCC side, mostly it follows the similar rationale and > structure as the jira entry. > And the details are in your webrev. > > Thanks, Roger > > > On 5/6/2015 11:43 AM, Peter Levart wrote: >> Cool! >> >> Do we need any CCC approval as this *is* a spec change isn't it? >> >> I haven't done such a thing yet, so please give me some pointers. I >> also intend to add a jtreg test that verifies this new behavior. >> >> Regards, Peter >> >> On 05/06/2015 05:06 PM, Stephen Colebourne wrote: >>> I am also happy with this change. >>> >>> Stephen >>> >>> >>> On 6 May 2015 at 15:43, Roger Riggs wrote: >>>> Hi Peter, >>>> >>>> Thanks for the analysis and followup. >>>> >>>> Yes, I think thesimple check as you propose is the desired >>>> clarification. >>>> I agree that the change should not affect any existing code outside >>>> the JDK >>>> and does not raise a compatibility issue. >>>> >>>> Roger >>>> >>>> >>>> >>>> On 5/4/2015 6:22 AM, Peter Levart wrote: >>>>> Hi, >>>>> >>>>> Now that JDK-8074003 is in, I'd like to discuss how to tackle >>>>> JDK-8079063. >>>>> >>>>> Package-private ZoneOffsetTransition constructor that takes >>>>> LocalDateTime >>>>> transition is invoked from the following 4 places: >>>>> >>>>> 1 - the public static factory method: >>>>> >>>>> /** >>>>> * Obtains an instance defining a transition between two >>>>> offsets. >>>>> *

    >>>>> * Applications should normally obtain an instance from {@link >>>>> ZoneRules}. >>>>> * This factory is only intended for use when creating {@link >>>>> ZoneRules}. >>>>> * >>>>> * @param transition the transition date-time at the >>>>> transition, >>>>> which never >>>>> * actually occurs, expressed local to the before offset, >>>>> not null >>>>> * @param offsetBefore the offset before the transition, not >>>>> null >>>>> * @param offsetAfter the offset at and after the >>>>> transition, not >>>>> null >>>>> * @return the transition, not null >>>>> * @throws IllegalArgumentException if {@code offsetBefore} >>>>> and {@code >>>>> offsetAfter} >>>>> * are equal, or {@code transition.getNano()} returns >>>>> non-zero >>>>> value >>>>> */ >>>>> public static ZoneOffsetTransition of(LocalDateTime transition, >>>>> ZoneOffset offsetBefore, ZoneOffset offsetAfter) { >>>>> >>>>> ...this one already disallows transition parameters that have >>>>> transition.getNano() != 0. >>>>> >>>>> >>>>> 2 - Lines 498..500 of ZoneOffsetTransitionRule: >>>>> >>>>> LocalDateTime localDT = LocalDateTime.of(date, time); >>>>> LocalDateTime transition = >>>>> timeDefinition.createDateTime(localDT, >>>>> standardOffset, offsetBefore); >>>>> return new ZoneOffsetTransition(transition, offsetBefore, >>>>> offsetAfter); >>>>> >>>>> ...where 'time' is an instance field of ZoneOffsetTransitionRule. The >>>>> ZoneOffsetTransitionRule public static factory method has the >>>>> following >>>>> definition: >>>>> >>>>> /** >>>>> * Obtains an instance defining the yearly rule to create >>>>> transitions >>>>> between two offsets. >>>>> *

    >>>>> * Applications should normally obtain an instance from {@link >>>>> ZoneRules}. >>>>> * This factory is only intended for use when creating {@link >>>>> ZoneRules}. >>>>> * >>>>> * @param month the month of the month-day of the first day >>>>> of the >>>>> cutover week, not null >>>>> * @param dayOfMonthIndicator the day of the month-day of >>>>> the cutover >>>>> week, positive if the week is that >>>>> * day or later, negative if the week is that day or earlier, >>>>> counting from the last day of the month, >>>>> * from -28 to 31 excluding 0 >>>>> * @param dayOfWeek the required day-of-week, null if the >>>>> month-day >>>>> should not be changed >>>>> * @param time the cutover time in the 'before' offset, not >>>>> null >>>>> * @param timeEndOfDay whether the time is midnight at the >>>>> end of day >>>>> * @param timeDefnition how to interpret the cutover >>>>> * @param standardOffset the standard offset in force at the >>>>> cutover, >>>>> not null >>>>> * @param offsetBefore the offset before the cutover, not null >>>>> * @param offsetAfter the offset after the cutover, not null >>>>> * @return the rule, not null >>>>> * @throws IllegalArgumentException if the day of month >>>>> indicator is >>>>> invalid >>>>> * @throws IllegalArgumentException if the end of day flag is >>>>> true >>>>> when the time is not midnight >>>>> */ >>>>> public static ZoneOffsetTransitionRule of( >>>>> Month month, >>>>> int dayOfMonthIndicator, >>>>> DayOfWeek dayOfWeek, >>>>> LocalTime time, >>>>> boolean timeEndOfDay, >>>>> TimeDefinition timeDefnition, >>>>> ZoneOffset standardOffset, >>>>> ZoneOffset offsetBefore, >>>>> ZoneOffset offsetAfter) { >>>>> Objects.requireNonNull(month, "month"); >>>>> Objects.requireNonNull(time, "time"); >>>>> Objects.requireNonNull(timeDefnition, "timeDefnition"); >>>>> Objects.requireNonNull(standardOffset, "standardOffset"); >>>>> Objects.requireNonNull(offsetBefore, "offsetBefore"); >>>>> Objects.requireNonNull(offsetAfter, "offsetAfter"); >>>>> if (dayOfMonthIndicator < -28 || dayOfMonthIndicator > 31 || >>>>> dayOfMonthIndicator == 0) { >>>>> throw new IllegalArgumentException("Day of month >>>>> indicator >>>>> must be between -28 and 31 inclusive excluding zero"); >>>>> } >>>>> if (timeEndOfDay && time.equals(LocalTime.MIDNIGHT) == >>>>> false) { >>>>> throw new IllegalArgumentException("Time must be >>>>> midnight when >>>>> end of day flag is true"); >>>>> } >>>>> return new ZoneOffsetTransitionRule(month, >>>>> dayOfMonthIndicator, >>>>> dayOfWeek, time, timeEndOfDay, timeDefnition, standardOffset, >>>>> offsetBefore, >>>>> offsetAfter); >>>>> } >>>>> >>>>> ...which allows 'time' parameter (that becomes >>>>> ZoneOffsetTransitionRule's >>>>> 'time' field) with no restriction as to whether it can contain >>>>> nanos != 0 or >>>>> not. >>>>> >>>>> >>>>> 3, 4 - Lines 668..678 of ZoneRules private getOffsetInfo method: >>>>> >>>>> LocalDateTime dtBefore = savingsLocalTransitions[index]; >>>>> LocalDateTime dtAfter = savingsLocalTransitions[index >>>>> + 1]; >>>>> ZoneOffset offsetBefore = wallOffsets[index / 2]; >>>>> ZoneOffset offsetAfter = wallOffsets[index / 2 + 1]; >>>>> if (offsetAfter.getTotalSeconds() > >>>>> offsetBefore.getTotalSeconds()) { >>>>> // gap >>>>> return new ZoneOffsetTransition(dtBefore, >>>>> offsetBefore, >>>>> offsetAfter); >>>>> } else { >>>>> // overlap >>>>> return new ZoneOffsetTransition(dtAfter, >>>>> offsetBefore, >>>>> offsetAfter); >>>>> } >>>>> >>>>> ...where dtBefore/dtAfter "transition" parameters are taken from >>>>> savingsLocalTransitions[] array that is filled-in in ZoneRules >>>>> constructors >>>>> from passed-in ZoneOffsetTransition objects. So here no nanos != 0 >>>>> can sneak >>>>> in if ZoneOffsetTransition invariant holds. >>>>> >>>>> The only place where nanos can sneak-in therefore seems to be the >>>>> public >>>>> ZoneOffsetTransitionRule.of() factory method. The question is >>>>> whether the >>>>> spec. could be changed so that ZoneOffsetTransitionRule.of() >>>>> factory method >>>>> would add another @throws definition: >>>>> >>>>> * @throws IllegalArgumentException if {@code time.getNano()} >>>>> returns >>>>> non-zero value >>>>> >>>>> >>>>> This, I think, would be consistent with ZoneOffsetTransition.of() >>>>> factory >>>>> method and I believe early enough in the live of the API so that >>>>> no custom >>>>> software would be affected: >>>>> >>>>> >>>>> http://cr.openjdk.java.net/~plevart/jdk9-dev/ZoneOffsetTransitionRule.time/webrev.01/ >>>>> >>>>> >>>>> What do you think? >>>>> >>>>> Regards, Peter >>>>> >> > From Roger.Riggs at Oracle.com Wed Jun 3 14:37:39 2015 From: Roger.Riggs at Oracle.com (Roger Riggs) Date: Wed, 03 Jun 2015 10:37:39 -0400 Subject: [9] RFR (XS) JDK-8081771: ProcessTool.createJavaProcessBuilder() needs new addTestVmAndJavaOptions argument In-Reply-To: <556E7263.4050601@oracle.com> References: <556E7263.4050601@oracle.com> Message-ID: <556F1133.3000200@Oracle.com> Hi Chris, The jdk testlibrary changes are fine. Roger On 6/2/2015 11:20 PM, Chris Plummer wrote: > Please review the following: > > Webrev: http://cr.openjdk.java.net/~cjplummer/8054386/webrev.02/ > Bug: https://bugs.openjdk.java.net/browse/JDK-8081771 > > This review only concerns the changes to ProcessTool.java. The > CDSJDITests and filemapp.cpp changes will be committed under CR > JDK-8054386, but they rely on this change to ProcessTool.java, so both > CRs will be pushed together. See the following thread for the > JDK-8054386 review: > > http://mail.openjdk.java.net/pipermail/hotspot-runtime-dev/2015-June/014923.html > > > thanks, > > Chris From rob.mckenna at oracle.com Wed Jun 3 15:20:13 2015 From: rob.mckenna at oracle.com (Rob McKenna) Date: Wed, 03 Jun 2015 16:20:13 +0100 Subject: RFR: 7130985: Four helper classes missing in Sun JDK Message-ID: <556F1B2D.8010807@oracle.com> Meant to get this sorted a while back. There was a thread on this last year ( http://mail.openjdk.java.net/pipermail/core-libs-dev/2014-July/027779.html ) A test has been added since then: http://cr.openjdk.java.net/~robm/7130985/webrev.corba/ http://cr.openjdk.java.net/~robm/7130985/webrev.j2se/ -Rob From dmitry.samersoff at oracle.com Wed Jun 3 15:29:48 2015 From: dmitry.samersoff at oracle.com (Dmitry Samersoff) Date: Wed, 03 Jun 2015 18:29:48 +0300 Subject: RFR(M, v13): JDK-8059036 : Implement Diagnostic Commands for heap and finalizerinfo In-Reply-To: <556EBF4B.6070400@oracle.com> References: <554894E9.8020908@oracle.com> <1AF219C7-1055-4D97-BB46-2FA390C4135D@oracle.com> <5548D65C.8040304@gmail.com> <554A8CEE.2090009@oracle.com> <55523418.5010708@oracle.com> <55527935.5010408@gmail.com> <5553B552.9060900@oracle.com> <555453BA.2070205@gmail.com> <55567748.6020304@oracle.com> <55567A68.9020802@oracle.com> <5556F34B.5050701@oracle.com> <55573A31.7050306@gmail.com> <55573C86.8030807@gmail.com> <5559D85B.2050500@oracle.com> <6FB8DF19-1D67-4702-9642-4C630765D2D1@oracle.com> <555C2EEB.1050505@oracle.com> <3941B430-FE6C-4AF0-A875-76D24411C654@oracle.com> <5564802A.2010403@oracle.com> <5565738A.4040109@gmail.com> <7EB0BE71-1608-4047-B71D-1A9EAA9083BB@oracle.com> <556727A2.9060209@gmail.com> <55675908.6030406@oracle.com> <556AF3CD.50900@oracle.com> <37227441-B016-4A9A-B9CA-BE100E992C9A@oracle.com> <556D8FB0.4070608@oracle.com> <556E9970.8010704@gmail.com> <556EBF4B.6070400@oracle.com> Message-ID: <556F1D6C.6010806@oracle.com> Everyone, Updated webrev: http://cr.openjdk.java.net/~dsamersoff/JDK-8059036/webrev.13 Changes to oop.inline.hpp is reverted and find_field used directly is diagnostic command. -Dmitry On 2015-06-03 11:48, Dmitry Samersoff wrote: > Everyone, > > Updated webrev: > > http://cr.openjdk.java.net/~dsamersoff/JDK-8059036/webrev.12 > > getFinalizerHistogram and FinalizerHistogramEntry moved to separate > package-private class. Hotspot code changed accordingly. > > getFinalizerHistogram updated to use Peter's code. > > -Dmitry > > > On 2015-06-03 09:06, Peter Levart wrote: >> Hi Dmitry, >> >> On 06/02/2015 01:12 PM, Dmitry Samersoff wrote: >>> Staffan, >>> >>>> Instead of hardcoding the field offsets, you can use >>>> InstanceKlass::find_field and fieldDescriptor::offset to find the >>>> offset at runtime. >>> Done. Please, see >>> >>> http://cr.openjdk.java.net/~dsamersoff/JDK-8059036/webrev.11 >> >> In the jdk part, now that you have a FinalizerHistogramEntry class, you >> can simplify the code even further: >> >> >> private static final class FinalizerHistogramEntry { >> int instanceCount; >> final String className; >> >> int getInstanceCount() { >> return instanceCount; >> } >> >> FinalizerHistogramEntry(String className) { >> this.className = className; >> } >> } >> >> static Object[] getFinalizerHistogram() { >> Map countMap = new HashMap<>(); >> queue.forEach(r -> { >> Object referent = r.get(); >> if (referent != null) { >> countMap.computeIfAbsent( >> referent.getClass().getName(), >> FinalizerHistogramEntry::new).instanceCount++; >> /* Clear stack slot containing this variable, to decrease >> the chances of false retention with a conservative GC */ >> referent = null; >> } >> }); >> >> FinalizerHistogramEntry fhe[] = countMap.values() >> .toArray(new FinalizerHistogramEntry[countMap.size()]); >> Arrays.sort(fhe, >> Comparator.comparingInt( >> >> FinalizerHistogramEntry::getInstanceCount).reversed()); >> return fhe; >> } >> >> >> ... see, no copying loop required. Also notice the access modifier used >> on the nested class and it's fields/method/constructor. This way the >> class is private and fields can be accessed from getFinalizerHistogram >> and lambda without compiler generating access bridges. >> >> >> Regards, Peter >> >>> >>> I put the function int get_filed_offset_by_name(Symbol*,Symbol*) to >>> oop.inline.hpp leaving a room for further cleanup because I found couple >>> of places in hotspot that implements mostly similar functionality. >>> >>> -Dmitry >>> >>> On 2015-06-01 10:18, Staffan Larsen wrote: >>>> Dmitry, >>>> >>>> Instead of hardcoding the field offsets, you can use InstanceKlass::find_field and fieldDescriptor::offset to find the offset at runtime. >>>> >>>> Thanks, >>>> /Staffan >>>> >>>>> On 31 maj 2015, at 13:43, Dmitry Samersoff wrote: >>>>> >>>>> Everyone, >>>>> >>>>> Please take a look at new version of the fix. >>>>> >>>>> http://cr.openjdk.java.net/~dsamersoff/JDK-8059036/webrev.10/ >>>>> >>>>> Changes (to previous version) are in >>>>> Finalizer.java and diagnosticCommand.cpp >>>>> >>>>> This version copy data from Map.Entry<> to array of >>>>> FinalizerHistogramEntry instances then, >>>>> VM prints content of this array. >>>>> >>>>> -Dmitry >>>>> >>>>> >>>>> On 2015-05-28 21:06, Mandy Chung wrote: >>>>>> On 05/28/2015 07:35 AM, Peter Levart wrote: >>>>>>> Hi Mandy, >>>>>>> >>>>>>> On 05/27/2015 03:32 PM, Mandy Chung wrote: >>>>>>>> Taking it further - is it simpler to return String[] of all >>>>>>>> classnames including the duplicated ones and have the VM do the >>>>>>>> count? Are you concerned with the size of the String[]? >>>>>>> Yes, the histogram is much smaller than the list of all instances. >>>>>>> There can be millions of instances waiting in finalizer queue, but >>>>>>> only a few distinct classes. >>>>>> Do you happen to know what libraries are the major contributors to these >>>>>> millions of finalizers? >>>>>> >>>>>> It has been strongly recommended to avoid finalizers (see Effective Java >>>>>> Item 7). I'm not surprised that existing code is still using >>>>>> finalizers while we should really encourage them to migrate away from it. >>>>>> >>>>>>> What could be done in Java to simplify things in native code but still >>>>>>> not format the output is to convert the array of Map.Entry(s) into an >>>>>>> Object[] array of alternating {String, int[], String, int[], .... } >>>>>>> >>>>>>> Would this simplify native code for the price of a little extra work >>>>>>> in Java? The sizes of old and new arrays are not big (# of distinct >>>>>>> classes of finalizable objects in the queue). >>>>>> I also prefer writing in Java and writing less native code (much >>>>>> simplified). It's an interface that we have to agree upon and keep it >>>>>> simple. Having the returned Object[] as alternate String and int[] is a >>>>>> reasonable compromise. >>>>>> >>>>>> ReferenceQueue.java - you can move @SuppressWarning from the method to >>>>>> just the field variable "rn" >>>>>> @SuppressWarnings("unchecked") >>>>>> Reference rn = r.next; >>>>>> >>>>>> Finalizer.java >>>>>> It's better to rename printFinalizationQueue as it inspects the >>>>>> finalizer reference queue (maybe getFinalizerHistogram?). Can you move >>>>>> this method to the end of this class and add the comment saying that >>>>>> this is invoked by VM for jcmd -finalizerinfo support and @return to >>>>>> describe the returned content. I also think you can remove >>>>>> @SuppressWarnings for this method. >>>>>> >>>>>> Mandy >>>>> >>>>> -- >>>>> Dmitry Samersoff >>>>> Oracle Java development team, Saint Petersburg, Russia >>>>> * I would love to change the world, but they won't give me the sources. >>> >> > > -- Dmitry Samersoff Oracle Java development team, Saint Petersburg, Russia * I would love to change the world, but they won't give me the sources. From paul.sandoz at oracle.com Wed Jun 3 15:53:27 2015 From: paul.sandoz at oracle.com (Paul Sandoz) Date: Wed, 3 Jun 2015 17:53:27 +0200 Subject: RFR 8072773 (fs) Files.lines needs a better splitting implementation for stream source Message-ID: Hi, Please review an optimization for Files.lines for certain charsets: http://cr.openjdk.java.net/~psandoz/jdk9/JDK-8072773-File-lines/webrev/ If a charset is say US-ASCII or UTF-8 it is possible to implement an efficient splitting Spliterator that scans bytes from a mid-point to search for line feed characters. Splitting uses a mapped byte buffer. Traversal uses FileChannel.reads at an offset. In previous incarnations i tried to use mapped byte buffer for both, but for some reason the traversal performance was not good (both on Mac and x86). In any case i am happy with the current approach as there is minimal layering between the FileChannel and BufferedReader leveraged to read the lines. Sequential performance is similar (same or better) than the current approach. Parallel performance is much better than the current approach. Some advice on two aspects would be most appreciated: 1) Is there an easy way to determine the sub-set of supported charsets that are applicable? 2) We should try and explicitly unmap the mapped byte buffer when the stream is closed, using some sort of shared secret. How can i do that? Paul. From paul.sandoz at oracle.com Wed Jun 3 16:06:48 2015 From: paul.sandoz at oracle.com (Paul Sandoz) Date: Wed, 3 Jun 2015 18:06:48 +0200 Subject: RFR 8071597 Add Stream dropWhile and takeWhile operations In-Reply-To: <0EA606C1-FBBB-4466-BBB7-DF89C32E0CEC@oracle.com> References: <79392DA7-6A8E-42AC-9E83-AA7F89D8F2A5@oracle.com> <0EA606C1-FBBB-4466-BBB7-DF89C32E0CEC@oracle.com> Message-ID: <609F6A85-5BD1-4BF0-8215-74F973A02DE7@oracle.com> On Jun 2, 2015, at 8:58 PM, Chris Hegarty wrote: > Very nice. I just looked over the spec, for now. > > * @param predicate a non-interfering, > * stateless > * predicate to apply elements to determine the longest > * prefix of elements. > > Is this missing a *to*, ?? predicate to apply to elements to determine the ?" > Thanks, updated. Paul. From richard.warburton at gmail.com Wed Jun 3 16:14:54 2015 From: richard.warburton at gmail.com (Richard Warburton) Date: Wed, 3 Jun 2015 17:14:54 +0100 Subject: Lower overhead String encoding/decoding In-Reply-To: References: <54200F07.4070604@oracle.com> <542441BB.600@oracle.com> <5444BB17.8080708@oracle.com> <544FADA5.7000403@oracle.com> <544FB0F7.8080909@univ-mlv.fr> <54737BC3.9010508@oracle.com> <5473A167.5040703@oracle.com> <54A458DE.1010204@oracle.com> Message-ID: Hi gents, My apology for the delay, things are little slow during this period of the >> year:-) >> I will sponsor the rfe and prepare for the CCC (internally). We may need >> go through >> the api and the implementation one more time. >> > > I was just wondering if there was any news on the CCC for this patch? > Just tracking back on this front as I've not had a reply for over 3 months. It would be good to know the status of this patch. regards, Richard Warburton http://insightfullogic.com @RichardWarburto From Alan.Bateman at oracle.com Wed Jun 3 16:18:55 2015 From: Alan.Bateman at oracle.com (Alan Bateman) Date: Wed, 03 Jun 2015 17:18:55 +0100 Subject: RFR 8072773 (fs) Files.lines needs a better splitting implementation for stream source In-Reply-To: References: Message-ID: <556F28EF.1000201@oracle.com> On 03/06/2015 16:53, Paul Sandoz wrote: > Hi, > > Please review an optimization for Files.lines for certain charsets: > > http://cr.openjdk.java.net/~psandoz/jdk9/JDK-8072773-File-lines/webrev/ > > If a charset is say US-ASCII or UTF-8 it is possible to implement an efficient splitting Spliterator that scans bytes from a mid-point to search for line feed characters. > > Splitting uses a mapped byte buffer. Traversal uses FileChannel.reads at an offset. In previous incarnations i tried to use mapped byte buffer for both, but for some reason the traversal performance was not good (both on Mac and x86). In any case i am happy with the current approach as there is minimal layering between the FileChannel and BufferedReader leveraged to read the lines. > > Sequential performance is similar (same or better) than the current approach. Parallel performance is much better than the current approach. > > Some advice on two aspects would be most appreciated: > > 1) Is there an easy way to determine the sub-set of supported charsets that are applicable? > > 2) We should try and explicitly unmap the mapped byte buffer when the stream is closed, using some sort of shared secret. How can i do that? > As this code path is only for the default provider case then there's a good chance that it will be a FileChannelImpl, in which case you can call its unmap method (directly or via a shared secret). It is possible to interpose on the default provider so you can't be guaranteed it is a FileChannelImpl of course. In passing, you might consider moving ByteBufferLinesSpliterator to its own source file because Files is getting very big. -Alan. From paul.sandoz at oracle.com Wed Jun 3 16:21:21 2015 From: paul.sandoz at oracle.com (Paul Sandoz) Date: Wed, 3 Jun 2015 18:21:21 +0200 Subject: RFR 8071597 Add Stream dropWhile and takeWhile operations In-Reply-To: <556E60F3.7060800@oracle.com> References: <79392DA7-6A8E-42AC-9E83-AA7F89D8F2A5@oracle.com> <556E60F3.7060800@oracle.com> Message-ID: Hi Stuart, I had prepared an alternative rendition stashed away just in case this came up :-) I still want to retain a punchy short first paragraph. What do you think about the following? Stream skip(long n); /** - * Returns a stream consisting of the longest prefix of elements taken from - * this stream that match the given predicate. + * Returns a stream consisting of elements taken from this stream that match + * the given predicate. * - *

    If this stream is ordered then the prefix is a contiguous sequence of - * elements of this stream. All elements of the sequence match the given - * predicate, the first element of the sequence is the first element - * (if any) of this stream, and the element (if any) immediately following - * the last element of the sequence does not match the given predicate. + *

    If this stream is ordered then the longest prefix of elements is + * taken from this stream. The prefix is a contiguous sequence of elements + * of this stream. All elements of the sequence match the given predicate. + * The first element of the sequence is the first element (if any) of this + * stream, and the element (if any) immediately following the last element + * of the sequence does not match the given predicate. * - *

    If this stream is unordered then the prefix is a subset of elements of + *

    If this stream is unordered then a subset of elements is taken from * this stream. All elements (if any) of the subset match the given * predicate. In this case the behavior of this operation is - * nondeterministic; it is free to select any valid subset as the prefix. + * nondeterministic; it is free to take any valid subset. * *

    This is a short-circuiting * stateful intermediate operation. @@ -521,8 +522,8 @@ * * @param predicate a non-interfering, * stateless - * predicate to apply to elements to determine the longest - * prefix of elements. + * predicate to apply to elements to determine if they + * should be taken. * @return the new stream */ default Stream takeWhile(Predicate predicate) { @@ -535,20 +536,20 @@ } /** - * Returns a stream consisting of the remaining elements of this stream - * after dropping the longest prefix of elements that match the given - * predicate. + * Returns a stream consistent of the remaining elements of this stream + * after dropping elements that match the given predicate. * - *

    If this stream is ordered then the prefix is a contiguous sequence of - * elements of this stream. All elements of the sequence match the given - * predicate, the first element of the sequence is the first element - * (if any) of this stream, and the element (if any) immediately following - * the last element of the sequence does not match the given predicate. + *

    If this stream is ordered then the longest prefix of elements is + * dropped from this stream. The prefix is a contiguous sequence of + * elements of this stream. All elements of the sequence match the given + * predicate. The first element of the sequence is the first element (if + * any) of this stream, and the element (if any) immediately following the + * last element of the sequence does not match the given predicate. * - *

    If this stream is unordered then the prefix is a subset of elements of + *

    If this stream is unordered then a subset of elements is dropped from * this stream. All elements (if any) of the subset match the given * predicate. In this case the behavior of this operation is - * nondeterministic; it is free to select any valid subset as the prefix. + * nondeterministic; it is free to take any valid subset. * *

    This is a stateful * intermediate operation. @@ -577,8 +578,8 @@ * * @param predicate a non-interfering, * stateless - * predicate to apply to elements to determine the longest - * prefix of elements. + * predicate to apply to elements to determine if they + * should be dropped. * @return the new stream */ default Stream dropWhile(Predicate predicate) { Paul. On Jun 3, 2015, at 4:05 AM, Stuart Marks wrote: > Hi Paul, > > Some comments on the spec. > > On 6/2/15 6:13 AM, Paul Sandoz wrote: >> http://cr.openjdk.java.net/~psandoz/jdk9/JDK-8071597-take-drop-while/webrev/ >> >> I opted to weight the documentation of the operations towards ordered streams in the first paragraph. That is what makes most sense in terms of usage and what most people will read. Thus i refer to the "longest prefix" in the first paragraph then define what that means in subsequent paragraphs for ordered and unordered streams: > > I see what you're trying to do here. The first sentence of a doc comment is special and so it needs to describe concisely but very generally what the method is trying to do. > > The problem is that the first sentence is flatly contradicted by the third paragraph. If the stream is unordered "longest prefix" is completely ill-defined. "Prefix" is nonsensical for an unordered stream, but the third paragraph attempts to redefine it to fit. "Longest" doesn't apply, since any subset may be returned. > > Maybe the thing to do just give up on the idea of the first sentence trying to cover the semantics for both ordered and unordered streams. The problem is that something like "takeWhile" is really mostly applicable to ordered streams, so in the first paragraph just give people what they're expecting to see. Then have the second paragraph cover the case of unordered streams. > > Maybe something like this: > > ---------- > If this stream is ordered, returns a stream consisting of the longest prefix of elements taken from this stream, where the elements all match the given predicate. The prefix is a contiguous sequence of elements of this stream. The first element of the sequence is the first element (if any) of this stream, and the element (if any) immediately following the last element of the sequence does not match the given predicate. > > If this stream is unordered, this method returns a subset of elements of this stream. All elements (if any) of the subset match the given predicate. In this case the behavior of this operation is nondeterministic; it is free to select any valid subset of elements that match the predicate. > ---------- > > s'marks > From Alan.Bateman at oracle.com Wed Jun 3 16:27:20 2015 From: Alan.Bateman at oracle.com (Alan Bateman) Date: Wed, 03 Jun 2015 17:27:20 +0100 Subject: RFR [9] 8072839: JAX-B Plugability Layer: using java.util.ServiceLoader In-Reply-To: <556DA24C.9080708@oracle.com> References: <55560D2B.3040400@oracle.com> <555C8C7B.7060801@oracle.com> <5564F8A4.5050602@oracle.com> <556DA24C.9080708@oracle.com> Message-ID: <556F2AE8.9070009@oracle.com> On 02/06/2015 13:32, Miroslav Kos wrote: > Hi Alan, Daniel, > would you have some time to check the changes in this one? > The updated version looks okay to me. -Alan From konstantin.shefov at oracle.com Wed Jun 3 16:33:25 2015 From: konstantin.shefov at oracle.com (Konstantin Shefov) Date: Wed, 03 Jun 2015 19:33:25 +0300 Subject: [8u-dev] Review request : JDK-8068416: LFGarbageCollectedTest.java fails with OOME: "GC overhead limit exceeded" Message-ID: <556F2C55.5060304@oracle.com> Hello, Please review the test bug fix https://bugs.openjdk.java.net/browse/JDK-8068416 Webrev is http://cr.openjdk.java.net/~kshefov/8068416/webrev.00/ Test fails only against JDK 8u and passes against JDK 9. Fix is to reduce the number of iterations to 40. With that number of iterations the test passes on those hosts where it failed before. The number of iterations the test start to fail is 65. Before the fix the number of iterations was 84. Thanks -Konstantin From xueming.shen at oracle.com Wed Jun 3 16:34:52 2015 From: xueming.shen at oracle.com (Xueming Shen) Date: Wed, 03 Jun 2015 09:34:52 -0700 Subject: Lower overhead String encoding/decoding In-Reply-To: References: <54200F07.4070604@oracle.com> <542441BB.600@oracle.com> <5444BB17.8080708@oracle.com> <544FADA5.7000403@oracle.com> <544FB0F7.8080909@univ-mlv.fr> <54737BC3.9010508@oracle.com> <5473A167.5040703@oracle.com> <54A458DE.1010204@oracle.com> Message-ID: <556F2CAC.6000308@oracle.com> Richard, I'm still planning to be the sponsor for this RFE and get this one into the jdk9. We are current working on JEP 254 [1][2][3]in which the internal storage mechanism is changed from char[] to byte[]. If this JEP get approved and is scheduled to target JDK9, the decoding/encoding pieces will also need to be updated/ changed accordingly. Given that I would prefer to delay this RFE a little more till all pieces are settled. Regards, Sherman [1] http://openjdk.java.net/jeps/254 [2] https://bugs.openjdk.java.net/browse/JDK-8054307 [3] http://hg.openjdk.java.net/jdk9/sandbox/ On 6/3/15 9:14 AM, Richard Warburton wrote: > Hi gents, > > My apology for the delay, things are little slow during this > period of the year:-) > I will sponsor the rfe and prepare for the CCC (internally). > We may need go through > the api and the implementation one more time. > > > I was just wondering if there was any news on the CCC for this patch? > > > Just tracking back on this front as I've not had a reply for over 3 > months. It would be good to know the status of this patch. > > regards, > > Richard Warburton > > http://insightfullogic.com > @RichardWarburto https://bugs.openjdk.java.net/browse/JDK-8072392 From paul.sandoz at oracle.com Wed Jun 3 16:35:51 2015 From: paul.sandoz at oracle.com (Paul Sandoz) Date: Wed, 3 Jun 2015 18:35:51 +0200 Subject: RFR 8081678: Add Stream returning methods to classes where there currently exist only Enumeration returning methods In-Reply-To: <06A9E918-9FBF-41D9-B594-E94ED50A2089@oracle.com> References: <3093DBE7-AC02-4371-B7EC-12C3A15EA334@oracle.com> <06A9E918-9FBF-41D9-B594-E94ED50A2089@oracle.com> Message-ID: <586BD75D-F4D7-4CD6-BEA3-43BD21F43CBB@oracle.com> On Jun 3, 2015, at 12:06 AM, Lance Andersen wrote: > Hi Paul, > > All the changes seem reasonable. A couple minor suggestions > > - DriverManager.drivers() - I do not think we need to repeat the note from getDrivers(), otherwise, I would use {@code} vs in the new javadoc comment Removed. > - DriverManagerTests.test19() - For the new test, if you could add a simple comment about the test, it would keep it consistent with the other tests, otherwise I can do it later > Added. Webrev updated in place Thanks, Paul. > Best > Lance > > On Jun 2, 2015, at 9:37 AM, Paul Sandoz wrote: > >> Hi, >> >> Please review a follow up to Stuart's Enumeration.asIterator patch that adds some Stream return methods to classes where there is only Enumeration returning methods to support traversal: >> >> http://cr.openjdk.java.net/~psandoz/jdk9/JDK-8081678-enumeration-and-stream/webrev/ >> >> I took the opportunity to sprinkle some Enumerator.asIterator impls. >> >> I originally included some methods on ClassLoader but Alan wisely warned me away from doing that as this area is likely to be impacted by modularity. >> >> There is one small area of uncertainty with NetworkInterface. Can the following method ever return null? >> >> 342 public static Enumeration getNetworkInterfaces() >> 343 throws SocketException { >> 344 NetworkInterface[] netifs = getAll(); >> 345 >> 346 // specified to return null if no network interfaces >> 347 return netifs != null >> 348 ? enumerationFromArray(netifs) >> 349 : null; >> 350 } >> >> Contrary to the comment i cannot find any specification. For the stream returning method, networkInterfaces, i have specified this to return an empty stream, thus it might be good to update the enumeration returning method as well to say whether it returns null or an empty enumeration. >> >> Paul. > > > > Lance Andersen| Principal Member of Technical Staff | +1.781.442.2037 > Oracle Java Engineering > 1 Network Drive > Burlington, MA 01803 > Lance.Andersen at oracle.com > > > From paul.sandoz at oracle.com Wed Jun 3 16:47:17 2015 From: paul.sandoz at oracle.com (Paul Sandoz) Date: Wed, 3 Jun 2015 18:47:17 +0200 Subject: RFR 8081678: Add Stream returning methods to classes where there currently exist only Enumeration returning methods In-Reply-To: <556EDB10.9060708@oracle.com> References: <3093DBE7-AC02-4371-B7EC-12C3A15EA334@oracle.com> <556EDB10.9060708@oracle.com> Message-ID: <2E28B338-0289-4503-B713-B30E4E1704D0@oracle.com> On Jun 3, 2015, at 12:46 PM, Alan Bateman wrote: > > On 02/06/2015 14:37, Paul Sandoz wrote: > >> : >> >> There is one small area of uncertainty with NetworkInterface. Can the following method ever return null? >> >> 342 public static Enumeration getNetworkInterfaces() >> 343 throws SocketException { >> 344 NetworkInterface[] netifs = getAll(); >> 345 >> 346 // specified to return null if no network interfaces >> 347 return netifs != null >> 348 ? enumerationFromArray(netifs) >> 349 : null; >> 350 } >> >> Contrary to the comment i cannot find any specification. For the stream returning method, networkInterfaces, i have specified this to return an empty stream, thus it might be good to update the enumeration returning method as well to say whether it returns null or an empty enumeration. >> > cc'ing net-dev. AFAIK, getNetworkInterface was originally specified to return null but this was changed in Java SE 8 to always return with at least one element, the loopback interface. > > The underlying native getAll might still return null for exception cases (which will cause the SocketException to be thrown on return). So I think you can remove the null check. > Ok, i removed it but added an assert for the array being non-null and containing at least one element. I also refined the documentation of the stream returning method in light of this: http://cr.openjdk.java.net/~psandoz/jdk9/JDK-8081678-enumeration-and-stream/webrev/ Paul. From aleksey.shipilev at oracle.com Wed Jun 3 16:47:59 2015 From: aleksey.shipilev at oracle.com (Aleksey Shipilev) Date: Wed, 03 Jun 2015 19:47:59 +0300 Subject: Lower overhead String encoding/decoding In-Reply-To: <556F2CAC.6000308@oracle.com> References: <54200F07.4070604@oracle.com> <542441BB.600@oracle.com> <5444BB17.8080708@oracle.com> <544FADA5.7000403@oracle.com> <544FB0F7.8080909@univ-mlv.fr> <54737BC3.9010508@oracle.com> <5473A167.5040703@oracle.com> <54A458DE.1010204@oracle.com> <556F2CAC.6000308@oracle.com> Message-ID: <556F2FBF.6060103@oracle.com> On 06/03/2015 07:34 PM, Xueming Shen wrote: > I'm still planning to be the sponsor for this RFE and get this one > into the jdk9. We are current working on JEP 254 [1][2][3]in which > the internal storage mechanism is changed from char[] to byte[]. If > this JEP get approved and is scheduled to target JDK9, the > decoding/encoding pieces will also need to be updated/ changed > accordingly. Given that I would prefer to delay this RFE a little > more till all pieces are settled. Do we actually need to wait for Compact Strings to land? The failure scenario would be that Compact Strings are not making it to JDK 9, and this code lingers in history. I mean it's fine to see if there is a clash between the two, but I don't think Richard's change should be blocked by Compact Strings, given it is ready for integration (?), and Compact Strings are not yet. Thanks, -Aleksey From Roger.Riggs at Oracle.com Wed Jun 3 16:52:57 2015 From: Roger.Riggs at Oracle.com (Roger Riggs) Date: Wed, 03 Jun 2015 12:52:57 -0400 Subject: RFR: JDK-8079063: ZoneOffsetTransitionRule.of should throw IAE for non-zero nanoseconds In-Reply-To: <556F04F6.2090205@gmail.com> References: <55474866.3020506@gmail.com> <554A288A.4090305@Oracle.com> <554A36BC.40004@gmail.com> <554A39AA.5000303@Oracle.com> <556F04F6.2090205@gmail.com> Message-ID: <556F30E9.8080300@Oracle.com> Hi Peter, Looks good; thanks for the extra checks. Roger On 6/3/2015 9:45 AM, Peter Levart wrote: > Hi Roger, > > Now that CCC has approved the change in spec., I have prepared the > final fix: > > http://cr.openjdk.java.net/~plevart/jdk9-dev/ZoneOffsetTransitionRule.time/webrev.02/ > > > I added asserts to package-private constructors as a means to catch > possible future mistakes as constructors are sometimes called > internally among the package classes directly. At least the > constructor for ZoneOffsetTransition is. The ZoneOffsetTransitionRule > constructor is not and could be made private as an alternative. What > do you say? > > Regards, Peter > > On 05/06/2015 05:56 PM, Roger Riggs wrote: >> Hi Peter, >> >> I'll run the CCC side, mostly it follows the similar rationale and >> structure as the jira entry. >> And the details are in your webrev. >> >> Thanks, Roger >> >> >> On 5/6/2015 11:43 AM, Peter Levart wrote: >>> Cool! >>> >>> Do we need any CCC approval as this *is* a spec change isn't it? >>> >>> I haven't done such a thing yet, so please give me some pointers. I >>> also intend to add a jtreg test that verifies this new behavior. >>> >>> Regards, Peter >>> >>> On 05/06/2015 05:06 PM, Stephen Colebourne wrote: >>>> I am also happy with this change. >>>> >>>> Stephen >>>> >>>> >>>> On 6 May 2015 at 15:43, Roger Riggs wrote: >>>>> Hi Peter, >>>>> >>>>> Thanks for the analysis and followup. >>>>> >>>>> Yes, I think thesimple check as you propose is the desired >>>>> clarification. >>>>> I agree that the change should not affect any existing code >>>>> outside the JDK >>>>> and does not raise a compatibility issue. >>>>> >>>>> Roger >>>>> >>>>> >>>>> >>>>> On 5/4/2015 6:22 AM, Peter Levart wrote: >>>>>> Hi, >>>>>> >>>>>> Now that JDK-8074003 is in, I'd like to discuss how to tackle >>>>>> JDK-8079063. >>>>>> >>>>>> Package-private ZoneOffsetTransition constructor that takes >>>>>> LocalDateTime >>>>>> transition is invoked from the following 4 places: >>>>>> >>>>>> 1 - the public static factory method: >>>>>> >>>>>> /** >>>>>> * Obtains an instance defining a transition between two >>>>>> offsets. >>>>>> *

    >>>>>> * Applications should normally obtain an instance from {@link >>>>>> ZoneRules}. >>>>>> * This factory is only intended for use when creating {@link >>>>>> ZoneRules}. >>>>>> * >>>>>> * @param transition the transition date-time at the >>>>>> transition, >>>>>> which never >>>>>> * actually occurs, expressed local to the before offset, >>>>>> not null >>>>>> * @param offsetBefore the offset before the transition, >>>>>> not null >>>>>> * @param offsetAfter the offset at and after the >>>>>> transition, not >>>>>> null >>>>>> * @return the transition, not null >>>>>> * @throws IllegalArgumentException if {@code offsetBefore} >>>>>> and {@code >>>>>> offsetAfter} >>>>>> * are equal, or {@code transition.getNano()} >>>>>> returns non-zero >>>>>> value >>>>>> */ >>>>>> public static ZoneOffsetTransition of(LocalDateTime transition, >>>>>> ZoneOffset offsetBefore, ZoneOffset offsetAfter) { >>>>>> >>>>>> ...this one already disallows transition parameters that have >>>>>> transition.getNano() != 0. >>>>>> >>>>>> >>>>>> 2 - Lines 498..500 of ZoneOffsetTransitionRule: >>>>>> >>>>>> LocalDateTime localDT = LocalDateTime.of(date, time); >>>>>> LocalDateTime transition = >>>>>> timeDefinition.createDateTime(localDT, >>>>>> standardOffset, offsetBefore); >>>>>> return new ZoneOffsetTransition(transition, offsetBefore, >>>>>> offsetAfter); >>>>>> >>>>>> ...where 'time' is an instance field of ZoneOffsetTransitionRule. >>>>>> The >>>>>> ZoneOffsetTransitionRule public static factory method has the >>>>>> following >>>>>> definition: >>>>>> >>>>>> /** >>>>>> * Obtains an instance defining the yearly rule to create >>>>>> transitions >>>>>> between two offsets. >>>>>> *

    >>>>>> * Applications should normally obtain an instance from {@link >>>>>> ZoneRules}. >>>>>> * This factory is only intended for use when creating {@link >>>>>> ZoneRules}. >>>>>> * >>>>>> * @param month the month of the month-day of the first day >>>>>> of the >>>>>> cutover week, not null >>>>>> * @param dayOfMonthIndicator the day of the month-day of >>>>>> the cutover >>>>>> week, positive if the week is that >>>>>> * day or later, negative if the week is that day or earlier, >>>>>> counting from the last day of the month, >>>>>> * from -28 to 31 excluding 0 >>>>>> * @param dayOfWeek the required day-of-week, null if the >>>>>> month-day >>>>>> should not be changed >>>>>> * @param time the cutover time in the 'before' offset, not >>>>>> null >>>>>> * @param timeEndOfDay whether the time is midnight at the >>>>>> end of day >>>>>> * @param timeDefnition how to interpret the cutover >>>>>> * @param standardOffset the standard offset in force at >>>>>> the cutover, >>>>>> not null >>>>>> * @param offsetBefore the offset before the cutover, not null >>>>>> * @param offsetAfter the offset after the cutover, not null >>>>>> * @return the rule, not null >>>>>> * @throws IllegalArgumentException if the day of month >>>>>> indicator is >>>>>> invalid >>>>>> * @throws IllegalArgumentException if the end of day flag >>>>>> is true >>>>>> when the time is not midnight >>>>>> */ >>>>>> public static ZoneOffsetTransitionRule of( >>>>>> Month month, >>>>>> int dayOfMonthIndicator, >>>>>> DayOfWeek dayOfWeek, >>>>>> LocalTime time, >>>>>> boolean timeEndOfDay, >>>>>> TimeDefinition timeDefnition, >>>>>> ZoneOffset standardOffset, >>>>>> ZoneOffset offsetBefore, >>>>>> ZoneOffset offsetAfter) { >>>>>> Objects.requireNonNull(month, "month"); >>>>>> Objects.requireNonNull(time, "time"); >>>>>> Objects.requireNonNull(timeDefnition, "timeDefnition"); >>>>>> Objects.requireNonNull(standardOffset, "standardOffset"); >>>>>> Objects.requireNonNull(offsetBefore, "offsetBefore"); >>>>>> Objects.requireNonNull(offsetAfter, "offsetAfter"); >>>>>> if (dayOfMonthIndicator < -28 || dayOfMonthIndicator > >>>>>> 31 || >>>>>> dayOfMonthIndicator == 0) { >>>>>> throw new IllegalArgumentException("Day of month >>>>>> indicator >>>>>> must be between -28 and 31 inclusive excluding zero"); >>>>>> } >>>>>> if (timeEndOfDay && time.equals(LocalTime.MIDNIGHT) == >>>>>> false) { >>>>>> throw new IllegalArgumentException("Time must be >>>>>> midnight when >>>>>> end of day flag is true"); >>>>>> } >>>>>> return new ZoneOffsetTransitionRule(month, >>>>>> dayOfMonthIndicator, >>>>>> dayOfWeek, time, timeEndOfDay, timeDefnition, standardOffset, >>>>>> offsetBefore, >>>>>> offsetAfter); >>>>>> } >>>>>> >>>>>> ...which allows 'time' parameter (that becomes >>>>>> ZoneOffsetTransitionRule's >>>>>> 'time' field) with no restriction as to whether it can contain >>>>>> nanos != 0 or >>>>>> not. >>>>>> >>>>>> >>>>>> 3, 4 - Lines 668..678 of ZoneRules private getOffsetInfo method: >>>>>> >>>>>> LocalDateTime dtBefore = >>>>>> savingsLocalTransitions[index]; >>>>>> LocalDateTime dtAfter = >>>>>> savingsLocalTransitions[index + 1]; >>>>>> ZoneOffset offsetBefore = wallOffsets[index / 2]; >>>>>> ZoneOffset offsetAfter = wallOffsets[index / 2 + 1]; >>>>>> if (offsetAfter.getTotalSeconds() > >>>>>> offsetBefore.getTotalSeconds()) { >>>>>> // gap >>>>>> return new ZoneOffsetTransition(dtBefore, >>>>>> offsetBefore, >>>>>> offsetAfter); >>>>>> } else { >>>>>> // overlap >>>>>> return new ZoneOffsetTransition(dtAfter, >>>>>> offsetBefore, >>>>>> offsetAfter); >>>>>> } >>>>>> >>>>>> ...where dtBefore/dtAfter "transition" parameters are taken from >>>>>> savingsLocalTransitions[] array that is filled-in in ZoneRules >>>>>> constructors >>>>>> from passed-in ZoneOffsetTransition objects. So here no nanos != >>>>>> 0 can sneak >>>>>> in if ZoneOffsetTransition invariant holds. >>>>>> >>>>>> The only place where nanos can sneak-in therefore seems to be the >>>>>> public >>>>>> ZoneOffsetTransitionRule.of() factory method. The question is >>>>>> whether the >>>>>> spec. could be changed so that ZoneOffsetTransitionRule.of() >>>>>> factory method >>>>>> would add another @throws definition: >>>>>> >>>>>> * @throws IllegalArgumentException if {@code >>>>>> time.getNano()} returns >>>>>> non-zero value >>>>>> >>>>>> >>>>>> This, I think, would be consistent with ZoneOffsetTransition.of() >>>>>> factory >>>>>> method and I believe early enough in the live of the API so that >>>>>> no custom >>>>>> software would be affected: >>>>>> >>>>>> >>>>>> http://cr.openjdk.java.net/~plevart/jdk9-dev/ZoneOffsetTransitionRule.time/webrev.01/ >>>>>> >>>>>> >>>>>> What do you think? >>>>>> >>>>>> Regards, Peter >>>>>> >>> >> > From xueming.shen at oracle.com Wed Jun 3 17:45:35 2015 From: xueming.shen at oracle.com (Xueming Shen) Date: Wed, 03 Jun 2015 10:45:35 -0700 Subject: RFR 8080640: Reduce copying when reading JAR/ZIP entries In-Reply-To: <556DE778.1040300@oracle.com> References: <555A957F.5010102@oracle.com> <555CCAFB.4090805@oracle.com> <555E0C62.9090406@oracle.com> <555E1D49.9000707@oracle.com> <555F7862.9030307@oracle.com> <555F7AB8.1050202@oracle.com> <555F8E78.40404@oracle.com> <556DE778.1040300@oracle.com> Message-ID: <556F3D3F.3020906@oracle.com> Staffan, I'm not convinced that the benefit here is significant enough to change the getInputStream() to return a ByteArrayInputStream, given this can be easily achieved by wrapping the returned byte[] from getBytes(ZipEntry) at user's site. I would suggest to file a separate rfe on this disagreement and move on with the agreed getBytes() for now. Thanks, -Sherman On 06/02/2015 10:27 AM, Staffan Friberg wrote: > > On 05/22/2015 01:15 PM, Staffan Friberg wrote: >> On 05/22/2015 11:51 AM, Xueming Shen wrote: >>> On 05/22/2015 11:41 AM, Staffan Friberg wrote: >>>> >>>> On 05/21/2015 11:00 AM, Staffan Friberg wrote: >>>>> >>>>> On 05/21/2015 09:48 AM, Staffan Friberg wrote: >>>>>> >>>>>> On 05/20/2015 10:57 AM, Xueming Shen wrote: >>>>>>> On 05/18/2015 06:44 PM, Staffan Friberg wrote: >>>>>>>> Hi, >>>>>>>> >>>>>>>> Wanted to get reviews and feedback on this performance improvement for reading from JAR/ZIP files during classloading by reducing unnecessary copying and reading the entry in one go instead of in small portions. This shows a significant improvement when reading a single entry and for a large application with 10k classes and 500+ JAR files it improved the startup time by 4%. >>>>>>>> >>>>>>>> For more details on the background and performance results please see the RFE entry. >>>>>>>> >>>>>>>> RFE - https://bugs.openjdk.java.net/browse/JDK-8080640 >>>>>>>> WEBREV - http://cr.openjdk.java.net/~sfriberg/JDK-8080640/webrev.0 >>>>>>>> >>>>>>>> Cheers, >>>>>>>> Staffan >>>>>>> >>>>>>> Hi Staffan, >>>>>>> >>>>>>> If I did not miss something here, from your use scenario it appears to me the only thing you really >>>>>>> need here to help boost your performance is >>>>>>> >>>>>>> byte[] ZipFile.getAllBytes(ZipEntry ze); >>>>>>> >>>>>>> You are allocating a byte[] at use side and wrapping it with a ByteBuffer if the size is small enough, >>>>>>> otherwise, you letting the ZipFile to allocate a big enough one for you. It does not look like you >>>>>>> can re-use that byte[] (has to be wrapped by the ByteArrayInputStream and return), why do you >>>>>>> need two different methods here? The logic would be much easier to simply let the ZipFile to allocate >>>>>>> the needed buffer with appropriate size, fill the bytes and return, with a "OOME" if the entry size >>>>>>> is bigger than 2g. >>>>>>> >>>>>>> The only thing we use from the input ze is its name, get the size/csize from the jzentry, I don't think >>>>>>> jzentry.csize/size can be "unknown", they are from the "cen" table. >>>>>>> >>>>>>> If the real/final use of the bytes is to wrap it with a ByteArrayInputStream,why bother using ByteBuffer >>>>>>> here? Shouldn't a direct byte[] with exactly the size of the entry server better. >>>>>>> >>>>>>> -Sherman >>>>>>> >>>>>> Hi Sherman, >>>>>> >>>>>> Thanks for the comments. I agree, was starting out with bytebuffer because I was hoping to be able to cache things where the buffer was being used, but since the buffer is past along further I couldn't figure out a clean way to do it. >>>>>> Will rewrite it to simply just return a buffer, and only wrap it in the Resource class getByteBuffer. >>>>>> >>>>>> What would be your thought on updating the ZipFile.getInputStream to return ByteArrayInputStream for small entries? Currently I do that work outside in two places and moving it would potentially speed up others reading small entries as well. >>>>>> >>>>>> Thanks, >>>>>> Staffan >>>>> Just realized that my use of ByteArrayInputStream would miss Jar verification if enabled so the way to go hear would be to add it if possible to the ZipFile.getInputStream. >>>>> >>>>> //Staffan >>>> Hi, >>>> >>>> Here is an updated webrev which uses a byte[] directly and also uses ByteArrayInputStream in ZipFile for small entries below 128k. >>>> >>> >>> I'm not sure about the benefit of doing the ByteArrayInputStream in ZipFile.getInputStream. It has >>> the consequence of changing the "expected" behavior of getInputStream() (instead of return an >>> input stream waiting for reading, it now reads all bytes in advance), something we might not want >>> to do in a performance tuning. Though it might be reasonable to guess everyone get an input stream >>> is to read all bytes from it later. >>> >>> -Sherman >>> >>> >>>> http://cr.openjdk.java.net/~sfriberg/JDK-8080640/webrev.1 >>>> >>>> //Staffan >>> >> Agree that it will change the behavior slightly, but as you said it is probably expected that some one will read the stream eventually. >> We could reduce the size further if that makes a difference, if the size is below 65k we would not use more memory than the buffer allocated for the InflaterStream today. >> The total allocation would be slightly larger for deflated entries as we would allocate a byte[] for the compressed bytes, but it would be GC:able and not kept alive. So from a memory perspective the difference is very limited. >> >> //Staffan > Hi, > > Bumping this thread to get some more comments on the concern about changing the ZipFile.getInputStream behavior. The benefit of doing this change is that any read of small entries from ZIP and JAR files will be much faster and less resources will be held, including native resources normally held by the ZipInputStream. > > The behavior change that will occur is that the full entry will be read as part of creating the stream and not lazily as might be expected. However when getting a today InputStream zip file will be accessed to read information about the size of the entry, so the zip file is already touched when getting an InputStream, but not the compressed bytes. > > I'm fine with removing this part of the change and just push the private getBytes function and the updates to the JDK libraries to use it. > > Thanks, > Staffan From Roger.Riggs at Oracle.com Wed Jun 3 18:32:47 2015 From: Roger.Riggs at Oracle.com (Roger Riggs) Date: Wed, 03 Jun 2015 14:32:47 -0400 Subject: RFR 9: 8081567 : java/lang/ProcessHandle/InfoTest.java failed Cannot run program "whoami" In-Reply-To: <556ECD7D.7020703@oracle.com> References: <556CD0E1.2080102@Oracle.com> <556E21FD.10203@oracle.com> <556E2CA3.6050002@Oracle.com> <556ECD7D.7020703@oracle.com> Message-ID: <556F484F.2070901@Oracle.com> Hi Ivan, Corrections updated in place: http://cr.openjdk.java.net/~rriggs/webrev-whoami-8081567/ On 6/3/2015 5:48 AM, Ivan Gerasimov wrote: > > > On 03.06.2015 1:22, Roger Riggs wrote: >> Hi Ivan, >> >> Thanks for the review. >> >> Updated the webrev in place with 2 corrections. >> http://cr.openjdk.java.net/~rriggs/webrev-whoami-8081567 >> >> On 6/2/2015 5:37 PM, Ivan Gerasimov wrote: >>> 1) >>> >>> + if (domainLen > 0) { >>> + wcscat(domain, L"\\"); >>> + } else { >>> + domain[0] = L'\0'; >>> + } >>> + wcscat(domain, name); >>> >>> Here, domainLen is either equal to whatever LookupAccountSid has set >>> it to. >>> >>> In MSDN [1] it's written: >>> On input, specifies the size, in TCHARs, of the >>> lpReferencedDomainName buffer. If the function fails because the >>> buffer is too small or if cchReferencedDomainName is zero, >>> cchReferencedDomainName receives the required buffer size, including >>> the terminating null character. >>> >>> I see in the comments that in fact on success this parameter >>> receives the length of the resulting string. >>> But should we rely on that undocumented behavior? >> Returning the actual size is (one) convention of a number of Win APIs. > > My concern is that I don't see this convention explicitly stated, at > least in the documentation for LookupAccountSidW(). > On the other hand, the doc does promise that the string will be > null-terminated upon successful call. > > Thus, I think it's safer to just remove lines 423-427 and do > concatenation right away, unless you have evidence that > LookupAccountSidW may fail to produce null-terminated string, > contradicting the documentation. > In the later case, I think, it should be stated in the comments. ok; and similar to code in WindowsNativeDispatcher.c and WindowsUserPrincipals the concatenation should be unconditional. > >>> Does it really happen that LookupAccountSidW() returns 0, setting >>> domainLen to 0 and leaving the domain buffer uninitialized? >>> From the published example [2] it seems that this shouldn't happen. >> Only successful calls reach the code in question. >> If domainLen is returned as zero, the domain[0] is zero terminated; >> not counting on the API to do it. >> Otherwise, it returned (some) string and it is null terminated. >> Am I misunderstanding the case? >>> >>> 2) >>> I think domainLen should be set to >>> DWORD domainLen = sizeof(domain) - sizeof(name) - 1; >>> Otherwise LookupAccountSid is allowed to fill up the whole buffer, >>> so that 'name' could not be appended. >> Agree, but I think it should be -2; one for the delimiter and for >> the trailing \0. > > I was thinking of 256 (== sizeof(domain) - sizeof(name) - 1) for the > domain + \0. > Then the trailing \0 is replaced with the delimiter and the remaining > 256 WCHARs can be used for the name + \0; ok > >> I could not find a maximum length of a domain or name and picked 255. > > The example from MSDN [2] uses 256 for the name + terminating null, so > I think it should be fine. > To be extra-safe we could check GetLastError() for ERROR_NONE_MAPPED > and in such rare cases allocate the buffers dynamically. I don't see that NONE_MAPPED is associated with the output buffers. But taking the lead from the code in WindowsNativeDispatcher.c for producting the SID, I added code to fallback to the String version. Thanks, Roger > > Sincerely yours, > Ivan > >>> >>> 3) >>> Indentation isn't consistent: it should be multiple of 4. >> Thanks, helpful but not correct emacs c-mode. >> >> Roger >> >>> >>> Sincerely yours, >>> Ivan >>> >>> [1] >>> https://msdn.microsoft.com/en-us/library/windows/desktop/aa379166%28v=vs.85%29.aspx >>> [2] >>> https://msdn.microsoft.com/en-us/library/windows/desktop/aa379554(v=vs.85).aspx >>> >>>> Issue: >>>> https://bugs.openjdk.java.net/browse/JDK-8081567 >>>> >>>> Thanks, Roger >>>> >>>> >>> >> >> > From lance.andersen at oracle.com Wed Jun 3 18:33:05 2015 From: lance.andersen at oracle.com (Lance Andersen) Date: Wed, 3 Jun 2015 14:33:05 -0400 Subject: RFR 8081678: Add Stream returning methods to classes where there currently exist only Enumeration returning methods In-Reply-To: <586BD75D-F4D7-4CD6-BEA3-43BD21F43CBB@oracle.com> References: <3093DBE7-AC02-4371-B7EC-12C3A15EA334@oracle.com> <06A9E918-9FBF-41D9-B594-E94ED50A2089@oracle.com> <586BD75D-F4D7-4CD6-BEA3-43BD21F43CBB@oracle.com> Message-ID: looks good Paul, thank you Best Lance On Jun 3, 2015, at 12:35 PM, Paul Sandoz wrote: > > On Jun 3, 2015, at 12:06 AM, Lance Andersen wrote: > >> Hi Paul, >> >> All the changes seem reasonable. A couple minor suggestions >> >> - DriverManager.drivers() - I do not think we need to repeat the note from getDrivers(), otherwise, I would use {@code} vs in the new javadoc comment > > Removed. > > >> - DriverManagerTests.test19() - For the new test, if you could add a simple comment about the test, it would keep it consistent with the other tests, otherwise I can do it later >> > > Added. > > Webrev updated in place > > Thanks, > Paul. > >> Best >> Lance >> >> On Jun 2, 2015, at 9:37 AM, Paul Sandoz wrote: >> >>> Hi, >>> >>> Please review a follow up to Stuart's Enumeration.asIterator patch that adds some Stream return methods to classes where there is only Enumeration returning methods to support traversal: >>> >>> http://cr.openjdk.java.net/~psandoz/jdk9/JDK-8081678-enumeration-and-stream/webrev/ >>> >>> I took the opportunity to sprinkle some Enumerator.asIterator impls. >>> >>> I originally included some methods on ClassLoader but Alan wisely warned me away from doing that as this area is likely to be impacted by modularity. >>> >>> There is one small area of uncertainty with NetworkInterface. Can the following method ever return null? >>> >>> 342 public static Enumeration getNetworkInterfaces() >>> 343 throws SocketException { >>> 344 NetworkInterface[] netifs = getAll(); >>> 345 >>> 346 // specified to return null if no network interfaces >>> 347 return netifs != null >>> 348 ? enumerationFromArray(netifs) >>> 349 : null; >>> 350 } >>> >>> Contrary to the comment i cannot find any specification. For the stream returning method, networkInterfaces, i have specified this to return an empty stream, thus it might be good to update the enumeration returning method as well to say whether it returns null or an empty enumeration. >>> >>> Paul. >> >> >> >> Lance Andersen| Principal Member of Technical Staff | +1.781.442.2037 >> Oracle Java Engineering >> 1 Network Drive >> Burlington, MA 01803 >> Lance.Andersen at oracle.com >> >> >> > Lance Andersen| Principal Member of Technical Staff | +1.781.442.2037 Oracle Java Engineering 1 Network Drive Burlington, MA 01803 Lance.Andersen at oracle.com From Roger.Riggs at Oracle.com Wed Jun 3 18:36:40 2015 From: Roger.Riggs at Oracle.com (Roger Riggs) Date: Wed, 03 Jun 2015 14:36:40 -0400 Subject: RFR 9: 8067808 : java/lang/ProcessBuilder/Basic.java failed on Assertion In-Reply-To: <556EDDE3.9070807@oracle.com> References: <556DF6E0.2010000@Oracle.com> <556EDDE3.9070807@oracle.com> Message-ID: <556F4938.40807@Oracle.com> Hi Ivan, Thanks, Roger There is a pending enhancement to ProcessHandle to deal with uniqueness of pids. I can look at adding a note to process.getPid also. (The pid will be the pid when the process was alive.) Roger On 6/3/2015 6:58 AM, Ivan Gerasimov wrote: > Hi Roger! > > Seems Okay to me. > > Do you think it would make sense to make the doc for Process.getPid() > clearer that it guarantees to return the same pid even after calling > waitFor()? > This's true for current implementation, but from the OS point of view, > pid does not necessarily uniquely identifies the process after it has > terminated. > > Sincerely yours, > Ivan > > On 02.06.2015 21:33, Roger Riggs wrote: >> Please review test cleanup to make process id testing more reliable. >> It now spawns javaChild to report the child pid instead of parsing >> the output of OS specific commands. >> >> Webrev: >> http://cr.openjdk.java.net/~rriggs/webrev-builder-8067808/ >> >> Issue: >> http://cr.openjdk.java.net/~rriggs/webrev-builder-8067808/ >> >> Thanks, Roger >> >> p.s. I'm looking for a review also for: >> 8081567 : java/lang/ProcessHandle/InfoTest.java failed Cannot run >> program "whoami" >> >> >> > From mandy.chung at oracle.com Wed Jun 3 18:36:14 2015 From: mandy.chung at oracle.com (Mandy Chung) Date: Wed, 03 Jun 2015 11:36:14 -0700 Subject: RFR(M, v13): JDK-8059036 : Implement Diagnostic Commands for heap and finalizerinfo In-Reply-To: <556F1D6C.6010806@oracle.com> References: <554894E9.8020908@oracle.com> <1AF219C7-1055-4D97-BB46-2FA390C4135D@oracle.com> <5548D65C.8040304@gmail.com> <554A8CEE.2090009@oracle.com> <55523418.5010708@oracle.com> <55527935.5010408@gmail.com> <5553B552.9060900@oracle.com> <555453BA.2070205@gmail.com> <55567748.6020304@oracle.com> <55567A68.9020802@oracle.com> <5556F34B.5050701@oracle.com> <55573A31.7050306@gmail.com> <55573C86.8030807@gmail.com> <5559D85B.2050500@oracle.com> <6FB8DF19-1D67-4702-9642-4C630765D2D1@oracle.com> <555C2EEB.1050505@oracle.com> <3941B430-FE6C-4AF0-A875-76D24411C654@oracle.com> <5564802A.2010403@oracle.com> <5565738A.4040109@gmail.com> <7EB0BE71-1608-4047-B71D-1A9EAA9083BB@oracle.com> <556727A2.9060209@gmail.com> <55675908.6030406@oracle.com> <556AF3CD.50900@oracle.com> <37227441-B016-4A9A-B9CA-BE100E992C9A@oracle.com> <556D8FB0.4070608@oracle.com> <556E9970.8010704@gmail.com> <556EBF4B.6070400@oracle.com> <556F1D6C.6010806@oracle.com> Message-ID: <556F491E.3070709@oracle.com> On 06/03/2015 08:29 AM, Dmitry Samersoff wrote: > Updated webrev: > > http://cr.openjdk.java.net/~dsamersoff/JDK-8059036/webrev.13 I reviewed the jdk change. The version looks okay and some comments: ReferenceQueue.java - you should eliminate the use of rawtype: 84 Reference rn = r.next; Change Reference to Reference The forEach method - eliminate the use of raw type and move @SuppressWarning to the field variable in line 182: @SuppressWarnings("unchecked") Reference rn = r.next; FinalizerHistogram.java Copyright year needs update. I personally think the VM code would be much simpler if you stay with alternative entry of String and int than dealing with a FinalizerHistogramEntry private class. It's okay as FinalizerHistogram class is separated. The comment in line 35 is suitable to move to the class description and this is the suggestion. // This FinalizerHistogram class is for GC.finalizer_info diagnostic command support. // It is invoked by the VM. Should getFinalizerHistogram() return FinalizerHistogramEntry[]? The VM knows the returned type anyway. It's an inner class of FinalizerHistogram and it can simply be named as "Entry". I suggest to have Entry::increment method to replace ".instanceCount++". The tests are in hotspot/test. Can you add a unit test in jdk/test? Perhaps you can test FinalizerHistogram.getFinalizerHistogram() via reflection - just a sanity test. A minor naming comment - now you have a FinalizerHistogram class. Perhaps the getFinalizerHistogram method should be renamed e.g. "dump"? Mandy From kumar.x.srinivasan at oracle.com Wed Jun 3 18:48:47 2015 From: kumar.x.srinivasan at oracle.com (Kumar Srinivasan) Date: Wed, 03 Jun 2015 11:48:47 -0700 Subject: RFR: 8081824: Remove dead code GetPublicJREHome in the launcher Message-ID: <556F4C0F.2020704@oracle.com> Hello, Please review the removal of launcher/windows/registry dead code, the details of which are in the bug report. https://bugs.openjdk.java.net/browse/JDK-8081824 The webrev: http://cr.openjdk.java.net/~ksrini/8081824/webrev.00/ Thanks Kumar From mandy.chung at oracle.com Wed Jun 3 18:52:35 2015 From: mandy.chung at oracle.com (Mandy Chung) Date: Wed, 03 Jun 2015 11:52:35 -0700 Subject: RFR: 8081824: Remove dead code GetPublicJREHome in the launcher In-Reply-To: <556F4C0F.2020704@oracle.com> References: <556F4C0F.2020704@oracle.com> Message-ID: <556F4CF3.2040200@oracle.com> Looks good. Mandy On 06/03/2015 11:48 AM, Kumar Srinivasan wrote: > Hello, > > Please review the removal of launcher/windows/registry dead code, > the details of which are in the bug report. > https://bugs.openjdk.java.net/browse/JDK-8081824 > > The webrev: > http://cr.openjdk.java.net/~ksrini/8081824/webrev.00/ > > Thanks > Kumar > From peter.levart at gmail.com Wed Jun 3 18:53:42 2015 From: peter.levart at gmail.com (Peter Levart) Date: Wed, 03 Jun 2015 20:53:42 +0200 Subject: RFR 8071597 Add Stream dropWhile and takeWhile operations In-Reply-To: <79392DA7-6A8E-42AC-9E83-AA7F89D8F2A5@oracle.com> References: <79392DA7-6A8E-42AC-9E83-AA7F89D8F2A5@oracle.com> Message-ID: <556F4D36.5090701@gmail.com> Hi Paul, This is a usefull addition to Stream API for sequential ordered streams. But does it have any utility in unordered streams at all? Wouldn't it be better to just throw an IllegalStateException or something if the stream is not ordered? I can't imagine currently a situation where I would need to select an arbitrary sub-set of the stream where all elements match a predicate and I would not be given any guarantee about how big a subset I will get - the same with a stream where an arbitrary sub-set of elements matching predicate is taken away. If I wanted something similar for unordered stream, I would rather say: "Give me at most N elements of the stream matching predicate" for example - and that can already be achieved by .filter().limit(); So any use of [take|drop]While on unordered stream would most probably be unintended and consequently could be considered a bug, don't you think? Regards, Peter On 06/02/2015 03:13 PM, Paul Sandoz wrote: > Hi, > > Please review this webrev that adds take/dropWhile operations to streams: > > http://cr.openjdk.java.net/~psandoz/jdk9/JDK-8071597-take-drop-while/webrev/ > > I opted to weight the documentation of the operations towards ordered streams in the first paragraph. That is what makes most sense in terms of usage and what most people will read. Thus i refer to the "longest prefix" in the first paragraph then define what that means in subsequent paragraphs for ordered and unordered streams: > > 482 /** > 483 * Returns a stream consisting of the longest prefix of elements taken from > 484 * this stream that match the given predicate. > 485 * > 486 *

    If this stream is ordered then the prefix is a contiguous sequence of > 487 * elements of this stream. All elements of the sequence match the given > 488 * predicate, the first element of the sequence is the first element > 489 * (if any) of this stream, and the element (if any) immediately following > 490 * the last element of the sequence does not match the given predicate. > 491 * > 492 *

    If this stream is unordered then the prefix is a subset of elements of > 493 * this stream. All elements (if any) of the subset match the given > 494 * predicate. In this case the behavior of this operation is > 495 * nondeterministic; it is free to select any valid subset as the prefix. > 496 * > 497 *

    This is a short-circuiting > 498 * stateful intermediate operation. > 499 * > ... > 528 default Stream takeWhile(Predicate predicate) { > > 537 /** > 538 * Returns a stream consisting of the remaining elements of this stream > 539 * after dropping the longest prefix of elements that match the given > 540 * predicate. > 541 * > 542 *

    If this stream is ordered then the prefix is a contiguous sequence of > 543 * elements of this stream. All elements of the sequence match the given > 544 * predicate, the first element of the sequence is the first element > 545 * (if any) of this stream, and the element (if any) immediately following > 546 * the last element of the sequence does not match the given predicate. > 547 * > 548 *

    If this stream is unordered then the prefix is a subset of elements of > 549 * this stream. All elements (if any) of the subset match the given > 550 * predicate. In this case the behavior of this operation is > 551 * nondeterministic; it is free to select any valid subset as the prefix. > 552 * > ... > 584 default Stream dropWhile(Predicate predicate) { > > > After this has been reviewed i will follow up with a further issue regarding the specification of takeWhile, stateful predicates and cancellation. I avoided such specification here as it's likely to rathole :-) > > Basically the takeWhile operation is implemented such that one can do: > > long t = System.currentTimeMillis(); > List pps = Stream > .generate(() -> BigInteger.probablePrime(1024, ThreadLocalRandom.current())) > .parallel() > .takeWhile(e -> (System.currentTimeMillis() - t) < TimeUnit.SECONDS.toMillis(5)) > .collect(toList()); > > Paul. From sean.coffey at oracle.com Wed Jun 3 19:14:07 2015 From: sean.coffey at oracle.com (=?UTF-8?B?U2XDoW4gQ29mZmV5?=) Date: Wed, 03 Jun 2015 20:14:07 +0100 Subject: [8u-dev] Review request : JDK-8068416: LFGarbageCollectedTest.java fails with OOME: "GC overhead limit exceeded" In-Reply-To: <556F2C55.5060304@oracle.com> References: <556F2C55.5060304@oracle.com> Message-ID: <556F51FF.8000202@oracle.com> I bumped into this failure myself today. I think you've got a typo. 440 should be 40. Looks like a good approach otherwise. Regards, Sean. On 03/06/2015 17:33, Konstantin Shefov wrote: > Hello, > > Please review the test bug fix > https://bugs.openjdk.java.net/browse/JDK-8068416 > Webrev is http://cr.openjdk.java.net/~kshefov/8068416/webrev.00/ > Test fails only against JDK 8u and passes against JDK 9. > > Fix is to reduce the number of iterations to 40. With that number of > iterations the test passes on those hosts where it failed before. > The number of iterations the test start to fail is 65. > Before the fix the number of iterations was 84. > > Thanks > -Konstantin From xueming.shen at oracle.com Wed Jun 3 19:19:33 2015 From: xueming.shen at oracle.com (Xueming Shen) Date: Wed, 03 Jun 2015 12:19:33 -0700 Subject: RFR 8072773 (fs) Files.lines needs a better splitting implementation for stream source In-Reply-To: References: Message-ID: <556F5345.9060604@oracle.com> On 06/03/2015 08:53 AM, Paul Sandoz wrote: > Hi, > > Please review an optimization for Files.lines for certain charsets: > > http://cr.openjdk.java.net/~psandoz/jdk9/JDK-8072773-File-lines/webrev/ > > If a charset is say US-ASCII or UTF-8 it is possible to implement an efficient splitting Spliterator that scans bytes from a mid-point to search for line feed characters. > > Splitting uses a mapped byte buffer. Traversal uses FileChannel.reads at an offset. In previous incarnations i tried to use mapped byte buffer for both, but for some reason the traversal performance was not good (both on Mac and x86). In any case i am happy with the current approach as there is minimal layering between the FileChannel and BufferedReader leveraged to read the lines. > > Sequential performance is similar (same or better) than the current approach. Parallel performance is much better than the current approach. > > Some advice on two aspects would be most appreciated: > > 1) Is there an easy way to determine the sub-set of supported charsets that are applicable? > It's easy though a little heavy :-) getLFCR returns a byte[] for the "byte" form of the \n and \r in a particular encodings, if each one of them can be mapped into one byte. Then we can use b[0] for \n and b[1] for \r in trySplit(). This makes the new fast version work for most of charsets. private static byte[] getLFCR(Charset cs) { try { if (cs.canEncode()) { ByteBuffer bb = cs.newEncoder() .encode(CharBuffer.wrap(new char[] { '\n', '\r' })); if (bb.remaining() == 2) { CharBuffer cb = cs.newDecoder().decode(bb); if (cb.remaining() == 2 && cb.get() == '\n' && cb.get() == '\r') { bb.flip(); byte[] ba = new byte[2]; bb.get(ba); return ba; } } } } catch (Exception x) {} return null; } -sherman From peter.levart at gmail.com Wed Jun 3 19:22:22 2015 From: peter.levart at gmail.com (Peter Levart) Date: Wed, 03 Jun 2015 21:22:22 +0200 Subject: RFR(M, v13): JDK-8059036 : Implement Diagnostic Commands for heap and finalizerinfo In-Reply-To: <556F491E.3070709@oracle.com> References: <554894E9.8020908@oracle.com> <1AF219C7-1055-4D97-BB46-2FA390C4135D@oracle.com> <5548D65C.8040304@gmail.com> <554A8CEE.2090009@oracle.com> <55523418.5010708@oracle.com> <55527935.5010408@gmail.com> <5553B552.9060900@oracle.com> <555453BA.2070205@gmail.com> <55567748.6020304@oracle.com> <55567A68.9020802@oracle.com> <5556F34B.5050701@oracle.com> <55573A31.7050306@gmail.com> <55573C86.8030807@gmail.com> <5559D85B.2050500@oracle.com> <6FB8DF19-1D67-4702-9642-4C630765D2D1@oracle.com> <555C2EEB.1050505@oracle.com> <3941B430-FE6C-4AF0-A875-76D24411C654@oracle.com> <5564802A.2010403@oracle.com> <5565738A.4040109@gmail.com> <7EB0BE71-1608-4047-B71D-1A9EAA9083BB@oracle.com> <556727A2.9060209@gmail.com> <55675908.6030406@oracle.com> <556AF3CD.50900@oracle.com> <37227441-B016-4A9A-B9CA-BE100E992C9A@oracle.com> <556D8FB0.4070608@oracle.com> <556E9970.8010704@gmail.com> <556EBF4B.6070400@oracle.com> <556F1D6C.6010806@oracle.com> <556F491E.3070709@oracle.com> Message-ID: <556F53EE.3050207@gmail.com> On 06/03/2015 08:36 PM, Mandy Chung wrote: > > > On 06/03/2015 08:29 AM, Dmitry Samersoff wrote: >> Updated webrev: >> >> http://cr.openjdk.java.net/~dsamersoff/JDK-8059036/webrev.13 > > I reviewed the jdk change. The version looks okay and some comments: > > ReferenceQueue.java - you should eliminate the use of rawtype: > > 84 Reference rn = r.next; > > Change Reference to Reference > > The forEach method - eliminate the use of raw type and > move @SuppressWarning to the field variable in line 182: > > @SuppressWarnings("unchecked") > Reference rn = r.next; Thanks, Mandy. This was my doing. The @SuppressWarnings can be moved to the local var declaration in line 84 too. Here's how the fixed ReferenceQueue looks like after those two changes: http://cr.openjdk.java.net/~plevart/misc/Finalizer.printFinalizationQueue/webrev.03/ > > FinalizerHistogram.java > Copyright year needs update. > > I personally think the VM code would be much simpler if you stay with > alternative entry of String and int than dealing with a > FinalizerHistogramEntry private class. It's okay as > FinalizerHistogram class is separated. > > The comment in line 35 is suitable to move to the class description > and this is the suggestion. > > // This FinalizerHistogram class is for GC.finalizer_info > diagnostic command support. > // It is invoked by the VM. > > Should getFinalizerHistogram() return FinalizerHistogramEntry[]? The > VM knows the returned type anyway. It's an inner class of > FinalizerHistogram and it can simply be named as "Entry". I suggest > to have Entry::increment method to replace ".instanceCount++". This FinalizerHistogram.Entry naming part came to my mind too, yes. If there is a method for incrementing, then both fields can be marked private. Regards, Peter > > The tests are in hotspot/test. Can you add a unit test in > jdk/test? Perhaps you can test > FinalizerHistogram.getFinalizerHistogram() via reflection - just a > sanity test. > > A minor naming comment - now you have a FinalizerHistogram class. > Perhaps the getFinalizerHistogram method should be renamed e.g. "dump"? > > Mandy From Alan.Bateman at oracle.com Wed Jun 3 19:27:54 2015 From: Alan.Bateman at oracle.com (Alan Bateman) Date: Wed, 03 Jun 2015 20:27:54 +0100 Subject: RFR 8081678: Add Stream returning methods to classes where there currently exist only Enumeration returning methods In-Reply-To: <2E28B338-0289-4503-B713-B30E4E1704D0@oracle.com> References: <3093DBE7-AC02-4371-B7EC-12C3A15EA334@oracle.com> <556EDB10.9060708@oracle.com> <2E28B338-0289-4503-B713-B30E4E1704D0@oracle.com> Message-ID: <556F553A.1070901@oracle.com> On 03/06/2015 17:47, Paul Sandoz wrote: > : > Ok, i removed it but added an assert for the array being non-null and containing at least one element. I also refined the documentation of the stream returning method in light of this: > > http://cr.openjdk.java.net/~psandoz/jdk9/JDK-8081678-enumeration-and-stream/webrev/ > The update looks good. A minor point on getInetAddresses where the javadoc has been changed to "Get an Enumeration ...". I think that would be better as "Returns an Enumeration". Same thing in inetAddresses() where "Return" would read a bit better (IMO anyway). A niggle in inetAddresses is that it's got two /**. I was surprised to see PermissionCollection on the list, I don't know how often that is used. For the tests then @library ../../util/streambootlib doesn't seem right. Is it time to move some infrastructure to make it easier to get at in other parts of the suite? -Alan From paul.sandoz at oracle.com Wed Jun 3 19:58:58 2015 From: paul.sandoz at oracle.com (Paul Sandoz) Date: Wed, 3 Jun 2015 21:58:58 +0200 Subject: RFR 8072773 (fs) Files.lines needs a better splitting implementation for stream source In-Reply-To: <556F5345.9060604@oracle.com> References: <556F5345.9060604@oracle.com> Message-ID: <33E805FD-D51A-4398-8B7F-308C481AEDF3@oracle.com> On Jun 3, 2015, at 9:19 PM, Xueming Shen wrote: > On 06/03/2015 08:53 AM, Paul Sandoz wrote: >> Hi, >> >> Please review an optimization for Files.lines for certain charsets: >> >> http://cr.openjdk.java.net/~psandoz/jdk9/JDK-8072773-File-lines/webrev/ >> >> If a charset is say US-ASCII or UTF-8 it is possible to implement an efficient splitting Spliterator that scans bytes from a mid-point to search for line feed characters. >> >> Splitting uses a mapped byte buffer. Traversal uses FileChannel.reads at an offset. In previous incarnations i tried to use mapped byte buffer for both, but for some reason the traversal performance was not good (both on Mac and x86). In any case i am happy with the current approach as there is minimal layering between the FileChannel and BufferedReader leveraged to read the lines. >> >> Sequential performance is similar (same or better) than the current approach. Parallel performance is much better than the current approach. >> >> Some advice on two aspects would be most appreciated: >> >> 1) Is there an easy way to determine the sub-set of supported charsets that are applicable? >> > > It's easy though a little heavy :-) Thanks, that is a little heavy, but i suppose computed values for charsets could be stashed in a static CHM. Paul. > getLFCR returns a byte[] for the "byte" form of > the \n and \r in a particular encodings, if each one of them can be mapped into > one byte. Then we can use b[0] for \n and b[1] for \r in trySplit(). This makes > the new fast version work for most of charsets. > > private static byte[] getLFCR(Charset cs) { > try { > if (cs.canEncode()) { > ByteBuffer bb = cs.newEncoder() > .encode(CharBuffer.wrap(new char[] { '\n', '\r' })); > if (bb.remaining() == 2) { > CharBuffer cb = cs.newDecoder().decode(bb); > if (cb.remaining() == 2 && > cb.get() == '\n' && cb.get() == '\r') { > bb.flip(); > byte[] ba = new byte[2]; > bb.get(ba); > return ba; > } > } > } > } catch (Exception x) {} > return null; > > } > > -sherman From andreas.lundblad at oracle.com Wed Jun 3 20:16:07 2015 From: andreas.lundblad at oracle.com (Andreas Lundblad) Date: Wed, 3 Jun 2015 22:16:07 +0200 Subject: Why isn't Object.notify() a synchronized method? In-Reply-To: <556A8E9D.4040407@oracle.com> References: <55673D97.4010505@CoSoCo.de> <5567420A.9020406@redhat.com> <5568985C.6050507@CoSoCo.de> <556A8E9D.4040407@oracle.com> Message-ID: <20150603201607.GB4474@e6430> On Sun, May 31, 2015 at 02:31:25PM +1000, David Holmes wrote: > >As I recently fell into the trap of forgetting the synchronized block > >around a single notifyAll(), I believe, the current situation is just > >errorprone. > > How is it errorprone? You forgot to acquire the lock and you got an > IllegalMonitorStateException when you did notifyAll. That's pointing > out your error. The reason for not making wait/notify synchronized is *not* that "it would be unnecessary because you typically already hold the lock anyway". The reason is (as David Holms pointed out earlier) that it would be *meaningless* to make them synchronized. When you say it's errorprone it sounds like you first had notifyAll(); and then "fixed" the IllegalMonitorStateException by doing synchronized (this) { notifyAll(); } (and then wrote your original email asking why notifyAll didn't do this "on the inside"). If this is the case you have not understood the intention of using synchronized here. In a nutshell wait and notify is all about thread communication, and for that to work correctly you need to synchronize your execution. Let me try to explain why wrapping *just* notifyAll() in a synchronized block (or relying on making notifyAll synchronized) is broken: Suppose you have a producer / consumer thing going on. In produce() you have something like enqueue(value); synchronized (this) { notifyAll(); // because consumer may be waiting for a value } and in consume() you have something like synchronized (this) { while (noValueIsAvailable()) wait(); } value = retrieve(); Suppose now that ProducerThread enters produce(), enqueues a value and before reaching notifyAll, ConsumerThread comes along, enters consume(), consumes the value, processes the value, calls consume *again*, sees that noValueIsAvailable and calls wait. ProducerThread now continues it's execution and calls notifyAll(). ConsumerThread is woken up, but at this point no value is available despite there was a call to notify! (In the best case, this doesn't crash the program, in worst case, ProducerThread assumed that the value should be processed after notifyAll, in which case you may run into a deadlock. If there were more variables involved you could also have memory visibility issues involved and accidentally break class invariants by doing this.) I've written an answer to a similar question here: "Why must wait() always be in synchronized block" http://stackoverflow.com/a/2779674/276052 best regards, Andreas From paul.sandoz at oracle.com Wed Jun 3 20:17:39 2015 From: paul.sandoz at oracle.com (Paul Sandoz) Date: Wed, 3 Jun 2015 22:17:39 +0200 Subject: RFR 8081678: Add Stream returning methods to classes where there currently exist only Enumeration returning methods In-Reply-To: <556F553A.1070901@oracle.com> References: <3093DBE7-AC02-4371-B7EC-12C3A15EA334@oracle.com> <556EDB10.9060708@oracle.com> <2E28B338-0289-4503-B713-B30E4E1704D0@oracle.com> <556F553A.1070901@oracle.com> Message-ID: On Jun 3, 2015, at 9:27 PM, Alan Bateman wrote: > On 03/06/2015 17:47, Paul Sandoz wrote: >> : >> Ok, i removed it but added an assert for the array being non-null and containing at least one element. I also refined the documentation of the stream returning method in light of this: >> >> http://cr.openjdk.java.net/~psandoz/jdk9/JDK-8081678-enumeration-and-stream/webrev/ >> > The update looks good. > > A minor point on getInetAddresses where the javadoc has been changed to "Get an Enumeration ...". I think that would be better as "Returns an Enumeration". Same thing in inetAddresses() where "Return" would read a bit better (IMO anyway). > It's frustratingly inconsistent throughout, i tried to make it locally consistent (for example, see the instance methods for sub interfaces). The whole doc needs to be consistently updated. > A niggle in inetAddresses is that it's got two /**. > Ooops fixed. > I was surprised to see PermissionCollection on the list, I don't know how often that is used. > grepcode.com shows quite a few usages of PermissionsCollection.elements(). Twas cheap to add. > For the tests then @library ../../util/streambootlib doesn't seem right. Is it time to move some infrastructure to make it easier to get at in other parts of the suite? > Quite probably, but not with this fix. Thanks, Paul. From igor.ignatyev at oracle.com Wed Jun 3 20:47:44 2015 From: igor.ignatyev at oracle.com (Igor Ignatyev) Date: Wed, 03 Jun 2015 23:47:44 +0300 Subject: [8u-dev] Review request : JDK-8068416: LFGarbageCollectedTest.java fails with OOME: "GC overhead limit exceeded" In-Reply-To: <556F51FF.8000202@oracle.com> References: <556F2C55.5060304@oracle.com> <556F51FF.8000202@oracle.com> Message-ID: <556F67F0.1040504@oracle.com> Konstantin, do you have an explanation why the test passes on jdk 9? from my point of view, it indicates there is a product bug in 8u which should be fixed and your fix just hides it. Igor On 06/03/2015 10:14 PM, Se?n Coffey wrote: > I bumped into this failure myself today. I think you've got a typo. > 440 should be 40. Looks like a good approach otherwise. > > Regards, > Sean. > > On 03/06/2015 17:33, Konstantin Shefov wrote: >> Hello, >> >> Please review the test bug fix >> https://bugs.openjdk.java.net/browse/JDK-8068416 >> Webrev is http://cr.openjdk.java.net/~kshefov/8068416/webrev.00/ >> Test fails only against JDK 8u and passes against JDK 9. >> >> Fix is to reduce the number of iterations to 40. With that number of >> iterations the test passes on those hosts where it failed before. >> The number of iterations the test start to fail is 65. >> Before the fix the number of iterations was 84. >> >> Thanks >> -Konstantin > From Ulf.Zibis at CoSoCo.de Wed Jun 3 20:56:40 2015 From: Ulf.Zibis at CoSoCo.de (Ulf Zibis) Date: Wed, 03 Jun 2015 22:56:40 +0200 Subject: String.contains(CharSequence) calls toString on argument In-Reply-To: References: <550C3F7A.1020404@oracle.com> <550C42F0.6010506@oracle.com> <5566EF7D.9020104@CoSoCo.de> Message-ID: <556F6A08.90100@CoSoCo.de> Hi, thanks for your feedback. I think it would be more readable here, if you would use if ... else ... . To me a ternary operator is more readable, if there is one value to compute. Also it enhances readability over the whole code, if I use less lines as possible, in other words, I hate to scroll so much. -Ulf Am 28.05.2015 um 12:57 schrieb Tomasz Kowalczewski: > Hi, > > thank you for taking time to read my proposal. Is this a matter of some OpenJDK coding > conventions? I myself never use ternary operator and always use if with braces. My experience is > that it improves code readability. If I have to apply one of proposed changes I would choose first > or third. > > Regards, > Tomasz > > On Thu, May 28, 2015 at 12:35 PM, Ulf Zibis > wrote: > > Hi, > > I more would like: > > 2199 return (s instanceof String) > 2200 ? indexOf((String) s) >= 0; > 2201 : indexOf(value, 0, value.length, s, 0, s.length(), 0) >= 0; > > or: > > 2199 return (s instanceof String > 2200 ? indexOf((String) s) > 2201 : indexOf(value, 0, value.length, s, 0, s.length(), 0)) > 2202 >= 0; > > or: > > 2199 int index = (s instanceof String) > 2200 ? indexOf((String) s) > 2201 : indexOf(value, 0, value.length, s, 0, s.length(), 0); > 2202return index >= 0; > > -Ulf > > > > Am 28.05.2015 um 12:06 schrieb Tomasz Kowalczewski: > > Hello all, > > encouraged by your responses I did a simple implementation and performed > some JMH experiments. Details below. > > Webrev [1] contains my changes. I deliberately wanted to do minimal code > changes. The gist of it is implementing indexOf that works on CharSequence > instead of String's internal char array. Both versions are almost identical > (I did not change existing implementation in place to not impact all other > String-only paths that use it). > > In my first attempt I just inlined (and simplified) indexOf implementation > into String::contains, but in the end decided to create general purpose > (package private) version. This might prove useful for others[2] > > JMH test project is here [3]. All benchmarks were run using "java -jar > benchmarks.jar -wi 10 -i 10 -f -prof gc" on Amazon EC2 instance (c4.xlarge, > 4 virtual cores). I have run it against three java builds: > > a. stock java 8u40 > b. my own build from clean jdk9 sources > c. my own build from modified jdk9 sources > > Results with gc profiler enabled are included in benchmark repo. This gist > contains summary results [4]. > > I know that this test covers only very narrow space of possible performance > regressions (ideas form more comprehensive tests are welcome). The results > show that String only path is not impacted by my changes. When using actual > CharSequence (StringBuilder in this case) we are mostly winning, exception > is case when CharSequence is of medium size and has many partial matches in > searched string. > > I hope this exercise will be useful to someone and that this change might > be considered for inclusion in OpenJDK. If not then maybe at least tests > for String::indexOf? > > [1] http://openjdk.s3-website-us-east-1.amazonaws.com/ > [2] Changeset from > http://mail.openjdk.java.net/pipermail/core-libs-dev/2015-May/033678.html uses > indexOf that could accept CharSequence instead of String. > [3] https://github.com/tkowalcz/openjdk-jmh > [4] https://gist.github.com/tkowalcz/d8f5b9435e184f65fd2a > > Regards, > Tomasz Kowalczewski > > On Sat, Mar 21, 2015 at 9:21 AM, Tomasz Kowalczewski < > tomasz.kowalczewski at gmail.com > wrote: > > There are other places which accept CharSequence and iterate over it > not caring about possible concurrent modification (the only sane way > IMHO). Examples are String.contentEquals and String.join that uses > StringJoiner which iterates its argument char by char. > > It looks like contains is the exception in this regard. > > On 20 mar 2015, at 17:05, Vitaly Davidovich > wrote: > > If CharSequence is mutable and thread-safe, I would expect toString() > implementation to provide the atomic snapshot (e.g. do the > synchronization). Therefore, I find Sherman's argument interesting. > > > That's my point about "it ought to be up to the caller" -- they're in a > position to make this call, but String.contains() is playing defensive > because it has no clue of the context. > > On Fri, Mar 20, 2015 at 11:55 AM, Aleksey Shipilev < > aleksey.shipilev at oracle.com > wrote: > > If CharSequence is mutable and thread-safe, I would expect toString() > implementation to provide the atomic snapshot (e.g. do the > synchronization). Therefore, I find Sherman's argument interesting. > > On the other hand, we don't seem to be having any details/requirements > for contains(CharSequence) w.r.t. it's own argument. What if > String.contains pulls the values one charAt at a time? The concurrency > requirements are not enforceable in CharSequence alone then, unless you > call the supposed-to-be-atomic toString() first. > > -Aleksey. > > On 03/20/2015 06:48 PM, Vitaly Davidovich wrote: > Yes, but that ought to be for the caller to decide. Also, although the > resulting String is immutable, toString() itself may observe mutation. > > On Fri, Mar 20, 2015 at 11:40 AM, Xueming Shen < > > xueming.shen at oracle.com > > > wrote: > > On 03/20/2015 02:34 AM, Tomasz Kowalczewski wrote: > > Hello! > > Current implementation of String.contains that accepts CharSequence > > calls > > toString on it and passes resulting string to indexOf(String). This > > IMO > > defeats the purpose of using CharSequences (that is to have a mutable > character buffer and not allocate unnecessary objects). > > It is arguable that cs.toString() may serve the purpose of taking a > snapshot of an otherwise > "mutable" character buffer? > > -Sherman > > > Is changing this a desirable development? It seems pretty > > straightforward > > to port indexOf(String) to use CharSequence. > > If all you need is patch then I can work on it (I have signed OCA) > > just > > wanted to make sure it is not a futile work. > > > > > > > > > > -- > Tomasz Kowalczewski From chris.hegarty at oracle.com Wed Jun 3 21:00:20 2015 From: chris.hegarty at oracle.com (Chris Hegarty) Date: Wed, 3 Jun 2015 22:00:20 +0100 Subject: RFR 8081678: Add Stream returning methods to classes where there currently exist only Enumeration returning methods In-Reply-To: References: <3093DBE7-AC02-4371-B7EC-12C3A15EA334@oracle.com> <556EDB10.9060708@oracle.com> <2E28B338-0289-4503-B713-B30E4E1704D0@oracle.com> <556F553A.1070901@oracle.com> Message-ID: <557A7A1C-611B-4BE3-96AA-803148778848@oracle.com> Looks good Paul, just a few minor comments. ( looked at all, but *security* and *sql* ) NetworkInterface s/Enumertion/Stream L127 * will be returned in the Enumeration. However, if the caller has the s/an/a L131 * @return an Stream object with all or a subset of the InetAddresses L 209 * @return an Stream object with all of the subinterfaces -Chris. > On 3 Jun 2015, at 21:17, Paul Sandoz wrote: > > > On Jun 3, 2015, at 9:27 PM, Alan Bateman wrote: > >> On 03/06/2015 17:47, Paul Sandoz wrote: >>> : >>> Ok, i removed it but added an assert for the array being non-null and containing at least one element. I also refined the documentation of the stream returning method in light of this: >>> >>> http://cr.openjdk.java.net/~psandoz/jdk9/JDK-8081678-enumeration-and-stream/webrev/ >>> >> The update looks good. >> >> A minor point on getInetAddresses where the javadoc has been changed to "Get an Enumeration ...". I think that would be better as "Returns an Enumeration". Same thing in inetAddresses() where "Return" would read a bit better (IMO anyway). >> > > It's frustratingly inconsistent throughout, i tried to make it locally consistent (for example, see the instance methods for sub interfaces). The whole doc needs to be consistently updated. > > >> A niggle in inetAddresses is that it's got two /**. >> > > Ooops fixed. > > >> I was surprised to see PermissionCollection on the list, I don't know how often that is used. >> > > grepcode.com shows quite a few usages of PermissionsCollection.elements(). Twas cheap to add. > > >> For the tests then @library ../../util/streambootlib doesn't seem right. Is it time to move some infrastructure to make it easier to get at in other parts of the suite? >> > > Quite probably, but not with this fix. > > Thanks, > Paul. From ivan.gerasimov at oracle.com Wed Jun 3 21:58:33 2015 From: ivan.gerasimov at oracle.com (Ivan Gerasimov) Date: Thu, 04 Jun 2015 00:58:33 +0300 Subject: RFR 9: 8081567 : java/lang/ProcessHandle/InfoTest.java failed Cannot run program "whoami" In-Reply-To: <556F484F.2070901@Oracle.com> References: <556CD0E1.2080102@Oracle.com> <556E21FD.10203@oracle.com> <556E2CA3.6050002@Oracle.com> <556ECD7D.7020703@oracle.com> <556F484F.2070901@Oracle.com> Message-ID: <556F7889.1090902@oracle.com> Thanks! ------------------ test/java/lang/ProcessHandle/InfoTest.java ------------------ Looks good ------------------ src/java.base/windows/native/libjava/ProcessHandleImpl_win.c ------------------ The changes look good to me! I guess it does have more sense to use StringSID if the length of user name of domain exceeds 256 chars. > > Corrections updated in place: > http://cr.openjdk.java.net/~rriggs/webrev-whoami-8081567/ > >> To be extra-safe we could check GetLastError() for ERROR_NONE_MAPPED >> and in such rare cases allocate the buffers dynamically. > I don't see that NONE_MAPPED is associated with the output buffers. > Yes, my mistake. That's output value of domainLen and nameLen that have to be analyzed to detect 'buffer is too small' condition. But I think, what you have is fine. A couple of minor nits. I would be just a little bit happier, if it were 397 WCHAR domain[255 + 1 + 255 + 1]; // large enough to concat with '/' and name 398 WCHAR name[255 *+ 1*]; 399 DWORD domainLen = *sizeof(domain) - sizeof(name)*; Otherwise, we would never use the last char of 'domain' :-) You may want to remove the extra space in the signature of procToUser(_ ), in both the forward declaration and definition. Sincerely yours, Ivan From Roger.Riggs at Oracle.com Wed Jun 3 22:13:58 2015 From: Roger.Riggs at Oracle.com (Roger Riggs) Date: Wed, 03 Jun 2015 18:13:58 -0400 Subject: RFR 9: 8081567 : java/lang/ProcessHandle/InfoTest.java failed Cannot run program "whoami" In-Reply-To: <556F7889.1090902@oracle.com> References: <556CD0E1.2080102@Oracle.com> <556E21FD.10203@oracle.com> <556E2CA3.6050002@Oracle.com> <556ECD7D.7020703@oracle.com> <556F484F.2070901@Oracle.com> <556F7889.1090902@oracle.com> Message-ID: <556F7C26.5060203@Oracle.com> Hi Ivan, Thanks for the cleanup suggestions. Roger On 6/3/2015 5:58 PM, Ivan Gerasimov wrote: > Thanks! > > ------------------ > test/java/lang/ProcessHandle/InfoTest.java > ------------------ > Looks good > > > ------------------ > src/java.base/windows/native/libjava/ProcessHandleImpl_win.c > ------------------ > The changes look good to me! > I guess it does have more sense to use StringSID if the length of user > name of domain exceeds 256 chars. > >> >> Corrections updated in place: >> http://cr.openjdk.java.net/~rriggs/webrev-whoami-8081567/ >> >>> To be extra-safe we could check GetLastError() for ERROR_NONE_MAPPED >>> and in such rare cases allocate the buffers dynamically. >> I don't see that NONE_MAPPED is associated with the output buffers. >> > > Yes, my mistake. That's output value of domainLen and nameLen that > have to be analyzed to detect 'buffer is too small' condition. > But I think, what you have is fine. > > > A couple of minor nits. > I would be just a little bit happier, if it were > 397 WCHAR domain[255 + 1 + 255 + 1]; // large enough to concat > with '/' and name > 398 WCHAR name[255 *+ 1*]; > 399 DWORD domainLen = *sizeof(domain) - sizeof(name)*; > > Otherwise, we would never use the last char of 'domain' :-) > > You may want to remove the extra space in the signature of > procToUser(_ ), in both the forward declaration and definition. > > Sincerely yours, > Ivan > From mandy.chung at oracle.com Wed Jun 3 22:56:34 2015 From: mandy.chung at oracle.com (Mandy Chung) Date: Wed, 3 Jun 2015 15:56:34 -0700 Subject: RFR(M, v13): JDK-8059036 : Implement Diagnostic Commands for heap and finalizerinfo In-Reply-To: <556F53EE.3050207@gmail.com> References: <554894E9.8020908@oracle.com> <1AF219C7-1055-4D97-BB46-2FA390C4135D@oracle.com> <5548D65C.8040304@gmail.com> <554A8CEE.2090009@oracle.com> <55523418.5010708@oracle.com> <55527935.5010408@gmail.com> <5553B552.9060900@oracle.com> <555453BA.2070205@gmail.com> <55567748.6020304@oracle.com> <55567A68.9020802@oracle.com> <5556F34B.5050701@oracle.com> <55573A31.7050306@gmail.com> <55573C86.8030807@gmail.com> <5559D85B.2050500@oracle.com> <6FB8DF19-1D67-4702-9642-4C630765D2D1@oracle.com> <555C2EEB.1050505@oracle.com> <3941B430-FE6C-4AF0-A875-76D24411C654@oracle.com> <5564802A.2010403@oracle.com> <5565738A.4040109@gmail.com> <7EB0BE71-1608-4047-B71D-1A9EAA9083BB@oracle.com> <556727A2.9060209@gmail.com> <55675908.6030406@oracle.com> <556AF3CD.50900@oracle.com> <37227441-B016-4A9A-B9CA-BE100E992C9A@oracle.com> <556D8FB0.4070608@oracle.com> <556E9970.8010704@gmail.com> <556EBF4B.6070400@oracle.com> <556F1D6C.6010806@oracle.com> <556F491E.3070709@oracle.com> <556F53EE.3050207! @gmail.com> Message-ID: <523E93C8-B707-4CF6-A26E-B5400E04C402@oracle.com> > On Jun 3, 2015, at 12:22 PM, Peter Levart wrote: > > > On 06/03/2015 08:36 PM, Mandy Chung wrote: >> >> >> On 06/03/2015 08:29 AM, Dmitry Samersoff wrote: >>> Updated webrev: >>> >>> http://cr.openjdk.java.net/~dsamersoff/JDK-8059036/webrev.13 >> >> I reviewed the jdk change. The version looks okay and some comments: >> >> ReferenceQueue.java - you should eliminate the use of rawtype: >> >> 84 Reference rn = r.next; >> >> Change Reference to Reference >> >> The forEach method - eliminate the use of raw type and >> move @SuppressWarning to the field variable in line 182: >> >> @SuppressWarnings("unchecked") >> Reference rn = r.next; > > Thanks, Mandy. This was my doing. The @SuppressWarnings can be moved to the local var declaration in line 84 too. Here's how the fixed ReferenceQueue looks like after those two changes: > > http://cr.openjdk.java.net/~plevart/misc/Finalizer.printFinalizationQueue/webrev.03/ This looks good. > >> >> FinalizerHistogram.java >> Copyright year needs update. >> >> I personally think the VM code would be much simpler if you stay with alternative entry of String and int than dealing with a FinalizerHistogramEntry private class. It's okay as FinalizerHistogram class is separated. >> >> The comment in line 35 is suitable to move to the class description and this is the suggestion. >> >> // This FinalizerHistogram class is for GC.finalizer_info diagnostic command support. >> // It is invoked by the VM. >> >> Should getFinalizerHistogram() return FinalizerHistogramEntry[]? The VM knows the returned type anyway. It's an inner class of FinalizerHistogram and it can simply be named as "Entry". I suggest to have Entry::increment method to replace ".instanceCount++". > > This FinalizerHistogram.Entry naming part came to my mind too, yes. If there is a method for incrementing, then both fields can be marked private. Yup. Mandy > > Regards, Peter > >> >> The tests are in hotspot/test. Can you add a unit test in jdk/test? Perhaps you can test FinalizerHistogram.getFinalizerHistogram() via reflection - just a sanity test. >> >> A minor naming comment - now you have a FinalizerHistogram class. Perhaps the getFinalizerHistogram method should be renamed e.g. "dump"? >> >> Mandy > From chris.plummer at oracle.com Wed Jun 3 23:23:08 2015 From: chris.plummer at oracle.com (Chris Plummer) Date: Wed, 03 Jun 2015 16:23:08 -0700 Subject: [9] RFR (M) 8054386: Allow Java debugging when CDS is enabled In-Reply-To: <556EBD91.6030204@oracle.com> References: <555B47EA.3000900@oracle.com> <556D5CFD.1020709@oracle.com> <556DFC33.4060600@oracle.com> <556E73AF.3020606@oracle.com> <556EBD91.6030204@oracle.com> Message-ID: <556F8C5C.6070704@oracle.com> Hi Serguei, I'll take care of the rename. Can I also put you down for the ProcessTool.java changes, which are officially being reviewed on another thread: http://mail.openjdk.java.net/pipermail/core-libs-dev/2015-June/033892.html thanks, Chris On 6/3/15 1:40 AM, serguei.spitsyn at oracle.com wrote: > Chris, > > It looks good in general. > I'd suggest to rename the folder: > || test/com/sun/jdi/CDSJDITests > to: > test/com/sun/jdi/cds > > There is no need to spell "JDI" as it is already a sub-folder of the > com/sun/jdi > and there is no need to spell "Tests" too as it is in the test repo. > Also, all the folders are normally named in the lower case. > > Thanks, > Serguei > > > On 6/2/15 8:25 PM, Chris Plummer wrote: >> Ok, I'm going to keep this as one webrev, but I did create >> JDK-8081771 for the ProcessTool.java changes: >> >> https://bugs.openjdk.java.net/browse/JDK-8081771 >> >> I will commit ProcessTool.java under JDK-8081771, and the rest of the >> changes under JDK-8054386. Both will then be pushed together. I also >> started a new thread for the review of JDK-8081771: >> >> http://mail.openjdk.java.net/pipermail/hotspot-runtime-dev/2015-June/014930.html >> http://mail.openjdk.java.net/pipermail/core-libs-dev/2015-June/033892.html >> >> thanks, >> >> Chris >> >> On 6/2/15 11:55 AM, Chris Plummer wrote: >>> I'm going to have to separate out the ProcessTool.java changes into >>> a separate changeset (and CR). In the meantime, feel free to review >>> what I have below. The code won't be changing at all when I separate >>> out the ProcessTool.java changes. >>> >>> thanks, >>> >>> Chris >>> >>> On 6/2/15 12:36 AM, Chris Plummer wrote: >>>> [Adding core-libs-dev at openjdk.java.net since this update includes >>>> changes to jdk/test library code] >>>> >>>> Please review the updated webrev: >>>> >>>> Webrev: http://cr.openjdk.java.net/~cjplummer/8054386/webrev.02/ >>>> Bug: https://bugs.openjdk.java.net/browse/JDK-8054386 >>>> >>>> There were concerns about the new hotspot tests referencing jdk >>>> tests. One concern was that if the jdk tests change, they could >>>> break the hotspot tests, and this might initially go undetected. >>>> The other concern is that if the jdk and hotspot tests are placed >>>> in separate test bundles, then it would not be possible to run the >>>> hotspot tests. >>>> >>>> Because of these concerns, I moved the tests that were in >>>> hotspot/test/runtime/CDSJDITests to instead be in >>>> jdk/test/com/sun/jdi/CDSJDITests. There was a slight renaming of >>>> the tests in the process. Also, I had to update the jdk version of >>>> ProcessTool.java to include the createJavaProcessBuilder() variant >>>> that is in the hotspot version of ProcessTool.java. >>>> >>>> Lastly, in CDSJITTest.java I changed: >>>> >>>> OutputAnalyzer output = new OutputAnalyzer(pb.start()); >>>> >>>> to instead be: >>>> >>>> OutputAnalyzer output = ProcessTools.executeProcess(pb); >>>> >>>> I had to do this since the jdk version of the OutputAnalyzer >>>> constructor is not public. The 1st version is what is commonly used >>>> in hostspot tests, and the 2nd version is what is commonly used in >>>> jdk tests. I decided to adopt the jdk way rather than make the >>>> OutputAnalyzer constructors public, although this will probably >>>> happen eventually when the two versions are unified. >>>> >>>> thanks, >>>> >>>> Chris >>>> >>>> >>>> On 5/19/15 7:25 AM, Chris Plummer wrote: >>>>> Hi, >>>>> >>>>> Please review the following changes for allowing java debugging >>>>> when CDS is enabled. >>>>> >>>>> Webrev:http://cr.openjdk.java.net/~cjplummer/8054386/webrev.01/ >>>>> Bug:https://bugs.openjdk.java.net/browse/JDK-8054386 >>>>> >>>>> The VM changes are simple. I removed the check that prevents >>>>> debugging with CDS enabled, and added logic that will map the CDS >>>>> archive RW when debugging is enabled. >>>>> >>>>> The tests are a bit more complex. There are a bunch of existing >>>>> JDI tests for testing debugging support. Rather than start from >>>>> scratch or clone them, I instead just wrote wrapper tests that put >>>>> the relevant JDI test classes in the archive, and then invoke the >>>>> JDI test. I did this for 3 of the JDI tests. If you feel there are >>>>> others that would be good candidates, I'd be happy to add them. >>>>> I'm looking for ones that would result in modification of the RO >>>>> class metadata, such as setting a breakpoint (for which I already >>>>> added two tests). >>>>> >>>>> Testing done: >>>>> -Using JPRT to run the new jtreg tests on all platforms. >>>>> -Using JPRT to run all jtreg runtime tests on linux x86 and x_64. >>>>> -Regular JPRT "-testset hotspot" run >>>>> -Putting the JCK JVMTI tests in the archive and then running them. >>>>> -Putting the nsk jdb, jdwp, jvmti, and jdi tests in the archive >>>>> and then running them. >>>>> -Putting a simple test class in the archive and then setting a >>>>> breakpoint on it using jdb >>>>> >>>>> Some of the above testing resulted in the discovery of bugs that >>>>> still need to be addressed: JDK-8078644, JDK-8078730, and >>>>> JDK-8079181. >>>>> >>>>> I also verified that without the change to map the archive RW, the >>>>> above testing resulted in a SEGV, which is what you would expect >>>>> (and actually want to see to prove that the testing is effective). >>>>> >>>>> thanks, >>>>> >>>>> Chris >>>>> >>>> >>> >> > From serguei.spitsyn at oracle.com Thu Jun 4 00:01:10 2015 From: serguei.spitsyn at oracle.com (serguei.spitsyn at oracle.com) Date: Wed, 03 Jun 2015 17:01:10 -0700 Subject: [9] RFR (XS) JDK-8081771: ProcessTool.createJavaProcessBuilder() needs new addTestVmAndJavaOptions argument In-Reply-To: <556E7263.4050601@oracle.com> References: <556E7263.4050601@oracle.com> Message-ID: <556F9546.80801@oracle.com> It looks good to me. Reviewed all together. Thanks, Serguei Thanks, Serguei On 6/2/15 8:20 PM, Chris Plummer wrote: > Please review the following: > > Webrev: http://cr.openjdk.java.net/~cjplummer/8054386/webrev.02/ > Bug: https://bugs.openjdk.java.net/browse/JDK-8081771 > > This review only concerns the changes to ProcessTool.java. The > CDSJDITests and filemapp.cpp changes will be committed under CR > JDK-8054386, but they rely on this change to ProcessTool.java, so both > CRs will be pushed together. See the following thread for the > JDK-8054386 review: > > http://mail.openjdk.java.net/pipermail/hotspot-runtime-dev/2015-June/014923.html > > > thanks, > > Chris From serguei.spitsyn at oracle.com Thu Jun 4 00:02:54 2015 From: serguei.spitsyn at oracle.com (serguei.spitsyn at oracle.com) Date: Wed, 03 Jun 2015 17:02:54 -0700 Subject: [9] RFR (M) 8054386: Allow Java debugging when CDS is enabled In-Reply-To: <556F8C5C.6070704@oracle.com> References: <555B47EA.3000900@oracle.com> <556D5CFD.1020709@oracle.com> <556DFC33.4060600@oracle.com> <556E73AF.3020606@oracle.com> <556EBD91.6030204@oracle.com> <556F8C5C.6070704@oracle.com> Message-ID: <556F95AE.7020708@oracle.com> Hi Chris, I've replied with a thumbs up on another thread. Thanks, Serguei On 6/3/15 4:23 PM, Chris Plummer wrote: > Hi Serguei, > > I'll take care of the rename. Can I also put you down for the > ProcessTool.java changes, which are officially being reviewed on > another thread: > > http://mail.openjdk.java.net/pipermail/core-libs-dev/2015-June/033892.html > > > thanks, > > Chris > > On 6/3/15 1:40 AM, serguei.spitsyn at oracle.com wrote: >> Chris, >> >> It looks good in general. >> I'd suggest to rename the folder: >> || test/com/sun/jdi/CDSJDITests >> to: >> test/com/sun/jdi/cds >> >> There is no need to spell "JDI" as it is already a sub-folder of the >> com/sun/jdi >> and there is no need to spell "Tests" too as it is in the test repo. >> Also, all the folders are normally named in the lower case. >> >> Thanks, >> Serguei >> >> >> On 6/2/15 8:25 PM, Chris Plummer wrote: >>> Ok, I'm going to keep this as one webrev, but I did create >>> JDK-8081771 for the ProcessTool.java changes: >>> >>> https://bugs.openjdk.java.net/browse/JDK-8081771 >>> >>> I will commit ProcessTool.java under JDK-8081771, and the rest of >>> the changes under JDK-8054386. Both will then be pushed together. I >>> also started a new thread for the review of JDK-8081771: >>> >>> http://mail.openjdk.java.net/pipermail/hotspot-runtime-dev/2015-June/014930.html >>> >>> http://mail.openjdk.java.net/pipermail/core-libs-dev/2015-June/033892.html >>> >>> >>> thanks, >>> >>> Chris >>> >>> On 6/2/15 11:55 AM, Chris Plummer wrote: >>>> I'm going to have to separate out the ProcessTool.java changes into >>>> a separate changeset (and CR). In the meantime, feel free to review >>>> what I have below. The code won't be changing at all when I >>>> separate out the ProcessTool.java changes. >>>> >>>> thanks, >>>> >>>> Chris >>>> >>>> On 6/2/15 12:36 AM, Chris Plummer wrote: >>>>> [Adding core-libs-dev at openjdk.java.net since this update includes >>>>> changes to jdk/test library code] >>>>> >>>>> Please review the updated webrev: >>>>> >>>>> Webrev: http://cr.openjdk.java.net/~cjplummer/8054386/webrev.02/ >>>>> Bug: https://bugs.openjdk.java.net/browse/JDK-8054386 >>>>> >>>>> There were concerns about the new hotspot tests referencing jdk >>>>> tests. One concern was that if the jdk tests change, they could >>>>> break the hotspot tests, and this might initially go undetected. >>>>> The other concern is that if the jdk and hotspot tests are placed >>>>> in separate test bundles, then it would not be possible to run the >>>>> hotspot tests. >>>>> >>>>> Because of these concerns, I moved the tests that were in >>>>> hotspot/test/runtime/CDSJDITests to instead be in >>>>> jdk/test/com/sun/jdi/CDSJDITests. There was a slight renaming of >>>>> the tests in the process. Also, I had to update the jdk version of >>>>> ProcessTool.java to include the createJavaProcessBuilder() variant >>>>> that is in the hotspot version of ProcessTool.java. >>>>> >>>>> Lastly, in CDSJITTest.java I changed: >>>>> >>>>> OutputAnalyzer output = new OutputAnalyzer(pb.start()); >>>>> >>>>> to instead be: >>>>> >>>>> OutputAnalyzer output = ProcessTools.executeProcess(pb); >>>>> >>>>> I had to do this since the jdk version of the OutputAnalyzer >>>>> constructor is not public. The 1st version is what is commonly >>>>> used in hostspot tests, and the 2nd version is what is commonly >>>>> used in jdk tests. I decided to adopt the jdk way rather than make >>>>> the OutputAnalyzer constructors public, although this will >>>>> probably happen eventually when the two versions are unified. >>>>> >>>>> thanks, >>>>> >>>>> Chris >>>>> >>>>> >>>>> On 5/19/15 7:25 AM, Chris Plummer wrote: >>>>>> Hi, >>>>>> >>>>>> Please review the following changes for allowing java debugging >>>>>> when CDS is enabled. >>>>>> >>>>>> Webrev:http://cr.openjdk.java.net/~cjplummer/8054386/webrev.01/ >>>>>> Bug:https://bugs.openjdk.java.net/browse/JDK-8054386 >>>>>> >>>>>> The VM changes are simple. I removed the check that prevents >>>>>> debugging with CDS enabled, and added logic that will map the CDS >>>>>> archive RW when debugging is enabled. >>>>>> >>>>>> The tests are a bit more complex. There are a bunch of existing >>>>>> JDI tests for testing debugging support. Rather than start from >>>>>> scratch or clone them, I instead just wrote wrapper tests that >>>>>> put the relevant JDI test classes in the archive, and then invoke >>>>>> the JDI test. I did this for 3 of the JDI tests. If you feel >>>>>> there are others that would be good candidates, I'd be happy to >>>>>> add them. I'm looking for ones that would result in modification >>>>>> of the RO class metadata, such as setting a breakpoint (for which >>>>>> I already added two tests). >>>>>> >>>>>> Testing done: >>>>>> -Using JPRT to run the new jtreg tests on all platforms. >>>>>> -Using JPRT to run all jtreg runtime tests on linux x86 and x_64. >>>>>> -Regular JPRT "-testset hotspot" run >>>>>> -Putting the JCK JVMTI tests in the archive and then running them. >>>>>> -Putting the nsk jdb, jdwp, jvmti, and jdi tests in the archive >>>>>> and then running them. >>>>>> -Putting a simple test class in the archive and then setting a >>>>>> breakpoint on it using jdb >>>>>> >>>>>> Some of the above testing resulted in the discovery of bugs that >>>>>> still need to be addressed: JDK-8078644, JDK-8078730, and >>>>>> JDK-8079181. >>>>>> >>>>>> I also verified that without the change to map the archive RW, >>>>>> the above testing resulted in a SEGV, which is what you would >>>>>> expect (and actually want to see to prove that the testing is >>>>>> effective). >>>>>> >>>>>> thanks, >>>>>> >>>>>> Chris >>>>>> >>>>> >>>> >>> >> > From brian.goetz at oracle.com Thu Jun 4 00:14:19 2015 From: brian.goetz at oracle.com (Brian Goetz) Date: Wed, 03 Jun 2015 20:14:19 -0400 Subject: Why isn't Object.notify() a synchronized method? In-Reply-To: <5568985C.6050507@CoSoCo.de> References: <55673D97.4010505@CoSoCo.de> <5567420A.9020406@redhat.com> <5568985C.6050507@CoSoCo.de> Message-ID: <556F985B.5010009@oracle.com> The performance issue here is mostly a red herring. The reason notify() is not synchronized has much more to do with correctness; when you "forget" to wrap your notify call with lock acquisition, it is almost always a bug. (The rest of this explanation is probably clearer if you've read JCiP Ch14.) A thread calls notify/notifyAll if it has effected a state change that could cause a waiting thread to become unblocked. Blocking is generally associated with a state predicate ("queue is nonempty", "light is green"), which refers to some state, and that state needs to be guarded by the lock associated with the condition queue. If you've modified the state that underlies the predicate (i.e., you put something in the queue, or switched the light to green), you needed the lock anyway. (Because spurious wakeup is allowed, even the trivial cases like a one-shot "release all threads when timer expires" are better implemented with a boolean state predicate (or better, CountDownLatch) than simply calling notifyAll.) So, while its not out of the question that code that wants to call notify without locking isn't wrong, it probably is wrong, and having the library "conveniently" acquire the lock for you is just brushing the mistake under the rug. On 5/29/2015 12:48 PM, Ulf Zibis wrote: > Thanks for your hint David. That's the only reason I could imagine too. > Can somebody tell something about the cost for recursive lock > acquisition in comparison to the whole call, couldn't it be eliminated > by Hotspot? > > As I recently fell into the trap of forgetting the synchronized block > around a single notifyAll(), I believe, the current situation is just > errorprone. > > Any comments about the Javadoc issue? > > -Ulf > > > Am 28.05.2015 um 18:27 schrieb David M. Lloyd: >> Since most of the time you have to hold the lock anyway for other >> reasons, I think this would generally be an unwelcome change since I >> expect the cost of recursive lock acquisition is nonzero. >> >> On 05/28/2015 11:08 AM, Ulf Zibis wrote: >>> Hi all, >>> >>> in the Javadoc of notify(), notifyAll() and wait(...) I read, that this >>> methods should only be used with synchronisation on it's instance. >>> So I'm wondering, why they don't have the synchronized modifier out of >>> the box in Object class. >>> >>> Also I think, the following note should be moved from wait(long,int) to >>> wait(long): >>> /The current thread must own this object's monitor. The thread releases >>> ownership of this monitor and waits until either of the following two >>> conditions has occurred:// >>> / >>> >>> * /Another thread notifies threads waiting on this object's monitor to >>> wake up either through a >>> call to the notify method or the notifyAll method./ >>> * /The timeout period, specified by timeout milliseconds plus nanos >>> nanoseconds arguments, has >>> elapsed. / >>> >>> >>> >>> Cheers, >>> >>> Ulf >>> >> > From xuelei.fan at oracle.com Thu Jun 4 01:57:53 2015 From: xuelei.fan at oracle.com (Xuelei Fan) Date: Thu, 04 Jun 2015 09:57:53 +0800 Subject: [9] Review request for 8072515: Test Task: Develop new tests for JEP 219: Datagram Transport Layer Security (DTLS) In-Reply-To: References: Message-ID: <556FB0A1.6020900@oracle.com> Looks fine to me. It's nice to keep each line not exceed 80 characters. For example - * @run main/othervm -Dtest.security.protocol=DTLS -Dtest.mode=norm DTLSBufferOverflowUnderflowTest + * @run main/othervm -Dtest.security.protocol=DTLS + * -Dtest.mode=norm DTLSBufferOverflowUnderflowTest Thanks, Xuelei On 6/2/2015 8:15 PM, Konstantin Shefov wrote: > Hello, > > Please review new tests fro DTLS feature for JDK 9: > > bug: https://bugs.openjdk.java.net/browse/JDK-8072515 > webrev: http://cr.openjdk.java.net/~kshefov/8072515/webrev.00/ > > > Thanks > -Konstantin > From david.holmes at oracle.com Thu Jun 4 06:04:45 2015 From: david.holmes at oracle.com (David Holmes) Date: Thu, 04 Jun 2015 16:04:45 +1000 Subject: [9] RFR (XS) JDK-8081771: ProcessTool.createJavaProcessBuilder() needs new addTestVmAndJavaOptions argument In-Reply-To: <556E7263.4050601@oracle.com> References: <556E7263.4050601@oracle.com> Message-ID: <556FEA7D.2040107@oracle.com> Hi Chris, On 3/06/2015 1:20 PM, Chris Plummer wrote: > Please review the following: > > Webrev: http://cr.openjdk.java.net/~cjplummer/8054386/webrev.02/ > Bug: https://bugs.openjdk.java.net/browse/JDK-8081771 > > This review only concerns the changes to ProcessTool.java. The Your new method needs javadoc explaining its usage, and in particular what addTestVmAndJavaOptions actually refers to. Also a comment on why the explicit -cp is added, but on under that arg, would be useful. Thanks, David > CDSJDITests and filemapp.cpp changes will be committed under CR > JDK-8054386, but they rely on this change to ProcessTool.java, so both > CRs will be pushed together. See the following thread for the > JDK-8054386 review: > > http://mail.openjdk.java.net/pipermail/hotspot-runtime-dev/2015-June/014923.html > > > thanks, > > Chris From david.holmes at oracle.com Thu Jun 4 06:29:51 2015 From: david.holmes at oracle.com (David Holmes) Date: Thu, 04 Jun 2015 16:29:51 +1000 Subject: [9] RFR (M) 8054386: Allow Java debugging when CDS is enabled In-Reply-To: <556D5CFD.1020709@oracle.com> References: <555B47EA.3000900@oracle.com> <556D5CFD.1020709@oracle.com> Message-ID: <556FF05F.6060300@oracle.com> Hi Chris, Hotspot change is good. Only a couple of style nits with the tests (where are our Java style guidelines ???). Taking CDSJDITest.java as an example: If you are okay with this line length: 115 public static OutputAnalyzer executeAndLog(ProcessBuilder pb, String logName) throws Exception { then you can remove a number of line breaks in the headers of other functions. (I don't follow the 70-80 char line length dogma ;) ) If you do break a header of a function the { still stays on the same line as the last header component ie: private static void addToClassList(PrintStream ps, String classes[]) throws IOException { not: 139 private static void addToClassList(PrintStream ps, String classes[]) 140 throws IOException 141 { Cheers, David On 2/06/2015 5:36 PM, Chris Plummer wrote: > [Adding core-libs-dev at openjdk.java.net since this update includes > changes to jdk/test library code] > > Please review the updated webrev: > > Webrev: http://cr.openjdk.java.net/~cjplummer/8054386/webrev.02/ > Bug: https://bugs.openjdk.java.net/browse/JDK-8054386 > > There were concerns about the new hotspot tests referencing jdk tests. > One concern was that if the jdk tests change, they could break the > hotspot tests, and this might initially go undetected. The other concern > is that if the jdk and hotspot tests are placed in separate test > bundles, then it would not be possible to run the hotspot tests. > > Because of these concerns, I moved the tests that were in > hotspot/test/runtime/CDSJDITests to instead be in > jdk/test/com/sun/jdi/CDSJDITests. There was a slight renaming of the > tests in the process. Also, I had to update the jdk version of > ProcessTool.java to include the createJavaProcessBuilder() variant that > is in the hotspot version of ProcessTool.java. > > Lastly, in CDSJITTest.java I changed: > > OutputAnalyzer output = new OutputAnalyzer(pb.start()); > > to instead be: > > OutputAnalyzer output = ProcessTools.executeProcess(pb); > > I had to do this since the jdk version of the OutputAnalyzer constructor > is not public. The 1st version is what is commonly used in hostspot > tests, and the 2nd version is what is commonly used in jdk tests. I > decided to adopt the jdk way rather than make the OutputAnalyzer > constructors public, although this will probably happen eventually when > the two versions are unified. > > thanks, > > Chris > > > On 5/19/15 7:25 AM, Chris Plummer wrote: >> Hi, >> >> Please review the following changes for allowing java debugging when >> CDS is enabled. >> >> Webrev:http://cr.openjdk.java.net/~cjplummer/8054386/webrev.01/ >> Bug:https://bugs.openjdk.java.net/browse/JDK-8054386 >> >> The VM changes are simple. I removed the check that prevents debugging >> with CDS enabled, and added logic that will map the CDS archive RW >> when debugging is enabled. >> >> The tests are a bit more complex. There are a bunch of existing JDI >> tests for testing debugging support. Rather than start from scratch or >> clone them, I instead just wrote wrapper tests that put the relevant >> JDI test classes in the archive, and then invoke the JDI test. I did >> this for 3 of the JDI tests. If you feel there are others that would >> be good candidates, I'd be happy to add them. I'm looking for ones >> that would result in modification of the RO class metadata, such as >> setting a breakpoint (for which I already added two tests). >> >> Testing done: >> -Using JPRT to run the new jtreg tests on all platforms. >> -Using JPRT to run all jtreg runtime tests on linux x86 and x_64. >> -Regular JPRT "-testset hotspot" run >> -Putting the JCK JVMTI tests in the archive and then running them. >> -Putting the nsk jdb, jdwp, jvmti, and jdi tests in the archive and >> then running them. >> -Putting a simple test class in the archive and then setting a >> breakpoint on it using jdb >> >> Some of the above testing resulted in the discovery of bugs that still >> need to be addressed: JDK-8078644, JDK-8078730, and JDK-8079181. >> >> I also verified that without the change to map the archive RW, the >> above testing resulted in a SEGV, which is what you would expect (and >> actually want to see to prove that the testing is effective). >> >> thanks, >> >> Chris >> > From david.holmes at oracle.com Thu Jun 4 06:31:51 2015 From: david.holmes at oracle.com (David Holmes) Date: Thu, 04 Jun 2015 16:31:51 +1000 Subject: [9] RFR (XS) JDK-8081771: ProcessTool.createJavaProcessBuilder() needs new addTestVmAndJavaOptions argument In-Reply-To: <556FEA7D.2040107@oracle.com> References: <556E7263.4050601@oracle.com> <556FEA7D.2040107@oracle.com> Message-ID: <556FF0D7.3030207@oracle.com> Typo ... On 4/06/2015 4:04 PM, David Holmes wrote: > Hi Chris, > > On 3/06/2015 1:20 PM, Chris Plummer wrote: >> Please review the following: >> >> Webrev: http://cr.openjdk.java.net/~cjplummer/8054386/webrev.02/ >> Bug: https://bugs.openjdk.java.net/browse/JDK-8081771 >> >> This review only concerns the changes to ProcessTool.java. The > > Your new method needs javadoc explaining its usage, and in particular > what addTestVmAndJavaOptions actually refers to. Also a comment on why > the explicit -cp is added, but on under that arg, would be useful. but only under that arg ... David > Thanks, > David > > > >> CDSJDITests and filemapp.cpp changes will be committed under CR >> JDK-8054386, but they rely on this change to ProcessTool.java, so both >> CRs will be pushed together. See the following thread for the >> JDK-8054386 review: >> >> http://mail.openjdk.java.net/pipermail/hotspot-runtime-dev/2015-June/014923.html >> >> >> >> thanks, >> >> Chris From forax at univ-mlv.fr Thu Jun 4 07:04:54 2015 From: forax at univ-mlv.fr (Remi Forax) Date: Thu, 04 Jun 2015 09:04:54 +0200 Subject: RFR 8071597 Add Stream dropWhile and takeWhile operations In-Reply-To: <79392DA7-6A8E-42AC-9E83-AA7F89D8F2A5@oracle.com> References: <79392DA7-6A8E-42AC-9E83-AA7F89D8F2A5@oracle.com> Message-ID: <556FF896.6090808@univ-mlv.fr> Thinking a little more about dropWhile(), it can be written using filter() more or less like this: default Stream dropWhile(Predicate predicate) { return filter(new Predicate<>() { private boolean noDropAnymore; public boolean test(T t) { return noDropAnymore || (noDropAnymore = !predicate.test(t)); } }); } and i maybe wrong but implementing dropWhile with an op is not better than that in term of perf so for me dropWhile() doesn't pull its own weight. regards, R?mi On 06/02/2015 03:13 PM, Paul Sandoz wrote: > Hi, > > Please review this webrev that adds take/dropWhile operations to streams: > > http://cr.openjdk.java.net/~psandoz/jdk9/JDK-8071597-take-drop-while/webrev/ > > I opted to weight the documentation of the operations towards ordered streams in the first paragraph. That is what makes most sense in terms of usage and what most people will read. Thus i refer to the "longest prefix" in the first paragraph then define what that means in subsequent paragraphs for ordered and unordered streams: > > 482 /** > 483 * Returns a stream consisting of the longest prefix of elements taken from > 484 * this stream that match the given predicate. > 485 * > 486 *

    If this stream is ordered then the prefix is a contiguous sequence of > 487 * elements of this stream. All elements of the sequence match the given > 488 * predicate, the first element of the sequence is the first element > 489 * (if any) of this stream, and the element (if any) immediately following > 490 * the last element of the sequence does not match the given predicate. > 491 * > 492 *

    If this stream is unordered then the prefix is a subset of elements of > 493 * this stream. All elements (if any) of the subset match the given > 494 * predicate. In this case the behavior of this operation is > 495 * nondeterministic; it is free to select any valid subset as the prefix. > 496 * > 497 *

    This is a short-circuiting > 498 * stateful intermediate operation. > 499 * > ... > 528 default Stream takeWhile(Predicate predicate) { > > 537 /** > 538 * Returns a stream consisting of the remaining elements of this stream > 539 * after dropping the longest prefix of elements that match the given > 540 * predicate. > 541 * > 542 *

    If this stream is ordered then the prefix is a contiguous sequence of > 543 * elements of this stream. All elements of the sequence match the given > 544 * predicate, the first element of the sequence is the first element > 545 * (if any) of this stream, and the element (if any) immediately following > 546 * the last element of the sequence does not match the given predicate. > 547 * > 548 *

    If this stream is unordered then the prefix is a subset of elements of > 549 * this stream. All elements (if any) of the subset match the given > 550 * predicate. In this case the behavior of this operation is > 551 * nondeterministic; it is free to select any valid subset as the prefix. > 552 * > ... > 584 default Stream dropWhile(Predicate predicate) { > > > After this has been reviewed i will follow up with a further issue regarding the specification of takeWhile, stateful predicates and cancellation. I avoided such specification here as it's likely to rathole :-) > > Basically the takeWhile operation is implemented such that one can do: > > long t = System.currentTimeMillis(); > List pps = Stream > .generate(() -> BigInteger.probablePrime(1024, ThreadLocalRandom.current())) > .parallel() > .takeWhile(e -> (System.currentTimeMillis() - t) < TimeUnit.SECONDS.toMillis(5)) > .collect(toList()); > > Paul. From paul.sandoz at oracle.com Thu Jun 4 07:37:58 2015 From: paul.sandoz at oracle.com (Paul Sandoz) Date: Thu, 4 Jun 2015 09:37:58 +0200 Subject: RFR 8071597 Add Stream dropWhile and takeWhile operations In-Reply-To: <556FF896.6090808@univ-mlv.fr> References: <79392DA7-6A8E-42AC-9E83-AA7F89D8F2A5@oracle.com> <556FF896.6090808@univ-mlv.fr> Message-ID: <5975E711-D526-495F-8A29-184CEDC59D0E@oracle.com> On Jun 4, 2015, at 9:04 AM, Remi Forax wrote: > Thinking a little more about dropWhile(), > it can be written using filter() more or less like this: > default Stream dropWhile(Predicate predicate) { > return filter(new Predicate<>() { > private boolean noDropAnymore; > public boolean test(T t) { > return noDropAnymore || (noDropAnymore = !predicate.test(t)); > } > }); > } > and i maybe wrong but implementing dropWhile with an op is not better than that in term of perf so for me dropWhile() doesn't pull its own weight. > Try running that in parallel. (Stream.parallel() affects the whole pipeline, so it's not possible to implement the default with sequential().filter(p) where p is a stateful predicate.) Paul. From paul.sandoz at oracle.com Thu Jun 4 07:46:41 2015 From: paul.sandoz at oracle.com (Paul Sandoz) Date: Thu, 4 Jun 2015 09:46:41 +0200 Subject: RFR 8081678: Add Stream returning methods to classes where there currently exist only Enumeration returning methods In-Reply-To: <557A7A1C-611B-4BE3-96AA-803148778848@oracle.com> References: <3093DBE7-AC02-4371-B7EC-12C3A15EA334@oracle.com> <556EDB10.9060708@oracle.com> <2E28B338-0289-4503-B713-B30E4E1704D0@oracle.com> <556F553A.1070901@oracle.com> <557A7A1C-611B-4BE3-96AA-803148778848@oracle.com> Message-ID: On Jun 3, 2015, at 11:00 PM, Chris Hegarty wrote: > Looks good Paul, just a few minor comments. ( looked at all, but *security* and *sql* ) > > NetworkInterface > s/Enumertion/Stream > L127 * will be returned in the Enumeration. However, if the caller has the > > s/an/a > L131 * @return an Stream object with all or a subset of the InetAddresses > L 209 * @return an Stream object with all of the subinterfaces > Thanks, updated. http://cr.openjdk.java.net/~psandoz/jdk9/JDK-8081678-enumeration-and-stream/webrev/src/java.base/share/classes/java/net/NetworkInterface.java.sdiff.html Paul. From konstantin.shefov at oracle.com Thu Jun 4 08:02:50 2015 From: konstantin.shefov at oracle.com (Konstantin Shefov) Date: Thu, 04 Jun 2015 11:02:50 +0300 Subject: [8u-dev] Review request : JDK-8068416: LFGarbageCollectedTest.java fails with OOME: "GC overhead limit exceeded" In-Reply-To: <556F51FF.8000202@oracle.com> References: <556F2C55.5060304@oracle.com> <556F51FF.8000202@oracle.com> Message-ID: <5570062A.9060700@oracle.com> Hi Sean, there should be exactly 440, because 11 cycles is one iteration. -Konstantin On 06/03/2015 10:14 PM, Se?n Coffey wrote: > I bumped into this failure myself today. I think you've got a typo. > 440 should be 40. Looks like a good approach otherwise. > > Regards, > Sean. > > On 03/06/2015 17:33, Konstantin Shefov wrote: >> Hello, >> >> Please review the test bug fix >> https://bugs.openjdk.java.net/browse/JDK-8068416 >> Webrev is http://cr.openjdk.java.net/~kshefov/8068416/webrev.00/ >> Test fails only against JDK 8u and passes against JDK 9. >> >> Fix is to reduce the number of iterations to 40. With that number of >> iterations the test passes on those hosts where it failed before. >> The number of iterations the test start to fail is 65. >> Before the fix the number of iterations was 84. >> >> Thanks >> -Konstantin > From Alan.Bateman at oracle.com Thu Jun 4 08:06:10 2015 From: Alan.Bateman at oracle.com (Alan Bateman) Date: Thu, 04 Jun 2015 09:06:10 +0100 Subject: RFR 8081678: Add Stream returning methods to classes where there currently exist only Enumeration returning methods In-Reply-To: References: <3093DBE7-AC02-4371-B7EC-12C3A15EA334@oracle.com> <556EDB10.9060708@oracle.com> <2E28B338-0289-4503-B713-B30E4E1704D0@oracle.com> <556F553A.1070901@oracle.com> Message-ID: <557006F2.8010701@oracle.com> On 03/06/2015 21:17, Paul Sandoz wrote: > : >> For the tests then @library ../../util/streambootlib doesn't seem right. Is it time to move some infrastructure to make it easier to get at in other parts of the suite? >> > Quite probably, but not with this fix. > > Sure, but we should create a bug to get this done and avoid unexpected dependences between different areas of the test suite. -Alan From forax at univ-mlv.fr Thu Jun 4 08:09:52 2015 From: forax at univ-mlv.fr (Remi Forax) Date: Thu, 04 Jun 2015 10:09:52 +0200 Subject: RFR 8071597 Add Stream dropWhile and takeWhile operations In-Reply-To: <5975E711-D526-495F-8A29-184CEDC59D0E@oracle.com> References: <79392DA7-6A8E-42AC-9E83-AA7F89D8F2A5@oracle.com> <556FF896.6090808@univ-mlv.fr> <5975E711-D526-495F-8A29-184CEDC59D0E@oracle.com> Message-ID: <557007D0.1080700@univ-mlv.fr> On 06/04/2015 09:37 AM, Paul Sandoz wrote: > On Jun 4, 2015, at 9:04 AM, Remi Forax wrote: >> Thinking a little more about dropWhile(), >> it can be written using filter() more or less like this: >> default Stream dropWhile(Predicate predicate) { >> return filter(new Predicate<>() { >> private boolean noDropAnymore; >> public boolean test(T t) { >> return noDropAnymore || (noDropAnymore = !predicate.test(t)); >> } >> }); >> } >> and i maybe wrong but implementing dropWhile with an op is not better than that in term of perf so for me dropWhile() doesn't pull its own weight. >> > Try running that in parallel. I'm not sure it's a good idea if you want good perf, the cost of checking a volatile flag inside the loop for each thread (I don't see how to have something more lightweight) will cost more than just keeping the stream sequential, no ? > > (Stream.parallel() affects the whole pipeline, so it's not possible to implement the default with sequential().filter(p) where p is a stateful predicate.) it's possible, the default should return a proxy in front of the result of sequential().filter(p) that return this if parallel() is called. > > Paul. R?mi From peter.levart at gmail.com Thu Jun 4 08:27:58 2015 From: peter.levart at gmail.com (Peter Levart) Date: Thu, 04 Jun 2015 10:27:58 +0200 Subject: RFR 8071597 Add Stream dropWhile and takeWhile operations In-Reply-To: <556F4D36.5090701@gmail.com> References: <79392DA7-6A8E-42AC-9E83-AA7F89D8F2A5@oracle.com> <556F4D36.5090701@gmail.com> Message-ID: <55700C0E.5070104@gmail.com> On 06/03/2015 08:53 PM, Peter Levart wrote: > Hi Paul, > > This is a usefull addition to Stream API for sequential ordered > streams. But does it have any utility in unordered streams at all? > Wouldn't it be better to just throw an IllegalStateException or > something if the stream is not ordered? I can't imagine currently a > situation where I would need to select an arbitrary sub-set of the > stream where all elements match a predicate and I would not be given > any guarantee about how big a subset I will get - the same with a > stream where an arbitrary sub-set of elements matching predicate is > taken away. If I wanted something similar for unordered stream, I > would rather say: "Give me at most N elements of the stream matching > predicate" for example - and that can already be achieved by > .filter().limit(); > > So any use of [take|drop]While on unordered stream would most probably > be unintended and consequently could be considered a bug, don't you think? Ok, I take back. I was only thinking in terms of the [take|drop]While Predicate being a function of the stream element. Now that I see an example where it is a function of time, I can understand it's utility even for unordered streams. Regards, Peter > Regards, Peter > > On 06/02/2015 03:13 PM, Paul Sandoz wrote: >> Hi, >> >> Please review this webrev that adds take/dropWhile operations to streams: >> >> http://cr.openjdk.java.net/~psandoz/jdk9/JDK-8071597-take-drop-while/webrev/ >> >> I opted to weight the documentation of the operations towards ordered streams in the first paragraph. That is what makes most sense in terms of usage and what most people will read. Thus i refer to the "longest prefix" in the first paragraph then define what that means in subsequent paragraphs for ordered and unordered streams: >> >> 482 /** >> 483 * Returns a stream consisting of the longest prefix of elements taken from >> 484 * this stream that match the given predicate. >> 485 * >> 486 *

    If this stream is ordered then the prefix is a contiguous sequence of >> 487 * elements of this stream. All elements of the sequence match the given >> 488 * predicate, the first element of the sequence is the first element >> 489 * (if any) of this stream, and the element (if any) immediately following >> 490 * the last element of the sequence does not match the given predicate. >> 491 * >> 492 *

    If this stream is unordered then the prefix is a subset of elements of >> 493 * this stream. All elements (if any) of the subset match the given >> 494 * predicate. In this case the behavior of this operation is >> 495 * nondeterministic; it is free to select any valid subset as the prefix. >> 496 * >> 497 *

    This is a short-circuiting >> 498 * stateful intermediate operation. >> 499 * >> ... >> 528 default Stream takeWhile(Predicate predicate) { >> >> 537 /** >> 538 * Returns a stream consisting of the remaining elements of this stream >> 539 * after dropping the longest prefix of elements that match the given >> 540 * predicate. >> 541 * >> 542 *

    If this stream is ordered then the prefix is a contiguous sequence of >> 543 * elements of this stream. All elements of the sequence match the given >> 544 * predicate, the first element of the sequence is the first element >> 545 * (if any) of this stream, and the element (if any) immediately following >> 546 * the last element of the sequence does not match the given predicate. >> 547 * >> 548 *

    If this stream is unordered then the prefix is a subset of elements of >> 549 * this stream. All elements (if any) of the subset match the given >> 550 * predicate. In this case the behavior of this operation is >> 551 * nondeterministic; it is free to select any valid subset as the prefix. >> 552 * >> ... >> 584 default Stream dropWhile(Predicate predicate) { >> >> >> After this has been reviewed i will follow up with a further issue regarding the specification of takeWhile, stateful predicates and cancellation. I avoided such specification here as it's likely to rathole :-) >> >> Basically the takeWhile operation is implemented such that one can do: >> >> long t = System.currentTimeMillis(); >> List pps = Stream >> .generate(() -> BigInteger.probablePrime(1024, ThreadLocalRandom.current())) >> .parallel() >> .takeWhile(e -> (System.currentTimeMillis() - t) < TimeUnit.SECONDS.toMillis(5)) >> .collect(toList()); >> >> Paul. > From paul.sandoz at oracle.com Thu Jun 4 08:34:18 2015 From: paul.sandoz at oracle.com (Paul Sandoz) Date: Thu, 4 Jun 2015 10:34:18 +0200 Subject: RFR 8071597 Add Stream dropWhile and takeWhile operations In-Reply-To: <557007D0.1080700@univ-mlv.fr> References: <79392DA7-6A8E-42AC-9E83-AA7F89D8F2A5@oracle.com> <556FF896.6090808@univ-mlv.fr> <5975E711-D526-495F-8A29-184CEDC59D0E@oracle.com> <557007D0.1080700@univ-mlv.fr> Message-ID: <123E29BA-B40F-423E-BAD0-6D3D0F80962C@oracle.com> On Jun 4, 2015, at 10:09 AM, Remi Forax wrote: > On 06/04/2015 09:37 AM, Paul Sandoz wrote: >> On Jun 4, 2015, at 9:04 AM, Remi Forax wrote: >>> Thinking a little more about dropWhile(), >>> it can be written using filter() more or less like this: >>> default Stream dropWhile(Predicate predicate) { >>> return filter(new Predicate<>() { >>> private boolean noDropAnymore; >>> public boolean test(T t) { >>> return noDropAnymore || (noDropAnymore = !predicate.test(t)); >>> } >>> }); >>> } >>> and i maybe wrong but implementing dropWhile with an op is not better than that in term of perf so for me dropWhile() doesn't pull its own weight. >>> >> Try running that in parallel. > And try with an ordered parallel stream and see if it produces the same results as that produced sequentially. > I'm not sure it's a good idea if you want good perf, the cost of checking a volatile flag inside the loop for each thread (I don't see how to have something more lightweight) will cost more than just keeping the stream sequential, no ? > You are imposing a restriction that this should *only* run sequentially. Stateful ops such as take/drop and limit/skip have to be used with care when executed in parallel. That does not mean we should rule it out (however, it's reasonable for default implementations of take/drop to not split, which is what is currently implemented more so for code-sharing). In general parallel streams should be used with care, we keep saying don't sprinkle parallel on your streams like pixie dust and expect it to work magic :-) >> >> (Stream.parallel() affects the whole pipeline, so it's not possible to implement the default with sequential().filter(p) where p is a stateful predicate.) > > it's possible, > the default should return a proxy in front of the result of sequential().filter(p) that return this if parallel() is called. > A sequential slight of hand. Ugh :-) Paul. From paul.sandoz at oracle.com Thu Jun 4 08:34:58 2015 From: paul.sandoz at oracle.com (Paul Sandoz) Date: Thu, 4 Jun 2015 10:34:58 +0200 Subject: RFR 8071597 Add Stream dropWhile and takeWhile operations In-Reply-To: <55700C0E.5070104@gmail.com> References: <79392DA7-6A8E-42AC-9E83-AA7F89D8F2A5@oracle.com> <556F4D36.5090701@gmail.com> <55700C0E.5070104@gmail.com> Message-ID: <384246E5-3B6C-4925-B07F-5130D2B8A923@oracle.com> On Jun 4, 2015, at 10:27 AM, Peter Levart wrote: > > On 06/03/2015 08:53 PM, Peter Levart wrote: >> Hi Paul, >> >> This is a usefull addition to Stream API for sequential ordered streams. But does it have any utility in unordered streams at all? Wouldn't it be better to just throw an IllegalStateException or something if the stream is not ordered? I can't imagine currently a situation where I would need to select an arbitrary sub-set of the stream where all elements match a predicate and I would not be given any guarantee about how big a subset I will get - the same with a stream where an arbitrary sub-set of elements matching predicate is taken away. If I wanted something similar for unordered stream, I would rather say: "Give me at most N elements of the stream matching predicate" for example - and that can already be achieved by .filter().limit(); >> >> So any use of [take|drop]While on unordered stream would most probably be unintended and consequently could be considered a bug, don't you think? > > Ok, I take back. I was only thinking in terms of the [take|drop]While Predicate being a function of the stream element. Now that I see an example where it is a function of time, I can understand it's utility even for unordered streams. > Right, i was about to reply pointing out that aspect. This is a tricky area, i expect it will keep some people rather busy on stackoverflow :-) Paul. From konstantin.shefov at oracle.com Thu Jun 4 09:10:28 2015 From: konstantin.shefov at oracle.com (Konstantin Shefov) Date: Thu, 04 Jun 2015 12:10:28 +0300 Subject: [8u-dev] Review request : JDK-8068416: LFGarbageCollectedTest.java fails with OOME: "GC overhead limit exceeded" In-Reply-To: <556F67F0.1040504@oracle.com> References: <556F2C55.5060304@oracle.com> <556F51FF.8000202@oracle.com> <556F67F0.1040504@oracle.com> Message-ID: <55701604.6010408@oracle.com> Igor, It seems I have given you wrong information. This test fails with OOME against JDK 9 also, I managed to reproduce the failure now. It was hard to reproduce it because of randomness, I need to rerun the test 50 times. Although the test seems to fail with OOME more often against JDK 8u, but I think it is just factor of randomness in the test. So I do not think it is a product bug then. -Konstantin On 06/03/2015 11:47 PM, Igor Ignatyev wrote: > Konstantin, > > do you have an explanation why the test passes on jdk 9? > from my point of view, it indicates there is a product bug in 8u which > should be fixed and your fix just hides it. > > Igor > > On 06/03/2015 10:14 PM, Se?n Coffey wrote: >> I bumped into this failure myself today. I think you've got a typo. >> 440 should be 40. Looks like a good approach otherwise. >> >> Regards, >> Sean. >> >> On 03/06/2015 17:33, Konstantin Shefov wrote: >>> Hello, >>> >>> Please review the test bug fix >>> https://bugs.openjdk.java.net/browse/JDK-8068416 >>> Webrev is http://cr.openjdk.java.net/~kshefov/8068416/webrev.00/ >>> Test fails only against JDK 8u and passes against JDK 9. >>> >>> Fix is to reduce the number of iterations to 40. With that number of >>> iterations the test passes on those hosts where it failed before. >>> The number of iterations the test start to fail is 65. >>> Before the fix the number of iterations was 84. >>> >>> Thanks >>> -Konstantin >> From igor.ignatyev at oracle.com Thu Jun 4 09:15:10 2015 From: igor.ignatyev at oracle.com (Igor Ignatyev) Date: Thu, 04 Jun 2015 12:15:10 +0300 Subject: [8u-dev] Review request : JDK-8068416: LFGarbageCollectedTest.java fails with OOME: "GC overhead limit exceeded" In-Reply-To: <55701604.6010408@oracle.com> References: <556F2C55.5060304@oracle.com> <556F51FF.8000202@oracle.com> <556F67F0.1040504@oracle.com> <55701604.6010408@oracle.com> Message-ID: <5570171E.4000204@oracle.com> Konstantin, could you please run the test on JDK 9 w/ the seed values which lead to failures on JDK 8u? and please update the bug w/ gotten information. Thanks, Igor On 06/04/2015 12:10 PM, Konstantin Shefov wrote: > Igor, > > It seems I have given you wrong information. This test fails with OOME > against JDK 9 also, I managed to reproduce the failure now. > It was hard to reproduce it because of randomness, I need to rerun the > test 50 times. Although the test seems to fail with OOME more often > against JDK 8u, but I think it is just factor of randomness in the test. > > So I do not think it is a product bug then. > > -Konstantin > > On 06/03/2015 11:47 PM, Igor Ignatyev wrote:try to reproduce failres >> Konstantin, >> >> do you have an explanation why the test passes on jdk 9? >> from my point of view, it indicates there is a product bug in 8u which >> should be fixed and your fix just hides it. >> >> Igor >> >> On 06/03/2015 10:14 PM, Se?n Coffey wrote: >>> I bumped into this failure myself today. I think you've got a typo. >>> 440 should be 40. Looks like a good approach otherwise. >>> >>> Regards, >>> Sean. >>> >>> On 03/06/2015 17:33, Konstantin Shefov wrote: >>>> Hello, >>>> >>>> Please review the test bug fix >>>> https://bugs.openjdk.java.net/browse/JDK-8068416 >>>> Webrev is http://cr.openjdk.java.net/~kshefov/8068416/webrev.00/ >>>> Test fails only against JDK 8u and passes against JDK 9. >>>> >>>> Fix is to reduce the number of iterations to 40. With that number of >>>> iterations the test passes on those hosts where it failed before. >>>> The number of iterations the test start to fail is 65. >>>> Before the fix the number of iterations was 84. >>>> >>>> Thanks >>>> -Konstantin >>> > From konstantin.shefov at oracle.com Thu Jun 4 09:49:22 2015 From: konstantin.shefov at oracle.com (Konstantin Shefov) Date: Thu, 04 Jun 2015 12:49:22 +0300 Subject: [8u-dev] Review request : JDK-8068416: LFGarbageCollectedTest.java fails with OOME: "GC overhead limit exceeded" In-Reply-To: <5570171E.4000204@oracle.com> References: <556F2C55.5060304@oracle.com> <556F51FF.8000202@oracle.com> <556F67F0.1040504@oracle.com> <55701604.6010408@oracle.com> <5570171E.4000204@oracle.com> Message-ID: <55701F22.3070309@oracle.com> Igor, I have added comment to https://bugs.openjdk.java.net/browse/JDK-8068416 Tried 10 seeds, for all of them test fails with OOME for both 8u60 and 9. -Konstantin On 06/04/2015 12:15 PM, Igor Ignatyev wrote: > Konstantin, > > could you please run the test on JDK 9 w/ the seed values which lead > to failures on JDK 8u? > and please update the bug w/ gotten information. > > Thanks, > Igor > > On 06/04/2015 12:10 PM, Konstantin Shefov wrote: >> Igor, >> >> It seems I have given you wrong information. This test fails with OOME >> against JDK 9 also, I managed to reproduce the failure now. >> It was hard to reproduce it because of randomness, I need to rerun the >> test 50 times. Although the test seems to fail with OOME more often >> against JDK 8u, but I think it is just factor of randomness in the test. >> >> So I do not think it is a product bug then. >> >> -Konstantin >> >> On 06/03/2015 11:47 PM, Igor Ignatyev wrote:try to reproduce failres >>> Konstantin, >>> >>> do you have an explanation why the test passes on jdk 9? >>> from my point of view, it indicates there is a product bug in 8u which >>> should be fixed and your fix just hides it. >>> >>> Igor >>> >>> On 06/03/2015 10:14 PM, Se?n Coffey wrote: >>>> I bumped into this failure myself today. I think you've got a typo. >>>> 440 should be 40. Looks like a good approach otherwise. >>>> >>>> Regards, >>>> Sean. >>>> >>>> On 03/06/2015 17:33, Konstantin Shefov wrote: >>>>> Hello, >>>>> >>>>> Please review the test bug fix >>>>> https://bugs.openjdk.java.net/browse/JDK-8068416 >>>>> Webrev is http://cr.openjdk.java.net/~kshefov/8068416/webrev.00/ >>>>> Test fails only against JDK 8u and passes against JDK 9. >>>>> >>>>> Fix is to reduce the number of iterations to 40. With that number of >>>>> iterations the test passes on those hosts where it failed before. >>>>> The number of iterations the test start to fail is 65. >>>>> Before the fix the number of iterations was 84. >>>>> >>>>> Thanks >>>>> -Konstantin >>>> >> From vladimir.x.ivanov at oracle.com Thu Jun 4 09:50:36 2015 From: vladimir.x.ivanov at oracle.com (Vladimir Ivanov) Date: Thu, 04 Jun 2015 12:50:36 +0300 Subject: [8u-dev] Review request : JDK-8068416: LFGarbageCollectedTest.java fails with OOME: "GC overhead limit exceeded" In-Reply-To: <55701604.6010408@oracle.com> References: <556F2C55.5060304@oracle.com> <556F51FF.8000202@oracle.com> <556F67F0.1040504@oracle.com> <55701604.6010408@oracle.com> Message-ID: <55701F6C.8020006@oracle.com> Konstantin, Have you looked into the heap dump to understand why the test provokes an OOM? Limiting test iterations is counter-productive, because it defeats the purpose of the test. Probably, the failure is caused by BMHs which aren't collected (see JDK-8078602 [1]). In that case I'd prefer the test to be excluded until BMHs are converted to VM anonymous classes. Best regards, Vladimir Ivanov [1] https://bugs.openjdk.java.net/browse/JDK-8078602 On 6/4/15 12:10 PM, Konstantin Shefov wrote: > Igor, > > It seems I have given you wrong information. This test fails with OOME > against JDK 9 also, I managed to reproduce the failure now. > It was hard to reproduce it because of randomness, I need to rerun the > test 50 times. Although the test seems to fail with OOME more often > against JDK 8u, but I think it is just factor of randomness in the test. > > So I do not think it is a product bug then. > > -Konstantin > > On 06/03/2015 11:47 PM, Igor Ignatyev wrote: >> Konstantin, >> >> do you have an explanation why the test passes on jdk 9? >> from my point of view, it indicates there is a product bug in 8u which >> should be fixed and your fix just hides it. >> >> Igor >> >> On 06/03/2015 10:14 PM, Se?n Coffey wrote: >>> I bumped into this failure myself today. I think you've got a typo. >>> 440 should be 40. Looks like a good approach otherwise. >>> >>> Regards, >>> Sean. >>> >>> On 03/06/2015 17:33, Konstantin Shefov wrote: >>>> Hello, >>>> >>>> Please review the test bug fix >>>> https://bugs.openjdk.java.net/browse/JDK-8068416 >>>> Webrev is http://cr.openjdk.java.net/~kshefov/8068416/webrev.00/ >>>> Test fails only against JDK 8u and passes against JDK 9. >>>> >>>> Fix is to reduce the number of iterations to 40. With that number of >>>> iterations the test passes on those hosts where it failed before. >>>> The number of iterations the test start to fail is 65. >>>> Before the fix the number of iterations was 84. >>>> >>>> Thanks >>>> -Konstantin >>> > From srikanth.adayapalam at oracle.com Thu Jun 4 10:16:55 2015 From: srikanth.adayapalam at oracle.com (Srikanth) Date: Thu, 04 Jun 2015 15:46:55 +0530 Subject: RFR JDK-8054304: Clarify treatment of bounds in j.l.r.Annotated{WildcardType, TypeVariable} In-Reply-To: <5513AD47.2080407@oracle.com> References: <5513AD47.2080407@oracle.com> Message-ID: <55702597.6050907@oracle.com> Hello, Please review this fix for https://bugs.openjdk.java.net/browse/JDK-8054304 (twin for conformance test failures: https://bugs.openjdk.java.net/browse/JDK-8058595) JBS: https://bugs.openjdk.java.net/browse/JDK-8054304 Webrev: http://cr.openjdk.java.net/~sadayapalam/JDK-8054304/webrev.00/ CCC: http://ccc.us.oracle.com/8054304 While as outlined in https://bugs.openjdk.java.net/browse/JDK-8054304, the javadoc of several methods is adjusted, only in one case viz AnnotatedWildcardType.getAnnotatedUpperBounds() there is a behaviour change. All other cases involve javadoc change that explicitly spell out current & correct behaviour. The patch passes Tier 1 & Tier 2 tests for JDK (i.e identical results with and without) Our plan is to fix this for JDK9 and backport to 8u60. I don't know at the moment if there will be objections for the backport from a process p.o.v as this calls for a specification change. Thanks! Srikanth. From konstantin.shefov at oracle.com Thu Jun 4 10:50:46 2015 From: konstantin.shefov at oracle.com (Konstantin Shefov) Date: Thu, 04 Jun 2015 13:50:46 +0300 Subject: [8u-dev] Review request : JDK-8068416: LFGarbageCollectedTest.java fails with OOME: "GC overhead limit exceeded" In-Reply-To: <55701F6C.8020006@oracle.com> References: <556F2C55.5060304@oracle.com> <556F51FF.8000202@oracle.com> <556F67F0.1040504@oracle.com> <55701604.6010408@oracle.com> <55701F6C.8020006@oracle.com> Message-ID: <55702D86.3060306@oracle.com> Vladimir In all cases when OOME happens the test operates with BoundMethodHandle$SpeciesData class, so it indeed may be caused by JDK-8078602. But is it a good idea of excluding the test? LFGarbageCollectedTest.java fails not every time and may catch other product issues if they happen. We can just leave it unchanged and close JDK-8068416 as a duplicate of JDK-8078602. -Konstantin On 06/04/2015 12:50 PM, Vladimir Ivanov wrote: > Konstantin, > > Have you looked into the heap dump to understand why the test provokes > an OOM? Limiting test iterations is counter-productive, because it > defeats the purpose of the test. > > Probably, the failure is caused by BMHs which aren't collected (see > JDK-8078602 [1]). In that case I'd prefer the test to be excluded > until BMHs are converted to VM anonymous classes. > > Best regards, > Vladimir Ivanov > > [1] https://bugs.openjdk.java.net/browse/JDK-8078602 > > On 6/4/15 12:10 PM, Konstantin Shefov wrote: >> Igor, >> >> It seems I have given you wrong information. This test fails with OOME >> against JDK 9 also, I managed to reproduce the failure now. >> It was hard to reproduce it because of randomness, I need to rerun the >> test 50 times. Although the test seems to fail with OOME more often >> against JDK 8u, but I think it is just factor of randomness in the test. >> >> So I do not think it is a product bug then. >> >> -Konstantin >> >> On 06/03/2015 11:47 PM, Igor Ignatyev wrote: >>> Konstantin, >>> >>> do you have an explanation why the test passes on jdk 9? >>> from my point of view, it indicates there is a product bug in 8u which >>> should be fixed and your fix just hides it. >>> >>> Igor >>> >>> On 06/03/2015 10:14 PM, Se?n Coffey wrote: >>>> I bumped into this failure myself today. I think you've got a typo. >>>> 440 should be 40. Looks like a good approach otherwise. >>>> >>>> Regards, >>>> Sean. >>>> >>>> On 03/06/2015 17:33, Konstantin Shefov wrote: >>>>> Hello, >>>>> >>>>> Please review the test bug fix >>>>> https://bugs.openjdk.java.net/browse/JDK-8068416 >>>>> Webrev is http://cr.openjdk.java.net/~kshefov/8068416/webrev.00/ >>>>> Test fails only against JDK 8u and passes against JDK 9. >>>>> >>>>> Fix is to reduce the number of iterations to 40. With that number of >>>>> iterations the test passes on those hosts where it failed before. >>>>> The number of iterations the test start to fail is 65. >>>>> Before the fix the number of iterations was 84. >>>>> >>>>> Thanks >>>>> -Konstantin >>>> >> From chris.hegarty at oracle.com Thu Jun 4 10:54:39 2015 From: chris.hegarty at oracle.com (Chris Hegarty) Date: Thu, 4 Jun 2015 11:54:39 +0100 Subject: RFR [9]: Better failure output for test/java/util/Arrays/ParallelPrefix.java Message-ID: This is a review request to add better test failure output to jdk/test/java/util/Arrays/ParallelPrefix.java This test has been seen to fail occasionally on recent JDK 9 builds. It fails with a message similar to: ... test ParallelPrefix.testParallelPrefixForInt([I at 47a7a8ef, 1365, 2730, ParallelPrefix$$Lambda$56/1605520928 at 106dbc48): success test ParallelPrefix.testParallelPrefixForInt([I at 789e465e, 1365, 2731, ParallelPrefix$$Lambda$55/1326195546 at 2c8625ae): failure java.lang.AssertionError: expected [[I at 7b703d4a] but found [[I at 73625cb4] at org.testng.Assert.fail(Assert.java:94) at org.testng.Assert.failNotEquals(Assert.java:494) at org.testng.Assert.assertArrayEquals(Assert.java:144) at org.testng.Assert.assertEquals(Assert.java:117) at org.testng.Assert.assertEquals(Assert.java:165) at ParallelPrefix.testParallelPrefixForInt(ParallelPrefix.java:124) ? The output is not very helpful. Testng, or at least the version that we have in use in jtreg, does not provide pretty failure messages when dealing with arrays. It is trivial to add these to the test, see below. Note: I used the 3-args version of assertEquals as it will trigger the inclusion of the index of the array where the values differ. Hopefully, when we see this test fail again it can produce some meaningful output. -Chris. diff --git a/test/java/util/Arrays/ParallelPrefix.java b/test/java/util/Arrays/ParallelPrefix.java --- a/test/java/util/Arrays/ParallelPrefix.java +++ b/test/java/util/Arrays/ParallelPrefix.java @@ -39,6 +39,7 @@ import static org.testng.Assert.*; import org.testng.annotations.DataProvider; import org.testng.annotations.Test; +import static java.lang.System.out; public class ParallelPrefix { //Array size less than MIN_PARTITION @@ -121,11 +122,11 @@ int[] parallelResult = data.clone(); Arrays.parallelPrefix(parallelResult, fromIndex, toIndex, op); - assertEquals(parallelResult, sequentialResult); + assertArraysEqual(parallelResult, sequentialResult); int[] parallelRangeResult = Arrays.copyOfRange(data, fromIndex, toIndex); Arrays.parallelPrefix(parallelRangeResult, op); - assertEquals(parallelRangeResult, Arrays.copyOfRange(sequentialResult, fromIndex, toIndex)); + assertArraysEqual(parallelRangeResult, Arrays.copyOfRange(sequentialResult, fromIndex, toIndex)); } @Test(dataProvider="longSet") @@ -137,11 +138,11 @@ long[] parallelResult = data.clone(); Arrays.parallelPrefix(parallelResult, fromIndex, toIndex, op); - assertEquals(parallelResult, sequentialResult); + assertArraysEqual(parallelResult, sequentialResult); long[] parallelRangeResult = Arrays.copyOfRange(data, fromIndex, toIndex); Arrays.parallelPrefix(parallelRangeResult, op); - assertEquals(parallelRangeResult, Arrays.copyOfRange(sequentialResult, fromIndex, toIndex)); + assertArraysEqual(parallelRangeResult, Arrays.copyOfRange(sequentialResult, fromIndex, toIndex)); } @Test(dataProvider="doubleSet") @@ -153,11 +154,11 @@ double[] parallelResult = data.clone(); Arrays.parallelPrefix(parallelResult, fromIndex, toIndex, op); - assertEquals(parallelResult, sequentialResult); + assertArraysEqual(parallelResult, sequentialResult); double[] parallelRangeResult = Arrays.copyOfRange(data, fromIndex, toIndex); Arrays.parallelPrefix(parallelRangeResult, op); - assertEquals(parallelRangeResult, Arrays.copyOfRange(sequentialResult, fromIndex, toIndex)); + assertArraysEqual(parallelRangeResult, Arrays.copyOfRange(sequentialResult, fromIndex, toIndex)); } @Test(dataProvider="stringSet") @@ -169,11 +170,11 @@ String[] parallelResult = data.clone(); Arrays.parallelPrefix(parallelResult, fromIndex, toIndex, op); - assertEquals(parallelResult, sequentialResult); + assertArraysEqual(parallelResult, sequentialResult); String[] parallelRangeResult = Arrays.copyOfRange(data, fromIndex, toIndex); Arrays.parallelPrefix(parallelRangeResult, op); - assertEquals(parallelRangeResult, Arrays.copyOfRange(sequentialResult, fromIndex, toIndex)); + assertArraysEqual(parallelRangeResult, Arrays.copyOfRange(sequentialResult, fromIndex, toIndex)); } @Test @@ -293,5 +294,41 @@ public static void assertInstance(Object actual, Class expected, String message) { assertTrue(expected.isInstance(actual), message); } + + static void assertArraysEqual(int[] actual, int[] expected) { + try { + assertEquals(actual, expected, ""); + } catch (AssertionError x) { + out.printf("Expected:%s, actual:%s%n", Arrays.toString(expected), Arrays.toString(actual)); + throw x; + } + } + + static void assertArraysEqual(long[] actual, long[] expected) { + try { + assertEquals(actual, expected, ""); + } catch (AssertionError x) { + out.printf("Expected:%s, actual:%s%n", Arrays.toString(expected), Arrays.toString(actual)); + throw x; + } + } + + static void assertArraysEqual(double[] actual, double[] expected) { + try { + assertEquals(actual, expected, ""); + } catch (AssertionError x) { + out.printf("Expected:%s, actual:%s%n", Arrays.toString(expected), Arrays.toString(actual)); + throw x; + } + } + + static void assertArraysEqual(String[] actual, String[] expected) { + try { + assertEquals(actual, expected, ""); + } catch (AssertionError x) { + out.printf("Expected:%s, actual:%s%n", Arrays.toString(expected), Arrays.toString(actual)); + throw x; + } + } } From vladimir.x.ivanov at oracle.com Thu Jun 4 11:19:54 2015 From: vladimir.x.ivanov at oracle.com (Vladimir Ivanov) Date: Thu, 04 Jun 2015 14:19:54 +0300 Subject: [8u-dev] Review request : JDK-8068416: LFGarbageCollectedTest.java fails with OOME: "GC overhead limit exceeded" In-Reply-To: <55702D86.3060306@oracle.com> References: <556F2C55.5060304@oracle.com> <556F51FF.8000202@oracle.com> <556F67F0.1040504@oracle.com> <55701604.6010408@oracle.com> <55701F6C.8020006@oracle.com> <55702D86.3060306@oracle.com> Message-ID: <5570345A.2070905@oracle.com> Konstantin, > In all cases when OOME happens the test operates with > BoundMethodHandle$SpeciesData class, so it indeed may be caused by > JDK-8078602. It's not necessarily an evidence. Most of method handles are BMHs. So, I'd suggest to inspect the heap dump. > But is it a good idea of excluding the test? LFGarbageCollectedTest.java > fails not every time and may catch other product issues if they happen. > We can just leave it unchanged and close JDK-8068416 as a duplicate of > JDK-8078602. No, intermittent test failures are not acceptable. Unless there's a way to limit produced BMHs count, the test should be excluded until JDK-8078602 is fixed. Best regards, Vladimir Ivanov > > -Konstantin > > On 06/04/2015 12:50 PM, Vladimir Ivanov wrote: >> Konstantin, >> >> Have you looked into the heap dump to understand why the test provokes >> an OOM? Limiting test iterations is counter-productive, because it >> defeats the purpose of the test. >> >> Probably, the failure is caused by BMHs which aren't collected (see >> JDK-8078602 [1]). In that case I'd prefer the test to be excluded >> until BMHs are converted to VM anonymous classes. >> >> Best regards, >> Vladimir Ivanov >> >> [1] https://bugs.openjdk.java.net/browse/JDK-8078602 >> >> On 6/4/15 12:10 PM, Konstantin Shefov wrote: >>> Igor, >>> >>> It seems I have given you wrong information. This test fails with OOME >>> against JDK 9 also, I managed to reproduce the failure now. >>> It was hard to reproduce it because of randomness, I need to rerun the >>> test 50 times. Although the test seems to fail with OOME more often >>> against JDK 8u, but I think it is just factor of randomness in the test. >>> >>> So I do not think it is a product bug then. >>> >>> -Konstantin >>> >>> On 06/03/2015 11:47 PM, Igor Ignatyev wrote: >>>> Konstantin, >>>> >>>> do you have an explanation why the test passes on jdk 9? >>>> from my point of view, it indicates there is a product bug in 8u which >>>> should be fixed and your fix just hides it. >>>> >>>> Igor >>>> >>>> On 06/03/2015 10:14 PM, Se?n Coffey wrote: >>>>> I bumped into this failure myself today. I think you've got a typo. >>>>> 440 should be 40. Looks like a good approach otherwise. >>>>> >>>>> Regards, >>>>> Sean. >>>>> >>>>> On 03/06/2015 17:33, Konstantin Shefov wrote: >>>>>> Hello, >>>>>> >>>>>> Please review the test bug fix >>>>>> https://bugs.openjdk.java.net/browse/JDK-8068416 >>>>>> Webrev is http://cr.openjdk.java.net/~kshefov/8068416/webrev.00/ >>>>>> Test fails only against JDK 8u and passes against JDK 9. >>>>>> >>>>>> Fix is to reduce the number of iterations to 40. With that number of >>>>>> iterations the test passes on those hosts where it failed before. >>>>>> The number of iterations the test start to fail is 65. >>>>>> Before the fix the number of iterations was 84. >>>>>> >>>>>> Thanks >>>>>> -Konstantin >>>>> >>> > From konstantin.shefov at oracle.com Thu Jun 4 11:24:59 2015 From: konstantin.shefov at oracle.com (Konstantin Shefov) Date: Thu, 04 Jun 2015 14:24:59 +0300 Subject: [8u-dev] Review request : JDK-8062904: TEST_BUG: Tests java/lang/invoke/LFCaching fail when run with -Xcomp option In-Reply-To: <556ECC99.2000702@oracle.com> References: <5565D9BB.60003@oracle.com> <5565F6D5.7030900@oracle.com> <5566D9D8.4090401@oracle.com> <55675CCB.8010506@oracle.com> <5568281B.7040108@oracle.com> <55685264.20001@oracle.com> <556C86C6.2000409@oracle.com> <556DFB1C.7010301@oracle.com> <556ECC99.2000702@oracle.com> Message-ID: <5570358B.1010105@oracle.com> Vladimir, I have verified the failure again. Even if I use random seed that causes failure against JDK 8u60, the test passes against JDK 9 with this seed. So JDK-8062904 ([1]) is really reproduced only against JDK 8u, not JDK 9. And failure happens only with -Xcomp option. I think it is a product failure. [1] https://bugs.openjdk.java.net/browse/JDK-8062904 -Konstantin On 06/03/2015 12:44 PM, Konstantin Shefov wrote: > Hi Vladimir > > On 02.06.2015 21:51, Vladimir Ivanov wrote: >> Konstantin, >> >>> It seems we have only this bug that manifests the problem. As I >>> understand, this is a product issue, not test. >> My question was about the symptoms - how the test can fail. If the >> test ignores NSME & VME exceptions, will it always pass w.r.t. code >> cache overflows? >> >> Code cache overflow is definitely a JVM problem, but we don't plan to >> address it in the near future. >> >> So, either the test should be excluded or adjusted to be tolerant to >> code cache overflows. > > The test has the code cache overflow failure only when run with > "-Xcomp", all other failures has been fixed by > https://bugs.openjdk.java.net/browse/JDK-8058733 > So my suggestion is either to exclude this test when run with -Xcomp > or (better) to reduce iteration number to 1 when -Xcomp, so that no > code cache overflow in this case. > > -Konstantin > >> >> Best regards, >> Vladimir Ivanov >> >>> >>> -Konstantin >>> >>> On 29.05.2015 14:49, Vladimir Ivanov wrote: >>>>> What do you mean by ignore code cache overflow? Do you mean I >>>>> should fix >>>>> the test to ignore these errors or should I leave this test unfixed >>>>> because it is a product related issue? >>>> The former. How reliable the test is if it ignores >>>> NoSuchMethodException & VirtualMachineError? Are there other >>>> manifestations of the problem? >>>> >>>> Best regards, >>>> Vladimir Ivanov >>>> >>>>> On 28.05.2015 21:22, Vladimir Ivanov wrote: >>>>>> Got it, thanks. >>>>>> >>>>>> Can we ignore errors caused by code cache overflow for now? >>>>>> >>>>>> Best regards, >>>>>> Vladimir Ivanov >>>>>> >>>>>> On 5/28/15 12:03 PM, Konstantin Shefov wrote: >>>>>>> Vladimir, >>>>>>> >>>>>>> This fix is not for timeout issue, this fix is for >>>>>>> "java.lang.VirtualMachineError: out of space in CodeCache for >>>>>>> adapters". >>>>>>> >>>>>>> Timeout issue is other bug and should be filed separately. >>>>>>> I do not know why SQE added RULES with timeout to this bug. >>>>>>> >>>>>>> By the way, if -Xcomp is set on JDK 8u, test works if not more than >>>>>>> one >>>>>>> iteration is allowed. The same thing was for JDK 9 until >>>>>>> JDK-8046809 had >>>>>>> been fixed. >>>>>>> >>>>>>> -Konstantin >>>>>>> >>>>>>> On 27.05.2015 19:54, Vladimir Ivanov wrote: >>>>>>>> Have you tried to reduce iteration granularity? >>>>>>>> >>>>>>>> Probably, checking execution duration on every test case is more >>>>>>>> robust. >>>>>>>> >>>>>>>> Best regards, >>>>>>>> Vladimir Ivanov >>>>>>>> >>>>>>>> On 5/27/15 5:50 PM, Konstantin Shefov wrote: >>>>>>>>> Hello, >>>>>>>>> >>>>>>>>> Please review the test bug fix >>>>>>>>> https://bugs.openjdk.java.net/browse/JDK-8062904 >>>>>>>>> Webrev is http://cr.openjdk.java.net/~kshefov/8062904/webrev.01/ >>>>>>>>> Test fails only against JDK 8u and passes against JDK 9. >>>>>>>>> >>>>>>>>> Thanks >>>>>>>>> >>>>>>>>> -Konstantin >>>>>>> >>>>> >>> > From konstantin.shefov at oracle.com Thu Jun 4 11:28:09 2015 From: konstantin.shefov at oracle.com (Konstantin Shefov) Date: Thu, 04 Jun 2015 14:28:09 +0300 Subject: [8u-dev] Review request : JDK-8068416: LFGarbageCollectedTest.java fails with OOME: "GC overhead limit exceeded" In-Reply-To: <5570345A.2070905@oracle.com> References: <556F2C55.5060304@oracle.com> <556F51FF.8000202@oracle.com> <556F67F0.1040504@oracle.com> <55701604.6010408@oracle.com> <55701F6C.8020006@oracle.com> <55702D86.3060306@oracle.com> <5570345A.2070905@oracle.com> Message-ID: <55703649.9000406@oracle.com> Vladimir On 06/04/2015 02:19 PM, Vladimir Ivanov wrote: > Konstantin, > >> In all cases when OOME happens the test operates with >> BoundMethodHandle$SpeciesData class, so it indeed may be caused by >> JDK-8078602. > It's not necessarily an evidence. Most of method handles are BMHs. So, > I'd suggest to inspect the heap dump. I will inspect the heap dump. If it is really JDK-8078602, I will exclude the test. -Konstantin > >> But is it a good idea of excluding the test? LFGarbageCollectedTest.java >> fails not every time and may catch other product issues if they happen. >> We can just leave it unchanged and close JDK-8068416 as a duplicate of >> JDK-8078602. > No, intermittent test failures are not acceptable. Unless there's a > way to limit produced BMHs count, the test should be excluded until > JDK-8078602 is fixed. > > Best regards, > Vladimir Ivanov > >> >> -Konstantin >> >> On 06/04/2015 12:50 PM, Vladimir Ivanov wrote: >>> Konstantin, >>> >>> Have you looked into the heap dump to understand why the test provokes >>> an OOM? Limiting test iterations is counter-productive, because it >>> defeats the purpose of the test. >>> >>> Probably, the failure is caused by BMHs which aren't collected (see >>> JDK-8078602 [1]). In that case I'd prefer the test to be excluded >>> until BMHs are converted to VM anonymous classes. >>> >>> Best regards, >>> Vladimir Ivanov >>> >>> [1] https://bugs.openjdk.java.net/browse/JDK-8078602 >>> >>> On 6/4/15 12:10 PM, Konstantin Shefov wrote: >>>> Igor, >>>> >>>> It seems I have given you wrong information. This test fails with OOME >>>> against JDK 9 also, I managed to reproduce the failure now. >>>> It was hard to reproduce it because of randomness, I need to rerun the >>>> test 50 times. Although the test seems to fail with OOME more often >>>> against JDK 8u, but I think it is just factor of randomness in the >>>> test. >>>> >>>> So I do not think it is a product bug then. >>>> >>>> -Konstantin >>>> >>>> On 06/03/2015 11:47 PM, Igor Ignatyev wrote: >>>>> Konstantin, >>>>> >>>>> do you have an explanation why the test passes on jdk 9? >>>>> from my point of view, it indicates there is a product bug in 8u >>>>> which >>>>> should be fixed and your fix just hides it. >>>>> >>>>> Igor >>>>> >>>>> On 06/03/2015 10:14 PM, Se?n Coffey wrote: >>>>>> I bumped into this failure myself today. I think you've got a typo. >>>>>> 440 should be 40. Looks like a good approach otherwise. >>>>>> >>>>>> Regards, >>>>>> Sean. >>>>>> >>>>>> On 03/06/2015 17:33, Konstantin Shefov wrote: >>>>>>> Hello, >>>>>>> >>>>>>> Please review the test bug fix >>>>>>> https://bugs.openjdk.java.net/browse/JDK-8068416 >>>>>>> Webrev is http://cr.openjdk.java.net/~kshefov/8068416/webrev.00/ >>>>>>> Test fails only against JDK 8u and passes against JDK 9. >>>>>>> >>>>>>> Fix is to reduce the number of iterations to 40. With that >>>>>>> number of >>>>>>> iterations the test passes on those hosts where it failed before. >>>>>>> The number of iterations the test start to fail is 65. >>>>>>> Before the fix the number of iterations was 84. >>>>>>> >>>>>>> Thanks >>>>>>> -Konstantin >>>>>> >>>> >> From konstantin.shefov at oracle.com Thu Jun 4 11:51:30 2015 From: konstantin.shefov at oracle.com (Konstantin Shefov) Date: Thu, 04 Jun 2015 14:51:30 +0300 Subject: [9] Review request for 8072515: Test Task: Develop new tests for JEP 219: Datagram Transport Layer Security (DTLS) In-Reply-To: <556FB0A1.6020900@oracle.com> References: <556FB0A1.6020900@oracle.com> Message-ID: <55703BC2.5010203@oracle.com> Hi Xue-Lei Thanks for reviewing! I will reduce the line size, make JPRT testing and push then. -Konstantin On 06/04/2015 04:57 AM, Xuelei Fan wrote: > Looks fine to me. > > It's nice to keep each line not exceed 80 characters. For example > > - * @run main/othervm -Dtest.security.protocol=DTLS -Dtest.mode=norm > DTLSBufferOverflowUnderflowTest > + * @run main/othervm -Dtest.security.protocol=DTLS > + * -Dtest.mode=norm DTLSBufferOverflowUnderflowTest > > Thanks, > Xuelei > > On 6/2/2015 8:15 PM, Konstantin Shefov wrote: >> Hello, >> >> Please review new tests fro DTLS feature for JDK 9: >> >> bug: https://bugs.openjdk.java.net/browse/JDK-8072515 >> webrev: http://cr.openjdk.java.net/~kshefov/8072515/webrev.00/ >> >> >> Thanks >> -Konstantin >> From paul.sandoz at oracle.com Thu Jun 4 12:37:02 2015 From: paul.sandoz at oracle.com (Paul Sandoz) Date: Thu, 4 Jun 2015 14:37:02 +0200 Subject: RFR [9]: Better failure output for test/java/util/Arrays/ParallelPrefix.java In-Reply-To: References: Message-ID: <430F0010-35E0-41E2-815E-417765996C06@oracle.com> On Jun 4, 2015, at 12:54 PM, Chris Hegarty wrote: > This is a review request to add better test failure output to jdk/test/java/util/Arrays/ParallelPrefix.java > Looks ok. Printing the content should give some clues, but i expect it's only with some focused repeated execution on an appropriate machine that one is really gonna find out what is going on. Some additional things to consider, if you so desire. If you wanna go the extra mile it's useful for the data provider to supply a string description argument summarizing the test data. You might want to size the arrays in proportion to the parallelism since the threshold is calculated as: this.threshold = (p = (hi - lo) / (ForkJoinPool.getCommonPoolParallelism() << 3)) <= MIN_PARTITION ? MIN_PARTITION : p; So for a large machine with say a parallelism of 2^5 an array size of 2^12 is not sufficient to go above MIN_PARTITION. Paul. From ivan.gerasimov at oracle.com Thu Jun 4 12:59:02 2015 From: ivan.gerasimov at oracle.com (Ivan Gerasimov) Date: Thu, 04 Jun 2015 15:59:02 +0300 Subject: RFR [9]: Better failure output for test/java/util/Arrays/ParallelPrefix.java In-Reply-To: References: Message-ID: <55704B96.2040607@oracle.com> Hi Chris! Wouldn't it cause the additional error message be printed _before_ the line 'test:...... : failure' ? I guess, It may look confusing. Would it make sense to re-throw the exception with this additional message instead? + try { + assertEquals(actual, expected, ""); + } catch (AssertionError x) { + throw new AssertionError(String.format("Expected:%s, actual:%s%n", + Arrays.toString(expected), Arrays.toString(actual)), x); + } Sincerely yours, Ivan On 04.06.2015 13:54, Chris Hegarty wrote: > This is a review request to add better test failure output to jdk/test/java/util/Arrays/ParallelPrefix.java > > This test has been seen to fail occasionally on recent JDK 9 builds. It fails with a message similar to: > > ... > test ParallelPrefix.testParallelPrefixForInt([I at 47a7a8ef, 1365, 2730, ParallelPrefix$$Lambda$56/1605520928 at 106dbc48): success > test ParallelPrefix.testParallelPrefixForInt([I at 789e465e, 1365, 2731, ParallelPrefix$$Lambda$55/1326195546 at 2c8625ae): failure > java.lang.AssertionError: expected [[I at 7b703d4a] but found [[I at 73625cb4] > at org.testng.Assert.fail(Assert.java:94) > at org.testng.Assert.failNotEquals(Assert.java:494) > at org.testng.Assert.assertArrayEquals(Assert.java:144) > at org.testng.Assert.assertEquals(Assert.java:117) > at org.testng.Assert.assertEquals(Assert.java:165) > at ParallelPrefix.testParallelPrefixForInt(ParallelPrefix.java:124) > ? > > The output is not very helpful. Testng, or at least the version that we have in use in jtreg, does not provide pretty failure messages when dealing with arrays. It is trivial to add these to the test, see below. Note: I used the 3-args version of assertEquals as it will trigger the inclusion of the index of the array where the values differ. > > Hopefully, when we see this test fail again it can produce some meaningful output. > > -Chris. > > diff --git a/test/java/util/Arrays/ParallelPrefix.java b/test/java/util/Arrays/ParallelPrefix.java > --- a/test/java/util/Arrays/ParallelPrefix.java > +++ b/test/java/util/Arrays/ParallelPrefix.java > @@ -39,6 +39,7 @@ > import static org.testng.Assert.*; > import org.testng.annotations.DataProvider; > import org.testng.annotations.Test; > +import static java.lang.System.out; > > public class ParallelPrefix { > //Array size less than MIN_PARTITION > @@ -121,11 +122,11 @@ > > int[] parallelResult = data.clone(); > Arrays.parallelPrefix(parallelResult, fromIndex, toIndex, op); > - assertEquals(parallelResult, sequentialResult); > + assertArraysEqual(parallelResult, sequentialResult); > > int[] parallelRangeResult = Arrays.copyOfRange(data, fromIndex, toIndex); > Arrays.parallelPrefix(parallelRangeResult, op); > - assertEquals(parallelRangeResult, Arrays.copyOfRange(sequentialResult, fromIndex, toIndex)); > + assertArraysEqual(parallelRangeResult, Arrays.copyOfRange(sequentialResult, fromIndex, toIndex)); > } > > @Test(dataProvider="longSet") > @@ -137,11 +138,11 @@ > > long[] parallelResult = data.clone(); > Arrays.parallelPrefix(parallelResult, fromIndex, toIndex, op); > - assertEquals(parallelResult, sequentialResult); > + assertArraysEqual(parallelResult, sequentialResult); > > long[] parallelRangeResult = Arrays.copyOfRange(data, fromIndex, toIndex); > Arrays.parallelPrefix(parallelRangeResult, op); > - assertEquals(parallelRangeResult, Arrays.copyOfRange(sequentialResult, fromIndex, toIndex)); > + assertArraysEqual(parallelRangeResult, Arrays.copyOfRange(sequentialResult, fromIndex, toIndex)); > } > > @Test(dataProvider="doubleSet") > @@ -153,11 +154,11 @@ > > double[] parallelResult = data.clone(); > Arrays.parallelPrefix(parallelResult, fromIndex, toIndex, op); > - assertEquals(parallelResult, sequentialResult); > + assertArraysEqual(parallelResult, sequentialResult); > > double[] parallelRangeResult = Arrays.copyOfRange(data, fromIndex, toIndex); > Arrays.parallelPrefix(parallelRangeResult, op); > - assertEquals(parallelRangeResult, Arrays.copyOfRange(sequentialResult, fromIndex, toIndex)); > + assertArraysEqual(parallelRangeResult, Arrays.copyOfRange(sequentialResult, fromIndex, toIndex)); > } > > @Test(dataProvider="stringSet") > @@ -169,11 +170,11 @@ > > String[] parallelResult = data.clone(); > Arrays.parallelPrefix(parallelResult, fromIndex, toIndex, op); > - assertEquals(parallelResult, sequentialResult); > + assertArraysEqual(parallelResult, sequentialResult); > > String[] parallelRangeResult = Arrays.copyOfRange(data, fromIndex, toIndex); > Arrays.parallelPrefix(parallelRangeResult, op); > - assertEquals(parallelRangeResult, Arrays.copyOfRange(sequentialResult, fromIndex, toIndex)); > + assertArraysEqual(parallelRangeResult, Arrays.copyOfRange(sequentialResult, fromIndex, toIndex)); > } > > @Test > @@ -293,5 +294,41 @@ > public static void assertInstance(Object actual, Class expected, String message) { > assertTrue(expected.isInstance(actual), message); > } > + > + static void assertArraysEqual(int[] actual, int[] expected) { > + try { > + assertEquals(actual, expected, ""); > + } catch (AssertionError x) { > + out.printf("Expected:%s, actual:%s%n", Arrays.toString(expected), Arrays.toString(actual)); > + throw x; > + } > + } > + > + static void assertArraysEqual(long[] actual, long[] expected) { > + try { > + assertEquals(actual, expected, ""); > + } catch (AssertionError x) { > + out.printf("Expected:%s, actual:%s%n", Arrays.toString(expected), Arrays.toString(actual)); > + throw x; > + } > + } > + > + static void assertArraysEqual(double[] actual, double[] expected) { > + try { > + assertEquals(actual, expected, ""); > + } catch (AssertionError x) { > + out.printf("Expected:%s, actual:%s%n", Arrays.toString(expected), Arrays.toString(actual)); > + throw x; > + } > + } > + > + static void assertArraysEqual(String[] actual, String[] expected) { > + try { > + assertEquals(actual, expected, ""); > + } catch (AssertionError x) { > + out.printf("Expected:%s, actual:%s%n", Arrays.toString(expected), Arrays.toString(actual)); > + throw x; > + } > + } > } > > > From Paul.Sandoz at oracle.com Thu Jun 4 13:01:53 2015 From: Paul.Sandoz at oracle.com (Paul Sandoz) Date: Thu, 4 Jun 2015 15:01:53 +0200 Subject: RFR 8081678: Add Stream returning methods to classes where there currently exist only Enumeration returning methods In-Reply-To: <557006F2.8010701@oracle.com> References: <3093DBE7-AC02-4371-B7EC-12C3A15EA334@oracle.com> <556EDB10.9060708@oracle.com> <2E28B338-0289-4503-B713-B30E4E1704D0@oracle.com> <556F553A.1070901@oracle.com> <557006F2.8010701@oracle.com> Message-ID: On Jun 4, 2015, at 10:06 AM, Alan Bateman wrote: > > > On 03/06/2015 21:17, Paul Sandoz wrote: >> : >>> For the tests then @library ../../util/streambootlib doesn't seem right. Is it time to move some infrastructure to make it easier to get at in other parts of the suite? >>> >> Quite probably, but not with this fix. >> >> > Sure, but we should create a bug to get this done and avoid unexpected dependences between different areas of the test suite. > https://bugs.openjdk.java.net/browse/JDK-8085814 There is some history behind the current structure of the stream-based tests that needs to be unpicked. It's somewhat standalone nature made it easy to edit in IDEs and execute directly via TestNG. I would argue that the general structure the JDK test layout is somewhat hostile for productive IDE-based development, executing and debugging of tests. Paul. From konstantin.shefov at oracle.com Thu Jun 4 13:11:23 2015 From: konstantin.shefov at oracle.com (Konstantin Shefov) Date: Thu, 04 Jun 2015 16:11:23 +0300 Subject: [8u-dev] Review request : JDK-8068416: LFGarbageCollectedTest.java fails with OOME: "GC overhead limit exceeded" In-Reply-To: <55703649.9000406@oracle.com> References: <556F2C55.5060304@oracle.com> <556F51FF.8000202@oracle.com> <556F67F0.1040504@oracle.com> <55701604.6010408@oracle.com> <55701F6C.8020006@oracle.com> <55702D86.3060306@oracle.com> <5570345A.2070905@oracle.com> <55703649.9000406@oracle.com> Message-ID: <55704E7B.8090701@oracle.com> Vladimir I looked at heap dump using jhat tool. Heap contained 2452 occurences of java.lang.invoke.BoundMethodHandle$Species_* classes by the moment OOME happened. So looks like java.lang.invoke.BoundMethodHandle$Species_* classesare not unloading as it is said in JDK-8078602 [1]. So I will exclude the test with "@ignore 8078602" tag. [1] https://bugs.openjdk.java.net/browse/JDK-8078602 -Konstantin On 06/04/2015 02:28 PM, Konstantin Shefov wrote: > Vladimir > > On 06/04/2015 02:19 PM, Vladimir Ivanov wrote: >> Konstantin, >> >>> In all cases when OOME happens the test operates with >>> BoundMethodHandle$SpeciesData class, so it indeed may be caused by >>> JDK-8078602. >> It's not necessarily an evidence. Most of method handles are BMHs. >> So, I'd suggest to inspect the heap dump. > I will inspect the heap dump. If it is really JDK-8078602, I will > exclude the test. > > -Konstantin >> >>> But is it a good idea of excluding the test? >>> LFGarbageCollectedTest.java >>> fails not every time and may catch other product issues if they happen. >>> We can just leave it unchanged and close JDK-8068416 as a duplicate of >>> JDK-8078602. >> No, intermittent test failures are not acceptable. Unless there's a >> way to limit produced BMHs count, the test should be excluded until >> JDK-8078602 is fixed. >> >> Best regards, >> Vladimir Ivanov >> >>> >>> -Konstantin >>> >>> On 06/04/2015 12:50 PM, Vladimir Ivanov wrote: >>>> Konstantin, >>>> >>>> Have you looked into the heap dump to understand why the test provokes >>>> an OOM? Limiting test iterations is counter-productive, because it >>>> defeats the purpose of the test. >>>> >>>> Probably, the failure is caused by BMHs which aren't collected (see >>>> JDK-8078602 [1]). In that case I'd prefer the test to be excluded >>>> until BMHs are converted to VM anonymous classes. >>>> >>>> Best regards, >>>> Vladimir Ivanov >>>> >>>> [1] https://bugs.openjdk.java.net/browse/JDK-8078602 >>>> >>>> On 6/4/15 12:10 PM, Konstantin Shefov wrote: >>>>> Igor, >>>>> >>>>> It seems I have given you wrong information. This test fails with >>>>> OOME >>>>> against JDK 9 also, I managed to reproduce the failure now. >>>>> It was hard to reproduce it because of randomness, I need to rerun >>>>> the >>>>> test 50 times. Although the test seems to fail with OOME more often >>>>> against JDK 8u, but I think it is just factor of randomness in the >>>>> test. >>>>> >>>>> So I do not think it is a product bug then. >>>>> >>>>> -Konstantin >>>>> >>>>> On 06/03/2015 11:47 PM, Igor Ignatyev wrote: >>>>>> Konstantin, >>>>>> >>>>>> do you have an explanation why the test passes on jdk 9? >>>>>> from my point of view, it indicates there is a product bug in 8u >>>>>> which >>>>>> should be fixed and your fix just hides it. >>>>>> >>>>>> Igor >>>>>> >>>>>> On 06/03/2015 10:14 PM, Se?n Coffey wrote: >>>>>>> I bumped into this failure myself today. I think you've got a typo. >>>>>>> 440 should be 40. Looks like a good approach otherwise. >>>>>>> >>>>>>> Regards, >>>>>>> Sean. >>>>>>> >>>>>>> On 03/06/2015 17:33, Konstantin Shefov wrote: >>>>>>>> Hello, >>>>>>>> >>>>>>>> Please review the test bug fix >>>>>>>> https://bugs.openjdk.java.net/browse/JDK-8068416 >>>>>>>> Webrev is http://cr.openjdk.java.net/~kshefov/8068416/webrev.00/ >>>>>>>> Test fails only against JDK 8u and passes against JDK 9. >>>>>>>> >>>>>>>> Fix is to reduce the number of iterations to 40. With that >>>>>>>> number of >>>>>>>> iterations the test passes on those hosts where it failed before. >>>>>>>> The number of iterations the test start to fail is 65. >>>>>>>> Before the fix the number of iterations was 84. >>>>>>>> >>>>>>>> Thanks >>>>>>>> -Konstantin >>>>>>> >>>>> >>> > From dmitry.samersoff at oracle.com Thu Jun 4 13:38:15 2015 From: dmitry.samersoff at oracle.com (Dmitry Samersoff) Date: Thu, 04 Jun 2015 16:38:15 +0300 Subject: RFR(M, v14): JDK-8059036 : Implement Diagnostic Commands for heap and finalizerinfo In-Reply-To: <556F491E.3070709@oracle.com> References: <554894E9.8020908@oracle.com> <1AF219C7-1055-4D97-BB46-2FA390C4135D@oracle.com> <5548D65C.8040304@gmail.com> <554A8CEE.2090009@oracle.com> <55523418.5010708@oracle.com> <55527935.5010408@gmail.com> <5553B552.9060900@oracle.com> <555453BA.2070205@gmail.com> <55567748.6020304@oracle.com> <55567A68.9020802@oracle.com> <5556F34B.5050701@oracle.com> <55573A31.7050306@gmail.com> <55573C86.8030807@gmail.com> <5559D85B.2050500@oracle.com> <6FB8DF19-1D67-4702-9642-4C630765D2D1@oracle.com> <555C2EEB.1050505@oracle.com> <3941B430-FE6C-4AF0-A875-76D24411C654@oracle.com> <5564802A.2010403@oracle.com> <5565738A.4040109@gmail.com> <7EB0BE71-1608-4047-B71D-1A9EAA9083BB@oracle.com> <556727A2.9060209@gmail.com> <55675908.6030406@oracle.com> <556AF3CD.50900@oracle.com> <37227441-B016-4A9A-B9CA-BE100E992C9A@oracle.com> <556D8FB0.4070608@oracle.com> <556E9970.8010704@gmail.com> <556EBF4B.6070400@oracle.com> <556F1D6C.6010806@oracle.com> <556F491E.3070709@oracle.com> Message-ID: <557054C7.4030705@oracle.com> Mandy, Thank you for the review. Updated webrev is: http://cr.openjdk.java.net/~dsamersoff/JDK-8059036/webrev.14 addressed comments, added a unit test to JDK workspace. On 2015-06-03 21:36, Mandy Chung wrote: > Should getFinalizerHistogram() return FinalizerHistogramEntry[]? > The VM knows the returned type anyway. "[java/lang/ref/Entry" signature would need a separate symbol entry in swollen vmSymbols::init() so I would prefer to stay with more generic Object[] > Perhaps the getFinalizerHistogram method should be renamed > e.g. "dump"? The line in vmSymbols looks like: template( get_finalizer_histogram_name, "getFinalizerHistogram") I would prefer to keep method name specific enough to be able to find it by grep in jdk code. (other comments are addressed) -Dmitry On 2015-06-03 21:36, Mandy Chung wrote: > > > On 06/03/2015 08:29 AM, Dmitry Samersoff wrote: >> Updated webrev: >> >> http://cr.openjdk.java.net/~dsamersoff/JDK-8059036/webrev.13 > > I reviewed the jdk change. The version looks okay and some comments: > > ReferenceQueue.java - you should eliminate the use of rawtype: > > 84 Reference rn = r.next; > > Change Reference to Reference done. > > The forEach method - eliminate the use of raw type and > move @SuppressWarning to the field variable in line 182: > > @SuppressWarnings("unchecked") > Reference rn = r.next; done. > > FinalizerHistogram.java > Copyright year needs update. done. > > I personally think the VM code would be much simpler if you stay with > alternative entry of String and int than dealing with a > FinalizerHistogramEntry private class. It's okay as FinalizerHistogram > class is separated. > > The comment in line 35 is suitable to move to the class description and > this is the suggestion. > > // This FinalizerHistogram class is for GC.finalizer_info diagnostic > command support. > // It is invoked by the VM. done. > > Should getFinalizerHistogram() return FinalizerHistogramEntry[]? The VM > knows the returned type anyway. "[java/lang/ref/Entry" signature would need a separate symbol entry in swollen vmSymbols::init() so I would prefer to stay with more generic Object[] > It's an inner class of > FinalizerHistogram and it can simply be named as "Entry". I suggest to > have Entry::increment method to replace ".instanceCount++". done. > > The tests are in hotspot/test. Can you add a unit test in jdk/test? > Perhaps you can test FinalizerHistogram.getFinalizerHistogram() via > reflection - just a sanity test. done. The test repeats hotspot one, but uses reflection instead of VM call. > > A minor naming comment - now you have a FinalizerHistogram class. > Perhaps the getFinalizerHistogram method should be renamed e.g. "dump"? The line in vmSymbols looks like: template( get_finalizer_histogram_name, "getFinalizerHistogram") I would prefer to keep it specific enough to be able to find it by grep in jdk code. -- Dmitry Samersoff Oracle Java development team, Saint Petersburg, Russia * I would love to change the world, but they won't give me the sources. From mandy.chung at oracle.com Thu Jun 4 13:56:03 2015 From: mandy.chung at oracle.com (Mandy Chung) Date: Thu, 4 Jun 2015 06:56:03 -0700 Subject: RFR(M, v14): JDK-8059036 : Implement Diagnostic Commands for heap and finalizerinfo In-Reply-To: <557054C7.4030705@oracle.com> References: <554894E9.8020908@oracle.com> <1AF219C7-1055-4D97-BB46-2FA390C4135D@oracle.com> <5548D65C.8040304@gmail.com> <554A8CEE.2090009@oracle.com> <55523418.5010708@oracle.com> <55527935.5010408@gmail.com> <5553B552.9060900@oracle.com> <555453BA.2070205@gmail.com> <55567748.6020304@oracle.com> <55567A68.9020802@oracle.com> <5556F34B.5050701@oracle.com> <55573A31.7050306@gmail.com> <55573C86.8030807@gmail.com> <5559D85B.2050500@oracle.com> <6FB8DF19-1D67-4702-9642-4C630765D2D1@oracle.com> <555C2EEB.1050505@oracle.com> <3941B430-FE6C-4AF0-A875-76D24411C654@oracle.com> <5564802A.2010403@oracle.com> <5565738A.4040109@gmail.com> <7EB0BE71-1608-4047-B71D-1A9EAA9083BB@oracle.com> <556727A2.9060209@gmail.com> <55675908.6030406@oracle.com> <556AF3CD.50900@oracle.com> <37227441-B016-4A9A-B9CA-BE100E992C9A@oracle.com> <556D8FB0.4070608@oracle.com> <556E9970.8010704@gmail.com> <556EBF4B.6070400@oracle.com> <556F1D6C.6010806@oracle.com> <556F491E.3070709@oracle.com> <557054C7.4030705! @oracle.com> Message-ID: <3607C2B6-F247-4291-9198-279CF8ED2AF3@oracle.com> > On Jun 4, 2015, at 6:38 AM, Dmitry Samersoff wrote: > > Mandy, > > Thank you for the review. > > Updated webrev is: > > http://cr.openjdk.java.net/~dsamersoff/JDK-8059036/webrev.14 > Thanks for the update and the test. > addressed comments, added a unit test to JDK workspace. > This test does not need @library and @build, right? > > On 2015-06-03 21:36, Mandy Chung wrote: > >> Should getFinalizerHistogram() return FinalizerHistogramEntry[]? >> The VM knows the returned type anyway. > > "[java/lang/ref/Entry" signature would need a separate symbol entry in > swollen vmSymbols::init() so I would prefer to stay with more generic > Object[] > You already add several symbols - why is it an issue for another one? Or if adding vm symbols is a concern, you should revert to String and int[] that you decide not to. The decision should apply consistently (use String and int, or new Java type). >> Perhaps the getFinalizerHistogram method should be renamed >> e.g. "dump"? > > The line in vmSymbols looks like: > > template( > get_finalizer_histogram_name, "getFinalizerHistogram") > > I would prefer to keep method name specific enough to be able to > find it by grep in jdk code. Grep in jdk code is easy as you have a new class :) Mandy > > (other comments are addressed) > > -Dmitry > > > On 2015-06-03 21:36, Mandy Chung wrote: >> >> >> On 06/03/2015 08:29 AM, Dmitry Samersoff wrote: >>> Updated webrev: >>> >>> http://cr.openjdk.java.net/~dsamersoff/JDK-8059036/webrev.13 >> >> I reviewed the jdk change. The version looks okay and some comments: >> >> ReferenceQueue.java - you should eliminate the use of rawtype: >> >> 84 Reference rn = r.next; >> >> Change Reference to Reference > > done. > >> >> The forEach method - eliminate the use of raw type and >> move @SuppressWarning to the field variable in line 182: >> >> @SuppressWarnings("unchecked") >> Reference rn = r.next; > > done. > >> >> FinalizerHistogram.java >> Copyright year needs update. > > done. >> >> I personally think the VM code would be much simpler if you stay with >> alternative entry of String and int than dealing with a >> FinalizerHistogramEntry private class. It's okay as FinalizerHistogram >> class is separated. >> >> The comment in line 35 is suitable to move to the class description and >> this is the suggestion. >> >> // This FinalizerHistogram class is for GC.finalizer_info diagnostic >> command support. >> // It is invoked by the VM. > > done. > >> >> Should getFinalizerHistogram() return FinalizerHistogramEntry[]? The VM >> knows the returned type anyway. > > "[java/lang/ref/Entry" signature would need a separate symbol entry in > swollen vmSymbols::init() so I would prefer to stay with more generic > Object[] > >> It's an inner class of >> FinalizerHistogram and it can simply be named as "Entry". I suggest to >> have Entry::increment method to replace ".instanceCount++". > > done. > >> >> The tests are in hotspot/test. Can you add a unit test in jdk/test? >> Perhaps you can test FinalizerHistogram.getFinalizerHistogram() via >> reflection - just a sanity test. > > done. The test repeats hotspot one, but uses reflection instead of VM call. > >> >> A minor naming comment - now you have a FinalizerHistogram class. >> Perhaps the getFinalizerHistogram method should be renamed e.g. "dump"? > > The line in vmSymbols looks like: > > template( > get_finalizer_histogram_name, "getFinalizerHistogram") > > I would prefer to keep it specific enough to be able to > find it by grep in jdk code. > > > -- > Dmitry Samersoff > Oracle Java development team, Saint Petersburg, Russia > * I would love to change the world, but they won't give me the sources. From chris.hegarty at oracle.com Thu Jun 4 14:09:59 2015 From: chris.hegarty at oracle.com (Chris Hegarty) Date: Thu, 4 Jun 2015 15:09:59 +0100 Subject: RFR [9]: Better failure output for test/java/util/Arrays/ParallelPrefix.java In-Reply-To: <55704B96.2040607@oracle.com> References: <55704B96.2040607@oracle.com> Message-ID: <1959A855-CA2B-40C4-915E-27519202969B@oracle.com> Thanks for the reviews. On 4 Jun 2015, at 13:59, Ivan Gerasimov wrote: > Wouldn't it cause the additional error message be printed _before_ the line 'test:...... : failure' ? > I guess, It may look confusing. > > Would it make sense to re-throw the exception with this additional message instead? > > + try { > + assertEquals(actual, expected, ""); > + } catch (AssertionError x) { > + throw new AssertionError(String.format("Expected:%s, actual:%s%n", > + Arrays.toString(expected), Arrays.toString(actual)), x); > + } You are right. Adding the exception as the cause, or suppressed, is better. Changed. On 4 Jun 2015, at 13:37, Paul Sandoz wrote: > ... > If you wanna go the extra mile it's useful for the data provider to supply a string description argument summarizing the test data. Added. > You might want to size the arrays in proportion to the parallelism since the threshold is calculated as: > > this.threshold = > (p = (hi - lo) / (ForkJoinPool.getCommonPoolParallelism() << 3)) > <= MIN_PARTITION ? MIN_PARTITION : p; > > So for a large machine with say a parallelism of 2^5 an array size of 2^12 is not sufficient to go above MIN_PARTITION.. I could be wrong, and I accept your point about sizing as per parallelism, but I think 2^12 should be sufficient for most systems, given MIN_PARTITION = 16. With a parallelism of 2^5, then p will be greater than MIN_PARTITION, no? Latest changes below. -Chris. diff --git a/test/java/util/Arrays/ParallelPrefix.java b/test/java/util/Arrays/ParallelPrefix.java --- a/test/java/util/Arrays/ParallelPrefix.java +++ b/test/java/util/Arrays/ParallelPrefix.java @@ -60,7 +60,7 @@ LARGE_ARRAY_SIZE }; - @DataProvider + @DataProvider(name = "intSet") public static Object[][] intSet(){ return genericData(size -> IntStream.range(0, size).toArray(), new IntBinaryOperator[]{ @@ -68,7 +68,7 @@ Integer::min}); } - @DataProvider + @DataProvider(name = "longSet") public static Object[][] longSet(){ return genericData(size -> LongStream.range(0, size).toArray(), new LongBinaryOperator[]{ @@ -76,7 +76,7 @@ Long::min}); } - @DataProvider + @DataProvider(name = "doubleSet") public static Object[][] doubleSet(){ return genericData(size -> IntStream.range(0, size).mapToDouble(i -> (double)i).toArray(), new DoubleBinaryOperator[]{ @@ -84,7 +84,7 @@ Double::min}); } - @DataProvider + @DataProvider(name = "stringSet") public static Object[][] stringSet(){ Function stringsFunc = size -> IntStream.range(0, size).mapToObj(Integer::toString).toArray(String[]::new); @@ -121,11 +121,11 @@ int[] parallelResult = data.clone(); Arrays.parallelPrefix(parallelResult, fromIndex, toIndex, op); - assertEquals(parallelResult, sequentialResult); + assertArraysEqual(parallelResult, sequentialResult); int[] parallelRangeResult = Arrays.copyOfRange(data, fromIndex, toIndex); Arrays.parallelPrefix(parallelRangeResult, op); - assertEquals(parallelRangeResult, Arrays.copyOfRange(sequentialResult, fromIndex, toIndex)); + assertArraysEqual(parallelRangeResult, Arrays.copyOfRange(sequentialResult, fromIndex, toIndex)); } @Test(dataProvider="longSet") @@ -137,11 +137,11 @@ long[] parallelResult = data.clone(); Arrays.parallelPrefix(parallelResult, fromIndex, toIndex, op); - assertEquals(parallelResult, sequentialResult); + assertArraysEqual(parallelResult, sequentialResult); long[] parallelRangeResult = Arrays.copyOfRange(data, fromIndex, toIndex); Arrays.parallelPrefix(parallelRangeResult, op); - assertEquals(parallelRangeResult, Arrays.copyOfRange(sequentialResult, fromIndex, toIndex)); + assertArraysEqual(parallelRangeResult, Arrays.copyOfRange(sequentialResult, fromIndex, toIndex)); } @Test(dataProvider="doubleSet") @@ -153,11 +153,11 @@ double[] parallelResult = data.clone(); Arrays.parallelPrefix(parallelResult, fromIndex, toIndex, op); - assertEquals(parallelResult, sequentialResult); + assertArraysEqual(parallelResult, sequentialResult); double[] parallelRangeResult = Arrays.copyOfRange(data, fromIndex, toIndex); Arrays.parallelPrefix(parallelRangeResult, op); - assertEquals(parallelRangeResult, Arrays.copyOfRange(sequentialResult, fromIndex, toIndex)); + assertArraysEqual(parallelRangeResult, Arrays.copyOfRange(sequentialResult, fromIndex, toIndex)); } @Test(dataProvider="stringSet") @@ -169,11 +169,11 @@ String[] parallelResult = data.clone(); Arrays.parallelPrefix(parallelResult, fromIndex, toIndex, op); - assertEquals(parallelResult, sequentialResult); + assertArraysEqual(parallelResult, sequentialResult); String[] parallelRangeResult = Arrays.copyOfRange(data, fromIndex, toIndex); Arrays.parallelPrefix(parallelRangeResult, op); - assertEquals(parallelRangeResult, Arrays.copyOfRange(sequentialResult, fromIndex, toIndex)); + assertArraysEqual(parallelRangeResult, Arrays.copyOfRange(sequentialResult, fromIndex, toIndex)); } @Test @@ -293,5 +293,41 @@ public static void assertInstance(Object actual, Class expected, String message) { assertTrue(expected.isInstance(actual), message); } + + static void assertArraysEqual(int[] actual, int[] expected) { + try { + assertEquals(actual, expected, ""); + } catch (AssertionError x) { + throw new AssertionError(String.format("Expected:%s, actual:%s", + Arrays.toString(expected), Arrays.toString(actual)), x); + } + } + + static void assertArraysEqual(long[] actual, long[] expected) { + try { + assertEquals(actual, expected, ""); + } catch (AssertionError x) { + throw new AssertionError(String.format("Expected:%s, actual:%s", + Arrays.toString(expected), Arrays.toString(actual)), x); + } + } + + static void assertArraysEqual(double[] actual, double[] expected) { + try { + assertEquals(actual, expected, ""); + } catch (AssertionError x) { + throw new AssertionError(String.format("Expected:%s, actual:%s", + Arrays.toString(expected), Arrays.toString(actual)), x); + } + } + + static void assertArraysEqual(String[] actual, String[] expected) { + try { + assertEquals(actual, expected, ""); + } catch (AssertionError x) { + throw new AssertionError(String.format("Expected:%s, actual:%s", + Arrays.toString(expected), Arrays.toString(actual)), x); + } + } } From paul.sandoz at oracle.com Thu Jun 4 14:28:38 2015 From: paul.sandoz at oracle.com (Paul Sandoz) Date: Thu, 4 Jun 2015 16:28:38 +0200 Subject: RFR [9]: Better failure output for test/java/util/Arrays/ParallelPrefix.java In-Reply-To: <1959A855-CA2B-40C4-915E-27519202969B@oracle.com> References: <55704B96.2040607@oracle.com> <1959A855-CA2B-40C4-915E-27519202969B@oracle.com> Message-ID: <712E8224-4F5B-4F42-83DC-72714B6EB1D5@oracle.com> On Jun 4, 2015, at 4:09 PM, Chris Hegarty wrote: > On 4 Jun 2015, at 13:37, Paul Sandoz wrote: > >> ... >> If you wanna go the extra mile it's useful for the data provider to supply a string description argument summarizing the test data. > > Added. > I did not mean: > + @DataProvider(name = "intSet") I meant as a String parameter of the test method, so that some useful summary shows up in the test report. It's not terribly important. >> You might want to size the arrays in proportion to the parallelism since the threshold is calculated as: >> >> this.threshold = >> (p = (hi - lo) / (ForkJoinPool.getCommonPoolParallelism() << 3)) >> <= MIN_PARTITION ? MIN_PARTITION : p; >> >> So for a large machine with say a parallelism of 2^5 an array size of 2^12 is not sufficient to go above MIN_PARTITION.. > > I could be wrong, and I accept your point about sizing as per parallelism, but I think 2^12 should be sufficient for most systems, given MIN_PARTITION = 16. With a parallelism of 2^5, then p will be greater than MIN_PARTITION, no? > Equal to: 2^12 / (2^5 * 2^3) = 2^12 / 2^8 = 2^4 I think it would be ok to increase this to 2^14. Paul. From chris.hegarty at oracle.com Thu Jun 4 14:57:15 2015 From: chris.hegarty at oracle.com (Chris Hegarty) Date: Thu, 4 Jun 2015 15:57:15 +0100 Subject: RFR [9]: Better failure output for test/java/util/Arrays/ParallelPrefix.java In-Reply-To: <712E8224-4F5B-4F42-83DC-72714B6EB1D5@oracle.com> References: <55704B96.2040607@oracle.com> <1959A855-CA2B-40C4-915E-27519202969B@oracle.com> <712E8224-4F5B-4F42-83DC-72714B6EB1D5@oracle.com> Message-ID: <35B4508F-D6FA-4F28-8FA2-F43733426EE2@oracle.com> On 4 Jun 2015, at 15:28, Paul Sandoz wrote: > On Jun 4, 2015, at 4:09 PM, Chris Hegarty wrote: >> On 4 Jun 2015, at 13:37, Paul Sandoz wrote: >> >>> ... >>> If you wanna go the extra mile it's useful for the data provider to supply a string description argument summarizing the test data. >> >> Added. >> > > I did not mean: > >> + @DataProvider(name = "intSet") > > I meant as a String parameter of the test method, so that some useful summary shows up in the test report. It's not terribly important. D?oh! Sorry. >>> You might want to size the arrays in proportion to the parallelism since the threshold is calculated as: >>> >>> this.threshold = >>> (p = (hi - lo) / (ForkJoinPool.getCommonPoolParallelism() << 3)) >>> <= MIN_PARTITION ? MIN_PARTITION : p; >>> >>> So for a large machine with say a parallelism of 2^5 an array size of 2^12 is not sufficient to go above MIN_PARTITION.. >> >> I could be wrong, and I accept your point about sizing as per parallelism, but I think 2^12 should be sufficient for most systems, given MIN_PARTITION = 16. With a parallelism of 2^5, then p will be greater than MIN_PARTITION, no? >> > > Equal to: > > 2^12 / (2^5 * 2^3) = 2^12 / 2^8 = 2^4 Another d?oh! I missed the '<< 3' in my calculations. > I think it would be ok to increase this to 2^14. Increasing the size to 2^14 increases the execution time of the test from ~2 secs to ~(12-16) secs on my really old slow machine. So this should be ok. I?ll run some more tests with the increased size before pushing. Thanks, -Chris. > Paul. From dmitry.samersoff at oracle.com Thu Jun 4 15:49:31 2015 From: dmitry.samersoff at oracle.com (Dmitry Samersoff) Date: Thu, 04 Jun 2015 18:49:31 +0300 Subject: RFR(M, v14): JDK-8059036 : Implement Diagnostic Commands for heap and finalizerinfo In-Reply-To: <3607C2B6-F247-4291-9198-279CF8ED2AF3@oracle.com> References: <554894E9.8020908@oracle.com> <554A8CEE.2090009@oracle.com> <55523418.5010708@oracle.com> <55527935.5010408@gmail.com> <5553B552.9060900@oracle.com> <555453BA.2070205@gmail.com> <55567748.6020304@oracle.com> <55567A68.9020802@oracle.com> <5556F34B.5050701@oracle.com> <55573A31.7050306@gmail.com> <55573C86.8030807@gmail.com> <5559D85B.2050500@oracle.com> <6FB8DF19-1D67-4702-9642-4C630765D2D1@oracle.com> <555C2EEB.1050505@oracle.com> <3941B430-FE6C-4AF0-A875-76D24411C654@oracle.com> <5564802A.2010403@oracle.com> <5565738A.4040109@gmail.com> <7EB0BE71-1608-4047-B71D-1A9EAA9083BB@oracle.com> <556727A2.9060209@gmail.com> <55675908.6030406@oracle.com> <556AF3CD.50900@oracle.com> <37227441-B016-4A9A-B9CA-BE100E992C9A@oracle.com> <556D8FB0.4070608@oracle.com> <556E9970.8010704@gmail.com> <556EBF4B.6070400@oracle.com> <556F1D6C.6010806@oracle.com> <556F491E.3070709@oracle.com> <557054C7.4030705@oracle.com> <3607C2B6-F247-4291-9198-279CF8ED2AF3@oracle.com> Message-ID: <5570738B.5040603@oracle.com> Mandy, Webrev updated in-place (press shift-reload). http://cr.openjdk.java.net/~dsamersoff/JDK-8059036/webrev.14 getFinalizerHistogram() now returns Entry[] @library and @build removed from the test. -Dmitry On 2015-06-04 16:56, Mandy Chung wrote: > >> On Jun 4, 2015, at 6:38 AM, Dmitry Samersoff wrote: >> >> Mandy, >> >> Thank you for the review. >> >> Updated webrev is: >> >> http://cr.openjdk.java.net/~dsamersoff/JDK-8059036/webrev.14 >> > > Thanks for the update and the test. > >> addressed comments, added a unit test to JDK workspace. >> > > This test does not need @library and @build, right? > >> >> On 2015-06-03 21:36, Mandy Chung wrote: >> >>> Should getFinalizerHistogram() return FinalizerHistogramEntry[]? >>> The VM knows the returned type anyway. >> >> "[java/lang/ref/Entry" signature would need a separate symbol entry in >> swollen vmSymbols::init() so I would prefer to stay with more generic >> Object[] >> > > You already add several symbols - why is it an issue for another one? Or if adding vm symbols is a concern, you should revert to String and int[] that you decide not to. The decision should apply consistently (use String and int, or new Java type). > > >>> Perhaps the getFinalizerHistogram method should be renamed >>> e.g. "dump"? >> >> The line in vmSymbols looks like: >> >> template( >> get_finalizer_histogram_name, "getFinalizerHistogram") >> >> I would prefer to keep method name specific enough to be able to >> find it by grep in jdk code. > > > Grep in jdk code is easy as you have a new class :) > > Mandy > >> >> (other comments are addressed) >> >> -Dmitry >> >> >> On 2015-06-03 21:36, Mandy Chung wrote: >>> >>> >>> On 06/03/2015 08:29 AM, Dmitry Samersoff wrote: >>>> Updated webrev: >>>> >>>> http://cr.openjdk.java.net/~dsamersoff/JDK-8059036/webrev.13 >>> >>> I reviewed the jdk change. The version looks okay and some comments: >>> >>> ReferenceQueue.java - you should eliminate the use of rawtype: >>> >>> 84 Reference rn = r.next; >>> >>> Change Reference to Reference >> >> done. >> >>> >>> The forEach method - eliminate the use of raw type and >>> move @SuppressWarning to the field variable in line 182: >>> >>> @SuppressWarnings("unchecked") >>> Reference rn = r.next; >> >> done. >> >>> >>> FinalizerHistogram.java >>> Copyright year needs update. >> >> done. >>> >>> I personally think the VM code would be much simpler if you stay with >>> alternative entry of String and int than dealing with a >>> FinalizerHistogramEntry private class. It's okay as FinalizerHistogram >>> class is separated. >>> >>> The comment in line 35 is suitable to move to the class description and >>> this is the suggestion. >>> >>> // This FinalizerHistogram class is for GC.finalizer_info diagnostic >>> command support. >>> // It is invoked by the VM. >> >> done. >> >>> >>> Should getFinalizerHistogram() return FinalizerHistogramEntry[]? The VM >>> knows the returned type anyway. >> >> "[java/lang/ref/Entry" signature would need a separate symbol entry in >> swollen vmSymbols::init() so I would prefer to stay with more generic >> Object[] >> >>> It's an inner class of >>> FinalizerHistogram and it can simply be named as "Entry". I suggest to >>> have Entry::increment method to replace ".instanceCount++". >> >> done. >> >>> >>> The tests are in hotspot/test. Can you add a unit test in jdk/test? >>> Perhaps you can test FinalizerHistogram.getFinalizerHistogram() via >>> reflection - just a sanity test. >> >> done. The test repeats hotspot one, but uses reflection instead of VM call. >> >>> >>> A minor naming comment - now you have a FinalizerHistogram class. >>> Perhaps the getFinalizerHistogram method should be renamed e.g. "dump"? >> >> The line in vmSymbols looks like: >> >> template( >> get_finalizer_histogram_name, "getFinalizerHistogram") >> >> I would prefer to keep it specific enough to be able to >> find it by grep in jdk code. >> >> >> -- >> Dmitry Samersoff >> Oracle Java development team, Saint Petersburg, Russia >> * I would love to change the world, but they won't give me the sources. > -- Dmitry Samersoff Oracle Java development team, Saint Petersburg, Russia * I would love to change the world, but they won't give me the sources. From richard.warburton at gmail.com Thu Jun 4 19:54:07 2015 From: richard.warburton at gmail.com (Richard Warburton) Date: Thu, 4 Jun 2015 20:54:07 +0100 Subject: Lower overhead String encoding/decoding In-Reply-To: <556F2FBF.6060103@oracle.com> References: <54200F07.4070604@oracle.com> <542441BB.600@oracle.com> <5444BB17.8080708@oracle.com> <544FADA5.7000403@oracle.com> <544FB0F7.8080909@univ-mlv.fr> <54737BC3.9010508@oracle.com> <5473A167.5040703@oracle.com> <54A458DE.1010204@oracle.com> <556F2CAC.6000308@oracle.com> <556F2FBF.6060103@oracle.com> Message-ID: Hi, > I'm still planning to be the sponsor for this RFE and get this one > > into the jdk9. We are current working on JEP 254 [1][2][3]in which > > the internal storage mechanism is changed from char[] to byte[]. If > > this JEP get approved and is scheduled to target JDK9, the > > decoding/encoding pieces will also need to be updated/ changed > > accordingly. Given that I would prefer to delay this RFE a little > > more till all pieces are settled. > > Do we actually need to wait for Compact Strings to land? The failure > scenario would be that Compact Strings are not making it to JDK 9, and > this code lingers in history. > > I mean it's fine to see if there is a clash between the two, but I don't > think Richard's change should be blocked by Compact Strings, given it is > ready for integration (?), and Compact Strings are not yet. Thanks for your replies gents - most appreciated. IMO the changes are orthogonal at the moment. If compact strings happen then this patch will probably need to be extended, but I imagine at least the implemented code is the non-compact case and will remain - as would API decisions and Javadoc. Of course that's just my opinion and I'm not as informed on the compact strings work as you are ;) regards, Richard Warburton http://insightfullogic.com @RichardWarburto From paul.sandoz at oracle.com Thu Jun 4 20:06:34 2015 From: paul.sandoz at oracle.com (Paul Sandoz) Date: Thu, 4 Jun 2015 22:06:34 +0200 Subject: RFR 8072773 (fs) Files.lines needs a better splitting implementation for stream source In-Reply-To: <556F28EF.1000201@oracle.com> References: <556F28EF.1000201@oracle.com> Message-ID: <7CF56CF1-7A27-475B-81D3-8B784B785247@oracle.com> On Jun 3, 2015, at 6:18 PM, Alan Bateman wrote: > As this code path is only for the default provider case then there's a good chance that it will be a FileChannelImpl, in which case you can call its unmap method (directly or via a shared secret). It is possible to interpose on the default provider so you can't be guaranteed it is a FileChannelImpl of course. > Not yet done. I would like to revisit this as a separate issue if i may. > In passing, you might consider moving ByteBufferLinesSpliterator to its own source file because Files is getting very big. > Done. I am going to revisit the set of support charsets at another time. For the moment i have limited to a subset of standard charsets. Also the file lines test tickled a bug in the stream test library, it was not closing streams, which becomes an issue when ~15000 tests are executed leaving the process with no more file descriptors: http://cr.openjdk.java.net/~psandoz/jdk9/JDK-8072773-File-lines/webrev/ Paul. From mandy.chung at oracle.com Thu Jun 4 22:20:54 2015 From: mandy.chung at oracle.com (Mandy Chung) Date: Thu, 4 Jun 2015 15:20:54 -0700 Subject: RFR(M, v14): JDK-8059036 : Implement Diagnostic Commands for heap and finalizerinfo In-Reply-To: <5570738B.5040603@oracle.com> References: <554894E9.8020908@oracle.com> <554A8CEE.2090009@oracle.com> <55523418.5010708@oracle.com> <55527935.5010408@gmail.com> <5553B552.9060900@oracle.com> <555453BA.2070205@gmail.com> <55567748.6020304@oracle.com> <55567A68.9020802@oracle.com> <5556F34B.5050701@oracle.com> <55573A31.7050306@gmail.com> <55573C86.8030807@gmail.com> <5559D85B.2050500@oracle.com> <6FB8DF19-1D67-4702-9642-4C630765D2D1@oracle.com> <555C2EEB.1050505@oracle.com> <3941B430-FE6C-4AF0-A875-76D24411C654@oracle.com> <5564802A.2010403@oracle.com> <5565738A.4040109@gmail.com> <7EB0BE71-1608-4047-B71D-1A9EAA9083BB@oracle.com> <556727A2.9060209@gmail.com> <55675908.6030406@oracle.com> <556AF3CD.50900@oracle.com> <37227441-B016-4A9A-B9CA-BE100E992C9A@oracle.com> <556D8FB0.4070608@oracle.com> <556E9970.8010704@gmail.com> <556EBF4B.6070400@oracle.com> <556F1D6C.6010806@oracle.com> <556F491E.3070709@oracle.com> <557054C7.4030705@oracle.com> <3607C2B6-F247-4291-9198-279CF8ED2AF3@oracle.com> <5570738B.504060! 3@oracle.com> Message-ID: <00038522-9EC7-4491-9AE4-73BD16D95909@oracle.com> > On Jun 4, 2015, at 8:49 AM, Dmitry Samersoff wrote: > > Mandy, > > Webrev updated in-place (press shift-reload). > > http://cr.openjdk.java.net/~dsamersoff/JDK-8059036/webrev.14 > > getFinalizerHistogram() now returns Entry[] > > @library and @build removed from the test. Looks fine. Thanks Mandy > > -Dmitry > > On 2015-06-04 16:56, Mandy Chung wrote: >> >>> On Jun 4, 2015, at 6:38 AM, Dmitry Samersoff wrote: >>> >>> Mandy, >>> >>> Thank you for the review. >>> >>> Updated webrev is: >>> >>> http://cr.openjdk.java.net/~dsamersoff/JDK-8059036/webrev.14 >>> >> >> Thanks for the update and the test. >> >>> addressed comments, added a unit test to JDK workspace. >>> >> >> This test does not need @library and @build, right? >> >>> >>> On 2015-06-03 21:36, Mandy Chung wrote: >>> >>>> Should getFinalizerHistogram() return FinalizerHistogramEntry[]? >>>> The VM knows the returned type anyway. >>> >>> "[java/lang/ref/Entry" signature would need a separate symbol entry in >>> swollen vmSymbols::init() so I would prefer to stay with more generic >>> Object[] >>> >> >> You already add several symbols - why is it an issue for another one? Or if adding vm symbols is a concern, you should revert to String and int[] that you decide not to. The decision should apply consistently (use String and int, or new Java type). >> >> >>>> Perhaps the getFinalizerHistogram method should be renamed >>>> e.g. "dump"? >>> >>> The line in vmSymbols looks like: >>> >>> template( >>> get_finalizer_histogram_name, "getFinalizerHistogram") >>> >>> I would prefer to keep method name specific enough to be able to >>> find it by grep in jdk code. >> >> >> Grep in jdk code is easy as you have a new class :) >> >> Mandy >> >>> >>> (other comments are addressed) >>> >>> -Dmitry >>> >>> >>> On 2015-06-03 21:36, Mandy Chung wrote: >>>> >>>> >>>> On 06/03/2015 08:29 AM, Dmitry Samersoff wrote: >>>>> Updated webrev: >>>>> >>>>> http://cr.openjdk.java.net/~dsamersoff/JDK-8059036/webrev.13 >>>> >>>> I reviewed the jdk change. The version looks okay and some comments: >>>> >>>> ReferenceQueue.java - you should eliminate the use of rawtype: >>>> >>>> 84 Reference rn = r.next; >>>> >>>> Change Reference to Reference >>> >>> done. >>> >>>> >>>> The forEach method - eliminate the use of raw type and >>>> move @SuppressWarning to the field variable in line 182: >>>> >>>> @SuppressWarnings("unchecked") >>>> Reference rn = r.next; >>> >>> done. >>> >>>> >>>> FinalizerHistogram.java >>>> Copyright year needs update. >>> >>> done. >>>> >>>> I personally think the VM code would be much simpler if you stay with >>>> alternative entry of String and int than dealing with a >>>> FinalizerHistogramEntry private class. It's okay as FinalizerHistogram >>>> class is separated. >>>> >>>> The comment in line 35 is suitable to move to the class description and >>>> this is the suggestion. >>>> >>>> // This FinalizerHistogram class is for GC.finalizer_info diagnostic >>>> command support. >>>> // It is invoked by the VM. >>> >>> done. >>> >>>> >>>> Should getFinalizerHistogram() return FinalizerHistogramEntry[]? The VM >>>> knows the returned type anyway. >>> >>> "[java/lang/ref/Entry" signature would need a separate symbol entry in >>> swollen vmSymbols::init() so I would prefer to stay with more generic >>> Object[] >>> >>>> It's an inner class of >>>> FinalizerHistogram and it can simply be named as "Entry". I suggest to >>>> have Entry::increment method to replace ".instanceCount++". >>> >>> done. >>> >>>> >>>> The tests are in hotspot/test. Can you add a unit test in jdk/test? >>>> Perhaps you can test FinalizerHistogram.getFinalizerHistogram() via >>>> reflection - just a sanity test. >>> >>> done. The test repeats hotspot one, but uses reflection instead of VM call. >>> >>>> >>>> A minor naming comment - now you have a FinalizerHistogram class. >>>> Perhaps the getFinalizerHistogram method should be renamed e.g. "dump"? >>> >>> The line in vmSymbols looks like: >>> >>> template( >>> get_finalizer_histogram_name, "getFinalizerHistogram") >>> >>> I would prefer to keep it specific enough to be able to >>> find it by grep in jdk code. >>> >>> >>> -- >>> Dmitry Samersoff >>> Oracle Java development team, Saint Petersburg, Russia >>> * I would love to change the world, but they won't give me the sources. >> > > > -- > Dmitry Samersoff > Oracle Java development team, Saint Petersburg, Russia > * I would love to change the world, but they won't give me the sources. From spliterator at gmail.com Thu Jun 4 22:57:10 2015 From: spliterator at gmail.com (Stefan Zobel) Date: Fri, 5 Jun 2015 00:57:10 +0200 Subject: RFR 8072773 (fs) Files.lines needs a better splitting implementation for stream source In-Reply-To: <7CF56CF1-7A27-475B-81D3-8B784B785247@oracle.com> References: <556F28EF.1000201@oracle.com> <7CF56CF1-7A27-475B-81D3-8B784B785247@oracle.com> Message-ID: With respect to the 'stream test library not closing streams' bug: would you mind to create a separate bugtracker issue for that, or do you think it's too special to carry that weight? Thanks, Stefan 2015-06-04 22:06 GMT+02:00 Paul Sandoz : > On Jun 3, 2015, at 6:18 PM, Alan Bateman wrote: > > As this code path is only for the default provider case then there's a > good chance that it will be a FileChannelImpl, in which case you can call > its unmap method (directly or via a shared secret). It is possible to > interpose on the default provider so you can't be guaranteed it is a > FileChannelImpl of course. > > > > Not yet done. I would like to revisit this as a separate issue if i may. > > > > In passing, you might consider moving ByteBufferLinesSpliterator to its > own source file because Files is getting very big. > > > > Done. > > I am going to revisit the set of support charsets at another time. For the > moment i have limited to a subset of standard charsets. > > Also the file lines test tickled a bug in the stream test library, it was > not closing streams, which becomes an issue when ~15000 tests are executed > leaving the process with no more file descriptors: > > http://cr.openjdk.java.net/~psandoz/jdk9/JDK-8072773-File-lines/webrev/ > > Paul. > From chris.plummer at oracle.com Thu Jun 4 23:32:26 2015 From: chris.plummer at oracle.com (Chris Plummer) Date: Thu, 04 Jun 2015 16:32:26 -0700 Subject: [9] RFR (M) 8054386: Allow Java debugging when CDS is enabled In-Reply-To: <556FF05F.6060300@oracle.com> References: <555B47EA.3000900@oracle.com> <556D5CFD.1020709@oracle.com> <556FF05F.6060300@oracle.com> Message-ID: <5570E00A.8070400@oracle.com> Hi David, Here's an updated webrev: http://cr.openjdk.java.net/~cjplummer/8054386/webrev.04/ thanks, Chris On 6/3/15 11:29 PM, David Holmes wrote: > Hi Chris, > > Hotspot change is good. > > Only a couple of style nits with the tests (where are our Java style > guidelines ???). Taking CDSJDITest.java as an example: > > If you are okay with this line length: > > 115 public static OutputAnalyzer executeAndLog(ProcessBuilder pb, > String logName) throws Exception { > > then you can remove a number of line breaks in the headers of other > functions. (I don't follow the 70-80 char line length dogma ;) ) > > If you do break a header of a function the { still stays on the same > line as the last header component ie: > > private static void addToClassList(PrintStream ps, String classes[]) > throws IOException { > > not: > > 139 private static void addToClassList(PrintStream ps, String > classes[]) > 140 throws IOException > 141 { > > Cheers, > David > > On 2/06/2015 5:36 PM, Chris Plummer wrote: >> [Adding core-libs-dev at openjdk.java.net since this update includes >> changes to jdk/test library code] >> >> Please review the updated webrev: >> >> Webrev: http://cr.openjdk.java.net/~cjplummer/8054386/webrev.02/ >> Bug: https://bugs.openjdk.java.net/browse/JDK-8054386 >> >> There were concerns about the new hotspot tests referencing jdk tests. >> One concern was that if the jdk tests change, they could break the >> hotspot tests, and this might initially go undetected. The other concern >> is that if the jdk and hotspot tests are placed in separate test >> bundles, then it would not be possible to run the hotspot tests. >> >> Because of these concerns, I moved the tests that were in >> hotspot/test/runtime/CDSJDITests to instead be in >> jdk/test/com/sun/jdi/CDSJDITests. There was a slight renaming of the >> tests in the process. Also, I had to update the jdk version of >> ProcessTool.java to include the createJavaProcessBuilder() variant that >> is in the hotspot version of ProcessTool.java. >> >> Lastly, in CDSJITTest.java I changed: >> >> OutputAnalyzer output = new OutputAnalyzer(pb.start()); >> >> to instead be: >> >> OutputAnalyzer output = ProcessTools.executeProcess(pb); >> >> I had to do this since the jdk version of the OutputAnalyzer constructor >> is not public. The 1st version is what is commonly used in hostspot >> tests, and the 2nd version is what is commonly used in jdk tests. I >> decided to adopt the jdk way rather than make the OutputAnalyzer >> constructors public, although this will probably happen eventually when >> the two versions are unified. >> >> thanks, >> >> Chris >> >> >> On 5/19/15 7:25 AM, Chris Plummer wrote: >>> Hi, >>> >>> Please review the following changes for allowing java debugging when >>> CDS is enabled. >>> >>> Webrev:http://cr.openjdk.java.net/~cjplummer/8054386/webrev.01/ >>> Bug:https://bugs.openjdk.java.net/browse/JDK-8054386 >>> >>> The VM changes are simple. I removed the check that prevents debugging >>> with CDS enabled, and added logic that will map the CDS archive RW >>> when debugging is enabled. >>> >>> The tests are a bit more complex. There are a bunch of existing JDI >>> tests for testing debugging support. Rather than start from scratch or >>> clone them, I instead just wrote wrapper tests that put the relevant >>> JDI test classes in the archive, and then invoke the JDI test. I did >>> this for 3 of the JDI tests. If you feel there are others that would >>> be good candidates, I'd be happy to add them. I'm looking for ones >>> that would result in modification of the RO class metadata, such as >>> setting a breakpoint (for which I already added two tests). >>> >>> Testing done: >>> -Using JPRT to run the new jtreg tests on all platforms. >>> -Using JPRT to run all jtreg runtime tests on linux x86 and x_64. >>> -Regular JPRT "-testset hotspot" run >>> -Putting the JCK JVMTI tests in the archive and then running them. >>> -Putting the nsk jdb, jdwp, jvmti, and jdi tests in the archive and >>> then running them. >>> -Putting a simple test class in the archive and then setting a >>> breakpoint on it using jdb >>> >>> Some of the above testing resulted in the discovery of bugs that still >>> need to be addressed: JDK-8078644, JDK-8078730, and JDK-8079181. >>> >>> I also verified that without the change to map the archive RW, the >>> above testing resulted in a SEGV, which is what you would expect (and >>> actually want to see to prove that the testing is effective). >>> >>> thanks, >>> >>> Chris >>> >> From serguei.spitsyn at oracle.com Fri Jun 5 00:40:41 2015 From: serguei.spitsyn at oracle.com (serguei.spitsyn at oracle.com) Date: Thu, 04 Jun 2015 17:40:41 -0700 Subject: [9] RFR (M) 8054386: Allow Java debugging when CDS is enabled In-Reply-To: <5570E00A.8070400@oracle.com> References: <555B47EA.3000900@oracle.com> <556D5CFD.1020709@oracle.com> <556FF05F.6060300@oracle.com> <5570E00A.8070400@oracle.com> Message-ID: <5570F009.7040003@oracle.com> I guess, there is no need to re-review. It looks good anyway. Thanks, Serguei On 6/4/15 4:32 PM, Chris Plummer wrote: > Hi David, > > Here's an updated webrev: > > http://cr.openjdk.java.net/~cjplummer/8054386/webrev.04/ > > thanks, > > Chris > > On 6/3/15 11:29 PM, David Holmes wrote: >> Hi Chris, >> >> Hotspot change is good. >> >> Only a couple of style nits with the tests (where are our Java style >> guidelines ???). Taking CDSJDITest.java as an example: >> >> If you are okay with this line length: >> >> 115 public static OutputAnalyzer executeAndLog(ProcessBuilder >> pb, String logName) throws Exception { >> >> then you can remove a number of line breaks in the headers of other >> functions. (I don't follow the 70-80 char line length dogma ;) ) >> >> If you do break a header of a function the { still stays on the same >> line as the last header component ie: >> >> private static void addToClassList(PrintStream ps, String >> classes[]) >> throws IOException { >> >> not: >> >> 139 private static void addToClassList(PrintStream ps, String >> classes[]) >> 140 throws IOException >> 141 { >> >> Cheers, >> David >> >> On 2/06/2015 5:36 PM, Chris Plummer wrote: >>> [Adding core-libs-dev at openjdk.java.net since this update includes >>> changes to jdk/test library code] >>> >>> Please review the updated webrev: >>> >>> Webrev: http://cr.openjdk.java.net/~cjplummer/8054386/webrev.02/ >>> Bug: https://bugs.openjdk.java.net/browse/JDK-8054386 >>> >>> There were concerns about the new hotspot tests referencing jdk tests. >>> One concern was that if the jdk tests change, they could break the >>> hotspot tests, and this might initially go undetected. The other >>> concern >>> is that if the jdk and hotspot tests are placed in separate test >>> bundles, then it would not be possible to run the hotspot tests. >>> >>> Because of these concerns, I moved the tests that were in >>> hotspot/test/runtime/CDSJDITests to instead be in >>> jdk/test/com/sun/jdi/CDSJDITests. There was a slight renaming of the >>> tests in the process. Also, I had to update the jdk version of >>> ProcessTool.java to include the createJavaProcessBuilder() variant that >>> is in the hotspot version of ProcessTool.java. >>> >>> Lastly, in CDSJITTest.java I changed: >>> >>> OutputAnalyzer output = new OutputAnalyzer(pb.start()); >>> >>> to instead be: >>> >>> OutputAnalyzer output = ProcessTools.executeProcess(pb); >>> >>> I had to do this since the jdk version of the OutputAnalyzer >>> constructor >>> is not public. The 1st version is what is commonly used in hostspot >>> tests, and the 2nd version is what is commonly used in jdk tests. I >>> decided to adopt the jdk way rather than make the OutputAnalyzer >>> constructors public, although this will probably happen eventually when >>> the two versions are unified. >>> >>> thanks, >>> >>> Chris >>> >>> >>> On 5/19/15 7:25 AM, Chris Plummer wrote: >>>> Hi, >>>> >>>> Please review the following changes for allowing java debugging when >>>> CDS is enabled. >>>> >>>> Webrev:http://cr.openjdk.java.net/~cjplummer/8054386/webrev.01/ >>>> Bug:https://bugs.openjdk.java.net/browse/JDK-8054386 >>>> >>>> The VM changes are simple. I removed the check that prevents debugging >>>> with CDS enabled, and added logic that will map the CDS archive RW >>>> when debugging is enabled. >>>> >>>> The tests are a bit more complex. There are a bunch of existing JDI >>>> tests for testing debugging support. Rather than start from scratch or >>>> clone them, I instead just wrote wrapper tests that put the relevant >>>> JDI test classes in the archive, and then invoke the JDI test. I did >>>> this for 3 of the JDI tests. If you feel there are others that would >>>> be good candidates, I'd be happy to add them. I'm looking for ones >>>> that would result in modification of the RO class metadata, such as >>>> setting a breakpoint (for which I already added two tests). >>>> >>>> Testing done: >>>> -Using JPRT to run the new jtreg tests on all platforms. >>>> -Using JPRT to run all jtreg runtime tests on linux x86 and x_64. >>>> -Regular JPRT "-testset hotspot" run >>>> -Putting the JCK JVMTI tests in the archive and then running them. >>>> -Putting the nsk jdb, jdwp, jvmti, and jdi tests in the archive and >>>> then running them. >>>> -Putting a simple test class in the archive and then setting a >>>> breakpoint on it using jdb >>>> >>>> Some of the above testing resulted in the discovery of bugs that still >>>> need to be addressed: JDK-8078644, JDK-8078730, and JDK-8079181. >>>> >>>> I also verified that without the change to map the archive RW, the >>>> above testing resulted in a SEGV, which is what you would expect (and >>>> actually want to see to prove that the testing is effective). >>>> >>>> thanks, >>>> >>>> Chris >>>> >>> > From david.holmes at oracle.com Fri Jun 5 05:50:13 2015 From: david.holmes at oracle.com (David Holmes) Date: Fri, 05 Jun 2015 15:50:13 +1000 Subject: [9] RFR (M) 8054386: Allow Java debugging when CDS is enabled In-Reply-To: <5570E00A.8070400@oracle.com> References: <555B47EA.3000900@oracle.com> <556D5CFD.1020709@oracle.com> <556FF05F.6060300@oracle.com> <5570E00A.8070400@oracle.com> Message-ID: <55713895.4040601@oracle.com> On 5/06/2015 9:32 AM, Chris Plummer wrote: > Hi David, > > Here's an updated webrev: > > http://cr.openjdk.java.net/~cjplummer/8054386/webrev.04/ Thanks - should have said updated webrev not necessary :) Looks good. David > thanks, > > Chris > > On 6/3/15 11:29 PM, David Holmes wrote: >> Hi Chris, >> >> Hotspot change is good. >> >> Only a couple of style nits with the tests (where are our Java style >> guidelines ???). Taking CDSJDITest.java as an example: >> >> If you are okay with this line length: >> >> 115 public static OutputAnalyzer executeAndLog(ProcessBuilder pb, >> String logName) throws Exception { >> >> then you can remove a number of line breaks in the headers of other >> functions. (I don't follow the 70-80 char line length dogma ;) ) >> >> If you do break a header of a function the { still stays on the same >> line as the last header component ie: >> >> private static void addToClassList(PrintStream ps, String classes[]) >> throws IOException { >> >> not: >> >> 139 private static void addToClassList(PrintStream ps, String >> classes[]) >> 140 throws IOException >> 141 { >> >> Cheers, >> David >> >> On 2/06/2015 5:36 PM, Chris Plummer wrote: >>> [Adding core-libs-dev at openjdk.java.net since this update includes >>> changes to jdk/test library code] >>> >>> Please review the updated webrev: >>> >>> Webrev: http://cr.openjdk.java.net/~cjplummer/8054386/webrev.02/ >>> Bug: https://bugs.openjdk.java.net/browse/JDK-8054386 >>> >>> There were concerns about the new hotspot tests referencing jdk tests. >>> One concern was that if the jdk tests change, they could break the >>> hotspot tests, and this might initially go undetected. The other concern >>> is that if the jdk and hotspot tests are placed in separate test >>> bundles, then it would not be possible to run the hotspot tests. >>> >>> Because of these concerns, I moved the tests that were in >>> hotspot/test/runtime/CDSJDITests to instead be in >>> jdk/test/com/sun/jdi/CDSJDITests. There was a slight renaming of the >>> tests in the process. Also, I had to update the jdk version of >>> ProcessTool.java to include the createJavaProcessBuilder() variant that >>> is in the hotspot version of ProcessTool.java. >>> >>> Lastly, in CDSJITTest.java I changed: >>> >>> OutputAnalyzer output = new OutputAnalyzer(pb.start()); >>> >>> to instead be: >>> >>> OutputAnalyzer output = ProcessTools.executeProcess(pb); >>> >>> I had to do this since the jdk version of the OutputAnalyzer constructor >>> is not public. The 1st version is what is commonly used in hostspot >>> tests, and the 2nd version is what is commonly used in jdk tests. I >>> decided to adopt the jdk way rather than make the OutputAnalyzer >>> constructors public, although this will probably happen eventually when >>> the two versions are unified. >>> >>> thanks, >>> >>> Chris >>> >>> >>> On 5/19/15 7:25 AM, Chris Plummer wrote: >>>> Hi, >>>> >>>> Please review the following changes for allowing java debugging when >>>> CDS is enabled. >>>> >>>> Webrev:http://cr.openjdk.java.net/~cjplummer/8054386/webrev.01/ >>>> Bug:https://bugs.openjdk.java.net/browse/JDK-8054386 >>>> >>>> The VM changes are simple. I removed the check that prevents debugging >>>> with CDS enabled, and added logic that will map the CDS archive RW >>>> when debugging is enabled. >>>> >>>> The tests are a bit more complex. There are a bunch of existing JDI >>>> tests for testing debugging support. Rather than start from scratch or >>>> clone them, I instead just wrote wrapper tests that put the relevant >>>> JDI test classes in the archive, and then invoke the JDI test. I did >>>> this for 3 of the JDI tests. If you feel there are others that would >>>> be good candidates, I'd be happy to add them. I'm looking for ones >>>> that would result in modification of the RO class metadata, such as >>>> setting a breakpoint (for which I already added two tests). >>>> >>>> Testing done: >>>> -Using JPRT to run the new jtreg tests on all platforms. >>>> -Using JPRT to run all jtreg runtime tests on linux x86 and x_64. >>>> -Regular JPRT "-testset hotspot" run >>>> -Putting the JCK JVMTI tests in the archive and then running them. >>>> -Putting the nsk jdb, jdwp, jvmti, and jdi tests in the archive and >>>> then running them. >>>> -Putting a simple test class in the archive and then setting a >>>> breakpoint on it using jdb >>>> >>>> Some of the above testing resulted in the discovery of bugs that still >>>> need to be addressed: JDK-8078644, JDK-8078730, and JDK-8079181. >>>> >>>> I also verified that without the change to map the archive RW, the >>>> above testing resulted in a SEGV, which is what you would expect (and >>>> actually want to see to prove that the testing is effective). >>>> >>>> thanks, >>>> >>>> Chris >>>> >>> > From amy.lu at oracle.com Fri Jun 5 05:52:42 2015 From: amy.lu at oracle.com (Amy Lu) Date: Fri, 05 Jun 2015 13:52:42 +0800 Subject: JDK 9 RFR of JDK-8085879: Mark intermittently failing: java/util/Arrays/ParallelPrefix.java Message-ID: <5571392A.2030202@oracle.com> java/util/Arrays/ParallelPrefix.java This test has been observed to fail intermittently. This fix is to mark it accordingly with keyword 'intermittent'. bug: https://bugs.openjdk.java.net/browse/JDK-8085879 webrev: http://cr.openjdk.java.net/~amlu/8085879/webrev.00/ Thanks, Amy --- old/test/java/util/Arrays/ParallelPrefix.java 2015-06-05 13:44:24.000000000 +0800 +++ new/test/java/util/Arrays/ParallelPrefix.java 2015-06-05 13:44:24.000000000 +0800 @@ -26,6 +26,7 @@ * @summary unit test for Arrays.ParallelPrefix(). * @author Tristan Yan * @run testng ParallelPrefix + * @key intermittent */ import java.util.Arrays; From david.holmes at oracle.com Fri Jun 5 05:57:03 2015 From: david.holmes at oracle.com (David Holmes) Date: Fri, 05 Jun 2015 15:57:03 +1000 Subject: [9] RFR (XS) JDK-8081771: ProcessTool.createJavaProcessBuilder() needs new addTestVmAndJavaOptions argument In-Reply-To: <5570E075.4030008@oracle.com> References: <556E7263.4050601@oracle.com> <556FEA7D.2040107@oracle.com> <556FF0D7.3030207@oracle.com> <5570E075.4030008@oracle.com> Message-ID: <55713A2F.2030103@oracle.com> Hi Chris, On 5/06/2015 9:34 AM, Chris Plummer wrote: > On 6/3/15 11:31 PM, David Holmes wrote: >> Typo ... >> >> On 4/06/2015 4:04 PM, David Holmes wrote: >>> Hi Chris, >>> >>> On 3/06/2015 1:20 PM, Chris Plummer wrote: >>>> Please review the following: >>>> >>>> Webrev: http://cr.openjdk.java.net/~cjplummer/8054386/webrev.02/ >>>> Bug: https://bugs.openjdk.java.net/browse/JDK-8081771 >>>> >>>> This review only concerns the changes to ProcessTool.java. The >>> >>> Your new method needs javadoc explaining its usage, and in particular >>> what addTestVmAndJavaOptions actually refers to. Also a comment on why >>> the explicit -cp is added, but on under that arg, would be useful. >> >> but only under that arg ... > Hi David, > > Here's an updated webrev: > > http://cr.openjdk.java.net/~cjplummer/8054386/webrev.04/ > > Note, this is the first time I've ever done javadocs, and I'm not even > sure how to build the javadocs to make sure it formats correctly. So if > anything looks suspicious and you think needs verification, please let > me know how. I've never built javadocs for the tests/testlibrary either, but simply read them as code comments. What you have done is consistent with the other doc comments in the file (which have some style inconsistencies with other JDK code, but not enough to worry about here). The comment on -cp is somewhat longer than I had anticipated - might be one of the rare occasions where a reference to the CR would be better. But unless someone else complains ... Thanks, David > thanks, > > Chris >> >> David >> >>> Thanks, >>> David >>> >>> >>> >>>> CDSJDITests and filemapp.cpp changes will be committed under CR >>>> JDK-8054386, but they rely on this change to ProcessTool.java, so both >>>> CRs will be pushed together. See the following thread for the >>>> JDK-8054386 review: >>>> >>>> http://mail.openjdk.java.net/pipermail/hotspot-runtime-dev/2015-June/014923.html >>>> >>>> >>>> >>>> >>>> thanks, >>>> >>>> Chris > From joe.darcy at oracle.com Fri Jun 5 06:15:52 2015 From: joe.darcy at oracle.com (joe darcy) Date: Thu, 04 Jun 2015 23:15:52 -0700 Subject: JDK 9 RFR of JDK-8085879: Mark intermittently failing: java/util/Arrays/ParallelPrefix.java In-Reply-To: <5571392A.2030202@oracle.com> References: <5571392A.2030202@oracle.com> Message-ID: <55713E98.9000101@oracle.com> Look good; thanks, -Joe On 6/4/2015 10:52 PM, Amy Lu wrote: > > > This test has been observed to fail intermittently. This fix is to > mark it accordingly with keyword 'intermittent'. > > bug: https://bugs.openjdk.java.net/browse/JDK-8085879 > webrev: http://cr.openjdk.java.net/~amlu/8085879/webrev.00/ > > Thanks, > Amy > > --- old/test/java/util/Arrays/ParallelPrefix.java 2015-06-05 > 13:44:24.000000000 +0800 > +++ new/test/java/util/Arrays/ParallelPrefix.java 2015-06-05 > 13:44:24.000000000 +0800 > @@ -26,6 +26,7 @@ > * @summary unit test for Arrays.ParallelPrefix(). > * @author Tristan Yan > * @run testng ParallelPrefix > + * @key intermittent From elena.votchennikova at oracle.com Fri Jun 5 07:26:20 2015 From: elena.votchennikova at oracle.com (Elena Votchennikova) Date: Fri, 05 Jun 2015 10:26:20 +0300 Subject: RFR JDK-8054304: Clarify treatment of bounds in j.l.r.Annotated{WildcardType, TypeVariable} In-Reply-To: <55702597.6050907@oracle.com> References: <5513AD47.2080407@oracle.com> <55702597.6050907@oracle.com> Message-ID: <55714F1C.3040003@oracle.com> Hi Srikanth, I would mention that Object is not annotated in the following statements: - AnnotatedWildcardType.getAnnotatedUpperBounds(): "Note that if no upper bound is explicitly declared, the upper bound is Object." - AnnotatedTypeVariable.getAnnotatedBounds(): "Note that if no bound is explicitly declared, the bound is Object." - TypeVariable.getAnnotatedBounds(): "Note that if no upper bound is explicitly declared, the upper bound is Object." Thank you, Elena 04.06.2015 13:16, Srikanth wrote: > Hello, > > Please review this fix for > https://bugs.openjdk.java.net/browse/JDK-8054304 > (twin for conformance test failures: > https://bugs.openjdk.java.net/browse/JDK-8058595) > > JBS: > > https://bugs.openjdk.java.net/browse/JDK-8054304 > > Webrev: > > http://cr.openjdk.java.net/~sadayapalam/JDK-8054304/webrev.00/ > > CCC: > > http://ccc.us.oracle.com/8054304 > > While as outlined in https://bugs.openjdk.java.net/browse/JDK-8054304, > the javadoc > of several methods is adjusted, only in one case viz > > AnnotatedWildcardType.getAnnotatedUpperBounds() > > there is a behaviour change. All other cases involve javadoc change > that explicitly > spell out current & correct behaviour. > > The patch passes Tier 1 & Tier 2 tests for JDK (i.e identical results > with and without) > > Our plan is to fix this for JDK9 and backport to 8u60. I don't know at > the moment > if there will be objections for the backport from a process p.o.v as > this calls for > a specification change. > > Thanks! > Srikanth. From frank.yuan at oracle.com Fri Jun 5 07:45:39 2015 From: frank.yuan at oracle.com (Frank Yuan) Date: Fri, 5 Jun 2015 15:45:39 +0800 Subject: Review request for JDK-8080906 & JDK-8080908: Develop tests for JEP 255 Xerces Updates Message-ID: <013b01d09f63$a0d1ff40$e275fdc0$@oracle.com> Hi Joe and all I have been working on the test task of JEP 255 Xerces Updates. Here I would invite you to review the changes for 2 bugs of this task: 1. JDK-8080906 Develop test for Xerces Update: DOM L3 Serializer To verify default LSSerializer is Xalan dom 3 serializer 2. JDK-8080908 Develop test for Xerces Update: XPointer To verify Xerces revision 415823: XERCESJ-1134. It should have been fixed in JDK-6794483, however, the test Bug6794483Test.java need to be revised to really cover this bug. I also added module dependencies for exported API in this path, it's recommended for Jigsaw change, refer to https://wiki.se.oracle.com/display/JPG/@modules+in+JTReg+tests The webrev is at http://cr.openjdk.java.net/~fyuan/8080906_8080908/webrev.00/, your comment will be appreciated. Best Regards Frank From staffan.larsen at oracle.com Fri Jun 5 08:20:13 2015 From: staffan.larsen at oracle.com (Staffan Larsen) Date: Fri, 5 Jun 2015 10:20:13 +0200 Subject: RFR(M, v14): JDK-8059036 : Implement Diagnostic Commands for heap and finalizerinfo In-Reply-To: <00038522-9EC7-4491-9AE4-73BD16D95909@oracle.com> References: <554894E9.8020908@oracle.com> <554A8CEE.2090009@oracle.com> <55523418.5010708@oracle.com> <55527935.5010408@gmail.com> <5553B552.9060900@oracle.com> <555453BA.2070205@gmail.com> <55567748.6020304@oracle.com> <55567A68.9020802@oracle.com> <5556F34B.5050701@oracle.com> <55573A31.7050306@gmail.com> <55573C86.8030807@gmail.com> <5559D85B.2050500@oracle.com> <6FB8DF19-1D67-4702-9642-4C630765D2D1@oracle.com> <555C2EEB.1050505@oracle.com> <3941B430-FE6C-4AF0-A875-76D24411C654@oracle.com> <5564802A.2010403@oracle.com> <5565738A.4040109@gmail.com> <7EB0BE71-1608-4047-B71D-1A9EAA9083BB@oracle.com> <556727A2.9060209@gmail.com> <55675908.6030406@oracle.com> <556AF3CD.50900@oracle.com> <37227441-B016-4A9A-B9CA-BE100E992C9A@oracle.com> <556D8FB0.4070608@oracle.com> <556E9970.8010704@gmail.com> <556EBF4B.6070400@oracle.com> <556F1D6C.6010806@oracle.com> <556F491E.3070709@oracle.com> <557054C7.4030705@oracle.com> <3607C2B6-F247-4291-9198-279CF8ED2AF3@oracle.com> <5570738B.504060! 3@oracle.com> <00038522-9EC7-4491-9AE4-73BD16D95909@oracle.com> Message-ID: Dmitry, I?d like to propose the following change to get prettier output (more in line with GC.class_histogram): diff --git a/src/share/vm/services/diagnosticCommand.cpp b/src/share/vm/services/diagnosticCommand.cpp --- a/src/share/vm/services/diagnosticCommand.cpp +++ b/src/share/vm/services/diagnosticCommand.cpp @@ -341,7 +341,6 @@ void FinalizerInfoDCmd::execute(DCmdSource source, TRAPS) { ResourceMark rm; - output()->print_cr("Unreachable instances awaiting finalization:"); Klass* k = SystemDictionary::resolve_or_null( vmSymbols::finalizer_histogram_klass(), THREAD); assert(k != NULL, "FinalizerHistogram class is not accessible"); @@ -375,12 +374,15 @@ assert(count_res != NULL && name_res != NULL, "Unexpected layout of FinalizerHistogramEntry"); + output()->print_cr("Unreachable instances waiting for finalization"); + output()->print_cr("#instances class name"); + output()->print_cr("-----------------------"); for (int i = 0; i < result_oop->length(); ++i) { oop element_oop = result_oop->obj_at(i); oop str_oop = element_oop->obj_field(name_fd.offset()); char *name = java_lang_String::as_utf8_string(str_oop); int count = element_oop->int_field(count_fd.offset()); - output()->print_cr("Class %s - %d", name, count); + output()->print_cr("%10d %s", count, name); } } A couple of nits: diagnosticCommand.cpp:359 - extra space after = diagnosticCommand.cpp:361 - spelling: finlalization -> finalization FinalizerInfoTest.java:69: - spelling: intialized -> initialized FinalizerInfoTest.java:71 - I?d like to see the ; on a new line Thanks, /Staffan > On 5 jun 2015, at 00:20, Mandy Chung wrote: > > >> On Jun 4, 2015, at 8:49 AM, Dmitry Samersoff wrote: >> >> Mandy, >> >> Webrev updated in-place (press shift-reload). >> >> http://cr.openjdk.java.net/~dsamersoff/JDK-8059036/webrev.14 >> >> getFinalizerHistogram() now returns Entry[] >> >> @library and @build removed from the test. > > > Looks fine. > > Thanks > Mandy > >> >> -Dmitry >> >> On 2015-06-04 16:56, Mandy Chung wrote: >>> >>>> On Jun 4, 2015, at 6:38 AM, Dmitry Samersoff wrote: >>>> >>>> Mandy, >>>> >>>> Thank you for the review. >>>> >>>> Updated webrev is: >>>> >>>> http://cr.openjdk.java.net/~dsamersoff/JDK-8059036/webrev.14 >>>> >>> >>> Thanks for the update and the test. >>> >>>> addressed comments, added a unit test to JDK workspace. >>>> >>> >>> This test does not need @library and @build, right? >>> >>>> >>>> On 2015-06-03 21:36, Mandy Chung wrote: >>>> >>>>> Should getFinalizerHistogram() return FinalizerHistogramEntry[]? >>>>> The VM knows the returned type anyway. >>>> >>>> "[java/lang/ref/Entry" signature would need a separate symbol entry in >>>> swollen vmSymbols::init() so I would prefer to stay with more generic >>>> Object[] >>>> >>> >>> You already add several symbols - why is it an issue for another one? Or if adding vm symbols is a concern, you should revert to String and int[] that you decide not to. The decision should apply consistently (use String and int, or new Java type). >>> >>> >>>>> Perhaps the getFinalizerHistogram method should be renamed >>>>> e.g. "dump"? >>>> >>>> The line in vmSymbols looks like: >>>> >>>> template( >>>> get_finalizer_histogram_name, "getFinalizerHistogram") >>>> >>>> I would prefer to keep method name specific enough to be able to >>>> find it by grep in jdk code. >>> >>> >>> Grep in jdk code is easy as you have a new class :) >>> >>> Mandy >>> >>>> >>>> (other comments are addressed) >>>> >>>> -Dmitry >>>> >>>> >>>> On 2015-06-03 21:36, Mandy Chung wrote: >>>>> >>>>> >>>>> On 06/03/2015 08:29 AM, Dmitry Samersoff wrote: >>>>>> Updated webrev: >>>>>> >>>>>> http://cr.openjdk.java.net/~dsamersoff/JDK-8059036/webrev.13 >>>>> >>>>> I reviewed the jdk change. The version looks okay and some comments: >>>>> >>>>> ReferenceQueue.java - you should eliminate the use of rawtype: >>>>> >>>>> 84 Reference rn = r.next; >>>>> >>>>> Change Reference to Reference >>>> >>>> done. >>>> >>>>> >>>>> The forEach method - eliminate the use of raw type and >>>>> move @SuppressWarning to the field variable in line 182: >>>>> >>>>> @SuppressWarnings("unchecked") >>>>> Reference rn = r.next; >>>> >>>> done. >>>> >>>>> >>>>> FinalizerHistogram.java >>>>> Copyright year needs update. >>>> >>>> done. >>>>> >>>>> I personally think the VM code would be much simpler if you stay with >>>>> alternative entry of String and int than dealing with a >>>>> FinalizerHistogramEntry private class. It's okay as FinalizerHistogram >>>>> class is separated. >>>>> >>>>> The comment in line 35 is suitable to move to the class description and >>>>> this is the suggestion. >>>>> >>>>> // This FinalizerHistogram class is for GC.finalizer_info diagnostic >>>>> command support. >>>>> // It is invoked by the VM. >>>> >>>> done. >>>> >>>>> >>>>> Should getFinalizerHistogram() return FinalizerHistogramEntry[]? The VM >>>>> knows the returned type anyway. >>>> >>>> "[java/lang/ref/Entry" signature would need a separate symbol entry in >>>> swollen vmSymbols::init() so I would prefer to stay with more generic >>>> Object[] >>>> >>>>> It's an inner class of >>>>> FinalizerHistogram and it can simply be named as "Entry". I suggest to >>>>> have Entry::increment method to replace ".instanceCount++". >>>> >>>> done. >>>> >>>>> >>>>> The tests are in hotspot/test. Can you add a unit test in jdk/test? >>>>> Perhaps you can test FinalizerHistogram.getFinalizerHistogram() via >>>>> reflection - just a sanity test. >>>> >>>> done. The test repeats hotspot one, but uses reflection instead of VM call. >>>> >>>>> >>>>> A minor naming comment - now you have a FinalizerHistogram class. >>>>> Perhaps the getFinalizerHistogram method should be renamed e.g. "dump"? >>>> >>>> The line in vmSymbols looks like: >>>> >>>> template( >>>> get_finalizer_histogram_name, "getFinalizerHistogram") >>>> >>>> I would prefer to keep it specific enough to be able to >>>> find it by grep in jdk code. >>>> >>>> >>>> -- >>>> Dmitry Samersoff >>>> Oracle Java development team, Saint Petersburg, Russia >>>> * I would love to change the world, but they won't give me the sources. >>> >> >> >> -- >> Dmitry Samersoff >> Oracle Java development team, Saint Petersburg, Russia >> * I would love to change the world, but they won't give me the sources. > From forax at univ-mlv.fr Fri Jun 5 09:15:19 2015 From: forax at univ-mlv.fr (Remi Forax) Date: Fri, 05 Jun 2015 11:15:19 +0200 Subject: RFR 8071597 Add Stream dropWhile and takeWhile operations In-Reply-To: <123E29BA-B40F-423E-BAD0-6D3D0F80962C@oracle.com> References: <79392DA7-6A8E-42AC-9E83-AA7F89D8F2A5@oracle.com> <556FF896.6090808@univ-mlv.fr> <5975E711-D526-495F-8A29-184CEDC59D0E@oracle.com> <557007D0.1080700@univ-mlv.fr> <123E29BA-B40F-423E-BAD0-6D3D0F80962C@oracle.com> Message-ID: <557168A7.1070802@univ-mlv.fr> On 06/04/2015 10:34 AM, Paul Sandoz wrote: > On Jun 4, 2015, at 10:09 AM, Remi Forax wrote: > >> On 06/04/2015 09:37 AM, Paul Sandoz wrote: >>> On Jun 4, 2015, at 9:04 AM, Remi Forax wrote: >>>> Thinking a little more about dropWhile(), >>>> it can be written using filter() more or less like this: >>>> default Stream dropWhile(Predicate predicate) { >>>> return filter(new Predicate<>() { >>>> private boolean noDropAnymore; >>>> public boolean test(T t) { >>>> return noDropAnymore || (noDropAnymore = !predicate.test(t)); >>>> } >>>> }); >>>> } >>>> and i maybe wrong but implementing dropWhile with an op is not better than that in term of perf so for me dropWhile() doesn't pull its own weight. >>>> >>> Try running that in parallel. > And try with an ordered parallel stream and see if it produces the same results as that produced sequentially. It won't, you have to know how the splitting is done. > > >> I'm not sure it's a good idea if you want good perf, the cost of checking a volatile flag inside the loop for each thread (I don't see how to have something more lightweight) will cost more than just keeping the stream sequential, no ? >> > You are imposing a restriction that this should *only* run sequentially. > > Stateful ops such as take/drop and limit/skip have to be used with care when executed in parallel. That does not mean we should rule it out (however, it's reasonable for default implementations of take/drop to not split, which is what is currently implemented more so for code-sharing). > > In general parallel streams should be used with care, we keep saying don't sprinkle parallel on your streams like pixie dust and expect it to work magic :-) Ok. > > >>> (Stream.parallel() affects the whole pipeline, so it's not possible to implement the default with sequential().filter(p) where p is a stateful predicate.) >> it's possible, >> the default should return a proxy in front of the result of sequential().filter(p) that return this if parallel() is called. >> > A sequential slight of hand. Ugh :-) > > Paul. R?mi From konstantin.shefov at oracle.com Fri Jun 5 09:48:36 2015 From: konstantin.shefov at oracle.com (Konstantin Shefov) Date: Fri, 05 Jun 2015 12:48:36 +0300 Subject: [9] Review request : JDK-8068416: LFGarbageCollectedTest.java fails with OOME: "GC overhead limit exceeded" Message-ID: <55717074.6010103@oracle.com> Hello, Please review the test bug fix https://bugs.openjdk.java.net/browse/JDK-8068416 Webrev is http://cr.openjdk.java.net/~kshefov/8068416/webrev.01/ Test failure is caused by JDK-8078602 [1]. Suggestion is to exclude the test until [1] is fixed. [1] https://bugs.openjdk.java.net/browse/JDK-8078602 Thanks -Konstantin From vladimir.x.ivanov at oracle.com Fri Jun 5 10:00:04 2015 From: vladimir.x.ivanov at oracle.com (Vladimir Ivanov) Date: Fri, 05 Jun 2015 13:00:04 +0300 Subject: [9] Review request : JDK-8068416: LFGarbageCollectedTest.java fails with OOME: "GC overhead limit exceeded" In-Reply-To: <55717074.6010103@oracle.com> References: <55717074.6010103@oracle.com> Message-ID: <55717324.2040703@oracle.com> + * @ignore until 8078602 is fixed The standard way to exclude a test is to mention bug ID. Use the following: @ignore 8078602 Otherwise, looks good. Best regards, Vladimir Ivanov On 6/5/15 12:48 PM, Konstantin Shefov wrote: > Hello, > > Please review the test bug fix > https://bugs.openjdk.java.net/browse/JDK-8068416 > Webrev is http://cr.openjdk.java.net/~kshefov/8068416/webrev.01/ > > Test failure is caused by JDK-8078602 [1]. Suggestion is to exclude the > test until [1] is fixed. > > [1] https://bugs.openjdk.java.net/browse/JDK-8078602 > > Thanks > -Konstantin From konstantin.shefov at oracle.com Fri Jun 5 10:05:10 2015 From: konstantin.shefov at oracle.com (Konstantin Shefov) Date: Fri, 05 Jun 2015 13:05:10 +0300 Subject: [9] Review request : JDK-8068416: LFGarbageCollectedTest.java fails with OOME: "GC overhead limit exceeded" In-Reply-To: <55717324.2040703@oracle.com> References: <55717074.6010103@oracle.com> <55717324.2040703@oracle.com> Message-ID: <55717456.3010402@oracle.com> Vladimir Thanks for reviewing Here is corrected fix: http://cr.openjdk.java.net/~kshefov/8068416/webrev.02 -Konstantin On 06/05/2015 01:00 PM, Vladimir Ivanov wrote: > + * @ignore until 8078602 is fixed > > The standard way to exclude a test is to mention bug ID. > Use the following: > @ignore 8078602 > > Otherwise, looks good. > > Best regards, > Vladimir Ivanov > > On 6/5/15 12:48 PM, Konstantin Shefov wrote: >> Hello, >> >> Please review the test bug fix >> https://bugs.openjdk.java.net/browse/JDK-8068416 >> Webrev is http://cr.openjdk.java.net/~kshefov/8068416/webrev.01/ >> >> Test failure is caused by JDK-8078602 [1]. Suggestion is to exclude the >> test until [1] is fixed. >> >> [1] https://bugs.openjdk.java.net/browse/JDK-8078602 >> >> Thanks >> -Konstantin From vladimir.x.ivanov at oracle.com Fri Jun 5 10:07:06 2015 From: vladimir.x.ivanov at oracle.com (Vladimir Ivanov) Date: Fri, 05 Jun 2015 13:07:06 +0300 Subject: [9] Review request : JDK-8068416: LFGarbageCollectedTest.java fails with OOME: "GC overhead limit exceeded" In-Reply-To: <55717456.3010402@oracle.com> References: <55717074.6010103@oracle.com> <55717324.2040703@oracle.com> <55717456.3010402@oracle.com> Message-ID: <557174CA.4000901@oracle.com> Good. Best regards, Vladimir Ivanov On 6/5/15 1:05 PM, Konstantin Shefov wrote: > Vladimir > > Thanks for reviewing > Here is corrected fix: > http://cr.openjdk.java.net/~kshefov/8068416/webrev.02 > > -Konstantin > > On 06/05/2015 01:00 PM, Vladimir Ivanov wrote: >> + * @ignore until 8078602 is fixed >> >> The standard way to exclude a test is to mention bug ID. >> Use the following: >> @ignore 8078602 >> >> Otherwise, looks good. >> >> Best regards, >> Vladimir Ivanov >> >> On 6/5/15 12:48 PM, Konstantin Shefov wrote: >>> Hello, >>> >>> Please review the test bug fix >>> https://bugs.openjdk.java.net/browse/JDK-8068416 >>> Webrev is http://cr.openjdk.java.net/~kshefov/8068416/webrev.01/ >>> >>> Test failure is caused by JDK-8078602 [1]. Suggestion is to exclude the >>> test until [1] is fixed. >>> >>> [1] https://bugs.openjdk.java.net/browse/JDK-8078602 >>> >>> Thanks >>> -Konstantin > From vladimir.x.ivanov at oracle.com Fri Jun 5 10:16:51 2015 From: vladimir.x.ivanov at oracle.com (Vladimir Ivanov) Date: Fri, 05 Jun 2015 13:16:51 +0300 Subject: [8u-dev] Review request : JDK-8062904: TEST_BUG: Tests java/lang/invoke/LFCaching fail when run with -Xcomp option In-Reply-To: <556ECC99.2000702@oracle.com> References: <5565D9BB.60003@oracle.com> <5565F6D5.7030900@oracle.com> <5566D9D8.4090401@oracle.com> <55675CCB.8010506@oracle.com> <5568281B.7040108@oracle.com> <55685264.20001@oracle.com> <556C86C6.2000409@oracle.com> <556DFB1C.7010301@oracle.com> <556ECC99.2000702@oracle.com> Message-ID: <55717713.80606@oracle.com> Konstantin, > The test has the code cache overflow failure only when run with > "-Xcomp", all other failures has been fixed by > https://bugs.openjdk.java.net/browse/JDK-8058733 > So my suggestion is either to exclude this test when run with -Xcomp or > (better) to reduce iteration number to 1 when -Xcomp, so that no code > cache overflow in this case. I'd like to get an answer to my previous question: "How reliable the test is if it ignores NoSuchMethodException & VirtualMachineError? Are there other manifestations of the problem?" Most notably, have you seen VM crashes w/ -Xcomp running that test? Best regards, Vladimir Ivanov > > -Konstantin > >> >> Best regards, >> Vladimir Ivanov >> >>> >>> -Konstantin >>> >>> On 29.05.2015 14:49, Vladimir Ivanov wrote: >>>>> What do you mean by ignore code cache overflow? Do you mean I >>>>> should fix >>>>> the test to ignore these errors or should I leave this test unfixed >>>>> because it is a product related issue? >>>> The former. How reliable the test is if it ignores >>>> NoSuchMethodException & VirtualMachineError? Are there other >>>> manifestations of the problem? >>>> >>>> Best regards, >>>> Vladimir Ivanov >>>> >>>>> On 28.05.2015 21:22, Vladimir Ivanov wrote: >>>>>> Got it, thanks. >>>>>> >>>>>> Can we ignore errors caused by code cache overflow for now? >>>>>> >>>>>> Best regards, >>>>>> Vladimir Ivanov >>>>>> >>>>>> On 5/28/15 12:03 PM, Konstantin Shefov wrote: >>>>>>> Vladimir, >>>>>>> >>>>>>> This fix is not for timeout issue, this fix is for >>>>>>> "java.lang.VirtualMachineError: out of space in CodeCache for >>>>>>> adapters". >>>>>>> >>>>>>> Timeout issue is other bug and should be filed separately. >>>>>>> I do not know why SQE added RULES with timeout to this bug. >>>>>>> >>>>>>> By the way, if -Xcomp is set on JDK 8u, test works if not more than >>>>>>> one >>>>>>> iteration is allowed. The same thing was for JDK 9 until >>>>>>> JDK-8046809 had >>>>>>> been fixed. >>>>>>> >>>>>>> -Konstantin >>>>>>> >>>>>>> On 27.05.2015 19:54, Vladimir Ivanov wrote: >>>>>>>> Have you tried to reduce iteration granularity? >>>>>>>> >>>>>>>> Probably, checking execution duration on every test case is more >>>>>>>> robust. >>>>>>>> >>>>>>>> Best regards, >>>>>>>> Vladimir Ivanov >>>>>>>> >>>>>>>> On 5/27/15 5:50 PM, Konstantin Shefov wrote: >>>>>>>>> Hello, >>>>>>>>> >>>>>>>>> Please review the test bug fix >>>>>>>>> https://bugs.openjdk.java.net/browse/JDK-8062904 >>>>>>>>> Webrev is http://cr.openjdk.java.net/~kshefov/8062904/webrev.01/ >>>>>>>>> Test fails only against JDK 8u and passes against JDK 9. >>>>>>>>> >>>>>>>>> Thanks >>>>>>>>> >>>>>>>>> -Konstantin >>>>>>> >>>>> >>> > From konstantin.shefov at oracle.com Fri Jun 5 10:30:22 2015 From: konstantin.shefov at oracle.com (Konstantin Shefov) Date: Fri, 05 Jun 2015 13:30:22 +0300 Subject: [8u60] approval request for JDK-8068416: LFGarbageCollectedTest.java fails with OOME: "GC overhead limit exceeded" Message-ID: <55717A3E.4060209@oracle.com> Please approve direct backport. bug: https://bugs.openjdk.java.net/browse/JDK-8068416 jdk9 review thread: http://mail.openjdk.java.net/pipermail/core-libs-dev/2015-June/033998.html jdk8u-dev webrev: http://cr.openjdk.java.net/~kshefov/8068416/webrev.02 Thanks, -Konstantin From paul.sandoz at oracle.com Fri Jun 5 10:38:59 2015 From: paul.sandoz at oracle.com (Paul Sandoz) Date: Fri, 5 Jun 2015 12:38:59 +0200 Subject: RFR 8072773 (fs) Files.lines needs a better splitting implementation for stream source In-Reply-To: References: <556F28EF.1000201@oracle.com> <7CF56CF1-7A27-475B-81D3-8B784B785247@oracle.com> Message-ID: On Jun 5, 2015, at 12:57 AM, Stefan Zobel wrote: > With respect to the 'stream test library not closing streams' bug: > > would you mind to create a separate bugtracker issue for that, or do you > think it's too special to carry that weight? > Not too special just more more work than necessary to separate them out (since the file lines test is in effect also serving as test for the test library changes). Paul. From sean.coffey at oracle.com Fri Jun 5 10:43:54 2015 From: sean.coffey at oracle.com (=?UTF-8?B?U2XDoW4gQ29mZmV5?=) Date: Fri, 05 Jun 2015 11:43:54 +0100 Subject: [8u60] approval request for JDK-8068416: LFGarbageCollectedTest.java fails with OOME: "GC overhead limit exceeded" In-Reply-To: <55717A3E.4060209@oracle.com> References: <55717A3E.4060209@oracle.com> Message-ID: <55717D6A.70800@oracle.com> Approved. Regards, Sean. On 05/06/15 11:30, Konstantin Shefov wrote: > Please approve direct backport. > > bug: https://bugs.openjdk.java.net/browse/JDK-8068416 > jdk9 review thread: > http://mail.openjdk.java.net/pipermail/core-libs-dev/2015-June/033998.html > jdk8u-dev webrev: http://cr.openjdk.java.net/~kshefov/8068416/webrev.02 > > > Thanks, > -Konstantin From dmitry.samersoff at oracle.com Fri Jun 5 11:00:27 2015 From: dmitry.samersoff at oracle.com (Dmitry Samersoff) Date: Fri, 05 Jun 2015 14:00:27 +0300 Subject: RFR(M, v14): JDK-8059036 : Implement Diagnostic Commands for heap and finalizerinfo In-Reply-To: References: <554894E9.8020908@oracle.com> <555453BA.2070205@gmail.com> <55567748.6020304@oracle.com> <55567A68.9020802@oracle.com> <5556F34B.5050701@oracle.com> <55573A31.7050306@gmail.com> <55573C86.8030807@gmail.com> <5559D85B.2050500@oracle.com> <6FB8DF19-1D67-4702-9642-4C630765D2D1@oracle.com> <555C2EEB.1050505@oracle.com> <3941B430-FE6C-4AF0-A875-76D24411C654@oracle.com> <5564802A.2010403@oracle.com> <5565738A.4040109@gmail.com> <7EB0BE71-1608-4047-B71D-1A9EAA9083BB@oracle.com> <556727A2.9060209@gmail.com> <55675908.6030406@oracle.com> <556AF3CD.50900@oracle.com> <37227441-B016-4A9A-B9CA-BE100E992C9A@oracle.com> <556D8FB0.4070608@oracle.com> <556E9970.8010704@gmail.com> <556EBF4B.6070400@oracle.com> <556F1D6C.6010806@oracle.com> <556F491E.3070709@oracle.com> <557054C7.4030705@oracle.com> <3607C2B6-F247-4291-9198-279CF8ED2AF3@oracle.com> <5570738B.5040603@oracle.com> <00038522-9EC7-4491-9AE4-73BD16D95909@oracle.com> Message-ID: <5571814B.9060706@oracle.com> Staffan, Thank you for review! Done. Webrev updated in-place (press shift-reload). http://cr.openjdk.java.net/~dsamersoff/JDK-8059036/webrev.14 -Dmitry On 2015-06-05 11:20, Staffan Larsen wrote: > Dmitry, > > I?d like to propose the following change to get prettier output (more in line with GC.class_histogram): > > diff --git a/src/share/vm/services/diagnosticCommand.cpp b/src/share/vm/services/diagnosticCommand.cpp > --- a/src/share/vm/services/diagnosticCommand.cpp > +++ b/src/share/vm/services/diagnosticCommand.cpp > @@ -341,7 +341,6 @@ > void FinalizerInfoDCmd::execute(DCmdSource source, TRAPS) { > ResourceMark rm; > > - output()->print_cr("Unreachable instances awaiting finalization:"); > Klass* k = SystemDictionary::resolve_or_null( > vmSymbols::finalizer_histogram_klass(), THREAD); > assert(k != NULL, "FinalizerHistogram class is not accessible"); > @@ -375,12 +374,15 @@ > > assert(count_res != NULL && name_res != NULL, "Unexpected layout of FinalizerHistogramEntry"); > > + output()->print_cr("Unreachable instances waiting for finalization"); > + output()->print_cr("#instances class name"); > + output()->print_cr("-----------------------"); > for (int i = 0; i < result_oop->length(); ++i) { > oop element_oop = result_oop->obj_at(i); > oop str_oop = element_oop->obj_field(name_fd.offset()); > char *name = java_lang_String::as_utf8_string(str_oop); > int count = element_oop->int_field(count_fd.offset()); > - output()->print_cr("Class %s - %d", name, count); > + output()->print_cr("%10d %s", count, name); > } > } > > > A couple of nits: > > diagnosticCommand.cpp:359 - extra space after = > diagnosticCommand.cpp:361 - spelling: finlalization -> finalization > FinalizerInfoTest.java:69: - spelling: intialized -> initialized > FinalizerInfoTest.java:71 - I?d like to see the ; on a new line > > > Thanks, > /Staffan > > >> On 5 jun 2015, at 00:20, Mandy Chung wrote: >> >> >>> On Jun 4, 2015, at 8:49 AM, Dmitry Samersoff wrote: >>> >>> Mandy, >>> >>> Webrev updated in-place (press shift-reload). >>> >>> http://cr.openjdk.java.net/~dsamersoff/JDK-8059036/webrev.14 >>> >>> getFinalizerHistogram() now returns Entry[] >>> >>> @library and @build removed from the test. >> >> >> Looks fine. >> >> Thanks >> Mandy >> >>> >>> -Dmitry >>> >>> On 2015-06-04 16:56, Mandy Chung wrote: >>>> >>>>> On Jun 4, 2015, at 6:38 AM, Dmitry Samersoff wrote: >>>>> >>>>> Mandy, >>>>> >>>>> Thank you for the review. >>>>> >>>>> Updated webrev is: >>>>> >>>>> http://cr.openjdk.java.net/~dsamersoff/JDK-8059036/webrev.14 >>>>> >>>> >>>> Thanks for the update and the test. >>>> >>>>> addressed comments, added a unit test to JDK workspace. >>>>> >>>> >>>> This test does not need @library and @build, right? >>>> >>>>> >>>>> On 2015-06-03 21:36, Mandy Chung wrote: >>>>> >>>>>> Should getFinalizerHistogram() return FinalizerHistogramEntry[]? >>>>>> The VM knows the returned type anyway. >>>>> >>>>> "[java/lang/ref/Entry" signature would need a separate symbol entry in >>>>> swollen vmSymbols::init() so I would prefer to stay with more generic >>>>> Object[] >>>>> >>>> >>>> You already add several symbols - why is it an issue for another one? Or if adding vm symbols is a concern, you should revert to String and int[] that you decide not to. The decision should apply consistently (use String and int, or new Java type). >>>> >>>> >>>>>> Perhaps the getFinalizerHistogram method should be renamed >>>>>> e.g. "dump"? >>>>> >>>>> The line in vmSymbols looks like: >>>>> >>>>> template( >>>>> get_finalizer_histogram_name, "getFinalizerHistogram") >>>>> >>>>> I would prefer to keep method name specific enough to be able to >>>>> find it by grep in jdk code. >>>> >>>> >>>> Grep in jdk code is easy as you have a new class :) >>>> >>>> Mandy >>>> >>>>> >>>>> (other comments are addressed) >>>>> >>>>> -Dmitry >>>>> >>>>> >>>>> On 2015-06-03 21:36, Mandy Chung wrote: >>>>>> >>>>>> >>>>>> On 06/03/2015 08:29 AM, Dmitry Samersoff wrote: >>>>>>> Updated webrev: >>>>>>> >>>>>>> http://cr.openjdk.java.net/~dsamersoff/JDK-8059036/webrev.13 >>>>>> >>>>>> I reviewed the jdk change. The version looks okay and some comments: >>>>>> >>>>>> ReferenceQueue.java - you should eliminate the use of rawtype: >>>>>> >>>>>> 84 Reference rn = r.next; >>>>>> >>>>>> Change Reference to Reference >>>>> >>>>> done. >>>>> >>>>>> >>>>>> The forEach method - eliminate the use of raw type and >>>>>> move @SuppressWarning to the field variable in line 182: >>>>>> >>>>>> @SuppressWarnings("unchecked") >>>>>> Reference rn = r.next; >>>>> >>>>> done. >>>>> >>>>>> >>>>>> FinalizerHistogram.java >>>>>> Copyright year needs update. >>>>> >>>>> done. >>>>>> >>>>>> I personally think the VM code would be much simpler if you stay with >>>>>> alternative entry of String and int than dealing with a >>>>>> FinalizerHistogramEntry private class. It's okay as FinalizerHistogram >>>>>> class is separated. >>>>>> >>>>>> The comment in line 35 is suitable to move to the class description and >>>>>> this is the suggestion. >>>>>> >>>>>> // This FinalizerHistogram class is for GC.finalizer_info diagnostic >>>>>> command support. >>>>>> // It is invoked by the VM. >>>>> >>>>> done. >>>>> >>>>>> >>>>>> Should getFinalizerHistogram() return FinalizerHistogramEntry[]? The VM >>>>>> knows the returned type anyway. >>>>> >>>>> "[java/lang/ref/Entry" signature would need a separate symbol entry in >>>>> swollen vmSymbols::init() so I would prefer to stay with more generic >>>>> Object[] >>>>> >>>>>> It's an inner class of >>>>>> FinalizerHistogram and it can simply be named as "Entry". I suggest to >>>>>> have Entry::increment method to replace ".instanceCount++". >>>>> >>>>> done. >>>>> >>>>>> >>>>>> The tests are in hotspot/test. Can you add a unit test in jdk/test? >>>>>> Perhaps you can test FinalizerHistogram.getFinalizerHistogram() via >>>>>> reflection - just a sanity test. >>>>> >>>>> done. The test repeats hotspot one, but uses reflection instead of VM call. >>>>> >>>>>> >>>>>> A minor naming comment - now you have a FinalizerHistogram class. >>>>>> Perhaps the getFinalizerHistogram method should be renamed e.g. "dump"? >>>>> >>>>> The line in vmSymbols looks like: >>>>> >>>>> template( >>>>> get_finalizer_histogram_name, "getFinalizerHistogram") >>>>> >>>>> I would prefer to keep it specific enough to be able to >>>>> find it by grep in jdk code. >>>>> >>>>> >>>>> -- >>>>> Dmitry Samersoff >>>>> Oracle Java development team, Saint Petersburg, Russia >>>>> * I would love to change the world, but they won't give me the sources. >>>> >>> >>> >>> -- >>> Dmitry Samersoff >>> Oracle Java development team, Saint Petersburg, Russia >>> * I would love to change the world, but they won't give me the sources. >> > -- Dmitry Samersoff Oracle Java development team, Saint Petersburg, Russia * I would love to change the world, but they won't give me the sources. From lance.andersen at oracle.com Fri Jun 5 11:29:52 2015 From: lance.andersen at oracle.com (Lance Andersen) Date: Fri, 5 Jun 2015 07:29:52 -0400 Subject: Review request for JDK-8080906 & JDK-8080908: Develop tests for JEP 255 Xerces Updates In-Reply-To: <013b01d09f63$a0d1ff40$e275fdc0$@oracle.com> References: <013b01d09f63$a0d1ff40$e275fdc0$@oracle.com> Message-ID: <6860E5EE-B032-4F42-9C8A-A07973419767@oracle.com> Hi Frank, These seem OK. I am sure Joe will also review when he wakes up in his part of the world :-) Best Lance On Jun 5, 2015, at 3:45 AM, Frank Yuan wrote: > Hi Joe and all > > > > I have been working on the test task of JEP 255 Xerces Updates. > > > > Here I would invite you to review the changes for 2 bugs of this task: > > > > 1. JDK-8080906 Develop test for Xerces Update: DOM L3 Serializer > > To verify default LSSerializer is Xalan dom 3 serializer > > > > 2. JDK-8080908 Develop test for Xerces Update: XPointer > > To verify Xerces revision 415823: XERCESJ-1134. It should have been fixed > in JDK-6794483, however, the test Bug6794483Test.java need to be revised to > really cover this bug. > > > > > > I also added module dependencies for exported API in this path, it's > recommended for Jigsaw change, refer to > https://wiki.se.oracle.com/display/JPG/@modules+in+JTReg+tests > > > > The webrev is at > http://cr.openjdk.java.net/~fyuan/8080906_8080908/webrev.00/, your comment > will be appreciated. > > > > Best Regards > > Frank > Lance Andersen| Principal Member of Technical Staff | +1.781.442.2037 Oracle Java Engineering 1 Network Drive Burlington, MA 01803 Lance.Andersen at oracle.com From staffan.larsen at oracle.com Fri Jun 5 12:24:33 2015 From: staffan.larsen at oracle.com (Staffan Larsen) Date: Fri, 5 Jun 2015 14:24:33 +0200 Subject: RFR(M, v14): JDK-8059036 : Implement Diagnostic Commands for heap and finalizerinfo In-Reply-To: <5571814B.9060706@oracle.com> References: <554894E9.8020908@oracle.com> <555453BA.2070205@gmail.com> <55567748.6020304@oracle.com> <55567A68.9020802@oracle.com> <5556F34B.5050701@oracle.com> <55573A31.7050306@gmail.com> <55573C86.8030807@gmail.com> <5559D85B.2050500@oracle.com> <6FB8DF19-1D67-4702-9642-4C630765D2D1@oracle.com> <555C2EEB.1050505@oracle.com> <3941B430-FE6C-4AF0-A875-76D24411C654@oracle.com> <5564802A.2010403@oracle.com> <5565738A.4040109@gmail.com> <7EB0BE71-1608-4047-B71D-1A9EAA9083BB@oracle.com> <556727A2.9060209@gmail.com> <55675908.6030406@oracle.com> <556AF3CD.50900@oracle.com> <37227441-B016-4A9A-B9CA-BE100E992C9A@oracle.com> <556D8FB0.4070608@oracle.com> <556E9970.8010704@gmail.com> <556EBF4B.6070400@oracle.com> <556F1D6C.6010806@oracle.com> <556F491E.3070709@oracle.com> <557054C7.4030705@oracle.com> <3607C2B6-F247-4291-9198-279CF8ED2AF3@oracle.com> <5570738B.5040603@oracle.com> <00038522-9EC7-4491-9AE4-73BD16D95909@oracle.com> <5571! 814B.9060706@oracle.com> Message-ID: <0323E5E0-CFFA-4E11-8F51-270CB7F6964C@oracle.com> Thanks - this version looks good to me. /Staffan > On 5 jun 2015, at 13:00, Dmitry Samersoff wrote: > > Staffan, > > Thank you for review! > > Done. Webrev updated in-place (press shift-reload). > > http://cr.openjdk.java.net/~dsamersoff/JDK-8059036/webrev.14 > > -Dmitry > > On 2015-06-05 11:20, Staffan Larsen wrote: >> Dmitry, >> >> I?d like to propose the following change to get prettier output (more in line with GC.class_histogram): >> >> diff --git a/src/share/vm/services/diagnosticCommand.cpp b/src/share/vm/services/diagnosticCommand.cpp >> --- a/src/share/vm/services/diagnosticCommand.cpp >> +++ b/src/share/vm/services/diagnosticCommand.cpp >> @@ -341,7 +341,6 @@ >> void FinalizerInfoDCmd::execute(DCmdSource source, TRAPS) { >> ResourceMark rm; >> >> - output()->print_cr("Unreachable instances awaiting finalization:"); >> Klass* k = SystemDictionary::resolve_or_null( >> vmSymbols::finalizer_histogram_klass(), THREAD); >> assert(k != NULL, "FinalizerHistogram class is not accessible"); >> @@ -375,12 +374,15 @@ >> >> assert(count_res != NULL && name_res != NULL, "Unexpected layout of FinalizerHistogramEntry"); >> >> + output()->print_cr("Unreachable instances waiting for finalization"); >> + output()->print_cr("#instances class name"); >> + output()->print_cr("-----------------------"); >> for (int i = 0; i < result_oop->length(); ++i) { >> oop element_oop = result_oop->obj_at(i); >> oop str_oop = element_oop->obj_field(name_fd.offset()); >> char *name = java_lang_String::as_utf8_string(str_oop); >> int count = element_oop->int_field(count_fd.offset()); >> - output()->print_cr("Class %s - %d", name, count); >> + output()->print_cr("%10d %s", count, name); >> } >> } >> >> >> A couple of nits: >> >> diagnosticCommand.cpp:359 - extra space after = >> diagnosticCommand.cpp:361 - spelling: finlalization -> finalization >> FinalizerInfoTest.java:69: - spelling: intialized -> initialized >> FinalizerInfoTest.java:71 - I?d like to see the ; on a new line >> >> >> Thanks, >> /Staffan >> >> >>> On 5 jun 2015, at 00:20, Mandy Chung wrote: >>> >>> >>>> On Jun 4, 2015, at 8:49 AM, Dmitry Samersoff wrote: >>>> >>>> Mandy, >>>> >>>> Webrev updated in-place (press shift-reload). >>>> >>>> http://cr.openjdk.java.net/~dsamersoff/JDK-8059036/webrev.14 >>>> >>>> getFinalizerHistogram() now returns Entry[] >>>> >>>> @library and @build removed from the test. >>> >>> >>> Looks fine. >>> >>> Thanks >>> Mandy >>> >>>> >>>> -Dmitry >>>> >>>> On 2015-06-04 16:56, Mandy Chung wrote: >>>>> >>>>>> On Jun 4, 2015, at 6:38 AM, Dmitry Samersoff wrote: >>>>>> >>>>>> Mandy, >>>>>> >>>>>> Thank you for the review. >>>>>> >>>>>> Updated webrev is: >>>>>> >>>>>> http://cr.openjdk.java.net/~dsamersoff/JDK-8059036/webrev.14 >>>>>> >>>>> >>>>> Thanks for the update and the test. >>>>> >>>>>> addressed comments, added a unit test to JDK workspace. >>>>>> >>>>> >>>>> This test does not need @library and @build, right? >>>>> >>>>>> >>>>>> On 2015-06-03 21:36, Mandy Chung wrote: >>>>>> >>>>>>> Should getFinalizerHistogram() return FinalizerHistogramEntry[]? >>>>>>> The VM knows the returned type anyway. >>>>>> >>>>>> "[java/lang/ref/Entry" signature would need a separate symbol entry in >>>>>> swollen vmSymbols::init() so I would prefer to stay with more generic >>>>>> Object[] >>>>>> >>>>> >>>>> You already add several symbols - why is it an issue for another one? Or if adding vm symbols is a concern, you should revert to String and int[] that you decide not to. The decision should apply consistently (use String and int, or new Java type). >>>>> >>>>> >>>>>>> Perhaps the getFinalizerHistogram method should be renamed >>>>>>> e.g. "dump"? >>>>>> >>>>>> The line in vmSymbols looks like: >>>>>> >>>>>> template( >>>>>> get_finalizer_histogram_name, "getFinalizerHistogram") >>>>>> >>>>>> I would prefer to keep method name specific enough to be able to >>>>>> find it by grep in jdk code. >>>>> >>>>> >>>>> Grep in jdk code is easy as you have a new class :) >>>>> >>>>> Mandy >>>>> >>>>>> >>>>>> (other comments are addressed) >>>>>> >>>>>> -Dmitry >>>>>> >>>>>> >>>>>> On 2015-06-03 21:36, Mandy Chung wrote: >>>>>>> >>>>>>> >>>>>>> On 06/03/2015 08:29 AM, Dmitry Samersoff wrote: >>>>>>>> Updated webrev: >>>>>>>> >>>>>>>> http://cr.openjdk.java.net/~dsamersoff/JDK-8059036/webrev.13 >>>>>>> >>>>>>> I reviewed the jdk change. The version looks okay and some comments: >>>>>>> >>>>>>> ReferenceQueue.java - you should eliminate the use of rawtype: >>>>>>> >>>>>>> 84 Reference rn = r.next; >>>>>>> >>>>>>> Change Reference to Reference >>>>>> >>>>>> done. >>>>>> >>>>>>> >>>>>>> The forEach method - eliminate the use of raw type and >>>>>>> move @SuppressWarning to the field variable in line 182: >>>>>>> >>>>>>> @SuppressWarnings("unchecked") >>>>>>> Reference rn = r.next; >>>>>> >>>>>> done. >>>>>> >>>>>>> >>>>>>> FinalizerHistogram.java >>>>>>> Copyright year needs update. >>>>>> >>>>>> done. >>>>>>> >>>>>>> I personally think the VM code would be much simpler if you stay with >>>>>>> alternative entry of String and int than dealing with a >>>>>>> FinalizerHistogramEntry private class. It's okay as FinalizerHistogram >>>>>>> class is separated. >>>>>>> >>>>>>> The comment in line 35 is suitable to move to the class description and >>>>>>> this is the suggestion. >>>>>>> >>>>>>> // This FinalizerHistogram class is for GC.finalizer_info diagnostic >>>>>>> command support. >>>>>>> // It is invoked by the VM. >>>>>> >>>>>> done. >>>>>> >>>>>>> >>>>>>> Should getFinalizerHistogram() return FinalizerHistogramEntry[]? The VM >>>>>>> knows the returned type anyway. >>>>>> >>>>>> "[java/lang/ref/Entry" signature would need a separate symbol entry in >>>>>> swollen vmSymbols::init() so I would prefer to stay with more generic >>>>>> Object[] >>>>>> >>>>>>> It's an inner class of >>>>>>> FinalizerHistogram and it can simply be named as "Entry". I suggest to >>>>>>> have Entry::increment method to replace ".instanceCount++". >>>>>> >>>>>> done. >>>>>> >>>>>>> >>>>>>> The tests are in hotspot/test. Can you add a unit test in jdk/test? >>>>>>> Perhaps you can test FinalizerHistogram.getFinalizerHistogram() via >>>>>>> reflection - just a sanity test. >>>>>> >>>>>> done. The test repeats hotspot one, but uses reflection instead of VM call. >>>>>> >>>>>>> >>>>>>> A minor naming comment - now you have a FinalizerHistogram class. >>>>>>> Perhaps the getFinalizerHistogram method should be renamed e.g. "dump"? >>>>>> >>>>>> The line in vmSymbols looks like: >>>>>> >>>>>> template( >>>>>> get_finalizer_histogram_name, "getFinalizerHistogram") >>>>>> >>>>>> I would prefer to keep it specific enough to be able to >>>>>> find it by grep in jdk code. >>>>>> >>>>>> >>>>>> -- >>>>>> Dmitry Samersoff >>>>>> Oracle Java development team, Saint Petersburg, Russia >>>>>> * I would love to change the world, but they won't give me the sources. >>>>> >>>> >>>> >>>> -- >>>> Dmitry Samersoff >>>> Oracle Java development team, Saint Petersburg, Russia >>>> * I would love to change the world, but they won't give me the sources. >>> >> > > > -- > Dmitry Samersoff > Oracle Java development team, Saint Petersburg, Russia > * I would love to change the world, but they won't give me the sources. From paul.sandoz at oracle.com Fri Jun 5 12:25:01 2015 From: paul.sandoz at oracle.com (Paul Sandoz) Date: Fri, 5 Jun 2015 14:25:01 +0200 Subject: Patch to improve primitives Array.sort() In-Reply-To: <53868C2C-52D1-4D0C-8E7B-BEF86E467983@oracle.com> References: <1CF6FDFD0639DF478B44651D79ECE70401104C1917@GSCMHKP12EX.firmwide.corp.gs.com> <4911D592-3065-4416-A683-6B80275B4173@oracle.com> <553A0832.7070702@oracle.com> <4725EC08FFA7B84AB611B2463BDA97ED1B3F978D2D@GSCMAMP30EX.firmwide.corp.gs.com> <1CF6FDFD0639DF478B44651D79ECE70401146C4F10@GSCMHKP12EX.firmwide.corp.gs.com> <24BCB588-1A4F-4419-81A4-CD947828E5C2@oracle.com> <1CF6FDFD0639DF478B44651D79ECE70401146C4FB7@GSCMHKP12EX.firmwide.corp.gs.com> <2090E8E4-4AD4-4F7C-84AB-A77625FE221B@oracle.com> <6882C9A35DFB9B4FA2779F7BF5B9757D207611678C@GSCMAMP06EX.firmwide.corp.gs.com> <3251F9D5-C128-4C12-A514-803A17B65190@oracle.com> <6882C9A35DFB9B4FA2779F7BF5B9757D2076116796@GSCMAMP06EX.firmwide.corp.gs.com> <67BF3338-6879-445D-AA62-624D2C94B615@oracle.com> <53868C2C-52D1-4D0C-8E7B-BEF86E467983@oracle.com> Message-ID: <6DF7FEBC-89A2-4D08-B5D1-4C4A5565829B@oracle.com> On May 26, 2015, at 11:54 AM, Paul Sandoz wrote: > Here is an updated webrev: > > http://cr.openjdk.java.net/~psandoz/jdk9/JDK-8080945-nearly-sorted-primitives/webrev/ > Updated with some contributed JMH benchmarks located in the test area. To be moved when there is a better location (i really want to avoid a repeat experience of trying to find benchmarks in this area). I plan to push either today or Monday. Paul. From alexander.v.stepanov at oracle.com Fri Jun 5 12:51:54 2015 From: alexander.v.stepanov at oracle.com (alexander stepanov) Date: Fri, 05 Jun 2015 15:51:54 +0300 Subject: RFR [9] 8081517: minor cleanup for docs Message-ID: <55719B6A.4000904@oracle.com> Hello, Could you please review the fix http://cr.openjdk.java.net/~avstepan/8081517/webrev.00/ for https://bugs.openjdk.java.net/browse/JDK-8081517 Just some cleanup for docs. Thanks, Alexander From konstantin.shefov at oracle.com Fri Jun 5 13:23:34 2015 From: konstantin.shefov at oracle.com (Konstantin Shefov) Date: Fri, 05 Jun 2015 16:23:34 +0300 Subject: [8u-dev] Review request : JDK-8062904: TEST_BUG: Tests java/lang/invoke/LFCaching fail when run with -Xcomp option In-Reply-To: <55717713.80606@oracle.com> References: <5565D9BB.60003@oracle.com> <5565F6D5.7030900@oracle.com> <5566D9D8.4090401@oracle.com> <55675CCB.8010506@oracle.com> <5568281B.7040108@oracle.com> <55685264.20001@oracle.com> <556C86C6.2000409@oracle.com> <556DFB1C.7010301@oracle.com> <556ECC99.2000702@oracle.com> <55717713.80606@oracle.com> Message-ID: <5571A2D6.8020508@oracle.com> Vladimir On 06/05/2015 01:16 PM, Vladimir Ivanov wrote: > Konstantin, > >> The test has the code cache overflow failure only when run with >> "-Xcomp", all other failures has been fixed by >> https://bugs.openjdk.java.net/browse/JDK-8058733 >> So my suggestion is either to exclude this test when run with -Xcomp or >> (better) to reduce iteration number to 1 when -Xcomp, so that no code >> cache overflow in this case. > I'd like to get an answer to my previous question: > "How reliable the test is if it ignores NoSuchMethodException & > VirtualMachineError? Are there other manifestations of the problem?" > > Most notably, have you seen VM crashes w/ -Xcomp running that test? I have seen no crashes with -Xcomp in our tests results base, only NoSuchMethodException and VirtualMachineError. This failure does not occur with JDK 9 on -Xcomp, I used the same random seeds (-Dseed), and the problem exists only with JDK 8u, not JDK 9. Because there is no failure with JDK 9 I can suppose there could be a product issue in JDK 8u. -Konstantin > > Best regards, > Vladimir Ivanov > >> >> -Konstantin >> >>> >>> Best regards, >>> Vladimir Ivanov >>> >>>> >>>> -Konstantin >>>> >>>> On 29.05.2015 14:49, Vladimir Ivanov wrote: >>>>>> What do you mean by ignore code cache overflow? Do you mean I >>>>>> should fix >>>>>> the test to ignore these errors or should I leave this test unfixed >>>>>> because it is a product related issue? >>>>> The former. How reliable the test is if it ignores >>>>> NoSuchMethodException & VirtualMachineError? Are there other >>>>> manifestations of the problem? >>>>> >>>>> Best regards, >>>>> Vladimir Ivanov >>>>> >>>>>> On 28.05.2015 21:22, Vladimir Ivanov wrote: >>>>>>> Got it, thanks. >>>>>>> >>>>>>> Can we ignore errors caused by code cache overflow for now? >>>>>>> >>>>>>> Best regards, >>>>>>> Vladimir Ivanov >>>>>>> >>>>>>> On 5/28/15 12:03 PM, Konstantin Shefov wrote: >>>>>>>> Vladimir, >>>>>>>> >>>>>>>> This fix is not for timeout issue, this fix is for >>>>>>>> "java.lang.VirtualMachineError: out of space in CodeCache for >>>>>>>> adapters". >>>>>>>> >>>>>>>> Timeout issue is other bug and should be filed separately. >>>>>>>> I do not know why SQE added RULES with timeout to this bug. >>>>>>>> >>>>>>>> By the way, if -Xcomp is set on JDK 8u, test works if not more >>>>>>>> than >>>>>>>> one >>>>>>>> iteration is allowed. The same thing was for JDK 9 until >>>>>>>> JDK-8046809 had >>>>>>>> been fixed. >>>>>>>> >>>>>>>> -Konstantin >>>>>>>> >>>>>>>> On 27.05.2015 19:54, Vladimir Ivanov wrote: >>>>>>>>> Have you tried to reduce iteration granularity? >>>>>>>>> >>>>>>>>> Probably, checking execution duration on every test case is more >>>>>>>>> robust. >>>>>>>>> >>>>>>>>> Best regards, >>>>>>>>> Vladimir Ivanov >>>>>>>>> >>>>>>>>> On 5/27/15 5:50 PM, Konstantin Shefov wrote: >>>>>>>>>> Hello, >>>>>>>>>> >>>>>>>>>> Please review the test bug fix >>>>>>>>>> https://bugs.openjdk.java.net/browse/JDK-8062904 >>>>>>>>>> Webrev is http://cr.openjdk.java.net/~kshefov/8062904/webrev.01/ >>>>>>>>>> Test fails only against JDK 8u and passes against JDK 9. >>>>>>>>>> >>>>>>>>>> Thanks >>>>>>>>>> >>>>>>>>>> -Konstantin >>>>>>>> >>>>>> >>>> >> From lance.andersen at oracle.com Fri Jun 5 13:29:52 2015 From: lance.andersen at oracle.com (Lance Andersen) Date: Fri, 5 Jun 2015 09:29:52 -0400 Subject: RFR [9] 8081517: minor cleanup for docs In-Reply-To: <55719B6A.4000904@oracle.com> References: <55719B6A.4000904@oracle.com> Message-ID: Hi Alxander Overall. looks OK. A couple of things I thought could be changed; In FTPURLConnection, I probably would surround the code in an {@code} tag In VmIdentifier, I do not see think we still need

    in the

  • tags Best Lance On Jun 5, 2015, at 8:51 AM, alexander stepanov wrote: > Hello, > > Could you please review the fix > http://cr.openjdk.java.net/~avstepan/8081517/webrev.00/ > for > https://bugs.openjdk.java.net/browse/JDK-8081517 > > Just some cleanup for docs. > > Thanks, > Alexander Lance Andersen| Principal Member of Technical Staff | +1.781.442.2037 Oracle Java Engineering 1 Network Drive Burlington, MA 01803 Lance.Andersen at oracle.com From magnus.ihse.bursie at oracle.com Fri Jun 5 14:07:20 2015 From: magnus.ihse.bursie at oracle.com (Magnus Ihse Bursie) Date: Fri, 05 Jun 2015 16:07:20 +0200 Subject: RFR: JDK-8085822 JEP 223: New Version-String Scheme (initial integration) Message-ID: <5571AD18.7080701@oracle.com> This review request covers the main part of the work for JEP-223, the new version string format [1]. Basically, we'll call this release Java "9", instead of Java "1.9.0". This patch is a folding of all work that has been done so far in the branch JEP-223-branch in jdk9/sandbox. As you can see, it mostly covers build changes, with some code changes in hotspot, jdk, nashorn and langtools that either are corresponding changes in the product code due to the compiler define flags changing from the build, or follow-up changes to handle the new format. The JEP-223 work is not finished by this patch. In fact, there are known issues remaining even after this patch, typically by code that reads the "java.version" property and tries to parse it. However, this patch is not directly destined for jdk9/dev, but will go into the special verona/stage forest. As for all patches destined for verona/stage it will be code reviewed as if going to jdk9/dev. Once in verona/stage it will bide its time, and it will be complemented with follow-up patches to address remaining issues. When all such issues are resolved and JEP-223 is fully implemented, all changes will be pushed at once (without further code reviews) into jdk9/dev. This patch has been contributed by Magnus Ihse Bursie, Kumar Srinivasan and Alejandro Murillo. Bug: https://bugs.openjdk.java.net/browse/JDK-8085822 WebRev: http://cr.openjdk.java.net/~ihse/JDK-8085822-JEP-223-initial-patch/webrev.01 [1] http://openjdk.java.net/jeps/223 From Roger.Riggs at Oracle.com Fri Jun 5 14:08:39 2015 From: Roger.Riggs at Oracle.com (Roger Riggs) Date: Fri, 05 Jun 2015 10:08:39 -0400 Subject: RFR [9] 8081517: minor cleanup for docs In-Reply-To: <55719B6A.4000904@oracle.com> References: <55719B6A.4000904@oracle.com> Message-ID: <5571AD67.4040203@Oracle.com> Hi Alexander, On code examples, please use the template: 
    {@code

}
    (not just 
    ...
    ) See TraceClassVisitor.java: 92+ and FtpURLConnection.java:68 I would use @code instead of @literal in HttpAuthenticator.java:59; it will make the protocol stand out. Also in PlatformLogger.java:74+ 2) If indentation is in the scope of your cleanup... In JdbcRowSetImpl.java, the indentation of continued lines after @return or @param tags isn't correct; the continued line should be indented. And there are a some very long lines that should be wrapped; See +5637... "If the SQL AS clause was not..." +5784 the indentation needs to be fixed. +5840 also +5988 +6265 and after... in many cases the lines that follow of "/*" do not align the "* ". CachedRowSetWriter.java: - many method comment blocks are not aligned properly with the method of field that follows. For example, /** * The Connection object that this writer will use to make a * connection to the data source to which it will write data. * */ private transient Connection con; 3) CachedRowSetWriter.java: +1056: The tag "@ param" should be tighted up to "@param" Thanks, Roger On 6/5/2015 8:51 AM, alexander stepanov wrote: > Hello, > > Could you please review the fix > http://cr.openjdk.java.net/~avstepan/8081517/webrev.00/ > for > https://bugs.openjdk.java.net/browse/JDK-8081517 > > Just some cleanup for docs. > > Thanks, > Alexander From lance.andersen at oracle.com Fri Jun 5 14:12:49 2015 From: lance.andersen at oracle.com (Lance Andersen) Date: Fri, 5 Jun 2015 10:12:49 -0400 Subject: RFR [9] 8081517: minor cleanup for docs In-Reply-To: <5571AD67.4040203@Oracle.com> References: <55719B6A.4000904@oracle.com> <5571AD67.4040203@Oracle.com> Message-ID: Hi Roger On Jun 5, 2015, at 10:08 AM, Roger Riggs wrote: > > 2) > If indentation is in the scope of your cleanup? Agree with you and I did not comment on this as I felt it this should be done separately as there is a fair amount that can be done in these classes for clean up. Best lance > > > In JdbcRowSetImpl.java, the indentation of continued lines after @return or @param tags > isn't correct; the continued line should be indented. > > And there are a some very long lines that should be wrapped; > See +5637... "If the SQL AS clause was not..." > > +5784 the indentation needs to be fixed. > +5840 also > +5988 > +6265 and after... > in many cases the lines that follow of "/*" do not align the "* ". > > CachedRowSetWriter.java: > - many method comment blocks are not aligned properly with the method of field that follows. > For example, > /** > * The Connection object that this writer will use to make a > * connection to the data source to which it will write data. > * > */ > private transient Connection con; > > 3) > > CachedRowSetWriter.java: > +1056: The tag "@ param" should be tighted up to "@param" > > Thanks, Roger > > > > On 6/5/2015 8:51 AM, alexander stepanov wrote: >> Hello, >> >> Could you please review the fix >> http://cr.openjdk.java.net/~avstepan/8081517/webrev.00/ >> for >> https://bugs.openjdk.java.net/browse/JDK-8081517 >> >> Just some cleanup for docs. >> >> Thanks, >> Alexander > Lance Andersen| Principal Member of Technical Staff | +1.781.442.2037 Oracle Java Engineering 1 Network Drive Burlington, MA 01803 Lance.Andersen at oracle.com From rob.mckenna at oracle.com Fri Jun 5 14:41:49 2015 From: rob.mckenna at oracle.com (Rob McKenna) Date: Fri, 05 Jun 2015 15:41:49 +0100 Subject: RFR: 7130985: Four helper classes missing in Sun JDK In-Reply-To: <556F1B2D.8010807@oracle.com> References: <556F1B2D.8010807@oracle.com> Message-ID: <5571B52D.6030408@oracle.com> Added some cleanup code around the BufferedReaders & the leftover test files: http://cr.openjdk.java.net/~robm/7130985/webrev.02/ -Rob On 03/06/15 16:20, Rob McKenna wrote: > Meant to get this sorted a while back. > > There was a thread on this last year ( > http://mail.openjdk.java.net/pipermail/core-libs-dev/2014-July/027779.html > ) > > A test has been added since then: > > http://cr.openjdk.java.net/~robm/7130985/webrev.corba/ > http://cr.openjdk.java.net/~robm/7130985/webrev.j2se/ > > -Rob From alexander.v.stepanov at oracle.com Fri Jun 5 14:59:40 2015 From: alexander.v.stepanov at oracle.com (alexander stepanov) Date: Fri, 05 Jun 2015 17:59:40 +0300 Subject: RFR [9] 8081517: minor cleanup for docs In-Reply-To: References: <55719B6A.4000904@oracle.com> <5571AD67.4040203@Oracle.com> Message-ID: <5571B95C.3080606@oracle.com> > indentation is in the scope of your cleanup No, I didn't check the indents, lengths of lines etc. (sorry). The intention was just to fix some tidy warnings (like empty

    tags) and javadoc errors like invalid parameter names, invalid tags etc. Tags were replaced with {@code} in couple of files just because of some connected warnings like unclosed tags etc. Regards, Alexander On 05.06.2015 17:12, Lance Andersen wrote: > Hi Roger > On Jun 5, 2015, at 10:08 AM, Roger Riggs wrote: > >> 2) >> If indentation is in the scope of your cleanup? > Agree with you and I did not comment on this as I felt it this should be done separately as there is a fair amount that can be done in these classes for clean up. > > Best > lance >> >> In JdbcRowSetImpl.java, the indentation of continued lines after @return or @param tags >> isn't correct; the continued line should be indented. >> >> And there are a some very long lines that should be wrapped; >> See +5637... "If the SQL AS clause was not..." >> >> +5784 the indentation needs to be fixed. >> +5840 also >> +5988 >> +6265 and after... >> in many cases the lines that follow of "/*" do not align the "* ". >> >> CachedRowSetWriter.java: >> - many method comment blocks are not aligned properly with the method of field that follows. >> For example, >> /** >> * The Connection object that this writer will use to make a >> * connection to the data source to which it will write data. >> * >> */ >> private transient Connection con; >> >> 3) >> >> CachedRowSetWriter.java: >> +1056: The tag "@ param" should be tighted up to "@param" >> >> Thanks, Roger >> >> >> >> On 6/5/2015 8:51 AM, alexander stepanov wrote: >>> Hello, >>> >>> Could you please review the fix >>> http://cr.openjdk.java.net/~avstepan/8081517/webrev.00/ >>> for >>> https://bugs.openjdk.java.net/browse/JDK-8081517 >>> >>> Just some cleanup for docs. >>> >>> Thanks, >>> Alexander > > > Lance Andersen| Principal Member of Technical Staff | +1.781.442.2037 > Oracle Java Engineering > 1 Network Drive > Burlington, MA 01803 > Lance.Andersen at oracle.com > > > From Roger.Riggs at Oracle.com Fri Jun 5 15:10:59 2015 From: Roger.Riggs at Oracle.com (Roger Riggs) Date: Fri, 05 Jun 2015 11:10:59 -0400 Subject: RFR [9] 8081517: minor cleanup for docs In-Reply-To: <5571B95C.3080606@oracle.com> References: <55719B6A.4000904@oracle.com> <5571AD67.4040203@Oracle.com> <5571B95C.3080606@oracle.com> Message-ID: <5571BC03.2040601@Oracle.com> ok, as Lance mentioned, that can be deferred. Roger On 6/5/2015 10:59 AM, alexander stepanov wrote: > > indentation is in the scope of your cleanup > > No, I didn't check the indents, lengths of lines etc. (sorry). The > intention was just to fix some tidy warnings (like empty

    tags) and > javadoc errors like invalid parameter names, invalid tags etc. Tags > were replaced with {@code} in couple of files just > because of some connected warnings like unclosed tags etc. > > Regards, > Alexander > > On 05.06.2015 17:12, Lance Andersen wrote: >> Hi Roger >> On Jun 5, 2015, at 10:08 AM, Roger Riggs wrote: >> >>> 2) >>> If indentation is in the scope of your cleanup? >> Agree with you and I did not comment on this as I felt it this should >> be done separately as there is a fair amount that can be done in >> these classes for clean up. >> >> Best >> lance >>> >>> In JdbcRowSetImpl.java, the indentation of continued lines after >>> @return or @param tags >>> isn't correct; the continued line should be indented. >>> >>> And there are a some very long lines that should be wrapped; >>> See +5637... "If the SQL AS clause was not..." >>> >>> +5784 the indentation needs to be fixed. >>> +5840 also >>> +5988 >>> +6265 and after... >>> in many cases the lines that follow of "/*" do not align the "* ". >>> >>> CachedRowSetWriter.java: >>> - many method comment blocks are not aligned properly with the >>> method of field that follows. >>> For example, >>> /** >>> * The Connection object that this writer will use to >>> make a >>> * connection to the data source to which it will write data. >>> * >>> */ >>> private transient Connection con; >>> >>> 3) >>> >>> CachedRowSetWriter.java: >>> +1056: The tag "@ param" should be tighted up to "@param" >>> >>> Thanks, Roger >>> >>> >>> >>> On 6/5/2015 8:51 AM, alexander stepanov wrote: >>>> Hello, >>>> >>>> Could you please review the fix >>>> http://cr.openjdk.java.net/~avstepan/8081517/webrev.00/ >>>> for >>>> https://bugs.openjdk.java.net/browse/JDK-8081517 >>>> >>>> Just some cleanup for docs. >>>> >>>> Thanks, >>>> Alexander >> >> >> Lance Andersen| Principal Member of Technical Staff | +1.781.442.2037 >> Oracle Java Engineering >> 1 Network Drive >> Burlington, MA 01803 >> Lance.Andersen at oracle.com >> >> >> > From maxim.soloviev at oracle.com Fri Jun 5 15:39:13 2015 From: maxim.soloviev at oracle.com (Maxim Soloviev) Date: Fri, 05 Jun 2015 18:39:13 +0300 Subject: [8u60] approval request for JDK-8062198: Add RowSetMetaDataImpl Tests and add column range validation to isdefinitlyWritable Message-ID: <5571C2A1.6000300@oracle.com> Please approve direct backport. bug: https://bugs.openjdk.java.net/browse/JDK-8062198 jdk9 change set: http://hg.openjdk.java.net/jdk9/jdk9/jdk/rev/4162bcc663dc Thanks, Maxim From maxim.soloviev at oracle.com Fri Jun 5 15:45:47 2015 From: maxim.soloviev at oracle.com (Maxim Soloviev) Date: Fri, 05 Jun 2015 18:45:47 +0300 Subject: [8u60] approval request for JDK-8059411: RowSetWarning does not correctly chain warnings Message-ID: <5571C42B.7020701@oracle.com> Please approve direct backport. bug: https://bugs.openjdk.java.net/browse/JDK-8059411 jdk9 change set: http://hg.openjdk.java.net/jdk9/jdk9/jdk/rev/4110a7627857 Thanks, Maxim From maxim.soloviev at oracle.com Fri Jun 5 15:50:22 2015 From: maxim.soloviev at oracle.com (Maxim Soloviev) Date: Fri, 05 Jun 2015 18:50:22 +0300 Subject: [8u60] approval request for JDK-8066188: BaseRowSet returns the wrong default value for escape processing Message-ID: <5571C53E.3050908@oracle.com> Please approve direct backport. bug: https://bugs.openjdk.java.net/browse/JDK-8066188 jdk9 change set: http://hg.openjdk.java.net/jdk9/jdk9/jdk/rev/f619341171c0 Thanks, Maxim From rob.mckenna at oracle.com Fri Jun 5 15:51:35 2015 From: rob.mckenna at oracle.com (Rob McKenna) Date: Fri, 05 Jun 2015 16:51:35 +0100 Subject: [8u60] approval request for JDK-8062198: Add RowSetMetaDataImpl Tests and add column range validation to isdefinitlyWritable In-Reply-To: <5571C2A1.6000300@oracle.com> References: <5571C2A1.6000300@oracle.com> Message-ID: <5571C587.4030400@oracle.com> Approved. -Rob On 05/06/15 16:39, Maxim Soloviev wrote: > Please approve direct backport. > > bug: https://bugs.openjdk.java.net/browse/JDK-8062198 > jdk9 change set: http://hg.openjdk.java.net/jdk9/jdk9/jdk/rev/4162bcc663dc > > Thanks, > Maxim > From rob.mckenna at oracle.com Fri Jun 5 15:51:40 2015 From: rob.mckenna at oracle.com (Rob McKenna) Date: Fri, 05 Jun 2015 16:51:40 +0100 Subject: [8u60] approval request for JDK-8059411: RowSetWarning does not correctly chain warnings In-Reply-To: <5571C42B.7020701@oracle.com> References: <5571C42B.7020701@oracle.com> Message-ID: <5571C58C.1000809@oracle.com> Approved. -Rob On 05/06/15 16:45, Maxim Soloviev wrote: > Please approve direct backport. > > bug: https://bugs.openjdk.java.net/browse/JDK-8059411 > jdk9 change set: http://hg.openjdk.java.net/jdk9/jdk9/jdk/rev/4110a7627857 > > Thanks, > Maxim From rob.mckenna at oracle.com Fri Jun 5 15:52:35 2015 From: rob.mckenna at oracle.com (Rob McKenna) Date: Fri, 05 Jun 2015 16:52:35 +0100 Subject: [8u60] approval request for JDK-8066188: BaseRowSet returns the wrong default value for escape processing In-Reply-To: <5571C53E.3050908@oracle.com> References: <5571C53E.3050908@oracle.com> Message-ID: <5571C5C3.3070401@oracle.com> Approved. -Rob On 05/06/15 16:50, Maxim Soloviev wrote: > Please approve direct backport. > > bug: https://bugs.openjdk.java.net/browse/JDK-8066188 > jdk9 change set: http://hg.openjdk.java.net/jdk9/jdk9/jdk/rev/f619341171c0 > > Thanks, > Maxim From alexander.v.stepanov at oracle.com Fri Jun 5 17:33:42 2015 From: alexander.v.stepanov at oracle.com (alexander stepanov) Date: Fri, 05 Jun 2015 20:33:42 +0300 Subject: RFR [9] 8081517: minor cleanup for docs In-Reply-To: References: <55719B6A.4000904@oracle.com> <5571AD67.4040203@Oracle.com> Message-ID: <5571DD76.6040000@oracle.com> Hello Lance, Roger, Thank you for the notes, please see the updated webrev: http://cr.openjdk.java.net/~avstepan/8081517/webrev.01/index.html (files changed: FtpURLConnection.java, VmIdentifier.java, TraceClassVisitor.java, HttpAuthenticator.java, PlatformLogger.java and JdbcRowSetImpl.java - some indents were fixed) > SQL AS clause that's probably used for the SQL "AS" keyword. In such a case the upper case may be suitable. Regards, Alexander On 05.06.2015 17:12, Lance Andersen wrote: > Hi Roger > On Jun 5, 2015, at 10:08 AM, Roger Riggs wrote: > >> 2) >> If indentation is in the scope of your cleanup? > Agree with you and I did not comment on this as I felt it this should be done separately as there is a fair amount that can be done in these classes for clean up. > > Best > lance >> >> In JdbcRowSetImpl.java, the indentation of continued lines after @return or @param tags >> isn't correct; the continued line should be indented. >> >> And there are a some very long lines that should be wrapped; >> See +5637... "If the SQL AS clause was not..." >> >> +5784 the indentation needs to be fixed. >> +5840 also >> +5988 >> +6265 and after... >> in many cases the lines that follow of "/*" do not align the "* ". >> >> CachedRowSetWriter.java: >> - many method comment blocks are not aligned properly with the method of field that follows. >> For example, >> /** >> * The Connection object that this writer will use to make a >> * connection to the data source to which it will write data. >> * >> */ >> private transient Connection con; >> >> 3) >> >> CachedRowSetWriter.java: >> +1056: The tag "@ param" should be tighted up to "@param" >> >> Thanks, Roger >> >> >> >> On 6/5/2015 8:51 AM, alexander stepanov wrote: >>> Hello, >>> >>> Could you please review the fix >>> http://cr.openjdk.java.net/~avstepan/8081517/webrev.00/ >>> for >>> https://bugs.openjdk.java.net/browse/JDK-8081517 >>> >>> Just some cleanup for docs. >>> >>> Thanks, >>> Alexander > > > Lance Andersen| Principal Member of Technical Staff | +1.781.442.2037 > Oracle Java Engineering > 1 Network Drive > Burlington, MA 01803 > Lance.Andersen at oracle.com > > > From lance.andersen at oracle.com Fri Jun 5 18:05:22 2015 From: lance.andersen at oracle.com (Lance Andersen) Date: Fri, 5 Jun 2015 14:05:22 -0400 Subject: RFR [9] 8081517: minor cleanup for docs In-Reply-To: <5571DD76.6040000@oracle.com> References: <55719B6A.4000904@oracle.com> <5571AD67.4040203@Oracle.com> <5571DD76.6040000@oracle.com> Message-ID: <8513786A-B815-46AB-B70F-9D3B3F0C0FAD@oracle.com> Hi Alexander On Jun 5, 2015, at 1:33 PM, alexander stepanov wrote: > Hello Lance, Roger, > > Thank you for the notes, please see the updated webrev: > http://cr.openjdk.java.net/~avstepan/8081517/webrev.01/index.html > (files changed: FtpURLConnection.java, VmIdentifier.java, TraceClassVisitor.java, HttpAuthenticator.java, PlatformLogger.java and JdbcRowSetImpl.java - some indents were fixed) Looks OK to me. Thank you for making the additional updates > > > SQL AS clause > that's probably used for the SQL "AS" keyword. In such a case the upper case may be suitable. Yes should be upper case Best Lance > > Regards, > Alexander > > > On 05.06.2015 17:12, Lance Andersen wrote: >> Hi Roger >> On Jun 5, 2015, at 10:08 AM, Roger Riggs wrote: >> >>> 2) >>> If indentation is in the scope of your cleanup? >> Agree with you and I did not comment on this as I felt it this should be done separately as there is a fair amount that can be done in these classes for clean up. >> >> Best >> lance >>> >>> In JdbcRowSetImpl.java, the indentation of continued lines after @return or @param tags >>> isn't correct; the continued line should be indented. >>> >>> And there are a some very long lines that should be wrapped; >>> See +5637... "If the SQL AS clause was not..." >>> >>> +5784 the indentation needs to be fixed. >>> +5840 also >>> +5988 >>> +6265 and after... >>> in many cases the lines that follow of "/*" do not align the "* ". >>> >>> CachedRowSetWriter.java: >>> - many method comment blocks are not aligned properly with the method of field that follows. >>> For example, >>> /** >>> * The Connection object that this writer will use to make a >>> * connection to the data source to which it will write data. >>> * >>> */ >>> private transient Connection con; >>> >>> 3) >>> >>> CachedRowSetWriter.java: >>> +1056: The tag "@ param" should be tighted up to "@param" >>> >>> Thanks, Roger >>> >>> >>> >>> On 6/5/2015 8:51 AM, alexander stepanov wrote: >>>> Hello, >>>> >>>> Could you please review the fix >>>> http://cr.openjdk.java.net/~avstepan/8081517/webrev.00/ >>>> for >>>> https://bugs.openjdk.java.net/browse/JDK-8081517 >>>> >>>> Just some cleanup for docs. >>>> >>>> Thanks, >>>> Alexander >> >> >> Lance Andersen| Principal Member of Technical Staff | +1.781.442.2037 >> Oracle Java Engineering >> 1 Network Drive >> Burlington, MA 01803 >> Lance.Andersen at oracle.com >> >> >> > Lance Andersen| Principal Member of Technical Staff | +1.781.442.2037 Oracle Java Engineering 1 Network Drive Burlington, MA 01803 Lance.Andersen at oracle.com From vladimir.x.ivanov at oracle.com Fri Jun 5 18:05:27 2015 From: vladimir.x.ivanov at oracle.com (Vladimir Ivanov) Date: Fri, 05 Jun 2015 21:05:27 +0300 Subject: [8u-dev] Review request : JDK-8062904: TEST_BUG: Tests java/lang/invoke/LFCaching fail when run with -Xcomp option In-Reply-To: <5571A2D6.8020508@oracle.com> References: <5565D9BB.60003@oracle.com> <5565F6D5.7030900@oracle.com> <5566D9D8.4090401@oracle.com> <55675CCB.8010506@oracle.com> <5568281B.7040108@oracle.com> <55685264.20001@oracle.com> <556C86C6.2000409@oracle.com> <556DFB1C.7010301@oracle.com> <556ECC99.2000702@oracle.com> <55717713.80606@oracle.com> <5571A2D6.8020508@oracle.com> Message-ID: <5571E4E7.4050903@oracle.com> Konstantin, >> I'd like to get an answer to my previous question: >> "How reliable the test is if it ignores NoSuchMethodException & >> VirtualMachineError? Are there other manifestations of the problem?" >> >> Most notably, have you seen VM crashes w/ -Xcomp running that test? > I have seen no crashes with -Xcomp in our tests results base, only > NoSuchMethodException and VirtualMachineError. Good. Thanks for the info. > This failure does not occur with JDK 9 on -Xcomp, I used the same random > seeds (-Dseed), and the problem exists only with JDK 8u, not JDK 9. > > Because there is no failure with JDK 9 I can suppose there could be a > product issue in JDK 8u. That's strange. There should be no difference in MethodHandle stub management between 8u & 9 now (once allocated they stay forever). 9 has segmented code cache feature and it can lead to observable differences in behavior. How many iterations does the test perform on 8u & 9? What if you increase the number? Also, I'd suggest you to monitor code cache usage during the run and when the test finishes. Best regards, Vladimir Ivanov From staffan.friberg at oracle.com Fri Jun 5 18:09:48 2015 From: staffan.friberg at oracle.com (Staffan Friberg) Date: Fri, 05 Jun 2015 11:09:48 -0700 Subject: RFR 8080640: Reduce copying when reading JAR/ZIP entries In-Reply-To: <556F3D3F.3020906@oracle.com> References: <555A957F.5010102@oracle.com> <555CCAFB.4090805@oracle.com> <555E0C62.9090406@oracle.com> <555E1D49.9000707@oracle.com> <555F7862.9030307@oracle.com> <555F7AB8.1050202@oracle.com> <555F8E78.40404@oracle.com> <556DE778.1040300@oracle.com> <556F3D3F.3020906@oracle.com> Message-ID: <5571E5EC.6090907@oracle.com> Hi Sherman, I have a new webrev which reverts that part, http://cr.openjdk.java.net/~sfriberg/JDK-8080640/webrev.2/ Summary of changes Reduce lock region in ZipFile.getInputstream Add private ZipFile.getBytes that can be used in select places in the JDK where all bytes will be read Could you sponsor this change once it has been reviewed? Thanks, Staffan On 06/03/2015 10:45 AM, Xueming Shen wrote: > Staffan, > > I'm not convinced that the benefit here is significant enough to > change the > getInputStream() to return a ByteArrayInputStream, given this can be > easily > achieved by wrapping the returned byte[] from getBytes(ZipEntry) at > user's > site. I would suggest to file a separate rfe on this disagreement and > move on > with the agreed getBytes() for now. > > Thanks, > -Sherman > > On 06/02/2015 10:27 AM, Staffan Friberg wrote: >> >> On 05/22/2015 01:15 PM, Staffan Friberg wrote: >>> On 05/22/2015 11:51 AM, Xueming Shen wrote: >>>> On 05/22/2015 11:41 AM, Staffan Friberg wrote: >>>>> >>>>> On 05/21/2015 11:00 AM, Staffan Friberg wrote: >>>>>> >>>>>> On 05/21/2015 09:48 AM, Staffan Friberg wrote: >>>>>>> >>>>>>> On 05/20/2015 10:57 AM, Xueming Shen wrote: >>>>>>>> On 05/18/2015 06:44 PM, Staffan Friberg wrote: >>>>>>>>> Hi, >>>>>>>>> >>>>>>>>> Wanted to get reviews and feedback on this performance >>>>>>>>> improvement for reading from JAR/ZIP files during classloading >>>>>>>>> by reducing unnecessary copying and reading the entry in one >>>>>>>>> go instead of in small portions. This shows a significant >>>>>>>>> improvement when reading a single entry and for a large >>>>>>>>> application with 10k classes and 500+ JAR files it improved >>>>>>>>> the startup time by 4%. >>>>>>>>> >>>>>>>>> For more details on the background and performance results >>>>>>>>> please see the RFE entry. >>>>>>>>> >>>>>>>>> RFE - https://bugs.openjdk.java.net/browse/JDK-8080640 >>>>>>>>> WEBREV - >>>>>>>>> http://cr.openjdk.java.net/~sfriberg/JDK-8080640/webrev.0 >>>>>>>>> >>>>>>>>> Cheers, >>>>>>>>> Staffan >>>>>>>> >>>>>>>> Hi Staffan, >>>>>>>> >>>>>>>> If I did not miss something here, from your use scenario it >>>>>>>> appears to me the only thing you really >>>>>>>> need here to help boost your performance is >>>>>>>> >>>>>>>> byte[] ZipFile.getAllBytes(ZipEntry ze); >>>>>>>> >>>>>>>> You are allocating a byte[] at use side and wrapping it with a >>>>>>>> ByteBuffer if the size is small enough, >>>>>>>> otherwise, you letting the ZipFile to allocate a big enough one >>>>>>>> for you. It does not look like you >>>>>>>> can re-use that byte[] (has to be wrapped by the >>>>>>>> ByteArrayInputStream and return), why do you >>>>>>>> need two different methods here? The logic would be much easier >>>>>>>> to simply let the ZipFile to allocate >>>>>>>> the needed buffer with appropriate size, fill the bytes and >>>>>>>> return, with a "OOME" if the entry size >>>>>>>> is bigger than 2g. >>>>>>>> >>>>>>>> The only thing we use from the input ze is its name, get the >>>>>>>> size/csize from the jzentry, I don't think >>>>>>>> jzentry.csize/size can be "unknown", they are from the "cen" >>>>>>>> table. >>>>>>>> >>>>>>>> If the real/final use of the bytes is to wrap it with a >>>>>>>> ByteArrayInputStream,why bother using ByteBuffer >>>>>>>> here? Shouldn't a direct byte[] with exactly the size of the >>>>>>>> entry server better. >>>>>>>> >>>>>>>> -Sherman >>>>>>>> >>>>>>> Hi Sherman, >>>>>>> >>>>>>> Thanks for the comments. I agree, was starting out with >>>>>>> bytebuffer because I was hoping to be able to cache things where >>>>>>> the buffer was being used, but since the buffer is past along >>>>>>> further I couldn't figure out a clean way to do it. >>>>>>> Will rewrite it to simply just return a buffer, and only wrap it >>>>>>> in the Resource class getByteBuffer. >>>>>>> >>>>>>> What would be your thought on updating the >>>>>>> ZipFile.getInputStream to return ByteArrayInputStream for small >>>>>>> entries? Currently I do that work outside in two places and >>>>>>> moving it would potentially speed up others reading small >>>>>>> entries as well. >>>>>>> >>>>>>> Thanks, >>>>>>> Staffan >>>>>> Just realized that my use of ByteArrayInputStream would miss Jar >>>>>> verification if enabled so the way to go hear would be to add it >>>>>> if possible to the ZipFile.getInputStream. >>>>>> >>>>>> //Staffan >>>>> Hi, >>>>> >>>>> Here is an updated webrev which uses a byte[] directly and also >>>>> uses ByteArrayInputStream in ZipFile for small entries below 128k. >>>>> >>>> >>>> I'm not sure about the benefit of doing the ByteArrayInputStream in >>>> ZipFile.getInputStream. It has >>>> the consequence of changing the "expected" behavior of >>>> getInputStream() (instead of return an >>>> input stream waiting for reading, it now reads all bytes in >>>> advance), something we might not want >>>> to do in a performance tuning. Though it might be reasonable to >>>> guess everyone get an input stream >>>> is to read all bytes from it later. >>>> >>>> -Sherman >>>> >>>> >>>>> http://cr.openjdk.java.net/~sfriberg/JDK-8080640/webrev.1 >>>>> >>>>> //Staffan >>>> >>> Agree that it will change the behavior slightly, but as you said it >>> is probably expected that some one will read the stream eventually. >>> We could reduce the size further if that makes a difference, if the >>> size is below 65k we would not use more memory than the buffer >>> allocated for the InflaterStream today. >>> The total allocation would be slightly larger for deflated entries >>> as we would allocate a byte[] for the compressed bytes, but it would >>> be GC:able and not kept alive. So from a memory perspective the >>> difference is very limited. >>> >>> //Staffan >> Hi, >> >> Bumping this thread to get some more comments on the concern about >> changing the ZipFile.getInputStream behavior. The benefit of doing >> this change is that any read of small entries from ZIP and JAR files >> will be much faster and less resources will be held, including native >> resources normally held by the ZipInputStream. >> >> The behavior change that will occur is that the full entry will be >> read as part of creating the stream and not lazily as might be >> expected. However when getting a today InputStream zip file will be >> accessed to read information about the size of the entry, so the zip >> file is already touched when getting an InputStream, but not the >> compressed bytes. >> >> I'm fine with removing this part of the change and just push the >> private getBytes function and the updates to the JDK libraries to use >> it. >> >> Thanks, >> Staffan > From Roger.Riggs at Oracle.com Fri Jun 5 18:32:14 2015 From: Roger.Riggs at Oracle.com (Roger Riggs) Date: Fri, 05 Jun 2015 14:32:14 -0400 Subject: RFR [9] 8081517: minor cleanup for docs In-Reply-To: <5571DD76.6040000@oracle.com> References: <55719B6A.4000904@oracle.com> <5571AD67.4040203@Oracle.com> <5571DD76.6040000@oracle.com> Message-ID: <5571EB2E.8000200@Oracle.com> Hi Alexander, Looks good, thanks for the updates. Roger On 6/5/2015 1:33 PM, alexander stepanov wrote: > Hello Lance, Roger, > > Thank you for the notes, please see the updated webrev: > http://cr.openjdk.java.net/~avstepan/8081517/webrev.01/index.html > (files changed: FtpURLConnection.java, VmIdentifier.java, > TraceClassVisitor.java, HttpAuthenticator.java, PlatformLogger.java > and JdbcRowSetImpl.java - some indents were fixed) > > > SQL AS clause > that's probably used for the SQL "AS" keyword. In such a case the > upper case may be suitable. > > Regards, > Alexander > > > On 05.06.2015 17:12, Lance Andersen wrote: >> Hi Roger >> On Jun 5, 2015, at 10:08 AM, Roger Riggs wrote: >> >>> 2) >>> If indentation is in the scope of your cleanup? >> Agree with you and I did not comment on this as I felt it this should >> be done separately as there is a fair amount that can be done in >> these classes for clean up. >> >> Best >> lance >>> >>> In JdbcRowSetImpl.java, the indentation of continued lines after >>> @return or @param tags >>> isn't correct; the continued line should be indented. >>> >>> And there are a some very long lines that should be wrapped; >>> See +5637... "If the SQL AS clause was not..." >>> >>> +5784 the indentation needs to be fixed. >>> +5840 also >>> +5988 >>> +6265 and after... >>> in many cases the lines that follow of "/*" do not align the "* ". >>> >>> CachedRowSetWriter.java: >>> - many method comment blocks are not aligned properly with the >>> method of field that follows. >>> For example, >>> /** >>> * The Connection object that this writer will use to >>> make a >>> * connection to the data source to which it will write data. >>> * >>> */ >>> private transient Connection con; >>> >>> 3) >>> >>> CachedRowSetWriter.java: >>> +1056: The tag "@ param" should be tighted up to "@param" >>> >>> Thanks, Roger >>> >>> >>> >>> On 6/5/2015 8:51 AM, alexander stepanov wrote: >>>> Hello, >>>> >>>> Could you please review the fix >>>> http://cr.openjdk.java.net/~avstepan/8081517/webrev.00/ >>>> for >>>> https://bugs.openjdk.java.net/browse/JDK-8081517 >>>> >>>> Just some cleanup for docs. >>>> >>>> Thanks, >>>> Alexander >> >> >> Lance Andersen| Principal Member of Technical Staff | +1.781.442.2037 >> Oracle Java Engineering >> 1 Network Drive >> Burlington, MA 01803 >> Lance.Andersen at oracle.com >> >> >> > From huizhe.wang at oracle.com Fri Jun 5 19:15:04 2015 From: huizhe.wang at oracle.com (huizhe wang) Date: Fri, 05 Jun 2015 12:15:04 -0700 Subject: Review request for JDK-8080906 & JDK-8080908: Develop tests for JEP 255 Xerces Updates In-Reply-To: <6860E5EE-B032-4F42-9C8A-A07973419767@oracle.com> References: <013b01d09f63$a0d1ff40$e275fdc0$@oracle.com> <6860E5EE-B032-4F42-9C8A-A07973419767@oracle.com> Message-ID: <5571F538.40401@oracle.com> Yes, the tests look good to me too :-) I'd suggest a couple of things: Bug6794483Test: add 8080908 to the bug tag, and year "2015," to the header. I added a link to JDK-6794483 from JDK-8080908. LSSerializerTest: was missing a bug tag. I believe the original test was for 6439439, so add: @bug 6439439, 8080906 to the test class and @bug 8080906 to test "testDefaultLSSerializer". This test would have been helpful in detecting the failure of loading LSSerializerImpl in a Jigsaw build. You may add a note to the test to indicate that it would fail in a Jigsaw build, and that the issue will be fixed in JDK-8080266. The changeset will be split into two (JDK-8080906 & JDK-8080908) when check in. Thanks, Joe On 6/5/2015 4:29 AM, Lance Andersen wrote: > Hi Frank, > > These seem OK. I am sure Joe will also review when he wakes up in his > part of the world :-) > > Best > Lance > On Jun 5, 2015, at 3:45 AM, Frank Yuan > wrote: > >> Hi Joe and all >> >> >> >> I have been working on the test task of JEP 255 Xerces Updates. >> >> >> >> Here I would invite you to review the changes for 2 bugs of this task: >> >> >> >> 1. JDK-8080906 Develop test for Xerces Update: DOM L3 Serializer >> >> To verify default LSSerializer is Xalan dom 3 serializer >> >> >> >> 2. JDK-8080908 Develop test for Xerces Update: XPointer >> >> To verify Xerces revision 415823: XERCESJ-1134. It should have been >> fixed >> in JDK-6794483, however, the test Bug6794483Test.java need to be >> revised to >> really cover this bug. >> >> >> >> >> >> I also added module dependencies for exported API in this path, it's >> recommended for Jigsaw change, refer to >> https://wiki.se.oracle.com/display/JPG/@modules+in+JTReg+tests >> >> >> >> The webrev is at >> http://cr.openjdk.java.net/~fyuan/8080906_8080908/webrev.00/, your >> comment >> will be appreciated. >> >> >> >> Best Regards >> >> Frank >> > > > > Lance > Andersen| Principal Member of Technical Staff | +1.781.442.2037 > Oracle Java Engineering > 1 Network Drive > Burlington, MA 01803 > Lance.Andersen at oracle.com > > > From core-libs at jpayne.net Fri Jun 5 21:11:08 2015 From: core-libs at jpayne.net (Jonathan Payne) Date: Fri, 5 Jun 2015 22:11:08 +0100 Subject: Another take on Finalization Message-ID: <0C527AE7-DED0-4731-9522-CECA3F618B9F@jpayne.net> Hi. I have had an interest in the topic of finalization ever since it caused me to abandon the G1 collector 3 or 4 years ago. I?ve recently implemented a fix for my interpretation of the problem, which might be very different from the discussion currently ongoing in the thread entitled "JEP 132: More-prompt finalization?. My problem was that finalization was not being run at all with the G1 collector. Not at all. That would have been fine with me because none of the existing objects in the Finalizer queue actually needed the service anymore: the files, sockets, streams, etc. had all been properly closed by my application, otherwise the server would have long since failed completely. However, those objects started to accumulate in the VM and eventually (8 hours later) brought the server down. Which brings me to a few points: Finalization as conceived in the early JDKs was a bad idea. To make matters worse, the way we then made use of it in those early days was A REALLY REALLY bad idea. None of this mattered in those days because the GC ran often and quickly and finalization occurred during every GC cycle. There may be situations where finalization as a feature actually matters, but in the intervening years the JDK has added new technologies that provide a way to accomplish finalization on your own, in your own code. A few helper classes and it might even be easy when it?s necessary, which is hopefully almost never. Many of the uses of finalize() in the JDK today are bad and should be deleted. My fix, BTW, was to use a back door (that I added to SharedSecrets) in all the JDK classes that had a finalize() method, so that when a resource is properly closed, by calling the close() method for example, the back door would remove the Finalizer for the specified object from the linked list of Finalizer objects, thus removing it from the finalization equation altogether. I implemented this, and then the various tests of creating a huge number of objects with a finalize() method ran quickly and flawlessly with no horrific GCs or even a growing memory pool. The main problem with my solution was that there was this nasty SharedSecrets back door, so it has been rejected and probably rightly so. However, it proved a point. But now I am wondering why the actual right thing to do is not simply this: Remove the finalize() method from all the worst offenders in the JDK. I cannot remember all the places I patched when I implemented my fix, but the majority of them were pieces of code that absolutely had a close() method. If you don?t close objects when you?re done with them, your program PROBABLY SHOULD BE BROKEN. But even if you do not accept that, for all practical purposes, the program IS broken today because finalization is absolutely NOT run in a timely enough fashion. BTW - I never understood why CMS and other GC?s had absolutely no problem running finalization in a very timely fashion while the G1 collector just never seemed to get around to it. My interpretation of that fact has always led me to believe that it?s not a throughput issue with the finalization thread (not in real world examples, anyway) but rather a GC implementation that didn?t feel the need to be thorough enough to make sure something is ready to be finalized. I mean, when the G1 collector was forced to run a full collection (a death sentence on a 15Gb heap but it did occur) all the finalizable objects were found AND finalized immediately, all 15 or 20 million of them. So in summary: (1) The problem with finalization is that people use it. And more importantly, that the JDK is filled with inappropriate uses of it. (2) The main solution is probably just to delete the inappropriate uses in the JDK. But if that?s not OK, then some sort of patch like what I did which allows the JDK classes to unregister the Finalizer?s when they are no longer needed, i.e., when the object knows that it has cleaned itself up. I am curious to hear your thoughts. JP From ecki at zusammenkunft.net Fri Jun 5 23:15:16 2015 From: ecki at zusammenkunft.net (Bernd Eckenfels) Date: Sat, 6 Jun 2015 01:15:16 +0200 Subject: Another take on Finalization In-Reply-To: <0C527AE7-DED0-4731-9522-CECA3F618B9F@jpayne.net> References: <0C527AE7-DED0-4731-9522-CECA3F618B9F@jpayne.net> Message-ID: <20150606011516.00005523.ecki@zusammenkunft.net> Am Fri, 5 Jun 2015 22:11:08 +0100 schrieb Jonathan Payne : > My problem was that finalization was not being run at all with the G1 > collector. The problem that an object is not detected as unreachable and not enqueued into the finalizer queue is not only a G1 problem. When an application has very short transactions and no objects get promoted then G1 and CMS (and PrallelOld) trigger very seldom an old GC, so this leads to a long time that finalizeable objects do not get cleaned up. Not sure if G1 is especially bad in that regards. I think it might be related to the bug that it never responded to the DGC cleanup unlike other GCs? Something similar to calling System.gc() every hour but with less negative effects would be a helpful tool for those situations. This is also a problem for other memory pools (class loaders, code cache, constant pool, direct buffers) as well. So having a slow background cleanup is a good thing in all cases. > My fix, BTW, was to use a back door (that I added to SharedSecrets) > in all the JDK classes that had a finalize() method, so that when a > resource is properly closed, by calling the close() method for > example, the back door would remove the Finalizer for the specified > object from the linked list of Finalizer objects I think it would be good to offer such an API to early remove an object from the reference queue. Gruss Bernd From stuart.marks at oracle.com Fri Jun 5 23:20:12 2015 From: stuart.marks at oracle.com (Stuart Marks) Date: Fri, 05 Jun 2015 16:20:12 -0700 Subject: RFR 8071597 Add Stream dropWhile and takeWhile operations In-Reply-To: References: <79392DA7-6A8E-42AC-9E83-AA7F89D8F2A5@oracle.com> <556E60F3.7060800@oracle.com> Message-ID: <55722EAC.6020503@oracle.com> On 6/3/15 9:21 AM, Paul Sandoz wrote: > I had prepared an alternative rendition stashed away just in case this came up :-) > > I still want to retain a punchy short first paragraph. What do you think about the following? > > /** > - * Returns a stream consisting of the longest prefix of elements taken from > - * this stream that match the given predicate. > + * Returns a stream consisting of elements taken from this stream that match > + * the given predicate. Well, I'm sorry to say, this reminds me of that old tech support joke about the answer that is technically true but completely useless. :-) It's not incorrect, of course, but it's almost indistinguishable from the first sentence of Stream.filter(). People who know what "takeWhile" means will raise an eyebrow at this, go read the rest of the doc, and go "ohhh, yeah there's this complication with unordered streams." People who are trying to learn the system will go "Huh?" I agree on the need for a punchy first sentence since that's what goes into the Method Summary. The difficulty is making something sufficiently general that it covers all the cases, while not making it so vague as to be useless. Sorry, but I don't have any better suggestions at the moment. I might be able to come up with something in the next few days... but don't let this hold you up if you need to press forward. s'marks From peter.levart at gmail.com Sat Jun 6 15:02:51 2015 From: peter.levart at gmail.com (Peter Levart) Date: Sat, 06 Jun 2015 17:02:51 +0200 Subject: JEP 132: More-prompt finalization (Re: Another take on Finalization) In-Reply-To: <556C200B.3030603@gmail.com> References: <55674C6E.8050509@gmail.com> <5567CC5E.7000102@oracle.com> <8DBC7345-09D8-458F-9017-298D1E620F86@gmail.com> <6882C9A35DFB9B4FA2779F7BF5B9757D20761169A2@GSCMAMP06EX.firmwide.corp.gs.com> <556B61B0.5030708@gmail.com> <6882C9A35DFB9B4FA2779F7BF5B9757D20761169E3@GSCMAMP06EX.firmwide.corp.gs.com> <556C200B.3030603@gmail.com> Message-ID: <55730B9B.6060603@gmail.com> Hi Jonathan, On 06/05/2015 11:11 PM, Jonathan Payne wrote: > Hi. > > I have had an interest in the topic of finalization ever since it caused me to abandon the G1 collector 3 or 4 years ago. > > I?ve recently implemented a fix for my interpretation of the problem, which might be very different from the discussion currently ongoing in the thread entitled "JEP 132: More-prompt finalization?. > > My problem was that finalization was not being run at all with the G1 collector. Not at all. That would have been fine with me because none of the existing objects in the Finalizer queue actually needed the service anymore: the files, sockets, streams, etc. had all been properly closed by my application, otherwise the server would have long since failed completely. However, those objects started to accumulate in the VM and eventually (8 hours later) brought the server down. The most probable cause of the described behavior might be that your finalizable objects live long enough so that they are promoted to the old-generation together with their tracking Finalizer(s) (FinalReferences). We know that a finalizable object must undergo at least 2 GC cycles to be reclaimed. The 1st cycle merely discovers the final-reachable objects and links their FinalReference(s) to the pending chain. ReferenceHandler thread then pushes them to the finalization queue where their final() methods are invoked by finalizer thread which also clears their associated FinalReference(s) and unlinks them from the double-linked list. During the 2nd cycle, such finalizable objects are usually found unreachable and are reclaimed. It might be that G1 has (or had) a performance problem so that it couldn't discover final-reachable objects in old-generation soon enough to push them through 2 cycles before the system choked. This is roughly similar to a problem you might have if lots of normal objects get promoted to old-generation, but it might be the effects are more drastic in G1 when those objects are finalizable. So you might be able to tune your app by increasing the young generation and/or the time it takes before objects are promoted to old-generation. I understand that it would be desirable for a finalizable object to be made "untracked" as soon as it is manually cleaned-up. This would most certainly give a relief to GC as it could reclaim such untracked objects immediately like normal objects without pushing them through all the finalization steps. Such feature would need new API. Here's my take on such API incorporated in my prototype. This feature is mostly independent of other changes in the prototype and could be provided stand-alone. I choose to incorporate it into the prototype to compare it's overhead with classical finalization in unchanged and changed JDKs: http://cr.openjdk.java.net/~plevart/misc/JEP132/ReferenceHandling/webrev.03/ The java.lang.ref.Finalizator is a public subclass of package-private Finalizer which is a subclass of package-private FinalReference which is a subclass of public Reference. The public API therefore consists just of two types: Reference and Finalizator and the public methods either implemented or inherited by Finalizator. Finalizator is basically just a special kind of Reference, which can't be subclassed (is final), can't be registeread with a custom reference-queue, and can only be constructed using a Finalizator.create(finalizee, thunk) factory method taking a 'finalizee' to be tracked and a 'thunk' that is usually just an adapter for invoking a private cleanup method on the finalizee. Finalizator also implements java.lang.Runnable. It's run() method is invoked by finalization infrastructure or manually by user code that wishes to promptly trigger clean-up (from AutoCloseable.close() method for example). Here's how a classical finalizable class that also implements AutoCloseable might be implemented. Note that the class must implement it's own logic to make clean-up idempotent, since finalize() will be called even after close() has manually or automatically already been called: static final class Finalizable extends AtomicBoolean implements AutoCloseable { @Override protected void finalize() throws Throwable { close(); } @Override public void close() { // close must be idempotent if (compareAndSet(false, true)) { // clean-up invoked at most once } } } And here's how an alternative celanup might be implemented using Finalizator. Finalizator already guarantees that it's 'thunk' will be called at most once regardless of whether it was triggered by GC and/or manually: static final class Destroyable implements AutoCloseable { final Finalizator finalizator = Finalizator.create(this, Destroyable::destroy); void destroy() { // clean-up invoked at most once } @Override public void close() { // close just runs the finalizator finalizator.run(); } } As soon as Finalizator is run() 1st time, it is cleared and unlinked from the doubly-linked list. After that, GC can reclaim it and the finalizee right away without pushing them through the discovery and reference processing pipeline only to unlink the Finalizer from the doubly-linked list. I have done some testing and the results of creating and destroying 100M objects with a sustained rate of ~90 objects/ms with or without performing (AutoCloseable) clean-up immediately after construction gives the following results: Finalization, ORIGINAL real 2m5.958s user 0m33.855s sys 0m1.982s AutoCloseable combined with Finalization, ORIGINAL real 2m0.952s user 0m32.103s sys 0m1.730s Finalization, PATCHED real 2m0.519s user 0m16.664s sys 0m1.240s AutoCloseable combined with Finalization, PATCHED real 1m55.641s user 0m16.872s sys 0m1.218s Finalizator-based cleanup, PATCHED real 2m1.379s user 0m17.422s sys 0m1.321s AutoCloseable combined with Finalizator-based cleanup, PATCHED real 1m55.169s user 0m4.167s sys 0m1.139s We see what I have already shown before that my prototype practically halves the CPU overhead of finalization infrastructure. Just making an object AutoCloseable and promptly doing the clean-up can not reduce this overhead if the object is also finalizable. But if manual clean-up also "unregisters" the Finalizator from the doubly-linked list and clears it, it spares the finalization infrastructure from processing it as a finalizable object which further reduces the CPU overhead for a factor of 4, totaling just 1/8th of overhead of classic finalization with current JDK. Besides greatly reduced CPU overhead, such objects are also potentially more promptly reclaimed by GC, freeing memory for other more useful things... > Which brings me to a few points: > > Finalization as conceived in the early JDKs was a bad idea. To make matters worse, the way we then made use of it in those early days was A REALLY REALLY bad idea. > None of this mattered in those days because the GC ran often and quickly and finalization occurred during every GC cycle. > There may be situations where finalization as a feature actually matters, but in the intervening years the JDK has added new technologies that provide a way to accomplish finalization on your own, in your own code. A few helper classes and it might even be easy when it?s necessary, which is hopefully almost never. > Many of the uses of finalize() in the JDK today are bad and should be deleted. > > My fix, BTW, was to use a back door (that I added to SharedSecrets) in all the JDK classes that had a finalize() method, so that when a resource is properly closed, by calling the close() method for example, the back door would remove the Finalizer for the specified object from the linked list of Finalizer objects, thus removing it from the finalization equation altogether. I implemented this, and then the various tests of creating a huge number of objects with a finalize() method ran quickly and flawlessly with no horrific GCs or even a growing memory pool. The main problem with my solution was that there was this nasty SharedSecrets back door, so it has been rejected and probably rightly so. > > However, it proved a point. > > But now I am wondering why the actual right thing to do is not simply this: > > Remove the finalize() method from all the worst offenders in the JDK. > > I cannot remember all the places I patched when I implemented my fix, but the majority of them were pieces of code that absolutely had a close() method. If you don?t close objects when you?re done with them, your program PROBABLY SHOULD BE BROKEN. But even if you do not accept that, for all practical purposes, the program IS broken today because finalization is absolutely NOT run in a timely enough fashion. I have shown that we can have a cake and eat it too. Combining Finalizator-based clean-up with manual (or AutoCloseable) clean-up is a win-win situation. Programs that forget to call close() still work and those that do prompt close()ing will not be affected by finalization overhead. The migration of internal JDK code from finalize() methods to Finalizator-based cleanup should be simple and straight-forward. So what are we waiting for? ;-) > BTW - I never understood why CMS and other GC?s had absolutely no problem running finalization in a very timely fashion while the G1 collector just never seemed to get around to it. My interpretation of that fact has always led me to believe that it?s not a throughput issue with the finalization thread (not in real world examples, anyway) but rather a GC implementation that didn?t feel the need to be thorough enough to make sure something is ready to be finalized. I mean, when the G1 collector was forced to run a full collection (a death sentence on a 15Gb heap but it did occur) all the finalizable objects were found AND finalized immediately, all 15 or 20 million of them. > > So in summary: > > (1) The problem with finalization is that people use it. And more importantly, that the JDK is filled with inappropriate uses of it. > > (2) The main solution is probably just to delete the inappropriate uses in the JDK. But if that?s not OK, then some sort of patch like what I did which allows the JDK classes to unregister the Finalizer?s when they are no longer needed, i.e., when the object knows that it has cleaned itself up. > > I am curious to hear your thoughts. Thanks for the description of the problem you have with finalization. JDK has an internal alternative to finalization called sun.misc.Cleaner, which has basically the same API and implementation as my presented Finalizator with the following differences: - Cleaner is a PhantomReference which means that the referent is not obtainable any more when it is triggered, so clean-up code can only work on state that is not part of the referent (captured by Cleaner's thunk at the time of construction). This is suitable sometimes but not always. - Cleaner(s) are executed by the ReferenceHandler thread direclty which makes them unsuitable for public consumption as their thunk's must guarantee to be executed quickly or else the whole reference processing infrastructure blocks. Finalizator(s) are executed by the same thread(s) as Finalizer(s). While it would be possible to retro-fit internal JDK classes to use Cleaner(s) instead of finalize() methods, this would require more refactoring which is always tricky. Finalizators, on the other hand, could be used as a drop-in replacement for finalize() method. > JP > Regards, Peter From peter.levart at gmail.com Sun Jun 7 08:46:33 2015 From: peter.levart at gmail.com (Peter Levart) Date: Sun, 07 Jun 2015 10:46:33 +0200 Subject: Another take on Finalization In-Reply-To: <0C527AE7-DED0-4731-9522-CECA3F618B9F@jpayne.net> References: <0C527AE7-DED0-4731-9522-CECA3F618B9F@jpayne.net> Message-ID: <557404E9.7000907@gmail.com> Hi, On 06/05/2015 11:11 PM, Jonathan Payne wrote: > My problem was that finalization was not being run at all with the G1 collector. Not at all. That would have been fine with me because none of the existing objects in the Finalizer queue actually needed the service anymore: the files, sockets, streams, etc. had all been properly closed by my application, otherwise the server would have long since failed completely. However, those objects started to accumulate in the VM and eventually (8 hours later) brought the server down. I made a quick test comparing default (PS MarkSweep / PS Scavenge) and G1 collectors with -Xmx512m option. When finalizable objects are released immediately after construction with a sustained rate of ~90 objects/ms, the results for default collector are: Finalization, PATCHED, PS MarkSweep, PS Scavenge Heap: init = 260046848(253952K) used = 9175232(8960K) committed = 249561088(243712K) max = 477626368(466432K) construction threads: 1 instances per thread: 10000000 cons. per 1 ms sleep: 100 ~5s interval cumulative active pool queued t[ms] construct. destruct. in-flight con./ms des./ms con./ms des./ms thr. size tasks ------- ---------- ---------- ---------- ------- ------- ------- ------- ----- ----- ---------- 5000 439500 0 439500 87 0 87 0 0 0 0 10002 877703 0 877703 87 0 87 0 0 0 0 15004 1308200 238692 1069508 86 47 87 15 0 0 0 20006 1747600 238692 1508908 87 0 87 11 0 0 0 25008 2177600 478173 1699427 85 47 87 19 0 1 0 30010 2620800 478173 2142627 88 0 87 15 0 0 0 35012 3067800 478173 2589627 89 0 87 13 0 0 0 40014 3463600 3302385 161215 79 564 86 82 0 1 0 45016 3903200 3302385 600815 87 0 86 73 0 0 0 50018 4342800 3302385 1040415 87 0 86 66 0 0 0 55020 4772809 3564145 1208664 85 52 86 64 0 0 0 60021 5212000 3564145 1647855 87 0 86 59 0 0 0 65023 5611300 4096555 1514745 79 106 86 63 1 1 47 70024 6051400 5609375 442025 88 302 86 80 0 0 0 75025 6477000 6424132 52868 85 162 86 85 0 1 0 80026 6916000 6424132 491868 87 0 86 80 0 0 0 85027 7355600 6424132 931468 87 0 86 75 0 0 0 90028 7780400 7490435 289965 84 213 86 83 0 0 0 95029 8220800 7490435 730365 88 0 86 78 0 0 0 100029 8644500 8550627 93873 84 211 86 85 0 1 0 105031 9084400 8550627 533773 87 0 86 81 0 0 0 110032 9524100 8550627 973473 87 0 86 77 0 0 0 115033 9948900 9618391 330509 84 213 86 83 0 0 0 120033 10000000 9618391 381609 10 0 83 80 0 0 0 125205 10000000 10000000 0 0 73 79 79 0 0 0 And for G1: Finalization, PATCHED, G1 Young Generation, G1 Old Generation Heap: init = 260046848(253952K) used = 3145728(3072K) committed = 260046848(253952K) max = 536870912(524288K) construction threads: 1 instances per thread: 10000000 cons. per 1 ms sleep: 100 ~5s interval cumulative active pool queued t[ms] construct. destruct. in-flight con./ms des./ms con./ms des./ms thr. size tasks ------- ---------- ---------- ---------- ------- ------- ------- ------- ----- ----- ---------- 5000 437600 337879 99721 87 67 87 67 0 1 0 10002 876200 592612 283588 87 50 87 59 0 0 0 15004 1309600 1032801 276799 86 87 87 68 0 0 0 20006 1742200 1545349 196851 86 102 87 77 0 0 0 25008 2172300 2152632 19668 85 121 86 86 0 1 0 30010 2617800 2152632 465168 89 0 87 71 0 0 0 35012 3047400 2783284 264116 85 126 87 79 0 0 0 40139 3487900 2783284 704616 85 0 86 69 0 0 0 45140 3929800 3597818 331982 88 162 87 79 0 0 0 50142 4374400 3597818 776582 88 0 87 71 0 0 0 55144 4797700 4641649 156051 84 208 87 84 0 1 0 60145 5242800 4641649 601151 88 0 87 77 0 0 0 65147 5670900 5458835 212065 85 163 87 83 0 0 0 70148 6101400 6071013 30387 86 122 86 86 0 1 0 75150 6546100 6071013 475087 88 0 87 80 0 0 0 80151 6990100 6071013 919087 88 0 87 75 0 0 0 85153 7432600 6071013 1361587 88 0 87 71 0 0 0 90154 7850400 7506935 343465 83 287 87 83 0 0 0 95154 8280100 8175574 104526 85 133 87 85 0 1 0 100155 8713200 8675215 37985 86 99 86 86 0 1 0 105156 9158500 8675215 483285 89 0 87 82 0 0 0 110157 9603800 8675215 928585 89 0 87 78 0 0 0 115158 10000000 8675215 1324785 79 0 86 75 0 0 0 120761 10000000 10000000 0 0 236 82 82 0 0 0 ...which suggests that G1 is no less eager to process finalizable objects that are young garbage (see "in-flight" column). At least it doesn't seem that they accumulate. So your problem might have been with finalizable objects escaping to old-gen. Tuning the young-gen size and survivor ratio might help in your case. Regards, Peter From paul.sandoz at oracle.com Sun Jun 7 12:57:15 2015 From: paul.sandoz at oracle.com (Paul Sandoz) Date: Sun, 7 Jun 2015 14:57:15 +0200 Subject: RFR 8071597 Add Stream dropWhile and takeWhile operations In-Reply-To: <55722EAC.6020503@oracle.com> References: <79392DA7-6A8E-42AC-9E83-AA7F89D8F2A5@oracle.com> <556E60F3.7060800@oracle.com> <55722EAC.6020503@oracle.com> Message-ID: <7D02B5FD-D235-4844-8E87-5009F945D447@oracle.com> On Jun 6, 2015, at 1:20 AM, Stuart Marks wrote: > On 6/3/15 9:21 AM, Paul Sandoz wrote: >> I had prepared an alternative rendition stashed away just in case this came up :-) >> >> I still want to retain a punchy short first paragraph. What do you think about the following? >> >> /** >> - * Returns a stream consisting of the longest prefix of elements taken from >> - * this stream that match the given predicate. >> + * Returns a stream consisting of elements taken from this stream that match >> + * the given predicate. > > Well, I'm sorry to say, this reminds me of that old tech support joke about the answer that is technically true but completely useless. :-) > LOL. It does have a vacuity about it. I would argue it's mostly useless :-) Here is another variant: /** * Returns, if this stream is ordered, a stream consisting of the longest * prefix of elements taken from this stream that match the given predicate. * Otherwise returns, if this stream is unordered, a stream consisting of a * subset of elements taken from this stream that match the given predicate. * *

    If this stream is ordered then the longest prefix is a contiguous * sequence of elements of this stream that match the given predicate. The * first element of the sequence is the first element (if any) of this * stream, and the element (if any) immediately following the last element * of the sequence does not match the given predicate. * *

    If this stream is unordered then the behavior of this operation is * nondeterministic; it is free to take any valid subset. * ... */ default Stream takeWhile(Predicate predicate) { /** * Returns, if this stream is ordered, a stream consisting of the remaining * elements of this stream after dropping the longest prefix of elements * that match the given predicate. Otherwise returns, if this stream is * unordered, a stream consisting of the remaining elements of this stream * after dropping a subset of elements that match the given predicate. * *

    If this stream is ordered then the longest prefix is a contiguous * sequence of elements of this stream that match the given predicate. The * first element of the sequence is the first element (if any) of this * stream, and the element (if any) immediately following the last element * of the sequence does not match the given predicate. * *

    If this stream is unordered then the behavior of this operation is * nondeterministic; it is free to drop any valid subset. * ... */ default Stream dropWhile(Predicate predicate) { Unless there are strong objects i will stick with that layout. Paul. > It's not incorrect, of course, but it's almost indistinguishable from the first sentence of Stream.filter(). > > People who know what "takeWhile" means will raise an eyebrow at this, go read the rest of the doc, and go "ohhh, yeah there's this complication with unordered streams." People who are trying to learn the system will go "Huh?" > > I agree on the need for a punchy first sentence since that's what goes into the Method Summary. The difficulty is making something sufficiently general that it covers all the cases, while not making it so vague as to be useless. > > Sorry, but I don't have any better suggestions at the moment. I might be able to come up with something in the next few days... but don't let this hold you up if you need to press forward. > > s'marks From chris.hegarty at oracle.com Sun Jun 7 14:36:36 2015 From: chris.hegarty at oracle.com (Chris Hegarty) Date: Sun, 7 Jun 2015 15:36:36 +0100 Subject: RFR 8071597 Add Stream dropWhile and takeWhile operations In-Reply-To: <7D02B5FD-D235-4844-8E87-5009F945D447@oracle.com> References: <79392DA7-6A8E-42AC-9E83-AA7F89D8F2A5@oracle.com> <556E60F3.7060800@oracle.com> <55722EAC.6020503@oracle.com> <7D02B5FD-D235-4844-8E87-5009F945D447@oracle.com> Message-ID: <21325DE3-A718-423C-834C-7136AE0F7ED2@oracle.com> I think the original version reads better. Remember that the method summary description contains only the first sentence. I disagree with the objection about ordering. The stream has some order in which is "sees" elements. That order may be non-deterministic, but it still exists, we just don?t know what it is. takeWhile will still operate on a stream with non-deterministic order, and return the longest prefix, you just don?t know what that is. An appropriate note in the API warning about this should be sufficient. -Chris. > On 7 Jun 2015, at 13:57, Paul Sandoz wrote: > > > On Jun 6, 2015, at 1:20 AM, Stuart Marks wrote: > >> On 6/3/15 9:21 AM, Paul Sandoz wrote: >>> I had prepared an alternative rendition stashed away just in case this came up :-) >>> >>> I still want to retain a punchy short first paragraph. What do you think about the following? >>> >>> /** >>> - * Returns a stream consisting of the longest prefix of elements taken from >>> - * this stream that match the given predicate. >>> + * Returns a stream consisting of elements taken from this stream that match >>> + * the given predicate. >> >> Well, I'm sorry to say, this reminds me of that old tech support joke about the answer that is technically true but completely useless. :-) >> > > LOL. It does have a vacuity about it. I would argue it's mostly useless :-) > > Here is another variant: > > /** > * Returns, if this stream is ordered, a stream consisting of the longest > * prefix of elements taken from this stream that match the given predicate. > * Otherwise returns, if this stream is unordered, a stream consisting of a > * subset of elements taken from this stream that match the given predicate. > * > *

    If this stream is ordered then the longest prefix is a contiguous > * sequence of elements of this stream that match the given predicate. The > * first element of the sequence is the first element (if any) of this > * stream, and the element (if any) immediately following the last element > * of the sequence does not match the given predicate. > * > *

    If this stream is unordered then the behavior of this operation is > * nondeterministic; it is free to take any valid subset. > * > ... > */ > default Stream takeWhile(Predicate predicate) { > > > /** > * Returns, if this stream is ordered, a stream consisting of the remaining > * elements of this stream after dropping the longest prefix of elements > * that match the given predicate. Otherwise returns, if this stream is > * unordered, a stream consisting of the remaining elements of this stream > * after dropping a subset of elements that match the given predicate. > * > *

    If this stream is ordered then the longest prefix is a contiguous > * sequence of elements of this stream that match the given predicate. The > * first element of the sequence is the first element (if any) of this > * stream, and the element (if any) immediately following the last element > * of the sequence does not match the given predicate. > * > *

    If this stream is unordered then the behavior of this operation is > * nondeterministic; it is free to drop any valid subset. > * > ... > */ > default Stream dropWhile(Predicate predicate) { > > Unless there are strong objects i will stick with that layout. > > Paul. > >> It's not incorrect, of course, but it's almost indistinguishable from the first sentence of Stream.filter(). >> >> People who know what "takeWhile" means will raise an eyebrow at this, go read the rest of the doc, and go "ohhh, yeah there's this complication with unordered streams." People who are trying to learn the system will go "Huh?" >> >> I agree on the need for a punchy first sentence since that's what goes into the Method Summary. The difficulty is making something sufficiently general that it covers all the cases, while not making it so vague as to be useless. >> >> Sorry, but I don't have any better suggestions at the moment. I might be able to come up with something in the next few days... but don't let this hold you up if you need to press forward. >> >> s'marks > From spliterator at gmail.com Sun Jun 7 21:07:26 2015 From: spliterator at gmail.com (Stefan Zobel) Date: Sun, 7 Jun 2015 23:07:26 +0200 Subject: RFR 8071597 Add Stream dropWhile and takeWhile operations In-Reply-To: <79392DA7-6A8E-42AC-9E83-AA7F89D8F2A5@oracle.com> References: <79392DA7-6A8E-42AC-9E83-AA7F89D8F2A5@oracle.com> Message-ID: I'm still trying to wrap my head around the test logic for the (par & !ord) && (op == WhileOp.Drop) case in the whileResultAsserter() method in WhileOpTest. Wouldn't it be possible that, for an unordered parallel stream, dropWhile() won't drop anything at all (i.e., drops an empty set)? In that case, input.size() == output.size() and the set of matching output elements is no longer a _proper_ subset of the set of matching input elements. The whileResultAsserter() would fail even though dropWhile() works correctly (in a nondeterministic sense)? It's a bit late now and I guess it's all in my imagination. Still curious ... Thanks, Stefan 2015-06-02 15:13 GMT+02:00 Paul Sandoz : > Hi, > > Please review this webrev that adds take/dropWhile operations to streams: > > > http://cr.openjdk.java.net/~psandoz/jdk9/JDK-8071597-take-drop-while/webrev/ > > I opted to weight the documentation of the operations towards ordered > streams in the first paragraph. That is what makes most sense in terms of > usage and what most people will read. Thus i refer to the "longest prefix" > in the first paragraph then define what that means in subsequent paragraphs > for ordered and unordered streams: > > 482 /** > 483 * Returns a stream consisting of the longest prefix of elements > taken from > 484 * this stream that match the given predicate. > 485 * > 486 *

    If this stream is ordered then the prefix is a contiguous > sequence of > 487 * elements of this stream. All elements of the sequence match > the given > 488 * predicate, the first element of the sequence is the first > element > 489 * (if any) of this stream, and the element (if any) immediately > following > 490 * the last element of the sequence does not match the given > predicate. > 491 * > 492 *

    If this stream is unordered then the prefix is a subset of > elements of > 493 * this stream. All elements (if any) of the subset match the > given > 494 * predicate. In this case the behavior of this operation is > 495 * nondeterministic; it is free to select any valid subset as the > prefix. > 496 * > 497 *

    This is a href="package-summary.html#StreamOps">short-circuiting > 498 * stateful intermediate operation. > 499 * > ... > 528 default Stream takeWhile(Predicate predicate) { > > 537 /** > 538 * Returns a stream consisting of the remaining elements of this > stream > 539 * after dropping the longest prefix of elements that match the > given > 540 * predicate. > 541 * > 542 *

    If this stream is ordered then the prefix is a contiguous > sequence of > 543 * elements of this stream. All elements of the sequence match > the given > 544 * predicate, the first element of the sequence is the first > element > 545 * (if any) of this stream, and the element (if any) immediately > following > 546 * the last element of the sequence does not match the given > predicate. > 547 * > 548 *

    If this stream is unordered then the prefix is a subset of > elements of > 549 * this stream. All elements (if any) of the subset match the > given > 550 * predicate. In this case the behavior of this operation is > 551 * nondeterministic; it is free to select any valid subset as the > prefix. > 552 * > ... > 584 default Stream dropWhile(Predicate predicate) { > > > After this has been reviewed i will follow up with a further issue > regarding the specification of takeWhile, stateful predicates and > cancellation. I avoided such specification here as it's likely to rathole > :-) > > Basically the takeWhile operation is implemented such that one can do: > > long t = System.currentTimeMillis(); > List pps = Stream > .generate(() -> BigInteger.probablePrime(1024, > ThreadLocalRandom.current())) > .parallel() > .takeWhile(e -> (System.currentTimeMillis() - t) < > TimeUnit.SECONDS.toMillis(5)) > .collect(toList()); > > Paul. > From thomas.schatzl at oracle.com Mon Jun 8 07:41:05 2015 From: thomas.schatzl at oracle.com (Thomas Schatzl) Date: Mon, 08 Jun 2015 09:41:05 +0200 Subject: Another take on Finalization In-Reply-To: <557404E9.7000907@gmail.com> References: <0C527AE7-DED0-4731-9522-CECA3F618B9F@jpayne.net> <557404E9.7000907@gmail.com> Message-ID: <1433749265.2616.28.camel@oracle.com> Hi, On Sun, 2015-06-07 at 10:46 +0200, Peter Levart wrote: > Hi, > > On 06/05/2015 11:11 PM, Jonathan Payne wrote: > > My problem was that finalization was not being run at all with the G1 >>collector. Not at all. That would have been fine with me because none >>of the existing objects in the Finalizer queue actually needed the >>service anymore: the files, sockets, streams, etc. had all been >>properly closed by my application, otherwise the server would have >>long since failed completely. However, those objects started to >>accumulate in the VM and eventually (8 hours later) brought the server >>down. > > I made a quick test comparing default (PS MarkSweep / PS Scavenge) and > G1 collectors with -Xmx512m option. When finalizable objects are > released immediately after construction with a sustained rate of ~90 > objects/ms, the results for default collector are: [...] > ...which suggests that G1 is no less eager to process finalizable > objects that are young garbage (see "in-flight" column). At least it > doesn't seem that they accumulate. So your problem might have been with > finalizable objects escaping to old-gen. Tuning the young-gen size and > survivor ratio might help in your case. that might have been an artifact of old G1 implementations that did not do class unloading (and various other cleanups) during the concurrent marking cycle. That feature is in the official release since 8u40, so G1 should definitely also try to finalize objects that escaped into old gen for some time now, without full gc. Thanks, Thomas From erik.joelsson at oracle.com Mon Jun 8 09:05:02 2015 From: erik.joelsson at oracle.com (Erik Joelsson) Date: Mon, 08 Jun 2015 11:05:02 +0200 Subject: RFR: JDK-8085822 JEP 223: New Version-String Scheme (initial integration) In-Reply-To: <5571AD18.7080701@oracle.com> References: <5571AD18.7080701@oracle.com> Message-ID: <55755ABE.9010000@oracle.com> Hello, Looks pretty good. Found some typos: jdk_util.c: 99: specia_update_version jdk-version.m4: 31: assing 124, 132: --with--version-pre-base has a dash too many? I see this pattern consistently used though, am I missing something? /Erik On 2015-06-05 16:07, Magnus Ihse Bursie wrote: > This review request covers the main part of the work for JEP-223, the > new version string format [1]. Basically, we'll call this release Java > "9", instead of Java "1.9.0". > > This patch is a folding of all work that has been done so far in the > branch JEP-223-branch in jdk9/sandbox. As you can see, it mostly > covers build changes, with some code changes in hotspot, jdk, nashorn > and langtools that either are corresponding changes in the product > code due to the compiler define flags changing from the build, or > follow-up changes to handle the new format. > > The JEP-223 work is not finished by this patch. In fact, there are > known issues remaining even after this patch, typically by code that > reads the "java.version" property and tries to parse it. However, this > patch is not directly destined for jdk9/dev, but will go into the > special verona/stage forest. As for all patches destined for > verona/stage it will be code reviewed as if going to jdk9/dev. Once in > verona/stage it will bide its time, and it will be complemented with > follow-up patches to address remaining issues. When all such issues > are resolved and JEP-223 is fully implemented, all changes will be > pushed at once (without further code reviews) into jdk9/dev. > > This patch has been contributed by Magnus Ihse Bursie, Kumar > Srinivasan and Alejandro Murillo. > > Bug: https://bugs.openjdk.java.net/browse/JDK-8085822 > WebRev: > http://cr.openjdk.java.net/~ihse/JDK-8085822-JEP-223-initial-patch/webrev.01 > > [1] http://openjdk.java.net/jeps/223 > From Alan.Bateman at oracle.com Mon Jun 8 09:34:12 2015 From: Alan.Bateman at oracle.com (Alan Bateman) Date: Mon, 08 Jun 2015 10:34:12 +0100 Subject: RFR: JDK-8085822 JEP 223: New Version-String Scheme (initial integration) In-Reply-To: <5571AD18.7080701@oracle.com> References: <5571AD18.7080701@oracle.com> Message-ID: <55756194.3080506@oracle.com> On 05/06/2015 15:07, Magnus Ihse Bursie wrote: > This review request covers the main part of the work for JEP-223, the > new version string format [1]. Basically, we'll call this release Java > "9", instead of Java "1.9.0". > > This patch is a folding of all work that has been done so far in the > branch JEP-223-branch in jdk9/sandbox. As you can see, it mostly > covers build changes, with some code changes in hotspot, jdk, nashorn > and langtools that either are corresponding changes in the product > code due to the compiler define flags changing from the build, or > follow-up changes to handle the new format. > > The JEP-223 work is not finished by this patch. In fact, there are > known issues remaining even after this patch, typically by code that > reads the "java.version" property and tries to parse it. However, this > patch is not directly destined for jdk9/dev, but will go into the > special verona/stage forest. As for all patches destined for > verona/stage it will be code reviewed as if going to jdk9/dev. Once in > verona/stage it will bide its time, and it will be complemented with > follow-up patches to address remaining issues. When all such issues > are resolved and JEP-223 is fully implemented, all changes will be > pushed at once (without further code reviews) into jdk9/dev. > > This patch has been contributed by Magnus Ihse Bursie, Kumar > Srinivasan and Alejandro Murillo. > > Bug: https://bugs.openjdk.java.net/browse/JDK-8085822 > WebRev: > http://cr.openjdk.java.net/~ihse/JDK-8085822-JEP-223-initial-patch/webrev.01 I looked through the code changes, skipping most of the make files :-) Version.java.template - the comment in jvmSecurityVersion() still talks about 1.6 and newer. Can this be replaced to just say that it returns the security version? Will the update_version and special_update_version fields eventually be dropped from the jvm_version_info stricture? Related, there seems to be a typo in the comment in jdk_util.c where it has "specia_update_version". The webrev shows a change to this comment in jvm.h: "Third, this file contains various I/O and network operations needed by the standard Java I/O and network APIs." I think this comment can be removed because those JVM_* functions were removed some time ago. Otherwise looks okay to me. -Alan. From dmitry.samersoff at oracle.com Mon Jun 8 11:05:27 2015 From: dmitry.samersoff at oracle.com (Dmitry Samersoff) Date: Mon, 08 Jun 2015 14:05:27 +0300 Subject: RFR(M, v14): JDK-8059036 : Implement Diagnostic Commands for heap and finalizerinfo In-Reply-To: <0323E5E0-CFFA-4E11-8F51-270CB7F6964C@oracle.com> References: <554894E9.8020908@oracle.com> <5556F34B.5050701@oracle.com> <55573A31.7050306@gmail.com> <55573C86.8030807@gmail.com> <5559D85B.2050500@oracle.com> <6FB8DF19-1D67-4702-9642-4C630765D2D1@oracle.com> <555C2EEB.1050505@oracle.com> <3941B430-FE6C-4AF0-A875-76D24411C654@oracle.com> <5564802A.2010403@oracle.com> <5565738A.4040109@gmail.com> <7EB0BE71-1608-4047-B71D-1A9EAA9083BB@oracle.com> <556727A2.9060209@gmail.com> <55675908.6030406@oracle.com> <556AF3CD.50900@oracle.com> <37227441-B016-4A9A-B9CA-BE100E992C9A@oracle.com> <556D8FB0.4070608@oracle.com> <556E9970.8010704@gmail.com> <556EBF4B.6070400@oracle.com> <556F1D6C.6010806@oracle.com> <556F491E.3070709@oracle.com> <557054C7.4030705@oracle.com> <3607C2B6-F247-4291-9198-279CF8ED2AF3@oracle.com> <5570738B.5040603@oracle.com> <00038522-9EC7-4491-9AE4-73BD16D95909@oracle.com> <5571814B.9060706@oracle.com> <0323E5E0-CFFA-4E11-8F51-270CB7F6964C@oracle.com> Message-ID: <557576F7.4040000@oracle.com> Staffan, Could you review a CCC request. http://ccc.us.oracle.com/8059036 -Dmitry On 2015-06-05 15:24, Staffan Larsen wrote: > Thanks - this version looks good to me. > > /Staffan > >> On 5 jun 2015, at 13:00, Dmitry Samersoff wrote: >> >> Staffan, >> >> Thank you for review! >> >> Done. Webrev updated in-place (press shift-reload). >> >> http://cr.openjdk.java.net/~dsamersoff/JDK-8059036/webrev.14 >> >> -Dmitry >> >> On 2015-06-05 11:20, Staffan Larsen wrote: >>> Dmitry, >>> >>> I?d like to propose the following change to get prettier output (more in line with GC.class_histogram): >>> >>> diff --git a/src/share/vm/services/diagnosticCommand.cpp b/src/share/vm/services/diagnosticCommand.cpp >>> --- a/src/share/vm/services/diagnosticCommand.cpp >>> +++ b/src/share/vm/services/diagnosticCommand.cpp >>> @@ -341,7 +341,6 @@ >>> void FinalizerInfoDCmd::execute(DCmdSource source, TRAPS) { >>> ResourceMark rm; >>> >>> - output()->print_cr("Unreachable instances awaiting finalization:"); >>> Klass* k = SystemDictionary::resolve_or_null( >>> vmSymbols::finalizer_histogram_klass(), THREAD); >>> assert(k != NULL, "FinalizerHistogram class is not accessible"); >>> @@ -375,12 +374,15 @@ >>> >>> assert(count_res != NULL && name_res != NULL, "Unexpected layout of FinalizerHistogramEntry"); >>> >>> + output()->print_cr("Unreachable instances waiting for finalization"); >>> + output()->print_cr("#instances class name"); >>> + output()->print_cr("-----------------------"); >>> for (int i = 0; i < result_oop->length(); ++i) { >>> oop element_oop = result_oop->obj_at(i); >>> oop str_oop = element_oop->obj_field(name_fd.offset()); >>> char *name = java_lang_String::as_utf8_string(str_oop); >>> int count = element_oop->int_field(count_fd.offset()); >>> - output()->print_cr("Class %s - %d", name, count); >>> + output()->print_cr("%10d %s", count, name); >>> } >>> } >>> >>> >>> A couple of nits: >>> >>> diagnosticCommand.cpp:359 - extra space after = >>> diagnosticCommand.cpp:361 - spelling: finlalization -> finalization >>> FinalizerInfoTest.java:69: - spelling: intialized -> initialized >>> FinalizerInfoTest.java:71 - I?d like to see the ; on a new line >>> >>> >>> Thanks, >>> /Staffan >>> >>> >>>> On 5 jun 2015, at 00:20, Mandy Chung wrote: >>>> >>>> >>>>> On Jun 4, 2015, at 8:49 AM, Dmitry Samersoff wrote: >>>>> >>>>> Mandy, >>>>> >>>>> Webrev updated in-place (press shift-reload). >>>>> >>>>> http://cr.openjdk.java.net/~dsamersoff/JDK-8059036/webrev.14 >>>>> >>>>> getFinalizerHistogram() now returns Entry[] >>>>> >>>>> @library and @build removed from the test. >>>> >>>> >>>> Looks fine. >>>> >>>> Thanks >>>> Mandy >>>> >>>>> >>>>> -Dmitry >>>>> >>>>> On 2015-06-04 16:56, Mandy Chung wrote: >>>>>> >>>>>>> On Jun 4, 2015, at 6:38 AM, Dmitry Samersoff wrote: >>>>>>> >>>>>>> Mandy, >>>>>>> >>>>>>> Thank you for the review. >>>>>>> >>>>>>> Updated webrev is: >>>>>>> >>>>>>> http://cr.openjdk.java.net/~dsamersoff/JDK-8059036/webrev.14 >>>>>>> >>>>>> >>>>>> Thanks for the update and the test. >>>>>> >>>>>>> addressed comments, added a unit test to JDK workspace. >>>>>>> >>>>>> >>>>>> This test does not need @library and @build, right? >>>>>> >>>>>>> >>>>>>> On 2015-06-03 21:36, Mandy Chung wrote: >>>>>>> >>>>>>>> Should getFinalizerHistogram() return FinalizerHistogramEntry[]? >>>>>>>> The VM knows the returned type anyway. >>>>>>> >>>>>>> "[java/lang/ref/Entry" signature would need a separate symbol entry in >>>>>>> swollen vmSymbols::init() so I would prefer to stay with more generic >>>>>>> Object[] >>>>>>> >>>>>> >>>>>> You already add several symbols - why is it an issue for another one? Or if adding vm symbols is a concern, you should revert to String and int[] that you decide not to. The decision should apply consistently (use String and int, or new Java type). >>>>>> >>>>>> >>>>>>>> Perhaps the getFinalizerHistogram method should be renamed >>>>>>>> e.g. "dump"? >>>>>>> >>>>>>> The line in vmSymbols looks like: >>>>>>> >>>>>>> template( >>>>>>> get_finalizer_histogram_name, "getFinalizerHistogram") >>>>>>> >>>>>>> I would prefer to keep method name specific enough to be able to >>>>>>> find it by grep in jdk code. >>>>>> >>>>>> >>>>>> Grep in jdk code is easy as you have a new class :) >>>>>> >>>>>> Mandy >>>>>> >>>>>>> >>>>>>> (other comments are addressed) >>>>>>> >>>>>>> -Dmitry >>>>>>> >>>>>>> >>>>>>> On 2015-06-03 21:36, Mandy Chung wrote: >>>>>>>> >>>>>>>> >>>>>>>> On 06/03/2015 08:29 AM, Dmitry Samersoff wrote: >>>>>>>>> Updated webrev: >>>>>>>>> >>>>>>>>> http://cr.openjdk.java.net/~dsamersoff/JDK-8059036/webrev.13 >>>>>>>> >>>>>>>> I reviewed the jdk change. The version looks okay and some comments: >>>>>>>> >>>>>>>> ReferenceQueue.java - you should eliminate the use of rawtype: >>>>>>>> >>>>>>>> 84 Reference rn = r.next; >>>>>>>> >>>>>>>> Change Reference to Reference >>>>>>> >>>>>>> done. >>>>>>> >>>>>>>> >>>>>>>> The forEach method - eliminate the use of raw type and >>>>>>>> move @SuppressWarning to the field variable in line 182: >>>>>>>> >>>>>>>> @SuppressWarnings("unchecked") >>>>>>>> Reference rn = r.next; >>>>>>> >>>>>>> done. >>>>>>> >>>>>>>> >>>>>>>> FinalizerHistogram.java >>>>>>>> Copyright year needs update. >>>>>>> >>>>>>> done. >>>>>>>> >>>>>>>> I personally think the VM code would be much simpler if you stay with >>>>>>>> alternative entry of String and int than dealing with a >>>>>>>> FinalizerHistogramEntry private class. It's okay as FinalizerHistogram >>>>>>>> class is separated. >>>>>>>> >>>>>>>> The comment in line 35 is suitable to move to the class description and >>>>>>>> this is the suggestion. >>>>>>>> >>>>>>>> // This FinalizerHistogram class is for GC.finalizer_info diagnostic >>>>>>>> command support. >>>>>>>> // It is invoked by the VM. >>>>>>> >>>>>>> done. >>>>>>> >>>>>>>> >>>>>>>> Should getFinalizerHistogram() return FinalizerHistogramEntry[]? The VM >>>>>>>> knows the returned type anyway. >>>>>>> >>>>>>> "[java/lang/ref/Entry" signature would need a separate symbol entry in >>>>>>> swollen vmSymbols::init() so I would prefer to stay with more generic >>>>>>> Object[] >>>>>>> >>>>>>>> It's an inner class of >>>>>>>> FinalizerHistogram and it can simply be named as "Entry". I suggest to >>>>>>>> have Entry::increment method to replace ".instanceCount++". >>>>>>> >>>>>>> done. >>>>>>> >>>>>>>> >>>>>>>> The tests are in hotspot/test. Can you add a unit test in jdk/test? >>>>>>>> Perhaps you can test FinalizerHistogram.getFinalizerHistogram() via >>>>>>>> reflection - just a sanity test. >>>>>>> >>>>>>> done. The test repeats hotspot one, but uses reflection instead of VM call. >>>>>>> >>>>>>>> >>>>>>>> A minor naming comment - now you have a FinalizerHistogram class. >>>>>>>> Perhaps the getFinalizerHistogram method should be renamed e.g. "dump"? >>>>>>> >>>>>>> The line in vmSymbols looks like: >>>>>>> >>>>>>> template( >>>>>>> get_finalizer_histogram_name, "getFinalizerHistogram") >>>>>>> >>>>>>> I would prefer to keep it specific enough to be able to >>>>>>> find it by grep in jdk code. >>>>>>> >>>>>>> >>>>>>> -- >>>>>>> Dmitry Samersoff >>>>>>> Oracle Java development team, Saint Petersburg, Russia >>>>>>> * I would love to change the world, but they won't give me the sources. >>>>>> >>>>> >>>>> >>>>> -- >>>>> Dmitry Samersoff >>>>> Oracle Java development team, Saint Petersburg, Russia >>>>> * I would love to change the world, but they won't give me the sources. >>>> >>> >> >> >> -- >> Dmitry Samersoff >> Oracle Java development team, Saint Petersburg, Russia >> * I would love to change the world, but they won't give me the sources. > -- Dmitry Samersoff Oracle Java development team, Saint Petersburg, Russia * I would love to change the world, but they won't give me the sources. From alexander.v.stepanov at oracle.com Mon Jun 8 11:16:03 2015 From: alexander.v.stepanov at oracle.com (alexander stepanov) Date: Mon, 08 Jun 2015 14:16:03 +0300 Subject: RFR [9] 8081517: minor cleanup for docs In-Reply-To: <5571EB2E.8000200@Oracle.com> References: <55719B6A.4000904@oracle.com> <5571AD67.4040203@Oracle.com> <5571DD76.6040000@oracle.com> <5571EB2E.8000200@Oracle.com> Message-ID: <55757973.8020708@oracle.com> Thanks! On 05.06.2015 21:32, Roger Riggs wrote: > Hi Alexander, > > Looks good, thanks for the updates. > > Roger > > > On 6/5/2015 1:33 PM, alexander stepanov wrote: >> Hello Lance, Roger, >> >> Thank you for the notes, please see the updated webrev: >> http://cr.openjdk.java.net/~avstepan/8081517/webrev.01/index.html >> (files changed: FtpURLConnection.java, VmIdentifier.java, >> TraceClassVisitor.java, HttpAuthenticator.java, PlatformLogger.java >> and JdbcRowSetImpl.java - some indents were fixed) >> >> > SQL AS clause >> that's probably used for the SQL "AS" keyword. In such a case the >> upper case may be suitable. >> >> Regards, >> Alexander >> >> >> On 05.06.2015 17:12, Lance Andersen wrote: >>> Hi Roger >>> On Jun 5, 2015, at 10:08 AM, Roger Riggs >>> wrote: >>> >>>> 2) >>>> If indentation is in the scope of your cleanup? >>> Agree with you and I did not comment on this as I felt it this >>> should be done separately as there is a fair amount that can be done >>> in these classes for clean up. >>> >>> Best >>> lance >>>> >>>> In JdbcRowSetImpl.java, the indentation of continued lines after >>>> @return or @param tags >>>> isn't correct; the continued line should be indented. >>>> >>>> And there are a some very long lines that should be wrapped; >>>> See +5637... "If the SQL AS clause was not..." >>>> >>>> +5784 the indentation needs to be fixed. >>>> +5840 also >>>> +5988 >>>> +6265 and after... >>>> in many cases the lines that follow of "/*" do not align the "* ". >>>> >>>> CachedRowSetWriter.java: >>>> - many method comment blocks are not aligned properly with the >>>> method of field that follows. >>>> For example, >>>> /** >>>> * The Connection object that this writer will use to >>>> make a >>>> * connection to the data source to which it will write data. >>>> * >>>> */ >>>> private transient Connection con; >>>> >>>> 3) >>>> >>>> CachedRowSetWriter.java: >>>> +1056: The tag "@ param" should be tighted up to "@param" >>>> >>>> Thanks, Roger >>>> >>>> >>>> >>>> On 6/5/2015 8:51 AM, alexander stepanov wrote: >>>>> Hello, >>>>> >>>>> Could you please review the fix >>>>> http://cr.openjdk.java.net/~avstepan/8081517/webrev.00/ >>>>> for >>>>> https://bugs.openjdk.java.net/browse/JDK-8081517 >>>>> >>>>> Just some cleanup for docs. >>>>> >>>>> Thanks, >>>>> Alexander >>> >>> >>> Lance Andersen| Principal Member of Technical Staff | +1.781.442.2037 >>> Oracle Java Engineering >>> 1 Network Drive >>> Burlington, MA 01803 >>> Lance.Andersen at oracle.com >>> >>> >>> >> > From mark.sheppard at oracle.com Mon Jun 8 12:30:57 2015 From: mark.sheppard at oracle.com (Mark Sheppard) Date: Mon, 08 Jun 2015 13:30:57 +0100 Subject: RFR: 7130985: Four helper classes missing in Sun JDK In-Reply-To: <5571B52D.6030408@oracle.com> References: <556F1B2D.8010807@oracle.com> <5571B52D.6030408@oracle.com> Message-ID: <55758B01.8080005@oracle.com> Hi Rob, looks fine to me thanks for handling this issue regards Mark On 05/06/2015 15:41, Rob McKenna wrote: > Added some cleanup code around the BufferedReaders & the leftover test > files: > > http://cr.openjdk.java.net/~robm/7130985/webrev.02/ > > -Rob > > On 03/06/15 16:20, Rob McKenna wrote: >> Meant to get this sorted a while back. >> >> There was a thread on this last year ( >> http://mail.openjdk.java.net/pipermail/core-libs-dev/2014-July/027779.html >> >> ) >> >> A test has been added since then: >> >> http://cr.openjdk.java.net/~robm/7130985/webrev.corba/ >> http://cr.openjdk.java.net/~robm/7130985/webrev.j2se/ >> >> -Rob From magnus.ihse.bursie at oracle.com Mon Jun 8 12:37:01 2015 From: magnus.ihse.bursie at oracle.com (Magnus Ihse Bursie) Date: Mon, 8 Jun 2015 14:37:01 +0200 Subject: RFR: JDK-8085822 JEP 223: New Version-String Scheme (initial integration) In-Reply-To: <55756194.3080506@oracle.com> References: <5571AD18.7080701@oracle.com> <55756194.3080506@oracle.com> Message-ID: <1136B10A-72A2-421D-8333-1943353F662C@oracle.com> > 8 jun 2015 kl. 11:34 skrev Alan Bateman : > >> On 05/06/2015 15:07, Magnus Ihse Bursie wrote: >> This review request covers the main part of the work for JEP-223, the new version string format [1]. Basically, we'll call this release Java "9", instead of Java "1.9.0". >> >> This patch is a folding of all work that has been done so far in the branch JEP-223-branch in jdk9/sandbox. As you can see, it mostly covers build changes, with some code changes in hotspot, jdk, nashorn and langtools that either are corresponding changes in the product code due to the compiler define flags changing from the build, or follow-up changes to handle the new format. >> >> The JEP-223 work is not finished by this patch. In fact, there are known issues remaining even after this patch, typically by code that reads the "java.version" property and tries to parse it. However, this patch is not directly destined for jdk9/dev, but will go into the special verona/stage forest. As for all patches destined for verona/stage it will be code reviewed as if going to jdk9/dev. Once in verona/stage it will bide its time, and it will be complemented with follow-up patches to address remaining issues. When all such issues are resolved and JEP-223 is fully implemented, all changes will be pushed at once (without further code reviews) into jdk9/dev. >> >> This patch has been contributed by Magnus Ihse Bursie, Kumar Srinivasan and Alejandro Murillo. >> >> Bug: https://bugs.openjdk.java.net/browse/JDK-8085822 >> WebRev: http://cr.openjdk.java.net/~ihse/JDK-8085822-JEP-223-initial-patch/webrev.01 > > I looked through the code changes, skipping most of the make files :-) > > Version.java.template - the comment in jvmSecurityVersion() still talks about 1.6 and newer. Can this be replaced to just say that it returns the security version? > > Will the update_version and special_update_version fields eventually be dropped from the jvm_version_info stricture? Related, there seems to be a typo in the comment in jdk_util.c where it has "specia_update_version". > > The webrev shows a change to this comment in jvm.h: > "Third, this file contains various I/O and network operations needed by the standard Java I/O and network APIs." > I think this comment can be removed because those JVM_* functions were removed some time ago. > > Otherwise looks okay to me. The API functions in Version.java and jvm.h are not finished. The specification in the JEP talks about a java.util.Version, that I presume will replace the sun.misc.Version, and that will fully implement an API to access the version string and all it's parts, according to the JEP definition. Also, the native interface will have to be changed to accommodate a version number with an arbitrarily number of dot separated parts. These changes will be done later on in the verona/stage forest. Are you ok with addressing these concerns at such a later time? /Magnus > > -Alan. From sean.coffey at oracle.com Mon Jun 8 12:53:03 2015 From: sean.coffey at oracle.com (=?UTF-8?B?U2XDoW4gQ29mZmV5?=) Date: Mon, 08 Jun 2015 13:53:03 +0100 Subject: RFR: 7130985: Four helper classes missing in Sun JDK In-Reply-To: <55758B01.8080005@oracle.com> References: <556F1B2D.8010807@oracle.com> <5571B52D.6030408@oracle.com> <55758B01.8080005@oracle.com> Message-ID: <5575902F.9050200@oracle.com> Looks good to me too. Regards, Sean. On 08/06/15 13:30, Mark Sheppard wrote: > Hi Rob, > looks fine to me > > thanks for handling this issue > > regards > Mark > > On 05/06/2015 15:41, Rob McKenna wrote: >> Added some cleanup code around the BufferedReaders & the leftover >> test files: >> >> http://cr.openjdk.java.net/~robm/7130985/webrev.02/ >> >> -Rob >> >> On 03/06/15 16:20, Rob McKenna wrote: >>> Meant to get this sorted a while back. >>> >>> There was a thread on this last year ( >>> http://mail.openjdk.java.net/pipermail/core-libs-dev/2014-July/027779.html >>> >>> ) >>> >>> A test has been added since then: >>> >>> http://cr.openjdk.java.net/~robm/7130985/webrev.corba/ >>> http://cr.openjdk.java.net/~robm/7130985/webrev.j2se/ >>> >>> -Rob > From sundararajan.athijegannathan at oracle.com Mon Jun 8 12:55:17 2015 From: sundararajan.athijegannathan at oracle.com (A. Sundararajan) Date: Mon, 08 Jun 2015 18:25:17 +0530 Subject: RFR: JDK-8085822 JEP 223: New Version-String Scheme (initial integration) In-Reply-To: <1136B10A-72A2-421D-8333-1943353F662C@oracle.com> References: <5571AD18.7080701@oracle.com> <55756194.3080506@oracle.com> <1136B10A-72A2-421D-8333-1943353F662C@oracle.com> Message-ID: <557590B5.8090906@oracle.com> +1 on Nashorn changes. -Sundar On Monday 08 June 2015 06:07 PM, Magnus Ihse Bursie wrote: >> 8 jun 2015 kl. 11:34 skrev Alan Bateman : >> >>> On 05/06/2015 15:07, Magnus Ihse Bursie wrote: >>> This review request covers the main part of the work for JEP-223, the new version string format [1]. Basically, we'll call this release Java "9", instead of Java "1.9.0". >>> >>> This patch is a folding of all work that has been done so far in the branch JEP-223-branch in jdk9/sandbox. As you can see, it mostly covers build changes, with some code changes in hotspot, jdk, nashorn and langtools that either are corresponding changes in the product code due to the compiler define flags changing from the build, or follow-up changes to handle the new format. >>> >>> The JEP-223 work is not finished by this patch. In fact, there are known issues remaining even after this patch, typically by code that reads the "java.version" property and tries to parse it. However, this patch is not directly destined for jdk9/dev, but will go into the special verona/stage forest. As for all patches destined for verona/stage it will be code reviewed as if going to jdk9/dev. Once in verona/stage it will bide its time, and it will be complemented with follow-up patches to address remaining issues. When all such issues are resolved and JEP-223 is fully implemented, all changes will be pushed at once (without further code reviews) into jdk9/dev. >>> >>> This patch has been contributed by Magnus Ihse Bursie, Kumar Srinivasan and Alejandro Murillo. >>> >>> Bug: https://bugs.openjdk.java.net/browse/JDK-8085822 >>> WebRev: http://cr.openjdk.java.net/~ihse/JDK-8085822-JEP-223-initial-patch/webrev.01 >> I looked through the code changes, skipping most of the make files :-) >> >> Version.java.template - the comment in jvmSecurityVersion() still talks about 1.6 and newer. Can this be replaced to just say that it returns the security version? >> >> Will the update_version and special_update_version fields eventually be dropped from the jvm_version_info stricture? Related, there seems to be a typo in the comment in jdk_util.c where it has "specia_update_version". >> >> The webrev shows a change to this comment in jvm.h: >> "Third, this file contains various I/O and network operations needed by the standard Java I/O and network APIs." >> I think this comment can be removed because those JVM_* functions were removed some time ago. >> >> Otherwise looks okay to me. > The API functions in Version.java and jvm.h are not finished. The specification in the JEP talks about a java.util.Version, that I presume will replace the sun.misc.Version, and that will fully implement an API to access the version string and all it's parts, according to the JEP definition. Also, the native interface will have to be changed to accommodate a version number with an arbitrarily number of dot separated parts. These changes will be done later on in the verona/stage forest. > > Are you ok with addressing these concerns at such a later time? > > /Magnus > >> -Alan. From volker.simonis at gmail.com Mon Jun 8 14:23:12 2015 From: volker.simonis at gmail.com (Volker Simonis) Date: Mon, 8 Jun 2015 16:23:12 +0200 Subject: RFR(M): 8081674: EmptyStackException at startup if running with extended or unsupported charset In-Reply-To: References: Message-ID: Hi, can I please get a review at least for the part of this fix which is common for jdk8 and jdk9. It's only a few lines in src/share/vm/prims/jni.cpp for the hotspot repository and one line src/java.base/share/classes/java/lang/ClassLoader.java for the jdk repository. Thanks, Volker On Tue, Jun 2, 2015 at 6:12 PM, Volker Simonis wrote: > Hi, > > can I please have review (and a sponsor) for these changes: > > https://bugs.openjdk.java.net/browse/JDK-8081674 > http://cr.openjdk.java.net/~simonis/webrevs/2015/8081674.jdk > http://cr.openjdk.java.net/~simonis/webrevs/2015/8081674.hs > > Please notice that the fix requires synchronous changes in the jdk and the > hotspot forest. > > The changes themselves are by far not that big as this explanation but I > found the problem to be quite intricate so I tried to explain it as good as > I could. I'd suggest to read the HTML-formatted explanation in the webrev > although the content is equal to the one in this mail: > > Using an unsupported character encoding (e.g. vi_VN.TCVN on Linux) results > in an immediate VM failure with jdk 8 and 9: > > export LANG=vi_VN.TCVN > java -version > Error occurred during initialization of VM > java.util.EmptyStackException > at java.util.Stack.peek(Stack.java:102) > at java.lang.ClassLoader$NativeLibrary.getFromClass(ClassLoader.java:1751) > at java.lang.ClassLoader$NativeLibrary.findBuiltinLib(Native Method) > at java.lang.ClassLoader.loadLibrary0(ClassLoader.java:1862) > at java.lang.ClassLoader.loadLibrary(ClassLoader.java:1835) > at java.lang.Runtime.loadLibrary0(Runtime.java:870) > at java.lang.System.loadLibrary(System.java:1119) > at java.lang.System.initializeSystemClass(System.java:1194) > > This is a consequence of "8005716: Enhance JNI specification to allow > support of static JNI libraries in Embedded JREs". > > With jdk 9 we get this error even if we're running with a supported charset > which is in the ExtendedCharsets (as opposed to being in the > StandardCharsets) class which is a consequence of delegating the loading of > the ExtendedCharsets class to the ServiceLoader in jdk 9. > > export LANG=eo.iso-8859-3 > output-jdk9/images/jdk/bin/java -version > Error occurred during initialization of VM > java.util.EmptyStackException > at java.util.Stack.peek(Stack.java:102) > at java.lang.ClassLoader$NativeLibrary.getFromClass(ClassLoader.java:1737) > at java.lang.ClassLoader$NativeLibrary.findBuiltinLib(Native Method) > at java.lang.ClassLoader.loadLibrary0(ClassLoader.java:1866) > at java.lang.ClassLoader.loadLibrary(ClassLoader.java:1840) > at java.lang.Runtime.loadLibrary0(Runtime.java:874) > at java.lang.System.loadLibrary(System.java:1111) > at java.lang.System.initializeSystemClass(System.java:1186) > > Here's why the exception happens for an unsupported charset (see the mixed > stack trace below for the full details): > > java.lang.System.loadLibrary() wants to load libzip.so. It calls > java.lang.Runtime.loadLibrary0() which at the very beginning calls the > native method ClassLoader$NativeLibrary.findBuiltinLib() which checks if the > corresponding library is already statically linked into the VM (introduced > by 8005716). Java_java_lang_ClassLoader_00024NativeLibrary_findBuiltinLib(), > the native implementation of findBuiltinLib() in Classloader.c calls > GetStringPlatformChars() to convert the library name into the native > platform encoding. GetStringPlatformChars() calls the helper function > jnuEncodingSupported() to check if the platform encoding which is stored in > the property "sun.jnu.encoding" is supported by Java. jnuEncodingSupported() > is implemented as follows: > > static jboolean isJNUEncodingSupported = JNI_FALSE; > static jboolean jnuEncodingSupported(JNIEnv *env) { > jboolean exe; > if (isJNUEncodingSupported == JNI_TRUE) { > return JNI_TRUE; > } > isJNUEncodingSupported = (jboolean) JNU_CallStaticMethodByName ( > env, &exe, > "java/nio/charset/Charset", > "isSupported", > "(Ljava/lang/String;)Z", > jnuEncoding).z; > return isJNUEncodingSupported; > } > > Once the function finds that the platform encoding is supported (by calling > java.nio.charset.Charset.isSupported()) it caches this value and always > returns it on following calls. However if the platform encoding is not > supported, it ALWAYS calls java.nio.charset.Charset.isSupported() an every > subsequent invocation. > > In order to call the Java method Charset.isSupported() (in > JNU_CallStaticMethodByName() in file jni_util.c), we have to call > jni_FindClass() to convert the symbolic class name > "java.nio.charset.Charset" into a class reference. > > But unfortunately, jni_FindClass() (from jni.cpp in libjvm.so) has a special > handling if called from java.lang.ClassLoader$NativeLibrary to ensure that > JNI_OnLoad/JNI_OnUnload are executed in the correct class context: > > instanceKlassHandle k (THREAD, thread->security_get_caller_class(0)); > if (k.not_null()) { > loader = Handle(THREAD, k->class_loader()); > // Special handling to make sure JNI_OnLoad and JNI_OnUnload are > executed > // in the correct class context. > if (loader.is_null() && > k->name() == vmSymbols::java_lang_ClassLoader_NativeLibrary()) { > JavaValue result(T_OBJECT); > JavaCalls::call_static(&result, k, > vmSymbols::getFromClass_name(), > vmSymbols::void_class_signature(), > thread); > if (HAS_PENDING_EXCEPTION) { > Handle ex(thread, thread->pending_exception()); > CLEAR_PENDING_EXCEPTION; > THROW_HANDLE_0(ex); > } > > So if that's the case and jni_FindClass() was reallycalled from > ClassLoader$NativeLibrary, then jni_FindClass() calles > ClassLoader$NativeLibrary().getFromClass() to find out the corresponding > context class which is supposed to be saved there in a field of type > java.util.Stack named "nativeLibraryContext": > > // Invoked in the VM to determine the context class in > // JNI_Load/JNI_Unload > static Class getFromClass() { > return ClassLoader.nativeLibraryContext.peek().fromClass; > } > > Unfortunately, "nativeLibraryContext" doesn't contain any entry at this > point and the invocation of Stack.peek() will throw the exception shown > before. In general, the "nativeLibraryContext" stack will be filled later on > in Runtime.loadLibrary0() like this: > > NativeLibrary lib = new NativeLibrary(fromClass, name, isBuiltin); > nativeLibraryContext.push(lib); > try { > lib.load(name, isBuiltin); > } finally { > nativeLibraryContext.pop(); > } > > such that it always contains at least one element later when jni_FindClass() > will be invoked. > > So in summary, the problem is that the implementors of 8005716 didn't took > into account that calling ClassLoader$NativeLibrary.findBuiltinLib() may > trigger a call to jni_FindClass() if we are running on a system with an > unsupported character encoding. > > I'd suggest the following fix for this problem: > > Change ClassLoader$NativeLibrary().getFromClass() to return null if the > stack is empty instead of throwing an exception: > > static Class getFromClass() { > return ClassLoader.nativeLibraryContext.empty() ? > null : ClassLoader.nativeLibraryContext.peek().fromClass; > } > > Unfortunately this also requires a HotSpot change in jni_FindClass() in > order to properly handle the new 'null' return value: > > if (k.not_null()) { > loader = Handle(THREAD, k->class_loader()); > // Special handling to make sure JNI_OnLoad and JNI_OnUnload are > executed > // in the correct class context. > if (loader.is_null() && > k->name() == vmSymbols::java_lang_ClassLoader_NativeLibrary()) { > JavaValue result(T_OBJECT); > JavaCalls::call_static(&result, k, > vmSymbols::getFromClass_name(), > vmSymbols::void_class_signature(), > thread); > if (HAS_PENDING_EXCEPTION) { > Handle ex(thread, thread->pending_exception()); > CLEAR_PENDING_EXCEPTION; > THROW_HANDLE_0(ex); > } > oop mirror = (oop) result.get_jobject(); > if (oopDesc::is_null(mirror)) { > loader = Handle(THREAD, SystemDictionary::java_system_loader()); > } else { > loader = Handle(THREAD, > > InstanceKlass::cast(java_lang_Class::as_Klass(mirror))->class_loader()); > protection_domain = Handle(THREAD, > > InstanceKlass::cast(java_lang_Class::as_Klass(mirror))->protection_domain()); > } > } > } else { > // We call ClassLoader.getSystemClassLoader to obtain the system class > loader. > loader = Handle(THREAD, SystemDictionary::java_system_loader()); > } > > These changes are sufficient to solve the problem in Java 8. Unfortunately, > that's still not enough in Java 9 because there the loading of the extended > charsets has been delegated to ServiceLoader. But ServiceLoader calls > ClassLoader.getBootstrapResources() which calls > sun.misc.Launcher.getBootstrapClassPath(). This leads to another problem > during class initialization of sun.misc.Launcher if running on an > unsupported locale. > > The first thing done in sun.misc.Launcher. is the initialisation of > the bootstrap URLClassPath in the Launcher. However this initialisation will > eventually call Charset.isSupported() and if we are running on an > unsupported locale this will inevitably end in another recursive call to > ServiceLoader. But as explained below, ServiceLoader will query the > Launcher's bootstrap URLClassPath which will be still uninitialized at that > point. > > So we'll have to additionally guard guard against this situation on JDK 9 > like this: > > private static Charset lookupExtendedCharset(String charsetName) { > if (!sun.misc.VM.isBooted() || // see lookupViaProviders() > sun.misc.Launcher.getBootstrapClassPath() == null) > return null; > > This fixes the crashes, but still at the price of not having the extended > charsets available during initialization until > Launcher.getBootstrapClassPath is set up properly. This may be still a > problem if the jdk is installed in a directory which contains characters > specific to an extended encoding or if we have such characters in the > command line arguments. > > Thank you and best regards, > Volker > > > Mixed stack trace of the initial EmptyStackException for unsupported > charsets described before: > > Native frames: (J=compiled Java code, j=interpreted, Vv=VM code, C=native > code) > j java.util.Stack.peek()Ljava/lang/Object;+1 > j java.lang.ClassLoader$NativeLibrary.getFromClass()Ljava/lang/Class;+3 > v ~StubRoutines::call_stub > V [libjvm.so+0x9d279a] JavaCalls::call_helper(JavaValue*, methodHandle*, > JavaCallArguments*, Thread*)+0x6b4 > V [libjvm.so+0xcad591] os::os_exception_wrapper(void (*)(JavaValue*, > methodHandle*, JavaCallArguments*, Thread*), JavaValue*, methodHandle*, > JavaCallArguments*, Thread*)+0x45 > V [libjvm.so+0x9d20cf] JavaCalls::call(JavaValue*, methodHandle, > JavaCallArguments*, Thread*)+0x8b > V [libjvm.so+0x9d1d3b] JavaCalls::call_static(JavaValue*, KlassHandle, > Symbol*, Symbol*, JavaCallArguments*, Thread*)+0x139 > V [libjvm.so+0x9d1e3f] JavaCalls::call_static(JavaValue*, KlassHandle, > Symbol*, Symbol*, Thread*)+0x9d > V [libjvm.so+0x9e6588] jni_FindClass+0x428 > C [libjava.so+0x20208] JNU_CallStaticMethodByName+0xff > C [libjava.so+0x21cae] jnuEncodingSupported+0x61 > C [libjava.so+0x22125] JNU_GetStringPlatformChars+0x125 > C [libjava.so+0xedcd] > Java_java_lang_ClassLoader_00024NativeLibrary_findBuiltinLib+0x8b > j > java.lang.ClassLoader$NativeLibrary.findBuiltinLib(Ljava/lang/String;)Ljava/lang/String;+0 > j java.lang.ClassLoader.loadLibrary0(Ljava/lang/Class;Ljava/io/File;)Z+4 > j > java.lang.ClassLoader.loadLibrary(Ljava/lang/Class;Ljava/lang/String;Z)V+228 > j java.lang.Runtime.loadLibrary0(Ljava/lang/Class;Ljava/lang/String;)V+54 > j java.lang.System.loadLibrary(Ljava/lang/String;)V+7 > j java.lang.System.initializeSystemClass()V+113 > v ~StubRoutines::call_stub > V [libjvm.so+0x9d279a] JavaCalls::call_helper(JavaValue*, methodHandle*, > JavaCallArguments*, Thread*)+0x6b4 > V [libjvm.so+0xcad591] os::os_exception_wrapper(void (*)(JavaValue*, > methodHandle*, JavaCallArguments*, Thread*), JavaValue*, methodHandle*, > JavaCallArguments*, Thread*)+0x45 > V [libjvm.so+0x9d20cf] JavaCalls::call(JavaValue*, methodHandle, > JavaCallArguments*, Thread*)+0x8b > V [libjvm.so+0x9d1d3b] JavaCalls::call_static(JavaValue*, KlassHandle, > Symbol*, Symbol*, JavaCallArguments*, Thread*)+0x139 > V [libjvm.so+0x9d1e3f] JavaCalls::call_static(JavaValue*, KlassHandle, > Symbol*, Symbol*, Thread*)+0x9d > V [libjvm.so+0xe3cceb] call_initializeSystemClass(Thread*)+0xb0 > V [libjvm.so+0xe44444] Threads::initialize_java_lang_classes(JavaThread*, > Thread*)+0x21a > V [libjvm.so+0xe44b12] Threads::create_vm(JavaVMInitArgs*, bool*)+0x4a6 > V [libjvm.so+0xa19bd7] JNI_CreateJavaVM+0xc7 > C [libjli.so+0xa520] InitializeJVM+0x154 > C [libjli.so+0x8024] JavaMain+0xcc > > From konstantin.shefov at oracle.com Mon Jun 8 15:04:48 2015 From: konstantin.shefov at oracle.com (Konstantin Shefov) Date: Mon, 08 Jun 2015 18:04:48 +0300 Subject: [9] Review request for 8085979: Make some DTLS feature functional tests work also for TLS protocol Message-ID: <5575AF10.1000908@oracle.com> Hello, Please review distribution of some DTLS feature tests to TLS protocol. Some DTLS tests may also be used to test the same functionality in TLS protocol and its versions. It is test only improvement. bug: https://bugs.openjdk.java.net/browse/JDK-8085979 webrev: http://cr.openjdk.java.net/~kshefov/8085979/webrev.00/ Thanks -Konstantin From Mohammad.Rezaei at gs.com Mon Jun 8 16:53:25 2015 From: Mohammad.Rezaei at gs.com (Rezaei, Mohammad A.) Date: Mon, 8 Jun 2015 12:53:25 -0400 Subject: JEP 132: More-prompt finalization (Re: Another take on Finalization) In-Reply-To: <55730B9B.6060603@gmail.com> References: <55674C6E.8050509@gmail.com> <5567CC5E.7000102@oracle.com> <8DBC7345-09D8-458F-9017-298D1E620F86@gmail.com> <6882C9A35DFB9B4FA2779F7BF5B9757D20761169A2@GSCMAMP06EX.firmwide.corp.gs.com> <556B61B0.5030708@gmail.com> <6882C9A35DFB9B4FA2779F7BF5B9757D20761169E3@GSCMAMP06EX.firmwide.corp.gs.com> <556C200B.3030603@gmail.com> <55730B9B.6060603@gmail.com> Message-ID: <6882C9A35DFB9B4FA2779F7BF5B9757D2076116C62@GSCMAMP06EX.firmwide.corp.gs.com> Shouldn't we extend the same notion to other reference types? For Weak/Soft references, the (rough) equivalent would be a method of the kind: public void markUnqueueable() // suggestions for better method name most welcome! { this.q = ReferenceQueue.NULL; } This is quite useful when the reference is cleaned up before the referent is GC'ed, which happens, for example, when rehashing a weak map. Thanks Moh >-----Original Message----- >From: core-libs-dev [mailto:core-libs-dev-bounces at openjdk.java.net] On Behalf >Of Peter Levart >Sent: Saturday, June 06, 2015 11:03 AM >To: Jonathan Payne >Cc: 'hotspot-gc-dev at openjdk.java.net openjdk.java.net'; 'core-libs- >dev at openjdk.java.net' >Subject: Re: JEP 132: More-prompt finalization (Re: Another take on >Finalization) > >Hi Jonathan, > >On 06/05/2015 11:11 PM, Jonathan Payne wrote: >> Hi. >> >> I have had an interest in the topic of finalization ever since it caused me >to abandon the G1 collector 3 or 4 years ago. >> >> I've recently implemented a fix for my interpretation of the problem, which >might be very different from the discussion currently ongoing in the thread >entitled "JEP 132: More-prompt finalization". >> >> My problem was that finalization was not being run at all with the G1 >collector. Not at all. That would have been fine with me because none of the >existing objects in the Finalizer queue actually needed the service anymore: >the files, sockets, streams, etc. had all been properly closed by my >application, otherwise the server would have long since failed completely. >However, those objects started to accumulate in the VM and eventually (8 hours >later) brought the server down. > >The most probable cause of the described behavior might be that your >finalizable objects live long enough so that they are promoted to the >old-generation together with their tracking Finalizer(s) >(FinalReferences). We know that a finalizable object must undergo at >least 2 GC cycles to be reclaimed. The 1st cycle merely discovers the >final-reachable objects and links their FinalReference(s) to the pending >chain. ReferenceHandler thread then pushes them to the finalization >queue where their final() methods are invoked by finalizer thread which >also clears their associated FinalReference(s) and unlinks them from the >double-linked list. During the 2nd cycle, such finalizable objects are >usually found unreachable and are reclaimed. It might be that G1 has (or >had) a performance problem so that it couldn't discover final-reachable >objects in old-generation soon enough to push them through 2 cycles >before the system choked. > >This is roughly similar to a problem you might have if lots of normal >objects get promoted to old-generation, but it might be the effects are >more drastic in G1 when those objects are finalizable. So you might be >able to tune your app by increasing the young generation and/or the time >it takes before objects are promoted to old-generation. > >I understand that it would be desirable for a finalizable object to be >made "untracked" as soon as it is manually cleaned-up. This would most >certainly give a relief to GC as it could reclaim such untracked objects >immediately like normal objects without pushing them through all the >finalization steps. > >Such feature would need new API. Here's my take on such API incorporated >in my prototype. This feature is mostly independent of other changes in >the prototype and could be provided stand-alone. I choose to incorporate >it into the prototype to compare it's overhead with classical >finalization in unchanged and changed JDKs: > >http://cr.openjdk.java.net/~plevart/misc/JEP132/ReferenceHandling/webrev.03/ > >The java.lang.ref.Finalizator is a public subclass of package-private >Finalizer which is a subclass of package-private FinalReference which is >a subclass of public Reference. The public API therefore consists just >of two types: Reference and Finalizator and the public methods either >implemented or inherited by Finalizator. Finalizator is basically just a >special kind of Reference, which can't be subclassed (is final), can't >be registeread with a custom reference-queue, and can only be >constructed using a Finalizator.create(finalizee, thunk) factory method >taking a 'finalizee' to be tracked and a 'thunk' that is usually just an >adapter for invoking a private cleanup method on the finalizee. >Finalizator also implements java.lang.Runnable. It's run() method is >invoked by finalization infrastructure or manually by user code that >wishes to promptly trigger clean-up (from AutoCloseable.close() method >for example). > >Here's how a classical finalizable class that also implements >AutoCloseable might be implemented. Note that the class must implement >it's own logic to make clean-up idempotent, since finalize() will be >called even after close() has manually or automatically already been called: > > static final class Finalizable extends AtomicBoolean implements >AutoCloseable { > > @Override > protected void finalize() throws Throwable { > close(); > } > > @Override > public void close() { > // close must be idempotent > if (compareAndSet(false, true)) { > // clean-up invoked at most once > } > } > } > > >And here's how an alternative celanup might be implemented using >Finalizator. Finalizator already guarantees that it's 'thunk' will be >called at most once regardless of whether it was triggered by GC and/or >manually: > > > static final class Destroyable implements AutoCloseable { > final Finalizator finalizator = > Finalizator.create(this, Destroyable::destroy); > > void destroy() { > // clean-up invoked at most once > } > > @Override > public void close() { > // close just runs the finalizator > finalizator.run(); > } > } > > >As soon as Finalizator is run() 1st time, it is cleared and unlinked >from the doubly-linked list. After that, GC can reclaim it and the >finalizee right away without pushing them through the discovery and >reference processing pipeline only to unlink the Finalizer from the >doubly-linked list. > >I have done some testing and the results of creating and destroying 100M >objects with a sustained rate of ~90 objects/ms with or without >performing (AutoCloseable) clean-up immediately after construction gives >the following results: > > >Finalization, ORIGINAL > >real 2m5.958s >user 0m33.855s >sys 0m1.982s > > >AutoCloseable combined with Finalization, ORIGINAL > >real 2m0.952s >user 0m32.103s >sys 0m1.730s > > >Finalization, PATCHED > >real 2m0.519s >user 0m16.664s >sys 0m1.240s > > >AutoCloseable combined with Finalization, PATCHED > >real 1m55.641s >user 0m16.872s >sys 0m1.218s > > >Finalizator-based cleanup, PATCHED > >real 2m1.379s >user 0m17.422s >sys 0m1.321s > > >AutoCloseable combined with Finalizator-based cleanup, PATCHED > >real 1m55.169s >user 0m4.167s >sys 0m1.139s > > > >We see what I have already shown before that my prototype practically >halves the CPU overhead of finalization infrastructure. Just making an >object AutoCloseable and promptly doing the clean-up can not reduce this >overhead if the object is also finalizable. But if manual clean-up also >"unregisters" the Finalizator from the doubly-linked list and clears it, >it spares the finalization infrastructure from processing it as a >finalizable object which further reduces the CPU overhead for a factor >of 4, totaling just 1/8th of overhead of classic finalization with >current JDK. Besides greatly reduced CPU overhead, such objects are also >potentially more promptly reclaimed by GC, freeing memory for other more >useful things... > > >> Which brings me to a few points: >> >> Finalization as conceived in the early JDKs was a bad idea. To make matters >worse, the way we then made use of it in those early days was A REALLY REALLY >bad idea. >> None of this mattered in those days because the GC ran often and quickly and >finalization occurred during every GC cycle. >> There may be situations where finalization as a feature actually matters, >but in the intervening years the JDK has added new technologies that provide a >way to accomplish finalization on your own, in your own code. A few helper >classes and it might even be easy when it's necessary, which is hopefully >almost never. >> Many of the uses of finalize() in the JDK today are bad and should be >deleted. >> >> My fix, BTW, was to use a back door (that I added to SharedSecrets) in all >the JDK classes that had a finalize() method, so that when a resource is >properly closed, by calling the close() method for example, the back door >would remove the Finalizer for the specified object from the linked list of >Finalizer objects, thus removing it from the finalization equation altogether. >I implemented this, and then the various tests of creating a huge number of >objects with a finalize() method ran quickly and flawlessly with no horrific >GCs or even a growing memory pool. The main problem with my solution was that >there was this nasty SharedSecrets back door, so it has been rejected and >probably rightly so. >> >> However, it proved a point. >> >> But now I am wondering why the actual right thing to do is not simply this: >> >> Remove the finalize() method from all the worst offenders in the JDK. >> >> I cannot remember all the places I patched when I implemented my fix, but >the majority of them were pieces of code that absolutely had a close() method. >If you don't close objects when you're done with them, your program PROBABLY >SHOULD BE BROKEN. But even if you do not accept that, for all practical >purposes, the program IS broken today because finalization is absolutely NOT >run in a timely enough fashion. > >I have shown that we can have a cake and eat it too. Combining >Finalizator-based clean-up with manual (or AutoCloseable) clean-up is a >win-win situation. Programs that forget to call close() still work and >those that do prompt close()ing will not be affected by finalization >overhead. The migration of internal JDK code from finalize() methods to >Finalizator-based cleanup should be simple and straight-forward. > >So what are we waiting for? ;-) > >> BTW - I never understood why CMS and other GC's had absolutely no problem >running finalization in a very timely fashion while the G1 collector just >never seemed to get around to it. My interpretation of that fact has always >led me to believe that it's not a throughput issue with the finalization >thread (not in real world examples, anyway) but rather a GC implementation >that didn't feel the need to be thorough enough to make sure something is >ready to be finalized. I mean, when the G1 collector was forced to run a full >collection (a death sentence on a 15Gb heap but it did occur) all the >finalizable objects were found AND finalized immediately, all 15 or 20 million >of them. >> >> So in summary: >> >> (1) The problem with finalization is that people use it. And more >importantly, that the JDK is filled with inappropriate uses of it. >> >> (2) The main solution is probably just to delete the inappropriate uses in >the JDK. But if that's not OK, then some sort of patch like what I did which >allows the JDK classes to unregister the Finalizer's when they are no longer >needed, i.e., when the object knows that it has cleaned itself up. >> >> I am curious to hear your thoughts. > >Thanks for the description of the problem you have with finalization. >JDK has an internal alternative to finalization called sun.misc.Cleaner, >which has basically the same API and implementation as my presented >Finalizator with the following differences: > >- Cleaner is a PhantomReference which means that the referent is not >obtainable any more when it is triggered, so clean-up code can only work >on state that is not part of the referent (captured by Cleaner's thunk >at the time of construction). This is suitable sometimes but not always. >- Cleaner(s) are executed by the ReferenceHandler thread direclty which >makes them unsuitable for public consumption as their thunk's must >guarantee to be executed quickly or else the whole reference processing >infrastructure blocks. Finalizator(s) are executed by the same thread(s) >as Finalizer(s). > >While it would be possible to retro-fit internal JDK classes to use >Cleaner(s) instead of finalize() methods, this would require more >refactoring which is always tricky. Finalizators, on the other hand, >could be used as a drop-in replacement for finalize() method. > >> JP >> > >Regards, Peter From peter.levart at gmail.com Mon Jun 8 17:06:40 2015 From: peter.levart at gmail.com (Peter Levart) Date: Mon, 08 Jun 2015 19:06:40 +0200 Subject: JEP 132: More-prompt finalization (Re: Another take on Finalization) In-Reply-To: <6882C9A35DFB9B4FA2779F7BF5B9757D2076116C62@GSCMAMP06EX.firmwide.corp.gs.com> References: <55674C6E.8050509@gmail.com> <5567CC5E.7000102@oracle.com> <8DBC7345-09D8-458F-9017-298D1E620F86@gmail.com> <6882C9A35DFB9B4FA2779F7BF5B9757D20761169A2@GSCMAMP06EX.firmwide.corp.gs.com> <556B61B0.5030708@gmail.com> <6882C9A35DFB9B4FA2779F7BF5B9757D20761169E3@GSCMAMP06EX.firmwide.corp.gs.com> <556C200B.3030603@gmail.com> <55730B9B.6060603@gmail.com> <6882C9A35DFB9B4FA2779F7BF5B9757D2076116C62@GSCMAMP06EX.firmwide.corp.gs.com> Message-ID: <5575CBA0.2000204@gmail.com> On 06/08/2015 06:53 PM, Rezaei, Mohammad A. wrote: > Shouldn't we extend the same notion to other reference types? For Weak/Soft references, the (rough) equivalent would be a method of the kind: > > public void markUnqueueable() // suggestions for better method name most welcome! > { > this.q = ReferenceQueue.NULL; > } > > This is quite useful when the reference is cleaned up before the referent is GC'ed, which happens, for example, when rehashing a weak map. > > Thanks > Moh Isn't Reference.clear() what you are looking for? If a Reference is cleared before being discovered by GC, it will never be enqueued. Regards, Peter From Mohammad.Rezaei at gs.com Mon Jun 8 17:45:05 2015 From: Mohammad.Rezaei at gs.com (Rezaei, Mohammad A.) Date: Mon, 8 Jun 2015 13:45:05 -0400 Subject: JEP 132: More-prompt finalization (Re: Another take on Finalization) In-Reply-To: <5575CBA0.2000204@gmail.com> References: <55674C6E.8050509@gmail.com> <5567CC5E.7000102@oracle.com> <8DBC7345-09D8-458F-9017-298D1E620F86@gmail.com> <6882C9A35DFB9B4FA2779F7BF5B9757D20761169A2@GSCMAMP06EX.firmwide.corp.gs.com> <556B61B0.5030708@gmail.com> <6882C9A35DFB9B4FA2779F7BF5B9757D20761169E3@GSCMAMP06EX.firmwide.corp.gs.com> <556C200B.3030603@gmail.com> <55730B9B.6060603@gmail.com> <6882C9A35DFB9B4FA2779F7BF5B9757D2076116C62@GSCMAMP06EX.firmwide.corp.gs.com> <5575CBA0.2000204@gmail.com> Message-ID: <6882C9A35DFB9B4FA2779F7BF5B9757D2076116C6A@GSCMAMP06EX.firmwide.corp.gs.com> If that's the case, the documentation needs to be more clear :-) It currently says: "Clears this reference object. Invoking this method will not cause this object to be enqueued." I interpret that as meaning the reference will not be put on the queue as a part of the clear() call. The bit of code that prevents this queuing is nowhere to be seen and a na?ve reading of the rest of the code suggests it will be queued: ReferenceQueue q = r.queue; if (q != ReferenceQueue.NULL) q.enqueue(r); I don't recall seeing clear() mentioned anywhere. A quick search through the jdk code shows hundreds of usages of WeakReference, but only a handful of calls to clear(). It's missing from places I would consider prime usage sites, e.g. WeakHashMap. Maybe we could even add this.queue = ReferenceQueue.NULL in clear() to avoid confusion? Thanks Moh >-----Original Message----- >From: Peter Levart [mailto:peter.levart at gmail.com] >Sent: Monday, June 08, 2015 1:07 PM >To: Rezaei, Mohammad A. [Tech] >Cc: 'hotspot-gc-dev at openjdk.java.net openjdk.java.net'; 'core-libs- >dev at openjdk.java.net' >Subject: Re: JEP 132: More-prompt finalization (Re: Another take on >Finalization) > > > >On 06/08/2015 06:53 PM, Rezaei, Mohammad A. wrote: >> Shouldn't we extend the same notion to other reference types? For Weak/Soft >references, the (rough) equivalent would be a method of the kind: >> >> public void markUnqueueable() // suggestions for better method name most >welcome! >> { >> this.q = ReferenceQueue.NULL; >> } >> >> This is quite useful when the reference is cleaned up before the referent is >GC'ed, which happens, for example, when rehashing a weak map. >> >> Thanks >> Moh > >Isn't Reference.clear() what you are looking for? > >If a Reference is cleared before being discovered by GC, it will never >be enqueued. > >Regards, Peter From kim.barrett at oracle.com Mon Jun 8 19:47:33 2015 From: kim.barrett at oracle.com (Kim Barrett) Date: Mon, 8 Jun 2015 15:47:33 -0400 Subject: JEP 132: More-prompt finalization (Re: Another take on Finalization) In-Reply-To: <55730B9B.6060603@gmail.com> References: <55674C6E.8050509@gmail.com> <5567CC5E.7000102@oracle.com> <8DBC7345-09D8-458F-9017-298D1E620F86@gmail.com> <6882C9A35DFB9B4FA2779F7BF5B9757D20761169A2@GSCMAMP06EX.firmwide.corp.gs.com> <556B61B0.5030708@gmail.com> <6882C9A35DFB9B4FA2779F7BF5B9757D20761169E3@GSCMAMP06EX.firmwide.corp.gs.com> <556C200B.3030603@gmail.com> <55730B9B.6060603@gmail.com> Message-ID: <6028BE66-BE55-4C51-96C0-63252C9D1D00@oracle.com> On Jun 6, 2015, at 11:02 AM, Peter Levart wrote: > > I understand that it would be desirable for a finalizable object to be made "untracked" as soon as it is manually cleaned-up. This would most certainly give a relief to GC as it could reclaim such untracked objects immediately like normal objects without pushing them through all the finalization steps. > > Such feature would need new API. Here's my take on such API incorporated in my prototype. This feature is mostly independent of other changes in the prototype and could be provided stand-alone. I choose to incorporate it into the prototype to compare it's overhead with classical finalization in unchanged and changed JDKs: > > http://cr.openjdk.java.net/~plevart/misc/JEP132/ReferenceHandling/webrev.03/ While changes are needed to do this sort of thing, it seems to me a better change would be to not use finalize() at all, but instead use PhantomReference-based cleanup. It provides exactly that kind of behavior, and has other benefits besides. There is some discussion about doing exactly that for File{Input,Output}Stream (see https://bugs.openjdk.java.net/browse/JDK-8080225). From peter.levart at gmail.com Mon Jun 8 20:20:38 2015 From: peter.levart at gmail.com (Peter Levart) Date: Mon, 08 Jun 2015 22:20:38 +0200 Subject: JEP 132: More-prompt finalization (Re: Another take on Finalization) In-Reply-To: <6028BE66-BE55-4C51-96C0-63252C9D1D00@oracle.com> References: <55674C6E.8050509@gmail.com> <5567CC5E.7000102@oracle.com> <8DBC7345-09D8-458F-9017-298D1E620F86@gmail.com> <6882C9A35DFB9B4FA2779F7BF5B9757D20761169A2@GSCMAMP06EX.firmwide.corp.gs.com> <556B61B0.5030708@gmail.com> <6882C9A35DFB9B4FA2779F7BF5B9757D20761169E3@GSCMAMP06EX.firmwide.corp.gs.com> <556C200B.3030603@gmail.com> <55730B9B.6060603@gmail.com> <6028BE66-BE55-4C51-96C0-63252C9D1D00@oracle.com> Message-ID: <5575F916.1020507@gmail.com> On 06/08/2015 09:47 PM, Kim Barrett wrote: > On Jun 6, 2015, at 11:02 AM, Peter Levart wrote: >> I understand that it would be desirable for a finalizable object to be made "untracked" as soon as it is manually cleaned-up. This would most certainly give a relief to GC as it could reclaim such untracked objects immediately like normal objects without pushing them through all the finalization steps. >> >> Such feature would need new API. Here's my take on such API incorporated in my prototype. This feature is mostly independent of other changes in the prototype and could be provided stand-alone. I choose to incorporate it into the prototype to compare it's overhead with classical finalization in unchanged and changed JDKs: >> >> http://cr.openjdk.java.net/~plevart/misc/JEP132/ReferenceHandling/webrev.03/ > While changes are needed to do this sort of thing, it seems to me a better change would be to not use finalize() at all, You mean FinalReference, right? This is the only remaining Reference type that isn't cleared automatically by GC. So if a XxxReference is cleared when it is discovered with referent being xxx-ly-reachable and later enqueued by ReferenceHandler thread, then GC has less work with it? Why is automatically cleared Reference more lightweight than FinalReference? Because GC can reclaim such referent right away in the same GC cycle as hooking it on the pending list? > but instead use PhantomReference-based cleanup. It provides exactly that kind of behavior, and has other benefits besides. > > There is some discussion about doing exactly that for File{Input,Output}Stream (see https://bugs.openjdk.java.net/browse/JDK-8080225). > sun.misc.Cleaner is already such thing (a PhantomReference). Would it be of interest to have a public API similar to sun.misc.Cleaner, but with the following difference: - public "Cleaners" would be executed by same thread(s) as finalize() methods. This would detach their execution from ReferenceHandler thread which must dispatch all references and must not be disturbed by user code. Regards, Peter From Alan.Bateman at oracle.com Mon Jun 8 20:41:09 2015 From: Alan.Bateman at oracle.com (Alan Bateman) Date: Mon, 08 Jun 2015 21:41:09 +0100 Subject: RFR: JDK-8085822 JEP 223: New Version-String Scheme (initial integration) In-Reply-To: <1136B10A-72A2-421D-8333-1943353F662C@oracle.com> References: <5571AD18.7080701@oracle.com> <55756194.3080506@oracle.com> <1136B10A-72A2-421D-8333-1943353F662C@oracle.com> Message-ID: <5575FDE5.5020002@oracle.com> On 08/06/2015 13:37, Magnus Ihse Bursie wrote: > : > The API functions in Version.java and jvm.h are not finished. The specification in the JEP talks about a java.util.Version, that I presume will replace the sun.misc.Version, and that will fully implement an API to access the version string and all it's parts, according to the JEP definition. Also, the native interface will have to be changed to accommodate a version number with an arbitrarily number of dot separated parts. These changes will be done later on in the verona/stage forest. > > Are you ok with addressing these concerns at such a later time? > Sure, esp. since my comments are indeed in the area that isn't complete in this webrev. -Alan From xueming.shen at oracle.com Mon Jun 8 21:37:08 2015 From: xueming.shen at oracle.com (Xueming Shen) Date: Mon, 08 Jun 2015 14:37:08 -0700 Subject: RFR 8080640: Reduce copying when reading JAR/ZIP entries In-Reply-To: <5571E5EC.6090907@oracle.com> References: <555A957F.5010102@oracle.com> <555CCAFB.4090805@oracle.com> <555E0C62.9090406@oracle.com> <555E1D49.9000707@oracle.com> <555F7862.9030307@oracle.com> <555F7AB8.1050202@oracle.com> <555F8E78.40404@oracle.com> <556DE778.1040300@oracle.com> <556F3D3F.3020906@oracle.com> <5571E5EC.6090907@oracle.com> Message-ID: <55760B04.1030202@oracle.com> Staffan, (1) ByteArrayInputSteram is no longer needed in ZipFile (2) You've changed the lock region in ZipFile.getInputSteram. Given we are not doing ByteArrayInpusteram for this method, can we just not touch this method and the class ZipFileInputSteram()? The concern is that we did some changes in that area back to 2010 and triggered a complicated race condition regression [1], it was finally fixed after lot of rounds of discussion. I still have all those emails in my inbox. It would be better to keep whatever works for now, instead of re-fresh all the memory (read all those emails) to figure out if the latest change might have a negative impact. The getBytes() implementation looks good to me. Thanks! -Sherman [1] http://mail.openjdk.java.net/pipermail/core-libs-dev/2011-March/006355.html On 06/05/2015 11:09 AM, Staffan Friberg wrote: > Hi Sherman, > > I have a new webrev which reverts that part, http://cr.openjdk.java.net/~sfriberg/JDK-8080640/webrev.2/ > > Summary of changes > Reduce lock region in ZipFile.getInputstream > Add private ZipFile.getBytes that can be used in select places in the JDK where all bytes will be read > > Could you sponsor this change once it has been reviewed? > > Thanks, > Staffan > > On 06/03/2015 10:45 AM, Xueming Shen wrote: >> Staffan, >> >> I'm not convinced that the benefit here is significant enough to change the >> getInputStream() to return a ByteArrayInputStream, given this can be easily >> achieved by wrapping the returned byte[] from getBytes(ZipEntry) at user's >> site. I would suggest to file a separate rfe on this disagreement and move on >> with the agreed getBytes() for now. >> >> Thanks, >> -Sherman >> >> On 06/02/2015 10:27 AM, Staffan Friberg wrote: >>> >>> On 05/22/2015 01:15 PM, Staffan Friberg wrote: >>>> On 05/22/2015 11:51 AM, Xueming Shen wrote: >>>>> On 05/22/2015 11:41 AM, Staffan Friberg wrote: >>>>>> >>>>>> On 05/21/2015 11:00 AM, Staffan Friberg wrote: >>>>>>> >>>>>>> On 05/21/2015 09:48 AM, Staffan Friberg wrote: >>>>>>>> >>>>>>>> On 05/20/2015 10:57 AM, Xueming Shen wrote: >>>>>>>>> On 05/18/2015 06:44 PM, Staffan Friberg wrote: >>>>>>>>>> Hi, >>>>>>>>>> >>>>>>>>>> Wanted to get reviews and feedback on this performance improvement for reading from JAR/ZIP files during classloading by reducing unnecessary copying and reading the entry in one go instead of in small portions. This shows a significant improvement when reading a single entry and for a large application with 10k classes and 500+ JAR files it improved the startup time by 4%. >>>>>>>>>> >>>>>>>>>> For more details on the background and performance results please see the RFE entry. >>>>>>>>>> >>>>>>>>>> RFE - https://bugs.openjdk.java.net/browse/JDK-8080640 >>>>>>>>>> WEBREV - http://cr.openjdk.java.net/~sfriberg/JDK-8080640/webrev.0 >>>>>>>>>> >>>>>>>>>> Cheers, >>>>>>>>>> Staffan >>>>>>>>> >>>>>>>>> Hi Staffan, >>>>>>>>> >>>>>>>>> If I did not miss something here, from your use scenario it appears to me the only thing you really >>>>>>>>> need here to help boost your performance is >>>>>>>>> >>>>>>>>> byte[] ZipFile.getAllBytes(ZipEntry ze); >>>>>>>>> >>>>>>>>> You are allocating a byte[] at use side and wrapping it with a ByteBuffer if the size is small enough, >>>>>>>>> otherwise, you letting the ZipFile to allocate a big enough one for you. It does not look like you >>>>>>>>> can re-use that byte[] (has to be wrapped by the ByteArrayInputStream and return), why do you >>>>>>>>> need two different methods here? The logic would be much easier to simply let the ZipFile to allocate >>>>>>>>> the needed buffer with appropriate size, fill the bytes and return, with a "OOME" if the entry size >>>>>>>>> is bigger than 2g. >>>>>>>>> >>>>>>>>> The only thing we use from the input ze is its name, get the size/csize from the jzentry, I don't think >>>>>>>>> jzentry.csize/size can be "unknown", they are from the "cen" table. >>>>>>>>> >>>>>>>>> If the real/final use of the bytes is to wrap it with a ByteArrayInputStream,why bother using ByteBuffer >>>>>>>>> here? Shouldn't a direct byte[] with exactly the size of the entry server better. >>>>>>>>> >>>>>>>>> -Sherman >>>>>>>>> >>>>>>>> Hi Sherman, >>>>>>>> >>>>>>>> Thanks for the comments. I agree, was starting out with bytebuffer because I was hoping to be able to cache things where the buffer was being used, but since the buffer is past along further I couldn't figure out a clean way to do it. >>>>>>>> Will rewrite it to simply just return a buffer, and only wrap it in the Resource class getByteBuffer. >>>>>>>> >>>>>>>> What would be your thought on updating the ZipFile.getInputStream to return ByteArrayInputStream for small entries? Currently I do that work outside in two places and moving it would potentially speed up others reading small entries as well. >>>>>>>> >>>>>>>> Thanks, >>>>>>>> Staffan >>>>>>> Just realized that my use of ByteArrayInputStream would miss Jar verification if enabled so the way to go hear would be to add it if possible to the ZipFile.getInputStream. >>>>>>> >>>>>>> //Staffan >>>>>> Hi, >>>>>> >>>>>> Here is an updated webrev which uses a byte[] directly and also uses ByteArrayInputStream in ZipFile for small entries below 128k. >>>>>> >>>>> >>>>> I'm not sure about the benefit of doing the ByteArrayInputStream in ZipFile.getInputStream. It has >>>>> the consequence of changing the "expected" behavior of getInputStream() (instead of return an >>>>> input stream waiting for reading, it now reads all bytes in advance), something we might not want >>>>> to do in a performance tuning. Though it might be reasonable to guess everyone get an input stream >>>>> is to read all bytes from it later. >>>>> >>>>> -Sherman >>>>> >>>>> >>>>>> http://cr.openjdk.java.net/~sfriberg/JDK-8080640/webrev.1 >>>>>> >>>>>> //Staffan >>>>> >>>> Agree that it will change the behavior slightly, but as you said it is probably expected that some one will read the stream eventually. >>>> We could reduce the size further if that makes a difference, if the size is below 65k we would not use more memory than the buffer allocated for the InflaterStream today. >>>> The total allocation would be slightly larger for deflated entries as we would allocate a byte[] for the compressed bytes, but it would be GC:able and not kept alive. So from a memory perspective the difference is very limited. >>>> >>>> //Staffan >>> Hi, >>> >>> Bumping this thread to get some more comments on the concern about changing the ZipFile.getInputStream behavior. The benefit of doing this change is that any read of small entries from ZIP and JAR files will be much faster and less resources will be held, including native resources normally held by the ZipInputStream. >>> >>> The behavior change that will occur is that the full entry will be read as part of creating the stream and not lazily as might be expected. However when getting a today InputStream zip file will be accessed to read information about the size of the entry, so the zip file is already touched when getting an InputStream, but not the compressed bytes. >>> >>> I'm fine with removing this part of the change and just push the private getBytes function and the updates to the JDK libraries to use it. >>> >>> Thanks, >>> Staffan >> > From kim.barrett at oracle.com Mon Jun 8 22:58:23 2015 From: kim.barrett at oracle.com (Kim Barrett) Date: Mon, 8 Jun 2015 18:58:23 -0400 Subject: JEP 132: More-prompt finalization (Re: Another take on Finalization) In-Reply-To: <5575F916.1020507@gmail.com> References: <55674C6E.8050509@gmail.com> <5567CC5E.7000102@oracle.com> <8DBC7345-09D8-458F-9017-298D1E620F86@gmail.com> <6882C9A35DFB9B4FA2779F7BF5B9757D20761169A2@GSCMAMP06EX.firmwide.corp.gs.com> <556B61B0.5030708@gmail.com> <6882C9A35DFB9B4FA2779F7BF5B9757D20761169E3@GSCMAMP06EX.firmwide.corp.gs.com> <556C200B.3030603@gmail.com> <55730B9B.6060603@gmail.com> <6028BE66-BE55-4C51-96C0-63252C9D1D00@oracle.com> <5575F916.1020507@gmail.com> Message-ID: <77693AFE-2539-4B7A-A7D2-E8691DEAC3F8@oracle.com> On Jun 8, 2015, at 4:20 PM, Peter Levart wrote: > > > > On 06/08/2015 09:47 PM, Kim Barrett wrote: >> On Jun 6, 2015, at 11:02 AM, Peter Levart >> wrote: >> >>> I understand that it would be desirable for a finalizable object to be made "untracked" as soon as it is manually cleaned-up. This would most certainly give a relief to GC as it could reclaim such untracked objects immediately like normal objects without pushing them through all the finalization steps. >>> >>> Such feature would need new API. Here's my take on such API incorporated in my prototype. This feature is mostly independent of other changes in the prototype and could be provided stand-alone. I choose to incorporate it into the prototype to compare it's overhead with classical finalization in unchanged and changed JDKs: >>> >>> >>> http://cr.openjdk.java.net/~plevart/misc/JEP132/ReferenceHandling/webrev.03/ >> While changes are needed to do this sort of thing, it seems to me a better change would be to not use finalize() at all, > > You mean FinalReference, right? FinalReference is an implementation detail used in the implementation of finalize(). So yes, I mean FinalReference, but applications have no direct access to them. > This is the only remaining Reference type that isn't cleared automatically by GC. PhantomReference is also not automatically cleared, but see JDK-8071507. > So if a XxxReference is cleared when it is discovered with referent being xxx-ly-reachable and later enqueued by ReferenceHandler thread, then GC has less work with it? Why is automatically cleared Reference more lightweight than FinalReference? Because GC can reclaim such referent right away in the same GC cycle as hooking it on the pending list? That?s one reason, and is part of the rationale for making the change suggested for JDK-8071507. Another benefit for PhantomReference is that the code that creates the reference object can also record it for later clearing, such as by a close-like operation. If the reference is cleared by the application then it never gets added to the pending list at all. > >> but instead use PhantomReference-based cleanup. It provides exactly that kind of behavior, and has other benefits besides. >> >> There is some discussion about doing exactly that for File{Input,Output}Stream (see >> https://bugs.openjdk.java.net/browse/JDK-8080225 >> ). >> >> > > sun.misc.Cleaner is already such thing (a PhantomReference). Would it be of interest to have a public API similar to sun.misc.Cleaner, but with the following difference: > > - public "Cleaners" would be executed by same thread(s) as finalize() methods. This would detach their execution from ReferenceHandler thread which must dispatch all references and must not be disturbed by user code. There?s been some discussion of a Cleaner-like API for ordinary PhantomReference objects, along with some prototyping. Having a utility that packages up some of the boilerplate involved in using PhantomReference might help transition away from finalize(). From daniel.daugherty at oracle.com Mon Jun 8 23:31:20 2015 From: daniel.daugherty at oracle.com (Daniel D. Daugherty) Date: Mon, 08 Jun 2015 17:31:20 -0600 Subject: RFR: JDK-8085822 JEP 223: New Version-String Scheme (initial integration) In-Reply-To: <5571AD18.7080701@oracle.com> References: <5571AD18.7080701@oracle.com> Message-ID: <557625C8.5050200@oracle.com> > http://cr.openjdk.java.net/~ihse/JDK-8085822-JEP-223-initial-patch/webrev.01 General comment: Not all copyright years were updated. General comment: It looks like support for the 'patch' value is not completely implemented through all the Makefiles. I didn't audit for this, but it's just my impression. common/autoconf/configure.ac No comments. common/autoconf/flags.m4 No comments. common/autoconf/generated-configure.sh There are multiple Copyright notices in this file. Why? L4076: # Verify that a given string represent a valid version number, and assing it to L4077: # a variable. Fixed two typos and reformat a bit: # Verify that a given string represents a valid version number, and # assigning it to a variable. L20186-20189: indent for the block is off L20256-20259: indent for the block is off L20262: as_fn_error $? "--with--version-string must have a value" "$LINENO" 5 The '--with--version...' part doesn't match previous usages where '--with-version...' is used L20275: # Unspecified numerical fields is interpreted as 0. Grammar: 'is interpreted' -> 'are interpreted' L20286: as_fn_error $? "Version string contains + but both 'BUILD' and 'OPT' is missing" "$LINENO" 5 Grammar: 'is missing' -> 'are missing' L20292: as_fn_error $? "--with--version-string fails to parse The '--with--version...' part doesn't match previous usages where '--with-version...' is used L20297-L20302: indent for the block is off L20307: as_fn_error $? "--with--version-pre-base must have a value" "$LINENO" 5 L20315: { $as_echo "$as_me:${as_lineno-$LINENO}: WARNING: --with--version-pre-base value... L20316: $as_echo "$as_me: WARNING: --with--version-pre-base value... The '--with--version...' part doesn't match previous usages where '--with-version...' is used L20327-20332: indent for the block is off L20337: as_fn_error $? "--with--version-pre-debuglevel must have... L20345: { $as_echo "$as_me:${as_lineno-$LINENO}: WARNING: --with--version-pre-debuglevel value... L20346: $as_echo "$as_me: WARNING: --with--version-pre-debuglevel value The '--with--version...' part doesn't match previous usages where '--with-version...' is used L20361-20366: indent for the block is off L20371: as_fn_error $? "--with--version-opt must have... L20379: { $as_echo "$as_me:${as_lineno-$LINENO}: WARNING: --with--version-opt value L20380: $as_echo "$as_me: WARNING: --with--version-opt value has been The '--with--version...' part doesn't match previous usages where '--with-version...' is used At this point, I'm going to stop pointing out the '--with-version...' and '--with--version...' differences; don't know which usage is right. L20388-L20388: indent is off by one L20388: username=`$ECHO "$USER" | $TR -d -c '[a-z][A-Z][0-9]'` This assumes that the "USER" variable is set. Should there be a check for "" and perhaps use "no_user_specified" or something like that? Perhaps the build setup always makes sure that "USER" is set to something. Don't know. L20395-L20400: indent for the block is off L20413-L20431: indent of all blocks in this range are off L20444-L20449: indent for the block is off L20457-L20475: indent of all blocks in this range are off L20486-L20491: indent for the block is off L20504-L20522: indent of all blocks in this range are off L20533-L20538: indent for the block is off L20551-L20569: indent of all blocks in this range are off L20580-L20585: indent for the block is off L20598-L20616: indent of all blocks in this range are off common/autoconf/help.m4 No comments. common/autoconf/jdk-options.m4 Don't know why the 'elliptic curve crypto implementation' support is relocated as part of this changeset, but ... No comments. common/autoconf/spec.gmk.in No comments. common/autoconf/version-numbers No comments. common/nb_native/nbproject/configurations.xml No comments. make/Images.gmk No comments. make/Install.gmk No comments. make/Javadoc.gmk Did you mean to remove the 'clean' target? make/JrtfsJar.gmk No comments. make/MacBundles.gmk No comments. make/jprt.properties No comments. hotspot/make/Makefile No comments. hotspot/make/aix/Makefile No comments. hotspot/make/aix/makefiles/buildtree.make No comments. hotspot/make/aix/makefiles/defs.make No comments. hotspot/make/aix/makefiles/vm.make No comments. hotspot/make/bsd/Makefile No comments. hotspot/make/bsd/makefiles/buildtree.make No comments. hotspot/make/bsd/makefiles/defs.make No comments. hotspot/make/bsd/makefiles/vm.make No comments. hotspot/make/defs.make No comments. hotspot/make/jdk_version No comments. hotspot/make/linux/Makefile No comments. hotspot/make/linux/makefiles/buildtree.make No comments. hotspot/make/linux/makefiles/defs.make No comments. hotspot/make/linux/makefiles/vm.make No comments. hotspot/make/solaris/Makefile No comments. hotspot/make/solaris/makefiles/buildtree.make No comments. hotspot/make/solaris/makefiles/defs.make No comments. hotspot/make/solaris/makefiles/sparcWorks.make No comments. hotspot/make/solaris/makefiles/vm.make No comments. hotspot/make/windows/build.make No comments. hotspot/make/windows/makefiles/compile.make No changes in the frames view. Update: udiff view shows a blank line deleted at the end of the file. hotspot/make/windows/makefiles/debug.make No comments. hotspot/make/windows/makefiles/defs.make No comments. hotspot/make/windows/makefiles/fastdebug.make No comments. hotspot/make/windows/makefiles/product.make No comments. hotspot/make/windows/makefiles/vm.make No comments. hotspot/make/windows/projectfiles/common/Makefile No comments. hotspot/src/share/vm/prims/jvm.h No comments. hotspot/src/share/vm/runtime/arguments.cpp No comments. hotspot/src/share/vm/runtime/java.cpp L661: void JDK_Version::fully_initialize( L662: uint8_t major, uint8_t minor, uint8_t security, uint8_t update) { L663: // This is only called when current is less than 1.6 and we've gotten Since you're ripping out vestigial version support, I think this function can go away since this is version 9 and newer. Don't really know for sure, but based on that comment... hotspot/src/share/vm/runtime/java.hpp No comments. hotspot/src/share/vm/runtime/vmStructs.cpp L1240: please make the 'int' parameter align like the rest. hotspot/src/share/vm/runtime/vm_version.cpp L84: void Abstract_VM_Version::initialize() { L85: // FIXME: Initialization can probably be removed now. I agree, but that would entail also removing the _initialized and the uses of it... Follow on bug fix? hotspot/src/share/vm/runtime/vm_version.hpp No comments. hotspot/test/runtime/6981737/Test6981737.java No comments. jdk/make/CompileDemos.gmk No comments. jdk/make/data/mainmanifest/manifest.mf No comments. jdk/make/gensrc/GensrcMisc.gmk No comments. jdk/make/launcher/Launcher-jdk.accessibility.gmk No comments. jdk/make/launcher/Launcher-jdk.pack200.gmk No comments. jdk/make/launcher/LauncherCommon.gmk No comments. jdk/make/lib/CoreLibraries.gmk No comments. jdk/src/java.base/share/classes/sun/misc/Version.java.template L149: * Returns the security version of the running JVM if it's 1.6 or newer This JavaDoc update is wrong, but it might not be important if sun.misc.Version class is going away. jdk/src/java.base/share/native/include/jvm.h No comments. jdk/src/java.base/share/native/launcher/defines.h No comments. jdk/src/java.base/share/native/launcher/main.c No comments. jdk/src/java.base/share/native/libjava/System.c No comments. jdk/src/java.base/share/native/libjava/Version.c No comments. jdk/src/java.base/share/native/libjava/jdk_util.c No comments. jdk/src/java.base/windows/native/common/version.rc No comments. jdk/src/java.desktop/windows/native/libawt/windows/awt.rc No comments. jdk/src/jdk.accessibility/windows/native/common/AccessBridgeStatusWindow.RC No comments. jdk/test/sun/misc/Version/Version.java No comments. langtools/make/gensrc/GensrcCommon.gmk No comments. langtools/src/java.compiler/share/classes/javax/lang/model/SourceVersion.java old L171: case "1.9": new L171: case "9": Should this logic support both versions? Will dropping "1.9" here prevent a pre-version changeset JVM from being dropped into a JDK for triage purposes? Granted we don't often triage 'javac' with different JVMs, but... langtools/test/tools/javac/options/modes/InfoOptsTest.java No comments. langtools/test/tools/javac/options/modes/SourceTargetTest.java No comments. nashorn/make/BuildNashorn.gmk No comments. nashorn/make/build.xml No comments. nashorn/src/jdk.scripting.nashorn/share/classes/jdk/nashorn/internal/runtime/Version.java No comments. common/autoconf/jdk-version.m4 No comments. nashorn/src/jdk.scripting.nashorn/share/classes/jdk/nashorn/internal/runtime/resources/version.properties.template nashorn/src/jdk.scripting.nashorn/share/classes/jdk/nashorn/internal/runtime/resources/version.properties-template No comments. common/bin/test_builds.sh hotspot/make/jdk6_hotspot_distro No comments. Dan On 6/5/15 8:07 AM, Magnus Ihse Bursie wrote: > This review request covers the main part of the work for JEP-223, the > new version string format [1]. Basically, we'll call this release Java > "9", instead of Java "1.9.0". > > This patch is a folding of all work that has been done so far in the > branch JEP-223-branch in jdk9/sandbox. As you can see, it mostly > covers build changes, with some code changes in hotspot, jdk, nashorn > and langtools that either are corresponding changes in the product > code due to the compiler define flags changing from the build, or > follow-up changes to handle the new format. > > The JEP-223 work is not finished by this patch. In fact, there are > known issues remaining even after this patch, typically by code that > reads the "java.version" property and tries to parse it. However, this > patch is not directly destined for jdk9/dev, but will go into the > special verona/stage forest. As for all patches destined for > verona/stage it will be code reviewed as if going to jdk9/dev. Once in > verona/stage it will bide its time, and it will be complemented with > follow-up patches to address remaining issues. When all such issues > are resolved and JEP-223 is fully implemented, all changes will be > pushed at once (without further code reviews) into jdk9/dev. > > This patch has been contributed by Magnus Ihse Bursie, Kumar > Srinivasan and Alejandro Murillo. > > Bug: https://bugs.openjdk.java.net/browse/JDK-8085822 > WebRev: > http://cr.openjdk.java.net/~ihse/JDK-8085822-JEP-223-initial-patch/webrev.01 > > [1] http://openjdk.java.net/jeps/223 > From mandy.chung at oracle.com Mon Jun 8 23:35:06 2015 From: mandy.chung at oracle.com (Mandy Chung) Date: Mon, 08 Jun 2015 16:35:06 -0700 Subject: RFR(M): 8081674: EmptyStackException at startup if running with extended or unsupported charset In-Reply-To: References: Message-ID: <557626AA.6010008@oracle.com> Hi Volker, Your patch reminds me the question I have about loading zip library. In JDK 9 (and earlier release), zip library is loaded by the VM during startup (see ClassLoader::load_zip_library). I think loadLibrary("zip") is no longer needed to be called from System::initializeSystemClass method and instead it can be loaded by java.util.zip.ZipFile static initializer. Do you mind to try the patch (below)? That may be a simpler fix. I never like the way jni_FindClass to look for the class context by calling the NativeLibrary::getFromClass method. I will have to look deeper other alternative to clean that up. If taking out loadLibrary("zip") resolves your issue, this will give us time to come up with a better clean-up in the future. Mandy [1] https://bugs.openjdk.java.net/browse/JDK-4429040 diff --git a/src/share/classes/java/lang/System.java b/src/share/classes/java/lang/System.java --- a/src/share/classes/java/lang/System.java +++ b/src/share/classes/java/lang/System.java @@ -1192,10 +1192,6 @@ setOut0(newPrintStream(fdOut, props.getProperty("sun.stdout.encoding"))); setErr0(newPrintStream(fdErr, props.getProperty("sun.stderr.encoding"))); - // Load the zip library now in order to keep java.util.zip.ZipFile - // from trying to use itself to load this library later. - loadLibrary("zip"); - // Setup Java signal handlers for HUP, TERM, and INT (where available). Terminator.setup(); diff --git a/src/share/classes/java/util/zip/ZipFile.java b/src/share/classes/java/util/zip/ZipFile.java --- a/src/share/classes/java/util/zip/ZipFile.java +++ b/src/share/classes/java/util/zip/ZipFile.java @@ -83,6 +83,7 @@ static { /* Zip library is loaded from System.initializeSystemClass */ + System.loadLibrary("zip"); initIDs(); } On 06/08/2015 07:23 AM, Volker Simonis wrote: > Hi, > > can I please get a review at least for the part of this fix which is > common for jdk8 and jdk9. It's only a few lines in > src/share/vm/prims/jni.cpp for the hotspot repository and one line > src/java.base/share/classes/java/lang/ClassLoader.java for the jdk > repository. > > Thanks, > Volker > > > On Tue, Jun 2, 2015 at 6:12 PM, Volker Simonis wrote: >> Hi, >> >> can I please have review (and a sponsor) for these changes: >> >> https://bugs.openjdk.java.net/browse/JDK-8081674 >> http://cr.openjdk.java.net/~simonis/webrevs/2015/8081674.jdk >> http://cr.openjdk.java.net/~simonis/webrevs/2015/8081674.hs >> >> Please notice that the fix requires synchronous changes in the jdk and the >> hotspot forest. >> >> The changes themselves are by far not that big as this explanation but I >> found the problem to be quite intricate so I tried to explain it as good as >> I could. I'd suggest to read the HTML-formatted explanation in the webrev >> although the content is equal to the one in this mail: >> >> Using an unsupported character encoding (e.g. vi_VN.TCVN on Linux) results >> in an immediate VM failure with jdk 8 and 9: >> >> export LANG=vi_VN.TCVN >> java -version >> Error occurred during initialization of VM >> java.util.EmptyStackException >> at java.util.Stack.peek(Stack.java:102) >> at java.lang.ClassLoader$NativeLibrary.getFromClass(ClassLoader.java:1751) >> at java.lang.ClassLoader$NativeLibrary.findBuiltinLib(Native Method) >> at java.lang.ClassLoader.loadLibrary0(ClassLoader.java:1862) >> at java.lang.ClassLoader.loadLibrary(ClassLoader.java:1835) >> at java.lang.Runtime.loadLibrary0(Runtime.java:870) >> at java.lang.System.loadLibrary(System.java:1119) >> at java.lang.System.initializeSystemClass(System.java:1194) >> >> This is a consequence of "8005716: Enhance JNI specification to allow >> support of static JNI libraries in Embedded JREs". >> >> With jdk 9 we get this error even if we're running with a supported charset >> which is in the ExtendedCharsets (as opposed to being in the >> StandardCharsets) class which is a consequence of delegating the loading of >> the ExtendedCharsets class to the ServiceLoader in jdk 9. >> >> export LANG=eo.iso-8859-3 >> output-jdk9/images/jdk/bin/java -version >> Error occurred during initialization of VM >> java.util.EmptyStackException >> at java.util.Stack.peek(Stack.java:102) >> at java.lang.ClassLoader$NativeLibrary.getFromClass(ClassLoader.java:1737) >> at java.lang.ClassLoader$NativeLibrary.findBuiltinLib(Native Method) >> at java.lang.ClassLoader.loadLibrary0(ClassLoader.java:1866) >> at java.lang.ClassLoader.loadLibrary(ClassLoader.java:1840) >> at java.lang.Runtime.loadLibrary0(Runtime.java:874) >> at java.lang.System.loadLibrary(System.java:1111) >> at java.lang.System.initializeSystemClass(System.java:1186) >> >> Here's why the exception happens for an unsupported charset (see the mixed >> stack trace below for the full details): >> >> java.lang.System.loadLibrary() wants to load libzip.so. It calls >> java.lang.Runtime.loadLibrary0() which at the very beginning calls the >> native method ClassLoader$NativeLibrary.findBuiltinLib() which checks if the >> corresponding library is already statically linked into the VM (introduced >> by 8005716). Java_java_lang_ClassLoader_00024NativeLibrary_findBuiltinLib(), >> the native implementation of findBuiltinLib() in Classloader.c calls >> GetStringPlatformChars() to convert the library name into the native >> platform encoding. GetStringPlatformChars() calls the helper function >> jnuEncodingSupported() to check if the platform encoding which is stored in >> the property "sun.jnu.encoding" is supported by Java. jnuEncodingSupported() >> is implemented as follows: >> >> static jboolean isJNUEncodingSupported = JNI_FALSE; >> static jboolean jnuEncodingSupported(JNIEnv *env) { >> jboolean exe; >> if (isJNUEncodingSupported == JNI_TRUE) { >> return JNI_TRUE; >> } >> isJNUEncodingSupported = (jboolean) JNU_CallStaticMethodByName ( >> env, &exe, >> "java/nio/charset/Charset", >> "isSupported", >> "(Ljava/lang/String;)Z", >> jnuEncoding).z; >> return isJNUEncodingSupported; >> } >> >> Once the function finds that the platform encoding is supported (by calling >> java.nio.charset.Charset.isSupported()) it caches this value and always >> returns it on following calls. However if the platform encoding is not >> supported, it ALWAYS calls java.nio.charset.Charset.isSupported() an every >> subsequent invocation. >> >> In order to call the Java method Charset.isSupported() (in >> JNU_CallStaticMethodByName() in file jni_util.c), we have to call >> jni_FindClass() to convert the symbolic class name >> "java.nio.charset.Charset" into a class reference. >> >> But unfortunately, jni_FindClass() (from jni.cpp in libjvm.so) has a special >> handling if called from java.lang.ClassLoader$NativeLibrary to ensure that >> JNI_OnLoad/JNI_OnUnload are executed in the correct class context: >> >> instanceKlassHandle k (THREAD, thread->security_get_caller_class(0)); >> if (k.not_null()) { >> loader = Handle(THREAD, k->class_loader()); >> // Special handling to make sure JNI_OnLoad and JNI_OnUnload are >> executed >> // in the correct class context. >> if (loader.is_null() && >> k->name() == vmSymbols::java_lang_ClassLoader_NativeLibrary()) { >> JavaValue result(T_OBJECT); >> JavaCalls::call_static(&result, k, >> vmSymbols::getFromClass_name(), >> vmSymbols::void_class_signature(), >> thread); >> if (HAS_PENDING_EXCEPTION) { >> Handle ex(thread, thread->pending_exception()); >> CLEAR_PENDING_EXCEPTION; >> THROW_HANDLE_0(ex); >> } >> >> So if that's the case and jni_FindClass() was reallycalled from >> ClassLoader$NativeLibrary, then jni_FindClass() calles >> ClassLoader$NativeLibrary().getFromClass() to find out the corresponding >> context class which is supposed to be saved there in a field of type >> java.util.Stack named "nativeLibraryContext": >> >> // Invoked in the VM to determine the context class in >> // JNI_Load/JNI_Unload >> static Class getFromClass() { >> return ClassLoader.nativeLibraryContext.peek().fromClass; >> } >> >> Unfortunately, "nativeLibraryContext" doesn't contain any entry at this >> point and the invocation of Stack.peek() will throw the exception shown >> before. In general, the "nativeLibraryContext" stack will be filled later on >> in Runtime.loadLibrary0() like this: >> >> NativeLibrary lib = new NativeLibrary(fromClass, name, isBuiltin); >> nativeLibraryContext.push(lib); >> try { >> lib.load(name, isBuiltin); >> } finally { >> nativeLibraryContext.pop(); >> } >> >> such that it always contains at least one element later when jni_FindClass() >> will be invoked. >> >> So in summary, the problem is that the implementors of 8005716 didn't took >> into account that calling ClassLoader$NativeLibrary.findBuiltinLib() may >> trigger a call to jni_FindClass() if we are running on a system with an >> unsupported character encoding. >> >> I'd suggest the following fix for this problem: >> >> Change ClassLoader$NativeLibrary().getFromClass() to return null if the >> stack is empty instead of throwing an exception: >> >> static Class getFromClass() { >> return ClassLoader.nativeLibraryContext.empty() ? >> null : ClassLoader.nativeLibraryContext.peek().fromClass; >> } >> >> Unfortunately this also requires a HotSpot change in jni_FindClass() in >> order to properly handle the new 'null' return value: >> >> if (k.not_null()) { >> loader = Handle(THREAD, k->class_loader()); >> // Special handling to make sure JNI_OnLoad and JNI_OnUnload are >> executed >> // in the correct class context. >> if (loader.is_null() && >> k->name() == vmSymbols::java_lang_ClassLoader_NativeLibrary()) { >> JavaValue result(T_OBJECT); >> JavaCalls::call_static(&result, k, >> vmSymbols::getFromClass_name(), >> vmSymbols::void_class_signature(), >> thread); >> if (HAS_PENDING_EXCEPTION) { >> Handle ex(thread, thread->pending_exception()); >> CLEAR_PENDING_EXCEPTION; >> THROW_HANDLE_0(ex); >> } >> oop mirror = (oop) result.get_jobject(); >> if (oopDesc::is_null(mirror)) { >> loader = Handle(THREAD, SystemDictionary::java_system_loader()); >> } else { >> loader = Handle(THREAD, >> >> InstanceKlass::cast(java_lang_Class::as_Klass(mirror))->class_loader()); >> protection_domain = Handle(THREAD, >> >> InstanceKlass::cast(java_lang_Class::as_Klass(mirror))->protection_domain()); >> } >> } >> } else { >> // We call ClassLoader.getSystemClassLoader to obtain the system class >> loader. >> loader = Handle(THREAD, SystemDictionary::java_system_loader()); >> } >> >> These changes are sufficient to solve the problem in Java 8. Unfortunately, >> that's still not enough in Java 9 because there the loading of the extended >> charsets has been delegated to ServiceLoader. But ServiceLoader calls >> ClassLoader.getBootstrapResources() which calls >> sun.misc.Launcher.getBootstrapClassPath(). This leads to another problem >> during class initialization of sun.misc.Launcher if running on an >> unsupported locale. >> >> The first thing done in sun.misc.Launcher. is the initialisation of >> the bootstrap URLClassPath in the Launcher. However this initialisation will >> eventually call Charset.isSupported() and if we are running on an >> unsupported locale this will inevitably end in another recursive call to >> ServiceLoader. But as explained below, ServiceLoader will query the >> Launcher's bootstrap URLClassPath which will be still uninitialized at that >> point. >> >> So we'll have to additionally guard guard against this situation on JDK 9 >> like this: >> >> private static Charset lookupExtendedCharset(String charsetName) { >> if (!sun.misc.VM.isBooted() || // see lookupViaProviders() >> sun.misc.Launcher.getBootstrapClassPath() == null) >> return null; >> >> This fixes the crashes, but still at the price of not having the extended >> charsets available during initialization until >> Launcher.getBootstrapClassPath is set up properly. This may be still a >> problem if the jdk is installed in a directory which contains characters >> specific to an extended encoding or if we have such characters in the >> command line arguments. >> >> Thank you and best regards, >> Volker >> >> >> Mixed stack trace of the initial EmptyStackException for unsupported >> charsets described before: >> >> Native frames: (J=compiled Java code, j=interpreted, Vv=VM code, C=native >> code) >> j java.util.Stack.peek()Ljava/lang/Object;+1 >> j java.lang.ClassLoader$NativeLibrary.getFromClass()Ljava/lang/Class;+3 >> v ~StubRoutines::call_stub >> V [libjvm.so+0x9d279a] JavaCalls::call_helper(JavaValue*, methodHandle*, >> JavaCallArguments*, Thread*)+0x6b4 >> V [libjvm.so+0xcad591] os::os_exception_wrapper(void (*)(JavaValue*, >> methodHandle*, JavaCallArguments*, Thread*), JavaValue*, methodHandle*, >> JavaCallArguments*, Thread*)+0x45 >> V [libjvm.so+0x9d20cf] JavaCalls::call(JavaValue*, methodHandle, >> JavaCallArguments*, Thread*)+0x8b >> V [libjvm.so+0x9d1d3b] JavaCalls::call_static(JavaValue*, KlassHandle, >> Symbol*, Symbol*, JavaCallArguments*, Thread*)+0x139 >> V [libjvm.so+0x9d1e3f] JavaCalls::call_static(JavaValue*, KlassHandle, >> Symbol*, Symbol*, Thread*)+0x9d >> V [libjvm.so+0x9e6588] jni_FindClass+0x428 >> C [libjava.so+0x20208] JNU_CallStaticMethodByName+0xff >> C [libjava.so+0x21cae] jnuEncodingSupported+0x61 >> C [libjava.so+0x22125] JNU_GetStringPlatformChars+0x125 >> C [libjava.so+0xedcd] >> Java_java_lang_ClassLoader_00024NativeLibrary_findBuiltinLib+0x8b >> j >> java.lang.ClassLoader$NativeLibrary.findBuiltinLib(Ljava/lang/String;)Ljava/lang/String;+0 >> j java.lang.ClassLoader.loadLibrary0(Ljava/lang/Class;Ljava/io/File;)Z+4 >> j >> java.lang.ClassLoader.loadLibrary(Ljava/lang/Class;Ljava/lang/String;Z)V+228 >> j java.lang.Runtime.loadLibrary0(Ljava/lang/Class;Ljava/lang/String;)V+54 >> j java.lang.System.loadLibrary(Ljava/lang/String;)V+7 >> j java.lang.System.initializeSystemClass()V+113 >> v ~StubRoutines::call_stub >> V [libjvm.so+0x9d279a] JavaCalls::call_helper(JavaValue*, methodHandle*, >> JavaCallArguments*, Thread*)+0x6b4 >> V [libjvm.so+0xcad591] os::os_exception_wrapper(void (*)(JavaValue*, >> methodHandle*, JavaCallArguments*, Thread*), JavaValue*, methodHandle*, >> JavaCallArguments*, Thread*)+0x45 >> V [libjvm.so+0x9d20cf] JavaCalls::call(JavaValue*, methodHandle, >> JavaCallArguments*, Thread*)+0x8b >> V [libjvm.so+0x9d1d3b] JavaCalls::call_static(JavaValue*, KlassHandle, >> Symbol*, Symbol*, JavaCallArguments*, Thread*)+0x139 >> V [libjvm.so+0x9d1e3f] JavaCalls::call_static(JavaValue*, KlassHandle, >> Symbol*, Symbol*, Thread*)+0x9d >> V [libjvm.so+0xe3cceb] call_initializeSystemClass(Thread*)+0xb0 >> V [libjvm.so+0xe44444] Threads::initialize_java_lang_classes(JavaThread*, >> Thread*)+0x21a >> V [libjvm.so+0xe44b12] Threads::create_vm(JavaVMInitArgs*, bool*)+0x4a6 >> V [libjvm.so+0xa19bd7] JNI_CreateJavaVM+0xc7 >> C [libjli.so+0xa520] InitializeJVM+0x154 >> C [libjli.so+0x8024] JavaMain+0xcc >> >> From kim.barrett at oracle.com Mon Jun 8 23:41:21 2015 From: kim.barrett at oracle.com (Kim Barrett) Date: Mon, 8 Jun 2015 19:41:21 -0400 Subject: JEP 132: More-prompt finalization (Re: Another take on Finalization) In-Reply-To: <6882C9A35DFB9B4FA2779F7BF5B9757D2076116C6A@GSCMAMP06EX.firmwide.corp.gs.com> References: <55674C6E.8050509@gmail.com> <5567CC5E.7000102@oracle.com> <8DBC7345-09D8-458F-9017-298D1E620F86@gmail.com> <6882C9A35DFB9B4FA2779F7BF5B9757D20761169A2@GSCMAMP06EX.firmwide.corp.gs.com> <556B61B0.5030708@gmail.com> <6882C9A35DFB9B4FA2779F7BF5B9757D20761169E3@GSCMAMP06EX.firmwide.corp.gs.com> <556C200B.3030603@gmail.com> <55730B9B.6060603@gmail.com> <6882C9A35DFB9B4FA2779F7BF5B9757D2076116C62@GSCMAMP06EX.firmwide.corp.gs.com> <5575CBA0.2000204@gmail.com> <6882C9A35DFB9B4FA2779F7BF5B9757D2076116C6A@GSCMAMP06EX.firmwide.corp.gs.com> Message-ID: <770B325C-9DB1-42C7-AA5F-7E42B7FC0027@oracle.com> On Jun 8, 2015, at 1:45 PM, Rezaei, Mohammad A. wrote: > > If that's the case, the documentation needs to be more clear :-) It currently says: > > "Clears this reference object. Invoking this method will not cause this object to be enqueued." > > I interpret that as meaning the reference will not be put on the queue as a part of the clear() call. > > The bit of code that prevents this queuing is nowhere to be seen It?s in the VM; see callers of (C++) discover_reference, which all skip that call if the referent is null. For example: http://hg.openjdk.java.net/jdk9/hs-rt/hotspot/annotate/431b1333b1c1/src/share/vm/oops/instanceRefKlass.inline.hpp circa line 47 From joe.darcy at oracle.com Tue Jun 9 01:00:00 2015 From: joe.darcy at oracle.com (Joseph D. Darcy) Date: Mon, 08 Jun 2015 18:00:00 -0700 Subject: FYI, make file support for tiered testing targets underway, JDK-8075571: Support tiered testing make targets In-Reply-To: <556CD9BB.6040204@oracle.com> References: <556CD9BB.6040204@oracle.com> Message-ID: <55763A90.3080902@oracle.com> Hello jaxp and core-libs teams, FYI, I have some in-progress changes to add Makefile support for tiered testing: JDK-8075571: Support tiered testing make targets http://cr.openjdk.java.net/~darcy/8075571.0/ This involves adding some new TEST.group definitions in the jdk and jaxp repos. The review is happening on build-dev: http://mail.openjdk.java.net/pipermail/build-dev/2015-June/015133.html Thanks, -Joe From joe.darcy at oracle.com Tue Jun 9 01:27:26 2015 From: joe.darcy at oracle.com (joe darcy) Date: Mon, 08 Jun 2015 18:27:26 -0700 Subject: JDK 9 RFR of JDK-8086029: Fix doclint reference warnings in org.omg.CORBA Message-ID: <557640FE.1060801@oracle.com> Hello, As an addendum to the fix-the-warnings JEP, please review this change which fixes some bad @throws references and the like over in corba: 8086029: Fix doclint reference warnings in org.omg.CORBA http://cr.openjdk.java.net/~darcy/8086029.0/ Patch below. Thanks, -Joe --- old/src/java.corba/share/classes/org/omg/CORBA/DynAny.java 2015-06-08 18:21:46.550021735 -0700 +++ new/src/java.corba/share/classes/org/omg/CORBA/DynAny.java 2015-06-08 18:21:46.478021732 -0700 @@ -52,8 +52,8 @@ * * @param dyn_any the DynAny object whose contents * are assigned to this DynAny. - * @throws Invalid if the source DynAny is - * invalid + * @throws org.omg.CORBA.DynAnyPackage.Invalid if the source + * DynAny is invalid */ public void assign(org.omg.CORBA.DynAny dyn_any) throws org.omg.CORBA.DynAnyPackage.Invalid; @@ -63,8 +63,8 @@ * object. * * @param value the Any object. - * @throws Invalid if the source Any object is - * empty or bad + * @throws org.omg.CORBA.DynAnyPackage.Invalid if the source + * Any object is empty or bad */ public void from_any(org.omg.CORBA.Any value) throws org.omg.CORBA.DynAnyPackage.Invalid; @@ -74,9 +74,9 @@ * object. * * @return the Any object. - * @throws Invalid if this DynAny is empty or - * bad. - * created or does not contain a meaningful value + * @throws org.omg.CORBA.DynAnyPackage.Invalid if this + * DynAny is empty or bad. created or does not + * contain a meaningful value */ public org.omg.CORBA.Any to_any() throws org.omg.CORBA.DynAnyPackage.Invalid; --- old/src/java.corba/share/classes/org/omg/CORBA/DynArray.java 2015-06-08 18:21:46.866021744 -0700 +++ new/src/java.corba/share/classes/org/omg/CORBA/DynArray.java 2015-06-08 18:21:46.798021742 -0700 @@ -38,8 +38,8 @@ * Returns the value of all the elements of this array. * * @return the array of Any objects that is the value - * for this DynArray object - * @see #set_elements + * for this DynArray object + * @see #set_elements */ public org.omg.CORBA.Any[] get_elements(); @@ -48,8 +48,9 @@ * DynArray object to the given array. * * @param value the array of Any objects - * @exception InvalidSeq if the sequence is bad - * @see #get_elements + * @exception org.omg.CORBA.DynAnyPackage.InvalidSeq if the + * sequence is bad + * @see #get_elements */ public void set_elements(org.omg.CORBA.Any[] value) throws org.omg.CORBA.DynAnyPackage.InvalidSeq; --- old/src/java.corba/share/classes/org/omg/CORBA/DynSequence.java 2015-06-08 18:21:47.154021753 -0700 +++ new/src/java.corba/share/classes/org/omg/CORBA/DynSequence.java 2015-06-08 18:21:47.082021751 -0700 @@ -65,8 +65,9 @@ * array. * * @param value the array of Any objects to be set - * @exception InvalidSeq if the array of values is bad - * @see #get_elements + * @exception org.omg.CORBA.DynAnyPackage.InvalidSeq if the array + * of values is bad + * @see #get_elements */ public void set_elements(org.omg.CORBA.Any[] value) throws org.omg.CORBA.DynAnyPackage.InvalidSeq; --- old/src/java.corba/share/classes/org/omg/CORBA/ServerRequest.java 2015-06-08 18:21:47.450021762 -0700 +++ new/src/java.corba/share/classes/org/omg/CORBA/ServerRequest.java 2015-06-08 18:21:47.374021759 -0700 @@ -248,7 +248,7 @@ * contain an exception will result in a BAD_PARAM system exception. Passing * in an unlisted user exception will result in either the DIR receiving a * BAD_PARAM system exception or in the client receiving an - * UNKNOWN_EXCEPTION system exception. + * UNKNOWN system exception. * * @param any the Any object containing the exception * @deprecated use set_exception() @@ -272,13 +272,13 @@ * will cause a BAD_PARAM system exception to be thrown. Passing * in an unlisted user exception will result in either the DIR receiving a * BAD_PARAM system exception or in the client receiving an - * UNKNOWN_EXCEPTION system exception. + * UNKNOWN system exception. * * @param any the Any object containing the exception * @exception BAD_PARAM if the given Any object does not * contain an exception or the exception is an * unlisted user exception - * @exception UNKNOWN_EXCEPTION if the given exception is an unlisted + * @exception UNKNOWN if the given exception is an unlisted * user exception and the DIR did not * receive a BAD_PARAM exception * @see CORBA From lance.andersen at oracle.com Tue Jun 9 01:31:22 2015 From: lance.andersen at oracle.com (Lance @ Oracle) Date: Mon, 8 Jun 2015 21:31:22 -0400 Subject: JDK 9 RFR of JDK-8086029: Fix doclint reference warnings in org.omg.CORBA In-Reply-To: <557640FE.1060801@oracle.com> References: <557640FE.1060801@oracle.com> Message-ID: <7DEBAEEA-9AA4-4CD1-ADE7-9B99F8E18881@oracle.com> +1 Lance Andersen| Principal Member of Technical Staff | +1.781.442.2037 Oracle Java Engineering 1 Network Drive Burlington, MA 01803 Lance.Andersen at oracle.com Sent from my iPad > On Jun 8, 2015, at 9:27 PM, joe darcy wrote: > > Hello, > > As an addendum to the fix-the-warnings JEP, please review this change which fixes some bad @throws references and the like over in corba: > > 8086029: Fix doclint reference warnings in org.omg.CORBA > http://cr.openjdk.java.net/~darcy/8086029.0/ > > Patch below. > > Thanks, > > -Joe > > --- old/src/java.corba/share/classes/org/omg/CORBA/DynAny.java 2015-06-08 18:21:46.550021735 -0700 > +++ new/src/java.corba/share/classes/org/omg/CORBA/DynAny.java 2015-06-08 18:21:46.478021732 -0700 > @@ -52,8 +52,8 @@ > * > * @param dyn_any the DynAny object whose contents > * are assigned to this DynAny. > - * @throws Invalid if the source DynAny is > - * invalid > + * @throws org.omg.CORBA.DynAnyPackage.Invalid if the source > + * DynAny is invalid > */ > public void assign(org.omg.CORBA.DynAny dyn_any) > throws org.omg.CORBA.DynAnyPackage.Invalid; > @@ -63,8 +63,8 @@ > * object. > * > * @param value the Any object. > - * @throws Invalid if the source Any object is > - * empty or bad > + * @throws org.omg.CORBA.DynAnyPackage.Invalid if the source > + * Any object is empty or bad > */ > public void from_any(org.omg.CORBA.Any value) > throws org.omg.CORBA.DynAnyPackage.Invalid; > @@ -74,9 +74,9 @@ > * object. > * > * @return the Any object. > - * @throws Invalid if this DynAny is empty or > - * bad. > - * created or does not contain a meaningful value > + * @throws org.omg.CORBA.DynAnyPackage.Invalid if this > + * DynAny is empty or bad. created or does not > + * contain a meaningful value > */ > public org.omg.CORBA.Any to_any() > throws org.omg.CORBA.DynAnyPackage.Invalid; > --- old/src/java.corba/share/classes/org/omg/CORBA/DynArray.java 2015-06-08 18:21:46.866021744 -0700 > +++ new/src/java.corba/share/classes/org/omg/CORBA/DynArray.java 2015-06-08 18:21:46.798021742 -0700 > @@ -38,8 +38,8 @@ > * Returns the value of all the elements of this array. > * > * @return the array of Any objects that is the value > - * for this DynArray object > - * @see #set_elements > + * for this DynArray object > + * @see #set_elements > */ > public org.omg.CORBA.Any[] get_elements(); > @@ -48,8 +48,9 @@ > * DynArray object to the given array. > * > * @param value the array of Any objects > - * @exception InvalidSeq if the sequence is bad > - * @see #get_elements > + * @exception org.omg.CORBA.DynAnyPackage.InvalidSeq if the > + * sequence is bad > + * @see #get_elements > */ > public void set_elements(org.omg.CORBA.Any[] value) > throws org.omg.CORBA.DynAnyPackage.InvalidSeq; > --- old/src/java.corba/share/classes/org/omg/CORBA/DynSequence.java 2015-06-08 18:21:47.154021753 -0700 > +++ new/src/java.corba/share/classes/org/omg/CORBA/DynSequence.java 2015-06-08 18:21:47.082021751 -0700 > @@ -65,8 +65,9 @@ > * array. > * > * @param value the array of Any objects to be set > - * @exception InvalidSeq if the array of values is bad > - * @see #get_elements > + * @exception org.omg.CORBA.DynAnyPackage.InvalidSeq if the array > + * of values is bad > + * @see #get_elements > */ > public void set_elements(org.omg.CORBA.Any[] value) > throws org.omg.CORBA.DynAnyPackage.InvalidSeq; > --- old/src/java.corba/share/classes/org/omg/CORBA/ServerRequest.java 2015-06-08 18:21:47.450021762 -0700 > +++ new/src/java.corba/share/classes/org/omg/CORBA/ServerRequest.java 2015-06-08 18:21:47.374021759 -0700 > @@ -248,7 +248,7 @@ > * contain an exception will result in a BAD_PARAM system exception. Passing > * in an unlisted user exception will result in either the DIR receiving a > * BAD_PARAM system exception or in the client receiving an > - * UNKNOWN_EXCEPTION system exception. > + * UNKNOWN system exception. > * > * @param any the Any object containing the exception > * @deprecated use set_exception() > @@ -272,13 +272,13 @@ > * will cause a BAD_PARAM system exception to be thrown. Passing > * in an unlisted user exception will result in either the DIR receiving a > * BAD_PARAM system exception or in the client receiving an > - * UNKNOWN_EXCEPTION system exception. > + * UNKNOWN system exception. > * > * @param any the Any object containing the exception > * @exception BAD_PARAM if the given Any object does not > * contain an exception or the exception is an > * unlisted user exception > - * @exception UNKNOWN_EXCEPTION if the given exception is an unlisted > + * @exception UNKNOWN if the given exception is an unlisted > * user exception and the DIR did not > * receive a BAD_PARAM exception > * @see CORBA > > From kim.barrett at oracle.com Tue Jun 9 02:03:06 2015 From: kim.barrett at oracle.com (Kim Barrett) Date: Mon, 8 Jun 2015 22:03:06 -0400 Subject: JEP 132: More-prompt finalization In-Reply-To: <556B61B0.5030708@gmail.com> References: <55674C6E.8050509@gmail.com> <5567CC5E.7000102@oracle.com> <8DBC7345-09D8-458F-9017-298D1E620F86@gmail.com> <6882C9A35DFB9B4FA2779F7BF5B9757D20761169A2@GSCMAMP06EX.firmwide.corp.gs.com> <556B61B0.5030708@gmail.com> Message-ID: On May 31, 2015, at 3:32 PM, Peter Levart wrote: > > So, for the curious, here's the improved prototype: > > http://cr.openjdk.java.net/~plevart/misc/JEP132/ReferenceHandling/webrev.02/ > > And here are the improved benchmarks (with some results inline): > > http://cr.openjdk.java.net/~plevart/misc/JEP132/ReferenceHandling/refproc/ While perusing the offered code (webrev.02) (not a careful review yet), I think I've found a serious bug: In Reference.java in unhookPendingChunk 253 // assert r.next == r; 254 // move a chunk of pending/discovered references to a 255 // temporary local r/next chain ... 262 rd.next = r; I think that assertion suggested at line 253 does not hold, and line 262 may damage a queue. The problem is that the application could explicitly enqueue a reference while it is sitting in the pending list, waiting to be processed. So the enqueue places the reference on the associated queue's list, linked through the reference's next field. Then unhook comes along and clobbers the reference's next field. Oops! handlePendingChunk has similar problems with an application enqueue before the reference has been "handled". I haven't looked carefully at webrev.03, but at a quick glance it looks like it has the same problem. (Which is what I expected, given the description of the differences between webrev.02 and webrev.03.) I'm not sure why unhook is using the next field at all, rather than just continuing to use the discovered field. I've not studied this idea carefully, but I think it might work to leave the chunks linked through the discovered field until addChunk, with that being responsible for self-looping the discovered field and transferring linkage to the next field. That is, treat chunks as extensions of the pending list until addChunk-time. There might be some that makes using the discovered field that way a problem, but I'm not thinking of anything right now. Of course, this problem doesn't exist for FinalReference because application code doesn't have access to them to perform the problematic explicit enqueue. From jan.lahoda at oracle.com Tue Jun 9 04:57:58 2015 From: jan.lahoda at oracle.com (Jan Lahoda) Date: Tue, 09 Jun 2015 06:57:58 +0200 Subject: RFR: JDK-8085822 JEP 223: New Version-String Scheme (initial integration) In-Reply-To: <557625C8.5050200@oracle.com> References: <5571AD18.7080701@oracle.com> <557625C8.5050200@oracle.com> Message-ID: <55767256.3030802@oracle.com> On 9.6.2015 01:31, Daniel D. Daugherty wrote: > > > http://cr.openjdk.java.net/~ihse/JDK-8085822-JEP-223-initial-patch/webrev.01 > langtools/src/java.compiler/share/classes/javax/lang/model/SourceVersion.java > > old L171: case "1.9": > new L171: case "9": > Should this logic support both versions? Will dropping > "1.9" here prevent a pre-version changeset JVM from > being dropped into a JDK for triage purposes? > > Granted we don't often triage 'javac' with different JVMs, but... > +1 on keeping both "1.9" and "9" here. javac can be used independently on the rest of JDK to some extent, so support for running it on older builds of JDK 9 seems reasonable to me. (I wonder if current JDK 9 javac should be prepared for the new version scheme in advance.) Jan From peter.levart at gmail.com Tue Jun 9 05:44:23 2015 From: peter.levart at gmail.com (Peter Levart) Date: Tue, 09 Jun 2015 07:44:23 +0200 Subject: JEP 132: More-prompt finalization In-Reply-To: References: <55674C6E.8050509@gmail.com> <5567CC5E.7000102@oracle.com> <8DBC7345-09D8-458F-9017-298D1E620F86@gmail.com> <6882C9A35DFB9B4FA2779F7BF5B9757D20761169A2@GSCMAMP06EX.firmwide.corp.gs.com> <556B61B0.5030708@gmail.com> Message-ID: <55767D37.4080100@gmail.com> Hi Kim, Thanks for taking a look at this code. On 06/09/2015 04:03 AM, Kim Barrett wrote: > On May 31, 2015, at 3:32 PM, Peter Levart wrote: >> So, for the curious, here's the improved prototype: >> >> http://cr.openjdk.java.net/~plevart/misc/JEP132/ReferenceHandling/webrev.02/ >> >> And here are the improved benchmarks (with some results inline): >> >> http://cr.openjdk.java.net/~plevart/misc/JEP132/ReferenceHandling/refproc/ > While perusing the offered code (webrev.02) (not a careful review > yet), I think I've found a serious bug: > > In Reference.java > in unhookPendingChunk > 253 // assert r.next == r; > 254 // move a chunk of pending/discovered references to a > 255 // temporary local r/next chain > ... > 262 rd.next = r; > > I think that assertion suggested at line 253 does not hold, and line > 262 may damage a queue. > > The problem is that the application could explicitly enqueue a > reference while it is sitting in the pending list, waiting to be > processed. So the enqueue places the reference on the associated > queue's list, linked through the reference's next field. Then unhook > comes along and clobbers the reference's next field. Oops! > > handlePendingChunk has similar problems with an application enqueue > before the reference has been "handled". > > I haven't looked carefully at webrev.03, but at a quick glance it > looks like it has the same problem. (Which is what I expected, given > the description of the differences between webrev.02 and webrev.03.) > > I'm not sure why unhook is using the next field at all, rather than > just continuing to use the discovered field. I've not studied this > idea carefully, but I think it might work to leave the chunks linked > through the discovered field until addChunk, with that being > responsible for self-looping the discovered field and transferring > linkage to the next field. That is, treat chunks as extensions of the > pending list until addChunk-time. There might be some that makes using > the discovered field that way a problem, but I'm not thinking of > anything right now. > > Of course, this problem doesn't exist for FinalReference because > application code doesn't have access to them to perform the > problematic explicit enqueue. > > Ops, you're right. Explicit enqueue-ing is what skipped my mind since I've been 1st working on finalizers and then generalized the solution... I'm afraid to use the 'discovered' field since it is in the domain of VM and I think Java side can only reset it to null while holding the lock. I also would not like to add another field to Reference just to support "chunking". Perhaps the chunk could be formed as a re-usable array of references. Let me think about this some more to figure out how to solve it most elegantly... I'll follow-up when I have something. Regards, Peter From huizhe.wang at oracle.com Tue Jun 9 06:06:58 2015 From: huizhe.wang at oracle.com (huizhe wang) Date: Mon, 08 Jun 2015 23:06:58 -0700 Subject: FYI, make file support for tiered testing targets underway, JDK-8075571: Support tiered testing make targets In-Reply-To: <55763A90.3080902@oracle.com> References: <556CD9BB.6040204@oracle.com> <55763A90.3080902@oracle.com> Message-ID: <55768282.6010605@oracle.com> On 6/8/2015 6:00 PM, Joseph D. Darcy wrote: > Hello jaxp and core-libs teams, > > FYI, I have some in-progress changes to add Makefile support for > tiered testing: Thanks for the info. Joe > > JDK-8075571: Support tiered testing make targets > http://cr.openjdk.java.net/~darcy/8075571.0/ > > This involves adding some new TEST.group definitions in the jdk and > jaxp repos. The review is happening on build-dev: > > http://mail.openjdk.java.net/pipermail/build-dev/2015-June/015133.html > > Thanks, > > -Joe From Alan.Bateman at oracle.com Tue Jun 9 06:36:38 2015 From: Alan.Bateman at oracle.com (Alan Bateman) Date: Tue, 09 Jun 2015 07:36:38 +0100 Subject: JDK 9 RFR of JDK-8086029: Fix doclint reference warnings in org.omg.CORBA In-Reply-To: <557640FE.1060801@oracle.com> References: <557640FE.1060801@oracle.com> Message-ID: <55768976.90709@oracle.com> On 09/06/2015 02:27, joe darcy wrote: > Hello, > > As an addendum to the fix-the-warnings JEP, please review this change > which fixes some bad @throws references and the like over in corba: > > 8086029: Fix doclint reference warnings in org.omg.CORBA > http://cr.openjdk.java.net/~darcy/8086029.0/ > > Patch below. This looks okay to me except that it's probably best to keep the line wrap style consistent (even if odd) in this area. -Alan. From peter.levart at gmail.com Tue Jun 9 06:38:18 2015 From: peter.levart at gmail.com (Peter Levart) Date: Tue, 09 Jun 2015 08:38:18 +0200 Subject: JEP 132: More-prompt finalization (Re: Another take on Finalization) In-Reply-To: <770B325C-9DB1-42C7-AA5F-7E42B7FC0027@oracle.com> References: <55674C6E.8050509@gmail.com> <5567CC5E.7000102@oracle.com> <8DBC7345-09D8-458F-9017-298D1E620F86@gmail.com> <6882C9A35DFB9B4FA2779F7BF5B9757D20761169A2@GSCMAMP06EX.firmwide.corp.gs.com> <556B61B0.5030708@gmail.com> <6882C9A35DFB9B4FA2779F7BF5B9757D20761169E3@GSCMAMP06EX.firmwide.corp.gs.com> <556C200B.3030603@gmail.com> <55730B9B.6060603@gmail.com> <6882C9A35DFB9B4FA2779F7BF5B9757D2076116C62@GSCMAMP06EX.firmwide.corp.gs.com> <5575CBA0.2000204@gmail.com> <6882C9A35DFB9B4FA2779F7BF5B9757D2076116C6A@GSCMAMP06EX.firmwide.corp.gs.com> <770B325C-9DB1-42C7-AA5F-7E42B7FC0027@oracle.com> Message-ID: <557689DA.30300@gmail.com> On 06/09/2015 01:41 AM, Kim Barrett wrote: > On Jun 8, 2015, at 1:45 PM, Rezaei, Mohammad A. wrote: >> If that's the case, the documentation needs to be more clear :-) It currently says: >> >> "Clears this reference object. Invoking this method will not cause this object to be enqueued." >> >> I interpret that as meaning the reference will not be put on the queue as a part of the clear() call. >> >> The bit of code that prevents this queuing is nowhere to be seen > It?s in the VM; see callers of (C++) discover_reference, which all skip that call if the referent is null. > > For example: > http://hg.openjdk.java.net/jdk9/hs-rt/hotspot/annotate/431b1333b1c1/src/share/vm/oops/instanceRefKlass.inline.hpp > circa line 47 > Moh, If you're looking for javadoc part where this is described, then see the part of for example WeakReference class documentation which says: "Suppose that the garbage collector determines at a certain point in time that an object is *weakly reachable*. At that time it will atomically clear all weak references to that object and all weak references to any other weakly-reachable objects from which that object is reachable through a chain of strong and soft references. At the same time it will declare all of the formerly weakly-reachable objects to be finalizable. At the same time or at some later time it will enqueue those newly-cleared weak references that are registered with reference queues." ...and in addition, the description of "weakly-reachable" and reachability in general: * An object is /strongly reachable/ if it can be reached by some thread without traversing any reference objects. A newly-created object is strongly reachable by the thread that created it. * An object is /softly reachable/ if it is not strongly reachable but can be reached by traversing a soft reference. * An object is */weakly reachable/* if it is neither strongly nor softly reachable but can be reached by traversing a weak reference. When the weak references to a weakly-reachable object are cleared, the object becomes eligible for finalization. * An object is /phantom reachable/ if it is neither strongly, softly, nor weakly reachable, it has been finalized, and some phantom reference refers to it. * Finally, an object is /unreachable/, and therefore eligible for reclamation, when it is not reachable in any of the above ways. ...so for an object to be "weakly-reachable" it has to be reachable by traversing a WeakReference. If you manually clear all WeakReferences pointing to an object, It will not be weakly-reachable any more, so GC will not do any actions that would make WeakReferences that formerly pointed to that object being enqueued at a later time. It can't do any actions, since those WeakReferences are not in any way associated with a referent any more after they are manually cleared. Regards, Peter From Paul.Sandoz at oracle.com Tue Jun 9 07:20:40 2015 From: Paul.Sandoz at oracle.com (Paul Sandoz) Date: Tue, 9 Jun 2015 08:20:40 +0100 Subject: RFR 8071597 Add Stream dropWhile and takeWhile operations In-Reply-To: References: <79392DA7-6A8E-42AC-9E83-AA7F89D8F2A5@oracle.com> Message-ID: <1D517CCF-9124-44CF-9390-02A35C05F989@oracle.com> HI Stefan, On Jun 7, 2015, at 10:07 PM, Stefan Zobel wrote: > I'm still trying to wrap my head around the test logic for the (par & !ord) > && (op == WhileOp.Drop) case > in the whileResultAsserter() method in WhileOpTest. > > Wouldn't it be possible that, for an unordered parallel stream, dropWhile() > won't drop anything at all (i.e., > drops an empty set)? > Yes. > In that case, input.size() == output.size() and the set of matching output > elements is no longer a _proper_ subset > of the set of matching input elements. The whileResultAsserter() would fail > even though dropWhile() works correctly > (in a nondeterministic sense)? > > It's a bit late now and I guess it's all in my imagination. Still curious > ... > I don't think you are imagining things, thanks for looking. The proper subset is incorrect, i confused myself :-) For example, if we have the input set {1, 2, 3} and the predicate i < 3, then a valid output set is {1, 2, 3} if the imputed order observed is [3, 2, 1] or [3, 1, 2] I adjusted the test code accordingly. Thanks, Paul. From vladimir.x.ivanov at oracle.com Tue Jun 9 10:28:39 2015 From: vladimir.x.ivanov at oracle.com (Vladimir Ivanov) Date: Tue, 09 Jun 2015 13:28:39 +0300 Subject: [8u-dev] Review request : JDK-8062904: TEST_BUG: Tests java/lang/invoke/LFCaching fail when run with -Xcomp option In-Reply-To: <5575C15C.6080501@oracle.com> References: <5565D9BB.60003@oracle.com> <5565F6D5.7030900@oracle.com> <5566D9D8.4090401@oracle.com> <55675CCB.8010506@oracle.com> <5568281B.7040108@oracle.com> <55685264.20001@oracle.com> <556C86C6.2000409@oracle.com> <556DFB1C.7010301@oracle.com> <556ECC99.2000702@oracle.com> <55717713.80606@oracle.com> <5571A2D6.8020508@oracle.com> <5571E4B8.60101@oracle.com> <5575C15C.6080501@oracle.com> Message-ID: <5576BFD7.9040701@oracle.com> Konstantin, thanks for figuring that out. I had a private discussion with Igor and we agreed that catching "expected" exceptions (NSME and VME) is more robust than hand tuning test limits to particular configurations. The idea is to gracefully finish the test once code cache is full, instead of trying to avoid code cache overflow at all costs in all test configurations. In order to lower the risk of missing product issues, exception messages can be checked as well: NSME: "no such method: java.lang.invoke.MethodHandle.linkTo.*" VME: "out of space in CodeCache for adapters" Though checking error messages is fragile as well, IMO in lesser extent than hardcoded limits. Do you have any objections to such approach? As a strawman proposal, you can introduce a helper in test library: ignoreException(Runnable test, Function filter) Also, please, find and adjust other tests where hard coded limits are set. Best regards, Vladimir Ivanov On 6/8/15 7:22 PM, Konstantin Shefov wrote: > Vladimir, > > You were right when you said that problem is not in JDK 8u. > I have examined the test code once more and found the following. > > In JDK 9 because of the Segmented code cache feature I have modified the > tests to support it, and in JDK 9 the number of iterations is limited by > the size of "NonProfiledCodeHeapSize", while in JDK 8u the number of > iterations is limited by "ReservedCodeCacheSize". > > But "ReservedCodeCacheSize" is usually about 256 MB, while > "NonProfiledCodeHeapSize" is about 128 MB. So in JDK 8u there are twice > more iterations happen when in JDK 9. > > If we run JDK 8u tests against JDK 9 with the same "-Dseed", we will > have VirtualMachineError on JDK 9. > > So there is NO product issue. > > In the file "test/java/lang/invoke/LFCaching/LambdaFormTestCase.java" > there is a constant defined called "ITERATIONS_TO_CODE_CACHE_SIZE_RATIO". > This constant has been measured with respect to -Xmixed mode only. > > So my suggestion to fix this test is to define one more constant for > "-Xcomp" mode and use it in "-Xcomp" mode. > > -Konstantin > > On 06/05/2015 09:04 PM, Vladimir Ivanov wrote: >> Konstantin, >> >>>> I'd like to get an answer to my previous question: >>>> "How reliable the test is if it ignores NoSuchMethodException & >>>> VirtualMachineError? Are there other manifestations of the problem?" >>>> >>>> Most notably, have you seen VM crashes w/ -Xcomp running that test? >>> I have seen no crashes with -Xcomp in our tests results base, only >>> NoSuchMethodException and VirtualMachineError. >> Good. Thanks for the info. >> >>> This failure does not occur with JDK 9 on -Xcomp, I used the same random >>> seeds (-Dseed), and the problem exists only with JDK 8u, not JDK 9. >>> >>> Because there is no failure with JDK 9 I can suppose there could be a >>> product issue in JDK 8u. >> That's strange. There should be no difference in MethodHandle stub >> management between 8u & 9 now (once allocated they stay forever). 9 >> has segmented code cache feature and it can lead to observable >> differences in behavior. >> >> How many iterations does the test perform on 8u & 9? What if you >> increase the number? Also, I'd suggest you to monitor code cache usage >> during the run and when the test finishes. >> >> Best regards, >> Vladimir Ivanov > From magnus.ihse.bursie at oracle.com Tue Jun 9 13:12:19 2015 From: magnus.ihse.bursie at oracle.com (Magnus Ihse Bursie) Date: Tue, 09 Jun 2015 15:12:19 +0200 Subject: RFR: JDK-8085822 JEP 223: New Version-String Scheme (initial integration) In-Reply-To: <557625C8.5050200@oracle.com> References: <5571AD18.7080701@oracle.com> <557625C8.5050200@oracle.com> Message-ID: <5576E633.9050503@oracle.com> Hi Daniel, Thank you for your thorough review! On 2015-06-09 01:31, Daniel D. Daugherty wrote: > > > http://cr.openjdk.java.net/~ihse/JDK-8085822-JEP-223-initial-patch/webrev.01 > > > General comment: Not all copyright years were updated. > General comment: It looks like support for the 'patch' value is not > completely > implemented through all the Makefiles. I didn't audit for this, > but it's > just my impression. You are basically correct. The makefiles all propagate the patch value where needed, but the actual source code changes in hotspot and jdk ignores the patch field as of now. This will be corrected in a later add-on patch, submitted by someone from the jdk and/or hotspot team rather than the build team. > > common/autoconf/generated-configure.sh > There are multiple Copyright notices in this file. Why? Oh dear, you've reviewed the generated file. :-& I'm really impressed by your effort! However, the generated-configure.sh file is automatically created by the autoconf framework from the rest of the files in common/autoconf/*, so we cannot direcly modify it's contents - only indirectly. The reason it's checked in, is that otherwise every user would need to generate it before being able to run configure. In build reviews, we usually either revert changes to generated-configure.sh before posting a review to hide it (and re-generate it before pushing), or we just ignore it during reviews. I should have done that, or pointed out that it was not needed nor possible to review when I cross-posted. I'm sorry. > > L4076: # Verify that a given string represent a valid version > number, and assing it to > L4077: # a variable. > Fixed two typos and reformat a bit: > # Verify that a given string represents a valid version > number, and > # assigning it to a variable. I'll put that fix in the source .m4 file instead. Thanks. > > L20262: as_fn_error $? "--with--version-string must have a > value" "$LINENO" 5 > The '--with--version...' part doesn't match previous usages where > '--with-version...' is used Yes, you're right. Erik pointed it out as well. It's a typo in the error message. It's all over the place due to copied code. I'll fix it in the source .m4 file. (As a side note, I have a half-finished follow-up patch that will reduce the amount of code duplication, but it requires some framework changes so it'll have to be a separate thing.) > > L20275: # Unspecified numerical fields is interpreted as 0. > Grammar: 'is interpreted' -> 'are interpreted' > > L20286: as_fn_error $? "Version string contains + but both > 'BUILD' and 'OPT' is missing" "$LINENO" 5 > Grammar: 'is missing' -> 'are missing' Those darn English plural forms! Why can't you all do as we sensible Swedes and get rid of them? ;-) (I'll fix) > > L20388: username=`$ECHO "$USER" | $TR -d -c '[a-z][A-Z][0-9]'` > This assumes that the "USER" variable is set. Should there > be a check for "" and perhaps use "no_user_specified" or > something like that? Perhaps the build setup always makes > sure that "USER" is set to something. Don't know. Hm. I think the worst thing that'll happen is that the username part of the opt string gets empty. This part is basically copied right off the old build, where we have relied on the $USER variable being present for all the time with no problems, so I think it's quite safe to assume. > > common/autoconf/jdk-options.m4 > Don't know why the 'elliptic curve crypto implementation' support > is relocated as part of this changeset, but ... It was incorrectly placed not at the top indentation level, but in the middle of the function definition where the old versioning code were handled. (Which, mostly by chance, happens to work in autoconf, but is really bad style). > make/Javadoc.gmk > Did you mean to remove the 'clean' target? Yep. Cleaning is done by the top-level Main.gmk makefile nowadays, that's just some dead code. > > hotspot/make/windows/makefiles/compile.make > No changes in the frames view. > Update: udiff view shows a blank line deleted at the end of the file. That's probably the result of some change going forth and then back again during development. But then again, removing extra blank linkes is not a bad thing. (jcheck unfortunately does not enforce any style checks for make files :-( so they tend to detoriate over time.) > > hotspot/src/share/vm/runtime/java.cpp > L661: void JDK_Version::fully_initialize( > L662: uint8_t major, uint8_t minor, uint8_t security, uint8_t > update) { > L663: // This is only called when current is less than 1.6 and > we've gotten > > Since you're ripping out vestigial version support, I think this > function can go away since this is version 9 and newer. Don't > really > know for sure, but based on that comment... It probably can, yes. This and other core changes to the actual .cpp/.java files will be addressed in a follow-up patch. > > hotspot/src/share/vm/runtime/java.hpp > No comments. > > hotspot/src/share/vm/runtime/vmStructs.cpp > L1240: please make the 'int' parameter align like the rest. Are you okay with defering such a change to a follow-up patch? It's likely the entire struct will need changes to be able to handle a theoretically arbitrarily long version number. > > hotspot/src/share/vm/runtime/vm_version.cpp > L84: void Abstract_VM_Version::initialize() { > L85: // FIXME: Initialization can probably be removed now. > I agree, but that would entail also removing the > _initialized and the uses of it... Follow on bug fix? Definitely follow on. > jdk/src/java.base/share/classes/sun/misc/Version.java.template > L149: * Returns the security version of the running JVM if > it's 1.6 or newer > This JavaDoc update is wrong, but it might not be important > if sun.misc.Version class is going away. I'm not sure if it's going to be replaced by, or just complemented by java.util.Version, but in any case it will need a follow-up patch to clean up this and other issues. > langtools/src/java.compiler/share/classes/javax/lang/model/SourceVersion.java > > old L171: case "1.9": > new L171: case "9": > Should this logic support both versions? Will dropping > "1.9" here prevent a pre-version changeset JVM from > being dropped into a JDK for triage purposes? > > Granted we don't often triage 'javac' with different JVMs, but... I'll defer that question to Kumar, who wrote that piece of code. My guess is that when Hotspot express was dropped, the ability to use a JVM from another JDK release bit rotteded away. /Magnus From konstantin.shefov at oracle.com Tue Jun 9 13:11:02 2015 From: konstantin.shefov at oracle.com (Konstantin Shefov) Date: Tue, 09 Jun 2015 16:11:02 +0300 Subject: [8u-dev] Review request : JDK-8062904: TEST_BUG: Tests java/lang/invoke/LFCaching fail when run with -Xcomp option In-Reply-To: <5576BFD7.9040701@oracle.com> References: <5565D9BB.60003@oracle.com> <5565F6D5.7030900@oracle.com> <5566D9D8.4090401@oracle.com> <55675CCB.8010506@oracle.com> <5568281B.7040108@oracle.com> <55685264.20001@oracle.com> <556C86C6.2000409@oracle.com> <556DFB1C.7010301@oracle.com> <556ECC99.2000702@oracle.com> <55717713.80606@oracle.com> <5571A2D6.8020508@oracle.com> <5571E4B8.60101@oracle.com> <5575C15C.6080501@oracle.com> <5576BFD7.9040701@oracle.com> Message-ID: <5576E5E6.6030205@oracle.com> There is one more bug associated with this problem: https://bugs.openjdk.java.net/browse/JDK-8067005 I can change all 4 tests. Your idea sounds good, because we may expect more iterations to run before code cache overflows. -Konstantin On 06/09/2015 01:28 PM, Vladimir Ivanov wrote: > Konstantin, thanks for figuring that out. > > I had a private discussion with Igor and we agreed that catching > "expected" exceptions (NSME and VME) is more robust than hand tuning > test limits to particular configurations. > > The idea is to gracefully finish the test once code cache is full, > instead of trying to avoid code cache overflow at all costs in all > test configurations. > > In order to lower the risk of missing product issues, exception > messages can be checked as well: > NSME: "no such method: java.lang.invoke.MethodHandle.linkTo.*" > VME: "out of space in CodeCache for adapters" > > Though checking error messages is fragile as well, IMO in lesser > extent than hardcoded limits. Do you have any objections to such > approach? > > As a strawman proposal, you can introduce a helper in test library: > ignoreException(Runnable test, Function filter) > > Also, please, find and adjust other tests where hard coded limits are > set. > > Best regards, > Vladimir Ivanov > > On 6/8/15 7:22 PM, Konstantin Shefov wrote: >> Vladimir, >> >> You were right when you said that problem is not in JDK 8u. >> I have examined the test code once more and found the following. >> >> In JDK 9 because of the Segmented code cache feature I have modified the >> tests to support it, and in JDK 9 the number of iterations is limited by >> the size of "NonProfiledCodeHeapSize", while in JDK 8u the number of >> iterations is limited by "ReservedCodeCacheSize". >> >> But "ReservedCodeCacheSize" is usually about 256 MB, while >> "NonProfiledCodeHeapSize" is about 128 MB. So in JDK 8u there are twice >> more iterations happen when in JDK 9. >> >> If we run JDK 8u tests against JDK 9 with the same "-Dseed", we will >> have VirtualMachineError on JDK 9. >> >> So there is NO product issue. >> >> In the file "test/java/lang/invoke/LFCaching/LambdaFormTestCase.java" >> there is a constant defined called >> "ITERATIONS_TO_CODE_CACHE_SIZE_RATIO". >> This constant has been measured with respect to -Xmixed mode only. >> >> So my suggestion to fix this test is to define one more constant for >> "-Xcomp" mode and use it in "-Xcomp" mode. >> >> -Konstantin >> >> On 06/05/2015 09:04 PM, Vladimir Ivanov wrote: >>> Konstantin, >>> >>>>> I'd like to get an answer to my previous question: >>>>> "How reliable the test is if it ignores NoSuchMethodException & >>>>> VirtualMachineError? Are there other manifestations of the problem?" >>>>> >>>>> Most notably, have you seen VM crashes w/ -Xcomp running that test? >>>> I have seen no crashes with -Xcomp in our tests results base, only >>>> NoSuchMethodException and VirtualMachineError. >>> Good. Thanks for the info. >>> >>>> This failure does not occur with JDK 9 on -Xcomp, I used the same >>>> random >>>> seeds (-Dseed), and the problem exists only with JDK 8u, not JDK 9. >>>> >>>> Because there is no failure with JDK 9 I can suppose there could be a >>>> product issue in JDK 8u. >>> That's strange. There should be no difference in MethodHandle stub >>> management between 8u & 9 now (once allocated they stay forever). 9 >>> has segmented code cache feature and it can lead to observable >>> differences in behavior. >>> >>> How many iterations does the test perform on 8u & 9? What if you >>> increase the number? Also, I'd suggest you to monitor code cache usage >>> during the run and when the test finishes. >>> >>> Best regards, >>> Vladimir Ivanov >> From daniel.daugherty at oracle.com Tue Jun 9 13:20:27 2015 From: daniel.daugherty at oracle.com (Daniel D. Daugherty) Date: Tue, 09 Jun 2015 07:20:27 -0600 Subject: RFR: JDK-8085822 JEP 223: New Version-String Scheme (initial integration) In-Reply-To: <5576E633.9050503@oracle.com> References: <5571AD18.7080701@oracle.com> <557625C8.5050200@oracle.com> <5576E633.9050503@oracle.com> Message-ID: <5576E81B.8050703@oracle.com> On 6/9/15 7:12 AM, Magnus Ihse Bursie wrote: > Hi Daniel, > > Thank you for your thorough review! This was my (failing) attempt at a "fast pass" review... :-) > > On 2015-06-09 01:31, Daniel D. Daugherty wrote: >> > >> http://cr.openjdk.java.net/~ihse/JDK-8085822-JEP-223-initial-patch/webrev.01 >> >> >> General comment: Not all copyright years were updated. >> General comment: It looks like support for the 'patch' value is not >> completely >> implemented through all the Makefiles. I didn't audit for this, >> but it's >> just my impression. > > You are basically correct. The makefiles all propagate the patch value > where needed, but the actual source code changes in hotspot and jdk > ignores the patch field as of now. This will be corrected in a later > add-on patch, submitted by someone from the jdk and/or hotspot team > rather than the build team. > >> >> common/autoconf/generated-configure.sh >> There are multiple Copyright notices in this file. Why? > Oh dear, you've reviewed the generated file. :-& I'm really impressed > by your effort! Ahhh... now that you say it... it sounds familiar... I may have made this same mistake before. I'll try to remember next time. > However, the generated-configure.sh file is automatically created by > the autoconf framework from the rest of the files in > common/autoconf/*, so we cannot direcly modify it's contents - only > indirectly. The reason it's checked in, is that otherwise every user > would need to generate it before being able to run configure. In build > reviews, we usually either revert changes to generated-configure.sh > before posting a review to hide it (and re-generate it before > pushing), or we just ignore it during reviews. I should have done > that, or pointed out that it was not needed nor possible to review > when I cross-posted. I'm sorry. > >> >> L4076: # Verify that a given string represent a valid version >> number, and assing it to >> L4077: # a variable. >> Fixed two typos and reformat a bit: >> # Verify that a given string represents a valid version >> number, and >> # assigning it to a variable. > I'll put that fix in the source .m4 file instead. Thanks. >> >> L20262: as_fn_error $? "--with--version-string must have a >> value" "$LINENO" 5 >> The '--with--version...' part doesn't match previous usages >> where >> '--with-version...' is used > Yes, you're right. Erik pointed it out as well. It's a typo in the > error message. It's all over the place due to copied code. I'll fix it > in the source .m4 file. > > (As a side note, I have a half-finished follow-up patch that will > reduce the amount of code duplication, but it requires some framework > changes so it'll have to be a separate thing.) > >> >> L20275: # Unspecified numerical fields is interpreted as 0. >> Grammar: 'is interpreted' -> 'are interpreted' >> >> L20286: as_fn_error $? "Version string contains + but >> both 'BUILD' and 'OPT' is missing" "$LINENO" 5 >> Grammar: 'is missing' -> 'are missing' > Those darn English plural forms! Why can't you all do as we sensible > Swedes and get rid of them? ;-) > > (I'll fix) > >> >> L20388: username=`$ECHO "$USER" | $TR -d -c >> '[a-z][A-Z][0-9]'` >> This assumes that the "USER" variable is set. Should there >> be a check for "" and perhaps use "no_user_specified" or >> something like that? Perhaps the build setup always makes >> sure that "USER" is set to something. Don't know. > Hm. I think the worst thing that'll happen is that the username part > of the opt string gets empty. This part is basically copied right off > the old build, where we have relied on the $USER variable being > present for all the time with no problems, so I think it's quite safe > to assume. >> >> common/autoconf/jdk-options.m4 >> Don't know why the 'elliptic curve crypto implementation' support >> is relocated as part of this changeset, but ... > It was incorrectly placed not at the top indentation level, but in the > middle of the function definition where the old versioning code were > handled. (Which, mostly by chance, happens to work in autoconf, but is > really bad style). > >> make/Javadoc.gmk >> Did you mean to remove the 'clean' target? > Yep. Cleaning is done by the top-level Main.gmk makefile nowadays, > that's just some dead code. > >> >> hotspot/make/windows/makefiles/compile.make >> No changes in the frames view. >> Update: udiff view shows a blank line deleted at the end of the >> file. > > That's probably the result of some change going forth and then back > again during development. But then again, removing extra blank linkes > is not a bad thing. (jcheck unfortunately does not enforce any style > checks for make files :-( so they tend to detoriate over time.) > >> >> hotspot/src/share/vm/runtime/java.cpp >> L661: void JDK_Version::fully_initialize( >> L662: uint8_t major, uint8_t minor, uint8_t security, uint8_t >> update) { >> L663: // This is only called when current is less than 1.6 and >> we've gotten >> >> Since you're ripping out vestigial version support, I think this >> function can go away since this is version 9 and newer. Don't >> really >> know for sure, but based on that comment... > It probably can, yes. This and other core changes to the actual > .cpp/.java files will be addressed in a follow-up patch. >> >> hotspot/src/share/vm/runtime/java.hpp >> No comments. >> >> hotspot/src/share/vm/runtime/vmStructs.cpp >> L1240: please make the 'int' parameter align like the rest. > Are you okay with defering such a change to a follow-up patch? Yes. > It's likely the entire struct will need changes to be able to handle a > theoretically arbitrarily long version number. > >> >> hotspot/src/share/vm/runtime/vm_version.cpp >> L84: void Abstract_VM_Version::initialize() { >> L85: // FIXME: Initialization can probably be removed now. >> I agree, but that would entail also removing the >> _initialized and the uses of it... Follow on bug fix? > Definitely follow on. > >> jdk/src/java.base/share/classes/sun/misc/Version.java.template >> L149: * Returns the security version of the running JVM if >> it's 1.6 or newer >> This JavaDoc update is wrong, but it might not be important >> if sun.misc.Version class is going away. > I'm not sure if it's going to be replaced by, or just complemented by > java.util.Version, but in any case it will need a follow-up patch to > clean up this and other issues. >> langtools/src/java.compiler/share/classes/javax/lang/model/SourceVersion.java >> >> old L171: case "1.9": >> new L171: case "9": >> Should this logic support both versions? Will dropping >> "1.9" here prevent a pre-version changeset JVM from >> being dropped into a JDK for triage purposes? >> >> Granted we don't often triage 'javac' with different JVMs, >> but... > I'll defer that question to Kumar, who wrote that piece of code. My > guess is that when Hotspot express was dropped, the ability to use a > JVM from another JDK release bit rotteded away. > > /Magnus Dan From claes.redestad at oracle.com Tue Jun 9 13:26:27 2015 From: claes.redestad at oracle.com (Claes Redestad) Date: Tue, 09 Jun 2015 15:26:27 +0200 Subject: RFR: JDK-8085822 JEP 223: New Version-String Scheme (initial integration) In-Reply-To: <5576E633.9050503@oracle.com> References: <5571AD18.7080701@oracle.com> <557625C8.5050200@oracle.com> <5576E633.9050503@oracle.com> Message-ID: <5576E983.5020902@oracle.com> On 2015-06-09 15:12, Magnus Ihse Bursie wrote: >> langtools/src/java.compiler/share/classes/javax/lang/model/SourceVersion.java >> >> old L171: case "1.9": >> new L171: case "9": >> Should this logic support both versions? Will dropping >> "1.9" here prevent a pre-version changeset JVM from >> being dropped into a JDK for triage purposes? >> >> Granted we don't often triage 'javac' with different JVMs, >> but... > I'll defer that question to Kumar, who wrote that piece of code. My > guess is that when Hotspot express was dropped, the ability to use a > JVM from another JDK release bit rotteded away. > > /Magnus While we know there's no guarantee that swapping in an older VM will work, in the face of a regression in a promoted build we still routinely (automatically, even) swap out the VM with a recent VM to get a rough estimate of whether the regression was caused by a HotSpot or JDK/tools change, mostly since this currently saves us time in narrowing down the changes to bisect over/investigate. So, there's at least some value in not intentionally breaking build-to-build backwards compatibility, but we don't expect it to always work and wouldn't make much fuss about it breaking. If an extra case "1.9" is all it takes to make it work with last week's VM, however... /Claes From magnus.ihse.bursie at oracle.com Tue Jun 9 13:36:55 2015 From: magnus.ihse.bursie at oracle.com (Magnus Ihse Bursie) Date: Tue, 09 Jun 2015 15:36:55 +0200 Subject: RFR: JDK-8085822 JEP 223: New Version-String Scheme (initial integration) In-Reply-To: <5576E983.5020902@oracle.com> References: <5571AD18.7080701@oracle.com> <557625C8.5050200@oracle.com> <5576E633.9050503@oracle.com> <5576E983.5020902@oracle.com> Message-ID: <5576EBF7.40607@oracle.com> On 2015-06-09 15:26, Claes Redestad wrote: > On 2015-06-09 15:12, Magnus Ihse Bursie wrote: >>> langtools/src/java.compiler/share/classes/javax/lang/model/SourceVersion.java >>> >>> old L171: case "1.9": >>> new L171: case "9": >>> Should this logic support both versions? Will dropping >>> "1.9" here prevent a pre-version changeset JVM from >>> being dropped into a JDK for triage purposes? >>> >>> Granted we don't often triage 'javac' with different JVMs, >>> but... >> I'll defer that question to Kumar, who wrote that piece of code. My >> guess is that when Hotspot express was dropped, the ability to use a >> JVM from another JDK release bit rotteded away. >> >> /Magnus > > While we know there's no guarantee that swapping in an older VM will > work, in the face of a regression in a promoted build we still > routinely (automatically, even) swap out the VM with a recent VM to > get a rough estimate of whether the regression was caused by a HotSpot > or JDK/tools change, mostly since this currently saves us time in > narrowing down the changes to bisect over/investigate. So, there's at > least some value in not intentionally breaking build-to-build > backwards compatibility, but we don't expect it to always work and > wouldn't make much fuss about it breaking. If an extra case "1.9" is > all it takes to make it work with last week's VM, however... Actually, I think I messed up a bit there. I believe the real question here was not about mixing different versions of Hotspot and the JDK, but mixing different versions of javac and the JDK. I think. :) /Magnus From magnus.ihse.bursie at oracle.com Tue Jun 9 13:52:20 2015 From: magnus.ihse.bursie at oracle.com (Magnus Ihse Bursie) Date: Tue, 09 Jun 2015 15:52:20 +0200 Subject: RFR: JDK-8085822 JEP 223: New Version-String Scheme (initial integration) In-Reply-To: <5576E81B.8050703@oracle.com> References: <5571AD18.7080701@oracle.com> <557625C8.5050200@oracle.com> <5576E633.9050503@oracle.com> <5576E81B.8050703@oracle.com> Message-ID: <5576EF94.3010500@oracle.com> Here's an updated webrev, which fixes the typos that were pointed out by reviewers: http://cr.openjdk.java.net/~ihse/JDK-8085822-JEP-223-initial-patch/webrev.02/ And here's a (much simpler) delta webrev which shows just these changes: http://cr.openjdk.java.net/~ihse/JDK-8085822-JEP-223-initial-patch-delta-01/webrev.01/ /Magnus On 2015-06-09 15:20, Daniel D. Daugherty wrote: > On 6/9/15 7:12 AM, Magnus Ihse Bursie wrote: >> Hi Daniel, >> >> Thank you for your thorough review! > > This was my (failing) attempt at a "fast pass" review... :-) > > >> >> On 2015-06-09 01:31, Daniel D. Daugherty wrote: >>> > >>> http://cr.openjdk.java.net/~ihse/JDK-8085822-JEP-223-initial-patch/webrev.01 >>> >>> >>> General comment: Not all copyright years were updated. >>> General comment: It looks like support for the 'patch' value is not >>> completely >>> implemented through all the Makefiles. I didn't audit for this, >>> but it's >>> just my impression. >> >> You are basically correct. The makefiles all propagate the patch >> value where needed, but the actual source code changes in hotspot and >> jdk ignores the patch field as of now. This will be corrected in a >> later add-on patch, submitted by someone from the jdk and/or hotspot >> team rather than the build team. >> >>> >>> common/autoconf/generated-configure.sh >>> There are multiple Copyright notices in this file. Why? >> Oh dear, you've reviewed the generated file. :-& I'm really impressed >> by your effort! > > Ahhh... now that you say it... it sounds familiar... I may have > made this same mistake before. I'll try to remember next time. > > >> However, the generated-configure.sh file is automatically created by >> the autoconf framework from the rest of the files in >> common/autoconf/*, so we cannot direcly modify it's contents - only >> indirectly. The reason it's checked in, is that otherwise every user >> would need to generate it before being able to run configure. In >> build reviews, we usually either revert changes to >> generated-configure.sh before posting a review to hide it (and >> re-generate it before pushing), or we just ignore it during reviews. >> I should have done that, or pointed out that it was not needed nor >> possible to review when I cross-posted. I'm sorry. >> >>> >>> L4076: # Verify that a given string represent a valid version >>> number, and assing it to >>> L4077: # a variable. >>> Fixed two typos and reformat a bit: >>> # Verify that a given string represents a valid version >>> number, and >>> # assigning it to a variable. >> I'll put that fix in the source .m4 file instead. Thanks. >>> >>> L20262: as_fn_error $? "--with--version-string must have a >>> value" "$LINENO" 5 >>> The '--with--version...' part doesn't match previous usages >>> where >>> '--with-version...' is used >> Yes, you're right. Erik pointed it out as well. It's a typo in the >> error message. It's all over the place due to copied code. I'll fix >> it in the source .m4 file. >> >> (As a side note, I have a half-finished follow-up patch that will >> reduce the amount of code duplication, but it requires some framework >> changes so it'll have to be a separate thing.) >> >>> >>> L20275: # Unspecified numerical fields is interpreted as 0. >>> Grammar: 'is interpreted' -> 'are interpreted' >>> >>> L20286: as_fn_error $? "Version string contains + but >>> both 'BUILD' and 'OPT' is missing" "$LINENO" 5 >>> Grammar: 'is missing' -> 'are missing' >> Those darn English plural forms! Why can't you all do as we sensible >> Swedes and get rid of them? ;-) >> >> (I'll fix) >> >>> >>> L20388: username=`$ECHO "$USER" | $TR -d -c >>> '[a-z][A-Z][0-9]'` >>> This assumes that the "USER" variable is set. Should there >>> be a check for "" and perhaps use "no_user_specified" or >>> something like that? Perhaps the build setup always makes >>> sure that "USER" is set to something. Don't know. >> Hm. I think the worst thing that'll happen is that the username part >> of the opt string gets empty. This part is basically copied right off >> the old build, where we have relied on the $USER variable being >> present for all the time with no problems, so I think it's quite safe >> to assume. >>> >>> common/autoconf/jdk-options.m4 >>> Don't know why the 'elliptic curve crypto implementation' support >>> is relocated as part of this changeset, but ... >> It was incorrectly placed not at the top indentation level, but in >> the middle of the function definition where the old versioning code >> were handled. (Which, mostly by chance, happens to work in autoconf, >> but is really bad style). >> >>> make/Javadoc.gmk >>> Did you mean to remove the 'clean' target? >> Yep. Cleaning is done by the top-level Main.gmk makefile nowadays, >> that's just some dead code. >> >>> >>> hotspot/make/windows/makefiles/compile.make >>> No changes in the frames view. >>> Update: udiff view shows a blank line deleted at the end of the >>> file. >> >> That's probably the result of some change going forth and then back >> again during development. But then again, removing extra blank linkes >> is not a bad thing. (jcheck unfortunately does not enforce any style >> checks for make files :-( so they tend to detoriate over time.) >> >>> >>> hotspot/src/share/vm/runtime/java.cpp >>> L661: void JDK_Version::fully_initialize( >>> L662: uint8_t major, uint8_t minor, uint8_t security, >>> uint8_t update) { >>> L663: // This is only called when current is less than 1.6 and >>> we've gotten >>> >>> Since you're ripping out vestigial version support, I think >>> this >>> function can go away since this is version 9 and newer. >>> Don't really >>> know for sure, but based on that comment... >> It probably can, yes. This and other core changes to the actual >> .cpp/.java files will be addressed in a follow-up patch. >>> >>> hotspot/src/share/vm/runtime/java.hpp >>> No comments. >>> >>> hotspot/src/share/vm/runtime/vmStructs.cpp >>> L1240: please make the 'int' parameter align like the rest. >> Are you okay with defering such a change to a follow-up patch? > > Yes. > > >> It's likely the entire struct will need changes to be able to handle >> a theoretically arbitrarily long version number. >> >>> >>> hotspot/src/share/vm/runtime/vm_version.cpp >>> L84: void Abstract_VM_Version::initialize() { >>> L85: // FIXME: Initialization can probably be removed now. >>> I agree, but that would entail also removing the >>> _initialized and the uses of it... Follow on bug fix? >> Definitely follow on. >> >>> jdk/src/java.base/share/classes/sun/misc/Version.java.template >>> L149: * Returns the security version of the running JVM if >>> it's 1.6 or newer >>> This JavaDoc update is wrong, but it might not be important >>> if sun.misc.Version class is going away. >> I'm not sure if it's going to be replaced by, or just complemented by >> java.util.Version, but in any case it will need a follow-up patch to >> clean up this and other issues. >>> langtools/src/java.compiler/share/classes/javax/lang/model/SourceVersion.java >>> >>> old L171: case "1.9": >>> new L171: case "9": >>> Should this logic support both versions? Will dropping >>> "1.9" here prevent a pre-version changeset JVM from >>> being dropped into a JDK for triage purposes? >>> >>> Granted we don't often triage 'javac' with different JVMs, >>> but... >> I'll defer that question to Kumar, who wrote that piece of code. My >> guess is that when Hotspot express was dropped, the ability to use a >> JVM from another JDK release bit rotteded away. >> >> /Magnus > > Dan From kumar.x.srinivasan at oracle.com Tue Jun 9 14:25:45 2015 From: kumar.x.srinivasan at oracle.com (Kumar Srinivasan) Date: Tue, 09 Jun 2015 07:25:45 -0700 Subject: RFR: JDK-8085822 JEP 223: New Version-String Scheme (initial integration) In-Reply-To: <55767256.3030802@oracle.com> References: <5571AD18.7080701@oracle.com> <557625C8.5050200@oracle.com> <55767256.3030802@oracle.com> Message-ID: <5576F769.3060002@oracle.com> On 6/8/2015 9:57 PM, Jan Lahoda wrote: > On 9.6.2015 01:31, Daniel D. Daugherty wrote: >> > >> http://cr.openjdk.java.net/~ihse/JDK-8085822-JEP-223-initial-patch/webrev.01 >> >> langtools/src/java.compiler/share/classes/javax/lang/model/SourceVersion.java >> >> >> old L171: case "1.9": >> new L171: case "9": >> Should this logic support both versions? Will dropping >> "1.9" here prevent a pre-version changeset JVM from >> being dropped into a JDK for triage purposes? >> >> Granted we don't often triage 'javac' with different JVMs, >> but... >> > > +1 on keeping both "1.9" and "9" here. javac can be used independently > on the rest of JDK to some extent, so support for running it on older > builds of JDK 9 seems reasonable to me. (I wonder if current JDK 9 > javac should be prepared for the new version scheme in advance.) Yes, I think for the most part langtools seems to be working in the sandbox repo, with these changes, the remaining work is to align the version and fullversion of langtools/test to be compliant to the JDK version. Kumar > > Jan From xuelei.fan at oracle.com Tue Jun 9 14:57:48 2015 From: xuelei.fan at oracle.com (Xuelei Fan) Date: Tue, 09 Jun 2015 22:57:48 +0800 Subject: [9] Review request for 8085979: Make some DTLS feature functional tests work also for TLS protocol In-Reply-To: <5575AF10.1000908@oracle.com> References: <5575AF10.1000908@oracle.com> Message-ID: <5576FEEC.9040501@oracle.com> Looks fine to me. Nice port to TLS protocols. A very minior comment about the class name. TLSUnSupportedCiphersTest.java: replease "UnSupported" with "Unsupported". Thanks, Xuelei On 6/8/2015 11:04 PM, Konstantin Shefov wrote: > Hello, > > Please review distribution of some DTLS feature tests to TLS protocol. > Some DTLS tests may also be used to test the same functionality in TLS > protocol and its versions. > It is test only improvement. > > bug: https://bugs.openjdk.java.net/browse/JDK-8085979 > webrev: http://cr.openjdk.java.net/~kshefov/8085979/webrev.00/ > > > Thanks > -Konstantin From xuelei.fan at oracle.com Tue Jun 9 15:07:55 2015 From: xuelei.fan at oracle.com (Xuelei Fan) Date: Tue, 09 Jun 2015 23:07:55 +0800 Subject: [9] Review request for 8085979: Make some DTLS feature functional tests work also for TLS protocol In-Reply-To: <5576FEEC.9040501@oracle.com> References: <5575AF10.1000908@oracle.com> <5576FEEC.9040501@oracle.com> Message-ID: <5577014B.4080004@oracle.com> On 6/9/2015 10:57 PM, Xuelei Fan wrote: > Looks fine to me. Nice port to TLS protocols. > > A very minior comment about the class name. > TLSUnSupportedCiphersTest.java: > replease "UnSupported" with "Unsupported". > typo: replace "UnSupported" with "Unsupported". > Thanks, > Xuelei > > On 6/8/2015 11:04 PM, Konstantin Shefov wrote: >> Hello, >> >> Please review distribution of some DTLS feature tests to TLS protocol. >> Some DTLS tests may also be used to test the same functionality in TLS >> protocol and its versions. >> It is test only improvement. >> >> bug: https://bugs.openjdk.java.net/browse/JDK-8085979 >> webrev: http://cr.openjdk.java.net/~kshefov/8085979/webrev.00/ >> >> >> Thanks >> -Konstantin > From konstantin.shefov at oracle.com Tue Jun 9 15:31:23 2015 From: konstantin.shefov at oracle.com (Konstantin Shefov) Date: Tue, 09 Jun 2015 18:31:23 +0300 Subject: [9] Review request for 8085979: Make some DTLS feature functional tests work also for TLS protocol In-Reply-To: <5577014B.4080004@oracle.com> References: <5575AF10.1000908@oracle.com> <5576FEEC.9040501@oracle.com> <5577014B.4080004@oracle.com> Message-ID: <557706CB.30604@oracle.com> Xuelei, thanks for reviewing typo is corrected: http://cr.openjdk.java.net/~kshefov/8085979/webrev.01/ -Konstantin On 06/09/2015 06:07 PM, Xuelei Fan wrote: > On 6/9/2015 10:57 PM, Xuelei Fan wrote: >> Looks fine to me. Nice port to TLS protocols. >> >> A very minior comment about the class name. >> TLSUnSupportedCiphersTest.java: >> replease "UnSupported" with "Unsupported". >> > typo: replace "UnSupported" with "Unsupported". > >> Thanks, >> Xuelei >> >> On 6/8/2015 11:04 PM, Konstantin Shefov wrote: >>> Hello, >>> >>> Please review distribution of some DTLS feature tests to TLS protocol. >>> Some DTLS tests may also be used to test the same functionality in TLS >>> protocol and its versions. >>> It is test only improvement. >>> >>> bug: https://bugs.openjdk.java.net/browse/JDK-8085979 >>> webrev: http://cr.openjdk.java.net/~kshefov/8085979/webrev.00/ >>> >>> >>> Thanks >>> -Konstantin From xuelei.fan at oracle.com Tue Jun 9 15:47:46 2015 From: xuelei.fan at oracle.com (Xuelei Fan) Date: Tue, 09 Jun 2015 23:47:46 +0800 Subject: [9] Review request for 8085979: Make some DTLS feature functional tests work also for TLS protocol In-Reply-To: <557706CB.30604@oracle.com> References: <5575AF10.1000908@oracle.com> <5576FEEC.9040501@oracle.com> <5577014B.4080004@oracle.com> <557706CB.30604@oracle.com> Message-ID: <55770AA2.4060001@oracle.com> On 6/9/2015 11:31 PM, Konstantin Shefov wrote: > Xuelei, thanks for reviewing > > typo is corrected: > http://cr.openjdk.java.net/~kshefov/8085979/webrev.01/ > Why there are old "UnSupportedCiphersTest.java" files? If you have not committed the changeset, you can "hg forget" to undo a previous "hg add". As would avoid the unnecessary old file logs. Xuelei > -Konstantin > > On 06/09/2015 06:07 PM, Xuelei Fan wrote: >> On 6/9/2015 10:57 PM, Xuelei Fan wrote: >>> Looks fine to me. Nice port to TLS protocols. >>> >>> A very minior comment about the class name. >>> TLSUnSupportedCiphersTest.java: >>> replease "UnSupported" with "Unsupported". >>> >> typo: replace "UnSupported" with "Unsupported". >> >>> Thanks, >>> Xuelei >>> >>> On 6/8/2015 11:04 PM, Konstantin Shefov wrote: >>>> Hello, >>>> >>>> Please review distribution of some DTLS feature tests to TLS protocol. >>>> Some DTLS tests may also be used to test the same functionality in TLS >>>> protocol and its versions. >>>> It is test only improvement. >>>> >>>> bug: https://bugs.openjdk.java.net/browse/JDK-8085979 >>>> webrev: http://cr.openjdk.java.net/~kshefov/8085979/webrev.00/ >>>> >>>> >>>> Thanks >>>> -Konstantin > From xuelei.fan at oracle.com Tue Jun 9 16:00:02 2015 From: xuelei.fan at oracle.com (Xuelei Fan) Date: Wed, 10 Jun 2015 00:00:02 +0800 Subject: [9] Review request for 8085979: Make some DTLS feature functional tests work also for TLS protocol In-Reply-To: <55770AA2.4060001@oracle.com> References: <5575AF10.1000908@oracle.com> <5576FEEC.9040501@oracle.com> <5577014B.4080004@oracle.com> <557706CB.30604@oracle.com> <55770AA2.4060001@oracle.com> Message-ID: <55770D82.1050507@oracle.com> On 6/9/2015 11:47 PM, Xuelei Fan wrote: > On 6/9/2015 11:31 PM, Konstantin Shefov wrote: >> Xuelei, thanks for reviewing >> >> typo is corrected: >> http://cr.openjdk.java.net/~kshefov/8085979/webrev.01/ >> > Why there are old "UnSupportedCiphersTest.java" files? If you have not > committed the changeset, you can "hg forget" to undo a previous "hg > add". As would avoid the unnecessary old file logs. > Ooops, my fault. The old files are the previous DTLS tests. It's OK. Thanks for the update. Thanks, Xuelei > Xuelei > >> -Konstantin >> >> On 06/09/2015 06:07 PM, Xuelei Fan wrote: >>> On 6/9/2015 10:57 PM, Xuelei Fan wrote: >>>> Looks fine to me. Nice port to TLS protocols. >>>> >>>> A very minior comment about the class name. >>>> TLSUnSupportedCiphersTest.java: >>>> replease "UnSupported" with "Unsupported". >>>> >>> typo: replace "UnSupported" with "Unsupported". >>> >>>> Thanks, >>>> Xuelei >>>> >>>> On 6/8/2015 11:04 PM, Konstantin Shefov wrote: >>>>> Hello, >>>>> >>>>> Please review distribution of some DTLS feature tests to TLS protocol. >>>>> Some DTLS tests may also be used to test the same functionality in TLS >>>>> protocol and its versions. >>>>> It is test only improvement. >>>>> >>>>> bug: https://bugs.openjdk.java.net/browse/JDK-8085979 >>>>> webrev: http://cr.openjdk.java.net/~kshefov/8085979/webrev.00/ >>>>> >>>>> >>>>> Thanks >>>>> -Konstantin >> > From volker.simonis at gmail.com Tue Jun 9 17:09:04 2015 From: volker.simonis at gmail.com (Volker Simonis) Date: Tue, 9 Jun 2015 19:09:04 +0200 Subject: RFR(M): 8081674: EmptyStackException at startup if running with extended or unsupported charset In-Reply-To: <557626AA.6010008@oracle.com> References: <557626AA.6010008@oracle.com> Message-ID: Hi Mandy, thanks for looking into this. Uunfortunately your fix only helps to fix "java -version" :( Running even a minimal HelloWorld will still fail with the following stack trace which is still caused by the same EmptyStackException: Error: A JNI error has occurred, please check your installation and try again Exception in thread "main" java.lang.ExceptionInInitializerError at sun.misc.Unsafe.ensureClassInitialized(Native Method) at sun.misc.SharedSecrets.getJavaUtilZipFileAccess(SharedSecrets.java:158) at sun.misc.URLClassPath$JarLoader.(URLClassPath.java:765) at sun.misc.URLClassPath$3.run(URLClassPath.java:530) at sun.misc.URLClassPath$3.run(URLClassPath.java:520) at java.security.AccessController.doPrivileged(Native Method) at sun.misc.URLClassPath.getLoader(URLClassPath.java:519) at sun.misc.URLClassPath.getLoader(URLClassPath.java:492) at sun.misc.URLClassPath.getNextLoader(URLClassPath.java:457) at sun.misc.URLClassPath.access$100(URLClassPath.java:64) at sun.misc.URLClassPath$1.next(URLClassPath.java:239) at sun.misc.URLClassPath$1.hasMoreElements(URLClassPath.java:250) at java.net.URLClassLoader$3$1.run(URLClassLoader.java:601) at java.net.URLClassLoader$3$1.run(URLClassLoader.java:599) at java.security.AccessController.doPrivileged(Native Method) at java.net.URLClassLoader$3.next(URLClassLoader.java:598) at java.net.URLClassLoader$3.hasMoreElements(URLClassLoader.java:623) at sun.misc.CompoundEnumeration.next(CompoundEnumeration.java:45) at sun.misc.CompoundEnumeration.hasMoreElements(CompoundEnumeration.java:54) at sun.misc.CompoundEnumeration.next(CompoundEnumeration.java:45) at sun.misc.CompoundEnumeration.hasMoreElements(CompoundEnumeration.java:54) at java.util.ServiceLoader$LazyIterator.hasNextService(ServiceLoader.java:354) at java.util.ServiceLoader$LazyIterator.hasNext(ServiceLoader.java:393) at java.util.ServiceLoader$1.hasNext(ServiceLoader.java:474) at java.nio.charset.Charset$1.getNext(Charset.java:350) at java.nio.charset.Charset$1.hasNext(Charset.java:365) at java.nio.charset.Charset$2.run(Charset.java:410) at java.nio.charset.Charset$2.run(Charset.java:407) at java.security.AccessController.doPrivileged(Native Method) at java.nio.charset.Charset.lookupViaProviders(Charset.java:406) at java.nio.charset.Charset.lookup2(Charset.java:477) at java.nio.charset.Charset.lookup(Charset.java:464) at java.nio.charset.Charset.isSupported(Charset.java:505) at sun.launcher.LauncherHelper.makePlatformString(LauncherHelper.java:580) Caused by: java.util.EmptyStackException at java.util.Stack.peek(Stack.java:102) at java.lang.ClassLoader$NativeLibrary.getFromClass(ClassLoader.java:1759) at java.lang.ClassLoader$NativeLibrary.findBuiltinLib(Native Method) at java.lang.ClassLoader.loadLibrary0(ClassLoader.java:1870) at java.lang.ClassLoader.loadLibrary(ClassLoader.java:1843) at java.lang.Runtime.loadLibrary0(Runtime.java:870) at java.lang.System.loadLibrary(System.java:1122) at java.util.zip.ZipFile.(ZipFile.java:86) ... 34 more It's obvious that the way jni_FindClass is looking for the class context by calling the NativeLibrary::getFromClass method is hacky but I think that the proposed fix is the quite simple and non-intrusive. And we probably don't want a complete rework of this code for 8 anyway. So why not fix it the proposed way in 8 and 9 for now? That will still leave us time to come up with a better clean-up at least for 9. What do you think? Regards, Volker On Tue, Jun 9, 2015 at 1:35 AM, Mandy Chung wrote: > Hi Volker, > > Your patch reminds me the question I have about loading zip library. > > In JDK 9 (and earlier release), zip library is loaded by the VM during > startup (see ClassLoader::load_zip_library). I think loadLibrary("zip") is > no longer needed to be called from System::initializeSystemClass method and > instead it can be loaded by java.util.zip.ZipFile static initializer. > > Do you mind to try the patch (below)? That may be a simpler fix. > > I never like the way jni_FindClass to look for the class context by calling > the NativeLibrary::getFromClass method. I will have to look deeper other > alternative to clean that up. If taking out loadLibrary("zip") resolves > your issue, this will give us time to come up with a better clean-up in the > future. > > Mandy > [1] https://bugs.openjdk.java.net/browse/JDK-4429040 > > diff --git a/src/share/classes/java/lang/System.java > b/src/share/classes/java/lang/System.java > --- a/src/share/classes/java/lang/System.java > +++ b/src/share/classes/java/lang/System.java > @@ -1192,10 +1192,6 @@ > setOut0(newPrintStream(fdOut, > props.getProperty("sun.stdout.encoding"))); > setErr0(newPrintStream(fdErr, > props.getProperty("sun.stderr.encoding"))); > > - // Load the zip library now in order to keep java.util.zip.ZipFile > - // from trying to use itself to load this library later. > - loadLibrary("zip"); > - > // Setup Java signal handlers for HUP, TERM, and INT (where > available). > Terminator.setup(); > > diff --git a/src/share/classes/java/util/zip/ZipFile.java > b/src/share/classes/java/util/zip/ZipFile.java > --- a/src/share/classes/java/util/zip/ZipFile.java > +++ b/src/share/classes/java/util/zip/ZipFile.java > @@ -83,6 +83,7 @@ > > static { > /* Zip library is loaded from System.initializeSystemClass */ > + System.loadLibrary("zip"); > initIDs(); > > } > > > > On 06/08/2015 07:23 AM, Volker Simonis wrote: >> >> Hi, >> >> can I please get a review at least for the part of this fix which is >> common for jdk8 and jdk9. It's only a few lines in >> src/share/vm/prims/jni.cpp for the hotspot repository and one line >> src/java.base/share/classes/java/lang/ClassLoader.java for the jdk >> repository. >> >> Thanks, >> Volker >> >> >> On Tue, Jun 2, 2015 at 6:12 PM, Volker Simonis >> wrote: >>> >>> Hi, >>> >>> can I please have review (and a sponsor) for these changes: >>> >>> https://bugs.openjdk.java.net/browse/JDK-8081674 >>> http://cr.openjdk.java.net/~simonis/webrevs/2015/8081674.jdk >>> http://cr.openjdk.java.net/~simonis/webrevs/2015/8081674.hs >>> >>> Please notice that the fix requires synchronous changes in the jdk and >>> the >>> hotspot forest. >>> >>> The changes themselves are by far not that big as this explanation but I >>> found the problem to be quite intricate so I tried to explain it as good >>> as >>> I could. I'd suggest to read the HTML-formatted explanation in the webrev >>> although the content is equal to the one in this mail: >>> >>> Using an unsupported character encoding (e.g. vi_VN.TCVN on Linux) >>> results >>> in an immediate VM failure with jdk 8 and 9: >>> >>> export LANG=vi_VN.TCVN >>> java -version >>> Error occurred during initialization of VM >>> java.util.EmptyStackException >>> at java.util.Stack.peek(Stack.java:102) >>> at >>> java.lang.ClassLoader$NativeLibrary.getFromClass(ClassLoader.java:1751) >>> at java.lang.ClassLoader$NativeLibrary.findBuiltinLib(Native >>> Method) >>> at java.lang.ClassLoader.loadLibrary0(ClassLoader.java:1862) >>> at java.lang.ClassLoader.loadLibrary(ClassLoader.java:1835) >>> at java.lang.Runtime.loadLibrary0(Runtime.java:870) >>> at java.lang.System.loadLibrary(System.java:1119) >>> at java.lang.System.initializeSystemClass(System.java:1194) >>> >>> This is a consequence of "8005716: Enhance JNI specification to allow >>> support of static JNI libraries in Embedded JREs". >>> >>> With jdk 9 we get this error even if we're running with a supported >>> charset >>> which is in the ExtendedCharsets (as opposed to being in the >>> StandardCharsets) class which is a consequence of delegating the loading >>> of >>> the ExtendedCharsets class to the ServiceLoader in jdk 9. >>> >>> export LANG=eo.iso-8859-3 >>> output-jdk9/images/jdk/bin/java -version >>> Error occurred during initialization of VM >>> java.util.EmptyStackException >>> at java.util.Stack.peek(Stack.java:102) >>> at >>> java.lang.ClassLoader$NativeLibrary.getFromClass(ClassLoader.java:1737) >>> at java.lang.ClassLoader$NativeLibrary.findBuiltinLib(Native >>> Method) >>> at java.lang.ClassLoader.loadLibrary0(ClassLoader.java:1866) >>> at java.lang.ClassLoader.loadLibrary(ClassLoader.java:1840) >>> at java.lang.Runtime.loadLibrary0(Runtime.java:874) >>> at java.lang.System.loadLibrary(System.java:1111) >>> at java.lang.System.initializeSystemClass(System.java:1186) >>> >>> Here's why the exception happens for an unsupported charset (see the >>> mixed >>> stack trace below for the full details): >>> >>> java.lang.System.loadLibrary() wants to load libzip.so. It calls >>> java.lang.Runtime.loadLibrary0() which at the very beginning calls the >>> native method ClassLoader$NativeLibrary.findBuiltinLib() which checks if >>> the >>> corresponding library is already statically linked into the VM >>> (introduced >>> by 8005716). >>> Java_java_lang_ClassLoader_00024NativeLibrary_findBuiltinLib(), >>> the native implementation of findBuiltinLib() in Classloader.c calls >>> GetStringPlatformChars() to convert the library name into the native >>> platform encoding. GetStringPlatformChars() calls the helper function >>> jnuEncodingSupported() to check if the platform encoding which is stored >>> in >>> the property "sun.jnu.encoding" is supported by Java. >>> jnuEncodingSupported() >>> is implemented as follows: >>> >>> static jboolean isJNUEncodingSupported = JNI_FALSE; >>> static jboolean jnuEncodingSupported(JNIEnv *env) { >>> jboolean exe; >>> if (isJNUEncodingSupported == JNI_TRUE) { >>> return JNI_TRUE; >>> } >>> isJNUEncodingSupported = (jboolean) JNU_CallStaticMethodByName ( >>> env, &exe, >>> "java/nio/charset/Charset", >>> "isSupported", >>> "(Ljava/lang/String;)Z", >>> jnuEncoding).z; >>> return isJNUEncodingSupported; >>> } >>> >>> Once the function finds that the platform encoding is supported (by >>> calling >>> java.nio.charset.Charset.isSupported()) it caches this value and always >>> returns it on following calls. However if the platform encoding is not >>> supported, it ALWAYS calls java.nio.charset.Charset.isSupported() an >>> every >>> subsequent invocation. >>> >>> In order to call the Java method Charset.isSupported() (in >>> JNU_CallStaticMethodByName() in file jni_util.c), we have to call >>> jni_FindClass() to convert the symbolic class name >>> "java.nio.charset.Charset" into a class reference. >>> >>> But unfortunately, jni_FindClass() (from jni.cpp in libjvm.so) has a >>> special >>> handling if called from java.lang.ClassLoader$NativeLibrary to ensure >>> that >>> JNI_OnLoad/JNI_OnUnload are executed in the correct class context: >>> >>> instanceKlassHandle k (THREAD, thread->security_get_caller_class(0)); >>> if (k.not_null()) { >>> loader = Handle(THREAD, k->class_loader()); >>> // Special handling to make sure JNI_OnLoad and JNI_OnUnload are >>> executed >>> // in the correct class context. >>> if (loader.is_null() && >>> k->name() == vmSymbols::java_lang_ClassLoader_NativeLibrary()) { >>> JavaValue result(T_OBJECT); >>> JavaCalls::call_static(&result, k, >>> vmSymbols::getFromClass_name(), >>> vmSymbols::void_class_signature(), >>> thread); >>> if (HAS_PENDING_EXCEPTION) { >>> Handle ex(thread, thread->pending_exception()); >>> CLEAR_PENDING_EXCEPTION; >>> THROW_HANDLE_0(ex); >>> } >>> >>> So if that's the case and jni_FindClass() was reallycalled from >>> ClassLoader$NativeLibrary, then jni_FindClass() calles >>> ClassLoader$NativeLibrary().getFromClass() to find out the corresponding >>> context class which is supposed to be saved there in a field of type >>> java.util.Stack named "nativeLibraryContext": >>> >>> // Invoked in the VM to determine the context class in >>> // JNI_Load/JNI_Unload >>> static Class getFromClass() { >>> return ClassLoader.nativeLibraryContext.peek().fromClass; >>> } >>> >>> Unfortunately, "nativeLibraryContext" doesn't contain any entry at this >>> point and the invocation of Stack.peek() will throw the exception shown >>> before. In general, the "nativeLibraryContext" stack will be filled later >>> on >>> in Runtime.loadLibrary0() like this: >>> >>> NativeLibrary lib = new NativeLibrary(fromClass, name, isBuiltin); >>> nativeLibraryContext.push(lib); >>> try { >>> lib.load(name, isBuiltin); >>> } finally { >>> nativeLibraryContext.pop(); >>> } >>> >>> such that it always contains at least one element later when >>> jni_FindClass() >>> will be invoked. >>> >>> So in summary, the problem is that the implementors of 8005716 didn't >>> took >>> into account that calling ClassLoader$NativeLibrary.findBuiltinLib() may >>> trigger a call to jni_FindClass() if we are running on a system with an >>> unsupported character encoding. >>> >>> I'd suggest the following fix for this problem: >>> >>> Change ClassLoader$NativeLibrary().getFromClass() to return null if the >>> stack is empty instead of throwing an exception: >>> >>> static Class getFromClass() { >>> return ClassLoader.nativeLibraryContext.empty() ? >>> null : ClassLoader.nativeLibraryContext.peek().fromClass; >>> } >>> >>> Unfortunately this also requires a HotSpot change in jni_FindClass() in >>> order to properly handle the new 'null' return value: >>> >>> if (k.not_null()) { >>> loader = Handle(THREAD, k->class_loader()); >>> // Special handling to make sure JNI_OnLoad and JNI_OnUnload are >>> executed >>> // in the correct class context. >>> if (loader.is_null() && >>> k->name() == vmSymbols::java_lang_ClassLoader_NativeLibrary()) { >>> JavaValue result(T_OBJECT); >>> JavaCalls::call_static(&result, k, >>> vmSymbols::getFromClass_name(), >>> vmSymbols::void_class_signature(), >>> thread); >>> if (HAS_PENDING_EXCEPTION) { >>> Handle ex(thread, thread->pending_exception()); >>> CLEAR_PENDING_EXCEPTION; >>> THROW_HANDLE_0(ex); >>> } >>> oop mirror = (oop) result.get_jobject(); >>> if (oopDesc::is_null(mirror)) { >>> loader = Handle(THREAD, SystemDictionary::java_system_loader()); >>> } else { >>> loader = Handle(THREAD, >>> >>> InstanceKlass::cast(java_lang_Class::as_Klass(mirror))->class_loader()); >>> protection_domain = Handle(THREAD, >>> >>> >>> InstanceKlass::cast(java_lang_Class::as_Klass(mirror))->protection_domain()); >>> } >>> } >>> } else { >>> // We call ClassLoader.getSystemClassLoader to obtain the system >>> class >>> loader. >>> loader = Handle(THREAD, SystemDictionary::java_system_loader()); >>> } >>> >>> These changes are sufficient to solve the problem in Java 8. >>> Unfortunately, >>> that's still not enough in Java 9 because there the loading of the >>> extended >>> charsets has been delegated to ServiceLoader. But ServiceLoader calls >>> ClassLoader.getBootstrapResources() which calls >>> sun.misc.Launcher.getBootstrapClassPath(). This leads to another problem >>> during class initialization of sun.misc.Launcher if running on an >>> unsupported locale. >>> >>> The first thing done in sun.misc.Launcher. is the initialisation >>> of >>> the bootstrap URLClassPath in the Launcher. However this initialisation >>> will >>> eventually call Charset.isSupported() and if we are running on an >>> unsupported locale this will inevitably end in another recursive call to >>> ServiceLoader. But as explained below, ServiceLoader will query the >>> Launcher's bootstrap URLClassPath which will be still uninitialized at >>> that >>> point. >>> >>> So we'll have to additionally guard guard against this situation on JDK 9 >>> like this: >>> >>> private static Charset lookupExtendedCharset(String charsetName) { >>> if (!sun.misc.VM.isBooted() || // see >>> lookupViaProviders() >>> sun.misc.Launcher.getBootstrapClassPath() == null) >>> return null; >>> >>> This fixes the crashes, but still at the price of not having the extended >>> charsets available during initialization until >>> Launcher.getBootstrapClassPath is set up properly. This may be still a >>> problem if the jdk is installed in a directory which contains characters >>> specific to an extended encoding or if we have such characters in the >>> command line arguments. >>> >>> Thank you and best regards, >>> Volker >>> >>> >>> Mixed stack trace of the initial EmptyStackException for unsupported >>> charsets described before: >>> >>> Native frames: (J=compiled Java code, j=interpreted, Vv=VM code, C=native >>> code) >>> j java.util.Stack.peek()Ljava/lang/Object;+1 >>> j java.lang.ClassLoader$NativeLibrary.getFromClass()Ljava/lang/Class;+3 >>> v ~StubRoutines::call_stub >>> V [libjvm.so+0x9d279a] JavaCalls::call_helper(JavaValue*, >>> methodHandle*, >>> JavaCallArguments*, Thread*)+0x6b4 >>> V [libjvm.so+0xcad591] os::os_exception_wrapper(void (*)(JavaValue*, >>> methodHandle*, JavaCallArguments*, Thread*), JavaValue*, methodHandle*, >>> JavaCallArguments*, Thread*)+0x45 >>> V [libjvm.so+0x9d20cf] JavaCalls::call(JavaValue*, methodHandle, >>> JavaCallArguments*, Thread*)+0x8b >>> V [libjvm.so+0x9d1d3b] JavaCalls::call_static(JavaValue*, KlassHandle, >>> Symbol*, Symbol*, JavaCallArguments*, Thread*)+0x139 >>> V [libjvm.so+0x9d1e3f] JavaCalls::call_static(JavaValue*, KlassHandle, >>> Symbol*, Symbol*, Thread*)+0x9d >>> V [libjvm.so+0x9e6588] jni_FindClass+0x428 >>> C [libjava.so+0x20208] JNU_CallStaticMethodByName+0xff >>> C [libjava.so+0x21cae] jnuEncodingSupported+0x61 >>> C [libjava.so+0x22125] JNU_GetStringPlatformChars+0x125 >>> C [libjava.so+0xedcd] >>> Java_java_lang_ClassLoader_00024NativeLibrary_findBuiltinLib+0x8b >>> j >>> >>> java.lang.ClassLoader$NativeLibrary.findBuiltinLib(Ljava/lang/String;)Ljava/lang/String;+0 >>> j java.lang.ClassLoader.loadLibrary0(Ljava/lang/Class;Ljava/io/File;)Z+4 >>> j >>> >>> java.lang.ClassLoader.loadLibrary(Ljava/lang/Class;Ljava/lang/String;Z)V+228 >>> j >>> java.lang.Runtime.loadLibrary0(Ljava/lang/Class;Ljava/lang/String;)V+54 >>> j java.lang.System.loadLibrary(Ljava/lang/String;)V+7 >>> j java.lang.System.initializeSystemClass()V+113 >>> v ~StubRoutines::call_stub >>> V [libjvm.so+0x9d279a] JavaCalls::call_helper(JavaValue*, >>> methodHandle*, >>> JavaCallArguments*, Thread*)+0x6b4 >>> V [libjvm.so+0xcad591] os::os_exception_wrapper(void (*)(JavaValue*, >>> methodHandle*, JavaCallArguments*, Thread*), JavaValue*, methodHandle*, >>> JavaCallArguments*, Thread*)+0x45 >>> V [libjvm.so+0x9d20cf] JavaCalls::call(JavaValue*, methodHandle, >>> JavaCallArguments*, Thread*)+0x8b >>> V [libjvm.so+0x9d1d3b] JavaCalls::call_static(JavaValue*, KlassHandle, >>> Symbol*, Symbol*, JavaCallArguments*, Thread*)+0x139 >>> V [libjvm.so+0x9d1e3f] JavaCalls::call_static(JavaValue*, KlassHandle, >>> Symbol*, Symbol*, Thread*)+0x9d >>> V [libjvm.so+0xe3cceb] call_initializeSystemClass(Thread*)+0xb0 >>> V [libjvm.so+0xe44444] >>> Threads::initialize_java_lang_classes(JavaThread*, >>> Thread*)+0x21a >>> V [libjvm.so+0xe44b12] Threads::create_vm(JavaVMInitArgs*, bool*)+0x4a6 >>> V [libjvm.so+0xa19bd7] JNI_CreateJavaVM+0xc7 >>> C [libjli.so+0xa520] InitializeJVM+0x154 >>> C [libjli.so+0x8024] JavaMain+0xcc >>> >>> > From mandy.chung at oracle.com Tue Jun 9 18:03:02 2015 From: mandy.chung at oracle.com (Mandy Chung) Date: Tue, 09 Jun 2015 11:03:02 -0700 Subject: RFR(M): 8081674: EmptyStackException at startup if running with extended or unsupported charset In-Reply-To: References: <557626AA.6010008@oracle.com> Message-ID: <55772A56.2030308@oracle.com> Does my patch work fine on 8u? If it works fine, I prefer to get that simple fix in 8u and take the time to have a better fix in 9 (jdk9 is still under development and I have assumed that it's not a show stopper to you. Let me know if it blocks you). A question to Sherman - do we have adequate tests to verify the move of System.loadLibrary("zip") from system initialization to ZipFile initialization for 8u? Mandy On 06/09/2015 10:09 AM, Volker Simonis wrote: > Hi Mandy, > > thanks for looking into this. > Uunfortunately your fix only helps to fix "java -version" :( > > Running even a minimal HelloWorld will still fail with the following > stack trace which is still caused by the same EmptyStackException: > > Error: A JNI error has occurred, please check your installation and try again > Exception in thread "main" java.lang.ExceptionInInitializerError > at sun.misc.Unsafe.ensureClassInitialized(Native Method) > at sun.misc.SharedSecrets.getJavaUtilZipFileAccess(SharedSecrets.java:158) > at sun.misc.URLClassPath$JarLoader.(URLClassPath.java:765) > at sun.misc.URLClassPath$3.run(URLClassPath.java:530) > at sun.misc.URLClassPath$3.run(URLClassPath.java:520) > at java.security.AccessController.doPrivileged(Native Method) > at sun.misc.URLClassPath.getLoader(URLClassPath.java:519) > at sun.misc.URLClassPath.getLoader(URLClassPath.java:492) > at sun.misc.URLClassPath.getNextLoader(URLClassPath.java:457) > at sun.misc.URLClassPath.access$100(URLClassPath.java:64) > at sun.misc.URLClassPath$1.next(URLClassPath.java:239) > at sun.misc.URLClassPath$1.hasMoreElements(URLClassPath.java:250) > at java.net.URLClassLoader$3$1.run(URLClassLoader.java:601) > at java.net.URLClassLoader$3$1.run(URLClassLoader.java:599) > at java.security.AccessController.doPrivileged(Native Method) > at java.net.URLClassLoader$3.next(URLClassLoader.java:598) > at java.net.URLClassLoader$3.hasMoreElements(URLClassLoader.java:623) > at sun.misc.CompoundEnumeration.next(CompoundEnumeration.java:45) > at sun.misc.CompoundEnumeration.hasMoreElements(CompoundEnumeration.java:54) > at sun.misc.CompoundEnumeration.next(CompoundEnumeration.java:45) > at sun.misc.CompoundEnumeration.hasMoreElements(CompoundEnumeration.java:54) > at java.util.ServiceLoader$LazyIterator.hasNextService(ServiceLoader.java:354) > at java.util.ServiceLoader$LazyIterator.hasNext(ServiceLoader.java:393) > at java.util.ServiceLoader$1.hasNext(ServiceLoader.java:474) > at java.nio.charset.Charset$1.getNext(Charset.java:350) > at java.nio.charset.Charset$1.hasNext(Charset.java:365) > at java.nio.charset.Charset$2.run(Charset.java:410) > at java.nio.charset.Charset$2.run(Charset.java:407) > at java.security.AccessController.doPrivileged(Native Method) > at java.nio.charset.Charset.lookupViaProviders(Charset.java:406) > at java.nio.charset.Charset.lookup2(Charset.java:477) > at java.nio.charset.Charset.lookup(Charset.java:464) > at java.nio.charset.Charset.isSupported(Charset.java:505) > at sun.launcher.LauncherHelper.makePlatformString(LauncherHelper.java:580) > Caused by: java.util.EmptyStackException > at java.util.Stack.peek(Stack.java:102) > at java.lang.ClassLoader$NativeLibrary.getFromClass(ClassLoader.java:1759) > at java.lang.ClassLoader$NativeLibrary.findBuiltinLib(Native Method) > at java.lang.ClassLoader.loadLibrary0(ClassLoader.java:1870) > at java.lang.ClassLoader.loadLibrary(ClassLoader.java:1843) > at java.lang.Runtime.loadLibrary0(Runtime.java:870) > at java.lang.System.loadLibrary(System.java:1122) > at java.util.zip.ZipFile.(ZipFile.java:86) > ... 34 more > > It's obvious that the way jni_FindClass is looking for the class > context by calling the NativeLibrary::getFromClass method is hacky but > I think that the proposed fix is the quite simple and non-intrusive. > And we probably don't want a complete rework of this code for 8 > anyway. So why not fix it the proposed way in 8 and 9 for now? That > will still leave us time to come up with a better clean-up at least > for 9. > > What do you think? > > Regards, > Volker > > On Tue, Jun 9, 2015 at 1:35 AM, Mandy Chung wrote: >> Hi Volker, >> >> Your patch reminds me the question I have about loading zip library. >> >> In JDK 9 (and earlier release), zip library is loaded by the VM during >> startup (see ClassLoader::load_zip_library). I think loadLibrary("zip") is >> no longer needed to be called from System::initializeSystemClass method and >> instead it can be loaded by java.util.zip.ZipFile static initializer. >> >> Do you mind to try the patch (below)? That may be a simpler fix. >> >> I never like the way jni_FindClass to look for the class context by calling >> the NativeLibrary::getFromClass method. I will have to look deeper other >> alternative to clean that up. If taking out loadLibrary("zip") resolves >> your issue, this will give us time to come up with a better clean-up in the >> future. >> >> Mandy >> [1] https://bugs.openjdk.java.net/browse/JDK-4429040 >> >> diff --git a/src/share/classes/java/lang/System.java >> b/src/share/classes/java/lang/System.java >> --- a/src/share/classes/java/lang/System.java >> +++ b/src/share/classes/java/lang/System.java >> @@ -1192,10 +1192,6 @@ >> setOut0(newPrintStream(fdOut, >> props.getProperty("sun.stdout.encoding"))); >> setErr0(newPrintStream(fdErr, >> props.getProperty("sun.stderr.encoding"))); >> >> - // Load the zip library now in order to keep java.util.zip.ZipFile >> - // from trying to use itself to load this library later. >> - loadLibrary("zip"); >> - >> // Setup Java signal handlers for HUP, TERM, and INT (where >> available). >> Terminator.setup(); >> >> diff --git a/src/share/classes/java/util/zip/ZipFile.java >> b/src/share/classes/java/util/zip/ZipFile.java >> --- a/src/share/classes/java/util/zip/ZipFile.java >> +++ b/src/share/classes/java/util/zip/ZipFile.java >> @@ -83,6 +83,7 @@ >> >> static { >> /* Zip library is loaded from System.initializeSystemClass */ >> + System.loadLibrary("zip"); >> initIDs(); >> >> } >> >> >> >> On 06/08/2015 07:23 AM, Volker Simonis wrote: >>> Hi, >>> >>> can I please get a review at least for the part of this fix which is >>> common for jdk8 and jdk9. It's only a few lines in >>> src/share/vm/prims/jni.cpp for the hotspot repository and one line >>> src/java.base/share/classes/java/lang/ClassLoader.java for the jdk >>> repository. >>> >>> Thanks, >>> Volker >>> >>> >>> On Tue, Jun 2, 2015 at 6:12 PM, Volker Simonis >>> wrote: >>>> Hi, >>>> >>>> can I please have review (and a sponsor) for these changes: >>>> >>>> https://bugs.openjdk.java.net/browse/JDK-8081674 >>>> http://cr.openjdk.java.net/~simonis/webrevs/2015/8081674.jdk >>>> http://cr.openjdk.java.net/~simonis/webrevs/2015/8081674.hs >>>> >>>> Please notice that the fix requires synchronous changes in the jdk and >>>> the >>>> hotspot forest. >>>> >>>> The changes themselves are by far not that big as this explanation but I >>>> found the problem to be quite intricate so I tried to explain it as good >>>> as >>>> I could. I'd suggest to read the HTML-formatted explanation in the webrev >>>> although the content is equal to the one in this mail: >>>> >>>> Using an unsupported character encoding (e.g. vi_VN.TCVN on Linux) >>>> results >>>> in an immediate VM failure with jdk 8 and 9: >>>> >>>> export LANG=vi_VN.TCVN >>>> java -version >>>> Error occurred during initialization of VM >>>> java.util.EmptyStackException >>>> at java.util.Stack.peek(Stack.java:102) >>>> at >>>> java.lang.ClassLoader$NativeLibrary.getFromClass(ClassLoader.java:1751) >>>> at java.lang.ClassLoader$NativeLibrary.findBuiltinLib(Native >>>> Method) >>>> at java.lang.ClassLoader.loadLibrary0(ClassLoader.java:1862) >>>> at java.lang.ClassLoader.loadLibrary(ClassLoader.java:1835) >>>> at java.lang.Runtime.loadLibrary0(Runtime.java:870) >>>> at java.lang.System.loadLibrary(System.java:1119) >>>> at java.lang.System.initializeSystemClass(System.java:1194) >>>> >>>> This is a consequence of "8005716: Enhance JNI specification to allow >>>> support of static JNI libraries in Embedded JREs". >>>> >>>> With jdk 9 we get this error even if we're running with a supported >>>> charset >>>> which is in the ExtendedCharsets (as opposed to being in the >>>> StandardCharsets) class which is a consequence of delegating the loading >>>> of >>>> the ExtendedCharsets class to the ServiceLoader in jdk 9. >>>> >>>> export LANG=eo.iso-8859-3 >>>> output-jdk9/images/jdk/bin/java -version >>>> Error occurred during initialization of VM >>>> java.util.EmptyStackException >>>> at java.util.Stack.peek(Stack.java:102) >>>> at >>>> java.lang.ClassLoader$NativeLibrary.getFromClass(ClassLoader.java:1737) >>>> at java.lang.ClassLoader$NativeLibrary.findBuiltinLib(Native >>>> Method) >>>> at java.lang.ClassLoader.loadLibrary0(ClassLoader.java:1866) >>>> at java.lang.ClassLoader.loadLibrary(ClassLoader.java:1840) >>>> at java.lang.Runtime.loadLibrary0(Runtime.java:874) >>>> at java.lang.System.loadLibrary(System.java:1111) >>>> at java.lang.System.initializeSystemClass(System.java:1186) >>>> >>>> Here's why the exception happens for an unsupported charset (see the >>>> mixed >>>> stack trace below for the full details): >>>> >>>> java.lang.System.loadLibrary() wants to load libzip.so. It calls >>>> java.lang.Runtime.loadLibrary0() which at the very beginning calls the >>>> native method ClassLoader$NativeLibrary.findBuiltinLib() which checks if >>>> the >>>> corresponding library is already statically linked into the VM >>>> (introduced >>>> by 8005716). >>>> Java_java_lang_ClassLoader_00024NativeLibrary_findBuiltinLib(), >>>> the native implementation of findBuiltinLib() in Classloader.c calls >>>> GetStringPlatformChars() to convert the library name into the native >>>> platform encoding. GetStringPlatformChars() calls the helper function >>>> jnuEncodingSupported() to check if the platform encoding which is stored >>>> in >>>> the property "sun.jnu.encoding" is supported by Java. >>>> jnuEncodingSupported() >>>> is implemented as follows: >>>> >>>> static jboolean isJNUEncodingSupported = JNI_FALSE; >>>> static jboolean jnuEncodingSupported(JNIEnv *env) { >>>> jboolean exe; >>>> if (isJNUEncodingSupported == JNI_TRUE) { >>>> return JNI_TRUE; >>>> } >>>> isJNUEncodingSupported = (jboolean) JNU_CallStaticMethodByName ( >>>> env, &exe, >>>> "java/nio/charset/Charset", >>>> "isSupported", >>>> "(Ljava/lang/String;)Z", >>>> jnuEncoding).z; >>>> return isJNUEncodingSupported; >>>> } >>>> >>>> Once the function finds that the platform encoding is supported (by >>>> calling >>>> java.nio.charset.Charset.isSupported()) it caches this value and always >>>> returns it on following calls. However if the platform encoding is not >>>> supported, it ALWAYS calls java.nio.charset.Charset.isSupported() an >>>> every >>>> subsequent invocation. >>>> >>>> In order to call the Java method Charset.isSupported() (in >>>> JNU_CallStaticMethodByName() in file jni_util.c), we have to call >>>> jni_FindClass() to convert the symbolic class name >>>> "java.nio.charset.Charset" into a class reference. >>>> >>>> But unfortunately, jni_FindClass() (from jni.cpp in libjvm.so) has a >>>> special >>>> handling if called from java.lang.ClassLoader$NativeLibrary to ensure >>>> that >>>> JNI_OnLoad/JNI_OnUnload are executed in the correct class context: >>>> >>>> instanceKlassHandle k (THREAD, thread->security_get_caller_class(0)); >>>> if (k.not_null()) { >>>> loader = Handle(THREAD, k->class_loader()); >>>> // Special handling to make sure JNI_OnLoad and JNI_OnUnload are >>>> executed >>>> // in the correct class context. >>>> if (loader.is_null() && >>>> k->name() == vmSymbols::java_lang_ClassLoader_NativeLibrary()) { >>>> JavaValue result(T_OBJECT); >>>> JavaCalls::call_static(&result, k, >>>> vmSymbols::getFromClass_name(), >>>> vmSymbols::void_class_signature(), >>>> thread); >>>> if (HAS_PENDING_EXCEPTION) { >>>> Handle ex(thread, thread->pending_exception()); >>>> CLEAR_PENDING_EXCEPTION; >>>> THROW_HANDLE_0(ex); >>>> } >>>> >>>> So if that's the case and jni_FindClass() was reallycalled from >>>> ClassLoader$NativeLibrary, then jni_FindClass() calles >>>> ClassLoader$NativeLibrary().getFromClass() to find out the corresponding >>>> context class which is supposed to be saved there in a field of type >>>> java.util.Stack named "nativeLibraryContext": >>>> >>>> // Invoked in the VM to determine the context class in >>>> // JNI_Load/JNI_Unload >>>> static Class getFromClass() { >>>> return ClassLoader.nativeLibraryContext.peek().fromClass; >>>> } >>>> >>>> Unfortunately, "nativeLibraryContext" doesn't contain any entry at this >>>> point and the invocation of Stack.peek() will throw the exception shown >>>> before. In general, the "nativeLibraryContext" stack will be filled later >>>> on >>>> in Runtime.loadLibrary0() like this: >>>> >>>> NativeLibrary lib = new NativeLibrary(fromClass, name, isBuiltin); >>>> nativeLibraryContext.push(lib); >>>> try { >>>> lib.load(name, isBuiltin); >>>> } finally { >>>> nativeLibraryContext.pop(); >>>> } >>>> >>>> such that it always contains at least one element later when >>>> jni_FindClass() >>>> will be invoked. >>>> >>>> So in summary, the problem is that the implementors of 8005716 didn't >>>> took >>>> into account that calling ClassLoader$NativeLibrary.findBuiltinLib() may >>>> trigger a call to jni_FindClass() if we are running on a system with an >>>> unsupported character encoding. >>>> >>>> I'd suggest the following fix for this problem: >>>> >>>> Change ClassLoader$NativeLibrary().getFromClass() to return null if the >>>> stack is empty instead of throwing an exception: >>>> >>>> static Class getFromClass() { >>>> return ClassLoader.nativeLibraryContext.empty() ? >>>> null : ClassLoader.nativeLibraryContext.peek().fromClass; >>>> } >>>> >>>> Unfortunately this also requires a HotSpot change in jni_FindClass() in >>>> order to properly handle the new 'null' return value: >>>> >>>> if (k.not_null()) { >>>> loader = Handle(THREAD, k->class_loader()); >>>> // Special handling to make sure JNI_OnLoad and JNI_OnUnload are >>>> executed >>>> // in the correct class context. >>>> if (loader.is_null() && >>>> k->name() == vmSymbols::java_lang_ClassLoader_NativeLibrary()) { >>>> JavaValue result(T_OBJECT); >>>> JavaCalls::call_static(&result, k, >>>> vmSymbols::getFromClass_name(), >>>> vmSymbols::void_class_signature(), >>>> thread); >>>> if (HAS_PENDING_EXCEPTION) { >>>> Handle ex(thread, thread->pending_exception()); >>>> CLEAR_PENDING_EXCEPTION; >>>> THROW_HANDLE_0(ex); >>>> } >>>> oop mirror = (oop) result.get_jobject(); >>>> if (oopDesc::is_null(mirror)) { >>>> loader = Handle(THREAD, SystemDictionary::java_system_loader()); >>>> } else { >>>> loader = Handle(THREAD, >>>> >>>> InstanceKlass::cast(java_lang_Class::as_Klass(mirror))->class_loader()); >>>> protection_domain = Handle(THREAD, >>>> >>>> >>>> InstanceKlass::cast(java_lang_Class::as_Klass(mirror))->protection_domain()); >>>> } >>>> } >>>> } else { >>>> // We call ClassLoader.getSystemClassLoader to obtain the system >>>> class >>>> loader. >>>> loader = Handle(THREAD, SystemDictionary::java_system_loader()); >>>> } >>>> >>>> These changes are sufficient to solve the problem in Java 8. >>>> Unfortunately, >>>> that's still not enough in Java 9 because there the loading of the >>>> extended >>>> charsets has been delegated to ServiceLoader. But ServiceLoader calls >>>> ClassLoader.getBootstrapResources() which calls >>>> sun.misc.Launcher.getBootstrapClassPath(). This leads to another problem >>>> during class initialization of sun.misc.Launcher if running on an >>>> unsupported locale. >>>> >>>> The first thing done in sun.misc.Launcher. is the initialisation >>>> of >>>> the bootstrap URLClassPath in the Launcher. However this initialisation >>>> will >>>> eventually call Charset.isSupported() and if we are running on an >>>> unsupported locale this will inevitably end in another recursive call to >>>> ServiceLoader. But as explained below, ServiceLoader will query the >>>> Launcher's bootstrap URLClassPath which will be still uninitialized at >>>> that >>>> point. >>>> >>>> So we'll have to additionally guard guard against this situation on JDK 9 >>>> like this: >>>> >>>> private static Charset lookupExtendedCharset(String charsetName) { >>>> if (!sun.misc.VM.isBooted() || // see >>>> lookupViaProviders() >>>> sun.misc.Launcher.getBootstrapClassPath() == null) >>>> return null; >>>> >>>> This fixes the crashes, but still at the price of not having the extended >>>> charsets available during initialization until >>>> Launcher.getBootstrapClassPath is set up properly. This may be still a >>>> problem if the jdk is installed in a directory which contains characters >>>> specific to an extended encoding or if we have such characters in the >>>> command line arguments. >>>> >>>> Thank you and best regards, >>>> Volker >>>> >>>> >>>> Mixed stack trace of the initial EmptyStackException for unsupported >>>> charsets described before: >>>> >>>> Native frames: (J=compiled Java code, j=interpreted, Vv=VM code, C=native >>>> code) >>>> j java.util.Stack.peek()Ljava/lang/Object;+1 >>>> j java.lang.ClassLoader$NativeLibrary.getFromClass()Ljava/lang/Class;+3 >>>> v ~StubRoutines::call_stub >>>> V [libjvm.so+0x9d279a] JavaCalls::call_helper(JavaValue*, >>>> methodHandle*, >>>> JavaCallArguments*, Thread*)+0x6b4 >>>> V [libjvm.so+0xcad591] os::os_exception_wrapper(void (*)(JavaValue*, >>>> methodHandle*, JavaCallArguments*, Thread*), JavaValue*, methodHandle*, >>>> JavaCallArguments*, Thread*)+0x45 >>>> V [libjvm.so+0x9d20cf] JavaCalls::call(JavaValue*, methodHandle, >>>> JavaCallArguments*, Thread*)+0x8b >>>> V [libjvm.so+0x9d1d3b] JavaCalls::call_static(JavaValue*, KlassHandle, >>>> Symbol*, Symbol*, JavaCallArguments*, Thread*)+0x139 >>>> V [libjvm.so+0x9d1e3f] JavaCalls::call_static(JavaValue*, KlassHandle, >>>> Symbol*, Symbol*, Thread*)+0x9d >>>> V [libjvm.so+0x9e6588] jni_FindClass+0x428 >>>> C [libjava.so+0x20208] JNU_CallStaticMethodByName+0xff >>>> C [libjava.so+0x21cae] jnuEncodingSupported+0x61 >>>> C [libjava.so+0x22125] JNU_GetStringPlatformChars+0x125 >>>> C [libjava.so+0xedcd] >>>> Java_java_lang_ClassLoader_00024NativeLibrary_findBuiltinLib+0x8b >>>> j >>>> >>>> java.lang.ClassLoader$NativeLibrary.findBuiltinLib(Ljava/lang/String;)Ljava/lang/String;+0 >>>> j java.lang.ClassLoader.loadLibrary0(Ljava/lang/Class;Ljava/io/File;)Z+4 >>>> j >>>> >>>> java.lang.ClassLoader.loadLibrary(Ljava/lang/Class;Ljava/lang/String;Z)V+228 >>>> j >>>> java.lang.Runtime.loadLibrary0(Ljava/lang/Class;Ljava/lang/String;)V+54 >>>> j java.lang.System.loadLibrary(Ljava/lang/String;)V+7 >>>> j java.lang.System.initializeSystemClass()V+113 >>>> v ~StubRoutines::call_stub >>>> V [libjvm.so+0x9d279a] JavaCalls::call_helper(JavaValue*, >>>> methodHandle*, >>>> JavaCallArguments*, Thread*)+0x6b4 >>>> V [libjvm.so+0xcad591] os::os_exception_wrapper(void (*)(JavaValue*, >>>> methodHandle*, JavaCallArguments*, Thread*), JavaValue*, methodHandle*, >>>> JavaCallArguments*, Thread*)+0x45 >>>> V [libjvm.so+0x9d20cf] JavaCalls::call(JavaValue*, methodHandle, >>>> JavaCallArguments*, Thread*)+0x8b >>>> V [libjvm.so+0x9d1d3b] JavaCalls::call_static(JavaValue*, KlassHandle, >>>> Symbol*, Symbol*, JavaCallArguments*, Thread*)+0x139 >>>> V [libjvm.so+0x9d1e3f] JavaCalls::call_static(JavaValue*, KlassHandle, >>>> Symbol*, Symbol*, Thread*)+0x9d >>>> V [libjvm.so+0xe3cceb] call_initializeSystemClass(Thread*)+0xb0 >>>> V [libjvm.so+0xe44444] >>>> Threads::initialize_java_lang_classes(JavaThread*, >>>> Thread*)+0x21a >>>> V [libjvm.so+0xe44b12] Threads::create_vm(JavaVMInitArgs*, bool*)+0x4a6 >>>> V [libjvm.so+0xa19bd7] JNI_CreateJavaVM+0xc7 >>>> C [libjli.so+0xa520] InitializeJVM+0x154 >>>> C [libjli.so+0x8024] JavaMain+0xcc >>>> >>>> From xueming.shen at oracle.com Tue Jun 9 18:07:22 2015 From: xueming.shen at oracle.com (Xueming Shen) Date: Tue, 09 Jun 2015 11:07:22 -0700 Subject: RFR(M): 8081674: EmptyStackException at startup if running with extended or unsupported charset In-Reply-To: <55772A56.2030308@oracle.com> References: <557626AA.6010008@oracle.com> <55772A56.2030308@oracle.com> Message-ID: <55772B5A.4000206@oracle.com> On 06/09/2015 11:03 AM, Mandy Chung wrote: > Does my patch work fine on 8u? If it works fine, I prefer to get that simple fix in 8u and take the time to have a better fix in 9 (jdk9 is still under development and I have assumed that it's not a show stopper to you. Let me know if it blocks you). > > A question to Sherman - do we have adequate tests to verify the move of System.loadLibrary("zip") from system initialization to ZipFile initialization for 8u? I don't think we have "adequate" tests for the "zip" initialization. It had tricky problems in the past. I would assume it might be safe to move it from the System to the zip package for 9 as we no longer have those jar files to load the classes, but that probably not the case for 8u, yet(?). -Sherman > > Mandy > > On 06/09/2015 10:09 AM, Volker Simonis wrote: >> Hi Mandy, >> >> thanks for looking into this. >> Uunfortunately your fix only helps to fix "java -version" :( >> >> Running even a minimal HelloWorld will still fail with the following >> stack trace which is still caused by the same EmptyStackException: >> >> Error: A JNI error has occurred, please check your installation and try again >> Exception in thread "main" java.lang.ExceptionInInitializerError >> at sun.misc.Unsafe.ensureClassInitialized(Native Method) >> at sun.misc.SharedSecrets.getJavaUtilZipFileAccess(SharedSecrets.java:158) >> at sun.misc.URLClassPath$JarLoader.(URLClassPath.java:765) >> at sun.misc.URLClassPath$3.run(URLClassPath.java:530) >> at sun.misc.URLClassPath$3.run(URLClassPath.java:520) >> at java.security.AccessController.doPrivileged(Native Method) >> at sun.misc.URLClassPath.getLoader(URLClassPath.java:519) >> at sun.misc.URLClassPath.getLoader(URLClassPath.java:492) >> at sun.misc.URLClassPath.getNextLoader(URLClassPath.java:457) >> at sun.misc.URLClassPath.access$100(URLClassPath.java:64) >> at sun.misc.URLClassPath$1.next(URLClassPath.java:239) >> at sun.misc.URLClassPath$1.hasMoreElements(URLClassPath.java:250) >> at java.net.URLClassLoader$3$1.run(URLClassLoader.java:601) >> at java.net.URLClassLoader$3$1.run(URLClassLoader.java:599) >> at java.security.AccessController.doPrivileged(Native Method) >> at java.net.URLClassLoader$3.next(URLClassLoader.java:598) >> at java.net.URLClassLoader$3.hasMoreElements(URLClassLoader.java:623) >> at sun.misc.CompoundEnumeration.next(CompoundEnumeration.java:45) >> at sun.misc.CompoundEnumeration.hasMoreElements(CompoundEnumeration.java:54) >> at sun.misc.CompoundEnumeration.next(CompoundEnumeration.java:45) >> at sun.misc.CompoundEnumeration.hasMoreElements(CompoundEnumeration.java:54) >> at java.util.ServiceLoader$LazyIterator.hasNextService(ServiceLoader.java:354) >> at java.util.ServiceLoader$LazyIterator.hasNext(ServiceLoader.java:393) >> at java.util.ServiceLoader$1.hasNext(ServiceLoader.java:474) >> at java.nio.charset.Charset$1.getNext(Charset.java:350) >> at java.nio.charset.Charset$1.hasNext(Charset.java:365) >> at java.nio.charset.Charset$2.run(Charset.java:410) >> at java.nio.charset.Charset$2.run(Charset.java:407) >> at java.security.AccessController.doPrivileged(Native Method) >> at java.nio.charset.Charset.lookupViaProviders(Charset.java:406) >> at java.nio.charset.Charset.lookup2(Charset.java:477) >> at java.nio.charset.Charset.lookup(Charset.java:464) >> at java.nio.charset.Charset.isSupported(Charset.java:505) >> at sun.launcher.LauncherHelper.makePlatformString(LauncherHelper.java:580) >> Caused by: java.util.EmptyStackException >> at java.util.Stack.peek(Stack.java:102) >> at java.lang.ClassLoader$NativeLibrary.getFromClass(ClassLoader.java:1759) >> at java.lang.ClassLoader$NativeLibrary.findBuiltinLib(Native Method) >> at java.lang.ClassLoader.loadLibrary0(ClassLoader.java:1870) >> at java.lang.ClassLoader.loadLibrary(ClassLoader.java:1843) >> at java.lang.Runtime.loadLibrary0(Runtime.java:870) >> at java.lang.System.loadLibrary(System.java:1122) >> at java.util.zip.ZipFile.(ZipFile.java:86) >> ... 34 more >> >> It's obvious that the way jni_FindClass is looking for the class >> context by calling the NativeLibrary::getFromClass method is hacky but >> I think that the proposed fix is the quite simple and non-intrusive. >> And we probably don't want a complete rework of this code for 8 >> anyway. So why not fix it the proposed way in 8 and 9 for now? That >> will still leave us time to come up with a better clean-up at least >> for 9. >> >> What do you think? >> >> Regards, >> Volker >> >> On Tue, Jun 9, 2015 at 1:35 AM, Mandy Chung wrote: >>> Hi Volker, >>> >>> Your patch reminds me the question I have about loading zip library. >>> >>> In JDK 9 (and earlier release), zip library is loaded by the VM during >>> startup (see ClassLoader::load_zip_library). I think loadLibrary("zip") is >>> no longer needed to be called from System::initializeSystemClass method and >>> instead it can be loaded by java.util.zip.ZipFile static initializer. >>> >>> Do you mind to try the patch (below)? That may be a simpler fix. >>> >>> I never like the way jni_FindClass to look for the class context by calling >>> the NativeLibrary::getFromClass method. I will have to look deeper other >>> alternative to clean that up. If taking out loadLibrary("zip") resolves >>> your issue, this will give us time to come up with a better clean-up in the >>> future. >>> >>> Mandy >>> [1] https://bugs.openjdk.java.net/browse/JDK-4429040 >>> >>> diff --git a/src/share/classes/java/lang/System.java >>> b/src/share/classes/java/lang/System.java >>> --- a/src/share/classes/java/lang/System.java >>> +++ b/src/share/classes/java/lang/System.java >>> @@ -1192,10 +1192,6 @@ >>> setOut0(newPrintStream(fdOut, >>> props.getProperty("sun.stdout.encoding"))); >>> setErr0(newPrintStream(fdErr, >>> props.getProperty("sun.stderr.encoding"))); >>> >>> - // Load the zip library now in order to keep java.util.zip.ZipFile >>> - // from trying to use itself to load this library later. >>> - loadLibrary("zip"); >>> - >>> // Setup Java signal handlers for HUP, TERM, and INT (where >>> available). >>> Terminator.setup(); >>> >>> diff --git a/src/share/classes/java/util/zip/ZipFile.java >>> b/src/share/classes/java/util/zip/ZipFile.java >>> --- a/src/share/classes/java/util/zip/ZipFile.java >>> +++ b/src/share/classes/java/util/zip/ZipFile.java >>> @@ -83,6 +83,7 @@ >>> >>> static { >>> /* Zip library is loaded from System.initializeSystemClass */ >>> + System.loadLibrary("zip"); >>> initIDs(); >>> >>> } >>> >>> >>> >>> On 06/08/2015 07:23 AM, Volker Simonis wrote: >>>> Hi, >>>> >>>> can I please get a review at least for the part of this fix which is >>>> common for jdk8 and jdk9. It's only a few lines in >>>> src/share/vm/prims/jni.cpp for the hotspot repository and one line >>>> src/java.base/share/classes/java/lang/ClassLoader.java for the jdk >>>> repository. >>>> >>>> Thanks, >>>> Volker >>>> >>>> >>>> On Tue, Jun 2, 2015 at 6:12 PM, Volker Simonis >>>> wrote: >>>>> Hi, >>>>> >>>>> can I please have review (and a sponsor) for these changes: >>>>> >>>>> https://bugs.openjdk.java.net/browse/JDK-8081674 >>>>> http://cr.openjdk.java.net/~simonis/webrevs/2015/8081674.jdk >>>>> http://cr.openjdk.java.net/~simonis/webrevs/2015/8081674.hs >>>>> >>>>> Please notice that the fix requires synchronous changes in the jdk and >>>>> the >>>>> hotspot forest. >>>>> >>>>> The changes themselves are by far not that big as this explanation but I >>>>> found the problem to be quite intricate so I tried to explain it as good >>>>> as >>>>> I could. I'd suggest to read the HTML-formatted explanation in the webrev >>>>> although the content is equal to the one in this mail: >>>>> >>>>> Using an unsupported character encoding (e.g. vi_VN.TCVN on Linux) >>>>> results >>>>> in an immediate VM failure with jdk 8 and 9: >>>>> >>>>> export LANG=vi_VN.TCVN >>>>> java -version >>>>> Error occurred during initialization of VM >>>>> java.util.EmptyStackException >>>>> at java.util.Stack.peek(Stack.java:102) >>>>> at >>>>> java.lang.ClassLoader$NativeLibrary.getFromClass(ClassLoader.java:1751) >>>>> at java.lang.ClassLoader$NativeLibrary.findBuiltinLib(Native >>>>> Method) >>>>> at java.lang.ClassLoader.loadLibrary0(ClassLoader.java:1862) >>>>> at java.lang.ClassLoader.loadLibrary(ClassLoader.java:1835) >>>>> at java.lang.Runtime.loadLibrary0(Runtime.java:870) >>>>> at java.lang.System.loadLibrary(System.java:1119) >>>>> at java.lang.System.initializeSystemClass(System.java:1194) >>>>> >>>>> This is a consequence of "8005716: Enhance JNI specification to allow >>>>> support of static JNI libraries in Embedded JREs". >>>>> >>>>> With jdk 9 we get this error even if we're running with a supported >>>>> charset >>>>> which is in the ExtendedCharsets (as opposed to being in the >>>>> StandardCharsets) class which is a consequence of delegating the loading >>>>> of >>>>> the ExtendedCharsets class to the ServiceLoader in jdk 9. >>>>> >>>>> export LANG=eo.iso-8859-3 >>>>> output-jdk9/images/jdk/bin/java -version >>>>> Error occurred during initialization of VM >>>>> java.util.EmptyStackException >>>>> at java.util.Stack.peek(Stack.java:102) >>>>> at >>>>> java.lang.ClassLoader$NativeLibrary.getFromClass(ClassLoader.java:1737) >>>>> at java.lang.ClassLoader$NativeLibrary.findBuiltinLib(Native >>>>> Method) >>>>> at java.lang.ClassLoader.loadLibrary0(ClassLoader.java:1866) >>>>> at java.lang.ClassLoader.loadLibrary(ClassLoader.java:1840) >>>>> at java.lang.Runtime.loadLibrary0(Runtime.java:874) >>>>> at java.lang.System.loadLibrary(System.java:1111) >>>>> at java.lang.System.initializeSystemClass(System.java:1186) >>>>> >>>>> Here's why the exception happens for an unsupported charset (see the >>>>> mixed >>>>> stack trace below for the full details): >>>>> >>>>> java.lang.System.loadLibrary() wants to load libzip.so. It calls >>>>> java.lang.Runtime.loadLibrary0() which at the very beginning calls the >>>>> native method ClassLoader$NativeLibrary.findBuiltinLib() which checks if >>>>> the >>>>> corresponding library is already statically linked into the VM >>>>> (introduced >>>>> by 8005716). >>>>> Java_java_lang_ClassLoader_00024NativeLibrary_findBuiltinLib(), >>>>> the native implementation of findBuiltinLib() in Classloader.c calls >>>>> GetStringPlatformChars() to convert the library name into the native >>>>> platform encoding. GetStringPlatformChars() calls the helper function >>>>> jnuEncodingSupported() to check if the platform encoding which is stored >>>>> in >>>>> the property "sun.jnu.encoding" is supported by Java. >>>>> jnuEncodingSupported() >>>>> is implemented as follows: >>>>> >>>>> static jboolean isJNUEncodingSupported = JNI_FALSE; >>>>> static jboolean jnuEncodingSupported(JNIEnv *env) { >>>>> jboolean exe; >>>>> if (isJNUEncodingSupported == JNI_TRUE) { >>>>> return JNI_TRUE; >>>>> } >>>>> isJNUEncodingSupported = (jboolean) JNU_CallStaticMethodByName ( >>>>> env, &exe, >>>>> "java/nio/charset/Charset", >>>>> "isSupported", >>>>> "(Ljava/lang/String;)Z", >>>>> jnuEncoding).z; >>>>> return isJNUEncodingSupported; >>>>> } >>>>> >>>>> Once the function finds that the platform encoding is supported (by >>>>> calling >>>>> java.nio.charset.Charset.isSupported()) it caches this value and always >>>>> returns it on following calls. However if the platform encoding is not >>>>> supported, it ALWAYS calls java.nio.charset.Charset.isSupported() an >>>>> every >>>>> subsequent invocation. >>>>> >>>>> In order to call the Java method Charset.isSupported() (in >>>>> JNU_CallStaticMethodByName() in file jni_util.c), we have to call >>>>> jni_FindClass() to convert the symbolic class name >>>>> "java.nio.charset.Charset" into a class reference. >>>>> >>>>> But unfortunately, jni_FindClass() (from jni.cpp in libjvm.so) has a >>>>> special >>>>> handling if called from java.lang.ClassLoader$NativeLibrary to ensure >>>>> that >>>>> JNI_OnLoad/JNI_OnUnload are executed in the correct class context: >>>>> >>>>> instanceKlassHandle k (THREAD, thread->security_get_caller_class(0)); >>>>> if (k.not_null()) { >>>>> loader = Handle(THREAD, k->class_loader()); >>>>> // Special handling to make sure JNI_OnLoad and JNI_OnUnload are >>>>> executed >>>>> // in the correct class context. >>>>> if (loader.is_null() && >>>>> k->name() == vmSymbols::java_lang_ClassLoader_NativeLibrary()) { >>>>> JavaValue result(T_OBJECT); >>>>> JavaCalls::call_static(&result, k, >>>>> vmSymbols::getFromClass_name(), >>>>> vmSymbols::void_class_signature(), >>>>> thread); >>>>> if (HAS_PENDING_EXCEPTION) { >>>>> Handle ex(thread, thread->pending_exception()); >>>>> CLEAR_PENDING_EXCEPTION; >>>>> THROW_HANDLE_0(ex); >>>>> } >>>>> >>>>> So if that's the case and jni_FindClass() was reallycalled from >>>>> ClassLoader$NativeLibrary, then jni_FindClass() calles >>>>> ClassLoader$NativeLibrary().getFromClass() to find out the corresponding >>>>> context class which is supposed to be saved there in a field of type >>>>> java.util.Stack named "nativeLibraryContext": >>>>> >>>>> // Invoked in the VM to determine the context class in >>>>> // JNI_Load/JNI_Unload >>>>> static Class getFromClass() { >>>>> return ClassLoader.nativeLibraryContext.peek().fromClass; >>>>> } >>>>> >>>>> Unfortunately, "nativeLibraryContext" doesn't contain any entry at this >>>>> point and the invocation of Stack.peek() will throw the exception shown >>>>> before. In general, the "nativeLibraryContext" stack will be filled later >>>>> on >>>>> in Runtime.loadLibrary0() like this: >>>>> >>>>> NativeLibrary lib = new NativeLibrary(fromClass, name, isBuiltin); >>>>> nativeLibraryContext.push(lib); >>>>> try { >>>>> lib.load(name, isBuiltin); >>>>> } finally { >>>>> nativeLibraryContext.pop(); >>>>> } >>>>> >>>>> such that it always contains at least one element later when >>>>> jni_FindClass() >>>>> will be invoked. >>>>> >>>>> So in summary, the problem is that the implementors of 8005716 didn't >>>>> took >>>>> into account that calling ClassLoader$NativeLibrary.findBuiltinLib() may >>>>> trigger a call to jni_FindClass() if we are running on a system with an >>>>> unsupported character encoding. >>>>> >>>>> I'd suggest the following fix for this problem: >>>>> >>>>> Change ClassLoader$NativeLibrary().getFromClass() to return null if the >>>>> stack is empty instead of throwing an exception: >>>>> >>>>> static Class getFromClass() { >>>>> return ClassLoader.nativeLibraryContext.empty() ? >>>>> null : ClassLoader.nativeLibraryContext.peek().fromClass; >>>>> } >>>>> >>>>> Unfortunately this also requires a HotSpot change in jni_FindClass() in >>>>> order to properly handle the new 'null' return value: >>>>> >>>>> if (k.not_null()) { >>>>> loader = Handle(THREAD, k->class_loader()); >>>>> // Special handling to make sure JNI_OnLoad and JNI_OnUnload are >>>>> executed >>>>> // in the correct class context. >>>>> if (loader.is_null() && >>>>> k->name() == vmSymbols::java_lang_ClassLoader_NativeLibrary()) { >>>>> JavaValue result(T_OBJECT); >>>>> JavaCalls::call_static(&result, k, >>>>> vmSymbols::getFromClass_name(), >>>>> vmSymbols::void_class_signature(), >>>>> thread); >>>>> if (HAS_PENDING_EXCEPTION) { >>>>> Handle ex(thread, thread->pending_exception()); >>>>> CLEAR_PENDING_EXCEPTION; >>>>> THROW_HANDLE_0(ex); >>>>> } >>>>> oop mirror = (oop) result.get_jobject(); >>>>> if (oopDesc::is_null(mirror)) { >>>>> loader = Handle(THREAD, SystemDictionary::java_system_loader()); >>>>> } else { >>>>> loader = Handle(THREAD, >>>>> >>>>> InstanceKlass::cast(java_lang_Class::as_Klass(mirror))->class_loader()); >>>>> protection_domain = Handle(THREAD, >>>>> >>>>> >>>>> InstanceKlass::cast(java_lang_Class::as_Klass(mirror))->protection_domain()); >>>>> } >>>>> } >>>>> } else { >>>>> // We call ClassLoader.getSystemClassLoader to obtain the system >>>>> class >>>>> loader. >>>>> loader = Handle(THREAD, SystemDictionary::java_system_loader()); >>>>> } >>>>> >>>>> These changes are sufficient to solve the problem in Java 8. >>>>> Unfortunately, >>>>> that's still not enough in Java 9 because there the loading of the >>>>> extended >>>>> charsets has been delegated to ServiceLoader. But ServiceLoader calls >>>>> ClassLoader.getBootstrapResources() which calls >>>>> sun.misc.Launcher.getBootstrapClassPath(). This leads to another problem >>>>> during class initialization of sun.misc.Launcher if running on an >>>>> unsupported locale. >>>>> >>>>> The first thing done in sun.misc.Launcher. is the initialisation >>>>> of >>>>> the bootstrap URLClassPath in the Launcher. However this initialisation >>>>> will >>>>> eventually call Charset.isSupported() and if we are running on an >>>>> unsupported locale this will inevitably end in another recursive call to >>>>> ServiceLoader. But as explained below, ServiceLoader will query the >>>>> Launcher's bootstrap URLClassPath which will be still uninitialized at >>>>> that >>>>> point. >>>>> >>>>> So we'll have to additionally guard guard against this situation on JDK 9 >>>>> like this: >>>>> >>>>> private static Charset lookupExtendedCharset(String charsetName) { >>>>> if (!sun.misc.VM.isBooted() || // see >>>>> lookupViaProviders() >>>>> sun.misc.Launcher.getBootstrapClassPath() == null) >>>>> return null; >>>>> >>>>> This fixes the crashes, but still at the price of not having the extended >>>>> charsets available during initialization until >>>>> Launcher.getBootstrapClassPath is set up properly. This may be still a >>>>> problem if the jdk is installed in a directory which contains characters >>>>> specific to an extended encoding or if we have such characters in the >>>>> command line arguments. >>>>> >>>>> Thank you and best regards, >>>>> Volker >>>>> >>>>> >>>>> Mixed stack trace of the initial EmptyStackException for unsupported >>>>> charsets described before: >>>>> >>>>> Native frames: (J=compiled Java code, j=interpreted, Vv=VM code, C=native >>>>> code) >>>>> j java.util.Stack.peek()Ljava/lang/Object;+1 >>>>> j java.lang.ClassLoader$NativeLibrary.getFromClass()Ljava/lang/Class;+3 >>>>> v ~StubRoutines::call_stub >>>>> V [libjvm.so+0x9d279a] JavaCalls::call_helper(JavaValue*, >>>>> methodHandle*, >>>>> JavaCallArguments*, Thread*)+0x6b4 >>>>> V [libjvm.so+0xcad591] os::os_exception_wrapper(void (*)(JavaValue*, >>>>> methodHandle*, JavaCallArguments*, Thread*), JavaValue*, methodHandle*, >>>>> JavaCallArguments*, Thread*)+0x45 >>>>> V [libjvm.so+0x9d20cf] JavaCalls::call(JavaValue*, methodHandle, >>>>> JavaCallArguments*, Thread*)+0x8b >>>>> V [libjvm.so+0x9d1d3b] JavaCalls::call_static(JavaValue*, KlassHandle, >>>>> Symbol*, Symbol*, JavaCallArguments*, Thread*)+0x139 >>>>> V [libjvm.so+0x9d1e3f] JavaCalls::call_static(JavaValue*, KlassHandle, >>>>> Symbol*, Symbol*, Thread*)+0x9d >>>>> V [libjvm.so+0x9e6588] jni_FindClass+0x428 >>>>> C [libjava.so+0x20208] JNU_CallStaticMethodByName+0xff >>>>> C [libjava.so+0x21cae] jnuEncodingSupported+0x61 >>>>> C [libjava.so+0x22125] JNU_GetStringPlatformChars+0x125 >>>>> C [libjava.so+0xedcd] >>>>> Java_java_lang_ClassLoader_00024NativeLibrary_findBuiltinLib+0x8b >>>>> j >>>>> >>>>> java.lang.ClassLoader$NativeLibrary.findBuiltinLib(Ljava/lang/String;)Ljava/lang/String;+0 >>>>> j java.lang.ClassLoader.loadLibrary0(Ljava/lang/Class;Ljava/io/File;)Z+4 >>>>> j >>>>> >>>>> java.lang.ClassLoader.loadLibrary(Ljava/lang/Class;Ljava/lang/String;Z)V+228 >>>>> j >>>>> java.lang.Runtime.loadLibrary0(Ljava/lang/Class;Ljava/lang/String;)V+54 >>>>> j java.lang.System.loadLibrary(Ljava/lang/String;)V+7 >>>>> j java.lang.System.initializeSystemClass()V+113 >>>>> v ~StubRoutines::call_stub >>>>> V [libjvm.so+0x9d279a] JavaCalls::call_helper(JavaValue*, >>>>> methodHandle*, >>>>> JavaCallArguments*, Thread*)+0x6b4 >>>>> V [libjvm.so+0xcad591] os::os_exception_wrapper(void (*)(JavaValue*, >>>>> methodHandle*, JavaCallArguments*, Thread*), JavaValue*, methodHandle*, >>>>> JavaCallArguments*, Thread*)+0x45 >>>>> V [libjvm.so+0x9d20cf] JavaCalls::call(JavaValue*, methodHandle, >>>>> JavaCallArguments*, Thread*)+0x8b >>>>> V [libjvm.so+0x9d1d3b] JavaCalls::call_static(JavaValue*, KlassHandle, >>>>> Symbol*, Symbol*, JavaCallArguments*, Thread*)+0x139 >>>>> V [libjvm.so+0x9d1e3f] JavaCalls::call_static(JavaValue*, KlassHandle, >>>>> Symbol*, Symbol*, Thread*)+0x9d >>>>> V [libjvm.so+0xe3cceb] call_initializeSystemClass(Thread*)+0xb0 >>>>> V [libjvm.so+0xe44444] >>>>> Threads::initialize_java_lang_classes(JavaThread*, >>>>> Thread*)+0x21a >>>>> V [libjvm.so+0xe44b12] Threads::create_vm(JavaVMInitArgs*, bool*)+0x4a6 >>>>> V [libjvm.so+0xa19bd7] JNI_CreateJavaVM+0xc7 >>>>> C [libjli.so+0xa520] InitializeJVM+0x154 >>>>> C [libjli.so+0x8024] JavaMain+0xcc >>>>> >>>>> > From mandy.chung at oracle.com Tue Jun 9 23:12:56 2015 From: mandy.chung at oracle.com (Mandy Chung) Date: Tue, 09 Jun 2015 16:12:56 -0700 Subject: RFR(M): 8081674: EmptyStackException at startup if running with extended or unsupported charset In-Reply-To: <55772B5A.4000206@oracle.com> References: <557626AA.6010008@oracle.com> <55772A56.2030308@oracle.com> <55772B5A.4000206@oracle.com> Message-ID: <557772F8.6090906@oracle.com> Given that the zip initialization is tricky, I agree that it's risky to delay the loading of zip library for JDK 8u. Volker - thanks for the instruction to reproduce ESE. Does your patch address the ServiceLoader issue? Charset calling sun.misc.Launcher.getBootstrapClassPath would still cause the bootstrapping issue AFAICT. That one is tricky to resolve. As for ESE, the static findBuiltinLib method finds if JNI_OnLoad_$LIBNAME entry exists. It is not intended to trigger any shared library to be loaded. Your patch would work. As findBuiltinLib doesn't load any shared library, it confuses JNI FindClass to call NativeLibrary::getFromClass. Another option is to move the findBuiltinLib method out from NativeLibrary class: http://cr.openjdk.java.net/~mchung/jdk9/webrevs/8081674/webrev.00/ Mandy On 06/09/2015 11:07 AM, Xueming Shen wrote: > On 06/09/2015 11:03 AM, Mandy Chung wrote: >> Does my patch work fine on 8u? If it works fine, I prefer to get >> that simple fix in 8u and take the time to have a better fix in 9 >> (jdk9 is still under development and I have assumed that it's not a >> show stopper to you. Let me know if it blocks you). >> >> A question to Sherman - do we have adequate tests to verify the move >> of System.loadLibrary("zip") from system initialization to ZipFile >> initialization for 8u? > > I don't think we have "adequate" tests for the "zip" initialization. > It had tricky problems in the past. > I would assume it might be safe to move it from the System to the zip > package for 9 as we no longer > have those jar files to load the classes, but that probably not the > case for 8u, yet(?). > > -Sherman > >> >> Mandy >> >> On 06/09/2015 10:09 AM, Volker Simonis wrote: >>> Hi Mandy, >>> >>> thanks for looking into this. >>> Uunfortunately your fix only helps to fix "java -version" :( >>> >>> Running even a minimal HelloWorld will still fail with the following >>> stack trace which is still caused by the same EmptyStackException: >>> >>> Error: A JNI error has occurred, please check your installation and >>> try again >>> Exception in thread "main" java.lang.ExceptionInInitializerError >>> at sun.misc.Unsafe.ensureClassInitialized(Native Method) >>> at >>> sun.misc.SharedSecrets.getJavaUtilZipFileAccess(SharedSecrets.java:158) >>> at sun.misc.URLClassPath$JarLoader.(URLClassPath.java:765) >>> at sun.misc.URLClassPath$3.run(URLClassPath.java:530) >>> at sun.misc.URLClassPath$3.run(URLClassPath.java:520) >>> at java.security.AccessController.doPrivileged(Native Method) >>> at sun.misc.URLClassPath.getLoader(URLClassPath.java:519) >>> at sun.misc.URLClassPath.getLoader(URLClassPath.java:492) >>> at sun.misc.URLClassPath.getNextLoader(URLClassPath.java:457) >>> at sun.misc.URLClassPath.access$100(URLClassPath.java:64) >>> at sun.misc.URLClassPath$1.next(URLClassPath.java:239) >>> at sun.misc.URLClassPath$1.hasMoreElements(URLClassPath.java:250) >>> at java.net.URLClassLoader$3$1.run(URLClassLoader.java:601) >>> at java.net.URLClassLoader$3$1.run(URLClassLoader.java:599) >>> at java.security.AccessController.doPrivileged(Native Method) >>> at java.net.URLClassLoader$3.next(URLClassLoader.java:598) >>> at >>> java.net.URLClassLoader$3.hasMoreElements(URLClassLoader.java:623) >>> at sun.misc.CompoundEnumeration.next(CompoundEnumeration.java:45) >>> at >>> sun.misc.CompoundEnumeration.hasMoreElements(CompoundEnumeration.java:54) >>> at sun.misc.CompoundEnumeration.next(CompoundEnumeration.java:45) >>> at >>> sun.misc.CompoundEnumeration.hasMoreElements(CompoundEnumeration.java:54) >>> at >>> java.util.ServiceLoader$LazyIterator.hasNextService(ServiceLoader.java:354) >>> at >>> java.util.ServiceLoader$LazyIterator.hasNext(ServiceLoader.java:393) >>> at java.util.ServiceLoader$1.hasNext(ServiceLoader.java:474) >>> at java.nio.charset.Charset$1.getNext(Charset.java:350) >>> at java.nio.charset.Charset$1.hasNext(Charset.java:365) >>> at java.nio.charset.Charset$2.run(Charset.java:410) >>> at java.nio.charset.Charset$2.run(Charset.java:407) >>> at java.security.AccessController.doPrivileged(Native Method) >>> at java.nio.charset.Charset.lookupViaProviders(Charset.java:406) >>> at java.nio.charset.Charset.lookup2(Charset.java:477) >>> at java.nio.charset.Charset.lookup(Charset.java:464) >>> at java.nio.charset.Charset.isSupported(Charset.java:505) >>> at >>> sun.launcher.LauncherHelper.makePlatformString(LauncherHelper.java:580) >>> Caused by: java.util.EmptyStackException >>> at java.util.Stack.peek(Stack.java:102) >>> at >>> java.lang.ClassLoader$NativeLibrary.getFromClass(ClassLoader.java:1759) >>> at java.lang.ClassLoader$NativeLibrary.findBuiltinLib(Native >>> Method) >>> at java.lang.ClassLoader.loadLibrary0(ClassLoader.java:1870) >>> at java.lang.ClassLoader.loadLibrary(ClassLoader.java:1843) >>> at java.lang.Runtime.loadLibrary0(Runtime.java:870) >>> at java.lang.System.loadLibrary(System.java:1122) >>> at java.util.zip.ZipFile.(ZipFile.java:86) >>> ... 34 more >>> >>> It's obvious that the way jni_FindClass is looking for the class >>> context by calling the NativeLibrary::getFromClass method is hacky but >>> I think that the proposed fix is the quite simple and non-intrusive. >>> And we probably don't want a complete rework of this code for 8 >>> anyway. So why not fix it the proposed way in 8 and 9 for now? That >>> will still leave us time to come up with a better clean-up at least >>> for 9. >>> >>> What do you think? >>> >>> Regards, >>> Volker >>> >>> On Tue, Jun 9, 2015 at 1:35 AM, Mandy Chung >>> wrote: >>>> Hi Volker, >>>> >>>> Your patch reminds me the question I have about loading zip library. >>>> >>>> In JDK 9 (and earlier release), zip library is loaded by the VM during >>>> startup (see ClassLoader::load_zip_library). I think >>>> loadLibrary("zip") is >>>> no longer needed to be called from System::initializeSystemClass >>>> method and >>>> instead it can be loaded by java.util.zip.ZipFile static initializer. >>>> >>>> Do you mind to try the patch (below)? That may be a simpler fix. >>>> >>>> I never like the way jni_FindClass to look for the class context by >>>> calling >>>> the NativeLibrary::getFromClass method. I will have to look deeper >>>> other >>>> alternative to clean that up. If taking out loadLibrary("zip") >>>> resolves >>>> your issue, this will give us time to come up with a better >>>> clean-up in the >>>> future. >>>> >>>> Mandy >>>> [1] https://bugs.openjdk.java.net/browse/JDK-4429040 >>>> >>>> diff --git a/src/share/classes/java/lang/System.java >>>> b/src/share/classes/java/lang/System.java >>>> --- a/src/share/classes/java/lang/System.java >>>> +++ b/src/share/classes/java/lang/System.java >>>> @@ -1192,10 +1192,6 @@ >>>> setOut0(newPrintStream(fdOut, >>>> props.getProperty("sun.stdout.encoding"))); >>>> setErr0(newPrintStream(fdErr, >>>> props.getProperty("sun.stderr.encoding"))); >>>> >>>> - // Load the zip library now in order to keep >>>> java.util.zip.ZipFile >>>> - // from trying to use itself to load this library later. >>>> - loadLibrary("zip"); >>>> - >>>> // Setup Java signal handlers for HUP, TERM, and INT (where >>>> available). >>>> Terminator.setup(); >>>> >>>> diff --git a/src/share/classes/java/util/zip/ZipFile.java >>>> b/src/share/classes/java/util/zip/ZipFile.java >>>> --- a/src/share/classes/java/util/zip/ZipFile.java >>>> +++ b/src/share/classes/java/util/zip/ZipFile.java >>>> @@ -83,6 +83,7 @@ >>>> >>>> static { >>>> /* Zip library is loaded from >>>> System.initializeSystemClass */ >>>> + System.loadLibrary("zip"); >>>> initIDs(); >>>> >>>> } >>>> >>>> >>>> >>>> On 06/08/2015 07:23 AM, Volker Simonis wrote: >>>>> Hi, >>>>> >>>>> can I please get a review at least for the part of this fix which is >>>>> common for jdk8 and jdk9. It's only a few lines in >>>>> src/share/vm/prims/jni.cpp for the hotspot repository and one line >>>>> src/java.base/share/classes/java/lang/ClassLoader.java for the jdk >>>>> repository. >>>>> >>>>> Thanks, >>>>> Volker >>>>> >>>>> >>>>> On Tue, Jun 2, 2015 at 6:12 PM, Volker Simonis >>>>> >>>>> wrote: >>>>>> Hi, >>>>>> >>>>>> can I please have review (and a sponsor) for these changes: >>>>>> >>>>>> https://bugs.openjdk.java.net/browse/JDK-8081674 >>>>>> http://cr.openjdk.java.net/~simonis/webrevs/2015/8081674.jdk >>>>>> http://cr.openjdk.java.net/~simonis/webrevs/2015/8081674.hs >>>>>> >>>>>> Please notice that the fix requires synchronous changes in the >>>>>> jdk and >>>>>> the >>>>>> hotspot forest. >>>>>> >>>>>> The changes themselves are by far not that big as this >>>>>> explanation but I >>>>>> found the problem to be quite intricate so I tried to explain it >>>>>> as good >>>>>> as >>>>>> I could. I'd suggest to read the HTML-formatted explanation in >>>>>> the webrev >>>>>> although the content is equal to the one in this mail: >>>>>> >>>>>> Using an unsupported character encoding (e.g. vi_VN.TCVN on Linux) >>>>>> results >>>>>> in an immediate VM failure with jdk 8 and 9: >>>>>> >>>>>> export LANG=vi_VN.TCVN >>>>>> java -version >>>>>> Error occurred during initialization of VM >>>>>> java.util.EmptyStackException >>>>>> at java.util.Stack.peek(Stack.java:102) >>>>>> at >>>>>> java.lang.ClassLoader$NativeLibrary.getFromClass(ClassLoader.java:1751) >>>>>> >>>>>> at >>>>>> java.lang.ClassLoader$NativeLibrary.findBuiltinLib(Native >>>>>> Method) >>>>>> at >>>>>> java.lang.ClassLoader.loadLibrary0(ClassLoader.java:1862) >>>>>> at java.lang.ClassLoader.loadLibrary(ClassLoader.java:1835) >>>>>> at java.lang.Runtime.loadLibrary0(Runtime.java:870) >>>>>> at java.lang.System.loadLibrary(System.java:1119) >>>>>> at java.lang.System.initializeSystemClass(System.java:1194) >>>>>> >>>>>> This is a consequence of "8005716: Enhance JNI specification to >>>>>> allow >>>>>> support of static JNI libraries in Embedded JREs". >>>>>> >>>>>> With jdk 9 we get this error even if we're running with a supported >>>>>> charset >>>>>> which is in the ExtendedCharsets (as opposed to being in the >>>>>> StandardCharsets) class which is a consequence of delegating the >>>>>> loading >>>>>> of >>>>>> the ExtendedCharsets class to the ServiceLoader in jdk 9. >>>>>> >>>>>> export LANG=eo.iso-8859-3 >>>>>> output-jdk9/images/jdk/bin/java -version >>>>>> Error occurred during initialization of VM >>>>>> java.util.EmptyStackException >>>>>> at java.util.Stack.peek(Stack.java:102) >>>>>> at >>>>>> java.lang.ClassLoader$NativeLibrary.getFromClass(ClassLoader.java:1737) >>>>>> >>>>>> at >>>>>> java.lang.ClassLoader$NativeLibrary.findBuiltinLib(Native >>>>>> Method) >>>>>> at >>>>>> java.lang.ClassLoader.loadLibrary0(ClassLoader.java:1866) >>>>>> at java.lang.ClassLoader.loadLibrary(ClassLoader.java:1840) >>>>>> at java.lang.Runtime.loadLibrary0(Runtime.java:874) >>>>>> at java.lang.System.loadLibrary(System.java:1111) >>>>>> at java.lang.System.initializeSystemClass(System.java:1186) >>>>>> >>>>>> Here's why the exception happens for an unsupported charset (see the >>>>>> mixed >>>>>> stack trace below for the full details): >>>>>> >>>>>> java.lang.System.loadLibrary() wants to load libzip.so. It calls >>>>>> java.lang.Runtime.loadLibrary0() which at the very beginning >>>>>> calls the >>>>>> native method ClassLoader$NativeLibrary.findBuiltinLib() which >>>>>> checks if >>>>>> the >>>>>> corresponding library is already statically linked into the VM >>>>>> (introduced >>>>>> by 8005716). >>>>>> Java_java_lang_ClassLoader_00024NativeLibrary_findBuiltinLib(), >>>>>> the native implementation of findBuiltinLib() in Classloader.c calls >>>>>> GetStringPlatformChars() to convert the library name into the native >>>>>> platform encoding. GetStringPlatformChars() calls the helper >>>>>> function >>>>>> jnuEncodingSupported() to check if the platform encoding which is >>>>>> stored >>>>>> in >>>>>> the property "sun.jnu.encoding" is supported by Java. >>>>>> jnuEncodingSupported() >>>>>> is implemented as follows: >>>>>> >>>>>> static jboolean isJNUEncodingSupported = JNI_FALSE; >>>>>> static jboolean jnuEncodingSupported(JNIEnv *env) { >>>>>> jboolean exe; >>>>>> if (isJNUEncodingSupported == JNI_TRUE) { >>>>>> return JNI_TRUE; >>>>>> } >>>>>> isJNUEncodingSupported = (jboolean) >>>>>> JNU_CallStaticMethodByName ( >>>>>> env, &exe, >>>>>> "java/nio/charset/Charset", >>>>>> "isSupported", >>>>>> "(Ljava/lang/String;)Z", >>>>>> jnuEncoding).z; >>>>>> return isJNUEncodingSupported; >>>>>> } >>>>>> >>>>>> Once the function finds that the platform encoding is supported (by >>>>>> calling >>>>>> java.nio.charset.Charset.isSupported()) it caches this value and >>>>>> always >>>>>> returns it on following calls. However if the platform encoding >>>>>> is not >>>>>> supported, it ALWAYS calls java.nio.charset.Charset.isSupported() an >>>>>> every >>>>>> subsequent invocation. >>>>>> >>>>>> In order to call the Java method Charset.isSupported() (in >>>>>> JNU_CallStaticMethodByName() in file jni_util.c), we have to call >>>>>> jni_FindClass() to convert the symbolic class name >>>>>> "java.nio.charset.Charset" into a class reference. >>>>>> >>>>>> But unfortunately, jni_FindClass() (from jni.cpp in libjvm.so) has a >>>>>> special >>>>>> handling if called from java.lang.ClassLoader$NativeLibrary to >>>>>> ensure >>>>>> that >>>>>> JNI_OnLoad/JNI_OnUnload are executed in the correct class context: >>>>>> >>>>>> instanceKlassHandle k (THREAD, >>>>>> thread->security_get_caller_class(0)); >>>>>> if (k.not_null()) { >>>>>> loader = Handle(THREAD, k->class_loader()); >>>>>> // Special handling to make sure JNI_OnLoad and >>>>>> JNI_OnUnload are >>>>>> executed >>>>>> // in the correct class context. >>>>>> if (loader.is_null() && >>>>>> k->name() == >>>>>> vmSymbols::java_lang_ClassLoader_NativeLibrary()) { >>>>>> JavaValue result(T_OBJECT); >>>>>> JavaCalls::call_static(&result, k, >>>>>> vmSymbols::getFromClass_name(), >>>>>> vmSymbols::void_class_signature(), >>>>>> thread); >>>>>> if (HAS_PENDING_EXCEPTION) { >>>>>> Handle ex(thread, thread->pending_exception()); >>>>>> CLEAR_PENDING_EXCEPTION; >>>>>> THROW_HANDLE_0(ex); >>>>>> } >>>>>> >>>>>> So if that's the case and jni_FindClass() was reallycalled from >>>>>> ClassLoader$NativeLibrary, then jni_FindClass() calles >>>>>> ClassLoader$NativeLibrary().getFromClass() to find out the >>>>>> corresponding >>>>>> context class which is supposed to be saved there in a field of type >>>>>> java.util.Stack named "nativeLibraryContext": >>>>>> >>>>>> // Invoked in the VM to determine the context class in >>>>>> // JNI_Load/JNI_Unload >>>>>> static Class getFromClass() { >>>>>> return ClassLoader.nativeLibraryContext.peek().fromClass; >>>>>> } >>>>>> >>>>>> Unfortunately, "nativeLibraryContext" doesn't contain any entry >>>>>> at this >>>>>> point and the invocation of Stack.peek() will throw the exception >>>>>> shown >>>>>> before. In general, the "nativeLibraryContext" stack will be >>>>>> filled later >>>>>> on >>>>>> in Runtime.loadLibrary0() like this: >>>>>> >>>>>> NativeLibrary lib = new NativeLibrary(fromClass, name, isBuiltin); >>>>>> nativeLibraryContext.push(lib); >>>>>> try { >>>>>> lib.load(name, isBuiltin); >>>>>> } finally { >>>>>> nativeLibraryContext.pop(); >>>>>> } >>>>>> >>>>>> such that it always contains at least one element later when >>>>>> jni_FindClass() >>>>>> will be invoked. >>>>>> >>>>>> So in summary, the problem is that the implementors of 8005716 >>>>>> didn't >>>>>> took >>>>>> into account that calling >>>>>> ClassLoader$NativeLibrary.findBuiltinLib() may >>>>>> trigger a call to jni_FindClass() if we are running on a system >>>>>> with an >>>>>> unsupported character encoding. >>>>>> >>>>>> I'd suggest the following fix for this problem: >>>>>> >>>>>> Change ClassLoader$NativeLibrary().getFromClass() to return null >>>>>> if the >>>>>> stack is empty instead of throwing an exception: >>>>>> >>>>>> static Class getFromClass() { >>>>>> return ClassLoader.nativeLibraryContext.empty() ? >>>>>> null : ClassLoader.nativeLibraryContext.peek().fromClass; >>>>>> } >>>>>> >>>>>> Unfortunately this also requires a HotSpot change in >>>>>> jni_FindClass() in >>>>>> order to properly handle the new 'null' return value: >>>>>> >>>>>> if (k.not_null()) { >>>>>> loader = Handle(THREAD, k->class_loader()); >>>>>> // Special handling to make sure JNI_OnLoad and >>>>>> JNI_OnUnload are >>>>>> executed >>>>>> // in the correct class context. >>>>>> if (loader.is_null() && >>>>>> k->name() == >>>>>> vmSymbols::java_lang_ClassLoader_NativeLibrary()) { >>>>>> JavaValue result(T_OBJECT); >>>>>> JavaCalls::call_static(&result, k, >>>>>> vmSymbols::getFromClass_name(), >>>>>> vmSymbols::void_class_signature(), >>>>>> thread); >>>>>> if (HAS_PENDING_EXCEPTION) { >>>>>> Handle ex(thread, thread->pending_exception()); >>>>>> CLEAR_PENDING_EXCEPTION; >>>>>> THROW_HANDLE_0(ex); >>>>>> } >>>>>> oop mirror = (oop) result.get_jobject(); >>>>>> if (oopDesc::is_null(mirror)) { >>>>>> loader = Handle(THREAD, >>>>>> SystemDictionary::java_system_loader()); >>>>>> } else { >>>>>> loader = Handle(THREAD, >>>>>> >>>>>> InstanceKlass::cast(java_lang_Class::as_Klass(mirror))->class_loader()); >>>>>> >>>>>> protection_domain = Handle(THREAD, >>>>>> >>>>>> >>>>>> InstanceKlass::cast(java_lang_Class::as_Klass(mirror))->protection_domain()); >>>>>> >>>>>> } >>>>>> } >>>>>> } else { >>>>>> // We call ClassLoader.getSystemClassLoader to obtain the >>>>>> system >>>>>> class >>>>>> loader. >>>>>> loader = Handle(THREAD, >>>>>> SystemDictionary::java_system_loader()); >>>>>> } >>>>>> >>>>>> These changes are sufficient to solve the problem in Java 8. >>>>>> Unfortunately, >>>>>> that's still not enough in Java 9 because there the loading of the >>>>>> extended >>>>>> charsets has been delegated to ServiceLoader. But ServiceLoader >>>>>> calls >>>>>> ClassLoader.getBootstrapResources() which calls >>>>>> sun.misc.Launcher.getBootstrapClassPath(). This leads to another >>>>>> problem >>>>>> during class initialization of sun.misc.Launcher if running on an >>>>>> unsupported locale. >>>>>> >>>>>> The first thing done in sun.misc.Launcher. is the >>>>>> initialisation >>>>>> of >>>>>> the bootstrap URLClassPath in the Launcher. However this >>>>>> initialisation >>>>>> will >>>>>> eventually call Charset.isSupported() and if we are running on an >>>>>> unsupported locale this will inevitably end in another recursive >>>>>> call to >>>>>> ServiceLoader. But as explained below, ServiceLoader will query the >>>>>> Launcher's bootstrap URLClassPath which will be still >>>>>> uninitialized at >>>>>> that >>>>>> point. >>>>>> >>>>>> So we'll have to additionally guard guard against this situation >>>>>> on JDK 9 >>>>>> like this: >>>>>> >>>>>> private static Charset lookupExtendedCharset(String charsetName) { >>>>>> if (!sun.misc.VM.isBooted() || // see >>>>>> lookupViaProviders() >>>>>> sun.misc.Launcher.getBootstrapClassPath() == null) >>>>>> return null; >>>>>> >>>>>> This fixes the crashes, but still at the price of not having the >>>>>> extended >>>>>> charsets available during initialization until >>>>>> Launcher.getBootstrapClassPath is set up properly. This may be >>>>>> still a >>>>>> problem if the jdk is installed in a directory which contains >>>>>> characters >>>>>> specific to an extended encoding or if we have such characters in >>>>>> the >>>>>> command line arguments. >>>>>> >>>>>> Thank you and best regards, >>>>>> Volker >>>>>> >>>>>> >>>>>> Mixed stack trace of the initial EmptyStackException for unsupported >>>>>> charsets described before: >>>>>> >>>>>> Native frames: (J=compiled Java code, j=interpreted, Vv=VM code, >>>>>> C=native >>>>>> code) >>>>>> j java.util.Stack.peek()Ljava/lang/Object;+1 >>>>>> j >>>>>> java.lang.ClassLoader$NativeLibrary.getFromClass()Ljava/lang/Class;+3 >>>>>> >>>>>> v ~StubRoutines::call_stub >>>>>> V [libjvm.so+0x9d279a] JavaCalls::call_helper(JavaValue*, >>>>>> methodHandle*, >>>>>> JavaCallArguments*, Thread*)+0x6b4 >>>>>> V [libjvm.so+0xcad591] os::os_exception_wrapper(void >>>>>> (*)(JavaValue*, >>>>>> methodHandle*, JavaCallArguments*, Thread*), JavaValue*, >>>>>> methodHandle*, >>>>>> JavaCallArguments*, Thread*)+0x45 >>>>>> V [libjvm.so+0x9d20cf] JavaCalls::call(JavaValue*, methodHandle, >>>>>> JavaCallArguments*, Thread*)+0x8b >>>>>> V [libjvm.so+0x9d1d3b] JavaCalls::call_static(JavaValue*, >>>>>> KlassHandle, >>>>>> Symbol*, Symbol*, JavaCallArguments*, Thread*)+0x139 >>>>>> V [libjvm.so+0x9d1e3f] JavaCalls::call_static(JavaValue*, >>>>>> KlassHandle, >>>>>> Symbol*, Symbol*, Thread*)+0x9d >>>>>> V [libjvm.so+0x9e6588] jni_FindClass+0x428 >>>>>> C [libjava.so+0x20208] JNU_CallStaticMethodByName+0xff >>>>>> C [libjava.so+0x21cae] jnuEncodingSupported+0x61 >>>>>> C [libjava.so+0x22125] JNU_GetStringPlatformChars+0x125 >>>>>> C [libjava.so+0xedcd] >>>>>> Java_java_lang_ClassLoader_00024NativeLibrary_findBuiltinLib+0x8b >>>>>> j >>>>>> >>>>>> java.lang.ClassLoader$NativeLibrary.findBuiltinLib(Ljava/lang/String;)Ljava/lang/String;+0 >>>>>> >>>>>> j >>>>>> java.lang.ClassLoader.loadLibrary0(Ljava/lang/Class;Ljava/io/File;)Z+4 >>>>>> >>>>>> j >>>>>> >>>>>> java.lang.ClassLoader.loadLibrary(Ljava/lang/Class;Ljava/lang/String;Z)V+228 >>>>>> >>>>>> j >>>>>> java.lang.Runtime.loadLibrary0(Ljava/lang/Class;Ljava/lang/String;)V+54 >>>>>> >>>>>> j java.lang.System.loadLibrary(Ljava/lang/String;)V+7 >>>>>> j java.lang.System.initializeSystemClass()V+113 >>>>>> v ~StubRoutines::call_stub >>>>>> V [libjvm.so+0x9d279a] JavaCalls::call_helper(JavaValue*, >>>>>> methodHandle*, >>>>>> JavaCallArguments*, Thread*)+0x6b4 >>>>>> V [libjvm.so+0xcad591] os::os_exception_wrapper(void >>>>>> (*)(JavaValue*, >>>>>> methodHandle*, JavaCallArguments*, Thread*), JavaValue*, >>>>>> methodHandle*, >>>>>> JavaCallArguments*, Thread*)+0x45 >>>>>> V [libjvm.so+0x9d20cf] JavaCalls::call(JavaValue*, methodHandle, >>>>>> JavaCallArguments*, Thread*)+0x8b >>>>>> V [libjvm.so+0x9d1d3b] JavaCalls::call_static(JavaValue*, >>>>>> KlassHandle, >>>>>> Symbol*, Symbol*, JavaCallArguments*, Thread*)+0x139 >>>>>> V [libjvm.so+0x9d1e3f] JavaCalls::call_static(JavaValue*, >>>>>> KlassHandle, >>>>>> Symbol*, Symbol*, Thread*)+0x9d >>>>>> V [libjvm.so+0xe3cceb] call_initializeSystemClass(Thread*)+0xb0 >>>>>> V [libjvm.so+0xe44444] >>>>>> Threads::initialize_java_lang_classes(JavaThread*, >>>>>> Thread*)+0x21a >>>>>> V [libjvm.so+0xe44b12] Threads::create_vm(JavaVMInitArgs*, >>>>>> bool*)+0x4a6 >>>>>> V [libjvm.so+0xa19bd7] JNI_CreateJavaVM+0xc7 >>>>>> C [libjli.so+0xa520] InitializeJVM+0x154 >>>>>> C [libjli.so+0x8024] JavaMain+0xcc >>>>>> >>>>>> >> > From srikanth.adayapalam at oracle.com Wed Jun 10 04:41:15 2015 From: srikanth.adayapalam at oracle.com (Srikanth) Date: Wed, 10 Jun 2015 10:11:15 +0530 Subject: RFR JDK-8054304: Clarify treatment of bounds in j.l.r.Annotated{WildcardType, TypeVariable} In-Reply-To: <55702597.6050907@oracle.com> References: <5513AD47.2080407@oracle.com> <55702597.6050907@oracle.com> Message-ID: <5577BFEB.2030805@oracle.com> > Hi Srikanth, > > I would mention that Object is not annotated in the following statements: > - AnnotatedWildcardType.getAnnotatedUpperBounds(): "Note that if no > upper bound is explicitly declared, the upper bound is Object." > - AnnotatedTypeVariable.getAnnotatedBounds(): "Note that if no bound is > explicitly declared, the bound is Object." > - TypeVariable.getAnnotatedBounds(): "Note that if no upper bound is > explicitly declared, the upper bound is Object." Thanks for the comment Elena. I agree this suggestion is good. Incorporated here: http://cr.openjdk.java.net/~sadayapalam/JDK-8054304/webrev.01/ Thanks! Srikanth > > > Thank you, > Elena > > 04.06.2015 13:16, Srikanth wrote: > > Hello, > > > > Please review this fix for > > https://bugs.openjdk.java.net/browse/JDK-8054304 > > (twin for conformance test failures: > > https://bugs.openjdk.java.net/browse/JDK-8058595) > > > > JBS: > > > > https://bugs.openjdk.java.net/browse/JDK-8054304 > > > > Webrev: > > > > http://cr.openjdk.java.net/~sadayapalam/JDK-8054304/webrev.00/ [...] From erik.joelsson at oracle.com Wed Jun 10 07:18:03 2015 From: erik.joelsson at oracle.com (Erik Joelsson) Date: Wed, 10 Jun 2015 09:18:03 +0200 Subject: RFR: JDK-8085822 JEP 223: New Version-String Scheme (initial integration) In-Reply-To: <5576EF94.3010500@oracle.com> References: <5571AD18.7080701@oracle.com> <557625C8.5050200@oracle.com> <5576E633.9050503@oracle.com> <5576E81B.8050703@oracle.com> <5576EF94.3010500@oracle.com> Message-ID: <5577E4AB.4090305@oracle.com> Looks good. /Erik On 2015-06-09 15:52, Magnus Ihse Bursie wrote: > Here's an updated webrev, which fixes the typos that were pointed out > by reviewers: > http://cr.openjdk.java.net/~ihse/JDK-8085822-JEP-223-initial-patch/webrev.02/ > > > And here's a (much simpler) delta webrev which shows just these changes: > http://cr.openjdk.java.net/~ihse/JDK-8085822-JEP-223-initial-patch-delta-01/webrev.01/ > > > /Magnus > > On 2015-06-09 15:20, Daniel D. Daugherty wrote: >> On 6/9/15 7:12 AM, Magnus Ihse Bursie wrote: >>> Hi Daniel, >>> >>> Thank you for your thorough review! >> >> This was my (failing) attempt at a "fast pass" review... :-) >> >> >>> >>> On 2015-06-09 01:31, Daniel D. Daugherty wrote: >>>> > >>>> http://cr.openjdk.java.net/~ihse/JDK-8085822-JEP-223-initial-patch/webrev.01 >>>> >>>> >>>> General comment: Not all copyright years were updated. >>>> General comment: It looks like support for the 'patch' value is not >>>> completely >>>> implemented through all the Makefiles. I didn't audit for this, >>>> but it's >>>> just my impression. >>> >>> You are basically correct. The makefiles all propagate the patch >>> value where needed, but the actual source code changes in hotspot >>> and jdk ignores the patch field as of now. This will be corrected in >>> a later add-on patch, submitted by someone from the jdk and/or >>> hotspot team rather than the build team. >>> >>>> >>>> common/autoconf/generated-configure.sh >>>> There are multiple Copyright notices in this file. Why? >>> Oh dear, you've reviewed the generated file. :-& I'm really >>> impressed by your effort! >> >> Ahhh... now that you say it... it sounds familiar... I may have >> made this same mistake before. I'll try to remember next time. >> >> >>> However, the generated-configure.sh file is automatically created by >>> the autoconf framework from the rest of the files in >>> common/autoconf/*, so we cannot direcly modify it's contents - only >>> indirectly. The reason it's checked in, is that otherwise every user >>> would need to generate it before being able to run configure. In >>> build reviews, we usually either revert changes to >>> generated-configure.sh before posting a review to hide it (and >>> re-generate it before pushing), or we just ignore it during reviews. >>> I should have done that, or pointed out that it was not needed nor >>> possible to review when I cross-posted. I'm sorry. >>> >>>> >>>> L4076: # Verify that a given string represent a valid version >>>> number, and assing it to >>>> L4077: # a variable. >>>> Fixed two typos and reformat a bit: >>>> # Verify that a given string represents a valid version >>>> number, and >>>> # assigning it to a variable. >>> I'll put that fix in the source .m4 file instead. Thanks. >>>> >>>> L20262: as_fn_error $? "--with--version-string must have a >>>> value" "$LINENO" 5 >>>> The '--with--version...' part doesn't match previous usages >>>> where >>>> '--with-version...' is used >>> Yes, you're right. Erik pointed it out as well. It's a typo in the >>> error message. It's all over the place due to copied code. I'll fix >>> it in the source .m4 file. >>> >>> (As a side note, I have a half-finished follow-up patch that will >>> reduce the amount of code duplication, but it requires some >>> framework changes so it'll have to be a separate thing.) >>> >>>> >>>> L20275: # Unspecified numerical fields is interpreted as 0. >>>> Grammar: 'is interpreted' -> 'are interpreted' >>>> >>>> L20286: as_fn_error $? "Version string contains + but >>>> both 'BUILD' and 'OPT' is missing" "$LINENO" 5 >>>> Grammar: 'is missing' -> 'are missing' >>> Those darn English plural forms! Why can't you all do as we sensible >>> Swedes and get rid of them? ;-) >>> >>> (I'll fix) >>> >>>> >>>> L20388: username=`$ECHO "$USER" | $TR -d -c >>>> '[a-z][A-Z][0-9]'` >>>> This assumes that the "USER" variable is set. Should there >>>> be a check for "" and perhaps use "no_user_specified" or >>>> something like that? Perhaps the build setup always makes >>>> sure that "USER" is set to something. Don't know. >>> Hm. I think the worst thing that'll happen is that the username part >>> of the opt string gets empty. This part is basically copied right >>> off the old build, where we have relied on the $USER variable being >>> present for all the time with no problems, so I think it's quite >>> safe to assume. >>>> >>>> common/autoconf/jdk-options.m4 >>>> Don't know why the 'elliptic curve crypto implementation' support >>>> is relocated as part of this changeset, but ... >>> It was incorrectly placed not at the top indentation level, but in >>> the middle of the function definition where the old versioning code >>> were handled. (Which, mostly by chance, happens to work in autoconf, >>> but is really bad style). >>> >>>> make/Javadoc.gmk >>>> Did you mean to remove the 'clean' target? >>> Yep. Cleaning is done by the top-level Main.gmk makefile nowadays, >>> that's just some dead code. >>> >>>> >>>> hotspot/make/windows/makefiles/compile.make >>>> No changes in the frames view. >>>> Update: udiff view shows a blank line deleted at the end of the >>>> file. >>> >>> That's probably the result of some change going forth and then back >>> again during development. But then again, removing extra blank >>> linkes is not a bad thing. (jcheck unfortunately does not enforce >>> any style checks for make files :-( so they tend to detoriate over >>> time.) >>> >>>> >>>> hotspot/src/share/vm/runtime/java.cpp >>>> L661: void JDK_Version::fully_initialize( >>>> L662: uint8_t major, uint8_t minor, uint8_t security, >>>> uint8_t update) { >>>> L663: // This is only called when current is less than 1.6 >>>> and we've gotten >>>> >>>> Since you're ripping out vestigial version support, I think >>>> this >>>> function can go away since this is version 9 and newer. >>>> Don't really >>>> know for sure, but based on that comment... >>> It probably can, yes. This and other core changes to the actual >>> .cpp/.java files will be addressed in a follow-up patch. >>>> >>>> hotspot/src/share/vm/runtime/java.hpp >>>> No comments. >>>> >>>> hotspot/src/share/vm/runtime/vmStructs.cpp >>>> L1240: please make the 'int' parameter align like the rest. >>> Are you okay with defering such a change to a follow-up patch? >> >> Yes. >> >> >>> It's likely the entire struct will need changes to be able to handle >>> a theoretically arbitrarily long version number. >>> >>>> >>>> hotspot/src/share/vm/runtime/vm_version.cpp >>>> L84: void Abstract_VM_Version::initialize() { >>>> L85: // FIXME: Initialization can probably be removed now. >>>> I agree, but that would entail also removing the >>>> _initialized and the uses of it... Follow on bug fix? >>> Definitely follow on. >>> >>>> jdk/src/java.base/share/classes/sun/misc/Version.java.template >>>> L149: * Returns the security version of the running JVM if >>>> it's 1.6 or newer >>>> This JavaDoc update is wrong, but it might not be important >>>> if sun.misc.Version class is going away. >>> I'm not sure if it's going to be replaced by, or just complemented >>> by java.util.Version, but in any case it will need a follow-up patch >>> to clean up this and other issues. >>>> langtools/src/java.compiler/share/classes/javax/lang/model/SourceVersion.java >>>> >>>> old L171: case "1.9": >>>> new L171: case "9": >>>> Should this logic support both versions? Will dropping >>>> "1.9" here prevent a pre-version changeset JVM from >>>> being dropped into a JDK for triage purposes? >>>> >>>> Granted we don't often triage 'javac' with different JVMs, >>>> but... >>> I'll defer that question to Kumar, who wrote that piece of code. My >>> guess is that when Hotspot express was dropped, the ability to use a >>> JVM from another JDK release bit rotteded away. >>> >>> /Magnus >> >> Dan > From volker.simonis at gmail.com Wed Jun 10 07:34:11 2015 From: volker.simonis at gmail.com (Volker Simonis) Date: Wed, 10 Jun 2015 09:34:11 +0200 Subject: RFR(M): 8081674: EmptyStackException at startup if running with extended or unsupported charset In-Reply-To: <55772A56.2030308@oracle.com> References: <557626AA.6010008@oracle.com> <55772A56.2030308@oracle.com> Message-ID: Mandy, the example/stacktrace I sent was with 8u. The problem isn't related to libzip, it is only the first library which gets loaded with System.loadLibrary(). The problem will occur with every library which gets loaded by System.loadLibrary() because java.lang.ClassLoader$NativeLibrary.findBuiltinLib() always calls jni_FindClass() if we're running on a unsupported locale. On Tue, Jun 9, 2015 at 8:03 PM, Mandy Chung wrote: > Does my patch work fine on 8u? If it works fine, I prefer to get that > simple fix in 8u and take the time to have a better fix in 9 (jdk9 is still > under development and I have assumed that it's not a show stopper to you. > Let me know if it blocks you). > > A question to Sherman - do we have adequate tests to verify the move of > System.loadLibrary("zip") from system initialization to ZipFile > initialization for 8u? > > Mandy > > > On 06/09/2015 10:09 AM, Volker Simonis wrote: >> >> Hi Mandy, >> >> thanks for looking into this. >> Uunfortunately your fix only helps to fix "java -version" :( >> >> Running even a minimal HelloWorld will still fail with the following >> stack trace which is still caused by the same EmptyStackException: >> >> Error: A JNI error has occurred, please check your installation and try >> again >> Exception in thread "main" java.lang.ExceptionInInitializerError >> at sun.misc.Unsafe.ensureClassInitialized(Native Method) >> at >> sun.misc.SharedSecrets.getJavaUtilZipFileAccess(SharedSecrets.java:158) >> at sun.misc.URLClassPath$JarLoader.(URLClassPath.java:765) >> at sun.misc.URLClassPath$3.run(URLClassPath.java:530) >> at sun.misc.URLClassPath$3.run(URLClassPath.java:520) >> at java.security.AccessController.doPrivileged(Native Method) >> at sun.misc.URLClassPath.getLoader(URLClassPath.java:519) >> at sun.misc.URLClassPath.getLoader(URLClassPath.java:492) >> at sun.misc.URLClassPath.getNextLoader(URLClassPath.java:457) >> at sun.misc.URLClassPath.access$100(URLClassPath.java:64) >> at sun.misc.URLClassPath$1.next(URLClassPath.java:239) >> at sun.misc.URLClassPath$1.hasMoreElements(URLClassPath.java:250) >> at java.net.URLClassLoader$3$1.run(URLClassLoader.java:601) >> at java.net.URLClassLoader$3$1.run(URLClassLoader.java:599) >> at java.security.AccessController.doPrivileged(Native Method) >> at java.net.URLClassLoader$3.next(URLClassLoader.java:598) >> at java.net.URLClassLoader$3.hasMoreElements(URLClassLoader.java:623) >> at sun.misc.CompoundEnumeration.next(CompoundEnumeration.java:45) >> at >> sun.misc.CompoundEnumeration.hasMoreElements(CompoundEnumeration.java:54) >> at sun.misc.CompoundEnumeration.next(CompoundEnumeration.java:45) >> at >> sun.misc.CompoundEnumeration.hasMoreElements(CompoundEnumeration.java:54) >> at >> java.util.ServiceLoader$LazyIterator.hasNextService(ServiceLoader.java:354) >> at >> java.util.ServiceLoader$LazyIterator.hasNext(ServiceLoader.java:393) >> at java.util.ServiceLoader$1.hasNext(ServiceLoader.java:474) >> at java.nio.charset.Charset$1.getNext(Charset.java:350) >> at java.nio.charset.Charset$1.hasNext(Charset.java:365) >> at java.nio.charset.Charset$2.run(Charset.java:410) >> at java.nio.charset.Charset$2.run(Charset.java:407) >> at java.security.AccessController.doPrivileged(Native Method) >> at java.nio.charset.Charset.lookupViaProviders(Charset.java:406) >> at java.nio.charset.Charset.lookup2(Charset.java:477) >> at java.nio.charset.Charset.lookup(Charset.java:464) >> at java.nio.charset.Charset.isSupported(Charset.java:505) >> at >> sun.launcher.LauncherHelper.makePlatformString(LauncherHelper.java:580) >> Caused by: java.util.EmptyStackException >> at java.util.Stack.peek(Stack.java:102) >> at >> java.lang.ClassLoader$NativeLibrary.getFromClass(ClassLoader.java:1759) >> at java.lang.ClassLoader$NativeLibrary.findBuiltinLib(Native Method) >> at java.lang.ClassLoader.loadLibrary0(ClassLoader.java:1870) >> at java.lang.ClassLoader.loadLibrary(ClassLoader.java:1843) >> at java.lang.Runtime.loadLibrary0(Runtime.java:870) >> at java.lang.System.loadLibrary(System.java:1122) >> at java.util.zip.ZipFile.(ZipFile.java:86) >> ... 34 more >> >> It's obvious that the way jni_FindClass is looking for the class >> context by calling the NativeLibrary::getFromClass method is hacky but >> I think that the proposed fix is the quite simple and non-intrusive. >> And we probably don't want a complete rework of this code for 8 >> anyway. So why not fix it the proposed way in 8 and 9 for now? That >> will still leave us time to come up with a better clean-up at least >> for 9. >> >> What do you think? >> >> Regards, >> Volker >> >> On Tue, Jun 9, 2015 at 1:35 AM, Mandy Chung >> wrote: >>> >>> Hi Volker, >>> >>> Your patch reminds me the question I have about loading zip library. >>> >>> In JDK 9 (and earlier release), zip library is loaded by the VM during >>> startup (see ClassLoader::load_zip_library). I think loadLibrary("zip") >>> is >>> no longer needed to be called from System::initializeSystemClass method >>> and >>> instead it can be loaded by java.util.zip.ZipFile static initializer. >>> >>> Do you mind to try the patch (below)? That may be a simpler fix. >>> >>> I never like the way jni_FindClass to look for the class context by >>> calling >>> the NativeLibrary::getFromClass method. I will have to look deeper other >>> alternative to clean that up. If taking out loadLibrary("zip") resolves >>> your issue, this will give us time to come up with a better clean-up in >>> the >>> future. >>> >>> Mandy >>> [1] https://bugs.openjdk.java.net/browse/JDK-4429040 >>> >>> diff --git a/src/share/classes/java/lang/System.java >>> b/src/share/classes/java/lang/System.java >>> --- a/src/share/classes/java/lang/System.java >>> +++ b/src/share/classes/java/lang/System.java >>> @@ -1192,10 +1192,6 @@ >>> setOut0(newPrintStream(fdOut, >>> props.getProperty("sun.stdout.encoding"))); >>> setErr0(newPrintStream(fdErr, >>> props.getProperty("sun.stderr.encoding"))); >>> >>> - // Load the zip library now in order to keep >>> java.util.zip.ZipFile >>> - // from trying to use itself to load this library later. >>> - loadLibrary("zip"); >>> - >>> // Setup Java signal handlers for HUP, TERM, and INT (where >>> available). >>> Terminator.setup(); >>> >>> diff --git a/src/share/classes/java/util/zip/ZipFile.java >>> b/src/share/classes/java/util/zip/ZipFile.java >>> --- a/src/share/classes/java/util/zip/ZipFile.java >>> +++ b/src/share/classes/java/util/zip/ZipFile.java >>> @@ -83,6 +83,7 @@ >>> >>> static { >>> /* Zip library is loaded from System.initializeSystemClass */ >>> + System.loadLibrary("zip"); >>> initIDs(); >>> >>> } >>> >>> >>> >>> On 06/08/2015 07:23 AM, Volker Simonis wrote: >>>> >>>> Hi, >>>> >>>> can I please get a review at least for the part of this fix which is >>>> common for jdk8 and jdk9. It's only a few lines in >>>> src/share/vm/prims/jni.cpp for the hotspot repository and one line >>>> src/java.base/share/classes/java/lang/ClassLoader.java for the jdk >>>> repository. >>>> >>>> Thanks, >>>> Volker >>>> >>>> >>>> On Tue, Jun 2, 2015 at 6:12 PM, Volker Simonis >>>> >>>> wrote: >>>>> >>>>> Hi, >>>>> >>>>> can I please have review (and a sponsor) for these changes: >>>>> >>>>> https://bugs.openjdk.java.net/browse/JDK-8081674 >>>>> http://cr.openjdk.java.net/~simonis/webrevs/2015/8081674.jdk >>>>> http://cr.openjdk.java.net/~simonis/webrevs/2015/8081674.hs >>>>> >>>>> Please notice that the fix requires synchronous changes in the jdk and >>>>> the >>>>> hotspot forest. >>>>> >>>>> The changes themselves are by far not that big as this explanation but >>>>> I >>>>> found the problem to be quite intricate so I tried to explain it as >>>>> good >>>>> as >>>>> I could. I'd suggest to read the HTML-formatted explanation in the >>>>> webrev >>>>> although the content is equal to the one in this mail: >>>>> >>>>> Using an unsupported character encoding (e.g. vi_VN.TCVN on Linux) >>>>> results >>>>> in an immediate VM failure with jdk 8 and 9: >>>>> >>>>> export LANG=vi_VN.TCVN >>>>> java -version >>>>> Error occurred during initialization of VM >>>>> java.util.EmptyStackException >>>>> at java.util.Stack.peek(Stack.java:102) >>>>> at >>>>> java.lang.ClassLoader$NativeLibrary.getFromClass(ClassLoader.java:1751) >>>>> at java.lang.ClassLoader$NativeLibrary.findBuiltinLib(Native >>>>> Method) >>>>> at java.lang.ClassLoader.loadLibrary0(ClassLoader.java:1862) >>>>> at java.lang.ClassLoader.loadLibrary(ClassLoader.java:1835) >>>>> at java.lang.Runtime.loadLibrary0(Runtime.java:870) >>>>> at java.lang.System.loadLibrary(System.java:1119) >>>>> at java.lang.System.initializeSystemClass(System.java:1194) >>>>> >>>>> This is a consequence of "8005716: Enhance JNI specification to allow >>>>> support of static JNI libraries in Embedded JREs". >>>>> >>>>> With jdk 9 we get this error even if we're running with a supported >>>>> charset >>>>> which is in the ExtendedCharsets (as opposed to being in the >>>>> StandardCharsets) class which is a consequence of delegating the >>>>> loading >>>>> of >>>>> the ExtendedCharsets class to the ServiceLoader in jdk 9. >>>>> >>>>> export LANG=eo.iso-8859-3 >>>>> output-jdk9/images/jdk/bin/java -version >>>>> Error occurred during initialization of VM >>>>> java.util.EmptyStackException >>>>> at java.util.Stack.peek(Stack.java:102) >>>>> at >>>>> java.lang.ClassLoader$NativeLibrary.getFromClass(ClassLoader.java:1737) >>>>> at java.lang.ClassLoader$NativeLibrary.findBuiltinLib(Native >>>>> Method) >>>>> at java.lang.ClassLoader.loadLibrary0(ClassLoader.java:1866) >>>>> at java.lang.ClassLoader.loadLibrary(ClassLoader.java:1840) >>>>> at java.lang.Runtime.loadLibrary0(Runtime.java:874) >>>>> at java.lang.System.loadLibrary(System.java:1111) >>>>> at java.lang.System.initializeSystemClass(System.java:1186) >>>>> >>>>> Here's why the exception happens for an unsupported charset (see the >>>>> mixed >>>>> stack trace below for the full details): >>>>> >>>>> java.lang.System.loadLibrary() wants to load libzip.so. It calls >>>>> java.lang.Runtime.loadLibrary0() which at the very beginning calls the >>>>> native method ClassLoader$NativeLibrary.findBuiltinLib() which checks >>>>> if >>>>> the >>>>> corresponding library is already statically linked into the VM >>>>> (introduced >>>>> by 8005716). >>>>> Java_java_lang_ClassLoader_00024NativeLibrary_findBuiltinLib(), >>>>> the native implementation of findBuiltinLib() in Classloader.c calls >>>>> GetStringPlatformChars() to convert the library name into the native >>>>> platform encoding. GetStringPlatformChars() calls the helper function >>>>> jnuEncodingSupported() to check if the platform encoding which is >>>>> stored >>>>> in >>>>> the property "sun.jnu.encoding" is supported by Java. >>>>> jnuEncodingSupported() >>>>> is implemented as follows: >>>>> >>>>> static jboolean isJNUEncodingSupported = JNI_FALSE; >>>>> static jboolean jnuEncodingSupported(JNIEnv *env) { >>>>> jboolean exe; >>>>> if (isJNUEncodingSupported == JNI_TRUE) { >>>>> return JNI_TRUE; >>>>> } >>>>> isJNUEncodingSupported = (jboolean) JNU_CallStaticMethodByName ( >>>>> env, &exe, >>>>> "java/nio/charset/Charset", >>>>> "isSupported", >>>>> "(Ljava/lang/String;)Z", >>>>> jnuEncoding).z; >>>>> return isJNUEncodingSupported; >>>>> } >>>>> >>>>> Once the function finds that the platform encoding is supported (by >>>>> calling >>>>> java.nio.charset.Charset.isSupported()) it caches this value and always >>>>> returns it on following calls. However if the platform encoding is not >>>>> supported, it ALWAYS calls java.nio.charset.Charset.isSupported() an >>>>> every >>>>> subsequent invocation. >>>>> >>>>> In order to call the Java method Charset.isSupported() (in >>>>> JNU_CallStaticMethodByName() in file jni_util.c), we have to call >>>>> jni_FindClass() to convert the symbolic class name >>>>> "java.nio.charset.Charset" into a class reference. >>>>> >>>>> But unfortunately, jni_FindClass() (from jni.cpp in libjvm.so) has a >>>>> special >>>>> handling if called from java.lang.ClassLoader$NativeLibrary to ensure >>>>> that >>>>> JNI_OnLoad/JNI_OnUnload are executed in the correct class context: >>>>> >>>>> instanceKlassHandle k (THREAD, >>>>> thread->security_get_caller_class(0)); >>>>> if (k.not_null()) { >>>>> loader = Handle(THREAD, k->class_loader()); >>>>> // Special handling to make sure JNI_OnLoad and JNI_OnUnload are >>>>> executed >>>>> // in the correct class context. >>>>> if (loader.is_null() && >>>>> k->name() == >>>>> vmSymbols::java_lang_ClassLoader_NativeLibrary()) { >>>>> JavaValue result(T_OBJECT); >>>>> JavaCalls::call_static(&result, k, >>>>> vmSymbols::getFromClass_name(), >>>>> >>>>> vmSymbols::void_class_signature(), >>>>> thread); >>>>> if (HAS_PENDING_EXCEPTION) { >>>>> Handle ex(thread, thread->pending_exception()); >>>>> CLEAR_PENDING_EXCEPTION; >>>>> THROW_HANDLE_0(ex); >>>>> } >>>>> >>>>> So if that's the case and jni_FindClass() was reallycalled from >>>>> ClassLoader$NativeLibrary, then jni_FindClass() calles >>>>> ClassLoader$NativeLibrary().getFromClass() to find out the >>>>> corresponding >>>>> context class which is supposed to be saved there in a field of type >>>>> java.util.Stack named "nativeLibraryContext": >>>>> >>>>> // Invoked in the VM to determine the context class in >>>>> // JNI_Load/JNI_Unload >>>>> static Class getFromClass() { >>>>> return ClassLoader.nativeLibraryContext.peek().fromClass; >>>>> } >>>>> >>>>> Unfortunately, "nativeLibraryContext" doesn't contain any entry at this >>>>> point and the invocation of Stack.peek() will throw the exception shown >>>>> before. In general, the "nativeLibraryContext" stack will be filled >>>>> later >>>>> on >>>>> in Runtime.loadLibrary0() like this: >>>>> >>>>> NativeLibrary lib = new NativeLibrary(fromClass, name, isBuiltin); >>>>> nativeLibraryContext.push(lib); >>>>> try { >>>>> lib.load(name, isBuiltin); >>>>> } finally { >>>>> nativeLibraryContext.pop(); >>>>> } >>>>> >>>>> such that it always contains at least one element later when >>>>> jni_FindClass() >>>>> will be invoked. >>>>> >>>>> So in summary, the problem is that the implementors of 8005716 didn't >>>>> took >>>>> into account that calling ClassLoader$NativeLibrary.findBuiltinLib() >>>>> may >>>>> trigger a call to jni_FindClass() if we are running on a system with an >>>>> unsupported character encoding. >>>>> >>>>> I'd suggest the following fix for this problem: >>>>> >>>>> Change ClassLoader$NativeLibrary().getFromClass() to return null if the >>>>> stack is empty instead of throwing an exception: >>>>> >>>>> static Class getFromClass() { >>>>> return ClassLoader.nativeLibraryContext.empty() ? >>>>> null : ClassLoader.nativeLibraryContext.peek().fromClass; >>>>> } >>>>> >>>>> Unfortunately this also requires a HotSpot change in jni_FindClass() in >>>>> order to properly handle the new 'null' return value: >>>>> >>>>> if (k.not_null()) { >>>>> loader = Handle(THREAD, k->class_loader()); >>>>> // Special handling to make sure JNI_OnLoad and JNI_OnUnload are >>>>> executed >>>>> // in the correct class context. >>>>> if (loader.is_null() && >>>>> k->name() == >>>>> vmSymbols::java_lang_ClassLoader_NativeLibrary()) { >>>>> JavaValue result(T_OBJECT); >>>>> JavaCalls::call_static(&result, k, >>>>> vmSymbols::getFromClass_name(), >>>>> >>>>> vmSymbols::void_class_signature(), >>>>> thread); >>>>> if (HAS_PENDING_EXCEPTION) { >>>>> Handle ex(thread, thread->pending_exception()); >>>>> CLEAR_PENDING_EXCEPTION; >>>>> THROW_HANDLE_0(ex); >>>>> } >>>>> oop mirror = (oop) result.get_jobject(); >>>>> if (oopDesc::is_null(mirror)) { >>>>> loader = Handle(THREAD, >>>>> SystemDictionary::java_system_loader()); >>>>> } else { >>>>> loader = Handle(THREAD, >>>>> >>>>> >>>>> InstanceKlass::cast(java_lang_Class::as_Klass(mirror))->class_loader()); >>>>> protection_domain = Handle(THREAD, >>>>> >>>>> >>>>> >>>>> InstanceKlass::cast(java_lang_Class::as_Klass(mirror))->protection_domain()); >>>>> } >>>>> } >>>>> } else { >>>>> // We call ClassLoader.getSystemClassLoader to obtain the system >>>>> class >>>>> loader. >>>>> loader = Handle(THREAD, SystemDictionary::java_system_loader()); >>>>> } >>>>> >>>>> These changes are sufficient to solve the problem in Java 8. >>>>> Unfortunately, >>>>> that's still not enough in Java 9 because there the loading of the >>>>> extended >>>>> charsets has been delegated to ServiceLoader. But ServiceLoader calls >>>>> ClassLoader.getBootstrapResources() which calls >>>>> sun.misc.Launcher.getBootstrapClassPath(). This leads to another >>>>> problem >>>>> during class initialization of sun.misc.Launcher if running on an >>>>> unsupported locale. >>>>> >>>>> The first thing done in sun.misc.Launcher. is the >>>>> initialisation >>>>> of >>>>> the bootstrap URLClassPath in the Launcher. However this initialisation >>>>> will >>>>> eventually call Charset.isSupported() and if we are running on an >>>>> unsupported locale this will inevitably end in another recursive call >>>>> to >>>>> ServiceLoader. But as explained below, ServiceLoader will query the >>>>> Launcher's bootstrap URLClassPath which will be still uninitialized at >>>>> that >>>>> point. >>>>> >>>>> So we'll have to additionally guard guard against this situation on JDK >>>>> 9 >>>>> like this: >>>>> >>>>> private static Charset lookupExtendedCharset(String charsetName) { >>>>> if (!sun.misc.VM.isBooted() || // see >>>>> lookupViaProviders() >>>>> sun.misc.Launcher.getBootstrapClassPath() == null) >>>>> return null; >>>>> >>>>> This fixes the crashes, but still at the price of not having the >>>>> extended >>>>> charsets available during initialization until >>>>> Launcher.getBootstrapClassPath is set up properly. This may be still a >>>>> problem if the jdk is installed in a directory which contains >>>>> characters >>>>> specific to an extended encoding or if we have such characters in the >>>>> command line arguments. >>>>> >>>>> Thank you and best regards, >>>>> Volker >>>>> >>>>> >>>>> Mixed stack trace of the initial EmptyStackException for unsupported >>>>> charsets described before: >>>>> >>>>> Native frames: (J=compiled Java code, j=interpreted, Vv=VM code, >>>>> C=native >>>>> code) >>>>> j java.util.Stack.peek()Ljava/lang/Object;+1 >>>>> j >>>>> java.lang.ClassLoader$NativeLibrary.getFromClass()Ljava/lang/Class;+3 >>>>> v ~StubRoutines::call_stub >>>>> V [libjvm.so+0x9d279a] JavaCalls::call_helper(JavaValue*, >>>>> methodHandle*, >>>>> JavaCallArguments*, Thread*)+0x6b4 >>>>> V [libjvm.so+0xcad591] os::os_exception_wrapper(void (*)(JavaValue*, >>>>> methodHandle*, JavaCallArguments*, Thread*), JavaValue*, methodHandle*, >>>>> JavaCallArguments*, Thread*)+0x45 >>>>> V [libjvm.so+0x9d20cf] JavaCalls::call(JavaValue*, methodHandle, >>>>> JavaCallArguments*, Thread*)+0x8b >>>>> V [libjvm.so+0x9d1d3b] JavaCalls::call_static(JavaValue*, >>>>> KlassHandle, >>>>> Symbol*, Symbol*, JavaCallArguments*, Thread*)+0x139 >>>>> V [libjvm.so+0x9d1e3f] JavaCalls::call_static(JavaValue*, >>>>> KlassHandle, >>>>> Symbol*, Symbol*, Thread*)+0x9d >>>>> V [libjvm.so+0x9e6588] jni_FindClass+0x428 >>>>> C [libjava.so+0x20208] JNU_CallStaticMethodByName+0xff >>>>> C [libjava.so+0x21cae] jnuEncodingSupported+0x61 >>>>> C [libjava.so+0x22125] JNU_GetStringPlatformChars+0x125 >>>>> C [libjava.so+0xedcd] >>>>> Java_java_lang_ClassLoader_00024NativeLibrary_findBuiltinLib+0x8b >>>>> j >>>>> >>>>> >>>>> java.lang.ClassLoader$NativeLibrary.findBuiltinLib(Ljava/lang/String;)Ljava/lang/String;+0 >>>>> j >>>>> java.lang.ClassLoader.loadLibrary0(Ljava/lang/Class;Ljava/io/File;)Z+4 >>>>> j >>>>> >>>>> >>>>> java.lang.ClassLoader.loadLibrary(Ljava/lang/Class;Ljava/lang/String;Z)V+228 >>>>> j >>>>> java.lang.Runtime.loadLibrary0(Ljava/lang/Class;Ljava/lang/String;)V+54 >>>>> j java.lang.System.loadLibrary(Ljava/lang/String;)V+7 >>>>> j java.lang.System.initializeSystemClass()V+113 >>>>> v ~StubRoutines::call_stub >>>>> V [libjvm.so+0x9d279a] JavaCalls::call_helper(JavaValue*, >>>>> methodHandle*, >>>>> JavaCallArguments*, Thread*)+0x6b4 >>>>> V [libjvm.so+0xcad591] os::os_exception_wrapper(void (*)(JavaValue*, >>>>> methodHandle*, JavaCallArguments*, Thread*), JavaValue*, methodHandle*, >>>>> JavaCallArguments*, Thread*)+0x45 >>>>> V [libjvm.so+0x9d20cf] JavaCalls::call(JavaValue*, methodHandle, >>>>> JavaCallArguments*, Thread*)+0x8b >>>>> V [libjvm.so+0x9d1d3b] JavaCalls::call_static(JavaValue*, >>>>> KlassHandle, >>>>> Symbol*, Symbol*, JavaCallArguments*, Thread*)+0x139 >>>>> V [libjvm.so+0x9d1e3f] JavaCalls::call_static(JavaValue*, >>>>> KlassHandle, >>>>> Symbol*, Symbol*, Thread*)+0x9d >>>>> V [libjvm.so+0xe3cceb] call_initializeSystemClass(Thread*)+0xb0 >>>>> V [libjvm.so+0xe44444] >>>>> Threads::initialize_java_lang_classes(JavaThread*, >>>>> Thread*)+0x21a >>>>> V [libjvm.so+0xe44b12] Threads::create_vm(JavaVMInitArgs*, >>>>> bool*)+0x4a6 >>>>> V [libjvm.so+0xa19bd7] JNI_CreateJavaVM+0xc7 >>>>> C [libjli.so+0xa520] InitializeJVM+0x154 >>>>> C [libjli.so+0x8024] JavaMain+0xcc >>>>> >>>>> > From Roger.Riggs at Oracle.com Wed Jun 10 09:55:24 2015 From: Roger.Riggs at Oracle.com (Roger Riggs) Date: Wed, 10 Jun 2015 10:55:24 +0100 Subject: RFR 9: 8086117: java/lang/Runtime/exec/LotsOfOutput.java still fails intermittently with Process consumes memory Message-ID: <5578098C.4030103@Oracle.com> Please review a test change in the algorithm for detecting memory growth to resolve an intermittent reporting problem. Webrev: http://cr.openjdk.java.net/~rriggs/webrev-lots-8086117/ Issue: https://bugs.openjdk.java.net/browse/JDK-8086117 Thanks, Roger From david.holmes at oracle.com Wed Jun 10 09:58:11 2015 From: david.holmes at oracle.com (David Holmes) Date: Wed, 10 Jun 2015 19:58:11 +1000 Subject: RFR: JDK-8085822 JEP 223: New Version-String Scheme (initial integration) In-Reply-To: <5576EF94.3010500@oracle.com> References: <5571AD18.7080701@oracle.com> <557625C8.5050200@oracle.com> <5576E633.9050503@oracle.com> <5576E81B.8050703@oracle.com> <5576EF94.3010500@oracle.com> Message-ID: <55780A33.1010302@oracle.com> Hi Magnus, Generally looks good - a few comments/queries below. On 9/06/2015 11:52 PM, Magnus Ihse Bursie wrote: > Here's an updated webrev, which fixes the typos that were pointed out by > reviewers: > http://cr.openjdk.java.net/~ihse/JDK-8085822-JEP-223-initial-patch/webrev.02/ common/autoconf/version-numbers Shouldn't there be a DEFAULT_VERSION_XXX for each of the XXX parts of the version info? (Even if all zeroes presently.) --- Looking at hotspot changes I can't convince myself that the new streamlined "version" variables will capture things like "fastdebug". Will that filter through via configure variables? --- make/*/makefiles/vm.make Shouldn't the -DVERSION_XX=$(VERSION_XX) definitions be taken from the VERSION_CFLAGS in spec.gmk? (Or are you still allowing for a stand-alone hotspot build?) --- hotspot/src/share/vm/prims/jvm.h jdk/src/java.base/share/native/include/jvm.h I think this comment is way out of date: /* Build number is available only for RE build (i.e. JDK_BUILD_NUMBER is set to bNN) * It will be zero for internal builds. */ and can be completely removed. Even if still true, mention of an "RE build" has no place in OpenJDK sources. --- hotspot/src/share/vm/runtime/java.cpp I think a lot of the modified code is obsolete post Hotspot Express - which makes it hard to determine if the changes make sense. --- hotspot/src/share/vm/runtime/vmStructs.cpp Please fix the alignment (3 extra spaces on modified line): 1239 static_field(Abstract_VM_Version, _vm_minor_version, int) \ 1240 static_field(Abstract_VM_Version, _vm_security_version, int) \ --- hotspot/test/runtime/6981737/Test6981737.java This test is really only valid for the new version scheme now, so it should probably be rewritten - and in that case it really isn't testing 6981737 but should be replaced by a new test for the new version format. --- jdk/src/java.base/share/classes/sun/misc/Version.java.template This comment is nonsensical: /** ! * Returns the security version of the running JVM if it's 1.6 or newer * or any RE VM build. It will return 0 if it's an internal 1.5 or * 1.4.x build. * @since 1.6 */ as security version does not exist pre 9. Normally you should be adding a new method and deprecating the old one. The new one is @since 9. /** ! * Returns the security version of the running JDK. * @since 1.6 */ Ditto: @since 9 (but again old should be deprecated and new method added) 253 /** 254 * Returns the build number of the running JDK if it's a RE build 255 * It will return 0 if it's an internal build. As with jvm.h this seems obsolete commentary these days - not only RE builds define a build number. Thanks, David > > And here's a (much simpler) delta webrev which shows just these changes: > http://cr.openjdk.java.net/~ihse/JDK-8085822-JEP-223-initial-patch-delta-01/webrev.01/ > > > /Magnus > > On 2015-06-09 15:20, Daniel D. Daugherty wrote: >> On 6/9/15 7:12 AM, Magnus Ihse Bursie wrote: >>> Hi Daniel, >>> >>> Thank you for your thorough review! >> >> This was my (failing) attempt at a "fast pass" review... :-) >> >> >>> >>> On 2015-06-09 01:31, Daniel D. Daugherty wrote: >>>> > >>>> http://cr.openjdk.java.net/~ihse/JDK-8085822-JEP-223-initial-patch/webrev.01 >>>> >>>> >>>> General comment: Not all copyright years were updated. >>>> General comment: It looks like support for the 'patch' value is not >>>> completely >>>> implemented through all the Makefiles. I didn't audit for this, >>>> but it's >>>> just my impression. >>> >>> You are basically correct. The makefiles all propagate the patch >>> value where needed, but the actual source code changes in hotspot and >>> jdk ignores the patch field as of now. This will be corrected in a >>> later add-on patch, submitted by someone from the jdk and/or hotspot >>> team rather than the build team. >>> >>>> >>>> common/autoconf/generated-configure.sh >>>> There are multiple Copyright notices in this file. Why? >>> Oh dear, you've reviewed the generated file. :-& I'm really impressed >>> by your effort! >> >> Ahhh... now that you say it... it sounds familiar... I may have >> made this same mistake before. I'll try to remember next time. >> >> >>> However, the generated-configure.sh file is automatically created by >>> the autoconf framework from the rest of the files in >>> common/autoconf/*, so we cannot direcly modify it's contents - only >>> indirectly. The reason it's checked in, is that otherwise every user >>> would need to generate it before being able to run configure. In >>> build reviews, we usually either revert changes to >>> generated-configure.sh before posting a review to hide it (and >>> re-generate it before pushing), or we just ignore it during reviews. >>> I should have done that, or pointed out that it was not needed nor >>> possible to review when I cross-posted. I'm sorry. >>> >>>> >>>> L4076: # Verify that a given string represent a valid version >>>> number, and assing it to >>>> L4077: # a variable. >>>> Fixed two typos and reformat a bit: >>>> # Verify that a given string represents a valid version >>>> number, and >>>> # assigning it to a variable. >>> I'll put that fix in the source .m4 file instead. Thanks. >>>> >>>> L20262: as_fn_error $? "--with--version-string must have a >>>> value" "$LINENO" 5 >>>> The '--with--version...' part doesn't match previous usages >>>> where >>>> '--with-version...' is used >>> Yes, you're right. Erik pointed it out as well. It's a typo in the >>> error message. It's all over the place due to copied code. I'll fix >>> it in the source .m4 file. >>> >>> (As a side note, I have a half-finished follow-up patch that will >>> reduce the amount of code duplication, but it requires some framework >>> changes so it'll have to be a separate thing.) >>> >>>> >>>> L20275: # Unspecified numerical fields is interpreted as 0. >>>> Grammar: 'is interpreted' -> 'are interpreted' >>>> >>>> L20286: as_fn_error $? "Version string contains + but >>>> both 'BUILD' and 'OPT' is missing" "$LINENO" 5 >>>> Grammar: 'is missing' -> 'are missing' >>> Those darn English plural forms! Why can't you all do as we sensible >>> Swedes and get rid of them? ;-) >>> >>> (I'll fix) >>> >>>> >>>> L20388: username=`$ECHO "$USER" | $TR -d -c >>>> '[a-z][A-Z][0-9]'` >>>> This assumes that the "USER" variable is set. Should there >>>> be a check for "" and perhaps use "no_user_specified" or >>>> something like that? Perhaps the build setup always makes >>>> sure that "USER" is set to something. Don't know. >>> Hm. I think the worst thing that'll happen is that the username part >>> of the opt string gets empty. This part is basically copied right off >>> the old build, where we have relied on the $USER variable being >>> present for all the time with no problems, so I think it's quite safe >>> to assume. >>>> >>>> common/autoconf/jdk-options.m4 >>>> Don't know why the 'elliptic curve crypto implementation' support >>>> is relocated as part of this changeset, but ... >>> It was incorrectly placed not at the top indentation level, but in >>> the middle of the function definition where the old versioning code >>> were handled. (Which, mostly by chance, happens to work in autoconf, >>> but is really bad style). >>> >>>> make/Javadoc.gmk >>>> Did you mean to remove the 'clean' target? >>> Yep. Cleaning is done by the top-level Main.gmk makefile nowadays, >>> that's just some dead code. >>> >>>> >>>> hotspot/make/windows/makefiles/compile.make >>>> No changes in the frames view. >>>> Update: udiff view shows a blank line deleted at the end of the >>>> file. >>> >>> That's probably the result of some change going forth and then back >>> again during development. But then again, removing extra blank linkes >>> is not a bad thing. (jcheck unfortunately does not enforce any style >>> checks for make files :-( so they tend to detoriate over time.) >>> >>>> >>>> hotspot/src/share/vm/runtime/java.cpp >>>> L661: void JDK_Version::fully_initialize( >>>> L662: uint8_t major, uint8_t minor, uint8_t security, >>>> uint8_t update) { >>>> L663: // This is only called when current is less than 1.6 and >>>> we've gotten >>>> >>>> Since you're ripping out vestigial version support, I think >>>> this >>>> function can go away since this is version 9 and newer. >>>> Don't really >>>> know for sure, but based on that comment... >>> It probably can, yes. This and other core changes to the actual >>> .cpp/.java files will be addressed in a follow-up patch. >>>> >>>> hotspot/src/share/vm/runtime/java.hpp >>>> No comments. >>>> >>>> hotspot/src/share/vm/runtime/vmStructs.cpp >>>> L1240: please make the 'int' parameter align like the rest. >>> Are you okay with defering such a change to a follow-up patch? >> >> Yes. >> >> >>> It's likely the entire struct will need changes to be able to handle >>> a theoretically arbitrarily long version number. >>> >>>> >>>> hotspot/src/share/vm/runtime/vm_version.cpp >>>> L84: void Abstract_VM_Version::initialize() { >>>> L85: // FIXME: Initialization can probably be removed now. >>>> I agree, but that would entail also removing the >>>> _initialized and the uses of it... Follow on bug fix? >>> Definitely follow on. >>> >>>> jdk/src/java.base/share/classes/sun/misc/Version.java.template >>>> L149: * Returns the security version of the running JVM if >>>> it's 1.6 or newer >>>> This JavaDoc update is wrong, but it might not be important >>>> if sun.misc.Version class is going away. >>> I'm not sure if it's going to be replaced by, or just complemented by >>> java.util.Version, but in any case it will need a follow-up patch to >>> clean up this and other issues. >>>> langtools/src/java.compiler/share/classes/javax/lang/model/SourceVersion.java >>>> >>>> old L171: case "1.9": >>>> new L171: case "9": >>>> Should this logic support both versions? Will dropping >>>> "1.9" here prevent a pre-version changeset JVM from >>>> being dropped into a JDK for triage purposes? >>>> >>>> Granted we don't often triage 'javac' with different JVMs, >>>> but... >>> I'll defer that question to Kumar, who wrote that piece of code. My >>> guess is that when Hotspot express was dropped, the ability to use a >>> JVM from another JDK release bit rotteded away. >>> >>> /Magnus >> >> Dan > From chris.hegarty at oracle.com Wed Jun 10 10:12:13 2015 From: chris.hegarty at oracle.com (Chris Hegarty) Date: Wed, 10 Jun 2015 11:12:13 +0100 Subject: RFR 9: 8086117: java/lang/Runtime/exec/LotsOfOutput.java still fails intermittently with Process consumes memory In-Reply-To: <5578098C.4030103@Oracle.com> References: <5578098C.4030103@Oracle.com> Message-ID: <55780D7D.8000302@oracle.com> Roger, It seems reasonable to be more lenient in testing for memory consumption in this test, given that the test would OOM quickly if the original bug was present. Are you confident enough to remove the '@key intermittent' tag at the same time? -Chris. On 10/06/15 10:55, Roger Riggs wrote: > Please review a test change in the algorithm for detecting memory growth to > resolve an intermittent reporting problem. > > Webrev: > http://cr.openjdk.java.net/~rriggs/webrev-lots-8086117/ > > Issue: > https://bugs.openjdk.java.net/browse/JDK-8086117 > > Thanks, Roger From tomasz.kowalczewski at gmail.com Wed Jun 10 11:21:38 2015 From: tomasz.kowalczewski at gmail.com (Tomasz Kowalczewski) Date: Wed, 10 Jun 2015 13:21:38 +0200 Subject: TimeZone issue in 1.8u60 Message-ID: Hi, I am not sure where to write about this, I hope somebody will point me to right list if this one is not correct. I have been playing with newest Java 1.8u60 to try PreserveFramePointer functionality. Unfortunately none of our servers start on this version of java. It is because of REST call to Amazon services done during startup. None of these calls worked. Unless I am missing something it turns out to be issue with formatting time zone information as done by Joda Time. It uses calls to: DateTimeUtils.getDateFormatSymbols(Locale.ENGLISH).getZoneStrings(); to get list of timezones. This usually returned array of arrays of 5 elements. In 1.8u60 it returns array of arrays of 7 elements. I know that all this software is not related to OpenJDK and calling getZoneStrings is discouraged in the docs. But as I am unfamiliar with time zones mechanisms inside JDK (loading from bundles etc.) I was hoping that somebody will point me to change that may caused this for sake of better understanding the issue. -- Thanks, Tomasz Kowalczewski From Roger.Riggs at Oracle.com Wed Jun 10 11:49:48 2015 From: Roger.Riggs at Oracle.com (Roger Riggs) Date: Wed, 10 Jun 2015 12:49:48 +0100 Subject: RFR 9: 8086117: java/lang/Runtime/exec/LotsOfOutput.java still fails intermittently with Process consumes memory In-Reply-To: <55780D7D.8000302@oracle.com> References: <5578098C.4030103@Oracle.com> <55780D7D.8000302@oracle.com> Message-ID: <5578245C.8080302@Oracle.com> Hi Chris, Thanks for the review. In the previous iteration of this fix, it was recommended to retain the intermittent keyword until there is a track record of the test not failing. Roger On 6/10/15 11:12 AM, Chris Hegarty wrote: > Roger, > > It seems reasonable to be more lenient in testing for memory > consumption in this test, given that the test would OOM quickly if the > original bug was present. > > Are you confident enough to remove the '@key intermittent' tag at the > same time? > > -Chris. > > On 10/06/15 10:55, Roger Riggs wrote: >> Please review a test change in the algorithm for detecting memory >> growth to >> resolve an intermittent reporting problem. >> >> Webrev: >> http://cr.openjdk.java.net/~rriggs/webrev-lots-8086117/ >> >> Issue: >> https://bugs.openjdk.java.net/browse/JDK-8086117 >> >> Thanks, Roger From magnus.ihse.bursie at oracle.com Wed Jun 10 12:13:16 2015 From: magnus.ihse.bursie at oracle.com (Magnus Ihse Bursie) Date: Wed, 10 Jun 2015 14:13:16 +0200 Subject: RFR: JDK-8085822 JEP 223: New Version-String Scheme (initial integration) In-Reply-To: <55780A33.1010302@oracle.com> References: <5571AD18.7080701@oracle.com> <557625C8.5050200@oracle.com> <5576E633.9050503@oracle.com> <5576E81B.8050703@oracle.com> <5576EF94.3010500@oracle.com> <55780A33.1010302@oracle.com> Message-ID: <557829DC.8000405@oracle.com> On 2015-06-10 11:58, David Holmes wrote: > Hi Magnus, > > Generally looks good - a few comments/queries below. In general, I believe most issues you found are valid. :-) However, as I said before in this thread, I'd like to see them resolved in the needed follow-up patches. The source code changes in Hotspot and JDK are inadequate to fully implement JEP-223, so these areas will need to be rewritten anyhow. Are you okay with resolving these issues later? > > On 9/06/2015 11:52 PM, Magnus Ihse Bursie wrote: >> Here's an updated webrev, which fixes the typos that were pointed out by >> reviewers: >> http://cr.openjdk.java.net/~ihse/JDK-8085822-JEP-223-initial-patch/webrev.02/ >> > > common/autoconf/version-numbers > > Shouldn't there be a DEFAULT_VERSION_XXX for each of the XXX parts of > the version info? (Even if all zeroes presently.) So that's a tricky one. :-& The question here is, I think, does it make sense to update version-numbers when we do a security (9.0.1) or minor (9.1.0) release? Looking historically, the version-numbers file have not been changed for the 8u family. The only change was going from 8 to 9 when creating the new jdk9 forest. That was what I based my decition to only have the major number in the file. (The rest of the version string is set by configure flags when building, in Oracle case by the RE team.) If it makes sense to put the minor/security/patch numbers here, I won't oppose it, but I guess we have until the first such release to figure out what's best, and that likely includes discussion with RE and possibly other consumers in the community (RedHat etc). > > --- > > Looking at hotspot changes I can't convince myself that the new > streamlined "version" variables will capture things like "fastdebug". > Will that filter through via configure variables? The "fastdebug" label has been handled as a less valued token in the JEP-223 process. Right now there's no mention of it at all in the version string proposal in the JEP. As I understand it, Iris is about to propose an amendment which will just make fastdebug be a part of the OPT string. I'm not entirely happy with that myself, but that's the way it's decided. So wherever you can see the OPT string, you'll see the debug level tag. Currently, however, that's not how it is implemented in this patch. Since no decision was made on this (and it's still not formally decided), I had to pick a route to go forward. The current patch will instead put the "fastdebug" label as part of the PRE string (that's the reason for the pre-base and pre-debuglevel arguments to configure). If the planned changes goes into the JEP, this'll change to make the debuglevel tag a part of the OPT string instead. > --- > > make/*/makefiles/vm.make > > Shouldn't the -DVERSION_XX=$(VERSION_XX) definitions be taken from the > VERSION_CFLAGS in spec.gmk? (Or are you still allowing for a > stand-alone hotspot build?) The hotspot JEP-223 work initially made by Alejandro kept support for stand-alone hotspot builds. I made additional changes on top of that, which still tried to cater for stand-alone builds. At this very moment I'm not sure if stand-alone hotspot builds are supposed to work or not, but I've tried to not change the situation with this patch. There are two future changes coming down the pipe: One is the additional JEP-223 work needed for Hotspot. I know Alejandro had plans for redesigning the API between Hotspot and the JDK, so no JDK version strings should be compiled into the JVM, but all of it extracted during an API in runtime. That would make most (all?) of the makefile changes in hotspot irrelevant. However, that implementation is not even started, so it's needed for the time being. The second change is the build-infra hotspot makefile rewrite. When that happens, stand-alone builds will definitely disappear, and if Hotspot still needs make support for version strings, then the logical choice is to use VERSION_CFLAGS. Ok? > > --- > > hotspot/src/share/vm/prims/jvm.h > jdk/src/java.base/share/native/include/jvm.h > > I think this comment is way out of date: > > /* Build number is available only for RE build (i.e. JDK_BUILD_NUMBER > is set to bNN) > * It will be zero for internal builds. > */ > > and can be completely removed. Even if still true, mention of an "RE > build" has no place in OpenJDK sources. Yep, agree. Follow-up patch, right? > > --- > > hotspot/src/share/vm/runtime/java.cpp > > I think a lot of the modified code is obsolete post Hotspot Express - > which makes it hard to determine if the changes make sense. Yep, agree. Follow-up patch, right? > > --- > > hotspot/src/share/vm/runtime/vmStructs.cpp > > Please fix the alignment (3 extra spaces on modified line): > > 1239 static_field(Abstract_VM_Version, _vm_minor_version, > int) \ > 1240 static_field(Abstract_VM_Version, > _vm_security_version, int) \ I was about to say "follow-up patch", but ugly indentation really sucks so I can fix this. Ok if I skip redoing the review for that? > > --- > > hotspot/test/runtime/6981737/Test6981737.java > > This test is really only valid for the new version scheme now, so it > should probably be rewritten - and in that case it really isn't > testing 6981737 but should be replaced by a new test for the new > version format. Yep, agree. Follow-up patch, right? > > --- > > jdk/src/java.base/share/classes/sun/misc/Version.java.template > > This comment is nonsensical: > > /** > ! * Returns the security version of the running JVM if it's 1.6 > or newer > * or any RE VM build. It will return 0 if it's an internal 1.5 or > * 1.4.x build. > * @since 1.6 > */ > > as security version does not exist pre 9. Normally you should be > adding a new method and deprecating the old one. The new one is @since 9. Yep, agree. Follow-up patch, right? > > /** > ! * Returns the security version of the running JDK. > * @since 1.6 > */ > > Ditto: @since 9 (but again old should be deprecated and new method added) > > 253 /** > 254 * Returns the build number of the running JDK if it's a RE > build > 255 * It will return 0 if it's an internal build. > > As with jvm.h this seems obsolete commentary these days - not only RE > builds define a build number. Yep, agree. Follow-up patch, right? /Magnus > > Thanks, > David From peter.levart at gmail.com Wed Jun 10 13:29:21 2015 From: peter.levart at gmail.com (Peter Levart) Date: Wed, 10 Jun 2015 15:29:21 +0200 Subject: RFR 9: 8086117: java/lang/Runtime/exec/LotsOfOutput.java still fails intermittently with Process consumes memory In-Reply-To: <5578098C.4030103@Oracle.com> References: <5578098C.4030103@Oracle.com> Message-ID: <55783BB1.2080306@gmail.com> Hi Roger, I don't know how the tests are executed in your environment, but is it possible that, while you're running the test in a shared VM, some other concurrent activity allocates from heap and that the test fails because of that and not because of Process.exec() at all? Regards, Peter On 06/10/2015 11:55 AM, Roger Riggs wrote: > Please review a test change in the algorithm for detecting memory > growth to > resolve an intermittent reporting problem. > > Webrev: > http://cr.openjdk.java.net/~rriggs/webrev-lots-8086117/ > > Issue: > https://bugs.openjdk.java.net/browse/JDK-8086117 > > Thanks, Roger From magnus.ihse.bursie at oracle.com Wed Jun 10 13:44:41 2015 From: magnus.ihse.bursie at oracle.com (Magnus Ihse Bursie) Date: Wed, 10 Jun 2015 15:44:41 +0200 Subject: RFR: JDK-8085822 JEP 223: New Version-String Scheme (initial integration) In-Reply-To: <557625C8.5050200@oracle.com> References: <5571AD18.7080701@oracle.com> <557625C8.5050200@oracle.com> Message-ID: <55783F49.6030906@oracle.com> On 2015-06-09 01:31, Daniel D. Daugherty wrote: > > > http://cr.openjdk.java.net/~ihse/JDK-8085822-JEP-223-initial-patch/webrev.01 > > > General comment: Not all copyright years were updated. I realize I missed that part of the review. :-( I have now updated the copyright years. Here's a delta review: http://cr.openjdk.java.net/~ihse/JDK-8085822-JEP-223-initial-patch-delta-02/webrev.01/ Here's the complete review once again: http://cr.openjdk.java.net/~ihse/JDK-8085822-JEP-223-initial-patch/webrev.03 Hopefully this is now the final version to be pushed to verona, and that any additional problems can be resolved with follow-up patches. /Magnus From mandy.chung at oracle.com Wed Jun 10 15:14:53 2015 From: mandy.chung at oracle.com (Mandy Chung) Date: Wed, 10 Jun 2015 08:14:53 -0700 Subject: RFR(M): 8081674: EmptyStackException at startup if running with extended or unsupported charset In-Reply-To: References: <557626AA.6010008@oracle.com> <55772A56.2030308@oracle.com> Message-ID: Have you checked out this patch moving out findBuiltinLib from NativeLibrary class? http://cr.openjdk.java.net/~mchung/jdk9/webrevs/8081674/webrev.00/ Mandy > On Jun 10, 2015, at 12:34 AM, Volker Simonis wrote: > > Mandy, the example/stacktrace I sent was with 8u. > > The problem isn't related to libzip, it is only the first library > which gets loaded with System.loadLibrary(). > > The problem will occur with every library which gets loaded by > System.loadLibrary() because > java.lang.ClassLoader$NativeLibrary.findBuiltinLib() always calls > jni_FindClass() if we're running on a unsupported locale. > > On Tue, Jun 9, 2015 at 8:03 PM, Mandy Chung wrote: >> Does my patch work fine on 8u? If it works fine, I prefer to get that >> simple fix in 8u and take the time to have a better fix in 9 (jdk9 is still >> under development and I have assumed that it's not a show stopper to you. >> Let me know if it blocks you). >> >> A question to Sherman - do we have adequate tests to verify the move of >> System.loadLibrary("zip") from system initialization to ZipFile >> initialization for 8u? >> >> Mandy >> >> >> On 06/09/2015 10:09 AM, Volker Simonis wrote: >>> >>> Hi Mandy, >>> >>> thanks for looking into this. >>> Uunfortunately your fix only helps to fix "java -version" :( >>> >>> Running even a minimal HelloWorld will still fail with the following >>> stack trace which is still caused by the same EmptyStackException: >>> >>> Error: A JNI error has occurred, please check your installation and try >>> again >>> Exception in thread "main" java.lang.ExceptionInInitializerError >>> at sun.misc.Unsafe.ensureClassInitialized(Native Method) >>> at >>> sun.misc.SharedSecrets.getJavaUtilZipFileAccess(SharedSecrets.java:158) >>> at sun.misc.URLClassPath$JarLoader.(URLClassPath.java:765) >>> at sun.misc.URLClassPath$3.run(URLClassPath.java:530) >>> at sun.misc.URLClassPath$3.run(URLClassPath.java:520) >>> at java.security.AccessController.doPrivileged(Native Method) >>> at sun.misc.URLClassPath.getLoader(URLClassPath.java:519) >>> at sun.misc.URLClassPath.getLoader(URLClassPath.java:492) >>> at sun.misc.URLClassPath.getNextLoader(URLClassPath.java:457) >>> at sun.misc.URLClassPath.access$100(URLClassPath.java:64) >>> at sun.misc.URLClassPath$1.next(URLClassPath.java:239) >>> at sun.misc.URLClassPath$1.hasMoreElements(URLClassPath.java:250) >>> at java.net.URLClassLoader$3$1.run(URLClassLoader.java:601) >>> at java.net.URLClassLoader$3$1.run(URLClassLoader.java:599) >>> at java.security.AccessController.doPrivileged(Native Method) >>> at java.net.URLClassLoader$3.next(URLClassLoader.java:598) >>> at java.net.URLClassLoader$3.hasMoreElements(URLClassLoader.java:623) >>> at sun.misc.CompoundEnumeration.next(CompoundEnumeration.java:45) >>> at >>> sun.misc.CompoundEnumeration.hasMoreElements(CompoundEnumeration.java:54) >>> at sun.misc.CompoundEnumeration.next(CompoundEnumeration.java:45) >>> at >>> sun.misc.CompoundEnumeration.hasMoreElements(CompoundEnumeration.java:54) >>> at >>> java.util.ServiceLoader$LazyIterator.hasNextService(ServiceLoader.java:354) >>> at >>> java.util.ServiceLoader$LazyIterator.hasNext(ServiceLoader.java:393) >>> at java.util.ServiceLoader$1.hasNext(ServiceLoader.java:474) >>> at java.nio.charset.Charset$1.getNext(Charset.java:350) >>> at java.nio.charset.Charset$1.hasNext(Charset.java:365) >>> at java.nio.charset.Charset$2.run(Charset.java:410) >>> at java.nio.charset.Charset$2.run(Charset.java:407) >>> at java.security.AccessController.doPrivileged(Native Method) >>> at java.nio.charset.Charset.lookupViaProviders(Charset.java:406) >>> at java.nio.charset.Charset.lookup2(Charset.java:477) >>> at java.nio.charset.Charset.lookup(Charset.java:464) >>> at java.nio.charset.Charset.isSupported(Charset.java:505) >>> at >>> sun.launcher.LauncherHelper.makePlatformString(LauncherHelper.java:580) >>> Caused by: java.util.EmptyStackException >>> at java.util.Stack.peek(Stack.java:102) >>> at >>> java.lang.ClassLoader$NativeLibrary.getFromClass(ClassLoader.java:1759) >>> at java.lang.ClassLoader$NativeLibrary.findBuiltinLib(Native Method) >>> at java.lang.ClassLoader.loadLibrary0(ClassLoader.java:1870) >>> at java.lang.ClassLoader.loadLibrary(ClassLoader.java:1843) >>> at java.lang.Runtime.loadLibrary0(Runtime.java:870) >>> at java.lang.System.loadLibrary(System.java:1122) >>> at java.util.zip.ZipFile.(ZipFile.java:86) >>> ... 34 more >>> >>> It's obvious that the way jni_FindClass is looking for the class >>> context by calling the NativeLibrary::getFromClass method is hacky but >>> I think that the proposed fix is the quite simple and non-intrusive. >>> And we probably don't want a complete rework of this code for 8 >>> anyway. So why not fix it the proposed way in 8 and 9 for now? That >>> will still leave us time to come up with a better clean-up at least >>> for 9. >>> >>> What do you think? >>> >>> Regards, >>> Volker >>> >>> On Tue, Jun 9, 2015 at 1:35 AM, Mandy Chung >>> wrote: >>>> >>>> Hi Volker, >>>> >>>> Your patch reminds me the question I have about loading zip library. >>>> >>>> In JDK 9 (and earlier release), zip library is loaded by the VM during >>>> startup (see ClassLoader::load_zip_library). I think loadLibrary("zip") >>>> is >>>> no longer needed to be called from System::initializeSystemClass method >>>> and >>>> instead it can be loaded by java.util.zip.ZipFile static initializer. >>>> >>>> Do you mind to try the patch (below)? That may be a simpler fix. >>>> >>>> I never like the way jni_FindClass to look for the class context by >>>> calling >>>> the NativeLibrary::getFromClass method. I will have to look deeper other >>>> alternative to clean that up. If taking out loadLibrary("zip") resolves >>>> your issue, this will give us time to come up with a better clean-up in >>>> the >>>> future. >>>> >>>> Mandy >>>> [1] https://bugs.openjdk.java.net/browse/JDK-4429040 >>>> >>>> diff --git a/src/share/classes/java/lang/System.java >>>> b/src/share/classes/java/lang/System.java >>>> --- a/src/share/classes/java/lang/System.java >>>> +++ b/src/share/classes/java/lang/System.java >>>> @@ -1192,10 +1192,6 @@ >>>> setOut0(newPrintStream(fdOut, >>>> props.getProperty("sun.stdout.encoding"))); >>>> setErr0(newPrintStream(fdErr, >>>> props.getProperty("sun.stderr.encoding"))); >>>> >>>> - // Load the zip library now in order to keep >>>> java.util.zip.ZipFile >>>> - // from trying to use itself to load this library later. >>>> - loadLibrary("zip"); >>>> - >>>> // Setup Java signal handlers for HUP, TERM, and INT (where >>>> available). >>>> Terminator.setup(); >>>> >>>> diff --git a/src/share/classes/java/util/zip/ZipFile.java >>>> b/src/share/classes/java/util/zip/ZipFile.java >>>> --- a/src/share/classes/java/util/zip/ZipFile.java >>>> +++ b/src/share/classes/java/util/zip/ZipFile.java >>>> @@ -83,6 +83,7 @@ >>>> >>>> static { >>>> /* Zip library is loaded from System.initializeSystemClass */ >>>> + System.loadLibrary("zip"); >>>> initIDs(); >>>> >>>> } >>>> >>>> >>>> >>>> On 06/08/2015 07:23 AM, Volker Simonis wrote: >>>>> >>>>> Hi, >>>>> >>>>> can I please get a review at least for the part of this fix which is >>>>> common for jdk8 and jdk9. It's only a few lines in >>>>> src/share/vm/prims/jni.cpp for the hotspot repository and one line >>>>> src/java.base/share/classes/java/lang/ClassLoader.java for the jdk >>>>> repository. >>>>> >>>>> Thanks, >>>>> Volker >>>>> >>>>> >>>>> On Tue, Jun 2, 2015 at 6:12 PM, Volker Simonis >>>>> >>>>> wrote: >>>>>> >>>>>> Hi, >>>>>> >>>>>> can I please have review (and a sponsor) for these changes: >>>>>> >>>>>> https://bugs.openjdk.java.net/browse/JDK-8081674 >>>>>> http://cr.openjdk.java.net/~simonis/webrevs/2015/8081674.jdk >>>>>> http://cr.openjdk.java.net/~simonis/webrevs/2015/8081674.hs >>>>>> >>>>>> Please notice that the fix requires synchronous changes in the jdk and >>>>>> the >>>>>> hotspot forest. >>>>>> >>>>>> The changes themselves are by far not that big as this explanation but >>>>>> I >>>>>> found the problem to be quite intricate so I tried to explain it as >>>>>> good >>>>>> as >>>>>> I could. I'd suggest to read the HTML-formatted explanation in the >>>>>> webrev >>>>>> although the content is equal to the one in this mail: >>>>>> >>>>>> Using an unsupported character encoding (e.g. vi_VN.TCVN on Linux) >>>>>> results >>>>>> in an immediate VM failure with jdk 8 and 9: >>>>>> >>>>>> export LANG=vi_VN.TCVN >>>>>> java -version >>>>>> Error occurred during initialization of VM >>>>>> java.util.EmptyStackException >>>>>> at java.util.Stack.peek(Stack.java:102) >>>>>> at >>>>>> java.lang.ClassLoader$NativeLibrary.getFromClass(ClassLoader.java:1751) >>>>>> at java.lang.ClassLoader$NativeLibrary.findBuiltinLib(Native >>>>>> Method) >>>>>> at java.lang.ClassLoader.loadLibrary0(ClassLoader.java:1862) >>>>>> at java.lang.ClassLoader.loadLibrary(ClassLoader.java:1835) >>>>>> at java.lang.Runtime.loadLibrary0(Runtime.java:870) >>>>>> at java.lang.System.loadLibrary(System.java:1119) >>>>>> at java.lang.System.initializeSystemClass(System.java:1194) >>>>>> >>>>>> This is a consequence of "8005716: Enhance JNI specification to allow >>>>>> support of static JNI libraries in Embedded JREs". >>>>>> >>>>>> With jdk 9 we get this error even if we're running with a supported >>>>>> charset >>>>>> which is in the ExtendedCharsets (as opposed to being in the >>>>>> StandardCharsets) class which is a consequence of delegating the >>>>>> loading >>>>>> of >>>>>> the ExtendedCharsets class to the ServiceLoader in jdk 9. >>>>>> >>>>>> export LANG=eo.iso-8859-3 >>>>>> output-jdk9/images/jdk/bin/java -version >>>>>> Error occurred during initialization of VM >>>>>> java.util.EmptyStackException >>>>>> at java.util.Stack.peek(Stack.java:102) >>>>>> at >>>>>> java.lang.ClassLoader$NativeLibrary.getFromClass(ClassLoader.java:1737) >>>>>> at java.lang.ClassLoader$NativeLibrary.findBuiltinLib(Native >>>>>> Method) >>>>>> at java.lang.ClassLoader.loadLibrary0(ClassLoader.java:1866) >>>>>> at java.lang.ClassLoader.loadLibrary(ClassLoader.java:1840) >>>>>> at java.lang.Runtime.loadLibrary0(Runtime.java:874) >>>>>> at java.lang.System.loadLibrary(System.java:1111) >>>>>> at java.lang.System.initializeSystemClass(System.java:1186) >>>>>> >>>>>> Here's why the exception happens for an unsupported charset (see the >>>>>> mixed >>>>>> stack trace below for the full details): >>>>>> >>>>>> java.lang.System.loadLibrary() wants to load libzip.so. It calls >>>>>> java.lang.Runtime.loadLibrary0() which at the very beginning calls the >>>>>> native method ClassLoader$NativeLibrary.findBuiltinLib() which checks >>>>>> if >>>>>> the >>>>>> corresponding library is already statically linked into the VM >>>>>> (introduced >>>>>> by 8005716). >>>>>> Java_java_lang_ClassLoader_00024NativeLibrary_findBuiltinLib(), >>>>>> the native implementation of findBuiltinLib() in Classloader.c calls >>>>>> GetStringPlatformChars() to convert the library name into the native >>>>>> platform encoding. GetStringPlatformChars() calls the helper function >>>>>> jnuEncodingSupported() to check if the platform encoding which is >>>>>> stored >>>>>> in >>>>>> the property "sun.jnu.encoding" is supported by Java. >>>>>> jnuEncodingSupported() >>>>>> is implemented as follows: >>>>>> >>>>>> static jboolean isJNUEncodingSupported = JNI_FALSE; >>>>>> static jboolean jnuEncodingSupported(JNIEnv *env) { >>>>>> jboolean exe; >>>>>> if (isJNUEncodingSupported == JNI_TRUE) { >>>>>> return JNI_TRUE; >>>>>> } >>>>>> isJNUEncodingSupported = (jboolean) JNU_CallStaticMethodByName ( >>>>>> env, &exe, >>>>>> "java/nio/charset/Charset", >>>>>> "isSupported", >>>>>> "(Ljava/lang/String;)Z", >>>>>> jnuEncoding).z; >>>>>> return isJNUEncodingSupported; >>>>>> } >>>>>> >>>>>> Once the function finds that the platform encoding is supported (by >>>>>> calling >>>>>> java.nio.charset.Charset.isSupported()) it caches this value and always >>>>>> returns it on following calls. However if the platform encoding is not >>>>>> supported, it ALWAYS calls java.nio.charset.Charset.isSupported() an >>>>>> every >>>>>> subsequent invocation. >>>>>> >>>>>> In order to call the Java method Charset.isSupported() (in >>>>>> JNU_CallStaticMethodByName() in file jni_util.c), we have to call >>>>>> jni_FindClass() to convert the symbolic class name >>>>>> "java.nio.charset.Charset" into a class reference. >>>>>> >>>>>> But unfortunately, jni_FindClass() (from jni.cpp in libjvm.so) has a >>>>>> special >>>>>> handling if called from java.lang.ClassLoader$NativeLibrary to ensure >>>>>> that >>>>>> JNI_OnLoad/JNI_OnUnload are executed in the correct class context: >>>>>> >>>>>> instanceKlassHandle k (THREAD, >>>>>> thread->security_get_caller_class(0)); >>>>>> if (k.not_null()) { >>>>>> loader = Handle(THREAD, k->class_loader()); >>>>>> // Special handling to make sure JNI_OnLoad and JNI_OnUnload are >>>>>> executed >>>>>> // in the correct class context. >>>>>> if (loader.is_null() && >>>>>> k->name() == >>>>>> vmSymbols::java_lang_ClassLoader_NativeLibrary()) { >>>>>> JavaValue result(T_OBJECT); >>>>>> JavaCalls::call_static(&result, k, >>>>>> vmSymbols::getFromClass_name(), >>>>>> >>>>>> vmSymbols::void_class_signature(), >>>>>> thread); >>>>>> if (HAS_PENDING_EXCEPTION) { >>>>>> Handle ex(thread, thread->pending_exception()); >>>>>> CLEAR_PENDING_EXCEPTION; >>>>>> THROW_HANDLE_0(ex); >>>>>> } >>>>>> >>>>>> So if that's the case and jni_FindClass() was reallycalled from >>>>>> ClassLoader$NativeLibrary, then jni_FindClass() calles >>>>>> ClassLoader$NativeLibrary().getFromClass() to find out the >>>>>> corresponding >>>>>> context class which is supposed to be saved there in a field of type >>>>>> java.util.Stack named "nativeLibraryContext": >>>>>> >>>>>> // Invoked in the VM to determine the context class in >>>>>> // JNI_Load/JNI_Unload >>>>>> static Class getFromClass() { >>>>>> return ClassLoader.nativeLibraryContext.peek().fromClass; >>>>>> } >>>>>> >>>>>> Unfortunately, "nativeLibraryContext" doesn't contain any entry at this >>>>>> point and the invocation of Stack.peek() will throw the exception shown >>>>>> before. In general, the "nativeLibraryContext" stack will be filled >>>>>> later >>>>>> on >>>>>> in Runtime.loadLibrary0() like this: >>>>>> >>>>>> NativeLibrary lib = new NativeLibrary(fromClass, name, isBuiltin); >>>>>> nativeLibraryContext.push(lib); >>>>>> try { >>>>>> lib.load(name, isBuiltin); >>>>>> } finally { >>>>>> nativeLibraryContext.pop(); >>>>>> } >>>>>> >>>>>> such that it always contains at least one element later when >>>>>> jni_FindClass() >>>>>> will be invoked. >>>>>> >>>>>> So in summary, the problem is that the implementors of 8005716 didn't >>>>>> took >>>>>> into account that calling ClassLoader$NativeLibrary.findBuiltinLib() >>>>>> may >>>>>> trigger a call to jni_FindClass() if we are running on a system with an >>>>>> unsupported character encoding. >>>>>> >>>>>> I'd suggest the following fix for this problem: >>>>>> >>>>>> Change ClassLoader$NativeLibrary().getFromClass() to return null if the >>>>>> stack is empty instead of throwing an exception: >>>>>> >>>>>> static Class getFromClass() { >>>>>> return ClassLoader.nativeLibraryContext.empty() ? >>>>>> null : ClassLoader.nativeLibraryContext.peek().fromClass; >>>>>> } >>>>>> >>>>>> Unfortunately this also requires a HotSpot change in jni_FindClass() in >>>>>> order to properly handle the new 'null' return value: >>>>>> >>>>>> if (k.not_null()) { >>>>>> loader = Handle(THREAD, k->class_loader()); >>>>>> // Special handling to make sure JNI_OnLoad and JNI_OnUnload are >>>>>> executed >>>>>> // in the correct class context. >>>>>> if (loader.is_null() && >>>>>> k->name() == >>>>>> vmSymbols::java_lang_ClassLoader_NativeLibrary()) { >>>>>> JavaValue result(T_OBJECT); >>>>>> JavaCalls::call_static(&result, k, >>>>>> vmSymbols::getFromClass_name(), >>>>>> >>>>>> vmSymbols::void_class_signature(), >>>>>> thread); >>>>>> if (HAS_PENDING_EXCEPTION) { >>>>>> Handle ex(thread, thread->pending_exception()); >>>>>> CLEAR_PENDING_EXCEPTION; >>>>>> THROW_HANDLE_0(ex); >>>>>> } >>>>>> oop mirror = (oop) result.get_jobject(); >>>>>> if (oopDesc::is_null(mirror)) { >>>>>> loader = Handle(THREAD, >>>>>> SystemDictionary::java_system_loader()); >>>>>> } else { >>>>>> loader = Handle(THREAD, >>>>>> >>>>>> >>>>>> InstanceKlass::cast(java_lang_Class::as_Klass(mirror))->class_loader()); >>>>>> protection_domain = Handle(THREAD, >>>>>> >>>>>> >>>>>> >>>>>> InstanceKlass::cast(java_lang_Class::as_Klass(mirror))->protection_domain()); >>>>>> } >>>>>> } >>>>>> } else { >>>>>> // We call ClassLoader.getSystemClassLoader to obtain the system >>>>>> class >>>>>> loader. >>>>>> loader = Handle(THREAD, SystemDictionary::java_system_loader()); >>>>>> } >>>>>> >>>>>> These changes are sufficient to solve the problem in Java 8. >>>>>> Unfortunately, >>>>>> that's still not enough in Java 9 because there the loading of the >>>>>> extended >>>>>> charsets has been delegated to ServiceLoader. But ServiceLoader calls >>>>>> ClassLoader.getBootstrapResources() which calls >>>>>> sun.misc.Launcher.getBootstrapClassPath(). This leads to another >>>>>> problem >>>>>> during class initialization of sun.misc.Launcher if running on an >>>>>> unsupported locale. >>>>>> >>>>>> The first thing done in sun.misc.Launcher. is the >>>>>> initialisation >>>>>> of >>>>>> the bootstrap URLClassPath in the Launcher. However this initialisation >>>>>> will >>>>>> eventually call Charset.isSupported() and if we are running on an >>>>>> unsupported locale this will inevitably end in another recursive call >>>>>> to >>>>>> ServiceLoader. But as explained below, ServiceLoader will query the >>>>>> Launcher's bootstrap URLClassPath which will be still uninitialized at >>>>>> that >>>>>> point. >>>>>> >>>>>> So we'll have to additionally guard guard against this situation on JDK >>>>>> 9 >>>>>> like this: >>>>>> >>>>>> private static Charset lookupExtendedCharset(String charsetName) { >>>>>> if (!sun.misc.VM.isBooted() || // see >>>>>> lookupViaProviders() >>>>>> sun.misc.Launcher.getBootstrapClassPath() == null) >>>>>> return null; >>>>>> >>>>>> This fixes the crashes, but still at the price of not having the >>>>>> extended >>>>>> charsets available during initialization until >>>>>> Launcher.getBootstrapClassPath is set up properly. This may be still a >>>>>> problem if the jdk is installed in a directory which contains >>>>>> characters >>>>>> specific to an extended encoding or if we have such characters in the >>>>>> command line arguments. >>>>>> >>>>>> Thank you and best regards, >>>>>> Volker >>>>>> >>>>>> >>>>>> Mixed stack trace of the initial EmptyStackException for unsupported >>>>>> charsets described before: >>>>>> >>>>>> Native frames: (J=compiled Java code, j=interpreted, Vv=VM code, >>>>>> C=native >>>>>> code) >>>>>> j java.util.Stack.peek()Ljava/lang/Object;+1 >>>>>> j >>>>>> java.lang.ClassLoader$NativeLibrary.getFromClass()Ljava/lang/Class;+3 >>>>>> v ~StubRoutines::call_stub >>>>>> V [libjvm.so+0x9d279a] JavaCalls::call_helper(JavaValue*, >>>>>> methodHandle*, >>>>>> JavaCallArguments*, Thread*)+0x6b4 >>>>>> V [libjvm.so+0xcad591] os::os_exception_wrapper(void (*)(JavaValue*, >>>>>> methodHandle*, JavaCallArguments*, Thread*), JavaValue*, methodHandle*, >>>>>> JavaCallArguments*, Thread*)+0x45 >>>>>> V [libjvm.so+0x9d20cf] JavaCalls::call(JavaValue*, methodHandle, >>>>>> JavaCallArguments*, Thread*)+0x8b >>>>>> V [libjvm.so+0x9d1d3b] JavaCalls::call_static(JavaValue*, >>>>>> KlassHandle, >>>>>> Symbol*, Symbol*, JavaCallArguments*, Thread*)+0x139 >>>>>> V [libjvm.so+0x9d1e3f] JavaCalls::call_static(JavaValue*, >>>>>> KlassHandle, >>>>>> Symbol*, Symbol*, Thread*)+0x9d >>>>>> V [libjvm.so+0x9e6588] jni_FindClass+0x428 >>>>>> C [libjava.so+0x20208] JNU_CallStaticMethodByName+0xff >>>>>> C [libjava.so+0x21cae] jnuEncodingSupported+0x61 >>>>>> C [libjava.so+0x22125] JNU_GetStringPlatformChars+0x125 >>>>>> C [libjava.so+0xedcd] >>>>>> Java_java_lang_ClassLoader_00024NativeLibrary_findBuiltinLib+0x8b >>>>>> j >>>>>> >>>>>> >>>>>> java.lang.ClassLoader$NativeLibrary.findBuiltinLib(Ljava/lang/String;)Ljava/lang/String;+0 >>>>>> j >>>>>> java.lang.ClassLoader.loadLibrary0(Ljava/lang/Class;Ljava/io/File;)Z+4 >>>>>> j >>>>>> >>>>>> >>>>>> java.lang.ClassLoader.loadLibrary(Ljava/lang/Class;Ljava/lang/String;Z)V+228 >>>>>> j >>>>>> java.lang.Runtime.loadLibrary0(Ljava/lang/Class;Ljava/lang/String;)V+54 >>>>>> j java.lang.System.loadLibrary(Ljava/lang/String;)V+7 >>>>>> j java.lang.System.initializeSystemClass()V+113 >>>>>> v ~StubRoutines::call_stub >>>>>> V [libjvm.so+0x9d279a] JavaCalls::call_helper(JavaValue*, >>>>>> methodHandle*, >>>>>> JavaCallArguments*, Thread*)+0x6b4 >>>>>> V [libjvm.so+0xcad591] os::os_exception_wrapper(void (*)(JavaValue*, >>>>>> methodHandle*, JavaCallArguments*, Thread*), JavaValue*, methodHandle*, >>>>>> JavaCallArguments*, Thread*)+0x45 >>>>>> V [libjvm.so+0x9d20cf] JavaCalls::call(JavaValue*, methodHandle, >>>>>> JavaCallArguments*, Thread*)+0x8b >>>>>> V [libjvm.so+0x9d1d3b] JavaCalls::call_static(JavaValue*, >>>>>> KlassHandle, >>>>>> Symbol*, Symbol*, JavaCallArguments*, Thread*)+0x139 >>>>>> V [libjvm.so+0x9d1e3f] JavaCalls::call_static(JavaValue*, >>>>>> KlassHandle, >>>>>> Symbol*, Symbol*, Thread*)+0x9d >>>>>> V [libjvm.so+0xe3cceb] call_initializeSystemClass(Thread*)+0xb0 >>>>>> V [libjvm.so+0xe44444] >>>>>> Threads::initialize_java_lang_classes(JavaThread*, >>>>>> Thread*)+0x21a >>>>>> V [libjvm.so+0xe44b12] Threads::create_vm(JavaVMInitArgs*, >>>>>> bool*)+0x4a6 >>>>>> V [libjvm.so+0xa19bd7] JNI_CreateJavaVM+0xc7 >>>>>> C [libjli.so+0xa520] InitializeJVM+0x154 >>>>>> C [libjli.so+0x8024] JavaMain+0xcc >>>>>> >>>>>> >> From volker.simonis at gmail.com Wed Jun 10 16:23:47 2015 From: volker.simonis at gmail.com (Volker Simonis) Date: Wed, 10 Jun 2015 18:23:47 +0200 Subject: RFR(M): 8081674: EmptyStackException at startup if running with extended or unsupported charset In-Reply-To: References: <557626AA.6010008@oracle.com> <55772A56.2030308@oracle.com> Message-ID: Hy Mandy, that's a real good proposal and it only requires changes in the jdk repository which makes it even more attractive. I've just checked and it does indeed solve the initial EmptyStackException problem in both jdk8 and jdk9. Here are the corresponding webrevs: http://cr.openjdk.java.net/~simonis/webrevs/2015/8081674.jdk8/ http://cr.openjdk.java.net/~simonis/webrevs/2015/8081674.jdk9/ The only difference between 8 and 9 is the different location of the source files. Compared to your proposal I've only updated the copyright years where necessary and removed one comment in Java_java_lang_ClassLoader_findBuiltinLib() which seemed unnecessary to me now that findBuiltinLib() isn't located in NativeLibrary anymore. On jdk9 we are now running in another problem which is caused by the fact that the extended charsets are now being loaded by ServiceLoader (I've detailed that already in the bug report). But I think we can leave the fix of that problem to another change as this seems to require some more reasoning (see Alan's comments in the bug report). So are you OK now with pushing this change to jdk9 and requesting a downport to 8u-dev afterwards? Thank you and best regards, Volker On Wed, Jun 10, 2015 at 5:14 PM, Mandy Chung wrote: > Have you checked out this patch moving out findBuiltinLib from NativeLibrary class? > http://cr.openjdk.java.net/~mchung/jdk9/webrevs/8081674/webrev.00/ > > Mandy > >> On Jun 10, 2015, at 12:34 AM, Volker Simonis wrote: >> >> Mandy, the example/stacktrace I sent was with 8u. >> >> The problem isn't related to libzip, it is only the first library >> which gets loaded with System.loadLibrary(). >> >> The problem will occur with every library which gets loaded by >> System.loadLibrary() because >> java.lang.ClassLoader$NativeLibrary.findBuiltinLib() always calls >> jni_FindClass() if we're running on a unsupported locale. >> >> On Tue, Jun 9, 2015 at 8:03 PM, Mandy Chung wrote: >>> Does my patch work fine on 8u? If it works fine, I prefer to get that >>> simple fix in 8u and take the time to have a better fix in 9 (jdk9 is still >>> under development and I have assumed that it's not a show stopper to you. >>> Let me know if it blocks you). >>> >>> A question to Sherman - do we have adequate tests to verify the move of >>> System.loadLibrary("zip") from system initialization to ZipFile >>> initialization for 8u? >>> >>> Mandy >>> >>> >>> On 06/09/2015 10:09 AM, Volker Simonis wrote: >>>> >>>> Hi Mandy, >>>> >>>> thanks for looking into this. >>>> Uunfortunately your fix only helps to fix "java -version" :( >>>> >>>> Running even a minimal HelloWorld will still fail with the following >>>> stack trace which is still caused by the same EmptyStackException: >>>> >>>> Error: A JNI error has occurred, please check your installation and try >>>> again >>>> Exception in thread "main" java.lang.ExceptionInInitializerError >>>> at sun.misc.Unsafe.ensureClassInitialized(Native Method) >>>> at >>>> sun.misc.SharedSecrets.getJavaUtilZipFileAccess(SharedSecrets.java:158) >>>> at sun.misc.URLClassPath$JarLoader.(URLClassPath.java:765) >>>> at sun.misc.URLClassPath$3.run(URLClassPath.java:530) >>>> at sun.misc.URLClassPath$3.run(URLClassPath.java:520) >>>> at java.security.AccessController.doPrivileged(Native Method) >>>> at sun.misc.URLClassPath.getLoader(URLClassPath.java:519) >>>> at sun.misc.URLClassPath.getLoader(URLClassPath.java:492) >>>> at sun.misc.URLClassPath.getNextLoader(URLClassPath.java:457) >>>> at sun.misc.URLClassPath.access$100(URLClassPath.java:64) >>>> at sun.misc.URLClassPath$1.next(URLClassPath.java:239) >>>> at sun.misc.URLClassPath$1.hasMoreElements(URLClassPath.java:250) >>>> at java.net.URLClassLoader$3$1.run(URLClassLoader.java:601) >>>> at java.net.URLClassLoader$3$1.run(URLClassLoader.java:599) >>>> at java.security.AccessController.doPrivileged(Native Method) >>>> at java.net.URLClassLoader$3.next(URLClassLoader.java:598) >>>> at java.net.URLClassLoader$3.hasMoreElements(URLClassLoader.java:623) >>>> at sun.misc.CompoundEnumeration.next(CompoundEnumeration.java:45) >>>> at >>>> sun.misc.CompoundEnumeration.hasMoreElements(CompoundEnumeration.java:54) >>>> at sun.misc.CompoundEnumeration.next(CompoundEnumeration.java:45) >>>> at >>>> sun.misc.CompoundEnumeration.hasMoreElements(CompoundEnumeration.java:54) >>>> at >>>> java.util.ServiceLoader$LazyIterator.hasNextService(ServiceLoader.java:354) >>>> at >>>> java.util.ServiceLoader$LazyIterator.hasNext(ServiceLoader.java:393) >>>> at java.util.ServiceLoader$1.hasNext(ServiceLoader.java:474) >>>> at java.nio.charset.Charset$1.getNext(Charset.java:350) >>>> at java.nio.charset.Charset$1.hasNext(Charset.java:365) >>>> at java.nio.charset.Charset$2.run(Charset.java:410) >>>> at java.nio.charset.Charset$2.run(Charset.java:407) >>>> at java.security.AccessController.doPrivileged(Native Method) >>>> at java.nio.charset.Charset.lookupViaProviders(Charset.java:406) >>>> at java.nio.charset.Charset.lookup2(Charset.java:477) >>>> at java.nio.charset.Charset.lookup(Charset.java:464) >>>> at java.nio.charset.Charset.isSupported(Charset.java:505) >>>> at >>>> sun.launcher.LauncherHelper.makePlatformString(LauncherHelper.java:580) >>>> Caused by: java.util.EmptyStackException >>>> at java.util.Stack.peek(Stack.java:102) >>>> at >>>> java.lang.ClassLoader$NativeLibrary.getFromClass(ClassLoader.java:1759) >>>> at java.lang.ClassLoader$NativeLibrary.findBuiltinLib(Native Method) >>>> at java.lang.ClassLoader.loadLibrary0(ClassLoader.java:1870) >>>> at java.lang.ClassLoader.loadLibrary(ClassLoader.java:1843) >>>> at java.lang.Runtime.loadLibrary0(Runtime.java:870) >>>> at java.lang.System.loadLibrary(System.java:1122) >>>> at java.util.zip.ZipFile.(ZipFile.java:86) >>>> ... 34 more >>>> >>>> It's obvious that the way jni_FindClass is looking for the class >>>> context by calling the NativeLibrary::getFromClass method is hacky but >>>> I think that the proposed fix is the quite simple and non-intrusive. >>>> And we probably don't want a complete rework of this code for 8 >>>> anyway. So why not fix it the proposed way in 8 and 9 for now? That >>>> will still leave us time to come up with a better clean-up at least >>>> for 9. >>>> >>>> What do you think? >>>> >>>> Regards, >>>> Volker >>>> >>>> On Tue, Jun 9, 2015 at 1:35 AM, Mandy Chung >>>> wrote: >>>>> >>>>> Hi Volker, >>>>> >>>>> Your patch reminds me the question I have about loading zip library. >>>>> >>>>> In JDK 9 (and earlier release), zip library is loaded by the VM during >>>>> startup (see ClassLoader::load_zip_library). I think loadLibrary("zip") >>>>> is >>>>> no longer needed to be called from System::initializeSystemClass method >>>>> and >>>>> instead it can be loaded by java.util.zip.ZipFile static initializer. >>>>> >>>>> Do you mind to try the patch (below)? That may be a simpler fix. >>>>> >>>>> I never like the way jni_FindClass to look for the class context by >>>>> calling >>>>> the NativeLibrary::getFromClass method. I will have to look deeper other >>>>> alternative to clean that up. If taking out loadLibrary("zip") resolves >>>>> your issue, this will give us time to come up with a better clean-up in >>>>> the >>>>> future. >>>>> >>>>> Mandy >>>>> [1] https://bugs.openjdk.java.net/browse/JDK-4429040 >>>>> >>>>> diff --git a/src/share/classes/java/lang/System.java >>>>> b/src/share/classes/java/lang/System.java >>>>> --- a/src/share/classes/java/lang/System.java >>>>> +++ b/src/share/classes/java/lang/System.java >>>>> @@ -1192,10 +1192,6 @@ >>>>> setOut0(newPrintStream(fdOut, >>>>> props.getProperty("sun.stdout.encoding"))); >>>>> setErr0(newPrintStream(fdErr, >>>>> props.getProperty("sun.stderr.encoding"))); >>>>> >>>>> - // Load the zip library now in order to keep >>>>> java.util.zip.ZipFile >>>>> - // from trying to use itself to load this library later. >>>>> - loadLibrary("zip"); >>>>> - >>>>> // Setup Java signal handlers for HUP, TERM, and INT (where >>>>> available). >>>>> Terminator.setup(); >>>>> >>>>> diff --git a/src/share/classes/java/util/zip/ZipFile.java >>>>> b/src/share/classes/java/util/zip/ZipFile.java >>>>> --- a/src/share/classes/java/util/zip/ZipFile.java >>>>> +++ b/src/share/classes/java/util/zip/ZipFile.java >>>>> @@ -83,6 +83,7 @@ >>>>> >>>>> static { >>>>> /* Zip library is loaded from System.initializeSystemClass */ >>>>> + System.loadLibrary("zip"); >>>>> initIDs(); >>>>> >>>>> } >>>>> >>>>> >>>>> >>>>> On 06/08/2015 07:23 AM, Volker Simonis wrote: >>>>>> >>>>>> Hi, >>>>>> >>>>>> can I please get a review at least for the part of this fix which is >>>>>> common for jdk8 and jdk9. It's only a few lines in >>>>>> src/share/vm/prims/jni.cpp for the hotspot repository and one line >>>>>> src/java.base/share/classes/java/lang/ClassLoader.java for the jdk >>>>>> repository. >>>>>> >>>>>> Thanks, >>>>>> Volker >>>>>> >>>>>> >>>>>> On Tue, Jun 2, 2015 at 6:12 PM, Volker Simonis >>>>>> >>>>>> wrote: >>>>>>> >>>>>>> Hi, >>>>>>> >>>>>>> can I please have review (and a sponsor) for these changes: >>>>>>> >>>>>>> https://bugs.openjdk.java.net/browse/JDK-8081674 >>>>>>> http://cr.openjdk.java.net/~simonis/webrevs/2015/8081674.jdk >>>>>>> http://cr.openjdk.java.net/~simonis/webrevs/2015/8081674.hs >>>>>>> >>>>>>> Please notice that the fix requires synchronous changes in the jdk and >>>>>>> the >>>>>>> hotspot forest. >>>>>>> >>>>>>> The changes themselves are by far not that big as this explanation but >>>>>>> I >>>>>>> found the problem to be quite intricate so I tried to explain it as >>>>>>> good >>>>>>> as >>>>>>> I could. I'd suggest to read the HTML-formatted explanation in the >>>>>>> webrev >>>>>>> although the content is equal to the one in this mail: >>>>>>> >>>>>>> Using an unsupported character encoding (e.g. vi_VN.TCVN on Linux) >>>>>>> results >>>>>>> in an immediate VM failure with jdk 8 and 9: >>>>>>> >>>>>>> export LANG=vi_VN.TCVN >>>>>>> java -version >>>>>>> Error occurred during initialization of VM >>>>>>> java.util.EmptyStackException >>>>>>> at java.util.Stack.peek(Stack.java:102) >>>>>>> at >>>>>>> java.lang.ClassLoader$NativeLibrary.getFromClass(ClassLoader.java:1751) >>>>>>> at java.lang.ClassLoader$NativeLibrary.findBuiltinLib(Native >>>>>>> Method) >>>>>>> at java.lang.ClassLoader.loadLibrary0(ClassLoader.java:1862) >>>>>>> at java.lang.ClassLoader.loadLibrary(ClassLoader.java:1835) >>>>>>> at java.lang.Runtime.loadLibrary0(Runtime.java:870) >>>>>>> at java.lang.System.loadLibrary(System.java:1119) >>>>>>> at java.lang.System.initializeSystemClass(System.java:1194) >>>>>>> >>>>>>> This is a consequence of "8005716: Enhance JNI specification to allow >>>>>>> support of static JNI libraries in Embedded JREs". >>>>>>> >>>>>>> With jdk 9 we get this error even if we're running with a supported >>>>>>> charset >>>>>>> which is in the ExtendedCharsets (as opposed to being in the >>>>>>> StandardCharsets) class which is a consequence of delegating the >>>>>>> loading >>>>>>> of >>>>>>> the ExtendedCharsets class to the ServiceLoader in jdk 9. >>>>>>> >>>>>>> export LANG=eo.iso-8859-3 >>>>>>> output-jdk9/images/jdk/bin/java -version >>>>>>> Error occurred during initialization of VM >>>>>>> java.util.EmptyStackException >>>>>>> at java.util.Stack.peek(Stack.java:102) >>>>>>> at >>>>>>> java.lang.ClassLoader$NativeLibrary.getFromClass(ClassLoader.java:1737) >>>>>>> at java.lang.ClassLoader$NativeLibrary.findBuiltinLib(Native >>>>>>> Method) >>>>>>> at java.lang.ClassLoader.loadLibrary0(ClassLoader.java:1866) >>>>>>> at java.lang.ClassLoader.loadLibrary(ClassLoader.java:1840) >>>>>>> at java.lang.Runtime.loadLibrary0(Runtime.java:874) >>>>>>> at java.lang.System.loadLibrary(System.java:1111) >>>>>>> at java.lang.System.initializeSystemClass(System.java:1186) >>>>>>> >>>>>>> Here's why the exception happens for an unsupported charset (see the >>>>>>> mixed >>>>>>> stack trace below for the full details): >>>>>>> >>>>>>> java.lang.System.loadLibrary() wants to load libzip.so. It calls >>>>>>> java.lang.Runtime.loadLibrary0() which at the very beginning calls the >>>>>>> native method ClassLoader$NativeLibrary.findBuiltinLib() which checks >>>>>>> if >>>>>>> the >>>>>>> corresponding library is already statically linked into the VM >>>>>>> (introduced >>>>>>> by 8005716). >>>>>>> Java_java_lang_ClassLoader_00024NativeLibrary_findBuiltinLib(), >>>>>>> the native implementation of findBuiltinLib() in Classloader.c calls >>>>>>> GetStringPlatformChars() to convert the library name into the native >>>>>>> platform encoding. GetStringPlatformChars() calls the helper function >>>>>>> jnuEncodingSupported() to check if the platform encoding which is >>>>>>> stored >>>>>>> in >>>>>>> the property "sun.jnu.encoding" is supported by Java. >>>>>>> jnuEncodingSupported() >>>>>>> is implemented as follows: >>>>>>> >>>>>>> static jboolean isJNUEncodingSupported = JNI_FALSE; >>>>>>> static jboolean jnuEncodingSupported(JNIEnv *env) { >>>>>>> jboolean exe; >>>>>>> if (isJNUEncodingSupported == JNI_TRUE) { >>>>>>> return JNI_TRUE; >>>>>>> } >>>>>>> isJNUEncodingSupported = (jboolean) JNU_CallStaticMethodByName ( >>>>>>> env, &exe, >>>>>>> "java/nio/charset/Charset", >>>>>>> "isSupported", >>>>>>> "(Ljava/lang/String;)Z", >>>>>>> jnuEncoding).z; >>>>>>> return isJNUEncodingSupported; >>>>>>> } >>>>>>> >>>>>>> Once the function finds that the platform encoding is supported (by >>>>>>> calling >>>>>>> java.nio.charset.Charset.isSupported()) it caches this value and always >>>>>>> returns it on following calls. However if the platform encoding is not >>>>>>> supported, it ALWAYS calls java.nio.charset.Charset.isSupported() an >>>>>>> every >>>>>>> subsequent invocation. >>>>>>> >>>>>>> In order to call the Java method Charset.isSupported() (in >>>>>>> JNU_CallStaticMethodByName() in file jni_util.c), we have to call >>>>>>> jni_FindClass() to convert the symbolic class name >>>>>>> "java.nio.charset.Charset" into a class reference. >>>>>>> >>>>>>> But unfortunately, jni_FindClass() (from jni.cpp in libjvm.so) has a >>>>>>> special >>>>>>> handling if called from java.lang.ClassLoader$NativeLibrary to ensure >>>>>>> that >>>>>>> JNI_OnLoad/JNI_OnUnload are executed in the correct class context: >>>>>>> >>>>>>> instanceKlassHandle k (THREAD, >>>>>>> thread->security_get_caller_class(0)); >>>>>>> if (k.not_null()) { >>>>>>> loader = Handle(THREAD, k->class_loader()); >>>>>>> // Special handling to make sure JNI_OnLoad and JNI_OnUnload are >>>>>>> executed >>>>>>> // in the correct class context. >>>>>>> if (loader.is_null() && >>>>>>> k->name() == >>>>>>> vmSymbols::java_lang_ClassLoader_NativeLibrary()) { >>>>>>> JavaValue result(T_OBJECT); >>>>>>> JavaCalls::call_static(&result, k, >>>>>>> vmSymbols::getFromClass_name(), >>>>>>> >>>>>>> vmSymbols::void_class_signature(), >>>>>>> thread); >>>>>>> if (HAS_PENDING_EXCEPTION) { >>>>>>> Handle ex(thread, thread->pending_exception()); >>>>>>> CLEAR_PENDING_EXCEPTION; >>>>>>> THROW_HANDLE_0(ex); >>>>>>> } >>>>>>> >>>>>>> So if that's the case and jni_FindClass() was reallycalled from >>>>>>> ClassLoader$NativeLibrary, then jni_FindClass() calles >>>>>>> ClassLoader$NativeLibrary().getFromClass() to find out the >>>>>>> corresponding >>>>>>> context class which is supposed to be saved there in a field of type >>>>>>> java.util.Stack named "nativeLibraryContext": >>>>>>> >>>>>>> // Invoked in the VM to determine the context class in >>>>>>> // JNI_Load/JNI_Unload >>>>>>> static Class getFromClass() { >>>>>>> return ClassLoader.nativeLibraryContext.peek().fromClass; >>>>>>> } >>>>>>> >>>>>>> Unfortunately, "nativeLibraryContext" doesn't contain any entry at this >>>>>>> point and the invocation of Stack.peek() will throw the exception shown >>>>>>> before. In general, the "nativeLibraryContext" stack will be filled >>>>>>> later >>>>>>> on >>>>>>> in Runtime.loadLibrary0() like this: >>>>>>> >>>>>>> NativeLibrary lib = new NativeLibrary(fromClass, name, isBuiltin); >>>>>>> nativeLibraryContext.push(lib); >>>>>>> try { >>>>>>> lib.load(name, isBuiltin); >>>>>>> } finally { >>>>>>> nativeLibraryContext.pop(); >>>>>>> } >>>>>>> >>>>>>> such that it always contains at least one element later when >>>>>>> jni_FindClass() >>>>>>> will be invoked. >>>>>>> >>>>>>> So in summary, the problem is that the implementors of 8005716 didn't >>>>>>> took >>>>>>> into account that calling ClassLoader$NativeLibrary.findBuiltinLib() >>>>>>> may >>>>>>> trigger a call to jni_FindClass() if we are running on a system with an >>>>>>> unsupported character encoding. >>>>>>> >>>>>>> I'd suggest the following fix for this problem: >>>>>>> >>>>>>> Change ClassLoader$NativeLibrary().getFromClass() to return null if the >>>>>>> stack is empty instead of throwing an exception: >>>>>>> >>>>>>> static Class getFromClass() { >>>>>>> return ClassLoader.nativeLibraryContext.empty() ? >>>>>>> null : ClassLoader.nativeLibraryContext.peek().fromClass; >>>>>>> } >>>>>>> >>>>>>> Unfortunately this also requires a HotSpot change in jni_FindClass() in >>>>>>> order to properly handle the new 'null' return value: >>>>>>> >>>>>>> if (k.not_null()) { >>>>>>> loader = Handle(THREAD, k->class_loader()); >>>>>>> // Special handling to make sure JNI_OnLoad and JNI_OnUnload are >>>>>>> executed >>>>>>> // in the correct class context. >>>>>>> if (loader.is_null() && >>>>>>> k->name() == >>>>>>> vmSymbols::java_lang_ClassLoader_NativeLibrary()) { >>>>>>> JavaValue result(T_OBJECT); >>>>>>> JavaCalls::call_static(&result, k, >>>>>>> vmSymbols::getFromClass_name(), >>>>>>> >>>>>>> vmSymbols::void_class_signature(), >>>>>>> thread); >>>>>>> if (HAS_PENDING_EXCEPTION) { >>>>>>> Handle ex(thread, thread->pending_exception()); >>>>>>> CLEAR_PENDING_EXCEPTION; >>>>>>> THROW_HANDLE_0(ex); >>>>>>> } >>>>>>> oop mirror = (oop) result.get_jobject(); >>>>>>> if (oopDesc::is_null(mirror)) { >>>>>>> loader = Handle(THREAD, >>>>>>> SystemDictionary::java_system_loader()); >>>>>>> } else { >>>>>>> loader = Handle(THREAD, >>>>>>> >>>>>>> >>>>>>> InstanceKlass::cast(java_lang_Class::as_Klass(mirror))->class_loader()); >>>>>>> protection_domain = Handle(THREAD, >>>>>>> >>>>>>> >>>>>>> >>>>>>> InstanceKlass::cast(java_lang_Class::as_Klass(mirror))->protection_domain()); >>>>>>> } >>>>>>> } >>>>>>> } else { >>>>>>> // We call ClassLoader.getSystemClassLoader to obtain the system >>>>>>> class >>>>>>> loader. >>>>>>> loader = Handle(THREAD, SystemDictionary::java_system_loader()); >>>>>>> } >>>>>>> >>>>>>> These changes are sufficient to solve the problem in Java 8. >>>>>>> Unfortunately, >>>>>>> that's still not enough in Java 9 because there the loading of the >>>>>>> extended >>>>>>> charsets has been delegated to ServiceLoader. But ServiceLoader calls >>>>>>> ClassLoader.getBootstrapResources() which calls >>>>>>> sun.misc.Launcher.getBootstrapClassPath(). This leads to another >>>>>>> problem >>>>>>> during class initialization of sun.misc.Launcher if running on an >>>>>>> unsupported locale. >>>>>>> >>>>>>> The first thing done in sun.misc.Launcher. is the >>>>>>> initialisation >>>>>>> of >>>>>>> the bootstrap URLClassPath in the Launcher. However this initialisation >>>>>>> will >>>>>>> eventually call Charset.isSupported() and if we are running on an >>>>>>> unsupported locale this will inevitably end in another recursive call >>>>>>> to >>>>>>> ServiceLoader. But as explained below, ServiceLoader will query the >>>>>>> Launcher's bootstrap URLClassPath which will be still uninitialized at >>>>>>> that >>>>>>> point. >>>>>>> >>>>>>> So we'll have to additionally guard guard against this situation on JDK >>>>>>> 9 >>>>>>> like this: >>>>>>> >>>>>>> private static Charset lookupExtendedCharset(String charsetName) { >>>>>>> if (!sun.misc.VM.isBooted() || // see >>>>>>> lookupViaProviders() >>>>>>> sun.misc.Launcher.getBootstrapClassPath() == null) >>>>>>> return null; >>>>>>> >>>>>>> This fixes the crashes, but still at the price of not having the >>>>>>> extended >>>>>>> charsets available during initialization until >>>>>>> Launcher.getBootstrapClassPath is set up properly. This may be still a >>>>>>> problem if the jdk is installed in a directory which contains >>>>>>> characters >>>>>>> specific to an extended encoding or if we have such characters in the >>>>>>> command line arguments. >>>>>>> >>>>>>> Thank you and best regards, >>>>>>> Volker >>>>>>> >>>>>>> >>>>>>> Mixed stack trace of the initial EmptyStackException for unsupported >>>>>>> charsets described before: >>>>>>> >>>>>>> Native frames: (J=compiled Java code, j=interpreted, Vv=VM code, >>>>>>> C=native >>>>>>> code) >>>>>>> j java.util.Stack.peek()Ljava/lang/Object;+1 >>>>>>> j >>>>>>> java.lang.ClassLoader$NativeLibrary.getFromClass()Ljava/lang/Class;+3 >>>>>>> v ~StubRoutines::call_stub >>>>>>> V [libjvm.so+0x9d279a] JavaCalls::call_helper(JavaValue*, >>>>>>> methodHandle*, >>>>>>> JavaCallArguments*, Thread*)+0x6b4 >>>>>>> V [libjvm.so+0xcad591] os::os_exception_wrapper(void (*)(JavaValue*, >>>>>>> methodHandle*, JavaCallArguments*, Thread*), JavaValue*, methodHandle*, >>>>>>> JavaCallArguments*, Thread*)+0x45 >>>>>>> V [libjvm.so+0x9d20cf] JavaCalls::call(JavaValue*, methodHandle, >>>>>>> JavaCallArguments*, Thread*)+0x8b >>>>>>> V [libjvm.so+0x9d1d3b] JavaCalls::call_static(JavaValue*, >>>>>>> KlassHandle, >>>>>>> Symbol*, Symbol*, JavaCallArguments*, Thread*)+0x139 >>>>>>> V [libjvm.so+0x9d1e3f] JavaCalls::call_static(JavaValue*, >>>>>>> KlassHandle, >>>>>>> Symbol*, Symbol*, Thread*)+0x9d >>>>>>> V [libjvm.so+0x9e6588] jni_FindClass+0x428 >>>>>>> C [libjava.so+0x20208] JNU_CallStaticMethodByName+0xff >>>>>>> C [libjava.so+0x21cae] jnuEncodingSupported+0x61 >>>>>>> C [libjava.so+0x22125] JNU_GetStringPlatformChars+0x125 >>>>>>> C [libjava.so+0xedcd] >>>>>>> Java_java_lang_ClassLoader_00024NativeLibrary_findBuiltinLib+0x8b >>>>>>> j >>>>>>> >>>>>>> >>>>>>> java.lang.ClassLoader$NativeLibrary.findBuiltinLib(Ljava/lang/String;)Ljava/lang/String;+0 >>>>>>> j >>>>>>> java.lang.ClassLoader.loadLibrary0(Ljava/lang/Class;Ljava/io/File;)Z+4 >>>>>>> j >>>>>>> >>>>>>> >>>>>>> java.lang.ClassLoader.loadLibrary(Ljava/lang/Class;Ljava/lang/String;Z)V+228 >>>>>>> j >>>>>>> java.lang.Runtime.loadLibrary0(Ljava/lang/Class;Ljava/lang/String;)V+54 >>>>>>> j java.lang.System.loadLibrary(Ljava/lang/String;)V+7 >>>>>>> j java.lang.System.initializeSystemClass()V+113 >>>>>>> v ~StubRoutines::call_stub >>>>>>> V [libjvm.so+0x9d279a] JavaCalls::call_helper(JavaValue*, >>>>>>> methodHandle*, >>>>>>> JavaCallArguments*, Thread*)+0x6b4 >>>>>>> V [libjvm.so+0xcad591] os::os_exception_wrapper(void (*)(JavaValue*, >>>>>>> methodHandle*, JavaCallArguments*, Thread*), JavaValue*, methodHandle*, >>>>>>> JavaCallArguments*, Thread*)+0x45 >>>>>>> V [libjvm.so+0x9d20cf] JavaCalls::call(JavaValue*, methodHandle, >>>>>>> JavaCallArguments*, Thread*)+0x8b >>>>>>> V [libjvm.so+0x9d1d3b] JavaCalls::call_static(JavaValue*, >>>>>>> KlassHandle, >>>>>>> Symbol*, Symbol*, JavaCallArguments*, Thread*)+0x139 >>>>>>> V [libjvm.so+0x9d1e3f] JavaCalls::call_static(JavaValue*, >>>>>>> KlassHandle, >>>>>>> Symbol*, Symbol*, Thread*)+0x9d >>>>>>> V [libjvm.so+0xe3cceb] call_initializeSystemClass(Thread*)+0xb0 >>>>>>> V [libjvm.so+0xe44444] >>>>>>> Threads::initialize_java_lang_classes(JavaThread*, >>>>>>> Thread*)+0x21a >>>>>>> V [libjvm.so+0xe44b12] Threads::create_vm(JavaVMInitArgs*, >>>>>>> bool*)+0x4a6 >>>>>>> V [libjvm.so+0xa19bd7] JNI_CreateJavaVM+0xc7 >>>>>>> C [libjli.so+0xa520] InitializeJVM+0x154 >>>>>>> C [libjli.so+0x8024] JavaMain+0xcc >>>>>>> >>>>>>> >>> > From alejandro.murillo at oracle.com Wed Jun 10 17:21:59 2015 From: alejandro.murillo at oracle.com (Alejandro E Murillo) Date: Wed, 10 Jun 2015 11:21:59 -0600 Subject: RFR: JDK-8085822 JEP 223: New Version-String Scheme (initial integration) In-Reply-To: <557829DC.8000405@oracle.com> References: <5571AD18.7080701@oracle.com> <557625C8.5050200@oracle.com> <5576E633.9050503@oracle.com> <5576E81B.8050703@oracle.com> <5576EF94.3010500@oracle.com> <55780A33.1010302@oracle.com> <557829DC.8000405@oracle.com> Message-ID: <55787237.20802@oracle.com> On 6/10/2015 6:13 AM, Magnus Ihse Bursie wrote: > On 2015-06-10 11:58, David Holmes wrote: >> Hi Magnus, >> >> Generally looks good - a few comments/queries below. > > In general, I believe most issues you found are valid. :-) However, as > I said before in this thread, I'd like to see them resolved in the > needed follow-up patches. The source code changes in Hotspot and JDK > are inadequate to fully implement JEP-223, so these areas will need to > be rewritten anyhow. Are you okay with resolving these issues later? > >> >> On 9/06/2015 11:52 PM, Magnus Ihse Bursie wrote: >>> Here's an updated webrev, which fixes the typos that were pointed >>> out by >>> reviewers: >>> http://cr.openjdk.java.net/~ihse/JDK-8085822-JEP-223-initial-patch/webrev.02/ >>> >> >> common/autoconf/version-numbers >> >> Shouldn't there be a DEFAULT_VERSION_XXX for each of the XXX parts of >> the version info? (Even if all zeroes presently.) > So that's a tricky one. :-& The question here is, I think, does it > make sense to update version-numbers when we do a security (9.0.1) or > minor (9.1.0) release? Looking historically, the version-numbers file > have not been changed for the 8u family. The only change was going > from 8 to 9 when creating the new jdk9 forest. That was what I based > my decition to only have the major number in the file. (The rest of > the version string is set by configure flags when building, in Oracle > case by the RE team.) > > If it makes sense to put the minor/security/patch numbers here, I > won't oppose it, but I guess we have until the first such release to > figure out what's best, and that likely includes discussion with RE > and possibly other consumers in the community (RedHat etc). > >> >> --- >> >> Looking at hotspot changes I can't convince myself that the new >> streamlined "version" variables will capture things like "fastdebug". >> Will that filter through via configure variables? > > The "fastdebug" label has been handled as a less valued token in the > JEP-223 process. Right now there's no mention of it at all in the > version string proposal in the JEP. As I understand it, Iris is about > to propose an amendment which will just make fastdebug be a part of > the OPT string. I'm not entirely happy with that myself, but that's > the way it's decided. So wherever you can see the OPT string, you'll > see the debug level tag. > > Currently, however, that's not how it is implemented in this patch. > Since no decision was made on this (and it's still not formally > decided), I had to pick a route to go forward. The current patch will > instead put the "fastdebug" label as part of the PRE string (that's > the reason for the pre-base and pre-debuglevel arguments to > configure). If the planned changes goes into the JEP, this'll change > to make the debuglevel tag a part of the OPT string instead. > >> --- >> >> make/*/makefiles/vm.make >> >> Shouldn't the -DVERSION_XX=$(VERSION_XX) definitions be taken from >> the VERSION_CFLAGS in spec.gmk? (Or are you still allowing for a >> stand-alone hotspot build?) > The hotspot JEP-223 work initially made by Alejandro kept support for > stand-alone hotspot builds. I made additional changes on top of that, > which still tried to cater for stand-alone builds. At this very moment > I'm not sure if stand-alone hotspot builds are supposed to work or > not, but I've tried to not change the situation with this patch. > > There are two future changes coming down the pipe: One is the > additional JEP-223 work needed for Hotspot. I know Alejandro had plans > for redesigning the API between Hotspot and the JDK, so no JDK version > strings should be compiled into the JVM, but all of it extracted > during an API in runtime. That would make most (all?) of the makefile > changes in hotspot irrelevant. However, that implementation is not > even started, so it's needed for the time being. There's already an API used by Hotspot to get JDK version values. I'm investigating using that and extend it if required. yes, there is a lot stuff in the hotspot code that should be removed (mostly in the makefiles) and I'm not against making those changes now, I just think they are somewhat out of the scope of this project. And should be done as individual RFEs or as part of the upcoming hotspot makefile refactoring. Alejandro > > The second change is the build-infra hotspot makefile rewrite. When > that happens, stand-alone builds will definitely disappear, and if > Hotspot still needs make support for version strings, then the logical > choice is to use VERSION_CFLAGS. > > Ok? > >> >> --- >> >> hotspot/src/share/vm/prims/jvm.h >> jdk/src/java.base/share/native/include/jvm.h >> >> I think this comment is way out of date: >> >> /* Build number is available only for RE build (i.e. >> JDK_BUILD_NUMBER is set to bNN) >> * It will be zero for internal builds. >> */ >> >> and can be completely removed. Even if still true, mention of an "RE >> build" has no place in OpenJDK sources. > Yep, agree. Follow-up patch, right? > >> >> --- >> >> hotspot/src/share/vm/runtime/java.cpp >> >> I think a lot of the modified code is obsolete post Hotspot Express - >> which makes it hard to determine if the changes make sense. > Yep, agree. Follow-up patch, right? >> >> --- >> >> hotspot/src/share/vm/runtime/vmStructs.cpp >> >> Please fix the alignment (3 extra spaces on modified line): >> >> 1239 static_field(Abstract_VM_Version, _vm_minor_version, >> int) \ >> 1240 static_field(Abstract_VM_Version, _vm_security_version, >> int) \ > > I was about to say "follow-up patch", but ugly indentation really > sucks so I can fix this. Ok if I skip redoing the review for that? > >> >> --- >> >> hotspot/test/runtime/6981737/Test6981737.java >> >> This test is really only valid for the new version scheme now, so it >> should probably be rewritten - and in that case it really isn't >> testing 6981737 but should be replaced by a new test for the new >> version format. > Yep, agree. Follow-up patch, right? >> >> --- >> >> jdk/src/java.base/share/classes/sun/misc/Version.java.template >> >> This comment is nonsensical: >> >> /** >> ! * Returns the security version of the running JVM if it's 1.6 >> or newer >> * or any RE VM build. It will return 0 if it's an internal 1.5 or >> * 1.4.x build. >> * @since 1.6 >> */ >> >> as security version does not exist pre 9. Normally you should be >> adding a new method and deprecating the old one. The new one is >> @since 9. > Yep, agree. Follow-up patch, right? >> >> /** >> ! * Returns the security version of the running JDK. >> * @since 1.6 >> */ >> >> Ditto: @since 9 (but again old should be deprecated and new method >> added) >> >> 253 /** >> 254 * Returns the build number of the running JDK if it's a RE >> build >> 255 * It will return 0 if it's an internal build. >> >> As with jvm.h this seems obsolete commentary these days - not only RE >> builds define a build number. > Yep, agree. Follow-up patch, right? > > /Magnus > >> >> Thanks, >> David > -- Alejandro From mandy.chung at oracle.com Wed Jun 10 19:02:35 2015 From: mandy.chung at oracle.com (Mandy Chung) Date: Wed, 10 Jun 2015 12:02:35 -0700 Subject: RFR(M): 8081674: EmptyStackException at startup if running with extended or unsupported charset In-Reply-To: References: <557626AA.6010008@oracle.com> <55772A56.2030308@oracle.com> Message-ID: <7B3C08CF-028E-4AD2-B7F8-C803F2BA3C7D@oracle.com> > On Jun 10, 2015, at 9:23 AM, Volker Simonis wrote: > > Hy Mandy, > > that's a real good proposal and it only requires changes in the jdk > repository which makes it even more attractive. > > I've just checked and it does indeed solve the initial > EmptyStackException problem in both jdk8 and jdk9. Here are the > corresponding webrevs: > > http://cr.openjdk.java.net/~simonis/webrevs/2015/8081674.jdk8/ ClassLoader.java It should have done that. Nit: Can you move line 1693 to 1868 closer to loadLibrary0 (the caller of findBuiltinLib)? Also make it private. Same applies to jdk9. Otherwise looks good. No need to generate a webrev as long as you make the change before the push. > http://cr.openjdk.java.net/~simonis/webrevs/2015/8081674.jdk9/ > > The only difference between 8 and 9 is the different location of the > source files. > Compared to your proposal I've only updated the copyright years where > necessary and removed one comment in > Java_java_lang_ClassLoader_findBuiltinLib() which seemed unnecessary > to me now that findBuiltinLib() isn't located in NativeLibrary > anymore. > Thanks for taking it out. > On jdk9 we are now running in another problem which is caused by the > fact that the extended charsets are now being loaded by ServiceLoader > (I've detailed that already in the bug report). But I think we can > leave the fix of that problem to another change as this seems to > require some more reasoning (see Alan's comments in the bug report). Yes this is a harder problem that have to investigate further. > > So are you OK now with pushing this change to jdk9 and requesting a > downport to 8u-dev afterwards? > Yes. Approved. Mandy > Thank you and best regards, > Volker > > > On Wed, Jun 10, 2015 at 5:14 PM, Mandy Chung wrote: >> Have you checked out this patch moving out findBuiltinLib from NativeLibrary class? >> http://cr.openjdk.java.net/~mchung/jdk9/webrevs/8081674/webrev.00/ >> >> Mandy >> >>> On Jun 10, 2015, at 12:34 AM, Volker Simonis wrote: >>> >>> Mandy, the example/stacktrace I sent was with 8u. >>> >>> The problem isn't related to libzip, it is only the first library >>> which gets loaded with System.loadLibrary(). >>> >>> The problem will occur with every library which gets loaded by >>> System.loadLibrary() because >>> java.lang.ClassLoader$NativeLibrary.findBuiltinLib() always calls >>> jni_FindClass() if we're running on a unsupported locale. >>> >>> On Tue, Jun 9, 2015 at 8:03 PM, Mandy Chung wrote: >>>> Does my patch work fine on 8u? If it works fine, I prefer to get that >>>> simple fix in 8u and take the time to have a better fix in 9 (jdk9 is still >>>> under development and I have assumed that it's not a show stopper to you. >>>> Let me know if it blocks you). >>>> >>>> A question to Sherman - do we have adequate tests to verify the move of >>>> System.loadLibrary("zip") from system initialization to ZipFile >>>> initialization for 8u? >>>> >>>> Mandy >>>> >>>> >>>> On 06/09/2015 10:09 AM, Volker Simonis wrote: >>>>> >>>>> Hi Mandy, >>>>> >>>>> thanks for looking into this. >>>>> Uunfortunately your fix only helps to fix "java -version" :( >>>>> >>>>> Running even a minimal HelloWorld will still fail with the following >>>>> stack trace which is still caused by the same EmptyStackException: >>>>> >>>>> Error: A JNI error has occurred, please check your installation and try >>>>> again >>>>> Exception in thread "main" java.lang.ExceptionInInitializerError >>>>> at sun.misc.Unsafe.ensureClassInitialized(Native Method) >>>>> at >>>>> sun.misc.SharedSecrets.getJavaUtilZipFileAccess(SharedSecrets.java:158) >>>>> at sun.misc.URLClassPath$JarLoader.(URLClassPath.java:765) >>>>> at sun.misc.URLClassPath$3.run(URLClassPath.java:530) >>>>> at sun.misc.URLClassPath$3.run(URLClassPath.java:520) >>>>> at java.security.AccessController.doPrivileged(Native Method) >>>>> at sun.misc.URLClassPath.getLoader(URLClassPath.java:519) >>>>> at sun.misc.URLClassPath.getLoader(URLClassPath.java:492) >>>>> at sun.misc.URLClassPath.getNextLoader(URLClassPath.java:457) >>>>> at sun.misc.URLClassPath.access$100(URLClassPath.java:64) >>>>> at sun.misc.URLClassPath$1.next(URLClassPath.java:239) >>>>> at sun.misc.URLClassPath$1.hasMoreElements(URLClassPath.java:250) >>>>> at java.net.URLClassLoader$3$1.run(URLClassLoader.java:601) >>>>> at java.net.URLClassLoader$3$1.run(URLClassLoader.java:599) >>>>> at java.security.AccessController.doPrivileged(Native Method) >>>>> at java.net.URLClassLoader$3.next(URLClassLoader.java:598) >>>>> at java.net.URLClassLoader$3.hasMoreElements(URLClassLoader.java:623) >>>>> at sun.misc.CompoundEnumeration.next(CompoundEnumeration.java:45) >>>>> at >>>>> sun.misc.CompoundEnumeration.hasMoreElements(CompoundEnumeration.java:54) >>>>> at sun.misc.CompoundEnumeration.next(CompoundEnumeration.java:45) >>>>> at >>>>> sun.misc.CompoundEnumeration.hasMoreElements(CompoundEnumeration.java:54) >>>>> at >>>>> java.util.ServiceLoader$LazyIterator.hasNextService(ServiceLoader.java:354) >>>>> at >>>>> java.util.ServiceLoader$LazyIterator.hasNext(ServiceLoader.java:393) >>>>> at java.util.ServiceLoader$1.hasNext(ServiceLoader.java:474) >>>>> at java.nio.charset.Charset$1.getNext(Charset.java:350) >>>>> at java.nio.charset.Charset$1.hasNext(Charset.java:365) >>>>> at java.nio.charset.Charset$2.run(Charset.java:410) >>>>> at java.nio.charset.Charset$2.run(Charset.java:407) >>>>> at java.security.AccessController.doPrivileged(Native Method) >>>>> at java.nio.charset.Charset.lookupViaProviders(Charset.java:406) >>>>> at java.nio.charset.Charset.lookup2(Charset.java:477) >>>>> at java.nio.charset.Charset.lookup(Charset.java:464) >>>>> at java.nio.charset.Charset.isSupported(Charset.java:505) >>>>> at >>>>> sun.launcher.LauncherHelper.makePlatformString(LauncherHelper.java:580) >>>>> Caused by: java.util.EmptyStackException >>>>> at java.util.Stack.peek(Stack.java:102) >>>>> at >>>>> java.lang.ClassLoader$NativeLibrary.getFromClass(ClassLoader.java:1759) >>>>> at java.lang.ClassLoader$NativeLibrary.findBuiltinLib(Native Method) >>>>> at java.lang.ClassLoader.loadLibrary0(ClassLoader.java:1870) >>>>> at java.lang.ClassLoader.loadLibrary(ClassLoader.java:1843) >>>>> at java.lang.Runtime.loadLibrary0(Runtime.java:870) >>>>> at java.lang.System.loadLibrary(System.java:1122) >>>>> at java.util.zip.ZipFile.(ZipFile.java:86) >>>>> ... 34 more >>>>> >>>>> It's obvious that the way jni_FindClass is looking for the class >>>>> context by calling the NativeLibrary::getFromClass method is hacky but >>>>> I think that the proposed fix is the quite simple and non-intrusive. >>>>> And we probably don't want a complete rework of this code for 8 >>>>> anyway. So why not fix it the proposed way in 8 and 9 for now? That >>>>> will still leave us time to come up with a better clean-up at least >>>>> for 9. >>>>> >>>>> What do you think? >>>>> >>>>> Regards, >>>>> Volker >>>>> >>>>> On Tue, Jun 9, 2015 at 1:35 AM, Mandy Chung >>>>> wrote: >>>>>> >>>>>> Hi Volker, >>>>>> >>>>>> Your patch reminds me the question I have about loading zip library. >>>>>> >>>>>> In JDK 9 (and earlier release), zip library is loaded by the VM during >>>>>> startup (see ClassLoader::load_zip_library). I think loadLibrary("zip") >>>>>> is >>>>>> no longer needed to be called from System::initializeSystemClass method >>>>>> and >>>>>> instead it can be loaded by java.util.zip.ZipFile static initializer. >>>>>> >>>>>> Do you mind to try the patch (below)? That may be a simpler fix. >>>>>> >>>>>> I never like the way jni_FindClass to look for the class context by >>>>>> calling >>>>>> the NativeLibrary::getFromClass method. I will have to look deeper other >>>>>> alternative to clean that up. If taking out loadLibrary("zip") resolves >>>>>> your issue, this will give us time to come up with a better clean-up in >>>>>> the >>>>>> future. >>>>>> >>>>>> Mandy >>>>>> [1] https://bugs.openjdk.java.net/browse/JDK-4429040 >>>>>> >>>>>> diff --git a/src/share/classes/java/lang/System.java >>>>>> b/src/share/classes/java/lang/System.java >>>>>> --- a/src/share/classes/java/lang/System.java >>>>>> +++ b/src/share/classes/java/lang/System.java >>>>>> @@ -1192,10 +1192,6 @@ >>>>>> setOut0(newPrintStream(fdOut, >>>>>> props.getProperty("sun.stdout.encoding"))); >>>>>> setErr0(newPrintStream(fdErr, >>>>>> props.getProperty("sun.stderr.encoding"))); >>>>>> >>>>>> - // Load the zip library now in order to keep >>>>>> java.util.zip.ZipFile >>>>>> - // from trying to use itself to load this library later. >>>>>> - loadLibrary("zip"); >>>>>> - >>>>>> // Setup Java signal handlers for HUP, TERM, and INT (where >>>>>> available). >>>>>> Terminator.setup(); >>>>>> >>>>>> diff --git a/src/share/classes/java/util/zip/ZipFile.java >>>>>> b/src/share/classes/java/util/zip/ZipFile.java >>>>>> --- a/src/share/classes/java/util/zip/ZipFile.java >>>>>> +++ b/src/share/classes/java/util/zip/ZipFile.java >>>>>> @@ -83,6 +83,7 @@ >>>>>> >>>>>> static { >>>>>> /* Zip library is loaded from System.initializeSystemClass */ >>>>>> + System.loadLibrary("zip"); >>>>>> initIDs(); >>>>>> >>>>>> } >>>>>> >>>>>> >>>>>> >>>>>> On 06/08/2015 07:23 AM, Volker Simonis wrote: >>>>>>> >>>>>>> Hi, >>>>>>> >>>>>>> can I please get a review at least for the part of this fix which is >>>>>>> common for jdk8 and jdk9. It's only a few lines in >>>>>>> src/share/vm/prims/jni.cpp for the hotspot repository and one line >>>>>>> src/java.base/share/classes/java/lang/ClassLoader.java for the jdk >>>>>>> repository. >>>>>>> >>>>>>> Thanks, >>>>>>> Volker >>>>>>> >>>>>>> >>>>>>> On Tue, Jun 2, 2015 at 6:12 PM, Volker Simonis >>>>>>> >>>>>>> wrote: >>>>>>>> >>>>>>>> Hi, >>>>>>>> >>>>>>>> can I please have review (and a sponsor) for these changes: >>>>>>>> >>>>>>>> https://bugs.openjdk.java.net/browse/JDK-8081674 >>>>>>>> http://cr.openjdk.java.net/~simonis/webrevs/2015/8081674.jdk >>>>>>>> http://cr.openjdk.java.net/~simonis/webrevs/2015/8081674.hs >>>>>>>> >>>>>>>> Please notice that the fix requires synchronous changes in the jdk and >>>>>>>> the >>>>>>>> hotspot forest. >>>>>>>> >>>>>>>> The changes themselves are by far not that big as this explanation but >>>>>>>> I >>>>>>>> found the problem to be quite intricate so I tried to explain it as >>>>>>>> good >>>>>>>> as >>>>>>>> I could. I'd suggest to read the HTML-formatted explanation in the >>>>>>>> webrev >>>>>>>> although the content is equal to the one in this mail: >>>>>>>> >>>>>>>> Using an unsupported character encoding (e.g. vi_VN.TCVN on Linux) >>>>>>>> results >>>>>>>> in an immediate VM failure with jdk 8 and 9: >>>>>>>> >>>>>>>> export LANG=vi_VN.TCVN >>>>>>>> java -version >>>>>>>> Error occurred during initialization of VM >>>>>>>> java.util.EmptyStackException >>>>>>>> at java.util.Stack.peek(Stack.java:102) >>>>>>>> at >>>>>>>> java.lang.ClassLoader$NativeLibrary.getFromClass(ClassLoader.java:1751) >>>>>>>> at java.lang.ClassLoader$NativeLibrary.findBuiltinLib(Native >>>>>>>> Method) >>>>>>>> at java.lang.ClassLoader.loadLibrary0(ClassLoader.java:1862) >>>>>>>> at java.lang.ClassLoader.loadLibrary(ClassLoader.java:1835) >>>>>>>> at java.lang.Runtime.loadLibrary0(Runtime.java:870) >>>>>>>> at java.lang.System.loadLibrary(System.java:1119) >>>>>>>> at java.lang.System.initializeSystemClass(System.java:1194) >>>>>>>> >>>>>>>> This is a consequence of "8005716: Enhance JNI specification to allow >>>>>>>> support of static JNI libraries in Embedded JREs". >>>>>>>> >>>>>>>> With jdk 9 we get this error even if we're running with a supported >>>>>>>> charset >>>>>>>> which is in the ExtendedCharsets (as opposed to being in the >>>>>>>> StandardCharsets) class which is a consequence of delegating the >>>>>>>> loading >>>>>>>> of >>>>>>>> the ExtendedCharsets class to the ServiceLoader in jdk 9. >>>>>>>> >>>>>>>> export LANG=eo.iso-8859-3 >>>>>>>> output-jdk9/images/jdk/bin/java -version >>>>>>>> Error occurred during initialization of VM >>>>>>>> java.util.EmptyStackException >>>>>>>> at java.util.Stack.peek(Stack.java:102) >>>>>>>> at >>>>>>>> java.lang.ClassLoader$NativeLibrary.getFromClass(ClassLoader.java:1737) >>>>>>>> at java.lang.ClassLoader$NativeLibrary.findBuiltinLib(Native >>>>>>>> Method) >>>>>>>> at java.lang.ClassLoader.loadLibrary0(ClassLoader.java:1866) >>>>>>>> at java.lang.ClassLoader.loadLibrary(ClassLoader.java:1840) >>>>>>>> at java.lang.Runtime.loadLibrary0(Runtime.java:874) >>>>>>>> at java.lang.System.loadLibrary(System.java:1111) >>>>>>>> at java.lang.System.initializeSystemClass(System.java:1186) >>>>>>>> >>>>>>>> Here's why the exception happens for an unsupported charset (see the >>>>>>>> mixed >>>>>>>> stack trace below for the full details): >>>>>>>> >>>>>>>> java.lang.System.loadLibrary() wants to load libzip.so. It calls >>>>>>>> java.lang.Runtime.loadLibrary0() which at the very beginning calls the >>>>>>>> native method ClassLoader$NativeLibrary.findBuiltinLib() which checks >>>>>>>> if >>>>>>>> the >>>>>>>> corresponding library is already statically linked into the VM >>>>>>>> (introduced >>>>>>>> by 8005716). >>>>>>>> Java_java_lang_ClassLoader_00024NativeLibrary_findBuiltinLib(), >>>>>>>> the native implementation of findBuiltinLib() in Classloader.c calls >>>>>>>> GetStringPlatformChars() to convert the library name into the native >>>>>>>> platform encoding. GetStringPlatformChars() calls the helper function >>>>>>>> jnuEncodingSupported() to check if the platform encoding which is >>>>>>>> stored >>>>>>>> in >>>>>>>> the property "sun.jnu.encoding" is supported by Java. >>>>>>>> jnuEncodingSupported() >>>>>>>> is implemented as follows: >>>>>>>> >>>>>>>> static jboolean isJNUEncodingSupported = JNI_FALSE; >>>>>>>> static jboolean jnuEncodingSupported(JNIEnv *env) { >>>>>>>> jboolean exe; >>>>>>>> if (isJNUEncodingSupported == JNI_TRUE) { >>>>>>>> return JNI_TRUE; >>>>>>>> } >>>>>>>> isJNUEncodingSupported = (jboolean) JNU_CallStaticMethodByName ( >>>>>>>> env, &exe, >>>>>>>> "java/nio/charset/Charset", >>>>>>>> "isSupported", >>>>>>>> "(Ljava/lang/String;)Z", >>>>>>>> jnuEncoding).z; >>>>>>>> return isJNUEncodingSupported; >>>>>>>> } >>>>>>>> >>>>>>>> Once the function finds that the platform encoding is supported (by >>>>>>>> calling >>>>>>>> java.nio.charset.Charset.isSupported()) it caches this value and always >>>>>>>> returns it on following calls. However if the platform encoding is not >>>>>>>> supported, it ALWAYS calls java.nio.charset.Charset.isSupported() an >>>>>>>> every >>>>>>>> subsequent invocation. >>>>>>>> >>>>>>>> In order to call the Java method Charset.isSupported() (in >>>>>>>> JNU_CallStaticMethodByName() in file jni_util.c), we have to call >>>>>>>> jni_FindClass() to convert the symbolic class name >>>>>>>> "java.nio.charset.Charset" into a class reference. >>>>>>>> >>>>>>>> But unfortunately, jni_FindClass() (from jni.cpp in libjvm.so) has a >>>>>>>> special >>>>>>>> handling if called from java.lang.ClassLoader$NativeLibrary to ensure >>>>>>>> that >>>>>>>> JNI_OnLoad/JNI_OnUnload are executed in the correct class context: >>>>>>>> >>>>>>>> instanceKlassHandle k (THREAD, >>>>>>>> thread->security_get_caller_class(0)); >>>>>>>> if (k.not_null()) { >>>>>>>> loader = Handle(THREAD, k->class_loader()); >>>>>>>> // Special handling to make sure JNI_OnLoad and JNI_OnUnload are >>>>>>>> executed >>>>>>>> // in the correct class context. >>>>>>>> if (loader.is_null() && >>>>>>>> k->name() == >>>>>>>> vmSymbols::java_lang_ClassLoader_NativeLibrary()) { >>>>>>>> JavaValue result(T_OBJECT); >>>>>>>> JavaCalls::call_static(&result, k, >>>>>>>> vmSymbols::getFromClass_name(), >>>>>>>> >>>>>>>> vmSymbols::void_class_signature(), >>>>>>>> thread); >>>>>>>> if (HAS_PENDING_EXCEPTION) { >>>>>>>> Handle ex(thread, thread->pending_exception()); >>>>>>>> CLEAR_PENDING_EXCEPTION; >>>>>>>> THROW_HANDLE_0(ex); >>>>>>>> } >>>>>>>> >>>>>>>> So if that's the case and jni_FindClass() was reallycalled from >>>>>>>> ClassLoader$NativeLibrary, then jni_FindClass() calles >>>>>>>> ClassLoader$NativeLibrary().getFromClass() to find out the >>>>>>>> corresponding >>>>>>>> context class which is supposed to be saved there in a field of type >>>>>>>> java.util.Stack named "nativeLibraryContext": >>>>>>>> >>>>>>>> // Invoked in the VM to determine the context class in >>>>>>>> // JNI_Load/JNI_Unload >>>>>>>> static Class getFromClass() { >>>>>>>> return ClassLoader.nativeLibraryContext.peek().fromClass; >>>>>>>> } >>>>>>>> >>>>>>>> Unfortunately, "nativeLibraryContext" doesn't contain any entry at this >>>>>>>> point and the invocation of Stack.peek() will throw the exception shown >>>>>>>> before. In general, the "nativeLibraryContext" stack will be filled >>>>>>>> later >>>>>>>> on >>>>>>>> in Runtime.loadLibrary0() like this: >>>>>>>> >>>>>>>> NativeLibrary lib = new NativeLibrary(fromClass, name, isBuiltin); >>>>>>>> nativeLibraryContext.push(lib); >>>>>>>> try { >>>>>>>> lib.load(name, isBuiltin); >>>>>>>> } finally { >>>>>>>> nativeLibraryContext.pop(); >>>>>>>> } >>>>>>>> >>>>>>>> such that it always contains at least one element later when >>>>>>>> jni_FindClass() >>>>>>>> will be invoked. >>>>>>>> >>>>>>>> So in summary, the problem is that the implementors of 8005716 didn't >>>>>>>> took >>>>>>>> into account that calling ClassLoader$NativeLibrary.findBuiltinLib() >>>>>>>> may >>>>>>>> trigger a call to jni_FindClass() if we are running on a system with an >>>>>>>> unsupported character encoding. >>>>>>>> >>>>>>>> I'd suggest the following fix for this problem: >>>>>>>> >>>>>>>> Change ClassLoader$NativeLibrary().getFromClass() to return null if the >>>>>>>> stack is empty instead of throwing an exception: >>>>>>>> >>>>>>>> static Class getFromClass() { >>>>>>>> return ClassLoader.nativeLibraryContext.empty() ? >>>>>>>> null : ClassLoader.nativeLibraryContext.peek().fromClass; >>>>>>>> } >>>>>>>> >>>>>>>> Unfortunately this also requires a HotSpot change in jni_FindClass() in >>>>>>>> order to properly handle the new 'null' return value: >>>>>>>> >>>>>>>> if (k.not_null()) { >>>>>>>> loader = Handle(THREAD, k->class_loader()); >>>>>>>> // Special handling to make sure JNI_OnLoad and JNI_OnUnload are >>>>>>>> executed >>>>>>>> // in the correct class context. >>>>>>>> if (loader.is_null() && >>>>>>>> k->name() == >>>>>>>> vmSymbols::java_lang_ClassLoader_NativeLibrary()) { >>>>>>>> JavaValue result(T_OBJECT); >>>>>>>> JavaCalls::call_static(&result, k, >>>>>>>> vmSymbols::getFromClass_name(), >>>>>>>> >>>>>>>> vmSymbols::void_class_signature(), >>>>>>>> thread); >>>>>>>> if (HAS_PENDING_EXCEPTION) { >>>>>>>> Handle ex(thread, thread->pending_exception()); >>>>>>>> CLEAR_PENDING_EXCEPTION; >>>>>>>> THROW_HANDLE_0(ex); >>>>>>>> } >>>>>>>> oop mirror = (oop) result.get_jobject(); >>>>>>>> if (oopDesc::is_null(mirror)) { >>>>>>>> loader = Handle(THREAD, >>>>>>>> SystemDictionary::java_system_loader()); >>>>>>>> } else { >>>>>>>> loader = Handle(THREAD, >>>>>>>> >>>>>>>> >>>>>>>> InstanceKlass::cast(java_lang_Class::as_Klass(mirror))->class_loader()); >>>>>>>> protection_domain = Handle(THREAD, >>>>>>>> >>>>>>>> >>>>>>>> >>>>>>>> InstanceKlass::cast(java_lang_Class::as_Klass(mirror))->protection_domain()); >>>>>>>> } >>>>>>>> } >>>>>>>> } else { >>>>>>>> // We call ClassLoader.getSystemClassLoader to obtain the system >>>>>>>> class >>>>>>>> loader. >>>>>>>> loader = Handle(THREAD, SystemDictionary::java_system_loader()); >>>>>>>> } >>>>>>>> >>>>>>>> These changes are sufficient to solve the problem in Java 8. >>>>>>>> Unfortunately, >>>>>>>> that's still not enough in Java 9 because there the loading of the >>>>>>>> extended >>>>>>>> charsets has been delegated to ServiceLoader. But ServiceLoader calls >>>>>>>> ClassLoader.getBootstrapResources() which calls >>>>>>>> sun.misc.Launcher.getBootstrapClassPath(). This leads to another >>>>>>>> problem >>>>>>>> during class initialization of sun.misc.Launcher if running on an >>>>>>>> unsupported locale. >>>>>>>> >>>>>>>> The first thing done in sun.misc.Launcher. is the >>>>>>>> initialisation >>>>>>>> of >>>>>>>> the bootstrap URLClassPath in the Launcher. However this initialisation >>>>>>>> will >>>>>>>> eventually call Charset.isSupported() and if we are running on an >>>>>>>> unsupported locale this will inevitably end in another recursive call >>>>>>>> to >>>>>>>> ServiceLoader. But as explained below, ServiceLoader will query the >>>>>>>> Launcher's bootstrap URLClassPath which will be still uninitialized at >>>>>>>> that >>>>>>>> point. >>>>>>>> >>>>>>>> So we'll have to additionally guard guard against this situation on JDK >>>>>>>> 9 >>>>>>>> like this: >>>>>>>> >>>>>>>> private static Charset lookupExtendedCharset(String charsetName) { >>>>>>>> if (!sun.misc.VM.isBooted() || // see >>>>>>>> lookupViaProviders() >>>>>>>> sun.misc.Launcher.getBootstrapClassPath() == null) >>>>>>>> return null; >>>>>>>> >>>>>>>> This fixes the crashes, but still at the price of not having the >>>>>>>> extended >>>>>>>> charsets available during initialization until >>>>>>>> Launcher.getBootstrapClassPath is set up properly. This may be still a >>>>>>>> problem if the jdk is installed in a directory which contains >>>>>>>> characters >>>>>>>> specific to an extended encoding or if we have such characters in the >>>>>>>> command line arguments. >>>>>>>> >>>>>>>> Thank you and best regards, >>>>>>>> Volker >>>>>>>> >>>>>>>> >>>>>>>> Mixed stack trace of the initial EmptyStackException for unsupported >>>>>>>> charsets described before: >>>>>>>> >>>>>>>> Native frames: (J=compiled Java code, j=interpreted, Vv=VM code, >>>>>>>> C=native >>>>>>>> code) >>>>>>>> j java.util.Stack.peek()Ljava/lang/Object;+1 >>>>>>>> j >>>>>>>> java.lang.ClassLoader$NativeLibrary.getFromClass()Ljava/lang/Class;+3 >>>>>>>> v ~StubRoutines::call_stub >>>>>>>> V [libjvm.so+0x9d279a] JavaCalls::call_helper(JavaValue*, >>>>>>>> methodHandle*, >>>>>>>> JavaCallArguments*, Thread*)+0x6b4 >>>>>>>> V [libjvm.so+0xcad591] os::os_exception_wrapper(void (*)(JavaValue*, >>>>>>>> methodHandle*, JavaCallArguments*, Thread*), JavaValue*, methodHandle*, >>>>>>>> JavaCallArguments*, Thread*)+0x45 >>>>>>>> V [libjvm.so+0x9d20cf] JavaCalls::call(JavaValue*, methodHandle, >>>>>>>> JavaCallArguments*, Thread*)+0x8b >>>>>>>> V [libjvm.so+0x9d1d3b] JavaCalls::call_static(JavaValue*, >>>>>>>> KlassHandle, >>>>>>>> Symbol*, Symbol*, JavaCallArguments*, Thread*)+0x139 >>>>>>>> V [libjvm.so+0x9d1e3f] JavaCalls::call_static(JavaValue*, >>>>>>>> KlassHandle, >>>>>>>> Symbol*, Symbol*, Thread*)+0x9d >>>>>>>> V [libjvm.so+0x9e6588] jni_FindClass+0x428 >>>>>>>> C [libjava.so+0x20208] JNU_CallStaticMethodByName+0xff >>>>>>>> C [libjava.so+0x21cae] jnuEncodingSupported+0x61 >>>>>>>> C [libjava.so+0x22125] JNU_GetStringPlatformChars+0x125 >>>>>>>> C [libjava.so+0xedcd] >>>>>>>> Java_java_lang_ClassLoader_00024NativeLibrary_findBuiltinLib+0x8b >>>>>>>> j >>>>>>>> >>>>>>>> >>>>>>>> java.lang.ClassLoader$NativeLibrary.findBuiltinLib(Ljava/lang/String;)Ljava/lang/String;+0 >>>>>>>> j >>>>>>>> java.lang.ClassLoader.loadLibrary0(Ljava/lang/Class;Ljava/io/File;)Z+4 >>>>>>>> j >>>>>>>> >>>>>>>> >>>>>>>> java.lang.ClassLoader.loadLibrary(Ljava/lang/Class;Ljava/lang/String;Z)V+228 >>>>>>>> j >>>>>>>> java.lang.Runtime.loadLibrary0(Ljava/lang/Class;Ljava/lang/String;)V+54 >>>>>>>> j java.lang.System.loadLibrary(Ljava/lang/String;)V+7 >>>>>>>> j java.lang.System.initializeSystemClass()V+113 >>>>>>>> v ~StubRoutines::call_stub >>>>>>>> V [libjvm.so+0x9d279a] JavaCalls::call_helper(JavaValue*, >>>>>>>> methodHandle*, >>>>>>>> JavaCallArguments*, Thread*)+0x6b4 >>>>>>>> V [libjvm.so+0xcad591] os::os_exception_wrapper(void (*)(JavaValue*, >>>>>>>> methodHandle*, JavaCallArguments*, Thread*), JavaValue*, methodHandle*, >>>>>>>> JavaCallArguments*, Thread*)+0x45 >>>>>>>> V [libjvm.so+0x9d20cf] JavaCalls::call(JavaValue*, methodHandle, >>>>>>>> JavaCallArguments*, Thread*)+0x8b >>>>>>>> V [libjvm.so+0x9d1d3b] JavaCalls::call_static(JavaValue*, >>>>>>>> KlassHandle, >>>>>>>> Symbol*, Symbol*, JavaCallArguments*, Thread*)+0x139 >>>>>>>> V [libjvm.so+0x9d1e3f] JavaCalls::call_static(JavaValue*, >>>>>>>> KlassHandle, >>>>>>>> Symbol*, Symbol*, Thread*)+0x9d >>>>>>>> V [libjvm.so+0xe3cceb] call_initializeSystemClass(Thread*)+0xb0 >>>>>>>> V [libjvm.so+0xe44444] >>>>>>>> Threads::initialize_java_lang_classes(JavaThread*, >>>>>>>> Thread*)+0x21a >>>>>>>> V [libjvm.so+0xe44b12] Threads::create_vm(JavaVMInitArgs*, >>>>>>>> bool*)+0x4a6 >>>>>>>> V [libjvm.so+0xa19bd7] JNI_CreateJavaVM+0xc7 >>>>>>>> C [libjli.so+0xa520] InitializeJVM+0x154 >>>>>>>> C [libjli.so+0x8024] JavaMain+0xcc >>>>>>>> >>>>>>>> >>>> >> From naoto.sato at oracle.com Wed Jun 10 21:45:35 2015 From: naoto.sato at oracle.com (Naoto Sato) Date: Wed, 10 Jun 2015 14:45:35 -0700 Subject: TimeZone issue in 1.8u60 In-Reply-To: References: Message-ID: <5578AFFF.1010500@oracle.com> Hi Tomasz, The change was made to fix a performance regression in JDK8: https://bugs.openjdk.java.net/browse/JDK-8076287 Those time zone names weren't cached in JDK8, so the fix was to cache those arrays, which are also shared with ZoneId.getDisplayName() which can also return generic names. HTH, Naoto On 6/10/15 4:21 AM, Tomasz Kowalczewski wrote: > Hi, > > I am not sure where to write about this, I hope somebody will point me to > right list if this one is not correct. > > I have been playing with newest Java 1.8u60 to try PreserveFramePointer > functionality. Unfortunately none of our servers start on this version of > java. It is because of REST call to Amazon services done during startup. > None of these calls worked. Unless I am missing something it turns out to > be issue with formatting time zone information as done by Joda Time. It > uses calls to: > > DateTimeUtils.getDateFormatSymbols(Locale.ENGLISH).getZoneStrings(); > > to get list of timezones. This usually returned array of arrays of 5 > elements. In 1.8u60 it returns array of arrays of 7 elements. > > I know that all this software is not related to OpenJDK and calling > getZoneStrings is discouraged in the docs. But as I am unfamiliar with time > zones mechanisms inside JDK (loading from bundles etc.) I was hoping that > somebody will point me to change that may caused this for sake of better > understanding the issue. > From brent.christian at oracle.com Wed Jun 10 22:16:27 2015 From: brent.christian at oracle.com (Brent Christian) Date: Wed, 10 Jun 2015 15:16:27 -0700 Subject: RFR 9: 8064956 : Remove sun.misc.ExtensionInstallationProvider and relevant classes Message-ID: <5578B73B.7050703@oracle.com> Hi, Please review my change for: Bug: https://bugs.openjdk.java.net/browse/JDK-8064956 Webrev: http://cr.openjdk.java.net/~bchristi/8064956/ After clearing some internal dependencies following the removal of the extension mechanism from JDK 9[1], we can now remove 4 extension-related classes from sun.misc: ExtensionDependency.java ExtensionInfo.java ExtensionInstallationException.java ExtensionInstallationProvider.java I will submit a CCC following code review. Thanks, -Brent 1. https://bugs.openjdk.java.net/browse/JDK-8061971 From mandy.chung at oracle.com Wed Jun 10 22:54:42 2015 From: mandy.chung at oracle.com (Mandy Chung) Date: Wed, 10 Jun 2015 15:54:42 -0700 Subject: RFR 9: 8064956 : Remove sun.misc.ExtensionInstallationProvider and relevant classes In-Reply-To: <5578B73B.7050703@oracle.com> References: <5578B73B.7050703@oracle.com> Message-ID: <7D60932B-EE3B-4112-AD0A-03EBBE6353A2@oracle.com> Thumbs up. I like code deletion. Mandy > On Jun 10, 2015, at 3:16 PM, Brent Christian wrote: > > Hi, > > Please review my change for: > > Bug: > https://bugs.openjdk.java.net/browse/JDK-8064956 > Webrev: > http://cr.openjdk.java.net/~bchristi/8064956/ > > After clearing some internal dependencies following the removal of the extension mechanism from JDK 9[1], we can now remove 4 extension-related classes from sun.misc: > > ExtensionDependency.java > ExtensionInfo.java > ExtensionInstallationException.java > ExtensionInstallationProvider.java > > I will submit a CCC following code review. > > Thanks, > -Brent > > 1. https://bugs.openjdk.java.net/browse/JDK-8061971 > From brian.burkhalter at oracle.com Wed Jun 10 22:57:20 2015 From: brian.burkhalter at oracle.com (Brian Burkhalter) Date: Wed, 10 Jun 2015 15:57:20 -0700 Subject: [9] RFR of 8081843: (fs) FileStore.getTotalSpace returns unexpected results with >2TB file systems Message-ID: <2D69D5D5-EB8B-4235-A546-7C9BD3D06872@oracle.com> Please review at your convenience. Issue: https://bugs.openjdk.java.net/browse/JDK-8081843 Patch: http://cr.openjdk.java.net/~bpb/8081843/webrev.00/ Summary: On Mac OS X use BSD system call statfs() instead of Standard C Library call statvfs() to obtain file system size statistics. Standard regression test run succeeded. Thanks, Brian From volker.simonis at gmail.com Wed Jun 10 23:02:39 2015 From: volker.simonis at gmail.com (Volker Simonis) Date: Thu, 11 Jun 2015 01:02:39 +0200 Subject: RFR(M): 8081674: EmptyStackException at startup if running with extended or unsupported charset In-Reply-To: <7B3C08CF-028E-4AD2-B7F8-C803F2BA3C7D@oracle.com> References: <557626AA.6010008@oracle.com> <55772A56.2030308@oracle.com> <7B3C08CF-028E-4AD2-B7F8-C803F2BA3C7D@oracle.com> Message-ID: On Wednesday, June 10, 2015, Mandy Chung wrote: > > > On Jun 10, 2015, at 9:23 AM, Volker Simonis > wrote: > > > > Hy Mandy, > > > > that's a real good proposal and it only requires changes in the jdk > > repository which makes it even more attractive. > > > > I've just checked and it does indeed solve the initial > > EmptyStackException problem in both jdk8 and jdk9. Here are the > > corresponding webrevs: > > > > http://cr.openjdk.java.net/~simonis/webrevs/2015/8081674.jdk8/ > > ClassLoader.java > It should have done that. Nit: > Can you move line 1693 to 1868 closer to loadLibrary0 (the caller of > findBuiltinLib)? > Also make it private. > > Same applies to jdk9. > > Sure, I'll do that. Otherwise looks good. No need to generate a webrev as long as you make the > change before the push. > > Thanks for the review! Regards, Volker > > http://cr.openjdk.java.net/~simonis/webrevs/2015/8081674.jdk9/ > > > > The only difference between 8 and 9 is the different location of the > > source files. > > Compared to your proposal I've only updated the copyright years where > > necessary and removed one comment in > > Java_java_lang_ClassLoader_findBuiltinLib() which seemed unnecessary > > to me now that findBuiltinLib() isn't located in NativeLibrary > > anymore. > > > > Thanks for taking it out. > > > On jdk9 we are now running in another problem which is caused by the > > fact that the extended charsets are now being loaded by ServiceLoader > > (I've detailed that already in the bug report). But I think we can > > leave the fix of that problem to another change as this seems to > > require some more reasoning (see Alan's comments in the bug report). > > Yes this is a harder problem that have to investigate further. > > > > > So are you OK now with pushing this change to jdk9 and requesting a > > downport to 8u-dev afterwards? > > > > Yes. Approved. > > Mandy > > > Thank you and best regards, > > Volker > > > > > > On Wed, Jun 10, 2015 at 5:14 PM, Mandy Chung > wrote: > >> Have you checked out this patch moving out findBuiltinLib from > NativeLibrary class? > >> http://cr.openjdk.java.net/~mchung/jdk9/webrevs/8081674/webrev.00/ > >> > >> Mandy > >> > >>> On Jun 10, 2015, at 12:34 AM, Volker Simonis > wrote: > >>> > >>> Mandy, the example/stacktrace I sent was with 8u. > >>> > >>> The problem isn't related to libzip, it is only the first library > >>> which gets loaded with System.loadLibrary(). > >>> > >>> The problem will occur with every library which gets loaded by > >>> System.loadLibrary() because > >>> java.lang.ClassLoader$NativeLibrary.findBuiltinLib() always calls > >>> jni_FindClass() if we're running on a unsupported locale. > >>> > >>> On Tue, Jun 9, 2015 at 8:03 PM, Mandy Chung > wrote: > >>>> Does my patch work fine on 8u? If it works fine, I prefer to get > that > >>>> simple fix in 8u and take the time to have a better fix in 9 (jdk9 is > still > >>>> under development and I have assumed that it's not a show stopper to > you. > >>>> Let me know if it blocks you). > >>>> > >>>> A question to Sherman - do we have adequate tests to verify the move > of > >>>> System.loadLibrary("zip") from system initialization to ZipFile > >>>> initialization for 8u? > >>>> > >>>> Mandy > >>>> > >>>> > >>>> On 06/09/2015 10:09 AM, Volker Simonis wrote: > >>>>> > >>>>> Hi Mandy, > >>>>> > >>>>> thanks for looking into this. > >>>>> Uunfortunately your fix only helps to fix "java -version" :( > >>>>> > >>>>> Running even a minimal HelloWorld will still fail with the following > >>>>> stack trace which is still caused by the same EmptyStackException: > >>>>> > >>>>> Error: A JNI error has occurred, please check your installation and > try > >>>>> again > >>>>> Exception in thread "main" java.lang.ExceptionInInitializerError > >>>>> at sun.misc.Unsafe.ensureClassInitialized(Native Method) > >>>>> at > >>>>> > sun.misc.SharedSecrets.getJavaUtilZipFileAccess(SharedSecrets.java:158) > >>>>> at sun.misc.URLClassPath$JarLoader.(URLClassPath.java:765) > >>>>> at sun.misc.URLClassPath$3.run(URLClassPath.java:530) > >>>>> at sun.misc.URLClassPath$3.run(URLClassPath.java:520) > >>>>> at java.security.AccessController.doPrivileged(Native Method) > >>>>> at sun.misc.URLClassPath.getLoader(URLClassPath.java:519) > >>>>> at sun.misc.URLClassPath.getLoader(URLClassPath.java:492) > >>>>> at sun.misc.URLClassPath.getNextLoader(URLClassPath.java:457) > >>>>> at sun.misc.URLClassPath.access$100(URLClassPath.java:64) > >>>>> at sun.misc.URLClassPath$1.next(URLClassPath.java:239) > >>>>> at sun.misc.URLClassPath$1.hasMoreElements(URLClassPath.java:250) > >>>>> at java.net.URLClassLoader$3$1.run(URLClassLoader.java:601) > >>>>> at java.net.URLClassLoader$3$1.run(URLClassLoader.java:599) > >>>>> at java.security.AccessController.doPrivileged(Native Method) > >>>>> at java.net.URLClassLoader$3.next(URLClassLoader.java:598) > >>>>> at > java.net.URLClassLoader$3.hasMoreElements(URLClassLoader.java:623) > >>>>> at sun.misc.CompoundEnumeration.next(CompoundEnumeration.java:45) > >>>>> at > >>>>> > sun.misc.CompoundEnumeration.hasMoreElements(CompoundEnumeration.java:54) > >>>>> at sun.misc.CompoundEnumeration.next(CompoundEnumeration.java:45) > >>>>> at > >>>>> > sun.misc.CompoundEnumeration.hasMoreElements(CompoundEnumeration.java:54) > >>>>> at > >>>>> > java.util.ServiceLoader$LazyIterator.hasNextService(ServiceLoader.java:354) > >>>>> at > >>>>> java.util.ServiceLoader$LazyIterator.hasNext(ServiceLoader.java:393) > >>>>> at java.util.ServiceLoader$1.hasNext(ServiceLoader.java:474) > >>>>> at java.nio.charset.Charset$1.getNext(Charset.java:350) > >>>>> at java.nio.charset.Charset$1.hasNext(Charset.java:365) > >>>>> at java.nio.charset.Charset$2.run(Charset.java:410) > >>>>> at java.nio.charset.Charset$2.run(Charset.java:407) > >>>>> at java.security.AccessController.doPrivileged(Native Method) > >>>>> at java.nio.charset.Charset.lookupViaProviders(Charset.java:406) > >>>>> at java.nio.charset.Charset.lookup2(Charset.java:477) > >>>>> at java.nio.charset.Charset.lookup(Charset.java:464) > >>>>> at java.nio.charset.Charset.isSupported(Charset.java:505) > >>>>> at > >>>>> > sun.launcher.LauncherHelper.makePlatformString(LauncherHelper.java:580) > >>>>> Caused by: java.util.EmptyStackException > >>>>> at java.util.Stack.peek(Stack.java:102) > >>>>> at > >>>>> > java.lang.ClassLoader$NativeLibrary.getFromClass(ClassLoader.java:1759) > >>>>> at java.lang.ClassLoader$NativeLibrary.findBuiltinLib(Native > Method) > >>>>> at java.lang.ClassLoader.loadLibrary0(ClassLoader.java:1870) > >>>>> at java.lang.ClassLoader.loadLibrary(ClassLoader.java:1843) > >>>>> at java.lang.Runtime.loadLibrary0(Runtime.java:870) > >>>>> at java.lang.System.loadLibrary(System.java:1122) > >>>>> at java.util.zip.ZipFile.(ZipFile.java:86) > >>>>> ... 34 more > >>>>> > >>>>> It's obvious that the way jni_FindClass is looking for the class > >>>>> context by calling the NativeLibrary::getFromClass method is hacky > but > >>>>> I think that the proposed fix is the quite simple and non-intrusive. > >>>>> And we probably don't want a complete rework of this code for 8 > >>>>> anyway. So why not fix it the proposed way in 8 and 9 for now? That > >>>>> will still leave us time to come up with a better clean-up at least > >>>>> for 9. > >>>>> > >>>>> What do you think? > >>>>> > >>>>> Regards, > >>>>> Volker > >>>>> > >>>>> On Tue, Jun 9, 2015 at 1:35 AM, Mandy Chung > > >>>>> wrote: > >>>>>> > >>>>>> Hi Volker, > >>>>>> > >>>>>> Your patch reminds me the question I have about loading zip library. > >>>>>> > >>>>>> In JDK 9 (and earlier release), zip library is loaded by the VM > during > >>>>>> startup (see ClassLoader::load_zip_library). I think > loadLibrary("zip") > >>>>>> is > >>>>>> no longer needed to be called from System::initializeSystemClass > method > >>>>>> and > >>>>>> instead it can be loaded by java.util.zip.ZipFile static > initializer. > >>>>>> > >>>>>> Do you mind to try the patch (below)? That may be a simpler fix. > >>>>>> > >>>>>> I never like the way jni_FindClass to look for the class context by > >>>>>> calling > >>>>>> the NativeLibrary::getFromClass method. I will have to look deeper > other > >>>>>> alternative to clean that up. If taking out loadLibrary("zip") > resolves > >>>>>> your issue, this will give us time to come up with a better > clean-up in > >>>>>> the > >>>>>> future. > >>>>>> > >>>>>> Mandy > >>>>>> [1] https://bugs.openjdk.java.net/browse/JDK-4429040 > >>>>>> > >>>>>> diff --git a/src/share/classes/java/lang/System.java > >>>>>> b/src/share/classes/java/lang/System.java > >>>>>> --- a/src/share/classes/java/lang/System.java > >>>>>> +++ b/src/share/classes/java/lang/System.java > >>>>>> @@ -1192,10 +1192,6 @@ > >>>>>> setOut0(newPrintStream(fdOut, > >>>>>> props.getProperty("sun.stdout.encoding"))); > >>>>>> setErr0(newPrintStream(fdErr, > >>>>>> props.getProperty("sun.stderr.encoding"))); > >>>>>> > >>>>>> - // Load the zip library now in order to keep > >>>>>> java.util.zip.ZipFile > >>>>>> - // from trying to use itself to load this library later. > >>>>>> - loadLibrary("zip"); > >>>>>> - > >>>>>> // Setup Java signal handlers for HUP, TERM, and INT (where > >>>>>> available). > >>>>>> Terminator.setup(); > >>>>>> > >>>>>> diff --git a/src/share/classes/java/util/zip/ZipFile.java > >>>>>> b/src/share/classes/java/util/zip/ZipFile.java > >>>>>> --- a/src/share/classes/java/util/zip/ZipFile.java > >>>>>> +++ b/src/share/classes/java/util/zip/ZipFile.java > >>>>>> @@ -83,6 +83,7 @@ > >>>>>> > >>>>>> static { > >>>>>> /* Zip library is loaded from System.initializeSystemClass > */ > >>>>>> + System.loadLibrary("zip"); > >>>>>> initIDs(); > >>>>>> > >>>>>> } > >>>>>> > >>>>>> > >>>>>> > >>>>>> On 06/08/2015 07:23 AM, Volker Simonis wrote: > >>>>>>> > >>>>>>> Hi, > >>>>>>> > >>>>>>> can I please get a review at least for the part of this fix which > is > >>>>>>> common for jdk8 and jdk9. It's only a few lines in > >>>>>>> src/share/vm/prims/jni.cpp for the hotspot repository and one line > >>>>>>> src/java.base/share/classes/java/lang/ClassLoader.java for the jdk > >>>>>>> repository. > >>>>>>> > >>>>>>> Thanks, > >>>>>>> Volker > >>>>>>> > >>>>>>> > >>>>>>> On Tue, Jun 2, 2015 at 6:12 PM, Volker Simonis > >>>>>>> > > >>>>>>> wrote: > >>>>>>>> > >>>>>>>> Hi, > >>>>>>>> > >>>>>>>> can I please have review (and a sponsor) for these changes: > >>>>>>>> > >>>>>>>> https://bugs.openjdk.java.net/browse/JDK-8081674 > >>>>>>>> http://cr.openjdk.java.net/~simonis/webrevs/2015/8081674.jdk > >>>>>>>> http://cr.openjdk.java.net/~simonis/webrevs/2015/8081674.hs > >>>>>>>> > >>>>>>>> Please notice that the fix requires synchronous changes in the > jdk and > >>>>>>>> the > >>>>>>>> hotspot forest. > >>>>>>>> > >>>>>>>> The changes themselves are by far not that big as this > explanation but > >>>>>>>> I > >>>>>>>> found the problem to be quite intricate so I tried to explain it > as > >>>>>>>> good > >>>>>>>> as > >>>>>>>> I could. I'd suggest to read the HTML-formatted explanation in the > >>>>>>>> webrev > >>>>>>>> although the content is equal to the one in this mail: > >>>>>>>> > >>>>>>>> Using an unsupported character encoding (e.g. vi_VN.TCVN on Linux) > >>>>>>>> results > >>>>>>>> in an immediate VM failure with jdk 8 and 9: > >>>>>>>> > >>>>>>>> export LANG=vi_VN.TCVN > >>>>>>>> java -version > >>>>>>>> Error occurred during initialization of VM > >>>>>>>> java.util.EmptyStackException > >>>>>>>> at java.util.Stack.peek(Stack.java:102) > >>>>>>>> at > >>>>>>>> > java.lang.ClassLoader$NativeLibrary.getFromClass(ClassLoader.java:1751) > >>>>>>>> at > java.lang.ClassLoader$NativeLibrary.findBuiltinLib(Native > >>>>>>>> Method) > >>>>>>>> at > java.lang.ClassLoader.loadLibrary0(ClassLoader.java:1862) > >>>>>>>> at java.lang.ClassLoader.loadLibrary(ClassLoader.java:1835) > >>>>>>>> at java.lang.Runtime.loadLibrary0(Runtime.java:870) > >>>>>>>> at java.lang.System.loadLibrary(System.java:1119) > >>>>>>>> at java.lang.System.initializeSystemClass(System.java:1194) > >>>>>>>> > >>>>>>>> This is a consequence of "8005716: Enhance JNI specification to > allow > >>>>>>>> support of static JNI libraries in Embedded JREs". > >>>>>>>> > >>>>>>>> With jdk 9 we get this error even if we're running with a > supported > >>>>>>>> charset > >>>>>>>> which is in the ExtendedCharsets (as opposed to being in the > >>>>>>>> StandardCharsets) class which is a consequence of delegating the > >>>>>>>> loading > >>>>>>>> of > >>>>>>>> the ExtendedCharsets class to the ServiceLoader in jdk 9. > >>>>>>>> > >>>>>>>> export LANG=eo.iso-8859-3 > >>>>>>>> output-jdk9/images/jdk/bin/java -version > >>>>>>>> Error occurred during initialization of VM > >>>>>>>> java.util.EmptyStackException > >>>>>>>> at java.util.Stack.peek(Stack.java:102) > >>>>>>>> at > >>>>>>>> > java.lang.ClassLoader$NativeLibrary.getFromClass(ClassLoader.java:1737) > >>>>>>>> at > java.lang.ClassLoader$NativeLibrary.findBuiltinLib(Native > >>>>>>>> Method) > >>>>>>>> at > java.lang.ClassLoader.loadLibrary0(ClassLoader.java:1866) > >>>>>>>> at java.lang.ClassLoader.loadLibrary(ClassLoader.java:1840) > >>>>>>>> at java.lang.Runtime.loadLibrary0(Runtime.java:874) > >>>>>>>> at java.lang.System.loadLibrary(System.java:1111) > >>>>>>>> at java.lang.System.initializeSystemClass(System.java:1186) > >>>>>>>> > >>>>>>>> Here's why the exception happens for an unsupported charset (see > the > >>>>>>>> mixed > >>>>>>>> stack trace below for the full details): > >>>>>>>> > >>>>>>>> java.lang.System.loadLibrary() wants to load libzip.so. It calls > >>>>>>>> java.lang.Runtime.loadLibrary0() which at the very beginning > calls the > >>>>>>>> native method ClassLoader$NativeLibrary.findBuiltinLib() which > checks > >>>>>>>> if > >>>>>>>> the > >>>>>>>> corresponding library is already statically linked into the VM > >>>>>>>> (introduced > >>>>>>>> by 8005716). > >>>>>>>> Java_java_lang_ClassLoader_00024NativeLibrary_findBuiltinLib(), > >>>>>>>> the native implementation of findBuiltinLib() in Classloader.c > calls > >>>>>>>> GetStringPlatformChars() to convert the library name into the > native > >>>>>>>> platform encoding. GetStringPlatformChars() calls the helper > function > >>>>>>>> jnuEncodingSupported() to check if the platform encoding which is > >>>>>>>> stored > >>>>>>>> in > >>>>>>>> the property "sun.jnu.encoding" is supported by Java. > >>>>>>>> jnuEncodingSupported() > >>>>>>>> is implemented as follows: > >>>>>>>> > >>>>>>>> static jboolean isJNUEncodingSupported = JNI_FALSE; > >>>>>>>> static jboolean jnuEncodingSupported(JNIEnv *env) { > >>>>>>>> jboolean exe; > >>>>>>>> if (isJNUEncodingSupported == JNI_TRUE) { > >>>>>>>> return JNI_TRUE; > >>>>>>>> } > >>>>>>>> isJNUEncodingSupported = (jboolean) > JNU_CallStaticMethodByName ( > >>>>>>>> env, &exe, > >>>>>>>> "java/nio/charset/Charset", > >>>>>>>> "isSupported", > >>>>>>>> "(Ljava/lang/String;)Z", > >>>>>>>> jnuEncoding).z; > >>>>>>>> return isJNUEncodingSupported; > >>>>>>>> } > >>>>>>>> > >>>>>>>> Once the function finds that the platform encoding is supported > (by > >>>>>>>> calling > >>>>>>>> java.nio.charset.Charset.isSupported()) it caches this value and > always > >>>>>>>> returns it on following calls. However if the platform encoding > is not > >>>>>>>> supported, it ALWAYS calls java.nio.charset.Charset.isSupported() > an > >>>>>>>> every > >>>>>>>> subsequent invocation. > >>>>>>>> > >>>>>>>> In order to call the Java method Charset.isSupported() (in > >>>>>>>> JNU_CallStaticMethodByName() in file jni_util.c), we have to call > >>>>>>>> jni_FindClass() to convert the symbolic class name > >>>>>>>> "java.nio.charset.Charset" into a class reference. > >>>>>>>> > >>>>>>>> But unfortunately, jni_FindClass() (from jni.cpp in libjvm.so) > has a > >>>>>>>> special > >>>>>>>> handling if called from java.lang.ClassLoader$NativeLibrary to > ensure > >>>>>>>> that > >>>>>>>> JNI_OnLoad/JNI_OnUnload are executed in the correct class context: > >>>>>>>> > >>>>>>>> instanceKlassHandle k (THREAD, > >>>>>>>> thread->security_get_caller_class(0)); > >>>>>>>> if (k.not_null()) { > >>>>>>>> loader = Handle(THREAD, k->class_loader()); > >>>>>>>> // Special handling to make sure JNI_OnLoad and JNI_OnUnload > are > >>>>>>>> executed > >>>>>>>> // in the correct class context. > >>>>>>>> if (loader.is_null() && > >>>>>>>> k->name() == > >>>>>>>> vmSymbols::java_lang_ClassLoader_NativeLibrary()) { > >>>>>>>> JavaValue result(T_OBJECT); > >>>>>>>> JavaCalls::call_static(&result, k, > >>>>>>>> > vmSymbols::getFromClass_name(), > >>>>>>>> > >>>>>>>> vmSymbols::void_class_signature(), > >>>>>>>> thread); > >>>>>>>> if (HAS_PENDING_EXCEPTION) { > >>>>>>>> Handle ex(thread, thread->pending_exception()); > >>>>>>>> CLEAR_PENDING_EXCEPTION; > >>>>>>>> THROW_HANDLE_0(ex); > >>>>>>>> } > >>>>>>>> > >>>>>>>> So if that's the case and jni_FindClass() was reallycalled from > >>>>>>>> ClassLoader$NativeLibrary, then jni_FindClass() calles > >>>>>>>> ClassLoader$NativeLibrary().getFromClass() to find out the > >>>>>>>> corresponding > >>>>>>>> context class which is supposed to be saved there in a field of > type > >>>>>>>> java.util.Stack named "nativeLibraryContext": > >>>>>>>> > >>>>>>>> // Invoked in the VM to determine the context class in > >>>>>>>> // JNI_Load/JNI_Unload > >>>>>>>> static Class getFromClass() { > >>>>>>>> return ClassLoader.nativeLibraryContext.peek().fromClass; > >>>>>>>> } > >>>>>>>> > >>>>>>>> Unfortunately, "nativeLibraryContext" doesn't contain any entry > at this > >>>>>>>> point and the invocation of Stack.peek() will throw the exception > shown > >>>>>>>> before. In general, the "nativeLibraryContext" stack will be > filled > >>>>>>>> later > >>>>>>>> on > >>>>>>>> in Runtime.loadLibrary0() like this: > >>>>>>>> > >>>>>>>> NativeLibrary lib = new NativeLibrary(fromClass, name, isBuiltin); > >>>>>>>> nativeLibraryContext.push(lib); > >>>>>>>> try { > >>>>>>>> lib.load(name, isBuiltin); > >>>>>>>> } finally { > >>>>>>>> nativeLibraryContext.pop(); > >>>>>>>> } > >>>>>>>> > >>>>>>>> such that it always contains at least one element later when > >>>>>>>> jni_FindClass() > >>>>>>>> will be invoked. > >>>>>>>> > >>>>>>>> So in summary, the problem is that the implementors of 8005716 > didn't > >>>>>>>> took > >>>>>>>> into account that calling > ClassLoader$NativeLibrary.findBuiltinLib() > >>>>>>>> may > >>>>>>>> trigger a call to jni_FindClass() if we are running on a system > with an > >>>>>>>> unsupported character encoding. > >>>>>>>> > >>>>>>>> I'd suggest the following fix for this problem: > >>>>>>>> > >>>>>>>> Change ClassLoader$NativeLibrary().getFromClass() to return null > if the > >>>>>>>> stack is empty instead of throwing an exception: > >>>>>>>> > >>>>>>>> static Class getFromClass() { > >>>>>>>> return ClassLoader.nativeLibraryContext.empty() ? > >>>>>>>> null : ClassLoader.nativeLibraryContext.peek().fromClass; > >>>>>>>> } > >>>>>>>> > >>>>>>>> Unfortunately this also requires a HotSpot change in > jni_FindClass() in > >>>>>>>> order to properly handle the new 'null' return value: > >>>>>>>> > >>>>>>>> if (k.not_null()) { > >>>>>>>> loader = Handle(THREAD, k->class_loader()); > >>>>>>>> // Special handling to make sure JNI_OnLoad and JNI_OnUnload > are > >>>>>>>> executed > >>>>>>>> // in the correct class context. > >>>>>>>> if (loader.is_null() && > >>>>>>>> k->name() == > >>>>>>>> vmSymbols::java_lang_ClassLoader_NativeLibrary()) { > >>>>>>>> JavaValue result(T_OBJECT); > >>>>>>>> JavaCalls::call_static(&result, k, > >>>>>>>> > vmSymbols::getFromClass_name(), > >>>>>>>> > >>>>>>>> vmSymbols::void_class_signature(), > >>>>>>>> thread); > >>>>>>>> if (HAS_PENDING_EXCEPTION) { > >>>>>>>> Handle ex(thread, thread->pending_exception()); > >>>>>>>> CLEAR_PENDING_EXCEPTION; > >>>>>>>> THROW_HANDLE_0(ex); > >>>>>>>> } > >>>>>>>> oop mirror = (oop) result.get_jobject(); > >>>>>>>> if (oopDesc::is_null(mirror)) { > >>>>>>>> loader = Handle(THREAD, > >>>>>>>> SystemDictionary::java_system_loader()); > >>>>>>>> } else { > >>>>>>>> loader = Handle(THREAD, > >>>>>>>> > >>>>>>>> > >>>>>>>> > InstanceKlass::cast(java_lang_Class::as_Klass(mirror))->class_loader()); > >>>>>>>> protection_domain = Handle(THREAD, > >>>>>>>> > >>>>>>>> > >>>>>>>> > >>>>>>>> > InstanceKlass::cast(java_lang_Class::as_Klass(mirror))->protection_domain()); > >>>>>>>> } > >>>>>>>> } > >>>>>>>> } else { > >>>>>>>> // We call ClassLoader.getSystemClassLoader to obtain the > system > >>>>>>>> class > >>>>>>>> loader. > >>>>>>>> loader = Handle(THREAD, > SystemDictionary::java_system_loader()); > >>>>>>>> } > >>>>>>>> > >>>>>>>> These changes are sufficient to solve the problem in Java 8. > >>>>>>>> Unfortunately, > >>>>>>>> that's still not enough in Java 9 because there the loading of the > >>>>>>>> extended > >>>>>>>> charsets has been delegated to ServiceLoader. But ServiceLoader > calls > >>>>>>>> ClassLoader.getBootstrapResources() which calls > >>>>>>>> sun.misc.Launcher.getBootstrapClassPath(). This leads to another > >>>>>>>> problem > >>>>>>>> during class initialization of sun.misc.Launcher if running on an > >>>>>>>> unsupported locale. > >>>>>>>> > >>>>>>>> The first thing done in sun.misc.Launcher. is the > >>>>>>>> initialisation > >>>>>>>> of > >>>>>>>> the bootstrap URLClassPath in the Launcher. However this > initialisation > >>>>>>>> will > >>>>>>>> eventually call Charset.isSupported() and if we are running on an > >>>>>>>> unsupported locale this will inevitably end in another recursive > call > >>>>>>>> to > >>>>>>>> ServiceLoader. But as explained below, ServiceLoader will query > the > >>>>>>>> Launcher's bootstrap URLClassPath which will be still > uninitialized at > >>>>>>>> that > >>>>>>>> point. > >>>>>>>> > >>>>>>>> So we'll have to additionally guard guard against this situation > on JDK > >>>>>>>> 9 > >>>>>>>> like this: > >>>>>>>> > >>>>>>>> private static Charset lookupExtendedCharset(String charsetName) { > >>>>>>>> if (!sun.misc.VM.isBooted() || // see > >>>>>>>> lookupViaProviders() > >>>>>>>> sun.misc.Launcher.getBootstrapClassPath() == null) > >>>>>>>> return null; > >>>>>>>> > >>>>>>>> This fixes the crashes, but still at the price of not having the > >>>>>>>> extended > >>>>>>>> charsets available during initialization until > >>>>>>>> Launcher.getBootstrapClassPath is set up properly. This may be > still a > >>>>>>>> problem if the jdk is installed in a directory which contains > >>>>>>>> characters > >>>>>>>> specific to an extended encoding or if we have such characters in > the > >>>>>>>> command line arguments. > >>>>>>>> > >>>>>>>> Thank you and best regards, > >>>>>>>> Volker > >>>>>>>> > >>>>>>>> > >>>>>>>> Mixed stack trace of the initial EmptyStackException for > unsupported > >>>>>>>> charsets described before: > >>>>>>>> > >>>>>>>> Native frames: (J=compiled Java code, j=interpreted, Vv=VM code, > >>>>>>>> C=native > >>>>>>>> code) > >>>>>>>> j java.util.Stack.peek()Ljava/lang/Object;+1 > >>>>>>>> j > >>>>>>>> > java.lang.ClassLoader$NativeLibrary.getFromClass()Ljava/lang/Class;+3 > >>>>>>>> v ~StubRoutines::call_stub > >>>>>>>> V [libjvm.so+0x9d279a] JavaCalls::call_helper(JavaValue*, > >>>>>>>> methodHandle*, > >>>>>>>> JavaCallArguments*, Thread*)+0x6b4 > >>>>>>>> V [libjvm.so+0xcad591] os::os_exception_wrapper(void > (*)(JavaValue*, > >>>>>>>> methodHandle*, JavaCallArguments*, Thread*), JavaValue*, > methodHandle*, > >>>>>>>> JavaCallArguments*, Thread*)+0x45 > >>>>>>>> V [libjvm.so+0x9d20cf] JavaCalls::call(JavaValue*, methodHandle, > >>>>>>>> JavaCallArguments*, Thread*)+0x8b > >>>>>>>> V [libjvm.so+0x9d1d3b] JavaCalls::call_static(JavaValue*, > >>>>>>>> KlassHandle, > >>>>>>>> Symbol*, Symbol*, JavaCallArguments*, Thread*)+0x139 > >>>>>>>> V [libjvm.so+0x9d1e3f] JavaCalls::call_static(JavaValue*, > >>>>>>>> KlassHandle, > >>>>>>>> Symbol*, Symbol*, Thread*)+0x9d > >>>>>>>> V [libjvm.so+0x9e6588] jni_FindClass+0x428 > >>>>>>>> C [libjava.so+0x20208] JNU_CallStaticMethodByName+0xff > >>>>>>>> C [libjava.so+0x21cae] jnuEncodingSupported+0x61 > >>>>>>>> C [libjava.so+0x22125] JNU_GetStringPlatformChars+0x125 > >>>>>>>> C [libjava.so+0xedcd] > >>>>>>>> Java_java_lang_ClassLoader_00024NativeLibrary_findBuiltinLib+0x8b > >>>>>>>> j > >>>>>>>> > >>>>>>>> > >>>>>>>> > java.lang.ClassLoader$NativeLibrary.findBuiltinLib(Ljava/lang/String;)Ljava/lang/String;+0 > >>>>>>>> j > >>>>>>>> > java.lang.ClassLoader.loadLibrary0(Ljava/lang/Class;Ljava/io/File;)Z+4 > >>>>>>>> j > >>>>>>>> > >>>>>>>> > >>>>>>>> > java.lang.ClassLoader.loadLibrary(Ljava/lang/Class;Ljava/lang/String;Z)V+228 > >>>>>>>> j > >>>>>>>> > java.lang.Runtime.loadLibrary0(Ljava/lang/Class;Ljava/lang/String;)V+54 > >>>>>>>> j java.lang.System.loadLibrary(Ljava/lang/String;)V+7 > >>>>>>>> j java.lang.System.initializeSystemClass()V+113 > >>>>>>>> v ~StubRoutines::call_stub > >>>>>>>> V [libjvm.so+0x9d279a] JavaCalls::call_helper(JavaValue*, > >>>>>>>> methodHandle*, > >>>>>>>> JavaCallArguments*, Thread*)+0x6b4 > >>>>>>>> V [libjvm.so+0xcad591] os::os_exception_wrapper(void > (*)(JavaValue*, > >>>>>>>> methodHandle*, JavaCallArguments*, Thread*), JavaValue*, > methodHandle*, > >>>>>>>> JavaCallArguments*, Thread*)+0x45 > >>>>>>>> V [libjvm.so+0x9d20cf] JavaCalls::call(JavaValue*, methodHandle, > >>>>>>>> JavaCallArguments*, Thread*)+0x8b > >>>>>>>> V [libjvm.so+0x9d1d3b] JavaCalls::call_static(JavaValue*, > >>>>>>>> KlassHandle, > >>>>>>>> Symbol*, Symbol*, JavaCallArguments*, Thread*)+0x139 > >>>>>>>> V [libjvm.so+0x9d1e3f] JavaCalls::call_static(JavaValue*, > >>>>>>>> KlassHandle, > >>>>>>>> Symbol*, Symbol*, Thread*)+0x9d > >>>>>>>> V [libjvm.so+0xe3cceb] call_initializeSystemClass(Thread*)+0xb0 > >>>>>>>> V [libjvm.so+0xe44444] > >>>>>>>> Threads::initialize_java_lang_classes(JavaThread*, > >>>>>>>> Thread*)+0x21a > >>>>>>>> V [libjvm.so+0xe44b12] Threads::create_vm(JavaVMInitArgs*, > >>>>>>>> bool*)+0x4a6 > >>>>>>>> V [libjvm.so+0xa19bd7] JNI_CreateJavaVM+0xc7 > >>>>>>>> C [libjli.so+0xa520] InitializeJVM+0x154 > >>>>>>>> C [libjli.so+0x8024] JavaMain+0xcc > >>>>>>>> > >>>>>>>> > >>>> > >> > > From brian.burkhalter at oracle.com Wed Jun 10 23:05:41 2015 From: brian.burkhalter at oracle.com (Brian Burkhalter) Date: Wed, 10 Jun 2015 16:05:41 -0700 Subject: [9] RFR of 8081843: (fs) FileStore.getTotalSpace returns unexpected results with >2TB file systems In-Reply-To: <2D69D5D5-EB8B-4235-A546-7C9BD3D06872@oracle.com> References: <2D69D5D5-EB8B-4235-A546-7C9BD3D06872@oracle.com> Message-ID: <59920CE5-D926-42A9-AAF8-326045D768E9@oracle.com> Redirecting this thread here http://mail.openjdk.java.net/pipermail/nio-dev/2015-June/003187.html to where it should have gone. On Jun 10, 2015, at 3:57 PM, Brian Burkhalter wrote: > Please review at your convenience. > > Issue: https://bugs.openjdk.java.net/browse/JDK-8081843 > Patch: http://cr.openjdk.java.net/~bpb/8081843/webrev.00/ > > Summary: On Mac OS X use BSD system call statfs() instead of Standard C Library call statvfs() to obtain file system size statistics. > > Standard regression test run succeeded. > > Thanks, > > Brian > From david.holmes at oracle.com Thu Jun 11 04:52:53 2015 From: david.holmes at oracle.com (David Holmes) Date: Thu, 11 Jun 2015 14:52:53 +1000 Subject: RFR: JDK-8085822 JEP 223: New Version-String Scheme (initial integration) In-Reply-To: <557829DC.8000405@oracle.com> References: <5571AD18.7080701@oracle.com> <557625C8.5050200@oracle.com> <5576E633.9050503@oracle.com> <5576E81B.8050703@oracle.com> <5576EF94.3010500@oracle.com> <55780A33.1010302@oracle.com> <557829DC.8000405@oracle.com> Message-ID: <55791425.8070709@oracle.com> Hi Magnus, On 10/06/2015 10:13 PM, Magnus Ihse Bursie wrote: > On 2015-06-10 11:58, David Holmes wrote: >> Hi Magnus, >> >> Generally looks good - a few comments/queries below. > > In general, I believe most issues you found are valid. :-) However, as I > said before in this thread, I'd like to see them resolved in the needed > follow-up patches. The source code changes in Hotspot and JDK are > inadequate to fully implement JEP-223, so these areas will need to be > rewritten anyhow. Are you okay with resolving these issues later? Given this is going to a staging repo I'm okay with deferring things - I just have a concern whether such things may be overlooked given that the integration with the staging repo will not be undergoing a final review. I would prefer to see the Version.java.template doc comments corrected, even if the issue of whether to add and deprecate is deferred, but again as this is to a staging area I can let it slide for now. I saw the fix to hotspot/src/share/vm/runtime/vmStructs.cpp. Thanks, David >> >> On 9/06/2015 11:52 PM, Magnus Ihse Bursie wrote: >>> Here's an updated webrev, which fixes the typos that were pointed out by >>> reviewers: >>> http://cr.openjdk.java.net/~ihse/JDK-8085822-JEP-223-initial-patch/webrev.02/ >>> >> >> common/autoconf/version-numbers >> >> Shouldn't there be a DEFAULT_VERSION_XXX for each of the XXX parts of >> the version info? (Even if all zeroes presently.) > So that's a tricky one. :-& The question here is, I think, does it make > sense to update version-numbers when we do a security (9.0.1) or minor > (9.1.0) release? Looking historically, the version-numbers file have not > been changed for the 8u family. The only change was going from 8 to 9 > when creating the new jdk9 forest. That was what I based my decition to > only have the major number in the file. (The rest of the version string > is set by configure flags when building, in Oracle case by the RE team.) > > If it makes sense to put the minor/security/patch numbers here, I won't > oppose it, but I guess we have until the first such release to figure > out what's best, and that likely includes discussion with RE and > possibly other consumers in the community (RedHat etc). > >> >> --- >> >> Looking at hotspot changes I can't convince myself that the new >> streamlined "version" variables will capture things like "fastdebug". >> Will that filter through via configure variables? > > The "fastdebug" label has been handled as a less valued token in the > JEP-223 process. Right now there's no mention of it at all in the > version string proposal in the JEP. As I understand it, Iris is about to > propose an amendment which will just make fastdebug be a part of the OPT > string. I'm not entirely happy with that myself, but that's the way it's > decided. So wherever you can see the OPT string, you'll see the debug > level tag. > > Currently, however, that's not how it is implemented in this patch. > Since no decision was made on this (and it's still not formally > decided), I had to pick a route to go forward. The current patch will > instead put the "fastdebug" label as part of the PRE string (that's the > reason for the pre-base and pre-debuglevel arguments to configure). If > the planned changes goes into the JEP, this'll change to make the > debuglevel tag a part of the OPT string instead. > >> --- >> >> make/*/makefiles/vm.make >> >> Shouldn't the -DVERSION_XX=$(VERSION_XX) definitions be taken from the >> VERSION_CFLAGS in spec.gmk? (Or are you still allowing for a >> stand-alone hotspot build?) > The hotspot JEP-223 work initially made by Alejandro kept support for > stand-alone hotspot builds. I made additional changes on top of that, > which still tried to cater for stand-alone builds. At this very moment > I'm not sure if stand-alone hotspot builds are supposed to work or not, > but I've tried to not change the situation with this patch. > > There are two future changes coming down the pipe: One is the additional > JEP-223 work needed for Hotspot. I know Alejandro had plans for > redesigning the API between Hotspot and the JDK, so no JDK version > strings should be compiled into the JVM, but all of it extracted during > an API in runtime. That would make most (all?) of the makefile changes > in hotspot irrelevant. However, that implementation is not even started, > so it's needed for the time being. > > The second change is the build-infra hotspot makefile rewrite. When that > happens, stand-alone builds will definitely disappear, and if Hotspot > still needs make support for version strings, then the logical choice is > to use VERSION_CFLAGS. > > Ok? > >> >> --- >> >> hotspot/src/share/vm/prims/jvm.h >> jdk/src/java.base/share/native/include/jvm.h >> >> I think this comment is way out of date: >> >> /* Build number is available only for RE build (i.e. JDK_BUILD_NUMBER >> is set to bNN) >> * It will be zero for internal builds. >> */ >> >> and can be completely removed. Even if still true, mention of an "RE >> build" has no place in OpenJDK sources. > Yep, agree. Follow-up patch, right? > >> >> --- >> >> hotspot/src/share/vm/runtime/java.cpp >> >> I think a lot of the modified code is obsolete post Hotspot Express - >> which makes it hard to determine if the changes make sense. > Yep, agree. Follow-up patch, right? >> >> --- >> >> hotspot/src/share/vm/runtime/vmStructs.cpp >> >> Please fix the alignment (3 extra spaces on modified line): >> >> 1239 static_field(Abstract_VM_Version, _vm_minor_version, >> int) \ >> 1240 static_field(Abstract_VM_Version, >> _vm_security_version, int) \ > > I was about to say "follow-up patch", but ugly indentation really sucks > so I can fix this. Ok if I skip redoing the review for that? > >> >> --- >> >> hotspot/test/runtime/6981737/Test6981737.java >> >> This test is really only valid for the new version scheme now, so it >> should probably be rewritten - and in that case it really isn't >> testing 6981737 but should be replaced by a new test for the new >> version format. > Yep, agree. Follow-up patch, right? >> >> --- >> >> jdk/src/java.base/share/classes/sun/misc/Version.java.template >> >> This comment is nonsensical: >> >> /** >> ! * Returns the security version of the running JVM if it's 1.6 >> or newer >> * or any RE VM build. It will return 0 if it's an internal 1.5 or >> * 1.4.x build. >> * @since 1.6 >> */ >> >> as security version does not exist pre 9. Normally you should be >> adding a new method and deprecating the old one. The new one is @since 9. > Yep, agree. Follow-up patch, right? >> >> /** >> ! * Returns the security version of the running JDK. >> * @since 1.6 >> */ >> >> Ditto: @since 9 (but again old should be deprecated and new method added) >> >> 253 /** >> 254 * Returns the build number of the running JDK if it's a RE >> build >> 255 * It will return 0 if it's an internal build. >> >> As with jvm.h this seems obsolete commentary these days - not only RE >> builds define a build number. > Yep, agree. Follow-up patch, right? > > /Magnus > >> >> Thanks, >> David > From david.holmes at oracle.com Thu Jun 11 05:35:57 2015 From: david.holmes at oracle.com (David Holmes) Date: Thu, 11 Jun 2015 15:35:57 +1000 Subject: RFR 9: 8086117: java/lang/Runtime/exec/LotsOfOutput.java still fails intermittently with Process consumes memory In-Reply-To: <55783BB1.2080306@gmail.com> References: <5578098C.4030103@Oracle.com> <55783BB1.2080306@gmail.com> Message-ID: <55791E3D.8010905@oracle.com> On 10/06/2015 11:29 PM, Peter Levart wrote: > Hi Roger, > > I don't know how the tests are executed in your environment, but is it > possible that, while you're running the test in a shared VM, some other > concurrent activity allocates from heap and that the test fails because > of that and not because of Process.exec() at all? I tend to agree - perhaps the test should be run in /othervm mode? David > Regards, Peter > > On 06/10/2015 11:55 AM, Roger Riggs wrote: >> Please review a test change in the algorithm for detecting memory >> growth to >> resolve an intermittent reporting problem. >> >> Webrev: >> http://cr.openjdk.java.net/~rriggs/webrev-lots-8086117/ >> >> Issue: >> https://bugs.openjdk.java.net/browse/JDK-8086117 >> >> Thanks, Roger > From erik.joelsson at oracle.com Thu Jun 11 06:58:41 2015 From: erik.joelsson at oracle.com (Erik Joelsson) Date: Thu, 11 Jun 2015 08:58:41 +0200 Subject: RFR: JDK-8085822 JEP 223: New Version-String Scheme (initial integration) In-Reply-To: <55783F49.6030906@oracle.com> References: <5571AD18.7080701@oracle.com> <557625C8.5050200@oracle.com> <55783F49.6030906@oracle.com> Message-ID: <557931A1.6030406@oracle.com> Looks good. /Erik On 2015-06-10 15:44, Magnus Ihse Bursie wrote: > On 2015-06-09 01:31, Daniel D. Daugherty wrote: >> > >> http://cr.openjdk.java.net/~ihse/JDK-8085822-JEP-223-initial-patch/webrev.01 >> >> >> General comment: Not all copyright years were updated. > > I realize I missed that part of the review. :-( > > I have now updated the copyright years. Here's a delta review: > > http://cr.openjdk.java.net/~ihse/JDK-8085822-JEP-223-initial-patch-delta-02/webrev.01/ > > > Here's the complete review once again: > > http://cr.openjdk.java.net/~ihse/JDK-8085822-JEP-223-initial-patch/webrev.03 > > > Hopefully this is now the final version to be pushed to verona, and > that any additional problems can be resolved with follow-up patches. > > /Magnus From Alan.Bateman at oracle.com Thu Jun 11 07:42:18 2015 From: Alan.Bateman at oracle.com (Alan Bateman) Date: Thu, 11 Jun 2015 08:42:18 +0100 Subject: RFR(M): 8081674: EmptyStackException at startup if running with extended or unsupported charset In-Reply-To: <7B3C08CF-028E-4AD2-B7F8-C803F2BA3C7D@oracle.com> References: <557626AA.6010008@oracle.com> <55772A56.2030308@oracle.com> <7B3C08CF-028E-4AD2-B7F8-C803F2BA3C7D@oracle.com> Message-ID: <55793BDA.8000204@oracle.com> On 10/06/2015 20:02, Mandy Chung wrote: > : > ClassLoader.java > It should have done that. Nit: > Can you move line 1693 to 1868 closer to loadLibrary0 (the caller of findBuiltinLib)? > Also make it private. > > Same applies to jdk9. > > Otherwise looks good. No need to generate a webrev as long as you make the change before the push. Looks good to me too (assuming these changes). > : > >> On jdk9 we are now running in another problem which is caused by the >> fact that the extended charsets are now being loaded by ServiceLoader >> (I've detailed that already in the bug report). But I think we can >> leave the fix of that problem to another change as this seems to >> require some more reasoning (see Alan's comments in the bug report). > Yes this is a harder problem that have to investigate further. > Right, the main issue here is that java.base must contain all charsets that are needed during start up / VM initialization. There is also no guarantee the jdk.charsets module will be linked into the run-time image. There are two scenarios that come from this. One is the LANG setting as we have here, the other is attempting to run with file.encoding set. The choice as to whether to fail gracefully or switch to UTF-8 might be different for these two scenarios, it just investigation and a proposal. -Alan From Alan.Bateman at oracle.com Thu Jun 11 07:43:28 2015 From: Alan.Bateman at oracle.com (Alan Bateman) Date: Thu, 11 Jun 2015 08:43:28 +0100 Subject: RFR 9: 8064956 : Remove sun.misc.ExtensionInstallationProvider and relevant classes In-Reply-To: <5578B73B.7050703@oracle.com> References: <5578B73B.7050703@oracle.com> Message-ID: <55793C20.508@oracle.com> On 10/06/2015 23:16, Brent Christian wrote: > Hi, > > Please review my change for: > > Bug: > https://bugs.openjdk.java.net/browse/JDK-8064956 > Webrev: > http://cr.openjdk.java.net/~bchristi/8064956/ > > After clearing some internal dependencies following the removal of the > extension mechanism from JDK 9[1], we can now remove 4 > extension-related classes from sun.misc: > > ExtensionDependency.java > ExtensionInfo.java > ExtensionInstallationException.java > ExtensionInstallationProvider.java Good to see these going away. -Alan From Alan.Bateman at oracle.com Thu Jun 11 07:51:06 2015 From: Alan.Bateman at oracle.com (Alan Bateman) Date: Thu, 11 Jun 2015 08:51:06 +0100 Subject: RFR 9: 8086117: java/lang/Runtime/exec/LotsOfOutput.java still fails intermittently with Process consumes memory In-Reply-To: <55791E3D.8010905@oracle.com> References: <5578098C.4030103@Oracle.com> <55783BB1.2080306@gmail.com> <55791E3D.8010905@oracle.com> Message-ID: <55793DEA.2090204@oracle.com> On 11/06/2015 06:35, David Holmes wrote: > On 10/06/2015 11:29 PM, Peter Levart wrote: >> Hi Roger, >> >> I don't know how the tests are executed in your environment, but is it >> possible that, while you're running the test in a shared VM, some other >> concurrent activity allocates from heap and that the test fails because >> of that and not because of Process.exec() at all? > > I tend to agree - perhaps the test should be run in /othervm mode? Sound right although just on Peter's comment, I'm not aware of any jtreg mode that runs tests concurrently in the same VM. The agent VM mode (that I assume most of us use nowadays) executes the tests sequentially in each VM. There is also the possibility of tests leaving looping threads behind or leaking resources and that can cause issues for tests that run subsequently in the same VM. The other thing is that it's possible to have a pool of agent VMs so many tests may be running concurrently in different VMs and this creates another form of potential interference. -Alan. From Roger.Riggs at Oracle.com Thu Jun 11 09:30:15 2015 From: Roger.Riggs at Oracle.com (Roger Riggs) Date: Thu, 11 Jun 2015 10:30:15 +0100 Subject: RFR 9: 8086117: java/lang/Runtime/exec/LotsOfOutput.java still fails intermittently with Process consumes memory In-Reply-To: <55793DEA.2090204@oracle.com> References: <5578098C.4030103@Oracle.com> <55783BB1.2080306@gmail.com> <55791E3D.8010905@oracle.com> <55793DEA.2090204@oracle.com> Message-ID: <55795527.2050703@Oracle.com> Hi, Yes a reasonable suggestion; I added a note to the bug for next if there is one. Looking at the failing logs the memory usage only increases on the first cycle through the loop. In the latest case, it increased by just a bit more than 1Mb and so triggered the failure. Since it seems like a one-time growth (and not a continuous one) updated test should not fail. (emoji for fingers crossed). Thanks, Roger On 6/11/15 8:51 AM, Alan Bateman wrote: > > > On 11/06/2015 06:35, David Holmes wrote: >> On 10/06/2015 11:29 PM, Peter Levart wrote: >>> Hi Roger, >>> >>> I don't know how the tests are executed in your environment, but is it >>> possible that, while you're running the test in a shared VM, some other >>> concurrent activity allocates from heap and that the test fails because >>> of that and not because of Process.exec() at all? >> >> I tend to agree - perhaps the test should be run in /othervm mode? > Sound right although just on Peter's comment, I'm not aware of any > jtreg mode that runs tests concurrently in the same VM. The agent VM > mode (that I assume most of us use nowadays) executes the tests > sequentially in each VM. There is also the possibility of tests > leaving looping threads behind or leaking resources and that can cause > issues for tests that run subsequently in the same VM. The other thing > is that it's possible to have a pool of agent VMs so many tests may be > running concurrently in different VMs and this creates another form of > potential interference. > > -Alan. From andreas.eriksson at oracle.com Thu Jun 11 12:06:56 2015 From: andreas.eriksson at oracle.com (Andreas Eriksson) Date: Thu, 11 Jun 2015 14:06:56 +0200 Subject: RFR(XS): 8081412: Remove MHIllegalAccess.java from the problem list Message-ID: <557979E0.5020109@oracle.com> Hi, Id like to remove java/lang/invoke/8022701/MHIllegalAccess.java from the problem list. It was added while the test fix propagated to the correct repos. [1] Now the fix has propagated. [2] Change, generated by running hg backout: diff --git a/test/ProblemList.txt b/test/ProblemList.txt --- a/test/ProblemList.txt +++ b/test/ProblemList.txt @@ -123,9 +123,6 @@ # 8029891 java/lang/ClassLoader/deadlock/GetResource.java generic-all -# 8080428 -java/lang/invoke/8022701/MHIllegalAccess.java generic-all - ############################################################################ # jdk_instrument Thanks, Andreas [1]: http://mail.openjdk.java.net/pipermail/core-libs-dev/2015-May/033760.html [2]: http://hg.openjdk.java.net/jdk9/dev/jdk/rev/8cad1a9fee4a From magnus.ihse.bursie at oracle.com Thu Jun 11 12:14:59 2015 From: magnus.ihse.bursie at oracle.com (Magnus Ihse Bursie) Date: Thu, 11 Jun 2015 14:14:59 +0200 Subject: RFR: JDK-8085822 JEP 223: New Version-String Scheme (initial integration) In-Reply-To: <55791425.8070709@oracle.com> References: <5571AD18.7080701@oracle.com> <557625C8.5050200@oracle.com> <5576E633.9050503@oracle.com> <5576E81B.8050703@oracle.com> <5576EF94.3010500@oracle.com> <55780A33.1010302@oracle.com> <557829DC.8000405@oracle.com> <55791425.8070709@oracle.com> Message-ID: <55797BC3.9080202@oracle.com> On 2015-06-11 06:52, David Holmes wrote: > Hi Magnus, > > On 10/06/2015 10:13 PM, Magnus Ihse Bursie wrote: >> On 2015-06-10 11:58, David Holmes wrote: >>> Hi Magnus, >>> >>> Generally looks good - a few comments/queries below. >> >> In general, I believe most issues you found are valid. :-) However, as I >> said before in this thread, I'd like to see them resolved in the needed >> follow-up patches. The source code changes in Hotspot and JDK are >> inadequate to fully implement JEP-223, so these areas will need to be >> rewritten anyhow. Are you okay with resolving these issues later? > > Given this is going to a staging repo I'm okay with deferring things - > I just have a concern whether such things may be overlooked given that > the integration with the staging repo will not be undergoing a final > review. I agree completely with your concern. I have summarized the issues that were raised but not addressed during this review, and created JBS bugs, one per component. I will do my best to make sure that fixing them does not get lost from the Verona project agenda. The three bugs are: https://bugs.openjdk.java.net/browse/JDK-8087202 https://bugs.openjdk.java.net/browse/JDK-8087203 https://bugs.openjdk.java.net/browse/JDK-8087205 Here's the core content of them. If I have missed something, please add it to the bug reports: HOTSPOT: Alan Bateman: Will the update_version and special_update_version fields eventually be dropped from the jvm_version_info structure? The webrev shows a change to this comment in jvm.h: "Third, this file contains various I/O and network operations needed by the standard Java I/O and network APIs." I think this comment can be removed because those JVM_* functions were removed some time ago. Daniel D. Daugherty: General comment: It looks like support for the 'patch' value is not completely implemented through all the Makefiles. I didn't audit for this, but it's just my impression. hotspot/src/share/vm/runtime/java.cpp L661: void JDK_Version::fully_initialize( L662: uint8_t major, uint8_t minor, uint8_t security, uint8_t update) { L663: // This is only called when current is less than 1.6 and we've gotten Since you're ripping out vestigial version support, I think this function can go away since this is version 9 and newer. Don't really know for sure, but based on that comment... hotspot/src/share/vm/runtime/vm_version.cpp L84: void Abstract_VM_Version::initialize() { L85: // FIXME: Initialization can probably be removed now. I agree, but that would entail also removing the _initialized and the uses of it... Follow on bug fix? David Holmes: make/*/makefiles/vm.make Shouldn't the -DVERSION_XX=$(VERSION_XX) definitions be taken from the VERSION_CFLAGS in spec.gmk? (Or are you still allowing for a stand-alone hotspot build?) hotspot/src/share/vm/prims/jvm.h I think this comment is way out of date: /* Build number is available only for RE build (i.e. JDK_BUILD_NUMBER is set to bNN) * It will be zero for internal builds. */ and can be completely removed. Even if still true, mention of an "RE build" has no place in OpenJDK sources. hotspot/src/share/vm/runtime/java.cpp I think a lot of the modified code is obsolete post Hotspot Express - which makes it hard to determine if the changes make sense. hotspot/test/runtime/6981737/Test6981737.java This test is really only valid for the new version scheme now, so it should probably be rewritten - and in that case it really isn't testing 6981737 but should be replaced by a new test for the new version format. JDK: Alan Bateman: Version.java.template - the comment in jvmSecurityVersion() still talks about 1.6 and newer. Can this be replaced to just say that it returns the security version? Daniel D. Daugherty: jdk/src/java.base/share/classes/sun/misc/Version.java.template L149: * Returns the security version of the running JVM if it's 1.6 or newer This JavaDoc update is wrong, but it might not be important if sun.misc.Version class is going away. David Holmes: jdk/src/java.base/share/classes/sun/misc/Version.java.template This comment is nonsensical: /** ! * Returns the security version of the running JVM if it's 1.6 or newer * or any RE VM build. It will return 0 if it's an internal 1.5 or * 1.4.x build. * @since 1.6 */ as security version does not exist pre 9. Normally you should be adding a new method and deprecating the old one. The new one is @since 9. /** ! * Returns the security version of the running JDK. * @since 1.6 */ Ditto: @since 9 (but again old should be deprecated and new method added) 253 /** 254 * Returns the build number of the running JDK if it's a RE build 255 * It will return 0 if it's an internal build. As with jvm.h this seems obsolete commentary these days - not only RE builds define a build number. LANGTOOLS: Daniel D. Daugherty: langtools/src/java.compiler/share/classes/javax/lang/model/SourceVersion.java old L171: case "1.9": new L171: case "9": Should this logic support both versions? Will dropping "1.9" here prevent a pre-version changeset JVM from being dropped into a JDK for triage purposes? Granted we don't often triage 'javac' with different JVMs, but... Jan Lahoda: +1 on keeping both "1.9" and "9" here. javac can be used independently on the rest of JDK to some extent, so support for running it on older builds of JDK 9 seems reasonable to me. (I wonder if current JDK 9 javac should be prepared for the new version scheme in advance.) /Magnus > > I would prefer to see the Version.java.template doc comments > corrected, even if the issue of whether to add and deprecate is > deferred, but again as this is to a staging area I can let it slide > for now. > > I saw the fix to hotspot/src/share/vm/runtime/vmStructs.cpp. > > Thanks, > David > >>> >>> On 9/06/2015 11:52 PM, Magnus Ihse Bursie wrote: >>>> Here's an updated webrev, which fixes the typos that were pointed >>>> out by >>>> reviewers: >>>> http://cr.openjdk.java.net/~ihse/JDK-8085822-JEP-223-initial-patch/webrev.02/ >>>> >>>> >>> >>> common/autoconf/version-numbers >>> >>> Shouldn't there be a DEFAULT_VERSION_XXX for each of the XXX parts of >>> the version info? (Even if all zeroes presently.) >> So that's a tricky one. :-& The question here is, I think, does it make >> sense to update version-numbers when we do a security (9.0.1) or minor >> (9.1.0) release? Looking historically, the version-numbers file have not >> been changed for the 8u family. The only change was going from 8 to 9 >> when creating the new jdk9 forest. That was what I based my decition to >> only have the major number in the file. (The rest of the version string >> is set by configure flags when building, in Oracle case by the RE team.) >> >> If it makes sense to put the minor/security/patch numbers here, I won't >> oppose it, but I guess we have until the first such release to figure >> out what's best, and that likely includes discussion with RE and >> possibly other consumers in the community (RedHat etc). >> >>> >>> --- >>> >>> Looking at hotspot changes I can't convince myself that the new >>> streamlined "version" variables will capture things like "fastdebug". >>> Will that filter through via configure variables? >> >> The "fastdebug" label has been handled as a less valued token in the >> JEP-223 process. Right now there's no mention of it at all in the >> version string proposal in the JEP. As I understand it, Iris is about to >> propose an amendment which will just make fastdebug be a part of the OPT >> string. I'm not entirely happy with that myself, but that's the way it's >> decided. So wherever you can see the OPT string, you'll see the debug >> level tag. >> >> Currently, however, that's not how it is implemented in this patch. >> Since no decision was made on this (and it's still not formally >> decided), I had to pick a route to go forward. The current patch will >> instead put the "fastdebug" label as part of the PRE string (that's the >> reason for the pre-base and pre-debuglevel arguments to configure). If >> the planned changes goes into the JEP, this'll change to make the >> debuglevel tag a part of the OPT string instead. >> >>> --- >>> >>> make/*/makefiles/vm.make >>> >>> Shouldn't the -DVERSION_XX=$(VERSION_XX) definitions be taken from the >>> VERSION_CFLAGS in spec.gmk? (Or are you still allowing for a >>> stand-alone hotspot build?) >> The hotspot JEP-223 work initially made by Alejandro kept support for >> stand-alone hotspot builds. I made additional changes on top of that, >> which still tried to cater for stand-alone builds. At this very moment >> I'm not sure if stand-alone hotspot builds are supposed to work or not, >> but I've tried to not change the situation with this patch. >> >> There are two future changes coming down the pipe: One is the additional >> JEP-223 work needed for Hotspot. I know Alejandro had plans for >> redesigning the API between Hotspot and the JDK, so no JDK version >> strings should be compiled into the JVM, but all of it extracted during >> an API in runtime. That would make most (all?) of the makefile changes >> in hotspot irrelevant. However, that implementation is not even started, >> so it's needed for the time being. >> >> The second change is the build-infra hotspot makefile rewrite. When that >> happens, stand-alone builds will definitely disappear, and if Hotspot >> still needs make support for version strings, then the logical choice is >> to use VERSION_CFLAGS. >> >> Ok? >> >>> >>> --- >>> >>> hotspot/src/share/vm/prims/jvm.h >>> jdk/src/java.base/share/native/include/jvm.h >>> >>> I think this comment is way out of date: >>> >>> /* Build number is available only for RE build (i.e. JDK_BUILD_NUMBER >>> is set to bNN) >>> * It will be zero for internal builds. >>> */ >>> >>> and can be completely removed. Even if still true, mention of an "RE >>> build" has no place in OpenJDK sources. >> Yep, agree. Follow-up patch, right? >> >>> >>> --- >>> >>> hotspot/src/share/vm/runtime/java.cpp >>> >>> I think a lot of the modified code is obsolete post Hotspot Express - >>> which makes it hard to determine if the changes make sense. >> Yep, agree. Follow-up patch, right? >>> >>> --- >>> >>> hotspot/src/share/vm/runtime/vmStructs.cpp >>> >>> Please fix the alignment (3 extra spaces on modified line): >>> >>> 1239 static_field(Abstract_VM_Version, _vm_minor_version, >>> int) \ >>> 1240 static_field(Abstract_VM_Version, >>> _vm_security_version, int) \ >> >> I was about to say "follow-up patch", but ugly indentation really sucks >> so I can fix this. Ok if I skip redoing the review for that? >> >>> >>> --- >>> >>> hotspot/test/runtime/6981737/Test6981737.java >>> >>> This test is really only valid for the new version scheme now, so it >>> should probably be rewritten - and in that case it really isn't >>> testing 6981737 but should be replaced by a new test for the new >>> version format. >> Yep, agree. Follow-up patch, right? >>> >>> --- >>> >>> jdk/src/java.base/share/classes/sun/misc/Version.java.template >>> >>> This comment is nonsensical: >>> >>> /** >>> ! * Returns the security version of the running JVM if it's 1.6 >>> or newer >>> * or any RE VM build. It will return 0 if it's an internal >>> 1.5 or >>> * 1.4.x build. >>> * @since 1.6 >>> */ >>> >>> as security version does not exist pre 9. Normally you should be >>> adding a new method and deprecating the old one. The new one is >>> @since 9. >> Yep, agree. Follow-up patch, right? >>> >>> /** >>> ! * Returns the security version of the running JDK. >>> * @since 1.6 >>> */ >>> >>> Ditto: @since 9 (but again old should be deprecated and new method >>> added) >>> >>> 253 /** >>> 254 * Returns the build number of the running JDK if it's a RE >>> build >>> 255 * It will return 0 if it's an internal build. >>> >>> As with jvm.h this seems obsolete commentary these days - not only RE >>> builds define a build number. >> Yep, agree. Follow-up patch, right? >> >> /Magnus >> >>> >>> Thanks, >>> David >> From volker.simonis at gmail.com Thu Jun 11 16:11:44 2015 From: volker.simonis at gmail.com (Volker Simonis) Date: Thu, 11 Jun 2015 18:11:44 +0200 Subject: RFR(M): 8081674: EmptyStackException at startup if running with extended or unsupported charset In-Reply-To: <55793BDA.8000204@oracle.com> References: <557626AA.6010008@oracle.com> <55772A56.2030308@oracle.com> <7B3C08CF-028E-4AD2-B7F8-C803F2BA3C7D@oracle.com> <55793BDA.8000204@oracle.com> Message-ID: On Thu, Jun 11, 2015 at 9:42 AM, Alan Bateman wrote: > > On 10/06/2015 20:02, Mandy Chung wrote: >> >> : >> ClassLoader.java >> It should have done that. Nit: >> Can you move line 1693 to 1868 closer to loadLibrary0 (the caller of >> findBuiltinLib)? >> Also make it private. >> >> Same applies to jdk9. >> >> Otherwise looks good. No need to generate a webrev as long as you make >> the change before the push. > > Looks good to me too (assuming these changes). > Thanks! >> : >> >>> On jdk9 we are now running in another problem which is caused by the >>> fact that the extended charsets are now being loaded by ServiceLoader >>> (I've detailed that already in the bug report). But I think we can >>> leave the fix of that problem to another change as this seems to >>> require some more reasoning (see Alan's comments in the bug report). >> >> Yes this is a harder problem that have to investigate further. >> > Right, the main issue here is that java.base must contain all charsets that > are needed during start up / VM initialization. There is also no guarantee > the jdk.charsets module will be linked into the run-time image. > > There are two scenarios that come from this. One is the LANG setting as we > have here, the other is attempting to run with file.encoding set. The choice > as to whether to fail gracefully or switch to UTF-8 might be different for > these two scenarios, it just investigation and a proposal. > I'd vote for falling back to either UTF-8 or ISO-8859-1 and issuing a warning. Failing gracefully maybe difficult as we're in the middle of the initialization so throwing an exception is probably the best we can do (which I wouldn't consider "gracefully"). Regards, Volker > -Alan From brian.burkhalter at oracle.com Fri Jun 12 00:30:42 2015 From: brian.burkhalter at oracle.com (Brian Burkhalter) Date: Thu, 11 Jun 2015 17:30:42 -0700 Subject: [9] RFR of 8087288: File.get{Free, Total, Usable}Space may return unexpected results with >2TB file systems Message-ID: Please review at your convenience: Issue: https://bugs.openjdk.java.net/browse/JDK-8087288 Patch: http://cr.openjdk.java.net/~bpb/8087288/webrev.00/ The root cause of this problem is essentially the same as that of https://bugs.openjdk.java.net/browse/JDK-8081843 and the proposed fix is likewise similar: on Mac OS X replace statvfs64() and struct statvfs64 with statfs() and struct statfs, respectively, and f_frsize with f_bsize. Thanks, Brian From frank.yuan at oracle.com Fri Jun 12 07:27:15 2015 From: frank.yuan at oracle.com (Frank Yuan) Date: Fri, 12 Jun 2015 15:27:15 +0800 Subject: Review request for JDK-8080907 - Develop test for Xerces Update: XML Schema Validation Message-ID: <00fd01d0a4e1$36bd31f0$a43795d0$@oracle.com> Hi Joe, Lance and all Would you like to have a review for bug: https://bugs.openjdk.java.net/browse/JDK-8080907? I added some new tests and updated some existing tests to add more coverage for JAXP library. The webrev is at http://cr.openjdk.java.net/~fyuan/8080907/webrev.00/, your comment will be appreciated. Best Regards Frank From joe.darcy at oracle.com Fri Jun 12 16:12:43 2015 From: joe.darcy at oracle.com (joe darcy) Date: Fri, 12 Jun 2015 09:12:43 -0700 Subject: RFR(XS): 8081412: Remove MHIllegalAccess.java from the problem list In-Reply-To: <557979E0.5020109@oracle.com> References: <557979E0.5020109@oracle.com> Message-ID: <557B04FB.4090605@oracle.com> Looks fine; thanks, -Joe On 6/11/2015 5:06 AM, Andreas Eriksson wrote: > Hi, > > Id like to remove java/lang/invoke/8022701/MHIllegalAccess.java from > the problem list. > It was added while the test fix propagated to the correct repos. [1] > Now the fix has propagated. [2] > > Change, generated by running hg backout: > > diff --git a/test/ProblemList.txt b/test/ProblemList.txt > --- a/test/ProblemList.txt > +++ b/test/ProblemList.txt > @@ -123,9 +123,6 @@ > # 8029891 > java/lang/ClassLoader/deadlock/GetResource.java generic-all > > -# 8080428 > -java/lang/invoke/8022701/MHIllegalAccess.java generic-all > - > ############################################################################ > > > # jdk_instrument > > Thanks, > Andreas > > [1]: > http://mail.openjdk.java.net/pipermail/core-libs-dev/2015-May/033760.html > [2]: http://hg.openjdk.java.net/jdk9/dev/jdk/rev/8cad1a9fee4a From huizhe.wang at oracle.com Fri Jun 12 16:58:47 2015 From: huizhe.wang at oracle.com (huizhe wang) Date: Fri, 12 Jun 2015 09:58:47 -0700 Subject: Review request for JDK-8080907 - Develop test for Xerces Update: XML Schema Validation In-Reply-To: <00fd01d0a4e1$36bd31f0$a43795d0$@oracle.com> References: <00fd01d0a4e1$36bd31f0$a43795d0$@oracle.com> Message-ID: <557B0FC7.8070400@oracle.com> Hi Frank, The change looks good to me. Thanks, Joe On 6/12/2015 12:27 AM, Frank Yuan wrote: > > Hi Joe, Lance and all > > Would you like to have a review for bug: > https://bugs.openjdk.java.net/browse/JDK-8080907? > > I added some new tests and updated some existing tests to add more > coverage for JAXP library. > > The webrev is at http://cr.openjdk.java.net/~fyuan/8080907/webrev.00/ > , your comment > will be appreciated. > > Best Regards > > Frank > From sean.mullan at oracle.com Fri Jun 12 17:29:07 2015 From: sean.mullan at oracle.com (Sean Mullan) Date: Fri, 12 Jun 2015 13:29:07 -0400 Subject: RFR: 8087283 : Add support for the XML Signature here() function to the JDK XPath implementation Message-ID: <557B16E3.1010109@oracle.com> Please review this change to add support for the XML Signature here() function to the JDK XPath implementation. The here function is not defined in its own namespace. Therefore it needs to be implemented by extending the function library of XPath in the same namespace. Currently, the JDK XML Signature implementation uses reflection to invoke a method of an implementation-specific class in the JDK XPath library to register the here function. This fix provides a much cleaner solution that avoids use of reflection by extending the built-in set of XPath functions to include the here function. webrev: http://cr.openjdk.java.net/~mullan/webrevs/8087283/webrev.00/ bug: https://bugs.openjdk.java.net/browse/JDK-8087283 Thanks, Sean From huizhe.wang at oracle.com Fri Jun 12 19:23:34 2015 From: huizhe.wang at oracle.com (huizhe wang) Date: Fri, 12 Jun 2015 12:23:34 -0700 Subject: RFR: 8087283 : Add support for the XML Signature here() function to the JDK XPath implementation In-Reply-To: <557B16E3.1010109@oracle.com> References: <557B16E3.1010109@oracle.com> Message-ID: <557B31B6.5000100@oracle.com> Hi Sean, The change looks good to me. Thanks, Joe On 6/12/2015 10:29 AM, Sean Mullan wrote: > Please review this change to add support for the XML Signature here() > function to the JDK XPath implementation. The here function is not > defined in its own namespace. Therefore it needs to be implemented by > extending the function library of XPath in the same namespace. > > Currently, the JDK XML Signature implementation uses reflection to > invoke a method of an implementation-specific class in the JDK XPath > library to register the here function. This fix provides a much > cleaner solution that avoids use of reflection by extending the > built-in set of XPath functions to include the here function. > > webrev: http://cr.openjdk.java.net/~mullan/webrevs/8087283/webrev.00/ > bug: https://bugs.openjdk.java.net/browse/JDK-8087283 > > Thanks, > Sean From Alan.Bateman at oracle.com Fri Jun 12 19:28:14 2015 From: Alan.Bateman at oracle.com (Alan Bateman) Date: Fri, 12 Jun 2015 20:28:14 +0100 Subject: RFR: 8087283 : Add support for the XML Signature here() function to the JDK XPath implementation In-Reply-To: <557B16E3.1010109@oracle.com> References: <557B16E3.1010109@oracle.com> Message-ID: <557B32CE.6000400@oracle.com> On 12/06/2015 18:29, Sean Mullan wrote: > Please review this change to add support for the XML Signature here() > function to the JDK XPath implementation. The here function is not > defined in its own namespace. Therefore it needs to be implemented by > extending the function library of XPath in the same namespace. > > Currently, the JDK XML Signature implementation uses reflection to > invoke a method of an implementation-specific class in the JDK XPath > library to register the here function. This fix provides a much > cleaner solution that avoids use of reflection by extending the > built-in set of XPath functions to include the here function. > > webrev: http://cr.openjdk.java.net/~mullan/webrevs/8087283/webrev.00/ > bug: https://bugs.openjdk.java.net/browse/JDK-8087283 The changes look okay to me too. -Alan From Alan.Bateman at oracle.com Fri Jun 12 19:29:24 2015 From: Alan.Bateman at oracle.com (Alan Bateman) Date: Fri, 12 Jun 2015 20:29:24 +0100 Subject: [9] RFR of 8087288: File.get{Free, Total, Usable}Space may return unexpected results with >2TB file systems In-Reply-To: References: Message-ID: <557B3314.6080507@oracle.com> On 12/06/2015 01:30, Brian Burkhalter wrote: > Please review at your convenience: > > Issue: https://bugs.openjdk.java.net/browse/JDK-8087288 > Patch: http://cr.openjdk.java.net/~bpb/8087288/webrev.00/ > > The root cause of this problem is essentially the same as that of https://bugs.openjdk.java.net/browse/JDK-8081843 and the proposed fix is likewise similar: on Mac OS X replace statvfs64() and struct statvfs64 with statfs() and struct statfs, respectively, and f_frsize with f_bsize. > This looks okay to me. Maybe someday we can come these methods to use the new file system API and remove these native methods. -Alan From lance.andersen at oracle.com Fri Jun 12 21:01:39 2015 From: lance.andersen at oracle.com (Lance Andersen) Date: Fri, 12 Jun 2015 17:01:39 -0400 Subject: Review request for JDK-8080907 - Develop test for Xerces Update: XML Schema Validation In-Reply-To: <557B0FC7.8070400@oracle.com> References: <00fd01d0a4e1$36bd31f0$a43795d0$@oracle.com> <557B0FC7.8070400@oracle.com> Message-ID: I am fine with this also Frank Best Lance On Jun 12, 2015, at 12:58 PM, huizhe wang wrote: > Hi Frank, > > The change looks good to me. > > Thanks, > Joe > > On 6/12/2015 12:27 AM, Frank Yuan wrote: >> Hi Joe, Lance and all >> >> Would you like to have a review for bug: https://bugs.openjdk.java.net/browse/JDK-8080907? >> >> I added some new tests and updated some existing tests to add more coverage for JAXP library. >> >> The webrev is at http://cr.openjdk.java.net/~fyuan/8080907/webrev.00/, your comment will be appreciated. >> >> >> Best Regards >> Frank >> > Lance Andersen| Principal Member of Technical Staff | +1.781.442.2037 Oracle Java Engineering 1 Network Drive Burlington, MA 01803 Lance.Andersen at oracle.com From blackdrag at gmx.org Sun Jun 14 17:20:13 2015 From: blackdrag at gmx.org (Jochen Theodorou) Date: Sun, 14 Jun 2015 19:20:13 +0200 Subject: JDK-8057919 Class.getSimpleName() should work for non-JLS compliant class names Message-ID: <557DB7CD.7090902@gmx.org> Hi all, this mail is refering to http://mail.openjdk.java.net/pipermail/core-libs-dev/2015-April/032733.html and JDK-8057919 In a groovy program def cl = { -> '' } assert cl.class.anonymousClass == false the open block {->''} is compiled into an inner class. We had problems with this in the past, because for several reasons we did not follow the JLS naming conventions here. The change in JDK-8057919 was supposed to make things better, but the change also means that our open block class will now have an empty simple name. At the same time reflection tells me the class is not an anonymous class. If I see this right, then regardless of if we set the attributes right or not, reflection and getSimpleName have a differing understanding about the nature of that class. I consider that a bug. They should have a common understanding about what a anonymous inner class is, since getSimpleName is supposed to return an empty string for that case. plus... it would be really really nice if someone could tell me how a non-anonymous class in a method has to look like with javap, so that I can know from there if it is no anonymous and finally fix he flags for Groovy's class generation. Because javap and the asm lib really don't help much with getting inner classes and their flags right. bye jochen -- Jochen "blackdrag" Theodorou blog: http://blackdragsview.blogspot.com/ From tomasz.kowalczewski at gmail.com Sun Jun 14 22:32:23 2015 From: tomasz.kowalczewski at gmail.com (Tomasz Kowalczewski) Date: Mon, 15 Jun 2015 00:32:23 +0200 Subject: TimeZone issue in 1.8u60 In-Reply-To: <5578AFFF.1010500@oracle.com> References: <5578AFFF.1010500@oracle.com> Message-ID: Thank you for this explanation. Do I understand correctly that first five elements of this seven element array are the same as five element array in Java versions before 1.8u60? On Wed, Jun 10, 2015 at 11:45 PM, Naoto Sato wrote: > Hi Tomasz, > > The change was made to fix a performance regression in JDK8: > > https://bugs.openjdk.java.net/browse/JDK-8076287 > > Those time zone names weren't cached in JDK8, so the fix was to cache > those arrays, which are also shared with ZoneId.getDisplayName() which can > also return generic names. > > HTH, > Naoto > > > On 6/10/15 4:21 AM, Tomasz Kowalczewski wrote: > >> Hi, >> >> I am not sure where to write about this, I hope somebody will point me to >> right list if this one is not correct. >> >> I have been playing with newest Java 1.8u60 to try PreserveFramePointer >> functionality. Unfortunately none of our servers start on this version of >> java. It is because of REST call to Amazon services done during startup. >> None of these calls worked. Unless I am missing something it turns out to >> be issue with formatting time zone information as done by Joda Time. It >> uses calls to: >> >> DateTimeUtils.getDateFormatSymbols(Locale.ENGLISH).getZoneStrings(); >> >> to get list of timezones. This usually returned array of arrays of 5 >> elements. In 1.8u60 it returns array of arrays of 7 elements. >> >> I know that all this software is not related to OpenJDK and calling >> getZoneStrings is discouraged in the docs. But as I am unfamiliar with >> time >> zones mechanisms inside JDK (loading from bundles etc.) I was hoping that >> somebody will point me to change that may caused this for sake of better >> understanding the issue. >> >> -- Tomasz Kowalczewski From david.holmes at oracle.com Mon Jun 15 02:25:41 2015 From: david.holmes at oracle.com (David Holmes) Date: Mon, 15 Jun 2015 12:25:41 +1000 Subject: JDK-8057919 Class.getSimpleName() should work for non-JLS compliant class names In-Reply-To: <557DB7CD.7090902@gmx.org> References: <557DB7CD.7090902@gmx.org> Message-ID: <557E37A5.4000700@oracle.com> Hi Jochen, On 15/06/2015 3:20 AM, Jochen Theodorou wrote: > Hi all, > > this mail is refering to > http://mail.openjdk.java.net/pipermail/core-libs-dev/2015-April/032733.html > and JDK-8057919 > > In a groovy program > > def cl = { -> '' } > assert cl.class.anonymousClass == false > > the open block {->''} is compiled into an inner class. We had problems > with this in the past, because for several reasons we did not follow the > JLS naming conventions here. The change in JDK-8057919 was supposed to > make things better, but the change also means that our open block class > will now have an empty simple name. At the same time reflection tells me > the class is not an anonymous class. > > If I see this right, then regardless of if we set the attributes right > or not, reflection and getSimpleName have a differing understanding > about the nature of that class. I consider that a bug. They should have > a common understanding about what a anonymous inner class is, since > getSimpleName is supposed to return an empty string for that case. Which reflection method are you referring to? Class.isAnonymousClass is defined in terms of getSimpleName(): public boolean isAnonymousClass() { return "".equals(getSimpleName()); } David > plus... it would be really really nice if someone could tell me how a > non-anonymous class in a method has to look like with javap, so that I > can know from there if it is no anonymous and finally fix he flags for > Groovy's class generation. > > Because javap and the asm lib really don't help much with getting inner > classes and their flags right. > > bye jochen > From daniel.fuchs at oracle.com Mon Jun 15 08:26:41 2015 From: daniel.fuchs at oracle.com (Daniel Fuchs) Date: Mon, 15 Jun 2015 10:26:41 +0200 Subject: RfR - 8080933: LogManager.demandSystemLogger should accept a 'caller' argument. Message-ID: <557E8C41.7090904@oracle.com> Hi, Please find below a fix for https://bugs.openjdk.java.net/browse/JDK-8080933 8080933: LogManager.demandSystemLogger should accept a 'caller' argument. http://cr.openjdk.java.net/~dfuchs/webrev_8080933/webrev.00 LogManager.demandLogger accepts a caller argument which is later used to resolve ResourceBundle. However LogManager.demandSystemLogger doesn't have that caller argument - which could later lead to some inconsistency when resource bundles are resolved. As for now this is more an inconsistency than an issue since by definition the caller that creates a system logger is a system class in the bootstrap class loader. However it could become an issue in a modular JDK - where different system classes may come from different modules and where information on the actual system caller class may be needed. best regards, -- daniel From paul.sandoz at oracle.com Mon Jun 15 09:47:21 2015 From: paul.sandoz at oracle.com (Paul Sandoz) Date: Mon, 15 Jun 2015 11:47:21 +0200 Subject: RFR 8085978 LinkedTransferQueue.spliterator can report LTQ.Node object, not T Message-ID: Hi, Please review: http://cr.openjdk.java.net/~psandoz/jdk9/JDK-8085978-linked-transfer-queue-traverse/webrev/ The LinkedTransferQueue's spliterator can erroneously report internal nodes when traversing. The fix is cherry picked from the 166 repo (other mostly refactoring-based changes will occur later on with bulk updates). I would also like to backport to 8u60. Paul. From naoto.sato at oracle.com Mon Jun 15 15:15:06 2015 From: naoto.sato at oracle.com (Naoto Sato) Date: Mon, 15 Jun 2015 08:15:06 -0700 Subject: TimeZone issue in 1.8u60 In-Reply-To: References: <5578AFFF.1010500@oracle.com> Message-ID: <557EEBFA.2030002@oracle.com> Yes. They are specified in the javadoc for DateFormatSymbols.getZoneStrings(). http://docs.oracle.com/javase/8/docs/api/java/text/DateFormatSymbols.html#getZoneStrings-- Naoto On 6/14/15 3:32 PM, Tomasz Kowalczewski wrote: > Thank you for this explanation. Do I understand correctly that first > five elements of this seven element array are the same as five element > array in Java versions before 1.8u60? > > On Wed, Jun 10, 2015 at 11:45 PM, Naoto Sato > wrote: > > Hi Tomasz, > > The change was made to fix a performance regression in JDK8: > > https://bugs.openjdk.java.net/browse/JDK-8076287 > > Those time zone names weren't cached in JDK8, so the fix was to > cache those arrays, which are also shared with > ZoneId.getDisplayName() which can also return generic names. > > HTH, > Naoto > > > On 6/10/15 4:21 AM, Tomasz Kowalczewski wrote: > > Hi, > > I am not sure where to write about this, I hope somebody will > point me to > right list if this one is not correct. > > I have been playing with newest Java 1.8u60 to try > PreserveFramePointer > functionality. Unfortunately none of our servers start on this > version of > java. It is because of REST call to Amazon services done during > startup. > None of these calls worked. Unless I am missing something it > turns out to > be issue with formatting time zone information as done by Joda > Time. It > uses calls to: > > DateTimeUtils.getDateFormatSymbols(Locale.ENGLISH).getZoneStrings(); > > to get list of timezones. This usually returned array of arrays of 5 > elements. In 1.8u60 it returns array of arrays of 7 elements. > > I know that all this software is not related to OpenJDK and calling > getZoneStrings is discouraged in the docs. But as I am > unfamiliar with time > zones mechanisms inside JDK (loading from bundles etc.) I was > hoping that > somebody will point me to change that may caused this for sake > of better > understanding the issue. > > > > > -- > Tomasz Kowalczewski From vladimir.x.ivanov at oracle.com Mon Jun 15 16:04:30 2015 From: vladimir.x.ivanov at oracle.com (Vladimir Ivanov) Date: Mon, 15 Jun 2015 19:04:30 +0300 Subject: JDK-8057919 Class.getSimpleName() should work for non-JLS compliant class names In-Reply-To: <557DB7CD.7090902@gmx.org> References: <557DB7CD.7090902@gmx.org> Message-ID: <557EF78E.90202@oracle.com> Jochen, > plus... it would be really really nice if someone could tell me how a > non-anonymous class in a method has to look like with javap, so that I > can know from there if it is no anonymous and finally fix he flags for > Groovy's class generation. Every entry in InnerClass attribute contains 4 fields: (1) inner_class_info_index; (2) outer_class_info_index; (3) inner_name_index; (4) inner_class_access_flags. The following InnerClass attribute is generated for your test case: InnerClasses: #2; // class test$_run_closure1 It corresponds to the following entry in InnerClass attribute: inner_class_info = #2 // class test$_run_closure1 outer_class_info_index = #0 inner_name_index = #0 inner_class_access_flags = 0 According to JVMS [1], it corresponds to an anonymous class: " 4.7.6. The InnerClasses Attribute ... outer_class_info_index If C is not a member of a class or an interface (that is, if C is ... an anonymous class (JLS ?15.9.5)), the value of the outer_class_info_index item must be zero. ... inner_name_index If C is anonymous (JLS ?15.9.5), the value of the inner_name_index item must be zero. ... " In order to make the class non-anonymous again, you have to specify inner_name_index and, optionally, outer_class_info_index. Best regards, Vladimir Ivanov [1] https://docs.oracle.com/javase/specs/jvms/se8/html/jvms-4.html#jvms-4.7.6 From martinrb at google.com Mon Jun 15 16:18:35 2015 From: martinrb at google.com (Martin Buchholz) Date: Mon, 15 Jun 2015 09:18:35 -0700 Subject: RFR 8085978 LinkedTransferQueue.spliterator can report LTQ.Node object, not T In-Reply-To: References: Message-ID: This is a fine fix for a jdk8 backport. In pursuit of even more perfection for jdk9, we might do some more fiddling later. It bothers me that not all the traversal loops in LTQ work the same. Other traversal loops simply skip over matched request nodes, but the spliterator traversals do not (although the difference may be undetectable). On Mon, Jun 15, 2015 at 2:47 AM, Paul Sandoz wrote: > Hi, > > Please review: > > > http://cr.openjdk.java.net/~psandoz/jdk9/JDK-8085978-linked-transfer-queue-traverse/webrev/ > > The LinkedTransferQueue's spliterator can erroneously report internal > nodes when traversing. > > The fix is cherry picked from the 166 repo (other mostly refactoring-based > changes will occur later on with bulk updates). I would also like to > backport to 8u60. > > Paul. > From mandy.chung at oracle.com Mon Jun 15 16:32:52 2015 From: mandy.chung at oracle.com (Mandy Chung) Date: Mon, 15 Jun 2015 09:32:52 -0700 Subject: RfR - 8080933: LogManager.demandSystemLogger should accept a 'caller' argument. In-Reply-To: <557E8C41.7090904@oracle.com> References: <557E8C41.7090904@oracle.com> Message-ID: <557EFE34.6080205@oracle.com> On 06/15/2015 01:26 AM, Daniel Fuchs wrote: > Hi, > > Please find below a fix for > > https://bugs.openjdk.java.net/browse/JDK-8080933 > 8080933: LogManager.demandSystemLogger should accept a > 'caller' argument. > > http://cr.openjdk.java.net/~dfuchs/webrev_8080933/webrev.00 Looks good to me. Thanks Mandy From martinrb at google.com Mon Jun 15 16:34:47 2015 From: martinrb at google.com (Martin Buchholz) Date: Mon, 15 Jun 2015 09:34:47 -0700 Subject: Spliterators of iterators Message-ID: Staring at LinkedTransferQueue.java, I see: return Spliterators.spliterator (a, 0, i, (Spliterator.ORDERED | Spliterator.NONNULL | Spliterator.CONCURRENT)); but ... that's a spliterator over an immutable array, so shouldn't we have IMMUTABLE instead of CONCURRENT? Looking further, I see ArraySpliterator requires its input array to be unmodified during operation * @param array the array, assumed to be unmodified during use ("assumed" is weird in javadoc. maybe "required"?) but ArraySpliterator does not include IMMUTABLE in characteristics, even though Arrays.spliterator does. I suggest fixing all these array spliterator characteristics to always report IMMUTABLE and never report CONCURRENT, and to document this fact, as we already do with SIZED and SUBSIZED. From paul.sandoz at oracle.com Mon Jun 15 17:16:50 2015 From: paul.sandoz at oracle.com (Paul Sandoz) Date: Mon, 15 Jun 2015 19:16:50 +0200 Subject: Spliterators of iterators In-Reply-To: References: Message-ID: <10639AC3-18AD-49DA-801E-96D8AF21081B@oracle.com> On Jun 15, 2015, at 6:34 PM, Martin Buchholz wrote: > Staring at LinkedTransferQueue.java, I see: > > return Spliterators.spliterator > (a, 0, i, (Spliterator.ORDERED | > Spliterator.NONNULL | > Spliterator.CONCURRENT)); > > but ... that's a spliterator over an immutable array, so shouldn't we have IMMUTABLE instead of CONCURRENT? > The intent here is that the sub-spliterators "inherit" certain characteristics of their top-level spliterator or source, rather than from that which might be derived from an internal data structure used to hold a partitioned set of elements. Note that the specification of Spliterator.CONCURRENT allows for sub-spliterators to report SIZED/SUBSIZED. Paul. > Looking further, I see ArraySpliterator requires its input array to be unmodified during operation > > * @param array the array, assumed to be unmodified during use > > ("assumed" is weird in javadoc. maybe "required"?) > > but ArraySpliterator does not include IMMUTABLE in characteristics, even though Arrays.spliterator does. > > I suggest fixing all these array spliterator characteristics to always report IMMUTABLE and never report CONCURRENT, and to document this fact, as we already do with SIZED and SUBSIZED. From martinrb at google.com Mon Jun 15 17:31:16 2015 From: martinrb at google.com (Martin Buchholz) Date: Mon, 15 Jun 2015 10:31:16 -0700 Subject: Spliterators of iterators In-Reply-To: <10639AC3-18AD-49DA-801E-96D8AF21081B@oracle.com> References: <10639AC3-18AD-49DA-801E-96D8AF21081B@oracle.com> Message-ID: On Mon, Jun 15, 2015 at 10:16 AM, Paul Sandoz wrote: > On Jun 15, 2015, at 6:34 PM, Martin Buchholz wrote: > > Staring at LinkedTransferQueue.java, I see: > > > > return Spliterators.spliterator > > (a, 0, i, (Spliterator.ORDERED | > > Spliterator.NONNULL | > > Spliterator.CONCURRENT)); > > > > but ... that's a spliterator over an immutable array, so shouldn't we > have IMMUTABLE instead of CONCURRENT? > > > > The intent here is that the sub-spliterators "inherit" certain > characteristics of their top-level spliterator or source, It makes sense for some characteristics to be inherited, like NONNULL. > rather than from that which might be derived from an internal data > structure used to hold a partitioned set of elements. > > Note that the specification of Spliterator.CONCURRENT allows for > sub-spliterators to report SIZED/SUBSIZED. > How is SIZED/SUBSIZED different from IMMUTABLE ? They both describe internal data structures, so yeah ... there is some kind of leak of implementation details, but these characteristics are there to be used by clients, so shouldn't we try to make them accurate? Alternatively: currently some spliterator characteristics describe "this" spliterator, and some describe the "parent", which is very confusing. From joe.darcy at oracle.com Mon Jun 15 18:01:29 2015 From: joe.darcy at oracle.com (joe darcy) Date: Mon, 15 Jun 2015 11:01:29 -0700 Subject: JDK 9 RFR of JDK-8098582: Remove redundant package.html file in javax.xml.ws/wsaddressing Message-ID: <557F12F9.9000605@oracle.com> Hello, Not sure the best place to send this out for review / send it upstream, in any case, pressing forward please review this change to fix JDK-8098582: Remove redundant package.html file in javax.xml.ws.wsaddressing http://cr.openjdk.java.net/~darcy/8098582.0/ The package javax.xml.ws.wsaddressing has both a package-info.java file and a package.html file. While belts and suspenders are sometimes helpful, this is not one of those times. Having both files causing a javadoc warning; the fix is to copy the documentation from package.html into package-info.java and delete package.html. Thanks, -Joe From Roger.Riggs at Oracle.com Mon Jun 15 20:37:08 2015 From: Roger.Riggs at Oracle.com (Roger Riggs) Date: Mon, 15 Jun 2015 16:37:08 -0400 Subject: RFR 9: 8086208 : java/lang/ProcessHandle/OnExitTest.java: IllegalThreadStateException: process hasn't exited Message-ID: <557F3774.9050904@Oracle.com> Please review a fix for a (Unix) race condition for the exit status of Process.onExit. And some source cleanup of OnExitTest. Webrev: http://cr.openjdk.java.net/~rriggs/webrev-race-8086208/ Issue: https://bugs.openjdk.java.net/browse/JDK-8086208 Roger From martinrb at google.com Mon Jun 15 20:46:34 2015 From: martinrb at google.com (Martin Buchholz) Date: Mon, 15 Jun 2015 13:46:34 -0700 Subject: RFR 9: 8086208 : java/lang/ProcessHandle/OnExitTest.java: IllegalThreadStateException: process hasn't exited In-Reply-To: <557F3774.9050904@Oracle.com> References: <557F3774.9050904@Oracle.com> Message-ID: if you do get interrupted, the interrupt is swallowed, which seems wrong. Other waiting methods have "uninterruptible" variants, that restore the interrupt status, like Semaphore.acquireUninterruptibly. Should there be a Process.waitForUninterruptibly? On Mon, Jun 15, 2015 at 1:37 PM, Roger Riggs wrote: > Please review a fix for a (Unix) race condition for the exit status of > Process.onExit. > And some source cleanup of OnExitTest. > > Webrev: > http://cr.openjdk.java.net/~rriggs/webrev-race-8086208/ > > Issue: > https://bugs.openjdk.java.net/browse/JDK-8086208 > > Roger > > From Roger.Riggs at Oracle.com Mon Jun 15 21:21:24 2015 From: Roger.Riggs at Oracle.com (Roger Riggs) Date: Mon, 15 Jun 2015 17:21:24 -0400 Subject: RFR 9: 8086208 : java/lang/ProcessHandle/OnExitTest.java: IllegalThreadStateException: process hasn't exited In-Reply-To: References: <557F3774.9050904@Oracle.com> Message-ID: <557F41D3.1000902@Oracle.com> Hi Martin, Since the function is called from the FJ pool; what would result from propagating the interrupt? The FJPool reports the interrupted to the task but otherwise just clears it. It does not affect the completion status. Since it is known that the process has terminated and is only waiting for the Thread in ProcessImpl to post the status; it cannot/should not proceed until the exitValue is valid. The webrev has been updated to propagate the interrupted status but it will likely be ignored/discarded. Roger On 6/15/2015 4:46 PM, Martin Buchholz wrote: > if you do get interrupted, the interrupt is swallowed, which seems wrong. > > Other waiting methods have "uninterruptible" variants, that restore > the interrupt status, like Semaphore.acquireUninterruptibly. Should > there be a Process.waitForUninterruptibly? > > On Mon, Jun 15, 2015 at 1:37 PM, Roger Riggs > wrote: > > Please review a fix for a (Unix) race condition for the exit > status of Process.onExit. > And some source cleanup of OnExitTest. > > Webrev: > http://cr.openjdk.java.net/~rriggs/webrev-race-8086208/ > > > Issue: > https://bugs.openjdk.java.net/browse/JDK-8086208 > > Roger > > From Roger.Riggs at Oracle.com Mon Jun 15 21:58:22 2015 From: Roger.Riggs at Oracle.com (Roger Riggs) Date: Mon, 15 Jun 2015 17:58:22 -0400 Subject: RFR 9/8: 8066504: GetVersionEx in java.base/windows/native/libjava/java_props_md.c might not get correct Windows version Message-ID: <557F4A7E.1080203@Oracle.com> Please review code for Windows 10 that sets the System properties for os.name and os.version from the version of kernel32.dll. The update uses the same technique used by Hotspot in src/os/windows/vm/os_windows.cpp. The Windows link of CoreLibraries.gmk includes version.lib/dll. The behavior of GetVersionEx has changed so it reports whatever version the application asserts it is compatible with instead of the current version of Windows. See: https://msdn.microsoft.com/en-us/library/windows/desktop/ms724451%28v=vs.85%29.aspx The same change is desirable for 8u60. Webrev: http://cr.openjdk.java.net/~rriggs/webrev-win-ver-8066504/ Issue: https://bugs.openjdk.java.net/browse/JDK-8066504 Thanks, Roger From martinrb at google.com Mon Jun 15 22:19:40 2015 From: martinrb at google.com (Martin Buchholz) Date: Mon, 15 Jun 2015 15:19:40 -0700 Subject: RFR 9: 8086208 : java/lang/ProcessHandle/OnExitTest.java: IllegalThreadStateException: process hasn't exited In-Reply-To: <557F41D3.1000902@Oracle.com> References: <557F3774.9050904@Oracle.com> <557F41D3.1000902@Oracle.com> Message-ID: OK, it wasn't clear to me this code was only running in a forkjoin pool. Hmmm ... is there guidance on how tasks in a forkjoin pool should handle InterruptedException? It should be very rare for forkjoin threads to be externally interrupted (task cancellation does not interrupt), so rare that maybe it should be interpreted as an emergency request and failing with InterruptedException is best? I've obviously lost touch with what this modern Process code is doing ... but: It looks fishy that you have a future that takes an exitStatus but calls waitFor in its body. If the exitStatus is already available, then the process must (should?) have already completed? It looks fishy to call handle (which gets any exception thrown) but then the exception is discarded. shouldReap looks weird to me - I'd expect to always have exactly one grim reaper thread per Process that always does the reaping. But yeah, I already had my chance to review this code ... maybe it's time for me to retire from java.lang.Process. On Mon, Jun 15, 2015 at 2:21 PM, Roger Riggs wrote: > Hi Martin, > > Since the function is called from the FJ pool; what would result from > propagating the interrupt? > The FJPool reports the interrupted to the task but otherwise just clears > it. > It does not affect the completion status. > > Since it is known that the process has terminated and is only waiting for > the > Thread in ProcessImpl to post the status; it cannot/should not proceed > until the > exitValue is valid. > > The webrev has been updated to propagate the interrupted status but it > will > likely be ignored/discarded. > > Roger > > > > > On 6/15/2015 4:46 PM, Martin Buchholz wrote: > > if you do get interrupted, the interrupt is swallowed, which seems wrong. > > Other waiting methods have "uninterruptible" variants, that restore the > interrupt status, like Semaphore.acquireUninterruptibly. Should there be a > Process.waitForUninterruptibly? > > On Mon, Jun 15, 2015 at 1:37 PM, Roger Riggs > wrote: > >> Please review a fix for a (Unix) race condition for the exit status of >> Process.onExit. >> And some source cleanup of OnExitTest. >> >> Webrev: >> http://cr.openjdk.java.net/~rriggs/webrev-race-8086208/ >> >> Issue: >> https://bugs.openjdk.java.net/browse/JDK-8086208 >> >> Roger >> >> > > From joe.darcy at oracle.com Tue Jun 16 00:12:26 2015 From: joe.darcy at oracle.com (joe darcy) Date: Mon, 15 Jun 2015 17:12:26 -0700 Subject: JDK 9 RFR of 8098790: Improve cross references and wording in java.lang.reflect.AnnotatedFoo type Message-ID: <557F69EA.5010209@oracle.com> Hello, Please review the small doc clarification for 8098790: Improve cross references and wording in java.lang.reflect.AnnotatedFoo type http://cr.openjdk.java.net/~darcy/8098790.0/ Patch below. In brief, various methods in java.lang.reflect.AnnotatedFoo would be clearer if they had @see links to the analogous method in java.lang.reflect.Foo. In addition some "Note that..." phrases are improperly used since they are normative text rather than just informative text. Thanks, -Joe --- old/src/java.base/share/classes/java/lang/reflect/AnnotatedArrayType.java 2015-06-15 17:06:47.895027600 -0700 +++ new/src/java.base/share/classes/java/lang/reflect/AnnotatedArrayType.java 2015-06-15 17:06:47.763027597 -0700 @@ -39,6 +39,7 @@ * Returns the potentially annotated generic component type of this array type. * * @return the potentially annotated generic component type of this array type + * @see GenericArrayType#getGenericComponentType() */ AnnotatedType getAnnotatedGenericComponentType(); } --- old/src/java.base/share/classes/java/lang/reflect/AnnotatedParameterizedType.java 2015-06-15 17:06:48.307027613 -0700 +++ new/src/java.base/share/classes/java/lang/reflect/AnnotatedParameterizedType.java 2015-06-15 17:06:48.163027609 -0700 @@ -38,6 +38,7 @@ * Returns the potentially annotated actual type arguments of this parameterized type. * * @return the potentially annotated actual type arguments of this parameterized type + * @see ParameterizedType#getActualTypeArguments() */ AnnotatedType[] getAnnotatedActualTypeArguments(); } --- old/src/java.base/share/classes/java/lang/reflect/AnnotatedTypeVariable.java 2015-06-15 17:06:48.723027625 -0700 +++ new/src/java.base/share/classes/java/lang/reflect/AnnotatedTypeVariable.java 2015-06-15 17:06:48.583027621 -0700 @@ -36,10 +36,11 @@ /** * Returns the potentially annotated bounds of this type variable. - * Note that if no bound is explicitly declared, the bound is unannotated + * If no bound is explicitly declared, the bound is unannotated * {@code Object}. * * @return the potentially annotated bounds of this type variable + * @see TypeVariable#getBounds() */ AnnotatedType[] getAnnotatedBounds(); } --- old/src/java.base/share/classes/java/lang/reflect/AnnotatedWildcardType.java 2015-06-15 17:06:49.131027638 -0700 +++ new/src/java.base/share/classes/java/lang/reflect/AnnotatedWildcardType.java 2015-06-15 17:06:48.995027634 -0700 @@ -36,20 +36,22 @@ /** * Returns the potentially annotated lower bounds of this wildcard type. - * Note that if no lower bound is explicitly declared, the lower bound is the + * If no lower bound is explicitly declared, the lower bound is the * type of null. In this case, a zero length array is returned. * * @return the potentially annotated lower bounds of this wildcard type or * an empty array if no lower bound is explicitly declared. + * @see WildcardType#getLowerBounds() */ AnnotatedType[] getAnnotatedLowerBounds(); /** * Returns the potentially annotated upper bounds of this wildcard type. - * Note that if no upper bound is explicitly declared, the upper bound is + * If no upper bound is explicitly declared, the upper bound is * unannotated {@code Object} * * @return the potentially annotated upper bounds of this wildcard type + * @see WildcardType#getUpperBounds() */ AnnotatedType[] getAnnotatedUpperBounds(); } --- old/src/java.base/share/classes/java/lang/reflect/TypeVariable.java 2015-06-15 17:06:49.535027650 -0700 +++ new/src/java.base/share/classes/java/lang/reflect/TypeVariable.java 2015-06-15 17:06:49.399027646 -0700 @@ -51,7 +51,7 @@ public interface TypeVariable extends Type, AnnotatedElement { /** * Returns an array of {@code Type} objects representing the - * upper bound(s) of this type variable. Note that if no upper bound is + * upper bound(s) of this type variable. If no upper bound is * explicitly declared, the upper bound is {@code Object}. * *

    For each upper bound B: