From peter.levart at gmail.com Sat Nov 1 17:49:35 2014 From: peter.levart at gmail.com (Peter Levart) Date: Sat, 01 Nov 2014 18:49:35 +0100 Subject: RFR: 8061950: Class.getMethods() exhibits quadratic time complexity In-Reply-To: References: <900D2AE3-B3A6-4B1B-B25A-C564B10334AA@oracle.com> <54492255.3040108@gmail.com> <544D58D4.7020909@gmail.com> <544D7B74.5060607@gmail.com> <5450CEE9.7060707@gmail.com> <545120D2.5010301@gmail.com> <54526D10.5060101@gmail.com> Message-ID: <54551D2F.407@gmail.com> Hi Martin, On 10/30/2014 07:13 PM, Martin Buchholz wrote: > Hi Peter! > > Thanks for all your hard work. > Your patch is hard to digest, but some initial comments anyways: > > --- > > I'm surprised you got this much performance gain loading rt.jar methods. > > It looks like you are creating a more sophisticated data structure > with more garbage, which won't show up as much on our small-memory > benchmarks. Which is why I was expecting to have to write an adaptive > data structure that switches from linear search to hash-based when > some threshold is exceeded. ...or two data structures. Since I have already arranged so that construction of MethodTable gets an upper limit on the total number of possible Method(s) that can be added to it, we can have two MethodTable implementations and switch based on this number: http://cr.openjdk.java.net/~plevart/jdk9-dev/Class.getMethods/webrev.05/ The question is where to draw the line. I have measured the time with a modified http://cr.openjdk.java.net/~plevart/jdk9-dev/Class.getMethods/GetAllRtMethods.java benchmark that executes with various MAX_ARRAY_SIZE values as it goes and results are pretty stable for MAX_ARRAY_SIZE up to about 90, then slowly start degrading (these results should not be compared with previous runs of this benchmark, since they are performed on a slightly slower computer - baseline for MAX_ARRAY_SIZE=-1 would be 0.70 seconds on my faster one which I don't have at the moment): MAX_ARRAY_SIZE=-1: 494141 methods obtained in 0.774282398 seconds; 0 ARRAY(s), 86614 HashMap(s) MAX_ARRAY_SIZE=0: 494141 methods obtained in 0.767787858 seconds; 8742 ARRAY(s), 77872 HashMap(s) MAX_ARRAY_SIZE=1: 494141 methods obtained in 0.769671542 seconds; 16464 ARRAY(s), 70150 HashMap(s) MAX_ARRAY_SIZE=2: 494141 methods obtained in 0.770154629 seconds; 19855 ARRAY(s), 66759 HashMap(s) MAX_ARRAY_SIZE=3: 494141 methods obtained in 0.781438807 seconds; 22341 ARRAY(s), 64273 HashMap(s) MAX_ARRAY_SIZE=4: 494141 methods obtained in 0.768229468 seconds; 24006 ARRAY(s), 62608 HashMap(s) MAX_ARRAY_SIZE=5: 494141 methods obtained in 0.77067988 seconds; 25007 ARRAY(s), 61607 HashMap(s) MAX_ARRAY_SIZE=6: 494141 methods obtained in 0.766908937 seconds; 25600 ARRAY(s), 61014 HashMap(s) MAX_ARRAY_SIZE=7: 494141 methods obtained in 0.770945373 seconds; 26364 ARRAY(s), 60250 HashMap(s) MAX_ARRAY_SIZE=8: 494141 methods obtained in 0.769324065 seconds; 27089 ARRAY(s), 59525 HashMap(s) MAX_ARRAY_SIZE=9: 494141 methods obtained in 0.766438124 seconds; 47479 ARRAY(s), 39135 HashMap(s) MAX_ARRAY_SIZE=10: 494141 methods obtained in 0.76463966 seconds; 48738 ARRAY(s), 37876 HashMap(s) MAX_ARRAY_SIZE=11: 494141 methods obtained in 0.768795889 seconds; 51308 ARRAY(s), 35306 HashMap(s) MAX_ARRAY_SIZE=12: 494141 methods obtained in 0.764373936 seconds; 53262 ARRAY(s), 33352 HashMap(s) MAX_ARRAY_SIZE=13: 494141 methods obtained in 0.762611993 seconds; 55051 ARRAY(s), 31563 HashMap(s) MAX_ARRAY_SIZE=14: 494141 methods obtained in 0.762725072 seconds; 56013 ARRAY(s), 30601 HashMap(s) MAX_ARRAY_SIZE=15: 494141 methods obtained in 0.753798798 seconds; 57471 ARRAY(s), 29143 HashMap(s) MAX_ARRAY_SIZE=16: 494141 methods obtained in 0.757654008 seconds; 58561 ARRAY(s), 28053 HashMap(s) MAX_ARRAY_SIZE=17: 494141 methods obtained in 0.754389446 seconds; 59584 ARRAY(s), 27030 HashMap(s) MAX_ARRAY_SIZE=18: 494141 methods obtained in 0.757966162 seconds; 60535 ARRAY(s), 26079 HashMap(s) ... ... MAX_ARRAY_SIZE=82: 494141 methods obtained in 0.759516432 seconds; 83375 ARRAY(s), 3239 HashMap(s) MAX_ARRAY_SIZE=83: 494141 methods obtained in 0.763177438 seconds; 83406 ARRAY(s), 3208 HashMap(s) MAX_ARRAY_SIZE=84: 494141 methods obtained in 0.761280607 seconds; 83425 ARRAY(s), 3189 HashMap(s) MAX_ARRAY_SIZE=85: 494141 methods obtained in 0.77047745 seconds; 83444 ARRAY(s), 3170 HashMap(s) MAX_ARRAY_SIZE=86: 494141 methods obtained in 0.758688921 seconds; 83486 ARRAY(s), 3128 HashMap(s) MAX_ARRAY_SIZE=87: 494141 methods obtained in 0.759290368 seconds; 83500 ARRAY(s), 3114 HashMap(s) MAX_ARRAY_SIZE=88: 494141 methods obtained in 0.758998186 seconds; 83518 ARRAY(s), 3096 HashMap(s) MAX_ARRAY_SIZE=89: 494141 methods obtained in 0.759510168 seconds; 83581 ARRAY(s), 3033 HashMap(s) MAX_ARRAY_SIZE=90: 494141 methods obtained in 0.757445598 seconds; 83594 ARRAY(s), 3020 HashMap(s) MAX_ARRAY_SIZE=91: 494141 methods obtained in 0.759975452 seconds; 83617 ARRAY(s), 2997 HashMap(s) MAX_ARRAY_SIZE=92: 494141 methods obtained in 0.762257197 seconds; 83635 ARRAY(s), 2979 HashMap(s) MAX_ARRAY_SIZE=93: 494141 methods obtained in 0.759114441 seconds; 83702 ARRAY(s), 2912 HashMap(s) MAX_ARRAY_SIZE=94: 494141 methods obtained in 0.762578409 seconds; 83709 ARRAY(s), 2905 HashMap(s) MAX_ARRAY_SIZE=95: 494141 methods obtained in 0.759205774 seconds; 83724 ARRAY(s), 2890 HashMap(s) MAX_ARRAY_SIZE=96: 494141 methods obtained in 0.762559129 seconds; 83739 ARRAY(s), 2875 HashMap(s) MAX_ARRAY_SIZE=97: 494141 methods obtained in 0.764749586 seconds; 83754 ARRAY(s), 2860 HashMap(s) MAX_ARRAY_SIZE=98: 494141 methods obtained in 0.762137989 seconds; 83762 ARRAY(s), 2852 HashMap(s) MAX_ARRAY_SIZE=99: 494141 methods obtained in 0.761563792 seconds; 83886 ARRAY(s), 2728 HashMap(s) MAX_ARRAY_SIZE=100: 494141 methods obtained in 0.762560357 seconds; 83965 ARRAY(s), 2649 HashMap(s) MAX_ARRAY_SIZE=101: 494141 methods obtained in 0.763043115 seconds; 83983 ARRAY(s), 2631 HashMap(s) MAX_ARRAY_SIZE=102: 494141 methods obtained in 0.763149662 seconds; 84073 ARRAY(s), 2541 HashMap(s) MAX_ARRAY_SIZE=103: 494141 methods obtained in 0.762778262 seconds; 84083 ARRAY(s), 2531 HashMap(s) MAX_ARRAY_SIZE=104: 494141 methods obtained in 0.770542513 seconds; 84103 ARRAY(s), 2511 HashMap(s) MAX_ARRAY_SIZE=105: 494141 methods obtained in 0.764989499 seconds; 84125 ARRAY(s), 2489 HashMap(s) MAX_ARRAY_SIZE=106: 494141 methods obtained in 0.762698653 seconds; 84155 ARRAY(s), 2459 HashMap(s) MAX_ARRAY_SIZE=107: 494141 methods obtained in 0.764577549 seconds; 84171 ARRAY(s), 2443 HashMap(s) MAX_ARRAY_SIZE=108: 494141 methods obtained in 0.766050758 seconds; 84190 ARRAY(s), 2424 HashMap(s) MAX_ARRAY_SIZE=109: 494141 methods obtained in 0.763050679 seconds; 84203 ARRAY(s), 2411 HashMap(s) MAX_ARRAY_SIZE=110: 494141 methods obtained in 0.762198567 seconds; 84217 ARRAY(s), 2397 HashMap(s) ... ... MAX_ARRAY_SIZE=435: 494141 methods obtained in 0.92093385 seconds; 86594 ARRAY(s), 20 HashMap(s) MAX_ARRAY_SIZE=436: 494141 methods obtained in 0.923851043 seconds; 86594 ARRAY(s), 20 HashMap(s) MAX_ARRAY_SIZE=437: 494141 methods obtained in 0.920907745 seconds; 86594 ARRAY(s), 20 HashMap(s) MAX_ARRAY_SIZE=438: 494141 methods obtained in 0.92080572 seconds; 86594 ARRAY(s), 20 HashMap(s) MAX_ARRAY_SIZE=439: 494141 methods obtained in 0.921864791 seconds; 86594 ARRAY(s), 20 HashMap(s) MAX_ARRAY_SIZE=440: 494141 methods obtained in 0.925055685 seconds; 86594 ARRAY(s), 20 HashMap(s) MAX_ARRAY_SIZE=441: 494141 methods obtained in 0.921136685 seconds; 86594 ARRAY(s), 20 HashMap(s) MAX_ARRAY_SIZE=442: 494141 methods obtained in 0.923417116 seconds; 86594 ARRAY(s), 20 HashMap(s) MAX_ARRAY_SIZE=443: 494141 methods obtained in 0.924828923 seconds; 86594 ARRAY(s), 20 HashMap(s) MAX_ARRAY_SIZE=444: 494141 methods obtained in 0.92331634 seconds; 86594 ARRAY(s), 20 HashMap(s) MAX_ARRAY_SIZE=445: 494141 methods obtained in 0.923586046 seconds; 86594 ARRAY(s), 20 HashMap(s) MAX_ARRAY_SIZE=446: 494141 methods obtained in 0.923619987 seconds; 86594 ARRAY(s), 20 HashMap(s) MAX_ARRAY_SIZE=447: 494141 methods obtained in 0.925436079 seconds; 86595 ARRAY(s), 19 HashMap(s) MAX_ARRAY_SIZE=448: 494141 methods obtained in 0.926022715 seconds; 86595 ARRAY(s), 19 HashMap(s) MAX_ARRAY_SIZE=449: 494141 methods obtained in 0.931290541 seconds; 86595 ARRAY(s), 19 HashMap(s) MAX_ARRAY_SIZE=450: 494141 methods obtained in 0.926193262 seconds; 86595 ARRAY(s), 19 HashMap(s) So for common cases (rt.jar classes), the MAX_ARRAY_SIZE could be quite high, but there will always be worst-case scenarios that perform poorly with array based implementation. I'll try to spend some time constructing some of them and measuring the performance with array based implementation. Array based MethodTable is about 32% better then original MethodArray for your ManyMethodsBenchmark, but still O(n^2) of course: Original code: Base class load time: 129.14 ms getDeclaredMethods: Methods: 65521, Total time: 63.59 ms, Time per method: 0.0010 ms getMethods : Methods: 65530, Total time: 49.37 ms, Time per method: 0.0008 ms Derived class load time: 33869.25 ms getDeclaredMethods: Methods: 65521, Total time: 33.65 ms, Time per method: 0.0005 ms getMethods : Methods: 65530, Total time: 8290.76 ms, Time per method: 0.1265 ms Patched (MethodTable with MAX_ARRAY_SIZE=Integer.MAX_VALUE): Base class load time: 132.17 ms getDeclaredMethods: Methods: 65521, Total time: 58.55 ms, Time per method: 0.0009 ms getMethods : Methods: 65530, Total time: 50.58 ms, Time per method: 0.0008 ms Derived class load time: 33735.93 ms getDeclaredMethods: Methods: 65521, Total time: 32.74 ms, Time per method: 0.0005 ms getMethods : Methods: 65530, Total time: 5601.14 ms, Time per method: 0.0855 ms > --- > > (The introduction of default methods into openjdk makes method lookup > even more confusing, and scares me, especially since I am uncertain we > have enough tests...) > > --- > > If your changes to StarInheritance.java are general improvements, we > could/should just check them in now (TDD?). We could. Should I prepare a separate webrev for them? > --- > > Use javadoc comments /** even for private methods > > + /* This method returns 'root' Constructor object. It MUST be > copied with ReflectionFactory > + * before handed to any code outside java.lang.Class or modified. > + */ > private Constructor getConstructor0(Class[] parameterTypes, > > > --- > > The java way is to spell intfcsMethods "interfacesMethods" Will correct this in next webrev. Regards, Peter > > > On Thu, Oct 30, 2014 at 9:53 AM, Peter Levart wrote: >> Hi, >> >> Here's a patch: >> >> http://cr.openjdk.java.net/~plevart/jdk9-dev/Class.getMethods/webrev.04/ >> >> for the following issue: >> >> https://bugs.openjdk.java.net/browse/JDK-8061950 >> >> For those following the thread "Loading classes with many methods is very >> expensive", this is a 4th iteration of this patch. I stepped back from the >> approach taken in 3rd webrev and rather refined approach taken in 2nd >> webrev. Instead of using a LinkedHashMap with values being linked-lists of >> nodes holding Method objects with same signature, which couldn't be made so >> that iteration would follow insertion order, I used plain HashMap this time. >> MethodNode(s) holding Method objects form a linked list of those sharing the >> same signature. In addition MethodNode(s) form a separate doubly-linked list >> of all nodes which is used to keep insertion order. The space use should be >> the same as I traded two object pointers in MethodNode for two less pointers >> in HashMap.Entry (vs. LinkedHashMap.Entry that was used before). So >> insertion order is not tracked by Map but instead by MethodTable now. I also >> track size of MethodTable now so there's no pre-traversing pass to allocate >> Method[] when requested. >> >> This version seems to be the fastest from all tried so far (measured by >> http://cr.openjdk.java.net/~plevart/jdk9-dev/Class.getMethods/GetAllRtMethods.java >> using "-Dsun.reflect.noCaches=true"): >> >> Original: >> >> 19658 classes loaded in 2.027207954 seconds. >> 494392 methods obtained in 0.945519006 seconds. >> 494392 methods obtained in 0.95250834 seconds. >> 494392 methods obtained in 0.917841393 seconds. >> 494392 methods obtained in 0.971922977 seconds. >> 494392 methods obtained in 0.947145131 seconds. >> 494392 methods obtained in 0.937122672 seconds. >> 494392 methods obtained in 0.893975586 seconds. >> 494392 methods obtained in 0.90307736 seconds. >> 494392 methods obtained in 0.918782446 seconds. >> 494392 methods obtained in 0.881968668 seconds. >> >> Patched: >> >> 19658 classes loaded in 2.034081706 seconds. >> 494392 methods obtained in 0.8082686 seconds. >> 494392 methods obtained in 0.743307034 seconds. >> 494392 methods obtained in 0.751591979 seconds. >> 494392 methods obtained in 0.726954964 seconds. >> 494392 methods obtained in 0.761067189 seconds. >> 494392 methods obtained in 0.70825252 seconds. >> 494392 methods obtained in 0.696489363 seconds. >> 494392 methods obtained in 0.692555238 seconds. >> 494392 methods obtained in 0.695150116 seconds. >> 494392 methods obtained in 0.697665068 seconds. >> >> >> I also updated the test that checks multiple inheritance of abstract methods >> as I found that my 1st iteration of the algorithm was not getting the same >> results as original code although all jtreg tests were passing. I added 4 >> cases that include abstract classes as well as interfaces to the mix. Some >> of them would fail with my 1st version of algorithm. >> >> All 86 jtreg tests in java/lang/reflect/ and java/lang/Class/ pass with this >> patch applied. >> >> >> Regards, Peter >> From peter.levart at gmail.com Sat Nov 1 18:13:28 2014 From: peter.levart at gmail.com (Peter Levart) Date: Sat, 01 Nov 2014 19:13:28 +0100 Subject: RFR: 8061950: Class.getMethods() exhibits quadratic time complexity In-Reply-To: References: <900D2AE3-B3A6-4B1B-B25A-C564B10334AA@oracle.com> <54492255.3040108@gmail.com> <544D58D4.7020909@gmail.com> <544D7B74.5060607@gmail.com> <5450CEE9.7060707@gmail.com> <545120D2.5010301@gmail.com> <54526D10.5060101@gmail.com> Message-ID: <545522C8.5050902@gmail.com> On 10/31/2014 11:50 AM, Stanimir Simeonoff wrote: > Hi, > > Personally I have no objections to the latest webrev. > > It looks like you are creating a more sophisticated data structure > with more garbage, which won't show up as much on our small-memory > benchmarks. Which is why I was expecting to have to write an adaptive > data structure that switches from linear search to hash-based when > some threshold is exceeded. > > There is a hidden cost in calling Method.getParameterTypes() even with > linear search. Although it can be optimized to always first check > Method.getName() for reference equality > > So the new approach creates 2 more short lived objects per Method - > HashMap.Node and the MethodList. MethodTable.Impl and its underlying > Object[] are one time off as they are properly sized. > I have no objection if the instances die trivially in the young gen. webrev.02, 04 (and 05) are basically same as garbage is concerned (webrev.03 is a dead-end): webrev.02: creates a LinkedHashMap.Entry + Class.MethodList for each Method webrev.04: creates a HashMap.Entry + MethodTable.MethodNode for each Method webrev.05: is same as 04 + it contains an array based alternative MethodTable which doesn't create any additional objects per Method all approaches create an underlying array. > > A possible optimization would be using a short-circuit in case of no > interfaces and extending directly java.lang.Object (quite a common > case). But even then the Object has quite a few public methods > [getClass(), notify(), notiftyAll(), wait(), wait(long, int), > wait(long), toString(), hashCode(), equals(Object)], so it requires > another HashMap and checking how many of them are overridden. There is a possibility of optimization in Class.getMethod() (single) when there are no (super)interfaces. And for Class.getMethods() (plural) when invoked for and interface with no (super)interfaces - just get declared methods. And of course, the same for Object.class. Any other class has to combine it's own declared methods with at least Object methods, to get a compilation of public methods. > That also reminds me: Object methods should always be cached, > regardless of the config flag. Object class cannot be redefined and > the memory is constant. The flag is not to enable tracking re-definition of classes (even when cache is used, they get properly tracked as cache is invalidated), but for memory constrained environments (and useful for testing too). I don't know if always caching Object methods would help much - perhaps optimizing getMethods() to only return declared public methods for Object.class is enough for memory constrained environments, as the Method objects must always be cloned anyway. Regards, Peter > > Stanimir > --- > > > (The introduction of default methods into openjdk makes method lookup > even more confusing, and scares me, especially since I am uncertain we > have enough tests...) > > --- > > If your changes to StarInheritance.java are general improvements, we > could/should just check them in now (TDD?). > > --- > > Use javadoc comments /** even for private methods > > + /* This method returns 'root' Constructor object. It MUST be > copied with ReflectionFactory > + * before handed to any code outside java.lang.Class or modified. > + */ > private Constructor getConstructor0(Class[] parameterTypes, > > > --- > > The java way is to spell intfcsMethods "interfacesMethods" > > > > On Thu, Oct 30, 2014 at 9:53 AM, Peter Levart > > wrote: > > Hi, > > > > Here's a patch: > > > > > http://cr.openjdk.java.net/~plevart/jdk9-dev/Class.getMethods/webrev.04/ > > > > > for the following issue: > > > > https://bugs.openjdk.java.net/browse/JDK-8061950 > > > > For those following the thread "Loading classes with many > methods is very > > expensive", this is a 4th iteration of this patch. I stepped > back from the > > approach taken in 3rd webrev and rather refined approach taken > in 2nd > > webrev. Instead of using a LinkedHashMap with values being > linked-lists of > > nodes holding Method objects with same signature, which couldn't > be made so > > that iteration would follow insertion order, I used plain > HashMap this time. > > MethodNode(s) holding Method objects form a linked list of those > sharing the > > same signature. In addition MethodNode(s) form a separate > doubly-linked list > > of all nodes which is used to keep insertion order. The space > use should be > > the same as I traded two object pointers in MethodNode for two > less pointers > > in HashMap.Entry (vs. LinkedHashMap.Entry that was used before). So > > insertion order is not tracked by Map but instead by MethodTable > now. I also > > track size of MethodTable now so there's no pre-traversing pass > to allocate > > Method[] when requested. > > > > This version seems to be the fastest from all tried so far > (measured by > > > http://cr.openjdk.java.net/~plevart/jdk9-dev/Class.getMethods/GetAllRtMethods.java > > > using "-Dsun.reflect.noCaches=true"): > > > > Original: > > > > 19658 classes loaded in 2.027207954 seconds. > > 494392 methods obtained in 0.945519006 seconds. > > 494392 methods obtained in 0.95250834 seconds. > > 494392 methods obtained in 0.917841393 seconds. > > 494392 methods obtained in 0.971922977 seconds. > > 494392 methods obtained in 0.947145131 seconds. > > 494392 methods obtained in 0.937122672 seconds. > > 494392 methods obtained in 0.893975586 seconds. > > 494392 methods obtained in 0.90307736 seconds. > > 494392 methods obtained in 0.918782446 seconds. > > 494392 methods obtained in 0.881968668 seconds. > > > > Patched: > > > > 19658 classes loaded in 2.034081706 seconds. > > 494392 methods obtained in 0.8082686 seconds. > > 494392 methods obtained in 0.743307034 seconds. > > 494392 methods obtained in 0.751591979 seconds. > > 494392 methods obtained in 0.726954964 seconds. > > 494392 methods obtained in 0.761067189 seconds. > > 494392 methods obtained in 0.70825252 seconds. > > 494392 methods obtained in 0.696489363 seconds. > > 494392 methods obtained in 0.692555238 seconds. > > 494392 methods obtained in 0.695150116 seconds. > > 494392 methods obtained in 0.697665068 seconds. > > > > > > I also updated the test that checks multiple inheritance of > abstract methods > > as I found that my 1st iteration of the algorithm was not > getting the same > > results as original code although all jtreg tests were passing. > I added 4 > > cases that include abstract classes as well as interfaces to the > mix. Some > > of them would fail with my 1st version of algorithm. > > > > All 86 jtreg tests in java/lang/reflect/ and java/lang/Class/ > pass with this > > patch applied. > > > > > > Regards, Peter > > > From otaviojava at java.net Sun Nov 2 14:45:59 2014 From: otaviojava at java.net (=?UTF-8?Q?Ot=C3=A1vio_Gon=C3=A7alves_de_Santana?=) Date: Sun, 2 Nov 2014 12:45:59 -0200 Subject: Review request: JDK-8055723 Replace concat String to append in StringBuilder parameters In-Reply-To: References: <5445A836.8090005@CoSoCo.de> Message-ID: Could another reviewer look these codes, please. http://cr.openjdk.java.net/~weijun/8055723/webrev.00/ On Fri, Oct 24, 2014 at 3:25 AM, Ot?vio Gon?alves de Santana < otaviojava at java.net> wrote: > Thank you Ulf. > I removed the fix in toString method and in debug classes: > http://cr.openjdk.java.net/~weijun/8055723/webrev.00/ > > On Mon, Oct 20, 2014 at 10:26 PM, Ulf Zibis wrote: > >> >> Am 21.10.2014 um 01:02 schrieb Ot?vio Gon?alves de Santana: >> >>> BUGURL: https://bugs.openjdk.java.net/browse/JDK-8055723 >>> >>> >>> WEBREV: http://cr.openjdk.java.net/~weijun/8055723/client/webrev.02/ >>> WEBREV: http://cr.openjdk.java.net/~weijun/8055723/core/webrev.03/ >>> >> >> I did not look through all sources. >> In Scanner.java I discovered: >> 1307 sb.append("[delimiters=").append(delimPattern).append(']'); >> 1308 sb.append("[position=").append(position).append(']'); >> ... >> Maybe better: >> 1307 sb.append("[delimiters=").append(delimPattern); >> 1308 sb.append("][position=").append(position); >> ... >> >> -Ulf >> >> > > > -- > Ot?vio Gon?alves de Santana > > blog: http://otaviosantana.blogspot.com.br/ > twitter: http://twitter.com/otaviojava > site: *http://about.me/otaviojava * > 55 (11) 98255-3513 > -- Ot?vio Gon?alves de Santana blog: http://otaviosantana.blogspot.com.br/ twitter: http://twitter.com/otaviojava site: *http://about.me/otaviojava * 55 (11) 98255-3513 From david.holmes at oracle.com Mon Nov 3 04:50:09 2014 From: david.holmes at oracle.com (David Holmes) Date: Mon, 03 Nov 2014 14:50:09 +1000 Subject: Review request: JDK-8062556: Add jdk tests for JDK-8058322 and JDK-8058313 In-Reply-To: <54539625.7070302@oracle.com> References: <5452CCA3.5040001@oracle.com> <5452ECF5.7020607@oracle.com> <54539625.7070302@oracle.com> Message-ID: <54570981.2090601@oracle.com> Hi Erik, webrevs still broken for some reason. On 1/11/2014 12:01 AM, Eric McCorkle wrote: > I went through and added comments in the binary data indicating where > the MethodParameters attributes are, and a breakdown of their contents. > I went ahead and did this for all the bad class files, not just the new > ones. > > There is a larger picture here: there's an outstanding task I filed > around the time these tests were written to find a better way for > langtools to run jtreg tests that involve bad class files. > Unfortunately, doing that is rather difficult, as you can see. The only > real way to do it is to generate a class file, convert it to signed > bytes (you can't even use hex; you get an unsigned/signed byte > conversion problem), then modify the data by hand. The intent is to > replace this with a better method at some point. OK. New comments an improvement. Please give the new test the correct initial copyright year of 2014. I know updates to the year are handled automatically (eventually) but we should at least have things correct to start with. Thanks, David > On 10/30/14 21:59, David Holmes wrote: >> Hi Erik, >> >> On 31/10/2014 9:41 AM, Eric McCorkle wrote: >>> Hello, >>> >>> Please review this patch which adds tests to the JDK test suite for two >>> reflection bugs that require hotspot changes (JDK-8058322 and >>> JDK-8058313) >>> >>> The webrev is here: >>> http://cr.openjdk.java.net/~emc/8062556/ >> >> I second Brian's comment re the source of the bad classes. >> >> Your webrev is broken btw - no top-level html files. >> >> The new test needs a copyright year of 2014 not 2013. >> >> Thanks, >> David >> From attila.szegedi at oracle.com Mon Nov 3 07:30:37 2014 From: attila.szegedi at oracle.com (Attila Szegedi) Date: Mon, 3 Nov 2014 08:30:37 +0100 Subject: [9] RFR (XS): 8060483: NPE with explicitCastArguments unboxing null In-Reply-To: References: <543E64BD.9000103@oracle.com> Message-ID: <7A9A3DA4-B7BB-4E29-889B-1BD69C984E58@oracle.com> Thanks for fixing this. Can we expect a backport to 8u-dev soon? On Oct 15, 2014, at 4:46 PM, Attila Szegedi wrote: > +1 > > On Wed, Oct 15, 2014 at 2:12 PM, Vladimir Ivanov < > vladimir.x.ivanov at oracle.com> wrote: > >> http://cr.openjdk.java.net/~vlivanov/8060483/webrev.00/ >> https://bugs.openjdk.java.net/browse/JDK-8060483 >> >> Recent changes broke MHs.explicitCastArguments for null values when >> unboxing taking place. >> >> The fix corrects the equivalence check for MHs.eCA & MH.asType() in MT. >> explicitCastEquivalentToAsType(). >> >> Also, will file a followup enhancement request to improve test coverage >> for MHs.eCA method. >> >> Testing: regression test, jck (api/java_lang/invoke), jdk/java/lang/invoke >> >> Thanks! >> >> Best regards, >> Vladimir Ivanov >> _______________________________________________ >> mlvm-dev mailing list >> mlvm-dev at openjdk.java.net >> http://mail.openjdk.java.net/mailman/listinfo/mlvm-dev >> From david.r.chase at oracle.com Mon Nov 3 12:49:55 2014 From: david.r.chase at oracle.com (David Chase) Date: Mon, 3 Nov 2014 07:49:55 -0500 Subject: [9] RFR(L) 8013267 : move MemberNameTable from native code to Java heap, use to intern MemberNames In-Reply-To: <5456FB59.60905@oracle.com> References: <594FE416-D445-426E-B7A9-C75F012ADE16@oracle.com> <0BDCD5DD-CBFB-4A3D-9BAD-7F9912E9237C@oracle.com> <5453C230.8010709@oracle.com> <9747A3A3-B7F4-407A-8E00-1E647D9DC2D1@oracle.com> <1D195ED3-8BD2-46CB-9D70-29CF809D9F5A@oracle.com> <5456FB59.60905@oracle.com> Message-ID: <632A5C98-B386-4625-BE12-355241581955@oracle.com> On 2014-11-02, at 10:49 PM, David Holmes wrote: >> The change is to load the volatile size for the loop bound; this stops the stores >> in the loop from moving earlier, right? > > Treating volatile accesses like memory barriers is playing a bit fast-and-loose with the spec+implementation. The basic happens-before relationship for volatiles states that if a volatile read sees a value X, then the volatile write that wrote X happened-before the read [1]. But in this code there are no checks of the values of the volatile fields. Instead you are relying on a volatile read "acting like acquire()" and a volatile write "acting like release()". > > That said you are trying to "synchronize" the hotspot code with the JDK code so you have stepped outside the JMM in any case and reasoning about what is and is not allowed is somewhat moot - unless the hotspot code always uses Java-style accesses to the Java-level variables. My main concern is that the compiler is inhibited from any peculiar code motion; I assume that taking a safe point has a bit of barrier built into it anyway, especially given that the worry case is safepoint + JVMTI. Given the worry, what?s the best way to spell ?barrier? here? I could synchronize on classData (it would be a recursive lock in the current version of the code) synchronized (this) { size++; } or I could synchronize on elementData (no longer used for a lock elsewhere, so always uncontended) synchronized (elementData) { size++; } or is there some Unsafe thing that would be better? (core-libs-dev ? there will be another webrev coming. This is a runtime+jdk patch.) David > BTW the Java side of this needs to be reviewed on core-libs-dev at openjdk.java.net > > David H. > > [1] http://docs.oracle.com/javase/specs/jls/se8/html/jls-17.html#jls-17.4.4 > > >> David From eric.mccorkle at oracle.com Mon Nov 3 15:42:40 2014 From: eric.mccorkle at oracle.com (Eric McCorkle) Date: Mon, 03 Nov 2014 10:42:40 -0500 Subject: Review request: JDK-8062556: Add jdk tests for JDK-8058322 and JDK-8058313 In-Reply-To: <54570981.2090601@oracle.com> References: <5452CCA3.5040001@oracle.com> <5452ECF5.7020607@oracle.com> <54539625.7070302@oracle.com> <54570981.2090601@oracle.com> Message-ID: <5457A270.9010009@oracle.com> I have been having issues with webrev, which I reported earlier. Webrev reports a syntax error when I try to use it, and curiously, it fails to produce top-level files in this case (and this case only, as evidenced by my other webrevs). Unfortunately, there's nothing I can do about the missing top-level files; however, you can still look at the individual files just fine. On 11/02/14 23:50, David Holmes wrote: > Hi Erik, > > webrevs still broken for some reason. > On 1/11/2014 12:01 AM, Eric McCorkle wrote: >> I went through and added comments in the binary data indicating where >> the MethodParameters attributes are, and a breakdown of their contents. >> I went ahead and did this for all the bad class files, not just the new >> ones. >> >> There is a larger picture here: there's an outstanding task I filed >> around the time these tests were written to find a better way for >> langtools to run jtreg tests that involve bad class files. >> Unfortunately, doing that is rather difficult, as you can see. The only >> real way to do it is to generate a class file, convert it to signed >> bytes (you can't even use hex; you get an unsigned/signed byte >> conversion problem), then modify the data by hand. The intent is to >> replace this with a better method at some point. > > OK. New comments an improvement. > > Please give the new test the correct initial copyright year of 2014. I > know updates to the year are handled automatically (eventually) but we > should at least have things correct to start with. > > Thanks, > David > >> On 10/30/14 21:59, David Holmes wrote: >>> Hi Erik, >>> >>> On 31/10/2014 9:41 AM, Eric McCorkle wrote: >>>> Hello, >>>> >>>> Please review this patch which adds tests to the JDK test suite for two >>>> reflection bugs that require hotspot changes (JDK-8058322 and >>>> JDK-8058313) >>>> >>>> The webrev is here: >>>> http://cr.openjdk.java.net/~emc/8062556/ >>> >>> I second Brian's comment re the source of the bad classes. >>> >>> Your webrev is broken btw - no top-level html files. >>> >>> The new test needs a copyright year of 2014 not 2013. >>> >>> Thanks, >>> David >>> From peter.levart at gmail.com Mon Nov 3 16:16:53 2014 From: peter.levart at gmail.com (Peter Levart) Date: Mon, 03 Nov 2014 17:16:53 +0100 Subject: [9] RFR(L) 8013267 : move MemberNameTable from native code to Java heap, use to intern MemberNames In-Reply-To: <632A5C98-B386-4625-BE12-355241581955@oracle.com> References: <594FE416-D445-426E-B7A9-C75F012ADE16@oracle.com> <0BDCD5DD-CBFB-4A3D-9BAD-7F9912E9237C@oracle.com> <5453C230.8010709@oracle.com> <9747A3A3-B7F4-407A-8E00-1E647D9DC2D1@oracle.com> <1D195ED3-8BD2-46CB-9D70-29CF809D9F5A@oracle.com> <5456FB59.60905@oracle.com> <632A5C98-B386-4625-BE12-355241581955@oracle.com> Message-ID: <5457AA75.8090103@gmail.com> On 11/03/2014 01:49 PM, David Chase wrote: > On 2014-11-02, at 10:49 PM, David Holmes wrote: >>> The change is to load the volatile size for the loop bound; this stops the stores >>> in the loop from moving earlier, right? >> Treating volatile accesses like memory barriers is playing a bit fast-and-loose with the spec+implementation. The basic happens-before relationship for volatiles states that if a volatile read sees a value X, then the volatile write that wrote X happened-before the read [1]. But in this code there are no checks of the values of the volatile fields. Instead you are relying on a volatile read "acting like acquire()" and a volatile write "acting like release()". >> >> That said you are trying to "synchronize" the hotspot code with the JDK code so you have stepped outside the JMM in any case and reasoning about what is and is not allowed is somewhat moot - unless the hotspot code always uses Java-style accesses to the Java-level variables. > My main concern is that the compiler is inhibited from any peculiar code motion; I assume that taking a safe point has a bit of barrier built into it anyway, especially given that the worry case is safepoint + JVMTI. > > Given the worry, what?s the best way to spell ?barrier? here? > I could synchronize on classData (it would be a recursive lock in the current version of the code) > synchronized (this) { size++; } > or I could synchronize on elementData (no longer used for a lock elsewhere, so always uncontended) > synchronized (elementData) { size++; } > or is there some Unsafe thing that would be better? > > (core-libs-dev ? there will be another webrev coming. This is a runtime+jdk patch.) > > David Hi David, You're worried that writes moving array elements up for one slot would bubble up before write of size = size+1, right? If that happens, VM could skip an existing (last) element and not update it. It seems that Unsafe.storeFence() between size++ and moving of elements could do, as the javadoc for it says: /** * Ensures lack of reordering of stores before the fence * with loads or stores after the fence. * @since 1.8 */ public native void storeFence(); Regards, Peter > >> BTW the Java side of this needs to be reviewed on core-libs-dev at openjdk.java.net >> >> David H. >> >> [1] http://docs.oracle.com/javase/specs/jls/se8/html/jls-17.html#jls-17.4.4 >> >> >>> David From konstantin.shefov at oracle.com Mon Nov 3 16:28:17 2014 From: konstantin.shefov at oracle.com (Konstantin Shefov) Date: Mon, 03 Nov 2014 19:28:17 +0300 Subject: [9] Review request : JDK-8059070: [TESTBUG] java/lang/invoke/LFCaching/LFMultiThreadCachingTest.java failed - timeout In-Reply-To: <5450F8E4.2060808@oracle.com> References: <543D1DEE.8030206@oracle.com> <543F8520.9090408@oracle.com> <8F9DD322-1793-4739-8155-61CE7D2C5AAB@oracle.com> <543F9800.7040406@oracle.com> <543FC71F.8020504@oracle.com> <5440E39C.8070001@oracle.com> <5448E5A2.9030804@oracle.com> <544E1B65.1090400@oracle.com> <5450F8E4.2060808@oracle.com> Message-ID: <5457AD21.4070909@oracle.com> Gently reminder 29.10.2014 17:25, Konstantin Shefov ?????: > Please, review a test bug fix. > http://cr.openjdk.java.net/~kshefov/8059070/webrev.01/ > > -Konstantin > > On 27.10.2014 13:16, Konstantin Shefov wrote: >> Kindly reminder >> >> On 23.10.2014 19:04, Paul Sandoz wrote: >>> On Oct 23, 2014, at 1:25 PM, Konstantin Shefov >>> wrote: >>>> Gently reminder >>>> >>>> On 17.10.2014 13:38, Konstantin Shefov wrote: >>>>> Hi, >>>>> >>>>> I have updated the webrev: >>>>> http://cr.openjdk.java.net/~kshefov/8059070/webrev.01/ >>>>> >>> +1 >>> >>> Sorry for the delay, >>> Paul. >> > From david.r.chase at oracle.com Mon Nov 3 16:36:59 2014 From: david.r.chase at oracle.com (David Chase) Date: Mon, 3 Nov 2014 11:36:59 -0500 Subject: [9] RFR(L) 8013267 : move MemberNameTable from native code to Java heap, use to intern MemberNames In-Reply-To: <5457AA75.8090103@gmail.com> References: <594FE416-D445-426E-B7A9-C75F012ADE16@oracle.com> <0BDCD5DD-CBFB-4A3D-9BAD-7F9912E9237C@oracle.com> <5453C230.8010709@oracle.com> <9747A3A3-B7F4-407A-8E00-1E647D9DC2D1@oracle.com> <1D195ED3-8BD2-46CB-9D70-29CF809D9F5A@oracle.com> <5456FB59.60905@oracle.com> <632A5C98-B386-4625-BE12-355241581955@oracle.com> <5457AA75.8090103@gmail.com> Message-ID: >> My main concern is that the compiler is inhibited from any peculiar code motion; I assume that taking a safe point has a bit of barrier built into it anyway, especially given that the worry case is safepoint + JVMTI. >> >> Given the worry, what?s the best way to spell ?barrier? here? >> I could synchronize on classData (it would be a recursive lock in the current version of the code) >> synchronized (this) { size++; } >> or I could synchronize on elementData (no longer used for a lock elsewhere, so always uncontended) >> synchronized (elementData) { size++; } >> or is there some Unsafe thing that would be better? > > You're worried that writes moving array elements up for one slot would bubble up before write of size = size+1, right? If that happens, VM could skip an existing (last) element and not update it. exactly, with the restriction that it would be compiler-induced bubbling, not architectural. Which is both better, and worse ? I don?t have to worry about crazy hardware, but the rules of java/jvm "memory model" are not as thoroughly defined as those for java itself. I added a method to Atomic (.storeFence() ). New webrev to come after I rebuild and retest. Thanks much, David > It seems that Unsafe.storeFence() between size++ and moving of elements could do, as the javadoc for it says: > > /** > * Ensures lack of reordering of stores before the fence > * with loads or stores after the fence. > * @since 1.8 > */ > public native void storeFence(); From peter.levart at gmail.com Mon Nov 3 16:42:30 2014 From: peter.levart at gmail.com (Peter Levart) Date: Mon, 03 Nov 2014 17:42:30 +0100 Subject: [9] RFR(L) 8013267 : move MemberNameTable from native code to Java heap, use to intern MemberNames In-Reply-To: <5457AA75.8090103@gmail.com> References: <594FE416-D445-426E-B7A9-C75F012ADE16@oracle.com> <0BDCD5DD-CBFB-4A3D-9BAD-7F9912E9237C@oracle.com> <5453C230.8010709@oracle.com> <9747A3A3-B7F4-407A-8E00-1E647D9DC2D1@oracle.com> <1D195ED3-8BD2-46CB-9D70-29CF809D9F5A@oracle.com> <5456FB59.60905@oracle.com> <632A5C98-B386-4625-BE12-355241581955@oracle.com> <5457AA75.8090103@gmail.com> Message-ID: <5457B076.10205@gmail.com> On 11/03/2014 05:16 PM, Peter Levart wrote: > On 11/03/2014 01:49 PM, David Chase wrote: >> On 2014-11-02, at 10:49 PM, David Holmes >> wrote: >>>> The change is to load the volatile size for the loop bound; this >>>> stops the stores >>>> in the loop from moving earlier, right? >>> Treating volatile accesses like memory barriers is playing a bit >>> fast-and-loose with the spec+implementation. The basic >>> happens-before relationship for volatiles states that if a volatile >>> read sees a value X, then the volatile write that wrote X >>> happened-before the read [1]. But in this code there are no checks >>> of the values of the volatile fields. Instead you are relying on a >>> volatile read "acting like acquire()" and a volatile write "acting >>> like release()". >>> >>> That said you are trying to "synchronize" the hotspot code with the >>> JDK code so you have stepped outside the JMM in any case and >>> reasoning about what is and is not allowed is somewhat moot - unless >>> the hotspot code always uses Java-style accesses to the Java-level >>> variables. >> My main concern is that the compiler is inhibited from any peculiar >> code motion; I assume that taking a safe point has a bit of barrier >> built into it anyway, especially given that the worry case is >> safepoint + JVMTI. >> >> Given the worry, what?s the best way to spell ?barrier? here? >> I could synchronize on classData (it would be a recursive lock in the >> current version of the code) >> synchronized (this) { size++; } >> or I could synchronize on elementData (no longer used for a lock >> elsewhere, so always uncontended) >> synchronized (elementData) { size++; } >> or is there some Unsafe thing that would be better? >> >> (core-libs-dev ? there will be another webrev coming. This is a >> runtime+jdk patch.) >> >> David > > Hi David, > > You're worried that writes moving array elements up for one slot would > bubble up before write of size = size+1, right? If that happens, VM > could skip an existing (last) element and not update it. > > It seems that Unsafe.storeFence() between size++ and moving of > elements could do, as the javadoc for it says: > > /** > * Ensures lack of reordering of stores before the fence > * with loads or stores after the fence. > * @since 1.8 > */ > public native void storeFence(); You might need a storeFence() between each two writes into the array too. Your moving loop is the following: 2544 for (int i = oldCapacity; i > index; i--) { 2545 // pre: element_data[i] is duplicated at [i+1] 2546 element_data[i] = element_data[i - 1]; 2547 // post: element_data[i-1] is duplicated at [i] 2548 } If we start unrolling, it becomes: w1: element_data[old_capacity - 0] = element_data[old_capacity - 1]; w2: element_data[old_capacity - 1] = element_data[old_capacity - 2]; w3: element_data[old_capacity - 2] = element_data[old_capacity - 3]; ... Can compiler reorder w2 and w3 (just writes - not the whole statements)? Say that it reads a chunk of elements into the registers and then writes them out, but in different order, and a check for safepoint comes inside this chunk of writes... This is hypothetical, but it could do it without breaking the local semantics... Peter > > > Regards, Peter > > > >> >>> BTW the Java side of this needs to be reviewed on >>> core-libs-dev at openjdk.java.net >>> >>> David H. >>> >>> [1] >>> http://docs.oracle.com/javase/specs/jls/se8/html/jls-17.html#jls-17.4.4 >>> >>> >>>> David > From david.r.chase at oracle.com Mon Nov 3 17:28:00 2014 From: david.r.chase at oracle.com (David Chase) Date: Mon, 3 Nov 2014 12:28:00 -0500 Subject: [9] RFR(L) 8013267 : move MemberNameTable from native code to Java heap, use to intern MemberNames In-Reply-To: <5457B076.10205@gmail.com> References: <594FE416-D445-426E-B7A9-C75F012ADE16@oracle.com> <0BDCD5DD-CBFB-4A3D-9BAD-7F9912E9237C@oracle.com> <5453C230.8010709@oracle.com> <9747A3A3-B7F4-407A-8E00-1E647D9DC2D1@oracle.com> <1D195ED3-8BD2-46CB-9D70-29CF809D9F5A@oracle.com> <5456FB59.60905@oracle.com> <632A5C98-B386-4625-BE12-355241581955@oracle.com> <5457AA75.8090103@gmail.com> <5457B076.10205@gmail.com> Message-ID: <73CF1882-22F1-4D9A-B37A-EAC9BCF675B0@oracle.com> On 2014-11-03, at 11:42 AM, Peter Levart wrote: >> You're worried that writes moving array elements up for one slot would bubble up before write of size = size+1, right? If that happens, VM could skip an existing (last) element and not update it. >> >> It seems that Unsafe.storeFence() between size++ and moving of elements could do, as the javadoc for it says: >> >> /** >> * Ensures lack of reordering of stores before the fence >> * with loads or stores after the fence. >> * @since 1.8 >> */ >> public native void storeFence(); > You might need a storeFence() between each two writes into the array too. Your moving loop is the following: > > 2544 for (int i = oldCapacity; i > index; i--) { > 2545 // pre: element_data[i] is duplicated at [i+1] > 2546 element_data[i] = element_data[i - 1]; > 2547 // post: element_data[i-1] is duplicated at [i] > 2548 } > > > If we start unrolling, it becomes: > > w1: element_data[old_capacity - 0] = element_data[old_capacity - 1]; > w2: element_data[old_capacity - 1] = element_data[old_capacity - 2]; > w3: element_data[old_capacity - 2] = element_data[old_capacity - 3]; > ... > > Can compiler reorder w2 and w3 (just writes - not the whole statements)? Say that it reads a chunk of elements into the registers and then writes them out, but in different order, and a check for safepoint comes inside this chunk of writes... This is hypothetical, but it could do it without breaking the local semantics? I think you are right, certainly in theory, and if I don?t hear someone else declaring that in practice we?re both just being paranoid, I?ll do that too. Seems like it might eventually slow things down to do all those fences. David From peter.levart at gmail.com Mon Nov 3 20:09:29 2014 From: peter.levart at gmail.com (Peter Levart) Date: Mon, 03 Nov 2014 21:09:29 +0100 Subject: [9] RFR(L) 8013267 : move MemberNameTable from native code to Java heap, use to intern MemberNames In-Reply-To: References: <594FE416-D445-426E-B7A9-C75F012ADE16@oracle.com> <0BDCD5DD-CBFB-4A3D-9BAD-7F9912E9237C@oracle.com> <5453C230.8010709@oracle.com> <9747A3A3-B7F4-407A-8E00-1E647D9DC2D1@oracle.com> <1D195ED3-8BD2-46CB-9D70-29CF809D9F5A@oracle.com> <5456FB59.60905@oracle.com> <632A5C98-B386-4625-BE12-355241581955@oracle.com> <5457AA75.8090103@gmail.com> Message-ID: <5457E0F9.8090004@gmail.com> Hi David, I was thinking about the fact that java.lang.invoke code is leaking into java.lang.Class. Perhaps, If you don't mind rewriting the code, a better code structure would be, if j.l.Class changes only consisted of adding a simple: + // A reference to canonicalizing cache of java.lang.invoke.MemberName(s) + // for members declared by class represented by this Class object + private transient volatile Object memberNameData; ...and nothing else. All the logic could live in MemberName itself (together with Unsafe machinery for accessing/cas-ing Class.memberNameData). Now to an idea about implementation. Since VM code is not doing any binary-search and only linearly scans the array when it has to update MemberNames, the code could be changed to scan a linked-list of MemberName(s) instead. You could add a field to MemberName: class MemberName { ... // next MemberName in chain of interned MemberNames for particular declaring class private MemberName next; Have a volatile field in MemberNameData (or ClassData - whatever you call it): class MemberNameData { ... // a chain of interned MemberName(s) for particular declaring class // accessed by VM when it has to modify them in-place private volatile MemberName memberNames; MemberName add(Class klass, int index, MemberName mn, int redefined_count) { mn.next = memberNames; memberNames = mn; if (jla.getClassRedefinedCount(klass) == redefined_count) { // no changes to class ... ... code to update array of sorted MemberName(s) with new 'mn' ... return mn; } // lost race, undo insertion memberNames = mn.next; return null; } This way all the worries about ordering of writes into array and/or size are gone. The array is still used to quickly search for an element, but VM only scans the linked-list. What do you think of this? Regards, Peter On 11/03/2014 05:36 PM, David Chase wrote: >>> My main concern is that the compiler is inhibited from any peculiar code motion; I assume that taking a safe point has a bit of barrier built into it anyway, especially given that the worry case is safepoint + JVMTI. >>> >>> Given the worry, what?s the best way to spell ?barrier? here? >>> I could synchronize on classData (it would be a recursive lock in the current version of the code) >>> synchronized (this) { size++; } >>> or I could synchronize on elementData (no longer used for a lock elsewhere, so always uncontended) >>> synchronized (elementData) { size++; } >>> or is there some Unsafe thing that would be better? >> You're worried that writes moving array elements up for one slot would bubble up before write of size = size+1, right? If that happens, VM could skip an existing (last) element and not update it. > exactly, with the restriction that it would be compiler-induced bubbling, not architectural. > Which is both better, and worse ? I don?t have to worry about crazy hardware, but the rules > of java/jvm "memory model" are not as thoroughly defined as those for java itself. > > I added a method to Atomic (.storeFence() ). New webrev to come after I rebuild and retest. > > Thanks much, > > David > >> It seems that Unsafe.storeFence() between size++ and moving of elements could do, as the javadoc for it says: >> >> /** >> * Ensures lack of reordering of stores before the fence >> * with loads or stores after the fence. >> * @since 1.8 >> */ >> public native void storeFence(); From david.r.chase at oracle.com Mon Nov 3 20:41:30 2014 From: david.r.chase at oracle.com (David Chase) Date: Mon, 3 Nov 2014 15:41:30 -0500 Subject: [9] RFR(L) 8013267 : move MemberNameTable from native code to Java heap, use to intern MemberNames In-Reply-To: <5457E0F9.8090004@gmail.com> References: <594FE416-D445-426E-B7A9-C75F012ADE16@oracle.com> <0BDCD5DD-CBFB-4A3D-9BAD-7F9912E9237C@oracle.com> <5453C230.8010709@oracle.com> <9747A3A3-B7F4-407A-8E00-1E647D9DC2D1@oracle.com> <1D195ED3-8BD2-46CB-9D70-29CF809D9F5A@oracle.com> <5456FB59.60905@oracle.com> <632A5C98-B386-4625-BE12-355241581955@oracle.com> <5457AA75.8090103@gmail.com> <5457E0F9.8090004@gmail.com> Message-ID: On 2014-11-03, at 3:09 PM, Peter Levart wrote: > Hi David, > > I was thinking about the fact that java.lang.invoke code is leaking into java.lang.Class. Perhaps, If you don't mind rewriting the code, a better code structure would be, if j.l.Class changes only consisted of adding a simple: > ? > This way all the worries about ordering of writes into array and/or size are gone. The array is still used to quickly search for an element, but VM only scans the linked-list. > > What do you think of this? I?m not sure. I know Coleen Ph would like to see that happen. A couple of people have vague plans to move more of the MemberName resolution into core libs. (Years ago I worked on a VM where *all* of this occurred in Java, but some of it was ahead of time.) I heard mention of ?we want to put more stuff in there? but I got the impression that already happened (there?s reflection data, for example) so I?m not sure that makes sense. There?s also a proposal from people in the runtime to just use a jmethodid, take the hit of an extra indirection, and no need to for this worrisome jvm/java concurrency. And if we instead wrote a hash table that only grew, and never relocated elements, we could (I think) allow non-synchronized O(1) probes of the table from the Java side, synchronized O(1) insertions from the Java side, and because nothing moves, a smaller dance with the VM. I?m rather tempted to look into this ? given the amount of work it would take to do the benchmarking to see if (a) jmethodid would have acceptable performance or (b) the existing costs are too high, I could instead just write fast code and be done. And another way to view this is that we?re now quibbling about performance, when we still have an existing correctness problem that this patch solves, so maybe we should just get this done and then file an RFE. David From huizhe.wang at oracle.com Mon Nov 3 21:09:41 2014 From: huizhe.wang at oracle.com (huizhe wang) Date: Mon, 03 Nov 2014 13:09:41 -0800 Subject: RFR (JAXP) 6770436 : Entity callback order differs between Java1.5 and Java1.6 Message-ID: <5457EF15.6040208@oracle.com> Hi, The order of the Character event for a built-in entity reference was wrong after the integration of StAX in JDK 6. The error was introduced when all Character event was moved to the scanDocument method. The Character event in handleCharacter method was then commented out to avoid duplicate Character event. However, that change also changed the event sequence, startEntity - characters - endEntity became startEntity - endEntity - characters. The following fix adds a flag builtInRefCharacterHandled to indicate whether or not the Character event has been handled. If yes, an ENTITY_REFERENCE is returned, which does nothing. http://cr.openjdk.java.net/~joehw/jdk9/6770436/webrev/ Please review. Thanks, Joe From christian.thalinger at oracle.com Mon Nov 3 21:30:07 2014 From: christian.thalinger at oracle.com (Christian Thalinger) Date: Mon, 3 Nov 2014 13:30:07 -0800 Subject: [9] RFR(L) 8013267 : move MemberNameTable from native code to Java heap, use to intern MemberNames In-Reply-To: References: <594FE416-D445-426E-B7A9-C75F012ADE16@oracle.com> <0BDCD5DD-CBFB-4A3D-9BAD-7F9912E9237C@oracle.com> <5453C230.8010709@oracle.com> <9747A3A3-B7F4-407A-8E00-1E647D9DC2D1@oracle.com> <1D195ED3-8BD2-46CB-9D70-29CF809D9F5A@oracle.com> <5456FB59.60905@oracle.com> <632A5C98-B386-4625-BE12-355241581955@oracle.com> <5457AA75.8090103@gmail.com> <5457E0F9.8090004@gmail.com> Message-ID: <00703487-00EB-4E43-9613-01EE9EE64147@oracle.com> > On Nov 3, 2014, at 12:41 PM, David Chase wrote: > > > On 2014-11-03, at 3:09 PM, Peter Levart wrote: > >> Hi David, >> >> I was thinking about the fact that java.lang.invoke code is leaking into java.lang.Class. Perhaps, If you don't mind rewriting the code, a better code structure would be, if j.l.Class changes only consisted of adding a simple: >> ? > >> This way all the worries about ordering of writes into array and/or size are gone. The array is still used to quickly search for an element, but VM only scans the linked-list. >> >> What do you think of this? > > I?m not sure. I know Coleen Ph would like to see that happen. > > A couple of people have vague plans to move more of the MemberName resolution into core libs. > (Years ago I worked on a VM where *all* of this occurred in Java, but some of it was ahead of time.) > > I heard mention of ?we want to put more stuff in there? but I got the impression that already happened > (there?s reflection data, for example) so I?m not sure that makes sense. > > There?s also a proposal from people in the runtime to just use a jmethodid, take the hit of an extra indirection, > and no need to for this worrisome jvm/java concurrency. > > And if we instead wrote a hash table that only grew, and never relocated elements, we could > (I think) allow non-synchronized O(1) probes of the table from the Java side, synchronized > O(1) insertions from the Java side, and because nothing moves, a smaller dance with the > VM. I?m rather tempted to look into this ? given the amount of work it would take to do the > benchmarking to see if (a) jmethodid would have acceptable performance or (b) the existing > costs are too high, I could instead just write fast code and be done. ?but you still have to do the benchmarking. Let?s not forget that there was a performance regression with the first C++ implementation of this. > > And another way to view this is that we?re now quibbling about performance, when we still > have an existing correctness problem that this patch solves, so maybe we should just get this > done and then file an RFE. > > David > From eric.mccorkle at oracle.com Mon Nov 3 21:34:23 2014 From: eric.mccorkle at oracle.com (Eric McCorkle) Date: Mon, 03 Nov 2014 16:34:23 -0500 Subject: Review request for 8055063: Parameter#toString() fails w/ AIOOBE for ctr of inner class w/ generic type In-Reply-To: <5453B58A.1090507@oracle.com> References: <5453B58A.1090507@oracle.com> Message-ID: <5457F4DF.5070703@oracle.com> This issue is an 8u40 issue, so it needs to be reviewed ASAP in order to backport it in time. On 10/31/14 12:15, Eric McCorkle wrote: > Hello, > > Please review this patch which fixes issues that arise with > getGenericParameterTypes() and getAnnotatedParameterTypes() when there > are generic signatures and synthetic parameters. > > Please note that a complete fix is not possible for all cases. See > discussion on https://bugs.openjdk.java.net/browse/JDK-8062582 for details. > > This patch will cause Executable.getAnnotatedParameterTypes(), > Parameter.getAnnotatedType(), and Parameter.getParameterizedType() to > report the correct types in the following cases: > > * No generic signature is present. > * Both a generic signature and method parameters information are present > * A generic signature is present, but method parameters information is > not present, but the number of parameters in the generic signature and > the number of parameters in the method descriptor are the same. > > In the problematic case, where there is a generic signature, no method > parameters information, and the generic signature does not match the > method descriptor, these methods will return the correct /non/-generic > type, as there is no general way of associating parameters in the > generic signature with those in the method descriptor in this case. > > Please also note that there is currently a bug in javac which causes > type annotations' parameter indexes to be wrong when synthetic > parameters are generated: https://bugs.openjdk.java.net/browse/JDK-8029012. > > The bug report is here: > https://bugs.openjdk.java.net/browse/JDK-8055063 > > The webrev is here: > http://cr.openjdk.java.net/~emc/8055063/ > From huizhe.wang at oracle.com Mon Nov 3 23:55:01 2014 From: huizhe.wang at oracle.com (huizhe wang) Date: Mon, 03 Nov 2014 15:55:01 -0800 Subject: RFR 8047962: XML test colocation: AuctionPortal test for bid.com In-Reply-To: References: Message-ID: <545815D5.5020009@oracle.com> Hi Tristan, Looks good overall. Once again, it's great to see that you've made the tests a lot cleaner, getting rid of the old report system and etc. The only issue I see is with movies.xml. If I use patch to apply your patch to my workspace, I get no movies.xml that in turn causes AuctionController to fail. It's probably related how your webrev was generated (note the error: Unexpected Error occurred reading `diff -e /dev/null new/test/javax/xml/jaxp/functional/test/content/movies.xml`: $?=0, err= 1 ). hg diff works fine generally, but not for changes in binary files. Use the git option, "-git", can fix the problem. You may see the following in your configuration: [diff] git = true Thanks, Joe On 10/31/2014 12:16 PM, Tristan Yan wrote: > Hi Joe, Alan and all others > > Would you please help reviewing these tests? The intent is moving some > JAXP tests from closed to open. The associated bug number is > https://bugs.openjdk.java.net/browse/JDK-8047962. > > These tests have been ran with and without the Security Manager. All > the tests pass under both scenarios. > > http://cr.openjdk.java.net/~tyan/JDK-8047962/webrev.01/ > > Thank you. > Tristan From peter.levart at gmail.com Tue Nov 4 10:07:56 2014 From: peter.levart at gmail.com (Peter Levart) Date: Tue, 04 Nov 2014 11:07:56 +0100 Subject: [9] RFR(L) 8013267 : move MemberNameTable from native code to Java heap, use to intern MemberNames In-Reply-To: References: <594FE416-D445-426E-B7A9-C75F012ADE16@oracle.com> <0BDCD5DD-CBFB-4A3D-9BAD-7F9912E9237C@oracle.com> <5453C230.8010709@oracle.com> <9747A3A3-B7F4-407A-8E00-1E647D9DC2D1@oracle.com> <1D195ED3-8BD2-46CB-9D70-29CF809D9F5A@oracle.com> <5456FB59.60905@oracle.com> <632A5C98-B386-4625-BE12-355241581955@oracle.com> <5457AA75.8090103@gmail.com> <5457E0F9.8090004@gmail.com> Message-ID: <5458A57C.4060208@gmail.com> On 11/03/2014 09:41 PM, David Chase wrote: > On 2014-11-03, at 3:09 PM, Peter Levart wrote: > >> Hi David, >> >> I was thinking about the fact that java.lang.invoke code is leaking into java.lang.Class. Perhaps, If you don't mind rewriting the code, a better code structure would be, if j.l.Class changes only consisted of adding a simple: >> ? >> This way all the worries about ordering of writes into array and/or size are gone. The array is still used to quickly search for an element, but VM only scans the linked-list. >> >> What do you think of this? > I?m not sure. I know Coleen Ph would like to see that happen. > > A couple of people have vague plans to move more of the MemberName resolution into core libs. > (Years ago I worked on a VM where *all* of this occurred in Java, but some of it was ahead of time.) Hi David, > > I heard mention of ?we want to put more stuff in there? but I got the impression that already happened > (there?s reflection data, for example) so I?m not sure that makes sense. Reflection is an API that is rooted in j.l.Class. If the plans are to move some of the java.lang.invoke public API to java.lang package (into the j.l.Class, ...), then this is understandable. > > There?s also a proposal from people in the runtime to just use a jmethodid, take the hit of an extra indirection, > and no need to for this worrisome jvm/java concurrency. The linked list of MemberName(s) is also worry-less and doesn't need an extra indirection via jmethodid. Does the hit of extra indirection occur when invoking a MethodHandle? > > And if we instead wrote a hash table that only grew, and never relocated elements, we could > (I think) allow non-synchronized O(1) probes of the table from the Java side, synchronized > O(1) insertions from the Java side, and because nothing moves, a smaller dance with the > VM. I?m rather tempted to look into this ? given the amount of work it would take to do the > benchmarking to see if (a) jmethodid would have acceptable performance or (b) the existing > costs are too high, I could instead just write fast code and be done. Are you thinking of an IdentityHashMap type of hash table (no linked-list of elements for same bucket, just search for 1st free slot on insert)? The problem would be how to pre-size the array. Count declared members? > > And another way to view this is that we?re now quibbling about performance, when we still > have an existing correctness problem that this patch solves, so maybe we should just get this > done and then file an RFE. Perhaps, yes. But note that questions about JMM and ordering of writes to array elements are about correctness, not performance. Regards, Peter > > David > From david.r.chase at oracle.com Tue Nov 4 15:19:24 2014 From: david.r.chase at oracle.com (David Chase) Date: Tue, 4 Nov 2014 10:19:24 -0500 Subject: [9] RFR(L) 8013267 : move MemberNameTable from native code to Java heap, use to intern MemberNames In-Reply-To: <5458A57C.4060208@gmail.com> References: <594FE416-D445-426E-B7A9-C75F012ADE16@oracle.com> <0BDCD5DD-CBFB-4A3D-9BAD-7F9912E9237C@oracle.com> <5453C230.8010709@oracle.com> <9747A3A3-B7F4-407A-8E00-1E647D9DC2D1@oracle.com> <1D195ED3-8BD2-46CB-9D70-29CF809D9F5A@oracle.com> <5456FB59.60905@oracle.com> <632A5C98-B386-4625-BE12-355241581955@oracle.com> <5457AA75.8090103@gmail.com> <5457E0F9.8090004@gmail.com> <5458A57C.4060208@gmail.com> Message-ID: <260D49F5-6380-4FC3-A900-6CD9AB3ED6F7@oracle.com> On 2014-11-04, at 5:07 AM, Peter Levart wrote: > Are you thinking of an IdentityHashMap type of hash table (no linked-list of elements for same bucket, just search for 1st free slot on insert)? The problem would be how to pre-size the array. Count declared members? It can?t be an identityHashMap, because we are interning member names. In spite of my grumbling about benchmarking, I?m inclined to do that and try a couple of experiments. One possibility would be to use two data structures, one for interning, the other for communication with the VM. Because there?s no lookup in the VM data stucture it can just be an array that gets elements appended, and the synchronization dance is much simpler. For interning, maybe I use a ConcurrentHashMap, and I try the following idiom: mn = resolve(args) // deal with any errors mn? = chm.get(mn) if (mn? != null) return mn? // hoped-for-common-case synchronized (something) { mn? = chm.get(mn) if (mn? != null) return mn? txn_class = mn.getDeclaringClass() while (true) { redef_count = txn_class.redefCount() mn = resolve(args) shared_array.add(mn); // barrier, because we are a paranoid if (redef_count = redef_count.redefCount()) { chm.add(mn); // safe to publish to other Java threads. return mn; } shared_array.drop_last(); // Try again } } (Idiom gets slightly revised for the one or two other intern use cases, but this is the basic idea). David >> >> And another way to view this is that we?re now quibbling about performance, when we still >> have an existing correctness problem that this patch solves, so maybe we should just get this >> done and then file an RFE. > > Perhaps, yes. But note that questions about JMM and ordering of writes to array elements are about correctness, not performance. > > Regards, Peter > >> >> David From lance.andersen at oracle.com Tue Nov 4 15:39:09 2014 From: lance.andersen at oracle.com (Lance Andersen) Date: Tue, 4 Nov 2014 10:39:09 -0500 Subject: RFR (JAXP) 6770436 : Entity callback order differs between Java1.5 and Java1.6 In-Reply-To: <5457EF15.6040208@oracle.com> References: <5457EF15.6040208@oracle.com> Message-ID: <9C4931E1-7A9E-4FF6-BD55-6D21BD1DA415@oracle.com> Hi Joe, looks OK. Best Lance On Nov 3, 2014, at 4:09 PM, huizhe wang wrote: > Hi, > > The order of the Character event for a built-in entity reference was wrong after the integration of StAX in JDK 6. The error was introduced when all Character event was moved to the scanDocument method. The Character event in handleCharacter method was then commented out to avoid duplicate Character event. However, that change also changed the event sequence, startEntity - characters - endEntity became startEntity - endEntity - characters. > > The following fix adds a flag builtInRefCharacterHandled to indicate whether or not the Character event has been handled. If yes, an ENTITY_REFERENCE is returned, which does nothing. > > http://cr.openjdk.java.net/~joehw/jdk9/6770436/webrev/ > > Please review. > > Thanks, > Joe Lance Andersen| Principal Member of Technical Staff | +1.781.442.2037 Oracle Java Engineering 1 Network Drive Burlington, MA 01803 Lance.Andersen at oracle.com From peter.levart at gmail.com Tue Nov 4 16:48:14 2014 From: peter.levart at gmail.com (Peter Levart) Date: Tue, 04 Nov 2014 17:48:14 +0100 Subject: [9] RFR(L) 8013267 : move MemberNameTable from native code to Java heap, use to intern MemberNames In-Reply-To: <260D49F5-6380-4FC3-A900-6CD9AB3ED6F7@oracle.com> References: <594FE416-D445-426E-B7A9-C75F012ADE16@oracle.com> <0BDCD5DD-CBFB-4A3D-9BAD-7F9912E9237C@oracle.com> <5453C230.8010709@oracle.com> <9747A3A3-B7F4-407A-8E00-1E647D9DC2D1@oracle.com> <1D195ED3-8BD2-46CB-9D70-29CF809D9F5A@oracle.com> <5456FB59.60905@oracle.com> <632A5C98-B386-4625-BE12-355241581955@oracle.com> <5457AA75.8090103@gmail.com> <5457E0F9.8090004@gmail.com> <5458A57C.4060208@gmail.com> <260D49F5-6380-4FC3-A900-6CD9AB3ED6F7@oracle.com> Message-ID: <5459034E.8070809@gmail.com> On 11/04/2014 04:19 PM, David Chase wrote: > On 2014-11-04, at 5:07 AM, Peter Levart wrote: >> Are you thinking of an IdentityHashMap type of hash table (no linked-list of elements for same bucket, just search for 1st free slot on insert)? The problem would be how to pre-size the array. Count declared members? > It can?t be an identityHashMap, because we are interning member names. I know it can't be IdentityHashMap - I just wondered if you were thinking of an IdentityHashMap-like data structure in contrast to standard HashMap-like. Not in terms of equality/hashCode used, but in terms of internal data structure. IdentityHashMap is just an array of elements (well pairs of them - key, value are placed in two consecutive array slots). Lookup searches for element linearly in the array starting from hashCode based index to the element if found or 1st empty array slot. It's very easy to implement if the only operations are get() and put() and could be used for interning and as a shared structure for VM to scan, but array has to be sized to at least 3/2 the number of elements for performance to not degrade. > In spite of my grumbling about benchmarking, I?m inclined to do that and try a couple of experiments. > One possibility would be to use two data structures, one for interning, the other for communication with the VM. > Because there?s no lookup in the VM data stucture it can just be an array that gets elements appended, > and the synchronization dance is much simpler. > > For interning, maybe I use a ConcurrentHashMap, and I try the following idiom: > > mn = resolve(args) > // deal with any errors > mn? = chm.get(mn) > if (mn? != null) return mn? // hoped-for-common-case > > synchronized (something) { > mn? = chm.get(mn) > if (mn? != null) return mn? > > txn_class = mn.getDeclaringClass() > > while (true) { > redef_count = txn_class.redefCount() > mn = resolve(args) > > shared_array.add(mn); > // barrier, because we are a paranoid > if (redef_count = redef_count.redefCount()) { > chm.add(mn); // safe to publish to other Java threads. > return mn; > } > shared_array.drop_last(); // Try again > } > } > > (Idiom gets slightly revised for the one or two other intern use cases, but this is the basic idea). Yes, that's similar to what I suggested by using a linked-list of MemberName(s) instead of the "shared_array" (easier to reason about ordering of writes) and a sorted array of MemberName(s) instead of the "chm" in your scheme above. ConcurrentHashMap would certainly be the most performant solution in terms of lookup/insertion-time and concurrent throughput, but it will use more heap than a simple packed array of MemberNames. CHM is much better now in JDK8 though regarding heap use. A combination of the two approaches is also possible: - instead of maintaining a "shared_array" of MemberName(s), have them form a linked-list (you trade a slot in array for 'next' pointer in MemberName) - use ConcurrentHashMap for interning. Regards, Peter > > David > >>> And another way to view this is that we?re now quibbling about performance, when we still >>> have an existing correctness problem that this patch solves, so maybe we should just get this >>> done and then file an RFE. >> Perhaps, yes. But note that questions about JMM and ordering of writes to array elements are about correctness, not performance. >> >> Regards, Peter >> >>> David From daniel.fuchs at oracle.com Tue Nov 4 16:48:40 2014 From: daniel.fuchs at oracle.com (Daniel Fuchs) Date: Tue, 04 Nov 2014 17:48:40 +0100 Subject: RFR: 8060132: Handlers configured on abstract nodes in logging.properties are not always properly closed In-Reply-To: <544AC9B3.8090202@oracle.com> References: <5437F268.9020600@oracle.com> <5437FDCB.30803@oracle.com> <544A9B16.6030500@oracle.com> <544AC9B3.8090202@oracle.com> Message-ID: <54590368.5070309@oracle.com> Hi, Here is a revised patch that introduces a new .handlers.ensureCloseOnReset=true|false property. This property can be used as a fallback to turn the new behavior off in the improbable case where an application may need to do so. I have updated the LogManager class-level API documentation. http://cr.openjdk.java.net/~dfuchs/webrev_8060132/webrev.02/ best regards, -- daniel On 24/10/14 23:50, Daniel Fuchs wrote: > Hi Mandy, >> Is this only problem to the abstract node of Loggers (i.e. the ancestor >> is in the logging configuration but the library/app never creates such >> a logger)? > No - the problem is with any logger that has handlers configured from > the logging.properties, and which the application creates and releases > before the Cleaner thread is run. > >> Your patch will keep all loggers defined in the configuration strong >> reference. > > Not all loggers defined in the configuration. Only those loggers for > which a handler and is explicitly defined in the configuration and > which have been created at least once - directly or indirectly (= > through a child) by the application. > >> Would that cause memory leak? > > The main source of memory leak - and probably the only one we should > care about is when a Logger that we strong reference has a reference to > an object whose class would be otherwise garbage collected. > In other words - if the logger is the only thing that prevents an > object, and > its class, and its class loader, from being garbage collected. > > If I'm not mistaken - there are only three objects that could cause such > an issue: > - Instances of Level > - Instances of Handler > - Instances of ResourceBundle. > > Instances of Level are already held by Level.knownLevel - this is a > known bug, > and until that is fixed Logger won't be the only thing preventing a custom > instance of Level from being garbage collected. > In addition such a custom instance of Level (an instance of a custom > Level subclass) > can't be set directly from the configuration - so this can only happen > if the > application has set this level programmatically on the Logger. > > If I remember well Handlers created from the configuration are looked up > from the > System ClassLoader - so that shouldn't cause a memory leak either. > > Remains resource bundles - which might be an issue. > > However - the set of loggers for which a Handler is configured from the > configuration file is by nature bounded. Therefore - it is my opinion that > the proposed patch does not create a memory leak per-se: > A memory leak can only happen if an application constantly (and > programmatically) adds new Handler to the existing logger, believing > that it will get a new instance of Logger each time because the old > one should have been gc'ed. > In that case - I believe the fault would be in the application, not in the > Logger... > > If you think that we should still leave the door open for an > application to explicitly request the old behavior - we could > define a new property for the configuration file. > e.g. something like: > > .handlers = // no changes here > .handlers.ensureCloseOnReset = false // new property > // True by default when a logger has handlers configured from > logging.properties. > // Can be explicitly turned off for a given logger, in the > configuration. > // Ignored if the logger has no handler configured. > // Not inherited by child loggers > > If .handlers.ensureCloseOnReset is explicitly set to > false then the LogManager will not add the logger to the > persistentLoggers list. > > Do you think we should do that? > > best regards, > > -- daniel > > On 10/24/14 8:31 PM, Mandy Chung wrote: >> >> On 10/10/2014 8:39 AM, Daniel Fuchs wrote: >>> http://cr.openjdk.java.net/~dfuchs/webrev_8060132/webrev.01 >> >> Sorry for the delay. I have been pondering if there is a better >> alternative and looks like you have considered a few other options >> that none of them is a good one. >> >> typos: >> >> 174 explicitely >> 925: persitentLoggers >> >> Is this only problem to the abstract node of Loggers (i.e. the ancestor >> is in the logging configuration but the library/app never creates such >> a logger)? >> >> Your patch will keep all loggers defined in the configuration strong >> reference. What if the application really wants the loggers say >> com.foo.FooLogger to get GC'ed (e.g. it wants the logger class and >> its defining class loader to be garbage collected) but the configuration >> does name com.foo.FooLogger.handler = .... >> >> Would that cause memory leak? >> >> Mandy > From huizhe.wang at oracle.com Tue Nov 4 17:02:10 2014 From: huizhe.wang at oracle.com (huizhe wang) Date: Tue, 04 Nov 2014 09:02:10 -0800 Subject: RFR (JAXP) 6770436 : Entity callback order differs between Java1.5 and Java1.6 In-Reply-To: <9C4931E1-7A9E-4FF6-BD55-6D21BD1DA415@oracle.com> References: <5457EF15.6040208@oracle.com> <9C4931E1-7A9E-4FF6-BD55-6D21BD1DA415@oracle.com> Message-ID: <54590692.1070606@oracle.com> Thanks Lance! Joe On 11/4/2014 7:39 AM, Lance Andersen wrote: > Hi Joe, > > looks OK. > > Best > Lance > On Nov 3, 2014, at 4:09 PM, huizhe wang > wrote: > >> Hi, >> >> The order of the Character event for a built-in entity reference was >> wrong after the integration of StAX in JDK 6. The error was >> introduced when all Character event was moved to the scanDocument >> method. The Character event in handleCharacter method was then >> commented out to avoid duplicate Character event. However, that >> change also changed the event sequence, startEntity - characters - >> endEntity became startEntity - endEntity - characters. >> >> The following fix adds a flag builtInRefCharacterHandled to indicate >> whether or not the Character event has been handled. If yes, an >> ENTITY_REFERENCE is returned, which does nothing. >> >> http://cr.openjdk.java.net/~joehw/jdk9/6770436/webrev/ >> >> >> Please review. >> >> Thanks, >> Joe > > > > Lance > Andersen| Principal Member of Technical Staff | +1.781.442.2037 > Oracle Java Engineering > 1 Network Drive > Burlington, MA 01803 > Lance.Andersen at oracle.com > > > From martinrb at google.com Tue Nov 4 17:08:22 2014 From: martinrb at google.com (Martin Buchholz) Date: Tue, 4 Nov 2014 09:08:22 -0800 Subject: RFR: 8062773: Clarifications for Class specification In-Reply-To: <7746DAA6-2F3E-4891-B269-FC716D352B81@oracle.com> References: <7746DAA6-2F3E-4891-B269-FC716D352B81@oracle.com> Message-ID: [+core-libs-dev] I'm sorry I keep forgetting to add the mailing list to my review requests. I blame google's code review mail tool, which cc's the right mailing lists automagically. Webrev updated to add a few more "direct"s, as suggested by Paul. https://bugs.openjdk.java.net/browse/JDK-8062773 http://cr.openjdk.java.net/~martin/webrevs/openjdk9/Class-spec/ On Tue, Nov 4, 2014 at 3:02 AM, Paul Sandoz wrote: > > On Nov 3, 2014, at 11:35 PM, Martin Buchholz wrote: > > Hi Joe, Paul, > > I'd like you to do a code review. > > https://bugs.openjdk.java.net/browse/JDK-8062773 > http://cr.openjdk.java.net/~martin/webrevs/openjdk9/Class-spec/ > > > > Looks ok to me. > > 730 * @return the superclass of the class represented by this object. > > You could tweak that to say "the direct superclass..." > > Paul. From pavel.rappo at oracle.com Tue Nov 4 17:10:32 2014 From: pavel.rappo at oracle.com (Pavel Rappo) Date: Tue, 4 Nov 2014 17:10:32 +0000 Subject: RFR JDK-8062759: Update test/javax/naming/spi/providers/InitialContextTest.java to use classpath In-Reply-To: <1F65F028-5F02-4608-9221-EF10498CF8A7@oracle.com> References: <1F65F028-5F02-4608-9221-EF10498CF8A7@oracle.com> Message-ID: <06D95449-112A-4251-8FC8-E8666FF31DCC@oracle.com> Hi everyone, Could you please review my change for JDK-8062759? http://cr.openjdk.java.net/~prappo/8062759/webrev.00/ ---------------------------------------------------------------- P.S. The actual change is located at lines [215, 219]. Everything else is just a minor refactoring. P.P.S. I decided that existing check (InitialContextTest.verifyOutput) of the output for traces of "ServiceLoader" is good enough to verify that in the 'initialContextFactoryAsService' scenario JNDI provider is loaded by ServiceLoader rather than by VersionHelper.loadClass Thanks -Pavel From martinrb at google.com Tue Nov 4 17:53:40 2014 From: martinrb at google.com (Martin Buchholz) Date: Tue, 4 Nov 2014 09:53:40 -0800 Subject: RFR: 8062771: Core reflection should use final fields whenever possible In-Reply-To: References: Message-ID: [+core-libs-dev, Alexey] On Tue, Nov 4, 2014 at 2:13 AM, Paul Sandoz wrote: > Hi Martin, > > Should this and the other review also be posted on core-libs? (sorry for forgetting the mailing list) > On Nov 3, 2014, at 11:18 PM, Martin Buchholz wrote: > >> Hi Joe, Paul, >> >> I'd like you to do a code review. >> >> http://cr.openjdk.java.net/~martin/webrevs/openjdk9/reflection-final-fields/ >> https://bugs.openjdk.java.net/browse/JDK-8062771 > > Kind of scary. > > I suspect the fix JDK-8016236 also fixed this bug because the field GenericInfo.genericInfo was made volatile, which stamped in a barrier ensuring the publication was safe and the AbstractRepository.factory was not null. If my suspicions are correct it might be worth adding that analysis to the test (plus it might be easy to verify in your environment). I was misled by the synopsis of JDK-8016236 Class.getGenericInterfaces performance improvement because it's a correctness fix as well as a performance improvement. I recommend y'all do a little correctness backport to jdk7 Class.java as follows: @@ -2450,7 +2450,7 @@ private native String getGenericSignature(); // Generic info repository; lazily initialized - private transient ClassRepository genericInfo; + private volatile transient ClassRepository genericInfo; // accessor for factory private GenericsFactory getFactory() { @@ -2460,11 +2460,13 @@ // accessor for generic info repository private ClassRepository getGenericInfo() { + ClassRepository genericInfo = this.genericInfo; // lazily initialize repository if necessary if (genericInfo == null) { // create and cache generic info repository genericInfo = ClassRepository.make(getGenericSignature(), getFactory()); + this.genericInfo = genericInfo; } return genericInfo; //return cached repository } > Using final fields where at all possible is a good cleanup. Right. (even though this change fixes no known bug ... in openjdk9) > Is it possible to make EmptyClass be a static inner class of the test? IIUC if its enclosed within ThreadSafety it should still work with URLClassLoader. Done. Webrev regenerated. From stanimir at riflexo.com Tue Nov 4 17:54:47 2014 From: stanimir at riflexo.com (Stanimir Simeonoff) Date: Tue, 4 Nov 2014 19:54:47 +0200 Subject: RFR: 8060132: Handlers configured on abstract nodes in logging.properties are not always properly closed In-Reply-To: <54590368.5070309@oracle.com> References: <5437F268.9020600@oracle.com> <5437FDCB.30803@oracle.com> <544A9B16.6030500@oracle.com> <544AC9B3.8090202@oracle.com> <54590368.5070309@oracle.com> Message-ID: Hi, 917 final boolean ensureCloseOnReset = names.length == 0 ? false 918 : getBooleanProperty(handlersPropertyName + ".ensureCloseOnReset", 919 names.length > 0); I think the statement would be a lot easier to read as: final boolean ensureCloseOnReset = names.length > 0 && getBooleanProperty(handlersPropertyName+ ".ensureCloseOnReset", true) Also, probably adding a default for ensureCloseOnReset (via system property -Djava.util.logging.ensureCloseOnReset) would be less daunting to revert to the default behavior than setting it in the concrete properties files (that might be bundled a jar already). Stanimir On Tue, Nov 4, 2014 at 6:48 PM, Daniel Fuchs wrote: > Hi, > > Here is a revised patch that introduces a new > .handlers.ensureCloseOnReset=true|false > property. > > This property can be used as a fallback to turn the new behavior > off in the improbable case where an application may need to do > so. I have updated the LogManager class-level API documentation. > > http://cr.openjdk.java.net/~dfuchs/webrev_8060132/webrev.02/ > > best regards, > > -- daniel > > > On 24/10/14 23:50, Daniel Fuchs wrote: > >> Hi Mandy, >> >>> Is this only problem to the abstract node of Loggers (i.e. the ancestor >>> is in the logging configuration but the library/app never creates such >>> a logger)? >>> >> No - the problem is with any logger that has handlers configured from >> the logging.properties, and which the application creates and releases >> before the Cleaner thread is run. >> >> Your patch will keep all loggers defined in the configuration strong >>> reference. >>> >> >> Not all loggers defined in the configuration. Only those loggers for >> which a handler and is explicitly defined in the configuration and >> which have been created at least once - directly or indirectly (= >> through a child) by the application. >> >> Would that cause memory leak? >>> >> >> The main source of memory leak - and probably the only one we should >> care about is when a Logger that we strong reference has a reference to >> an object whose class would be otherwise garbage collected. >> In other words - if the logger is the only thing that prevents an >> object, and >> its class, and its class loader, from being garbage collected. >> >> If I'm not mistaken - there are only three objects that could cause such >> an issue: >> - Instances of Level >> - Instances of Handler >> - Instances of ResourceBundle. >> >> Instances of Level are already held by Level.knownLevel - this is a >> known bug, >> and until that is fixed Logger won't be the only thing preventing a custom >> instance of Level from being garbage collected. >> In addition such a custom instance of Level (an instance of a custom >> Level subclass) >> can't be set directly from the configuration - so this can only happen >> if the >> application has set this level programmatically on the Logger. >> >> If I remember well Handlers created from the configuration are looked up >> from the >> System ClassLoader - so that shouldn't cause a memory leak either. >> >> Remains resource bundles - which might be an issue. >> >> However - the set of loggers for which a Handler is configured from the >> configuration file is by nature bounded. Therefore - it is my opinion that >> the proposed patch does not create a memory leak per-se: >> A memory leak can only happen if an application constantly (and >> programmatically) adds new Handler to the existing logger, believing >> that it will get a new instance of Logger each time because the old >> one should have been gc'ed. >> In that case - I believe the fault would be in the application, not in the >> Logger... >> >> If you think that we should still leave the door open for an >> application to explicitly request the old behavior - we could >> define a new property for the configuration file. >> e.g. something like: >> >> .handlers = // no changes here >> .handlers.ensureCloseOnReset = false // new property >> // True by default when a logger has handlers configured from >> logging.properties. >> // Can be explicitly turned off for a given logger, in the >> configuration. >> // Ignored if the logger has no handler configured. >> // Not inherited by child loggers >> >> If .handlers.ensureCloseOnReset is explicitly set to >> false then the LogManager will not add the logger to the >> persistentLoggers list. >> >> Do you think we should do that? >> >> best regards, >> >> -- daniel >> >> On 10/24/14 8:31 PM, Mandy Chung wrote: >> >>> >>> On 10/10/2014 8:39 AM, Daniel Fuchs wrote: >>> >>>> http://cr.openjdk.java.net/~dfuchs/webrev_8060132/webrev.01 >>>> >>> >>> Sorry for the delay. I have been pondering if there is a better >>> alternative and looks like you have considered a few other options >>> that none of them is a good one. >>> >>> typos: >>> >>> 174 explicitely >>> 925: persitentLoggers >>> >>> Is this only problem to the abstract node of Loggers (i.e. the ancestor >>> is in the logging configuration but the library/app never creates such >>> a logger)? >>> >>> Your patch will keep all loggers defined in the configuration strong >>> reference. What if the application really wants the loggers say >>> com.foo.FooLogger to get GC'ed (e.g. it wants the logger class and >>> its defining class loader to be garbage collected) but the configuration >>> does name com.foo.FooLogger.handler = .... >>> >>> Would that cause memory leak? >>> >>> Mandy >>> >> >> > From daniel.fuchs at oracle.com Tue Nov 4 18:35:10 2014 From: daniel.fuchs at oracle.com (Daniel Fuchs) Date: Tue, 04 Nov 2014 19:35:10 +0100 Subject: RFR: 8060132: Handlers configured on abstract nodes in logging.properties are not always properly closed In-Reply-To: References: <5437F268.9020600@oracle.com> <5437FDCB.30803@oracle.com> <544A9B16.6030500@oracle.com> <544AC9B3.8090202@oracle.com> <54590368.5070309@oracle.com> Message-ID: <54591C5E.1010301@oracle.com> On 04/11/14 18:54, Stanimir Simeonoff wrote: > Hi, > > 917 final boolean ensureCloseOnReset = names.length == 0 ? false > 918 : getBooleanProperty(handlersPropertyName + ".ensureCloseOnReset", > 919 names.length > 0); > > I think the statement would be a lot easier to read as:final boolean > ensureCloseOnReset = names.length > 0 && > getBooleanProperty(handlersPropertyName+ ".ensureCloseOnReset", true) > > Also, probably adding a default for ensureCloseOnReset (via system > property -Djava.util.logging.ensureCloseOnReset) would be less daunting > to revert to the default behavior than setting it in the concrete > properties files (that might be bundled a jar already). Hi Stanimir, Good points. I was thinking of adding a default value for ensureCloseOnReset in the property file - rather than hardcoding it to 'true'. Something like: final boolean defaultValue = getBooleanProperty("java.util.logging.ensureCloseOnReset", true); final boolean ensureCloseOnReset = names.length > 0 && getBooleanProperty(handlersPropertyName + ".ensureCloseOnReset", defaultValue); So 'true' would instead be the default of the default... -- daniel > > Stanimir > > > On Tue, Nov 4, 2014 at 6:48 PM, Daniel Fuchs > wrote: > > Hi, > > Here is a revised patch that introduces a new > .handlers.__ensureCloseOnReset=true|false > property. > > This property can be used as a fallback to turn the new behavior > off in the improbable case where an application may need to do > so. I have updated the LogManager class-level API documentation. > > http://cr.openjdk.java.net/~__dfuchs/webrev_8060132/webrev.__02/ > > > best regards, > > -- daniel > > > On 24/10/14 23:50, Daniel Fuchs wrote: > > Hi Mandy, > > Is this only problem to the abstract node of Loggers (i.e. > the ancestor > is in the logging configuration but the library/app never > creates such > a logger)? > > No - the problem is with any logger that has handlers configured > from > the logging.properties, and which the application creates and > releases > before the Cleaner thread is run. > > Your patch will keep all loggers defined in the > configuration strong > reference. > > > Not all loggers defined in the configuration. Only those loggers for > which a handler and is explicitly defined in the configuration and > which have been created at least once - directly or indirectly (= > through a child) by the application. > > Would that cause memory leak? > > > The main source of memory leak - and probably the only one we should > care about is when a Logger that we strong reference has a > reference to > an object whose class would be otherwise garbage collected. > In other words - if the logger is the only thing that prevents an > object, and > its class, and its class loader, from being garbage collected. > > If I'm not mistaken - there are only three objects that could > cause such > an issue: > - Instances of Level > - Instances of Handler > - Instances of ResourceBundle. > > Instances of Level are already held by Level.knownLevel - this is a > known bug, > and until that is fixed Logger won't be the only thing > preventing a custom > instance of Level from being garbage collected. > In addition such a custom instance of Level (an instance of a custom > Level subclass) > can't be set directly from the configuration - so this can only > happen > if the > application has set this level programmatically on the Logger. > > If I remember well Handlers created from the configuration are > looked up > from the > System ClassLoader - so that shouldn't cause a memory leak either. > > Remains resource bundles - which might be an issue. > > However - the set of loggers for which a Handler is configured > from the > configuration file is by nature bounded. Therefore - it is my > opinion that > the proposed patch does not create a memory leak per-se: > A memory leak can only happen if an application constantly (and > programmatically) adds new Handler to the existing logger, believing > that it will get a new instance of Logger each time because the old > one should have been gc'ed. > In that case - I believe the fault would be in the application, > not in the > Logger... > > If you think that we should still leave the door open for an > application to explicitly request the old behavior - we could > define a new property for the configuration file. > e.g. something like: > > .handlers = // no changes here > .handlers.__ensureCloseOnReset = false // new property > // True by default when a logger has handlers configured from > logging.properties. > // Can be explicitly turned off for a given logger, in the > configuration. > // Ignored if the logger has no handler configured. > // Not inherited by child loggers > > If .handlers.__ensureCloseOnReset is explicitly set to > false then the LogManager will not add the logger to the > persistentLoggers list. > > Do you think we should do that? > > best regards, > > -- daniel > > On 10/24/14 8:31 PM, Mandy Chung wrote: > > > On 10/10/2014 8:39 AM, Daniel Fuchs wrote: > > http://cr.openjdk.java.net/~__dfuchs/webrev_8060132/webrev.__01 > > > > Sorry for the delay. I have been pondering if there is a better > alternative and looks like you have considered a few other > options > that none of them is a good one. > > typos: > > 174 explicitely > 925: persitentLoggers > > Is this only problem to the abstract node of Loggers (i.e. > the ancestor > is in the logging configuration but the library/app never > creates such > a logger)? > > Your patch will keep all loggers defined in the > configuration strong > reference. What if the application really wants the loggers say > com.foo.FooLogger to get GC'ed (e.g. it wants the logger > class and > its defining class loader to be garbage collected) but the > configuration > does name com.foo.FooLogger.handler = .... > > Would that cause memory leak? > > Mandy > > > > From brent.christian at oracle.com Tue Nov 4 19:04:12 2014 From: brent.christian at oracle.com (Brent Christian) Date: Tue, 04 Nov 2014 11:04:12 -0800 Subject: Please Review Draft JEP: More memory-efficient internal representation for Strings Message-ID: <5459232C.6090003@oracle.com> Hi, Please review this draft JEP for: "More memory-efficient internal representation for Strings" https://bugs.openjdk.java.net/browse/JDK-8054307 Thanks, Brent From guy.steele at oracle.com Tue Nov 4 19:21:20 2014 From: guy.steele at oracle.com (Guy Steele) Date: Tue, 4 Nov 2014 14:21:20 -0500 Subject: Please Review Draft JEP: More memory-efficient internal representation for Strings In-Reply-To: <5459232C.6090003@oracle.com> References: <5459232C.6090003@oracle.com> Message-ID: As far as it goes, it looks good to me. I do note that so far it has no mention of the academic literature that has already explored this or related problems, and I hope that part of the activity would be to survey and summarize this literature before making implementation decisions. ?Guy On Nov 4, 2014, at 2:04 PM, Brent Christian wrote: > Hi, > > Please review this draft JEP for:As far as it goes, it loos > > "More memory-efficient internal representation for Strings" > https://bugs.openjdk.java.net/browse/JDK-8054307 > > Thanks, > Brent From aleksey.shipilev at oracle.com Tue Nov 4 19:55:34 2014 From: aleksey.shipilev at oracle.com (Aleksey Shipilev) Date: Tue, 04 Nov 2014 22:55:34 +0300 Subject: RFR: 8062771: Core reflection should use final fields whenever possible In-Reply-To: References: Message-ID: <54592F36.7050603@oracle.com> On 04.11.2014 20:53, Martin Buchholz wrote: > On Tue, Nov 4, 2014 at 2:13 AM, Paul Sandoz wrote: >> On Nov 3, 2014, at 11:18 PM, Martin Buchholz wrote: >>> http://cr.openjdk.java.net/~martin/webrevs/openjdk9/reflection-final-fields/ >>> https://bugs.openjdk.java.net/browse/JDK-8062771 Looks good. I always like more finals. >> Kind of scary. >> >> I suspect the fix JDK-8016236 also fixed this bug because the field GenericInfo.genericInfo was made volatile, which stamped in a barrier ensuring the publication was safe and the AbstractRepository.factory was not null. If my suspicions are correct it might be worth adding that analysis to the test (plus it might be easy to verify in your environment). > > I was misled by the synopsis of > JDK-8016236 Class.getGenericInterfaces performance improvement > because it's a correctness fix as well as a performance improvement. Ouch. I remember reviewing that change, but it never occurred to me the original code was broken. > I recommend y'all do a little correctness backport to jdk7 Class.java > as follows: Yes. Although I am not sure how we push the changes to jdk7 these days. > @@ -2450,7 +2450,7 @@ > private native String getGenericSignature(); If you do this: > // Generic info repository; lazily initialized > - private transient ClassRepository genericInfo; > + private volatile transient ClassRepository genericInfo; ...you don't need to do this for correctness (but may do for performance -- avoidable volatile read for ARM/PPC): > // accessor for generic info repository > private ClassRepository getGenericInfo() { > + ClassRepository genericInfo = this.genericInfo; > // lazily initialize repository if necessary > if (genericInfo == null) { > // create and cache generic info repository > genericInfo = ClassRepository.make(getGenericSignature(), > getFactory()); > + this.genericInfo = genericInfo; > } > return genericInfo; //return cached repository > } > Also, I wonder if we need a singleton ClassRepository instance, or we can tolerate two different ClassRepository-ies returned under concurrent init. Thanks, -Aleksey. From hendrikdev22 at gmail.com Tue Nov 4 20:10:35 2014 From: hendrikdev22 at gmail.com (Hendrik Dev) Date: Tue, 4 Nov 2014 21:10:35 +0100 Subject: sun.nio.cs.StreamDecoder readBytes() possible issue Message-ID: Javadoc for java.io.InputStream.read(byte[] b, int off , int len) says: "Reads up to len bytes of data from the input stream into an array of bytes. An attempt is made to read as many as len bytes, but a smaller number may be read. The number of bytes actually read is returned as an integer." So maybe the stream can return 0 but there is still data (if not data then -1 is assumed)? If so then: sun.nio.cs.StreamDecoder readBytes() does thow an exception if zero bytes are read instead of looping (https://github.com/openjdk-mirror/jdk7u-jdk/blob/master/src/share/classes/sun/nio/cs/StreamDecoder.java#L287) So it should maybe look more like this: assert rem > 0; int n = 0; do { n = in.read(bb.array(), bb.arrayOffset() + pos, rem); } while (n == 0); if (n < 0) return n; assert (n <= rem) : "n = " + n + ", rem = " + rem; Am i wrong? Thanks Hendrik -- Hendrik Saly (salyh, hendrikdev22) @hendrikdev22 PGP: 0x22D7F6EC From martinrb at google.com Tue Nov 4 20:11:03 2014 From: martinrb at google.com (Martin Buchholz) Date: Tue, 4 Nov 2014 12:11:03 -0800 Subject: RFR: 8062771: Core reflection should use final fields whenever possible In-Reply-To: <54592F36.7050603@oracle.com> References: <54592F36.7050603@oracle.com> Message-ID: Hi Aleksey, On Tue, Nov 4, 2014 at 11:55 AM, Aleksey Shipilev wrote: > On 04.11.2014 20:53, Martin Buchholz wrote: >> On Tue, Nov 4, 2014 at 2:13 AM, Paul Sandoz wrote: >>> On Nov 3, 2014, at 11:18 PM, Martin Buchholz wrote: > If you do this: > >> // Generic info repository; lazily initialized >> - private transient ClassRepository genericInfo; >> + private volatile transient ClassRepository genericInfo; > > ...you don't need to do this for correctness (but may do for performance > -- avoidable volatile read for ARM/PPC): It's just reflex to always read volatiles into locals - never hurts, except the line-of-code count. >> // accessor for generic info repository >> private ClassRepository getGenericInfo() { >> + ClassRepository genericInfo = this.genericInfo; >> // lazily initialize repository if necessary >> if (genericInfo == null) { >> // create and cache generic info repository >> genericInfo = ClassRepository.make(getGenericSignature(), >> getFactory()); >> + this.genericInfo = genericInfo; >> } >> return genericInfo; //return cached repository >> } >> > > Also, I wonder if we need a singleton ClassRepository instance, or we > can tolerate two different ClassRepository-ies returned under concurrent > init. Yeah, me too. But the existing code has this behavior as well. One could fix it if needed with double-checked-locking or whatever our fancy new volatile unsafe-less cas ends up looking like. From vitalyd at gmail.com Tue Nov 4 20:17:28 2014 From: vitalyd at gmail.com (Vitaly Davidovich) Date: Tue, 4 Nov 2014 15:17:28 -0500 Subject: sun.nio.cs.StreamDecoder readBytes() possible issue In-Reply-To: References: Message-ID: There's more detail in the InputStream.read javadoc, specifically (emphasis mine): This method blocks until input data is available, end of file is detected, > or an exception is thrown. > > If len is zero, then no bytes are read and 0 is returned; *otherwise, > there is an attempt to read at least one byte. If no byte is available > because the stream is at end of file, the value -1 is returned; otherwise, > at least one byte is read and stored into b. * On Tue, Nov 4, 2014 at 3:10 PM, Hendrik Dev wrote: > Javadoc for java.io.InputStream.read(byte[] b, int off , int len) says: > "Reads up to len bytes of data from the input stream into an array of > bytes. An attempt is made to read as many as len bytes, but a smaller > number may be read. The number of bytes actually read is returned as > an integer." > > So maybe the stream can return 0 but there is still data (if not data > then -1 is assumed)? > If so then: > > sun.nio.cs.StreamDecoder readBytes() does thow an exception if zero > bytes are read instead of looping > ( > https://github.com/openjdk-mirror/jdk7u-jdk/blob/master/src/share/classes/sun/nio/cs/StreamDecoder.java#L287 > ) > > So it should maybe look more like this: > > assert rem > 0; > int n = 0; > do { > n = in.read(bb.array(), bb.arrayOffset() + pos, rem); > } while (n == 0); > if (n < 0) > return n; > assert (n <= rem) : "n = " + n + ", rem = " + rem; > > Am i wrong? > > Thanks > Hendrik > > -- > Hendrik Saly (salyh, hendrikdev22) > @hendrikdev22 > PGP: 0x22D7F6EC > From aleksey.shipilev at oracle.com Tue Nov 4 20:34:22 2014 From: aleksey.shipilev at oracle.com (Aleksey Shipilev) Date: Tue, 04 Nov 2014 23:34:22 +0300 Subject: RFR: 8062771: Core reflection should use final fields whenever possible In-Reply-To: References: <54592F36.7050603@oracle.com> Message-ID: <5459384E.8020308@oracle.com> Hi Martin, On 04.11.2014 23:11, Martin Buchholz wrote: > On Tue, Nov 4, 2014 at 11:55 AM, Aleksey Shipilev > wrote: >> On 04.11.2014 20:53, Martin Buchholz wrote: >>> On Tue, Nov 4, 2014 at 2:13 AM, Paul Sandoz wrote: >>>> On Nov 3, 2014, at 11:18 PM, Martin Buchholz wrote: >> If you do this: >> >>> // Generic info repository; lazily initialized >>> - private transient ClassRepository genericInfo; >>> + private volatile transient ClassRepository genericInfo; >> >> ...you don't need to do this for correctness (but may do for performance >> -- avoidable volatile read for ARM/PPC): > > It's just reflex to always read volatiles into locals - never hurts, > except the line-of-code count. No objection here :) Current JDK 9 code also does this. I think we can make a partial backport of JDK-8016236 to jdk7, since this volatile construction fixes the obvious bug, and the fix was tested already in JDK 8 and 9. Can anyone from Core Libraries team attend to this? >> Also, I wonder if we need a singleton ClassRepository instance, or we >> can tolerate two different ClassRepository-ies returned under concurrent >> init. > > Yeah, me too. But the existing code has this behavior as well. One > could fix it if needed with double-checked-locking or whatever our > fancy new volatile unsafe-less cas ends up looking like. All right, ClassRepository seems to be an innocuous idempotent cache. Do not need double-checked locking here. -Aleksey. From david.r.chase at oracle.com Tue Nov 4 20:54:03 2014 From: david.r.chase at oracle.com (David Chase) Date: Tue, 4 Nov 2014 15:54:03 -0500 Subject: [9] RFR(L) 8013267 : move MemberNameTable from native code to Java heap, use to intern MemberNames In-Reply-To: <5459034E.8070809@gmail.com> References: <594FE416-D445-426E-B7A9-C75F012ADE16@oracle.com> <0BDCD5DD-CBFB-4A3D-9BAD-7F9912E9237C@oracle.com> <5453C230.8010709@oracle.com> <9747A3A3-B7F4-407A-8E00-1E647D9DC2D1@oracle.com> <1D195ED3-8BD2-46CB-9D70-29CF809D9F5A@oracle.com> <5456FB59.60905@oracle.com> <632A5C98-B386-4625-BE12-355241581955@oracle.com> <5457AA75.8090103@gmail.com> <5457E0F9.8090004@gmail.com> <5458A57C.4060208@gmail.com> <260D49F5-6380-4FC3-A900-6CD9AB3ED6F7@oracle.com> <5459034E.8070809@gmail.com> Message-ID: I?m working on the initial benchmarking, and so far this arrangement (with synchronization and binary search for lookup, lots of barriers and linear cost insertion) has not yet been any slower. I am nonetheless tempted by the 2-tables solution, because I think the simpler JVM-side interface that it allows is desirable. David On 2014-11-04, at 11:48 AM, Peter Levart wrote: > On 11/04/2014 04:19 PM, David Chase wrote: >> On 2014-11-04, at 5:07 AM, Peter Levart wrote: >>> Are you thinking of an IdentityHashMap type of hash table (no linked-list of elements for same bucket, just search for 1st free slot on insert)? The problem would be how to pre-size the array. Count declared members? >> It can?t be an identityHashMap, because we are interning member names. > > I know it can't be IdentityHashMap - I just wondered if you were thinking of an IdentityHashMap-like data structure in contrast to standard HashMap-like. Not in terms of equality/hashCode used, but in terms of internal data structure. IdentityHashMap is just an array of elements (well pairs of them - key, value are placed in two consecutive array slots). Lookup searches for element linearly in the array starting from hashCode based index to the element if found or 1st empty array slot. It's very easy to implement if the only operations are get() and put() and could be used for interning and as a shared structure for VM to scan, but array has to be sized to at least 3/2 the number of elements for performance to not degrade. > >> In spite of my grumbling about benchmarking, I?m inclined to do that and try a couple of experiments. >> One possibility would be to use two data structures, one for interning, the other for communication with the VM. >> Because there?s no lookup in the VM data stucture it can just be an array that gets elements appended, >> and the synchronization dance is much simpler. >> >> For interning, maybe I use a ConcurrentHashMap, and I try the following idiom: >> >> mn = resolve(args) >> // deal with any errors >> mn? = chm.get(mn) >> if (mn? != null) return mn? // hoped-for-common-case >> >> synchronized (something) { >> mn? = chm.get(mn) >> if (mn? != null) return mn? >> txn_class = mn.getDeclaringClass() >> >> while (true) { >> redef_count = txn_class.redefCount() >> mn = resolve(args) >> >> shared_array.add(mn); >> // barrier, because we are a paranoid >> if (redef_count = redef_count.redefCount()) { >> chm.add(mn); // safe to publish to other Java threads. >> return mn; >> } >> shared_array.drop_last(); // Try again >> } >> } >> >> (Idiom gets slightly revised for the one or two other intern use cases, but this is the basic idea). > > Yes, that's similar to what I suggested by using a linked-list of MemberName(s) instead of the "shared_array" (easier to reason about ordering of writes) and a sorted array of MemberName(s) instead of the "chm" in your scheme above. ConcurrentHashMap would certainly be the most performant solution in terms of lookup/insertion-time and concurrent throughput, but it will use more heap than a simple packed array of MemberNames. CHM is much better now in JDK8 though regarding heap use. > > A combination of the two approaches is also possible: > > - instead of maintaining a "shared_array" of MemberName(s), have them form a linked-list (you trade a slot in array for 'next' pointer in MemberName) > - use ConcurrentHashMap for interning. > > Regards, Peter > >> >> David >> >>>> And another way to view this is that we?re now quibbling about performance, when we still >>>> have an existing correctness problem that this patch solves, so maybe we should just get this >>>> done and then file an RFE. >>> Perhaps, yes. But note that questions about JMM and ordering of writes to array elements are about correctness, not performance. >>> >>> Regards, Peter >>> >>>> David From claes.redestad at oracle.com Tue Nov 4 20:57:26 2014 From: claes.redestad at oracle.com (Claes Redestad) Date: Tue, 04 Nov 2014 21:57:26 +0100 Subject: RFR 8062180: MethodHandleImpl.makeArrays throws and swallows java.lang.NoSuchFieldError in normal flow Message-ID: <54593DB6.1040305@oracle.com> Hi, please review this minor improvement to the initialization of MethodHandleImpl$Lazy. Running any code which initializes MethodHandlesImpl$Lazy will trigger throwing of java.lang.NoSuchFieldErrors, which has a measurable impact on startup. Additionally, the current lookup via findCollector in varargsArray is pointless (and will throw additional java.lang.NoSuchFieldErrors the first time a varargsArray of arity > 11 is requested) since all static methods which can be found will have been constructed already. bug: https://bugs.openjdk.java.net/browse/JDK-8062180 webrev: http://cr.openjdk.java.net/~redestad/8062180/webrev.00/ A simple microbenchmark[1] show a very small (~0.5ms) improvement for Class.forName("java.lang.invoke.MethodHandleImpl$Lazy") Benchmark Mode Samples Score Score error Units o.s.CLBench.loadMethodHandleImplLazy ss 100 46.364 1.740 ms o.s.CLBench.loadMethodHandleImplLazy ss 100 45.440 1.515 ms Thanks! /Claes [1] package org.sample; import org.openjdk.jmh.annotations.*; import javax.script.*; @State(Scope.Thread) public class CLBench { @Benchmark public Object loadMethodHandleImplLazy() throws Exception { return Class.forName("java.lang.invoke.MethodHandleImpl$Lazy"); } } From aleksey.shipilev at oracle.com Tue Nov 4 21:25:51 2014 From: aleksey.shipilev at oracle.com (Aleksey Shipilev) Date: Wed, 05 Nov 2014 00:25:51 +0300 Subject: RFR 8062180: MethodHandleImpl.makeArrays throws and swallows java.lang.NoSuchFieldError in normal flow In-Reply-To: <54593DB6.1040305@oracle.com> References: <54593DB6.1040305@oracle.com> Message-ID: <5459445F.8060405@oracle.com> Hi Claes, On 04.11.2014 23:57, Claes Redestad wrote: > please review this minor improvement to the initialization of > MethodHandleImpl$Lazy. Running any code which initializes > MethodHandlesImpl$Lazy will trigger throwing of > java.lang.NoSuchFieldErrors, which has a measurable impact on startup. Do you actually see the impact on startup? I thought the entire MHI$Lazy thing is here to avoid instantiation on startup. If that is not true, can you do a proper startup measurement on HelloWorld application? Here is a handy one-liner: for S in `seq 1 1000`; do time java Hello; done 2>&1 | grep real | sed -e "s/0m//g" -e "s/s//g" | awk '{ sum += $2; n++ } END { if (n > 0) print sum / n; }' > Additionally, the current lookup via findCollector in varargsArray is > pointless (and will throw additional java.lang.NoSuchFieldErrors the > first time a varargsArray of arity > 11 is requested) since all static > methods which can be found will have been constructed already. > > bug: https://bugs.openjdk.java.net/browse/JDK-8062180 > webrev: http://cr.openjdk.java.net/~redestad/8062180/webrev.00/ Seems rather minor, but every little thing helps startup. - Would we be better off asserting (mh == null) instead of breaking and asserting afterwards? - Would we save a few more bytecodes by using the method-local "final int ARRAYS_COUNT" to handle the magic number, instead of doing a full-blown field? Ditto for FILL_ARRAYS_COUNT. > A simple microbenchmark[1] show a very small (~0.5ms) improvement for > Class.forName("java.lang.invoke.MethodHandleImpl$Lazy") > > Benchmark Mode Samples Score Score > error Units > o.s.CLBench.loadMethodHandleImplLazy ss 100 46.364 > 1.740 ms > o.s.CLBench.loadMethodHandleImplLazy ss 100 45.440 > 1.515 ms Strictly speaking, there is no improvement. Run with more samples to shrink the error. -Aleksey. From mandy.chung at oracle.com Tue Nov 4 21:51:06 2014 From: mandy.chung at oracle.com (Mandy Chung) Date: Tue, 04 Nov 2014 13:51:06 -0800 Subject: RFR JDK-8062759: Update test/javax/naming/spi/providers/InitialContextTest.java to use classpath In-Reply-To: <06D95449-112A-4251-8FC8-E8666FF31DCC@oracle.com> References: <1F65F028-5F02-4608-9221-EF10498CF8A7@oracle.com> <06D95449-112A-4251-8FC8-E8666FF31DCC@oracle.com> Message-ID: <54594A4A.3010406@oracle.com> Hi Pavel, On 11/4/14 9:10 AM, Pavel Rappo wrote: > Hi everyone, > > Could you please review my change for JDK-8062759? > > http://cr.openjdk.java.net/~prappo/8062759/webrev.00/ This looks okay to me. Mandy From mandy.chung at oracle.com Tue Nov 4 22:40:07 2014 From: mandy.chung at oracle.com (Mandy Chung) Date: Tue, 04 Nov 2014 14:40:07 -0800 Subject: RFR: 8060132: Handlers configured on abstract nodes in logging.properties are not always properly closed In-Reply-To: <54590368.5070309@oracle.com> References: <5437F268.9020600@oracle.com> <5437FDCB.30803@oracle.com> <544A9B16.6030500@oracle.com> <544AC9B3.8090202@oracle.com> <54590368.5070309@oracle.com> Message-ID: <545955C7.4010909@oracle.com> On 11/4/14 8:48 AM, Daniel Fuchs wrote: > Hi, > > Here is a revised patch that introduces a new > .handlers.ensureCloseOnReset=true|false > property. > You have explored several different options and as you have found, none of them is ideal. I think having a property to specify ensureCloseOnReset would be reasonable to address the memory leak concern I have. Would it be better to name it .handlers.closeOnReset? I am less sure if we need to have a global switch to revert the default - do we expect the number of .handlers is large? I would expect there would be small number. > This property can be used as a fallback to turn the new behavior > off in the improbable case where an application may need to do > so. I have updated the LogManager class-level API documentation. > > http://cr.openjdk.java.net/~dfuchs/webrev_8060132/webrev.02/ > line 104-114: s/"true"/{@code true} {@code reset} should be {@link ...} to the reset method (the first occurrance) It doesn't seem necessary to specify "strongly referenced". Maybe it's good to say "it will ensure the handlers associated with the loggers will be closed at shutdown or at reset. If "closeOnReset is false", application should ensure the handlers be closed before the logger gets garbage collected." - something along that line. typos: 186: explicitely -> explicitly Perhaps PersistentLogger can be renamed to CloseOnResetLogger to make it clear what these loggers are kept for. Similarly for the variable name "persistentLogger" may be renamed too. 917: final might be adding noise 940: persitentLoggers line 917: you don't need (names.length == 0) to determine the default, right? if names.length == 0, it wouldn't add to the persistentLoggers list as no iteration in the for loop. Mandy From martinrb at google.com Wed Nov 5 00:06:44 2014 From: martinrb at google.com (Martin Buchholz) Date: Tue, 4 Nov 2014 16:06:44 -0800 Subject: RFR: 8062194: java.util.jar.Attributes should use insertion-ordered iteration In-Reply-To: <544FF3ED.2020901@oracle.com> References: <544E96AC.90904@oracle.com> <544FF3ED.2020901@oracle.com> Message-ID: On Tue, Oct 28, 2014 at 12:52 PM, Alan Bateman wrote: > On 28/10/2014 18:09, Martin Buchholz wrote: >> >> Xueming, I understand that getting CCC approval is a fair amount of >> work. At your option, we could leave the spec unchanged and do >> without CCC, given that Attributes' iteration order has changed in >> every past release. Or we could split the spec change off as a >> separate improvement. >> > If we eventually want to specify that the iteration order is predictable > then it's probably best to do it all together. > > The additional class description looks okay. One comment on the constructor > that takes the Attributes parameter is that the newly created Attributes > might have a different order to the Attributes that it was created with. > This would only arise if Attributes is sub-classes by something that uses a > different iteration order to its super. So ... there is the question whether we are tightening the spec for Attributes and its subclasses or only for Attributes. Subclasses of Attribute that would change the order are likely to be very rare (is there a use case?) - I'm OK with those becoming non-spec-compliant subclasses, especially given that they will continue to work just fine in practice. So in my proposal the tightening applies to subclasses as well. From joe.darcy at oracle.com Wed Nov 5 05:26:48 2014 From: joe.darcy at oracle.com (joe darcy) Date: Tue, 04 Nov 2014 21:26:48 -0800 Subject: RFR: 8062773: Clarifications for Class specification In-Reply-To: References: <7746DAA6-2F3E-4891-B269-FC716D352B81@oracle.com> Message-ID: <5459B518.1080206@oracle.com> Hi Martin, This description in getGenericInterfaces 876 *

If this object represents a class, the return value is an 877 * array containing objects representing all interfaces 878 * implemented by the class. The order of the interface objects in could also be direct-ized. In getGenericInterfaces, the new text 899 *

If this {@code Class} object represents an array type, the 900 * interfaces {@code Cloneable} and {@code java.io.Serializable} are 901 * returned in that order. is redundant with the existing text 881 * represented by this object. In the case of an array class, the 882 * interfaces {@code Cloneable} and {@code Serializable} are 883 * returned in that order. Exactly one copy of the Cloneable & Serializable assertion should be removed :-) With this change to getFields, 1540 * returns the public fields of the class and of all its superclasses and 1541 * superinterfaces. the bug will need a ccc request since the specification is changing (even if it is changing to match long-standing behavior). Thanks, -Joe On 11/4/2014 9:08 AM, Martin Buchholz wrote: > [+core-libs-dev] > > I'm sorry I keep forgetting to add the mailing list to my review > requests. I blame google's code review mail tool, which cc's the > right mailing lists automagically. > > Webrev updated to add a few more "direct"s, as suggested by Paul. > > https://bugs.openjdk.java.net/browse/JDK-8062773 > http://cr.openjdk.java.net/~martin/webrevs/openjdk9/Class-spec/ > > On Tue, Nov 4, 2014 at 3:02 AM, Paul Sandoz wrote: >> On Nov 3, 2014, at 11:35 PM, Martin Buchholz wrote: >> >> Hi Joe, Paul, >> >> I'd like you to do a code review. >> >> https://bugs.openjdk.java.net/browse/JDK-8062773 >> http://cr.openjdk.java.net/~martin/webrevs/openjdk9/Class-spec/ >> >> >> >> Looks ok to me. >> >> 730 * @return the superclass of the class represented by this object. >> >> You could tweak that to say "the direct superclass..." >> >> Paul. From paul.sandoz at oracle.com Wed Nov 5 09:06:06 2014 From: paul.sandoz at oracle.com (Paul Sandoz) Date: Wed, 5 Nov 2014 10:06:06 +0100 Subject: RFR: 8062771: Core reflection should use final fields whenever possible In-Reply-To: References: Message-ID: <843282E1-DC2B-48B0-B0E4-A60B5078D75A@oracle.com> On Nov 4, 2014, at 6:53 PM, Martin Buchholz wrote: >> Is it possible to make EmptyClass be a static inner class of the test? IIUC if its enclosed within ThreadSafety it should still work with URLClassLoader. > > Done. Webrev regenerated. +1 Perhaps one reason why the test is not failing (on 7) is because certain code is not getting compiled/inlined, since there are not enough iterations. If you think it worth obsessing over you could shuffle the code a bit, increase the iterations of the inner loop, and use some hotspot compiler command options to force inline and/or compile. > I recommend y'all do a little correctness backport to jdk7 Class.java > as follows: > > @@ -2450,7 +2450,7 @@ > private native String getGenericSignature(); > > // Generic info repository; lazily initialized > - private transient ClassRepository genericInfo; > + private volatile transient ClassRepository genericInfo; > > // accessor for factory > private GenericsFactory getFactory() { > @@ -2460,11 +2460,13 @@ > > // accessor for generic info repository > private ClassRepository getGenericInfo() { > + ClassRepository genericInfo = this.genericInfo; > // lazily initialize repository if necessary > if (genericInfo == null) { > // create and cache generic info repository > genericInfo = ClassRepository.make(getGenericSignature(), > getFactory()); > + this.genericInfo = genericInfo; > } > return genericInfo; //return cached repository > } > Agreed. On Nov 4, 2014, at 9:34 PM, Aleksey Shipilev wrote: > I think we can make a partial backport of JDK-8016236 to jdk7, since > this volatile construction fixes the obvious bug, and the fix was tested > already in JDK 8 and 9. Can anyone from Core Libraries team attend to this? I am knuckling down into conference development mode this and next week for Devoxx, if no one jumps on it sooner i can sort it out later on. Paul. From paul.sandoz at oracle.com Wed Nov 5 09:09:42 2014 From: paul.sandoz at oracle.com (Paul Sandoz) Date: Wed, 5 Nov 2014 10:09:42 +0100 Subject: RFR: 8062773: Clarifications for Class specification In-Reply-To: <5459B518.1080206@oracle.com> References: <7746DAA6-2F3E-4891-B269-FC716D352B81@oracle.com> <5459B518.1080206@oracle.com> Message-ID: <4CC69B8A-3E6D-40B9-8E3E-D49EE1961471@oracle.com> On Nov 5, 2014, at 6:26 AM, joe darcy wrote: > Hi Martin, > > This description in getGenericInterfaces > > 876 *

If this object represents a class, the return value is an > 877 * array containing objects representing all interfaces > 878 * implemented by the class. The order of the interface objects in > > could also be direct-ized. > > In getGenericInterfaces, the new text > > 899 *

If this {@code Class} object represents an array type, the > 900 * interfaces {@code Cloneable} and {@code java.io.Serializable} are > 901 * returned in that order. > > is redundant with the existing text > > 881 * represented by this object. In the case of an array class, the > 882 * interfaces {@code Cloneable} and {@code Serializable} are > 883 * returned in that order. > > Exactly one copy of the Cloneable & Serializable assertion should be removed :-) > > With this change to getFields, > > 1540 * returns the public fields of the class and of all its superclasses and > 1541 * superinterfaces. > > the bug will need a ccc request since the specification is changing (even if it is changing to match long-standing behavior). > Ah, i did not realize that. Perhaps separate into a new issue? I can log the CC request. Paul. From frank.yuan at oracle.com Wed Nov 5 09:12:13 2014 From: frank.yuan at oracle.com (Frank Yuan) Date: Wed, 5 Nov 2014 17:12:13 +0800 Subject: Review request for XML JAXP unit test colocation Message-ID: <013601cff8d8$983b4800$c8b1d800$@oracle.com> Hi, Joe and All Thanks for your previous reviews and comments. Per your comments: I added description for every test, got them running with and without security manager and fixed the known test issues. Could you help review the changes for JAXP unittest co-location again? After your review, I will push the tests into JDK repo: jaxp/test. bug: https://bugs.openjdk.java.net/browse/JDK-8043090 webrev: http://sqeweb.us.oracle.com/jsn/users/yc/webrev/ test results: http://sqeweb.us.oracle.com/jsn/users/yc/result/JTreport/html/index.html Thanks, Frank From claes.redestad at oracle.com Wed Nov 5 09:23:26 2014 From: claes.redestad at oracle.com (Claes Redestad) Date: Wed, 05 Nov 2014 10:23:26 +0100 Subject: RFR 8062180: MethodHandleImpl.makeArrays throws and swallows java.lang.NoSuchFieldError in normal flow In-Reply-To: <5459445F.8060405@oracle.com> References: <54593DB6.1040305@oracle.com> <5459445F.8060405@oracle.com> Message-ID: <5459EC8E.3080100@oracle.com> Hi Aleksey, On 2014-11-04 22:25, Aleksey Shipilev wrote: > Hi Claes, > > On 04.11.2014 23:57, Claes Redestad wrote: >> please review this minor improvement to the initialization of >> MethodHandleImpl$Lazy. Running any code which initializes >> MethodHandlesImpl$Lazy will trigger throwing of >> java.lang.NoSuchFieldErrors, which has a measurable impact on startup. > Do you actually see the impact on startup? I thought the entire MHI$Lazy > thing is here to avoid instantiation on startup. If that is not true, > can you do a proper startup measurement on HelloWorld application? I should have written nashorn/jjs startup or similar: any program which initiates MethodHandlesImpl$Lazy. A simple java HelloWorld wouldn't be affected, while bin/jjs hello.js will, for any choice of hello.js. > > Here is a handy one-liner: > > for S in `seq 1 1000`; do time java Hello; done 2>&1 | grep real | sed > -e "s/0m//g" -e "s/s//g" | awk '{ sum += $2; n++ } END { if (n > 0) > print sum / n; }' > >> Additionally, the current lookup via findCollector in varargsArray is >> pointless (and will throw additional java.lang.NoSuchFieldErrors the >> first time a varargsArray of arity > 11 is requested) since all static >> methods which can be found will have been constructed already. >> >> bug: https://bugs.openjdk.java.net/browse/JDK-8062180 >> webrev: http://cr.openjdk.java.net/~redestad/8062180/webrev.00/ > Seems rather minor, but every little thing helps startup. Yes and yes. :) I could pitch this a cleanup effort as well, and would be happy to take this a step further and throw exceptions rather than returning nulls from findCollector, removing the assertions. > > - Would we be better off asserting (mh == null) instead of breaking and > asserting afterwards? assert(mh != null) does makes it more readable... > - Would we save a few more bytecodes by using the method-local "final > int ARRAYS_COUNT" to handle the magic number, instead of doing a > full-blown field? Ditto for FILL_ARRAYS_COUNT. ... and renders ARRAYS_COUNT redundant. The FILL_ARRAYS_COUNT is already used elsewhere, though. New webrev: http://cr.openjdk.java.net/~redestad/8062180/webrev.01 > >> A simple microbenchmark[1] show a very small (~0.5ms) improvement for >> Class.forName("java.lang.invoke.MethodHandleImpl$Lazy") >> >> Benchmark Mode Samples Score Score >> error Units >> o.s.CLBench.loadMethodHandleImplLazy ss 100 46.364 >> 1.740 ms >> o.s.CLBench.loadMethodHandleImplLazy ss 100 45.440 >> 1.515 ms > Strictly speaking, there is no improvement. Run with more samples to > shrink the error. Ran with the updated code, 20k forks get the error down and establish the improvement statistically: Benchmark Mode Samples Score Score error Units loadMethodHandleImplLazy ss 20000 40.257 0.102 ms loadMethodHandleImplLazy ss 20000 39.340 0.095 ms Thanks! /Claes > > -Aleksey. > From aleksey.shipilev at oracle.com Wed Nov 5 09:43:34 2014 From: aleksey.shipilev at oracle.com (Aleksey Shipilev) Date: Wed, 05 Nov 2014 12:43:34 +0300 Subject: RFR 8062180: MethodHandleImpl.makeArrays throws and swallows java.lang.NoSuchFieldError in normal flow In-Reply-To: <5459EC8E.3080100@oracle.com> References: <54593DB6.1040305@oracle.com> <5459445F.8060405@oracle.com> <5459EC8E.3080100@oracle.com> Message-ID: <5459F146.3010101@oracle.com> On 11/05/2014 12:23 PM, Claes Redestad wrote: > On 2014-11-04 22:25, Aleksey Shipilev wrote: >> On 04.11.2014 23:57, Claes Redestad wrote: >> - Would we be better off asserting (mh == null) instead of breaking and >> asserting afterwards? > > assert(mh != null) does makes it more readable... > >> - Would we save a few more bytecodes by using the method-local "final >> int ARRAYS_COUNT" to handle the magic number, instead of doing a >> full-blown field? Ditto for FILL_ARRAYS_COUNT. > > ... and renders ARRAYS_COUNT redundant. > > The FILL_ARRAYS_COUNT is already used elsewhere, though. > > New webrev: http://cr.openjdk.java.net/~redestad/8062180/webrev.01 Looks good to me. I would still introduce a final local variable for magic number 11, but that's a matter of personal style. > Ran with the updated code, 20k forks get the error down and establish > the improvement statistically: > > Benchmark Mode Samples Score Score error Units > loadMethodHandleImplLazy ss 20000 40.257 0.102 ms > loadMethodHandleImplLazy ss 20000 39.340 0.095 ms Now we are talking. -Aleksey. From daniel.fuchs at oracle.com Wed Nov 5 11:47:58 2014 From: daniel.fuchs at oracle.com (Daniel Fuchs) Date: Wed, 05 Nov 2014 12:47:58 +0100 Subject: RFR: 8060132: Handlers configured on abstract nodes in logging.properties are not always properly closed In-Reply-To: <545955C7.4010909@oracle.com> References: <5437F268.9020600@oracle.com> <5437FDCB.30803@oracle.com> <544A9B16.6030500@oracle.com> <544AC9B3.8090202@oracle.com> <54590368.5070309@oracle.com> <545955C7.4010909@oracle.com> Message-ID: <545A0E6E.5080700@oracle.com> On 04/11/14 23:40, Mandy Chung wrote: > On 11/4/14 8:48 AM, Daniel Fuchs wrote: >> Hi, >> >> Here is a revised patch that introduces a new >> .handlers.ensureCloseOnReset=true|false >> property. >> > You have explored several different options and as you have found, > none of them is ideal. I think having a property to specify > ensureCloseOnReset would be reasonable to address the memory leak > concern I have. > > Would it be better to name it .handlers.closeOnReset? Well - closeOnReset is the normal behavior - if the logger is still around. So I believe ensureCloseOnReset is a better name. > I am less sure if we need to have a global switch to revert > the default - do we expect the number of .handlers > is large? I would expect there would be small number. I would expect the cases where you need to specify ensureCloseOnReset will be rare. So maybe we should wait until the case arises before implementing a global switch: we can add that later if it's needed. The local switches will provide a workable (even though possibly cumbersome) work around. >> This property can be used as a fallback to turn the new behavior >> off in the improbable case where an application may need to do >> so. I have updated the LogManager class-level API documentation. >> >> http://cr.openjdk.java.net/~dfuchs/webrev_8060132/webrev.02/ >> > line 104-114: s/"true"/{@code true} > {@code reset} should be {@link ...} to the reset method (the > first occurrance) Will do. > It doesn't seem necessary to specify "strongly referenced". > Maybe it's good to say "it will ensure the handlers associated > with the loggers will be closed at shutdown or at reset. OK > If "closeOnReset is false", application should ensure the > handlers be closed before the logger gets garbage collected." I will try to find something. > - something along that line. > > typos: > 186: explicitely -> explicitly I keep on doing this same mistake again and again :-( > Perhaps PersistentLogger can be renamed to CloseOnResetLogger > to make it clear what these loggers are kept for. Similarly > for the variable name "persistentLogger" may be renamed too. OK. > 917: final might be adding noise I like final ;-) > 940: persitentLoggers OK > line 917: you don't need (names.length == 0) to determine the default, > right? if names.length == 0, it wouldn't add to the persistentLoggers list > as no iteration in the for loop. I will use the syntax that Stanimir suggested. It's much simpler than what I originally wrote. If names.length == 0 we don't need to look up the ensureCloseOnReset property. Thanks Mandy, I will send a new webrev shortly - with both yours & Stanimir comments integrated (barring the global switch - as we could do that in a followup revision if that's really needed). best regards, -- daniel > > > Mandy > From vladimir.x.ivanov at oracle.com Wed Nov 5 12:47:39 2014 From: vladimir.x.ivanov at oracle.com (Vladimir Ivanov) Date: Wed, 05 Nov 2014 16:47:39 +0400 Subject: RFR 8062180: MethodHandleImpl.makeArrays throws and swallows java.lang.NoSuchFieldError in normal flow In-Reply-To: <5459EC8E.3080100@oracle.com> References: <54593DB6.1040305@oracle.com> <5459445F.8060405@oracle.com> <5459EC8E.3080100@oracle.com> Message-ID: <545A1C6B.6020706@oracle.com> Claes, Thanks for spotting this inefficiency. > New webrev: http://cr.openjdk.java.net/~redestad/8062180/webrev.01 The new code catches the case when a collector is erroneously removed, but not when a new one is introduced. Can you extract verification logic into separate method and check there that the number of collectors is equal to the hard-coded constant? Otherwise, looks good. Best regards, Vladimir Ivanov From pooja.chopra at oracle.com Wed Nov 5 14:39:16 2014 From: pooja.chopra at oracle.com (pooja chopra) Date: Wed, 05 Nov 2014 20:09:16 +0530 Subject: [9] Review Request: 8059753 Fix for java/security/Security/ClassLoaderDeadlock/Deadlock2.sh fails with exit code 1 In-Reply-To: <5433C6BA.3000609@oracle.com> References: <5433C6BA.3000609@oracle.com> Message-ID: <545A3694.1090506@oracle.com> Hi All, A gentle reminder . Please review below fix. Regards, Pooja On 10/7/2014 4:25 PM, pooja chopra wrote: > Hello, > Please review a fix for the issue: > 8059753 [TEST_BUG] Test > java/security/Security/ClassLoaderDeadlock/Deadlock2.sh fails with > exit code 1 > Test bug fix. > https://bugs.openjdk.java.net/browse/JDK-8059753 > The webrev is: http://cr.openjdk.java.net/~kshefov/8059753/webrev.00/ > > Thanks > Pooja From joel.franck at oracle.com Wed Nov 5 15:10:34 2014 From: joel.franck at oracle.com (=?windows-1252?Q?Joel_Borggr=E9n-Franck?=) Date: Wed, 5 Nov 2014 16:10:34 +0100 Subject: Review request for 8055063: Parameter#toString() fails w/ AIOOBE for ctr of inner class w/ generic type In-Reply-To: <5453B58A.1090507@oracle.com> References: <5453B58A.1090507@oracle.com> Message-ID: <55A63A8D-1D0E-4D48-8CC4-D9326A57586E@oracle.com> Hi Eric, I think caching the real parameter type array on Executable is the wrong thing to do considering we can have a lot of instances of Executable but fairly few will be examined for parameters and the vast majority of those will never have this issue. Also, the tests are a bit minimal, I would expect you to check that the type of the parameter is correct. The tests should also have a @bug line. I think you are overly cautious here. I?m not sure we need to work around potential bugs in non-Java compilers. If language Y inserts synthetic/mandated parameters anywhere else than as a prefix they better generate a correct Signature (or non at all). This is a discussion we can defer however. If you flesh out the test and remove the caching I think this is a fine fix for both 8u and 9. cheers /Joel On 31 okt 2014, at 17:15, Eric McCorkle wrote: > Hello, > > Please review this patch which fixes issues that arise with > getGenericParameterTypes() and getAnnotatedParameterTypes() when there > are generic signatures and synthetic parameters. > > Please note that a complete fix is not possible for all cases. See > discussion on https://bugs.openjdk.java.net/browse/JDK-8062582 for details. > > This patch will cause Executable.getAnnotatedParameterTypes(), > Parameter.getAnnotatedType(), and Parameter.getParameterizedType() to > report the correct types in the following cases: > > * No generic signature is present. > * Both a generic signature and method parameters information are present > * A generic signature is present, but method parameters information is > not present, but the number of parameters in the generic signature and > the number of parameters in the method descriptor are the same. > > In the problematic case, where there is a generic signature, no method > parameters information, and the generic signature does not match the > method descriptor, these methods will return the correct /non/-generic > type, as there is no general way of associating parameters in the > generic signature with those in the method descriptor in this case. > > Please also note that there is currently a bug in javac which causes > type annotations' parameter indexes to be wrong when synthetic > parameters are generated: https://bugs.openjdk.java.net/browse/JDK-8029012. > > The bug report is here: > https://bugs.openjdk.java.net/browse/JDK-8055063 > > The webrev is here: > http://cr.openjdk.java.net/~emc/8055063/ From daniel.fuchs at oracle.com Wed Nov 5 16:15:09 2014 From: daniel.fuchs at oracle.com (Daniel Fuchs) Date: Wed, 05 Nov 2014 17:15:09 +0100 Subject: RFR: 8060132: Handlers configured on abstract nodes in logging.properties are not always properly closed In-Reply-To: <545A0E6E.5080700@oracle.com> References: <5437F268.9020600@oracle.com> <5437FDCB.30803@oracle.com> <544A9B16.6030500@oracle.com> <544AC9B3.8090202@oracle.com> <54590368.5070309@oracle.com> <545955C7.4010909@oracle.com> <545A0E6E.5080700@oracle.com> Message-ID: <545A4D0D.7090105@oracle.com> Hi Mandy, Stanimir, Here is the new webrev: http://cr.openjdk.java.net/~dfuchs/webrev_8060132/webrev.03/index.html best regards, -- daniel On 05/11/14 12:47, Daniel Fuchs wrote: > On 04/11/14 23:40, Mandy Chung wrote: >> On 11/4/14 8:48 AM, Daniel Fuchs wrote: >>> Hi, >>> >>> Here is a revised patch that introduces a new >>> .handlers.ensureCloseOnReset=true|false >>> property. >>> >> You have explored several different options and as you have found, >> none of them is ideal. I think having a property to specify >> ensureCloseOnReset would be reasonable to address the memory leak >> concern I have. >> >> Would it be better to name it .handlers.closeOnReset? > > Well - closeOnReset is the normal behavior - if the logger > is still around. So I believe ensureCloseOnReset is a better name. > >> I am less sure if we need to have a global switch to revert >> the default - do we expect the number of .handlers >> is large? I would expect there would be small number. > > I would expect the cases where you need to specify > ensureCloseOnReset will be rare. So maybe we should wait > until the case arises before implementing a global switch: > we can add that later if it's needed. The local switches > will provide a workable (even though possibly cumbersome) > work around. > >>> This property can be used as a fallback to turn the new behavior >>> off in the improbable case where an application may need to do >>> so. I have updated the LogManager class-level API documentation. >>> >>> http://cr.openjdk.java.net/~dfuchs/webrev_8060132/webrev.02/ >>> >> line 104-114: s/"true"/{@code true} >> {@code reset} should be {@link ...} to the reset method (the >> first occurrance) > > Will do. > >> It doesn't seem necessary to specify "strongly referenced". >> Maybe it's good to say "it will ensure the handlers associated >> with the loggers will be closed at shutdown or at reset. > > OK > >> If "closeOnReset is false", application should ensure the >> handlers be closed before the logger gets garbage collected." > > I will try to find something. > >> - something along that line. >> >> typos: >> 186: explicitely -> explicitly > > I keep on doing this same mistake again and again :-( > >> Perhaps PersistentLogger can be renamed to CloseOnResetLogger >> to make it clear what these loggers are kept for. Similarly >> for the variable name "persistentLogger" may be renamed too. > > OK. > > >> 917: final might be adding noise > I like final ;-) > >> 940: persitentLoggers > OK > > >> line 917: you don't need (names.length == 0) to determine the default, >> right? if names.length == 0, it wouldn't add to the persistentLoggers >> list >> as no iteration in the for loop. > > I will use the syntax that Stanimir suggested. It's much simpler > than what I originally wrote. If names.length == 0 we don't need to > look up the ensureCloseOnReset property. > > Thanks Mandy, > > I will send a new webrev shortly - with both yours & Stanimir comments > integrated (barring the global switch - as we could do that in a > followup revision if that's really needed). > > best regards, > > -- daniel > >> >> >> Mandy >> > From claes.redestad at oracle.com Wed Nov 5 16:22:04 2014 From: claes.redestad at oracle.com (Claes Redestad) Date: Wed, 05 Nov 2014 17:22:04 +0100 Subject: RFR 8062180: MethodHandleImpl.makeArrays throws and swallows java.lang.NoSuchFieldError in normal flow In-Reply-To: <545A1C6B.6020706@oracle.com> References: <54593DB6.1040305@oracle.com> <5459445F.8060405@oracle.com> <5459EC8E.3080100@oracle.com> <545A1C6B.6020706@oracle.com> Message-ID: <545A4EAC.1000203@oracle.com> Hi Vladimir, On 11/05/2014 01:47 PM, Vladimir Ivanov wrote: > Claes, > > Thanks for spotting this inefficiency. > >> New webrev: http://cr.openjdk.java.net/~redestad/8062180/webrev.01 > The new code catches the case when a collector is erroneously removed, > but not when a new one is introduced. Can you extract verification > logic into separate method and check there that the number of > collectors is equal to the hard-coded constant? Done: http://cr.openjdk.java.net/~redestad/8062180/webrev.02 I've run some tests with -ea -esa to ensure the asserts pass as they should. > > Otherwise, looks good. Thanks! /Claes > > Best regards, > Vladimir Ivanov From vladimir.x.ivanov at oracle.com Wed Nov 5 15:25:29 2014 From: vladimir.x.ivanov at oracle.com (Vladimir Ivanov) Date: Wed, 05 Nov 2014 19:25:29 +0400 Subject: RFR 8062180: MethodHandleImpl.makeArrays throws and swallows java.lang.NoSuchFieldError in normal flow In-Reply-To: <545A4EAC.1000203@oracle.com> References: <54593DB6.1040305@oracle.com> <5459445F.8060405@oracle.com> <5459EC8E.3080100@oracle.com> <545A1C6B.6020706@oracle.com> <545A4EAC.1000203@oracle.com> Message-ID: <545A4169.7@oracle.com> Looks good. Best regards, Vladimir Ivanov On 11/5/14, 8:22 PM, Claes Redestad wrote: > Hi Vladimir, > > On 11/05/2014 01:47 PM, Vladimir Ivanov wrote: >> Claes, >> >> Thanks for spotting this inefficiency. >> >>> New webrev: http://cr.openjdk.java.net/~redestad/8062180/webrev.01 >> The new code catches the case when a collector is erroneously removed, >> but not when a new one is introduced. Can you extract verification >> logic into separate method and check there that the number of >> collectors is equal to the hard-coded constant? > > Done: > > http://cr.openjdk.java.net/~redestad/8062180/webrev.02 > > I've run some tests with -ea -esa to ensure the asserts pass as they > should. > >> >> Otherwise, looks good. > > Thanks! > > /Claes > >> >> Best regards, >> Vladimir Ivanov > From peter.levart at gmail.com Wed Nov 5 16:58:21 2014 From: peter.levart at gmail.com (Peter Levart) Date: Wed, 05 Nov 2014 17:58:21 +0100 Subject: RFR: 8061950: Class.getMethods() exhibits quadratic time complexity In-Reply-To: <54551D2F.407@gmail.com> References: <900D2AE3-B3A6-4B1B-B25A-C564B10334AA@oracle.com> <54492255.3040108@gmail.com> <544D58D4.7020909@gmail.com> <544D7B74.5060607@gmail.com> <5450CEE9.7060707@gmail.com> <545120D2.5010301@gmail.com> <54526D10.5060101@gmail.com> <54551D2F.407@gmail.com> Message-ID: <545A572D.3050608@gmail.com> On 11/01/2014 06:49 PM, Peter Levart wrote: >> On 10/30/2014 07:13 PM, Martin Buchholz wrote: >> Which is why I was expecting to have to write an adaptive >> data structure that switches from linear search to hash-based when >> some threshold is exceeded. > > ...or two data structures. Since I have already arranged so that > construction of MethodTable gets an upper limit on the total number of > possible Method(s) that can be added to it, we can have two > MethodTable implementations and switch based on this number: > > http://cr.openjdk.java.net/~plevart/jdk9-dev/Class.getMethods/webrev.05/ Hi, I made some progress on this front. I created new MethodTable implementation (the third one) with similar performance as HashMap based, but producing half the garbage. I also added some optimizations to getMethod(), getMethods() and MethodTable in general that deal with special cases / parameter types comparison and have measurable impact on performance (for example loading all rt.jar classes and calling getMethods() for them): Original: 19658 classes loaded in 2.003623648 seconds. 494392 methods obtained in 0.932151066 seconds. 494392 methods obtained in 0.968247076 seconds. 494392 methods obtained in 0.989218185 seconds. 494392 methods obtained in 1.018700179 seconds. 494392 methods obtained in 0.945078767 seconds. 494392 methods obtained in 0.930856897 seconds. 494392 methods obtained in 0.905420555 seconds. 494392 methods obtained in 0.89253006 seconds. 494392 methods obtained in 0.914590141 seconds. 494392 methods obtained in 0.891616716 seconds. 494392 methods obtained in 0.891839815 seconds. 494392 methods obtained in 0.892863985 seconds. 494392 methods obtained in 0.960643274 seconds. 494392 methods obtained in 0.959266392 seconds. 494392 methods obtained in 0.894688322 seconds. 494392 methods obtained in 0.892751891 seconds. 494392 methods obtained in 0.912542838 seconds. 494392 methods obtained in 0.912219636 seconds. 494392 methods obtained in 0.895791468 seconds. 494392 methods obtained in 0.891673959 seconds. Patched: 19658 classes loaded in 2.145270948 seconds. 494398 methods obtained in 0.722630874 seconds. 494398 methods obtained in 0.697521755 seconds. 494398 methods obtained in 0.689642554 seconds. 494398 methods obtained in 0.742724992 seconds. 494398 methods obtained in 0.695693313 seconds. 494398 methods obtained in 0.685169108 seconds. 494398 methods obtained in 0.663012634 seconds. 494398 methods obtained in 0.666446935 seconds. 494398 methods obtained in 0.675662884 seconds. 494398 methods obtained in 0.65369404 seconds. 494398 methods obtained in 0.656608178 seconds. 494398 methods obtained in 0.668384051 seconds. 494398 methods obtained in 0.657548957 seconds. 494398 methods obtained in 0.672332234 seconds. 494398 methods obtained in 0.655699295 seconds. 494398 methods obtained in 0.671819628 seconds. 494398 methods obtained in 0.657232183 seconds. 494398 methods obtained in 0.654462137 seconds. 494398 methods obtained in 0.659630473 seconds. 494398 methods obtained in 0.674219391 seconds. (the difference in method count is expected - patched code contains new methods in existing rt.jar classes ;-) Here's new webrev: http://cr.openjdk.java.net/~plevart/jdk9-dev/Class.getMethods/webrev.06/ The optimizations made from webrev.05 are: - getMethod() skips construction of MethodTable if there are no (super)interfaces. - getMethods() returns just declared public methods if there are no superclass and no (super)interfaces. - comparing method parameter types is optimized by adding two methods to Method/LangReflectAccess/ReflectionFactory. New MethodTable implementation based on a linear-probe hash table is a space/garbage improvement. I took IdentityHashMap, removed unneeded stuff and modified it's API. The result is a HashArray. It's API is similar in function and form to java.util.Map, but doesn't use separate keys and values. An element of HashArray is a key and a value at the same time. Elements are always non-null, so the method return values are unambiguous. As HashArray is a linear-probe hash table and there are no Map.Entry objects involved, the underlying data structure is very simple and memory efficient. It is just a sparse array of elements with length that is always a power of two and larger than 3 * size / 2. It also features overriddable element equals/hashCode methods. I made it a separate generic class because I think it can find it's usage elsewhere (for example as a cannonicalizing cache). Since HashArray based MethodTable is more space-efficient I moved the line between simple array based and HashArray based MethodTable down to 20 elements to minimize the worst-case scenario effect. Calling getMethods() on all rt.jar classes now constructs about 3/4 simple array based and 1/4 HashArray based MethodTables. Here's also Martin's ManyMethodsBenchmark: Original: Base class load time: 129.95 ms getDeclaredMethods: 65521 methods, 36.58 ms total time, 0.0006 ms per method getMethods : 65530 methods, 47.43 ms total time, 0.0007 ms per method Derived class load time: 32216.09 ms getDeclaredMethods: 65521 methods, 35.05 ms total time, 0.0005 ms per method getMethods : 65530 methods, 8068.66 ms total time, 0.1231 ms per method Patched (using HashArray based MethodTable): Base class load time: 126.00 ms getDeclaredMethods: 65521 methods, 36.83 ms total time, 0.0006 ms per method getMethods : 65530 methods, 45.08 ms total time, 0.0007 ms per method Derived class load time: 31865.27 ms getDeclaredMethods: 65521 methods, 35.01 ms total time, 0.0005 ms per method getMethods : 65530 methods, 78.05 ms total time, 0.0012 ms per method All 86 jtreg test in java.lang/Class/ and java/lang/reflect/ still pass. Regards, Peter From stanimir at riflexo.com Wed Nov 5 18:27:38 2014 From: stanimir at riflexo.com (Stanimir Simeonoff) Date: Wed, 5 Nov 2014 20:27:38 +0200 Subject: RFR: 8061950: Class.getMethods() exhibits quadratic time complexity In-Reply-To: <545A572D.3050608@gmail.com> References: <900D2AE3-B3A6-4B1B-B25A-C564B10334AA@oracle.com> <54492255.3040108@gmail.com> <544D58D4.7020909@gmail.com> <544D7B74.5060607@gmail.com> <5450CEE9.7060707@gmail.com> <545120D2.5010301@gmail.com> <54526D10.5060101@gmail.com> <54551D2F.407@gmail.com> <545A572D.3050608@gmail.com> Message-ID: A very small issue: MethodTable.java: 697 while (i < hwm && next == null) next = methods[i++]; In my opinion the condition should be while (next == null && i < hwm) as next will be non-null on each invocation of next() and generally it's more likely to exit the loop. I suppose HashMapImpl is unused . HashArray.java: 82 Integer.highestOneBit(expectedMaxSize + (expectedMaxSize << 1)) This effectively multiples expectedMaxSize by 3, I guess you meant >> instead. Perhaps extra a method about effective capacity? I will take a deeper look a bit later. Stanimir On Wed, Nov 5, 2014 at 6:58 PM, Peter Levart wrote: > On 11/01/2014 06:49 PM, Peter Levart wrote: > >> On 10/30/2014 07:13 PM, Martin Buchholz wrote: Which is why I was >>> expecting to have to write an adaptive >>> data structure that switches from linear search to hash-based when >>> some threshold is exceeded. >>> >> >> ...or two data structures. Since I have already arranged so that >> construction of MethodTable gets an upper limit on the total number of >> possible Method(s) that can be added to it, we can have two MethodTable >> implementations and switch based on this number: >> >> http://cr.openjdk.java.net/~plevart/jdk9-dev/Class.getMethods/webrev.05/ >> > > Hi, > > I made some progress on this front. I created new MethodTable > implementation (the third one) with similar performance as HashMap based, > but producing half the garbage. I also added some optimizations to > getMethod(), getMethods() and MethodTable in general that deal with special > cases / parameter types comparison and have measurable impact on > performance (for example loading all rt.jar classes and calling > getMethods() for them): > > Original: > > 19658 classes loaded in 2.003623648 seconds. > 494392 methods obtained in 0.932151066 seconds. > 494392 methods obtained in 0.968247076 seconds. > 494392 methods obtained in 0.989218185 seconds. > 494392 methods obtained in 1.018700179 seconds. > 494392 methods obtained in 0.945078767 seconds. > 494392 methods obtained in 0.930856897 seconds. > 494392 methods obtained in 0.905420555 seconds. > 494392 methods obtained in 0.89253006 seconds. > 494392 methods obtained in 0.914590141 seconds. > 494392 methods obtained in 0.891616716 seconds. > 494392 methods obtained in 0.891839815 seconds. > 494392 methods obtained in 0.892863985 seconds. > 494392 methods obtained in 0.960643274 seconds. > 494392 methods obtained in 0.959266392 seconds. > 494392 methods obtained in 0.894688322 seconds. > 494392 methods obtained in 0.892751891 seconds. > 494392 methods obtained in 0.912542838 seconds. > 494392 methods obtained in 0.912219636 seconds. > 494392 methods obtained in 0.895791468 seconds. > 494392 methods obtained in 0.891673959 seconds. > > Patched: > > 19658 classes loaded in 2.145270948 seconds. > 494398 methods obtained in 0.722630874 seconds. > 494398 methods obtained in 0.697521755 seconds. > 494398 methods obtained in 0.689642554 seconds. > 494398 methods obtained in 0.742724992 seconds. > 494398 methods obtained in 0.695693313 seconds. > 494398 methods obtained in 0.685169108 seconds. > 494398 methods obtained in 0.663012634 seconds. > 494398 methods obtained in 0.666446935 seconds. > 494398 methods obtained in 0.675662884 seconds. > 494398 methods obtained in 0.65369404 seconds. > 494398 methods obtained in 0.656608178 seconds. > 494398 methods obtained in 0.668384051 seconds. > 494398 methods obtained in 0.657548957 seconds. > 494398 methods obtained in 0.672332234 seconds. > 494398 methods obtained in 0.655699295 seconds. > 494398 methods obtained in 0.671819628 seconds. > 494398 methods obtained in 0.657232183 seconds. > 494398 methods obtained in 0.654462137 seconds. > 494398 methods obtained in 0.659630473 seconds. > 494398 methods obtained in 0.674219391 seconds. > > (the difference in method count is expected - patched code contains new > methods in existing rt.jar classes ;-) > > Here's new webrev: > > http://cr.openjdk.java.net/~plevart/jdk9-dev/Class.getMethods/webrev.06/ > > > The optimizations made from webrev.05 are: > > - getMethod() skips construction of MethodTable if there are no > (super)interfaces. > - getMethods() returns just declared public methods if there are no > superclass and no (super)interfaces. > - comparing method parameter types is optimized by adding two methods to > Method/LangReflectAccess/ReflectionFactory. > > New MethodTable implementation based on a linear-probe hash table is a > space/garbage improvement. I took IdentityHashMap, removed unneeded stuff > and modified it's API. The result is a HashArray. It's API is similar in > function and form to java.util.Map, but doesn't use separate keys and > values. An element of HashArray is a key and a value at the same time. > Elements are always non-null, so the method return values are unambiguous. > As HashArray is a linear-probe hash table and there are no Map.Entry > objects involved, the underlying data structure is very simple and memory > efficient. It is just a sparse array of elements with length that is always > a power of two and larger than 3 * size / 2. It also features overriddable > element equals/hashCode methods. I made it a separate generic class because > I think it can find it's usage elsewhere (for example as a cannonicalizing > cache). > > Since HashArray based MethodTable is more space-efficient I moved the line > between simple array based and HashArray based MethodTable down to 20 > elements to minimize the worst-case scenario effect. Calling getMethods() > on all rt.jar classes now constructs about 3/4 simple array based and 1/4 > HashArray based MethodTables. > > Here's also Martin's ManyMethodsBenchmark: > > Original: > > Base class load time: 129.95 ms > getDeclaredMethods: 65521 methods, 36.58 ms total time, 0.0006 ms per > method > getMethods : 65530 methods, 47.43 ms total time, 0.0007 ms per > method > Derived class load time: 32216.09 ms > getDeclaredMethods: 65521 methods, 35.05 ms total time, 0.0005 ms per > method > getMethods : 65530 methods, 8068.66 ms total time, 0.1231 ms per > method > > > Patched (using HashArray based MethodTable): > > Base class load time: 126.00 ms > getDeclaredMethods: 65521 methods, 36.83 ms total time, 0.0006 ms per > method > getMethods : 65530 methods, 45.08 ms total time, 0.0007 ms per > method > Derived class load time: 31865.27 ms > getDeclaredMethods: 65521 methods, 35.01 ms total time, 0.0005 ms per > method > getMethods : 65530 methods, 78.05 ms total time, 0.0012 ms per > method > > > All 86 jtreg test in java.lang/Class/ and java/lang/reflect/ still pass. > > > Regards, Peter > > From stanimir at riflexo.com Wed Nov 5 18:47:41 2014 From: stanimir at riflexo.com (Stanimir Simeonoff) Date: Wed, 5 Nov 2014 20:47:41 +0200 Subject: RFR: 8060132: Handlers configured on abstract nodes in logging.properties are not always properly closed In-Reply-To: <545A4D0D.7090105@oracle.com> References: <5437F268.9020600@oracle.com> <5437FDCB.30803@oracle.com> <544A9B16.6030500@oracle.com> <544AC9B3.8090202@oracle.com> <54590368.5070309@oracle.com> <545955C7.4010909@oracle.com> <545A0E6E.5080700@oracle.com> <545A4D0D.7090105@oracle.com> Message-ID: 945 } catch (Exception ex) { 946 System.err.println("Can't load log handler \"" + type + "\""); 947 System.err.println("" + ex); 948 ex.printStackTrace(); 949 } I'm not quite sure if handling Error (in particular any subclass of LinkageError) would make sense here (similar to the invocation of listeners). Of course there are other case handling just Exception and using word instead of 'type' but that's matter of another clean up. Stanimir On Wed, Nov 5, 2014 at 6:15 PM, Daniel Fuchs wrote: Hi Mandy, Stanimir, > > Here is the new webrev: > http://cr.openjdk.java.net/~dfuchs/webrev_8060132/webrev.03/index.html > > best regards, > > -- daniel > > > On 05/11/14 12:47, Daniel Fuchs wrote: > >> On 04/11/14 23:40, Mandy Chung wrote: >> >>> On 11/4/14 8:48 AM, Daniel Fuchs wrote: >>> >>>> Hi, >>>> >>>> Here is a revised patch that introduces a new >>>> .handlers.ensureCloseOnReset=true|false >>>> property. >>>> >>>> You have explored several different options and as you have found, >>> none of them is ideal. I think having a property to specify >>> ensureCloseOnReset would be reasonable to address the memory leak >>> concern I have. >>> >>> Would it be better to name it .handlers.closeOnReset? >>> >> >> Well - closeOnReset is the normal behavior - if the logger >> is still around. So I believe ensureCloseOnReset is a better name. >> >> I am less sure if we need to have a global switch to revert >>> the default - do we expect the number of .handlers >>> is large? I would expect there would be small number. >>> >> >> I would expect the cases where you need to specify >> ensureCloseOnReset will be rare. So maybe we should wait >> until the case arises before implementing a global switch: >> we can add that later if it's needed. The local switches >> will provide a workable (even though possibly cumbersome) >> work around. >> >> This property can be used as a fallback to turn the new behavior >>>> off in the improbable case where an application may need to do >>>> so. I have updated the LogManager class-level API documentation. >>>> >>>> http://cr.openjdk.java.net/~dfuchs/webrev_8060132/webrev.02/ >>>> >>>> line 104-114: s/"true"/{@code true} >>> {@code reset} should be {@link ...} to the reset method (the >>> first occurrance) >>> >> >> Will do. >> >> It doesn't seem necessary to specify "strongly referenced". >>> Maybe it's good to say "it will ensure the handlers associated >>> with the loggers will be closed at shutdown or at reset. >>> >> >> OK >> >> If "closeOnReset is false", application should ensure the >>> handlers be closed before the logger gets garbage collected." >>> >> >> I will try to find something. >> >> - something along that line. >>> >>> typos: >>> 186: explicitely -> explicitly >>> >> >> I keep on doing this same mistake again and again :-( >> >> Perhaps PersistentLogger can be renamed to CloseOnResetLogger >>> to make it clear what these loggers are kept for. Similarly >>> for the variable name "persistentLogger" may be renamed too. >>> >> >> OK. >> >> >> 917: final might be adding noise >>> >> I like final ;-) >> >> 940: persitentLoggers >>> >> OK >> >> >> line 917: you don't need (names.length == 0) to determine the default, >>> right? if names.length == 0, it wouldn't add to the persistentLoggers >>> list >>> as no iteration in the for loop. >>> >> >> I will use the syntax that Stanimir suggested. It's much simpler >> than what I originally wrote. If names.length == 0 we don't need to >> look up the ensureCloseOnReset property. >> >> Thanks Mandy, >> >> I will send a new webrev shortly - with both yours & Stanimir comments >> integrated (barring the global switch - as we could do that in a >> followup revision if that's really needed). >> >> best regards, >> >> -- daniel >> >> >>> >>> Mandy >>> >>> >> > From huizhe.wang at oracle.com Wed Nov 5 19:27:59 2014 From: huizhe.wang at oracle.com (huizhe wang) Date: Wed, 05 Nov 2014 11:27:59 -0800 Subject: Review request for XML JAXP unit test colocation In-Reply-To: <013601cff8d8$983b4800$c8b1d800$@oracle.com> References: <013601cff8d8$983b4800$c8b1d800$@oracle.com> Message-ID: <545A7A3F.5000407@oracle.com> Hi Frank, It's great to see that you re-categorized the tests and consolidated them into their proper categories. The webrev still shows Unexpected Error when creating diff for binary files, e.g. Hello.wsdl, an issue with how Mercurial handles them (never had problem with subversion before). I suggest removing these wsdl files, and creating the xml within the tests themselves as literal strings, for example for Bug6388460, replace the 'source' with the content of Hello.wsdl (Note NetBeans can do this automatically: type double quote and then paste the content of Hello.wsdl): Source source = new StreamSource(new StringReader( "\n" + "\n" + " \n" + " \n" + " \n" + " \n" + " \n" + " \n" + " \n" + " \n" + " \n" + " \n" + " \n" + " \n" + " \n" + " \n" + " \n" + " \n" + " \n" + " \n" + " \n" + " \n" + " \n" + " \n" + " \n" + " \n" + " \n" + " \n" + " \n" + " \n" + " \n" + " \n" + " \n" + "\n" )); Just spotted a minor typo in ProblemList.txt: exclued -> excluded Thanks, Joe On 11/5/2014 1:12 AM, Frank Yuan wrote: > Hi, Joe and All > > Thanks for your previous reviews and comments. Per your comments: I added > description for every test, got them running with and without security > manager and fixed the known test issues. Could you help review the changes > for JAXP unittest co-location again? After your review, I will push the > tests into JDK repo: jaxp/test. > > bug: https://bugs.openjdk.java.net/browse/JDK-8043090 > webrev: http://sqeweb.us.oracle.com/jsn/users/yc/webrev/ > test results: > http://sqeweb.us.oracle.com/jsn/users/yc/result/JTreport/html/index.html > > Thanks, > > Frank > From eric.mccorkle at oracle.com Wed Nov 5 21:54:12 2014 From: eric.mccorkle at oracle.com (Eric McCorkle) Date: Wed, 05 Nov 2014 16:54:12 -0500 Subject: Review request for 8055063: Parameter#toString() fails w/ AIOOBE for ctr of inner class w/ generic type In-Reply-To: <55A63A8D-1D0E-4D48-8CC4-D9326A57586E@oracle.com> References: <5453B58A.1090507@oracle.com> <55A63A8D-1D0E-4D48-8CC4-D9326A57586E@oracle.com> Message-ID: <545A9C84.1060805@oracle.com> Thanks, Joel. I've applied your fixes and re-upped the webrev. On 11/05/14 10:10, Joel Borggr?n-Franck wrote: > Hi Eric, > > I think caching the real parameter type array on Executable is the wrong thing to do considering we can have a lot of instances of Executable but fairly few will be examined for parameters and the vast majority of those will never have this issue. > > Also, the tests are a bit minimal, I would expect you to check that the type of the parameter is correct. The tests should also have a @bug line. > > I think you are overly cautious here. I?m not sure we need to work around potential bugs in non-Java compilers. If language Y inserts synthetic/mandated parameters anywhere else than as a prefix they better generate a correct Signature (or non at all). This is a discussion we can defer however. > > If you flesh out the test and remove the caching I think this is a fine fix for both 8u and 9. > > cheers > /Joel > > On 31 okt 2014, at 17:15, Eric McCorkle wrote: > >> Hello, >> >> Please review this patch which fixes issues that arise with >> getGenericParameterTypes() and getAnnotatedParameterTypes() when there >> are generic signatures and synthetic parameters. >> >> Please note that a complete fix is not possible for all cases. See >> discussion on https://bugs.openjdk.java.net/browse/JDK-8062582 for details. >> >> This patch will cause Executable.getAnnotatedParameterTypes(), >> Parameter.getAnnotatedType(), and Parameter.getParameterizedType() to >> report the correct types in the following cases: >> >> * No generic signature is present. >> * Both a generic signature and method parameters information are present >> * A generic signature is present, but method parameters information is >> not present, but the number of parameters in the generic signature and >> the number of parameters in the method descriptor are the same. >> >> In the problematic case, where there is a generic signature, no method >> parameters information, and the generic signature does not match the >> method descriptor, these methods will return the correct /non/-generic >> type, as there is no general way of associating parameters in the >> generic signature with those in the method descriptor in this case. >> >> Please also note that there is currently a bug in javac which causes >> type annotations' parameter indexes to be wrong when synthetic >> parameters are generated: https://bugs.openjdk.java.net/browse/JDK-8029012. >> >> The bug report is here: >> https://bugs.openjdk.java.net/browse/JDK-8055063 >> >> The webrev is here: >> http://cr.openjdk.java.net/~emc/8055063/ > From eric.mccorkle at oracle.com Wed Nov 5 21:57:26 2014 From: eric.mccorkle at oracle.com (Eric McCorkle) Date: Wed, 05 Nov 2014 16:57:26 -0500 Subject: Review request for 8055063: Parameter#toString() fails w/ AIOOBE for ctr of inner class w/ generic type In-Reply-To: <55A63A8D-1D0E-4D48-8CC4-D9326A57586E@oracle.com> References: <5453B58A.1090507@oracle.com> <55A63A8D-1D0E-4D48-8CC4-D9326A57586E@oracle.com> Message-ID: <545A9D46.9040002@oracle.com> A word on the tests: I don't think there's a stable way to check the types on getParameterizedType() and getAnnotatedType(). So I checked the type on getType() only. The problem is, you have to detect and cast from ParameterizedType and AnnotatedType respectively, but depending on the situation with generic signatures and method parameters information (and type annotations), you might not actually get back one of these. On 11/05/14 10:10, Joel Borggr?n-Franck wrote: > Hi Eric, > > I think caching the real parameter type array on Executable is the wrong thing to do considering we can have a lot of instances of Executable but fairly few will be examined for parameters and the vast majority of those will never have this issue. > > Also, the tests are a bit minimal, I would expect you to check that the type of the parameter is correct. The tests should also have a @bug line. > > I think you are overly cautious here. I?m not sure we need to work around potential bugs in non-Java compilers. If language Y inserts synthetic/mandated parameters anywhere else than as a prefix they better generate a correct Signature (or non at all). This is a discussion we can defer however. > > If you flesh out the test and remove the caching I think this is a fine fix for both 8u and 9. > > cheers > /Joel > > On 31 okt 2014, at 17:15, Eric McCorkle wrote: > >> Hello, >> >> Please review this patch which fixes issues that arise with >> getGenericParameterTypes() and getAnnotatedParameterTypes() when there >> are generic signatures and synthetic parameters. >> >> Please note that a complete fix is not possible for all cases. See >> discussion on https://bugs.openjdk.java.net/browse/JDK-8062582 for details. >> >> This patch will cause Executable.getAnnotatedParameterTypes(), >> Parameter.getAnnotatedType(), and Parameter.getParameterizedType() to >> report the correct types in the following cases: >> >> * No generic signature is present. >> * Both a generic signature and method parameters information are present >> * A generic signature is present, but method parameters information is >> not present, but the number of parameters in the generic signature and >> the number of parameters in the method descriptor are the same. >> >> In the problematic case, where there is a generic signature, no method >> parameters information, and the generic signature does not match the >> method descriptor, these methods will return the correct /non/-generic >> type, as there is no general way of associating parameters in the >> generic signature with those in the method descriptor in this case. >> >> Please also note that there is currently a bug in javac which causes >> type annotations' parameter indexes to be wrong when synthetic >> parameters are generated: https://bugs.openjdk.java.net/browse/JDK-8029012. >> >> The bug report is here: >> https://bugs.openjdk.java.net/browse/JDK-8055063 >> >> The webrev is here: >> http://cr.openjdk.java.net/~emc/8055063/ > From tristan.yan at oracle.com Wed Nov 5 22:57:08 2014 From: tristan.yan at oracle.com (Tristan Yan) Date: Wed, 5 Nov 2014 14:57:08 -0800 Subject: RFR 8047962: XML test colocation: AuctionPortal test for bid.com In-Reply-To: <545815D5.5020009@oracle.com> References: <545815D5.5020009@oracle.com> Message-ID: Thank you. Joe. Git plugin for mercurial works well for hg command but webrev script still doesn?t support the binary file. I did one small change to replace movies.xml with a Java string to suppress this error. Also I took your advice to move xml files into auction portal directory. Changed tests have been run with and without the Security Manager as usual. All the tests pass under both scenarios. http://cr.openjdk.java.net/~tyan/JDK-8047962/webrev.02/ Could you please sponsor this for me. Thank you very much. Tristan > On Nov 3, 2014, at 3:55 PM, huizhe wang wrote: > > Hi Tristan, > > Looks good overall. Once again, it's great to see that you've made the tests a lot cleaner, getting rid of the old report system and etc. > > The only issue I see is with movies.xml. If I use patch to apply your patch to my workspace, I get no movies.xml that in turn causes AuctionController to fail. It's probably related how your webrev was generated (note the error: Unexpected Error occurred reading `diff -e /dev/null new/test/javax/xml/jaxp/functional/test/content/movies.xml`: $?=0, err= 1 ). hg diff works fine generally, but not for changes in binary files. Use the git option, "-git", can fix the problem. You may see the following in your configuration: > > [diff] > git = true > > Thanks, > Joe > > On 10/31/2014 12:16 PM, Tristan Yan wrote: >> Hi Joe, Alan and all others >> >> Would you please help reviewing these tests? The intent is moving some JAXP tests from closed to open. The associated bug number is >> https://bugs.openjdk.java.net/browse/JDK-8047962 . >> >> These tests have been ran with and without the Security Manager. All the tests pass under both scenarios. >> >> http://cr.openjdk.java.net/~tyan/JDK-8047962/webrev.01/ >> Thank you. >> Tristan > From huizhe.wang at oracle.com Wed Nov 5 23:23:05 2014 From: huizhe.wang at oracle.com (huizhe wang) Date: Wed, 05 Nov 2014 15:23:05 -0800 Subject: RFR 8047962: XML test colocation: AuctionPortal test for bid.com In-Reply-To: References: <545815D5.5020009@oracle.com> Message-ID: <545AB159.9090905@oracle.com> Hi Tristan, It's good to see movie.xml is replaced with a literal string. Sorry if I wasn't clear on the directory structure. It would be nice if all files for AuctionPortal to be placed under AuctionPortal, that is: test/javax/xml/jaxp/functional/test/auctionportal/ content/ (xml and xsd) golden/ (gold files) Would that make sense? It's nice you have the paths in one place (HiBidConstants) that makes it easier to move files around. Yes, I'll sponsor that for you. Thanks, Joe On 11/5/2014 2:57 PM, Tristan Yan wrote: > Thank you. Joe. > Git plugin for mercurial works well for hg command but webrev script > still doesn?t support the binary file. I did one small change to > replace movies.xml with a Java string to suppress this error. Also I > took your advice to move xml files into auction portal directory. > Changed tests have been run with and without the Security Manager as > usual. All the tests pass under both scenarios. > > http://cr.openjdk.java.net/~tyan/JDK-8047962/webrev.02/ > > > Could you please sponsor this for me. > Thank you very much. > Tristan > >> On Nov 3, 2014, at 3:55 PM, huizhe wang > > wrote: >> >> Hi Tristan, >> >> Looks good overall. Once again, it's great to see that you've made >> the tests a lot cleaner, getting rid of the old report system and etc. >> >> The only issue I see is with movies.xml. If I use patch to apply your >> patch to my workspace, I get no movies.xml that in turn causes >> AuctionController to fail. It's probably related how your webrev was >> generated (note the error: Unexpected Error occurred reading `diff -e >> /dev/null >> new/test/javax/xml/jaxp/functional/test/content/movies.xml`: $?=0, >> err= 1 ). hg diff works fine generally, but not for changes in >> binary files. Use the git option, "-git", can fix the problem. You >> may see the following in your configuration: >> >> [diff] >> git = true >> >> Thanks, >> Joe >> >> On 10/31/2014 12:16 PM, Tristan Yan wrote: >>> Hi Joe, Alan and all others >>> >>> Would you please help reviewing these tests? The intent is moving >>> some JAXP tests from closed to open. The associated bug number is >>> https://bugs.openjdk.java.net/browse/JDK-8047962. >>> >>> These tests have been ran with and without the Security Manager. All >>> the tests pass under both scenarios. >>> >>> http://cr.openjdk.java.net/~tyan/JDK-8047962/webrev.01/ >>> >>> Thank you. >>> Tristan >> > From tristan.yan at oracle.com Thu Nov 6 00:01:12 2014 From: tristan.yan at oracle.com (Tristan Yan) Date: Wed, 5 Nov 2014 16:01:12 -0800 Subject: RFR 8047962: XML test colocation: AuctionPortal test for bid.com In-Reply-To: <545AB159.9090905@oracle.com> References: <545815D5.5020009@oracle.com> <545AB159.9090905@oracle.com> Message-ID: <97F180B9-DD34-4450-92B5-E9CBA369E8C8@oracle.com> Thanks again This makes more sense to me. Now they look clearer. http://cr.openjdk.java.net/~tyan/JDK-8047962/webrev.03/ Thank you for sponsoring this. Tristan > On Nov 5, 2014, at 3:23 PM, huizhe wang wrote: > > Hi Tristan, > > It's good to see movie.xml is replaced with a literal string. > > Sorry if I wasn't clear on the directory structure. It would be nice if all files for AuctionPortal to be placed under AuctionPortal, that is: > test/javax/xml/jaxp/functional/test/auctionportal/ > content/ (xml and xsd) > golden/ (gold files) > > > Would that make sense? It's nice you have the paths in one place (HiBidConstants) that makes it easier to move files around. > > Yes, I'll sponsor that for you. > > Thanks, > Joe > > On 11/5/2014 2:57 PM, Tristan Yan wrote: >> Thank you. Joe. >> Git plugin for mercurial works well for hg command but webrev script still doesn?t support the binary file. I did one small change to replace movies.xml with a Java string to suppress this error. Also I took your advice to move xml files into auction portal directory. Changed tests have been run with and without the Security Manager as usual. All the tests pass under both scenarios. >> >> http://cr.openjdk.java.net/~tyan/JDK-8047962/webrev.02/ >> >> Could you please sponsor this for me. >> Thank you very much. >> Tristan >> >>> On Nov 3, 2014, at 3:55 PM, huizhe wang > wrote: >>> >>> Hi Tristan, >>> >>> Looks good overall. Once again, it's great to see that you've made the tests a lot cleaner, getting rid of the old report system and etc. >>> >>> The only issue I see is with movies.xml. If I use patch to apply your patch to my workspace, I get no movies.xml that in turn causes AuctionController to fail. It's probably related how your webrev was generated (note the error: Unexpected Error occurred reading `diff -e /dev/null new/test/javax/xml/jaxp/functional/test/content/movies.xml`: $?=0, err= 1 ). hg diff works fine generally, but not for changes in binary files. Use the git option, "-git", can fix the problem. You may see the following in your configuration: >>> >>> [diff] >>> git = true >>> >>> Thanks, >>> Joe >>> >>> On 10/31/2014 12:16 PM, Tristan Yan wrote: >>>> Hi Joe, Alan and all others >>>> >>>> Would you please help reviewing these tests? The intent is moving some JAXP tests from closed to open. The associated bug number is >>>> https://bugs.openjdk.java.net/browse/JDK-8047962 . >>>> >>>> These tests have been ran with and without the Security Manager. All the tests pass under both scenarios. >>>> >>>> http://cr.openjdk.java.net/~tyan/JDK-8047962/webrev.01/ >>>> Thank you. >>>> Tristan >>> >> > From tristan.yan at oracle.com Thu Nov 6 03:41:14 2014 From: tristan.yan at oracle.com (Tristan Yan) Date: Wed, 5 Nov 2014 19:41:14 -0800 Subject: Review request for XML JAXP unit test colocation In-Reply-To: <006701cff96e$48682fa0$d9388ee0$@oracle.com> References: <013601cff8d8$983b4800$c8b1d800$@oracle.com> <006701cff96e$48682fa0$d9388ee0$@oracle.com> Message-ID: <7F801BA9-B3D1-4E3F-A8C5-637EF5CDF3D6@oracle.com> Hi Frank What I meant is uploading webrev for you. An open JDK review means you send your code to open JDK alias. Anyone is open JDK alias is eligible to review your code. There are some people they are not working on Oracle. They have no access to Oracle network. According the open JDK process your webrev has to be uploaded into somewhere that everyone can access. Most of people are using cr.openjdk.java.net because this is provided to people who has Author/Role in open JDK community. I can?t create an account for you. You need contribute before you get an open JDK id. Please see the detail information as following link. http://openjdk.java.net/contribute/ Thanks Tristan > On Nov 5, 2014, at 7:03 PM, Frank Yuan wrote: > > Oh, thanks for your reminder! > > Do you mean you can help me to generate an account to access cr.openjdk.java.net ? Certainly I want, thanks in advance. > > Best Regards > Frank > > From: Tristan Yan [mailto:tristan.yan at oracle.com] > Sent: Thursday, November 06, 2014 3:54 AM > To: Frank Yuan > Cc: Core Java Libraries SQE > Subject: Re: Review request for XML JAXP unit test colocation > > Hi Frank > I think you?re using internal link for open review. You need a standard one in cr.openjdk.java.net . Do you want me to help you to generate one? > Thanks > Tristan > >> On Nov 5, 2014, at 1:12 AM, Frank Yuan > wrote: >> >> Hi, Joe and All >> >> Thanks for your previous reviews and comments. Per your comments: I added >> description for every test, got them running with and without security >> manager and fixed the known test issues. Could you help review the changes >> for JAXP unittest co-location again? After your review, I will push the >> tests into JDK repo: jaxp/test. >> >> bug: https://bugs.openjdk.java.net/browse/JDK-8043090 >> webrev: http://sqeweb.us.oracle.com/jsn/users/yc/webrev/ >> test results: >> http://sqeweb.us.oracle.com/jsn/users/yc/result/JTreport/html/index.html >> >> Thanks, >> >> Frank >> From frank.yuan at oracle.com Thu Nov 6 07:31:30 2014 From: frank.yuan at oracle.com (Frank Yuan) Date: Thu, 6 Nov 2014 15:31:30 +0800 Subject: Review request for XML JAXP unit test colocation In-Reply-To: <545A7A3F.5000407@oracle.com> References: <013601cff8d8$983b4800$c8b1d800$@oracle.com> <545A7A3F.5000407@oracle.com> Message-ID: <016501cff993$b0aadc50$120094f0$@oracle.com> Hi Joe Thanks for your comments! I have corrected the typo error in ProblemList.txt. The Unexpected Error happened on the file beginning with BOM, the corresponding tests are to test BOM utf file. Because there is no mistake in the code, so I would not modify them for webrev tool, after all the reviewer can check the raw file. To Tristan Would you like to move the webrev and the test result to cr.openjdk.java.net for me? http://sqeweb.us.oracle.com/jsn/users/yc/webrev.zip http://sqeweb.us.oracle.com/jsn/users/yc/result.zip Best Regards Frank -----Original Message----- From: huizhe wang [mailto:huizhe.wang at oracle.com] Sent: Thursday, November 06, 2014 3:28 AM To: Frank Yuan Cc: 'Core-Libs-Dev'; 'jibing chen'; 'Gustavo Galimberti'; sandeep.konchady at oracle.com Subject: Re: Review request for XML JAXP unit test colocation Hi Frank, It's great to see that you re-categorized the tests and consolidated them into their proper categories. The webrev still shows Unexpected Error when creating diff for binary files, e.g. Hello.wsdl, an issue with how Mercurial handles them (never had problem with subversion before). I suggest removing these wsdl files, and creating the xml within the tests themselves as literal strings, for example for Bug6388460, replace the 'source' with the content of Hello.wsdl (Note NetBeans can do this automatically: type double quote and then paste the content of Hello.wsdl): Source source = new StreamSource(new StringReader( "\n" + "\n" + " \n" + " \n" + " \n" + " \n" + " \n" + " \n" + " \n" + " \n" + " \n" + " \n" + " \n" + " \n" + " \n" + " \n" + " \n" + " \n" + " \n" + " \n" + " \n" + " \n" + " \n" + " \n" + " \n" + " \n" + " \n" + " \n" + " \n" + " \n" + " \n" + " \n" + " \n" + "\n" )); Just spotted a minor typo in ProblemList.txt: exclued -> excluded Thanks, Joe On 11/5/2014 1:12 AM, Frank Yuan wrote: > Hi, Joe and All > > Thanks for your previous reviews and comments. Per your comments: I added > description for every test, got them running with and without security > manager and fixed the known test issues. Could you help review the changes > for JAXP unittest co-location again? After your review, I will push > the tests into JDK repo: jaxp/test. > > bug: https://bugs.openjdk.java.net/browse/JDK-8043090 > webrev: http://sqeweb.us.oracle.com/jsn/users/yc/webrev/ > test results: > http://sqeweb.us.oracle.com/jsn/users/yc/result/JTreport/html/index.ht > ml > > Thanks, > > Frank > From peter.levart at gmail.com Thu Nov 6 07:36:27 2014 From: peter.levart at gmail.com (Peter Levart) Date: Thu, 06 Nov 2014 08:36:27 +0100 Subject: RFR: 8061950: Class.getMethods() exhibits quadratic time complexity In-Reply-To: References: <900D2AE3-B3A6-4B1B-B25A-C564B10334AA@oracle.com> <54492255.3040108@gmail.com> <544D58D4.7020909@gmail.com> <544D7B74.5060607@gmail.com> <5450CEE9.7060707@gmail.com> <545120D2.5010301@gmail.com> <54526D10.5060101@gmail.com> <54551D2F.407@gmail.com> <545A572D.3050608@gmail.com> Message-ID: <545B24FB.9020002@gmail.com> Hi Stanimir, On 11/05/2014 07:27 PM, Stanimir Simeonoff wrote: > A very small issue: > > MethodTable.java: > > 697 while (i < hwm && next == null) next = methods[i++]; > > > In my opinion the condition should be while (next == null && i < hwm) as next > will be non-null on each invocation of next() and generally it's more > likely to exit the loop. You're right, the conditions order should be reversed as the (i < hwm) is less likely than (next == null). > I suppose HashMapImpl is unused . That's right. If HashArrayImpl turns out to be an acceptable solution, I'll remove HashMapImpl. > > HashArray.java: > 82 Integer.highestOneBit(expectedMaxSize + (expectedMaxSize << 1)) > > > This effectively multiples expectedMaxSize by 3, I guess you meant >> > instead. The intention of capacity(int expectedMaxSize) is to return the smallest power of 2 number that is greater than 3 * expectedMaxSize / 2. The method is copied from IdentityHashMap, so I'm sure it does it right ;-) > Perhaps extra a method about effective capacity? Don't know what you mean by effective capacity? The capacity() is meant to size the initial array. And it behaves correctly even for large requested expectedMaxSize values that would cause an overflow in naively written code. > > I will take a deeper look a bit later. Thanks. Regards, Peter > > Stanimir > > > On Wed, Nov 5, 2014 at 6:58 PM, Peter Levart wrote: > >> On 11/01/2014 06:49 PM, Peter Levart wrote: >> >>> On 10/30/2014 07:13 PM, Martin Buchholz wrote: Which is why I was >>>> expecting to have to write an adaptive >>>> data structure that switches from linear search to hash-based when >>>> some threshold is exceeded. >>>> >>> ...or two data structures. Since I have already arranged so that >>> construction of MethodTable gets an upper limit on the total number of >>> possible Method(s) that can be added to it, we can have two MethodTable >>> implementations and switch based on this number: >>> >>> http://cr.openjdk.java.net/~plevart/jdk9-dev/Class.getMethods/webrev.05/ >>> >> Hi, >> >> I made some progress on this front. I created new MethodTable >> implementation (the third one) with similar performance as HashMap based, >> but producing half the garbage. I also added some optimizations to >> getMethod(), getMethods() and MethodTable in general that deal with special >> cases / parameter types comparison and have measurable impact on >> performance (for example loading all rt.jar classes and calling >> getMethods() for them): >> >> Original: >> >> 19658 classes loaded in 2.003623648 seconds. >> 494392 methods obtained in 0.932151066 seconds. >> 494392 methods obtained in 0.968247076 seconds. >> 494392 methods obtained in 0.989218185 seconds. >> 494392 methods obtained in 1.018700179 seconds. >> 494392 methods obtained in 0.945078767 seconds. >> 494392 methods obtained in 0.930856897 seconds. >> 494392 methods obtained in 0.905420555 seconds. >> 494392 methods obtained in 0.89253006 seconds. >> 494392 methods obtained in 0.914590141 seconds. >> 494392 methods obtained in 0.891616716 seconds. >> 494392 methods obtained in 0.891839815 seconds. >> 494392 methods obtained in 0.892863985 seconds. >> 494392 methods obtained in 0.960643274 seconds. >> 494392 methods obtained in 0.959266392 seconds. >> 494392 methods obtained in 0.894688322 seconds. >> 494392 methods obtained in 0.892751891 seconds. >> 494392 methods obtained in 0.912542838 seconds. >> 494392 methods obtained in 0.912219636 seconds. >> 494392 methods obtained in 0.895791468 seconds. >> 494392 methods obtained in 0.891673959 seconds. >> >> Patched: >> >> 19658 classes loaded in 2.145270948 seconds. >> 494398 methods obtained in 0.722630874 seconds. >> 494398 methods obtained in 0.697521755 seconds. >> 494398 methods obtained in 0.689642554 seconds. >> 494398 methods obtained in 0.742724992 seconds. >> 494398 methods obtained in 0.695693313 seconds. >> 494398 methods obtained in 0.685169108 seconds. >> 494398 methods obtained in 0.663012634 seconds. >> 494398 methods obtained in 0.666446935 seconds. >> 494398 methods obtained in 0.675662884 seconds. >> 494398 methods obtained in 0.65369404 seconds. >> 494398 methods obtained in 0.656608178 seconds. >> 494398 methods obtained in 0.668384051 seconds. >> 494398 methods obtained in 0.657548957 seconds. >> 494398 methods obtained in 0.672332234 seconds. >> 494398 methods obtained in 0.655699295 seconds. >> 494398 methods obtained in 0.671819628 seconds. >> 494398 methods obtained in 0.657232183 seconds. >> 494398 methods obtained in 0.654462137 seconds. >> 494398 methods obtained in 0.659630473 seconds. >> 494398 methods obtained in 0.674219391 seconds. >> >> (the difference in method count is expected - patched code contains new >> methods in existing rt.jar classes ;-) >> >> Here's new webrev: >> >> http://cr.openjdk.java.net/~plevart/jdk9-dev/Class.getMethods/webrev.06/ >> >> >> The optimizations made from webrev.05 are: >> >> - getMethod() skips construction of MethodTable if there are no >> (super)interfaces. >> - getMethods() returns just declared public methods if there are no >> superclass and no (super)interfaces. >> - comparing method parameter types is optimized by adding two methods to >> Method/LangReflectAccess/ReflectionFactory. >> >> New MethodTable implementation based on a linear-probe hash table is a >> space/garbage improvement. I took IdentityHashMap, removed unneeded stuff >> and modified it's API. The result is a HashArray. It's API is similar in >> function and form to java.util.Map, but doesn't use separate keys and >> values. An element of HashArray is a key and a value at the same time. >> Elements are always non-null, so the method return values are unambiguous. >> As HashArray is a linear-probe hash table and there are no Map.Entry >> objects involved, the underlying data structure is very simple and memory >> efficient. It is just a sparse array of elements with length that is always >> a power of two and larger than 3 * size / 2. It also features overriddable >> element equals/hashCode methods. I made it a separate generic class because >> I think it can find it's usage elsewhere (for example as a cannonicalizing >> cache). >> >> Since HashArray based MethodTable is more space-efficient I moved the line >> between simple array based and HashArray based MethodTable down to 20 >> elements to minimize the worst-case scenario effect. Calling getMethods() >> on all rt.jar classes now constructs about 3/4 simple array based and 1/4 >> HashArray based MethodTables. >> >> Here's also Martin's ManyMethodsBenchmark: >> >> Original: >> >> Base class load time: 129.95 ms >> getDeclaredMethods: 65521 methods, 36.58 ms total time, 0.0006 ms per >> method >> getMethods : 65530 methods, 47.43 ms total time, 0.0007 ms per >> method >> Derived class load time: 32216.09 ms >> getDeclaredMethods: 65521 methods, 35.05 ms total time, 0.0005 ms per >> method >> getMethods : 65530 methods, 8068.66 ms total time, 0.1231 ms per >> method >> >> >> Patched (using HashArray based MethodTable): >> >> Base class load time: 126.00 ms >> getDeclaredMethods: 65521 methods, 36.83 ms total time, 0.0006 ms per >> method >> getMethods : 65530 methods, 45.08 ms total time, 0.0007 ms per >> method >> Derived class load time: 31865.27 ms >> getDeclaredMethods: 65521 methods, 35.01 ms total time, 0.0005 ms per >> method >> getMethods : 65530 methods, 78.05 ms total time, 0.0012 ms per >> method >> >> >> All 86 jtreg test in java.lang/Class/ and java/lang/reflect/ still pass. >> >> >> Regards, Peter >> >> From staffan.friberg at oracle.com Thu Nov 6 10:18:15 2014 From: staffan.friberg at oracle.com (Staffan Friberg) Date: Thu, 06 Nov 2014 11:18:15 +0100 Subject: RFR JDK-6321472: Add CRC-32C API In-Reply-To: <5446DB24.4000507@oracle.com> References: <544073EA.3060304@oracle.com> <5440F803.2000406@oracle.com> <544166B8.8090103@oracle.com> <5441A192.8090602@gmail.com> <544697D2.6080301@oracle.com> <5446AAB0.3090304@oracle.com> <5446B551.7050605@oracle.com> <5446BDE1.9090709@oracle.com> <5446CB91.3050503@oracle.com> <5446DB24.4000507@oracle.com> Message-ID: <545B4AE7.4000208@oracle.com> Anyone have time to be the second reviewer? http://cr.openjdk.java.net/~sfriberg/JDK-6321472/webrev.07 The CCC request has been approved for the new API. Regards; Staffan On 10/22/2014 12:16 AM, Staffan Friberg wrote: > Thanks for the review. Updated the @implSpec. > > Also includes Peter's good catch. > > Webrev: http://cr.openjdk.java.net/~sfriberg/JDK-6321472/webrev.06 > > //Staffan > > On 10/21/2014 02:09 PM, Xueming Shen wrote: >> Staffan, >> >> Thanks for the package.html update. >> >> Just wonder if it would be better to use >> >> buffer.remaining(), instead of the buffer.limit() - >> buffer.position() >> >> in Checksum.udpate(ByteBuffer)'s #implSpec >> >> The rest looks fine for me. >> >> -Sherman >> >> On 10/21/2014 01:11 PM, Staffan Friberg wrote: >>> Converted. >>> >>> http://cr.openjdk.java.net/~sfriberg/JDK-6321472/webrev.05 >>> >>> //Staffan >>> >>> On 10/21/2014 12:34 PM, Joe Darcy wrote: >>>> Hi Staffan, >>>> >>>> If you are updating package.html, please also hg mv the file to be >>>> a package-info.java file with the equivalent javadoc. >>>> >>>> Thanks, >>>> >>>> -Joe >>>> >>>> On 10/21/2014 11:49 AM, Staffan Friberg wrote: >>>>> Hi, >>>>> >>>>> Got an offline comment that the package.html should be update as >>>>> well to cover CRC-32C. >>>>> >>>>> Otherwise there are no code changes in this new webrev. >>>>> >>>>> http://cr.openjdk.java.net/~sfriberg/JDK-6321472/webrev.04 >>>>> >>>>> //Staffan >>>>> >>>>> On 10/21/2014 10:28 AM, Staffan Friberg wrote: >>>>>> Hi Peter, >>>>>> >>>>>> Thanks for the comments.. >>>>>>> >>>>>>> 217 if (Unsafe.ADDRESS_SIZE == 4) { >>>>>>> 218 // On 32 bit platforms read two ints >>>>>>> instead of a single 64bit long >>>>>>> >>>>>>> When you're reading from byte[] using Unsafe (updateBytes), you >>>>>>> have the option of reading 64bit values on 64bit platforms. When >>>>>>> you're reading from DirectByteBuffer memory >>>>>>> (updateDirectByteBuffer), you're only using 32bit reads. >>>>>> I will add a comment in the code for this decision. The reason is >>>>>> that read a long results in slightly worse performance in this >>>>>> case, in updateBytes it is faster. I was able to get it to run >>>>>> slightly faster by working directly with the address instead of >>>>>> always adding address + off, but this makes things worse in the >>>>>> 32bit case since all calculation will now be using long >>>>>> variables. So using the getInt as in the current code feels like >>>>>> the best solution as it strikes the best balance between 32 and >>>>>> 64bit. Below is how updateByteBuffer looked with the rewrite I >>>>>> mentioned. >>>>>> >>>>>> >>>>>> ong address = ((DirectBuffer) buffer).address(); >>>>>> crc = updateDirectByteBuffer(crc, address + pos, address + limit); >>>>>> >>>>>> >>>>>> private static int updateDirectByteBuffer(int crc, long adr, >>>>>> long end) { >>>>>> >>>>>> // Do only byte reads for arrays so short they can't be >>>>>> aligned >>>>>> if (end - adr >= 8) { >>>>>> >>>>>> // align on 8 bytes >>>>>> int alignLength = (8 - (int) (adr & 0x7)) & 0x7; >>>>>> for (long alignEnd = adr + alignLength; adr < >>>>>> alignEnd; adr++) { >>>>>> crc = (crc >>> 8) >>>>>> ^ byteTable[(crc ^ UNSAFE.getByte(adr)) & >>>>>> 0xFF]; >>>>>> } >>>>>> >>>>>> if (ByteOrder.nativeOrder() == ByteOrder.BIG_ENDIAN) { >>>>>> crc = Integer.reverseBytes(crc); >>>>>> } >>>>>> >>>>>> // slicing-by-8 >>>>>> for (; adr < (end - Long.BYTES); adr += Long.BYTES) { >>>>>> int firstHalf; >>>>>> int secondHalf; >>>>>> if (Unsafe.ADDRESS_SIZE == 4) { >>>>>> // On 32 bit platforms read two ints instead >>>>>> of a single 64bit long >>>>>> firstHalf = UNSAFE.getInt(adr); >>>>>> secondHalf = UNSAFE.getInt(adr + Integer.BYTES); >>>>>> } else { >>>>>> long value = UNSAFE.getLong(adr); >>>>>> if (ByteOrder.nativeOrder() == >>>>>> ByteOrder.LITTLE_ENDIAN) { >>>>>> firstHalf = (int) value; >>>>>> secondHalf = (int) (value >>> 32); >>>>>> } else { // ByteOrder.BIG_ENDIAN >>>>>> firstHalf = (int) (value >>> 32); >>>>>> secondHalf = (int) value; >>>>>> } >>>>>> } >>>>>> crc ^= firstHalf; >>>>>> if (ByteOrder.nativeOrder() == >>>>>> ByteOrder.LITTLE_ENDIAN) { >>>>>> crc = byteTable7[crc & 0xFF] >>>>>> ^ byteTable6[(crc >>> 8) & 0xFF] >>>>>> ^ byteTable5[(crc >>> 16) & 0xFF] >>>>>> ^ byteTable4[crc >>> 24] >>>>>> ^ byteTable3[secondHalf & 0xFF] >>>>>> ^ byteTable2[(secondHalf >>> 8) & 0xFF] >>>>>> ^ byteTable1[(secondHalf >>> 16) & 0xFF] >>>>>> ^ byteTable0[secondHalf >>> 24]; >>>>>> } else { // ByteOrder.BIG_ENDIAN >>>>>> crc = byteTable0[secondHalf & 0xFF] >>>>>> ^ byteTable1[(secondHalf >>> 8) & 0xFF] >>>>>> ^ byteTable2[(secondHalf >>> 16) & 0xFF] >>>>>> ^ byteTable3[secondHalf >>> 24] >>>>>> ^ byteTable4[crc & 0xFF] >>>>>> ^ byteTable5[(crc >>> 8) & 0xFF] >>>>>> ^ byteTable6[(crc >>> 16) & 0xFF] >>>>>> ^ byteTable7[crc >>> 24]; >>>>>> } >>>>>> } >>>>>> >>>>>> if (ByteOrder.nativeOrder() == ByteOrder.BIG_ENDIAN) { >>>>>> crc = Integer.reverseBytes(crc); >>>>>> } >>>>>> } >>>>>> >>>>>> // Tail >>>>>> for (; adr < end; adr++) { >>>>>> crc = (crc >>> 8) >>>>>> ^ byteTable[(crc ^ UNSAFE.getByte(adr)) & 0xFF]; >>>>>> } >>>>>> >>>>>> return crc; >>>>>> } >>>>>> >>>>>> >>>>>>> >>>>>>> Also, in updateBytes, the usage of >>>>>>> Unsafe.ARRAY_INT_INDEX_SCALE/ARRAY_LONG_INDEX_SCALE to index a >>>>>>> byte array sounds a little scary. To be ultra portable you could >>>>>>> check that ARRAY_BYTE_INDEX_SCALE == 1 first and refuse to use >>>>>>> Unsafe for byte arrays if it is not 1. Then use >>>>>>> Integer.BYTES/Long.BYTES to manipulate 'offsets' instead. In >>>>>>> updateDirectByteBuffer it would be more appropriate to use >>>>>>> Integer.BYTES/Long.BYTES too. >>>>>> Good idea. Added a check in the initial if statement and it will >>>>>> get automatically optimized away. >>>>>> >>>>>>> 225 firstHalf = (int) (value & 0xFFFFFFFF); >>>>>>> 226 secondHalf = (int) (value >>> 32); >>>>>>> 227 } else { // ByteOrder.BIG_ENDIAN >>>>>>> 228 firstHalf = (int) (value >>> 32); >>>>>>> 229 secondHalf = (int) (value & >>>>>>> 0xFFFFFFFF); >>>>>>> >>>>>>> firstHalf = (int) value; // this is equivalent for line 225 >>>>>>> secondHalf = (int) value; // this is equivalent for line 229 >>>>>> Done. >>>>>> >>>>>> Here is the latest webrev, >>>>>> http://cr.openjdk.java.net/~sfriberg/JDK-6321472/webrev.03 >>>>>> >>>>>> Cheers, >>>>>> Staffan >>>>> >>>> >>> >> > From konstantin.shefov at oracle.com Thu Nov 6 10:44:55 2014 From: konstantin.shefov at oracle.com (Konstantin Shefov) Date: Thu, 06 Nov 2014 13:44:55 +0300 Subject: [9] Review request : JDK-8062904: TEST_BUG: Tests java/lang/invoke/LFCaching fail when run with -Xcomp option Message-ID: <545B5127.5030903@oracle.com> Hello, Please review the test bug fix https://bugs.openjdk.java.net/browse/JDK-8062904 Webrev is http://cr.openjdk.java.net/~kshefov/8062904/webrev.00/ Thanks -Konstantin From joel.franck at oracle.com Thu Nov 6 10:48:18 2014 From: joel.franck at oracle.com (=?windows-1252?Q?Joel_Borggr=E9n-Franck?=) Date: Thu, 6 Nov 2014 11:48:18 +0100 Subject: RFR: 8062771: Core reflection should use final fields whenever possible In-Reply-To: <843282E1-DC2B-48B0-B0E4-A60B5078D75A@oracle.com> References: <843282E1-DC2B-48B0-B0E4-A60B5078D75A@oracle.com> Message-ID: Hi, I?m having a hard time following this thread, which webrev has been updated, for which release, fixing which issue? Martin, as far as I can see you are the only one of us who has replied to this thread who is a jdk7u committer (or reviewer). I think the assumption is that generic repositories of different kinds are ok with multiple initialization, but maybe we need to revisit this? cheers /Joel On 5 nov 2014, at 10:06, Paul Sandoz wrote: > On Nov 4, 2014, at 6:53 PM, Martin Buchholz wrote: >>> Is it possible to make EmptyClass be a static inner class of the test? IIUC if its enclosed within ThreadSafety it should still work with URLClassLoader. >> >> Done. Webrev regenerated. > > +1 > > Perhaps one reason why the test is not failing (on 7) is because certain code is not getting compiled/inlined, since there are not enough iterations. If you think it worth obsessing over you could shuffle the code a bit, increase the iterations of the inner loop, and use some hotspot compiler command options to force inline and/or compile. > > >> I recommend y'all do a little correctness backport to jdk7 Class.java >> as follows: >> >> @@ -2450,7 +2450,7 @@ >> private native String getGenericSignature(); >> >> // Generic info repository; lazily initialized >> - private transient ClassRepository genericInfo; >> + private volatile transient ClassRepository genericInfo; >> >> // accessor for factory >> private GenericsFactory getFactory() { >> @@ -2460,11 +2460,13 @@ >> >> // accessor for generic info repository >> private ClassRepository getGenericInfo() { >> + ClassRepository genericInfo = this.genericInfo; >> // lazily initialize repository if necessary >> if (genericInfo == null) { >> // create and cache generic info repository >> genericInfo = ClassRepository.make(getGenericSignature(), >> getFactory()); >> + this.genericInfo = genericInfo; >> } >> return genericInfo; //return cached repository >> } >> > > Agreed. > > > On Nov 4, 2014, at 9:34 PM, Aleksey Shipilev wrote: >> I think we can make a partial backport of JDK-8016236 to jdk7, since >> this volatile construction fixes the obvious bug, and the fix was tested >> already in JDK 8 and 9. Can anyone from Core Libraries team attend to this? > > I am knuckling down into conference development mode this and next week for Devoxx, if no one jumps on it sooner i can sort it out later on. > > Paul. From stanimir at riflexo.com Thu Nov 6 11:10:06 2014 From: stanimir at riflexo.com (Stanimir Simeonoff) Date: Thu, 6 Nov 2014 13:10:06 +0200 Subject: RFR: 8061950: Class.getMethods() exhibits quadratic time complexity In-Reply-To: <545B24FB.9020002@gmail.com> References: <900D2AE3-B3A6-4B1B-B25A-C564B10334AA@oracle.com> <54492255.3040108@gmail.com> <544D58D4.7020909@gmail.com> <544D7B74.5060607@gmail.com> <5450CEE9.7060707@gmail.com> <545120D2.5010301@gmail.com> <54526D10.5060101@gmail.com> <54551D2F.407@gmail.com> <545A572D.3050608@gmail.com> <545B24FB.9020002@gmail.com> Message-ID: > >> HashArray.java: >> 82 Integer.highestOneBit(expectedMaxSize + >> (expectedMaxSize << 1)) >> >> >> This effectively multiples expectedMaxSize by 3, I guess you meant >> >> instead. >> > > The intention of capacity(int expectedMaxSize) is to return the smallest > power of 2 number that is greater than 3 * expectedMaxSize / 2. > The method is copied from IdentityHashMap, so I'm sure it does it right ;-) > > Indeed it does so, it can lead to almost 3x times (i.e. fill factor of .33) waste for some numbers (11, 22, 43) . But I supposed it's a good tradeoff. > Perhaps extra a method about effective capacity? >> > > Don't know what you mean by effective capacity? The capacity() is meant to > size the initial array. And it behaves correctly even for large requested > expectedMaxSize values that would cause an overflow in naively written code. > > I meant to "extract" (not extra) a method for s + (s >> 1) but it's used once only, so please disregard this part. There is some code duplication in the different impl. of consolidate() but trying to refactor it (in a static method) would cause some performance degradation (I don't think there is an intrinsic for the native Class.isInterface()). So, I guess it can be left that way, as it's not too much. Stanimir >> On Wed, Nov 5, 2014 at 6:58 PM, Peter Levart >> wrote: >> >> On 11/01/2014 06:49 PM, Peter Levart wrote: >>> >>> On 10/30/2014 07:13 PM, Martin Buchholz wrote: Which is why I was >>>> >>>>> expecting to have to write an adaptive >>>>> data structure that switches from linear search to hash-based when >>>>> some threshold is exceeded. >>>>> >>>>> ...or two data structures. Since I have already arranged so that >>>> construction of MethodTable gets an upper limit on the total number of >>>> possible Method(s) that can be added to it, we can have two MethodTable >>>> implementations and switch based on this number: >>>> >>>> http://cr.openjdk.java.net/~plevart/jdk9-dev/Class.getMethods/webrev.05/ >>>> >>>> Hi, >>> >>> I made some progress on this front. I created new MethodTable >>> implementation (the third one) with similar performance as HashMap based, >>> but producing half the garbage. I also added some optimizations to >>> getMethod(), getMethods() and MethodTable in general that deal with >>> special >>> cases / parameter types comparison and have measurable impact on >>> performance (for example loading all rt.jar classes and calling >>> getMethods() for them): >>> >>> Original: >>> >>> 19658 classes loaded in 2.003623648 seconds. >>> 494392 methods obtained in 0.932151066 seconds. >>> 494392 methods obtained in 0.968247076 seconds. >>> 494392 methods obtained in 0.989218185 seconds. >>> 494392 methods obtained in 1.018700179 seconds. >>> 494392 methods obtained in 0.945078767 seconds. >>> 494392 methods obtained in 0.930856897 seconds. >>> 494392 methods obtained in 0.905420555 seconds. >>> 494392 methods obtained in 0.89253006 seconds. >>> 494392 methods obtained in 0.914590141 seconds. >>> 494392 methods obtained in 0.891616716 seconds. >>> 494392 methods obtained in 0.891839815 seconds. >>> 494392 methods obtained in 0.892863985 seconds. >>> 494392 methods obtained in 0.960643274 seconds. >>> 494392 methods obtained in 0.959266392 seconds. >>> 494392 methods obtained in 0.894688322 seconds. >>> 494392 methods obtained in 0.892751891 seconds. >>> 494392 methods obtained in 0.912542838 seconds. >>> 494392 methods obtained in 0.912219636 seconds. >>> 494392 methods obtained in 0.895791468 seconds. >>> 494392 methods obtained in 0.891673959 seconds. >>> >>> Patched: >>> >>> 19658 classes loaded in 2.145270948 seconds. >>> 494398 methods obtained in 0.722630874 seconds. >>> 494398 methods obtained in 0.697521755 seconds. >>> 494398 methods obtained in 0.689642554 seconds. >>> 494398 methods obtained in 0.742724992 seconds. >>> 494398 methods obtained in 0.695693313 seconds. >>> 494398 methods obtained in 0.685169108 seconds. >>> 494398 methods obtained in 0.663012634 seconds. >>> 494398 methods obtained in 0.666446935 seconds. >>> 494398 methods obtained in 0.675662884 seconds. >>> 494398 methods obtained in 0.65369404 seconds. >>> 494398 methods obtained in 0.656608178 seconds. >>> 494398 methods obtained in 0.668384051 seconds. >>> 494398 methods obtained in 0.657548957 seconds. >>> 494398 methods obtained in 0.672332234 seconds. >>> 494398 methods obtained in 0.655699295 seconds. >>> 494398 methods obtained in 0.671819628 seconds. >>> 494398 methods obtained in 0.657232183 seconds. >>> 494398 methods obtained in 0.654462137 seconds. >>> 494398 methods obtained in 0.659630473 seconds. >>> 494398 methods obtained in 0.674219391 seconds. >>> >>> (the difference in method count is expected - patched code contains new >>> methods in existing rt.jar classes ;-) >>> >>> Here's new webrev: >>> >>> http://cr.openjdk.java.net/~plevart/jdk9-dev/Class.getMethods/webrev.06/ >>> >>> >>> The optimizations made from webrev.05 are: >>> >>> - getMethod() skips construction of MethodTable if there are no >>> (super)interfaces. >>> - getMethods() returns just declared public methods if there are no >>> superclass and no (super)interfaces. >>> - comparing method parameter types is optimized by adding two methods to >>> Method/LangReflectAccess/ReflectionFactory. >>> >>> New MethodTable implementation based on a linear-probe hash table is a >>> space/garbage improvement. I took IdentityHashMap, removed unneeded stuff >>> and modified it's API. The result is a HashArray. It's API is similar in >>> function and form to java.util.Map, but doesn't use separate keys and >>> values. An element of HashArray is a key and a value at the same time. >>> Elements are always non-null, so the method return values are >>> unambiguous. >>> As HashArray is a linear-probe hash table and there are no Map.Entry >>> objects involved, the underlying data structure is very simple and memory >>> efficient. It is just a sparse array of elements with length that is >>> always >>> a power of two and larger than 3 * size / 2. It also features >>> overriddable >>> element equals/hashCode methods. I made it a separate generic class >>> because >>> I think it can find it's usage elsewhere (for example as a >>> cannonicalizing >>> cache). >>> >>> Since HashArray based MethodTable is more space-efficient I moved the >>> line >>> between simple array based and HashArray based MethodTable down to 20 >>> elements to minimize the worst-case scenario effect. Calling getMethods() >>> on all rt.jar classes now constructs about 3/4 simple array based and 1/4 >>> HashArray based MethodTables. >>> >>> Here's also Martin's ManyMethodsBenchmark: >>> >>> Original: >>> >>> Base class load time: 129.95 ms >>> getDeclaredMethods: 65521 methods, 36.58 ms total time, 0.0006 ms per >>> method >>> getMethods : 65530 methods, 47.43 ms total time, 0.0007 ms per >>> method >>> Derived class load time: 32216.09 ms >>> getDeclaredMethods: 65521 methods, 35.05 ms total time, 0.0005 ms per >>> method >>> getMethods : 65530 methods, 8068.66 ms total time, 0.1231 ms per >>> method >>> >>> >>> Patched (using HashArray based MethodTable): >>> >>> Base class load time: 126.00 ms >>> getDeclaredMethods: 65521 methods, 36.83 ms total time, 0.0006 ms per >>> method >>> getMethods : 65530 methods, 45.08 ms total time, 0.0007 ms per >>> method >>> Derived class load time: 31865.27 ms >>> getDeclaredMethods: 65521 methods, 35.01 ms total time, 0.0005 ms per >>> method >>> getMethods : 65530 methods, 78.05 ms total time, 0.0012 ms per >>> method >>> >>> >>> All 86 jtreg test in java.lang/Class/ and java/lang/reflect/ still pass. >>> >>> >>> Regards, Peter >>> >>> >>> > From stanimir at riflexo.com Thu Nov 6 11:26:20 2014 From: stanimir at riflexo.com (Stanimir Simeonoff) Date: Thu, 6 Nov 2014 13:26:20 +0200 Subject: RFR JDK-6321472: Add CRC-32C API In-Reply-To: <545B4AE7.4000208@oracle.com> References: <544073EA.3060304@oracle.com> <5440F803.2000406@oracle.com> <544166B8.8090103@oracle.com> <5441A192.8090602@gmail.com> <544697D2.6080301@oracle.com> <5446AAB0.3090304@oracle.com> <5446B551.7050605@oracle.com> <5446BDE1.9090709@oracle.com> <5446CB91.3050503@oracle.com> <5446DB24.4000507@oracle.com> <545B4AE7.4000208@oracle.com> Message-ID: Hi Staffan, Given the tables in the static class init : 66 private transient final static int[] byteTable0 = byteTables[0]; 67 private transient final static int[] byteTable1 = byteTables[1]; 68 private transient final static int[] byteTable2 = byteTables[2]; 69 private transient final static int[] byteTable3 = byteTables[3]; 70 private transient final static int[] byteTable4 = byteTables[4]; 71 private transient final static int[] byteTable5 = byteTables[5]; 72 private transient final static int[] byteTable6 = byteTables[6]; 73 private transient final static int[] byteTable7 = byteTables[7]; I was wondering if assigning the tables in the static part in similar fashion would make the code much more readable: 66 private transient final static int[] byteTable0 = byteTables[idx(0)]; 67 private transient final static int[] byteTable1 = byteTables[idx(1)]; 68 private transient final static int[] byteTable2 = byteTables[idx(2)]; 69 private transient final static int[] byteTable3 = byteTables[idx(3)]; 70 private transient final static int[] byteTable4 = byteTables[idx(4(]; 71 private transient final static int[] byteTable5 = byteTables[idx(5)]; 72 private transient final static int[] byteTable6 = byteTables[idx(6)]; 73 private transient final static int[] byteTable7 = byteTables[idx(7)]; private static int idx(int i){ return ByteOrder.nativeOrder() == ByteOrder.LITTLE_ENDIAN?7-i:i;}; Then in the loops you can skip checking the condition, something like: 296- if (ByteOrder.nativeOrder() == ByteOrder.LITTLE_ENDIAN) { 297- crc = byteTable7[crc & 0xFF] 298- ^ byteTable6[(crc >>> 8) & 0xFF] 299- ^ byteTable5[(crc >>> 16) & 0xFF] 300- ^ byteTable4[crc >>> 24] 301- ^ byteTable3[secondHalf & 0xFF] 302- ^ byteTable2[(secondHalf >>> 8) & 0xFF] 303- ^ byteTable1[(secondHalf >>> 16) & 0xFF] 304- ^ byteTable0[secondHalf >>> 24]; 305- } else { // ByteOrder.BIG_ENDIAN 306 crc = byteTable0[secondHalf & 0xFF] 307 ^ byteTable1[(secondHalf >>> 8) & 0xFF] 308 ^ byteTable2[(secondHalf >>> 16) & 0xFF] 309 ^ byteTable3[secondHalf >>> 24] 310 ^ byteTable4[crc & 0xFF] 311 ^ byteTable5[(crc >>> 8) & 0xFF] 312 ^ byteTable6[(crc >>> 16) & 0xFF] 313 ^ byteTable7[crc >>> 24]; 314 - } Since the byteTabeleN are properly assigned during the class init based on the endianess of the systen. Of course, there won't be any performance benefits since hotspot should constant fold the code but it'll be easier to maintain and read. Cheers Stanimir On Thu, Nov 6, 2014 at 12:18 PM, Staffan Friberg wrote: > Anyone have time to be the second reviewer? > > http://cr.openjdk.java.net/~sfriberg/JDK-6321472/webrev.07 > > The CCC request has been approved for the new API. > > Regards; > Staffan > > > On 10/22/2014 12:16 AM, Staffan Friberg wrote: > >> Thanks for the review. Updated the @implSpec. >> >> Also includes Peter's good catch. >> >> Webrev: http://cr.openjdk.java.net/~sfriberg/JDK-6321472/webrev.06 >> >> //Staffan >> >> On 10/21/2014 02:09 PM, Xueming Shen wrote: >> >>> Staffan, >>> >>> Thanks for the package.html update. >>> >>> Just wonder if it would be better to use >>> >>> buffer.remaining(), instead of the buffer.limit() - buffer.position() >>> >>> in Checksum.udpate(ByteBuffer)'s #implSpec >>> >>> The rest looks fine for me. >>> >>> -Sherman >>> >>> On 10/21/2014 01:11 PM, Staffan Friberg wrote: >>> >>>> Converted. >>>> >>>> http://cr.openjdk.java.net/~sfriberg/JDK-6321472/webrev.05 >>>> >>>> //Staffan >>>> >>>> On 10/21/2014 12:34 PM, Joe Darcy wrote: >>>> >>>>> Hi Staffan, >>>>> >>>>> If you are updating package.html, please also hg mv the file to be a >>>>> package-info.java file with the equivalent javadoc. >>>>> >>>>> Thanks, >>>>> >>>>> -Joe >>>>> >>>>> On 10/21/2014 11:49 AM, Staffan Friberg wrote: >>>>> >>>>>> Hi, >>>>>> >>>>>> Got an offline comment that the package.html should be update as well >>>>>> to cover CRC-32C. >>>>>> >>>>>> Otherwise there are no code changes in this new webrev. >>>>>> >>>>>> http://cr.openjdk.java.net/~sfriberg/JDK-6321472/webrev.04 >>>>>> >>>>>> //Staffan >>>>>> >>>>>> On 10/21/2014 10:28 AM, Staffan Friberg wrote: >>>>>> >>>>>>> Hi Peter, >>>>>>> >>>>>>> Thanks for the comments.. >>>>>>> >>>>>>>> >>>>>>>> 217 if (Unsafe.ADDRESS_SIZE == 4) { >>>>>>>> 218 // On 32 bit platforms read two ints >>>>>>>> instead of a single 64bit long >>>>>>>> >>>>>>>> When you're reading from byte[] using Unsafe (updateBytes), you >>>>>>>> have the option of reading 64bit values on 64bit platforms. When you're >>>>>>>> reading from DirectByteBuffer memory (updateDirectByteBuffer), you're only >>>>>>>> using 32bit reads. >>>>>>>> >>>>>>> I will add a comment in the code for this decision. The reason is >>>>>>> that read a long results in slightly worse performance in this case, in >>>>>>> updateBytes it is faster. I was able to get it to run slightly faster by >>>>>>> working directly with the address instead of always adding address + off, >>>>>>> but this makes things worse in the 32bit case since all calculation will >>>>>>> now be using long variables. So using the getInt as in the current code >>>>>>> feels like the best solution as it strikes the best balance between 32 and >>>>>>> 64bit. Below is how updateByteBuffer looked with the rewrite I mentioned. >>>>>>> >>>>>>> >>>>>>> ong address = ((DirectBuffer) buffer).address(); >>>>>>> crc = updateDirectByteBuffer(crc, address + pos, address + limit); >>>>>>> >>>>>>> >>>>>>> private static int updateDirectByteBuffer(int crc, long adr, >>>>>>> long end) { >>>>>>> >>>>>>> // Do only byte reads for arrays so short they can't be >>>>>>> aligned >>>>>>> if (end - adr >= 8) { >>>>>>> >>>>>>> // align on 8 bytes >>>>>>> int alignLength = (8 - (int) (adr & 0x7)) & 0x7; >>>>>>> for (long alignEnd = adr + alignLength; adr < alignEnd; >>>>>>> adr++) { >>>>>>> crc = (crc >>> 8) >>>>>>> ^ byteTable[(crc ^ UNSAFE.getByte(adr)) & >>>>>>> 0xFF]; >>>>>>> } >>>>>>> >>>>>>> if (ByteOrder.nativeOrder() == ByteOrder.BIG_ENDIAN) { >>>>>>> crc = Integer.reverseBytes(crc); >>>>>>> } >>>>>>> >>>>>>> // slicing-by-8 >>>>>>> for (; adr < (end - Long.BYTES); adr += Long.BYTES) { >>>>>>> int firstHalf; >>>>>>> int secondHalf; >>>>>>> if (Unsafe.ADDRESS_SIZE == 4) { >>>>>>> // On 32 bit platforms read two ints instead of >>>>>>> a single 64bit long >>>>>>> firstHalf = UNSAFE.getInt(adr); >>>>>>> secondHalf = UNSAFE.getInt(adr + Integer.BYTES); >>>>>>> } else { >>>>>>> long value = UNSAFE.getLong(adr); >>>>>>> if (ByteOrder.nativeOrder() == >>>>>>> ByteOrder.LITTLE_ENDIAN) { >>>>>>> firstHalf = (int) value; >>>>>>> secondHalf = (int) (value >>> 32); >>>>>>> } else { // ByteOrder.BIG_ENDIAN >>>>>>> firstHalf = (int) (value >>> 32); >>>>>>> secondHalf = (int) value; >>>>>>> } >>>>>>> } >>>>>>> crc ^= firstHalf; >>>>>>> if (ByteOrder.nativeOrder() == >>>>>>> ByteOrder.LITTLE_ENDIAN) { >>>>>>> crc = byteTable7[crc & 0xFF] >>>>>>> ^ byteTable6[(crc >>> 8) & 0xFF] >>>>>>> ^ byteTable5[(crc >>> 16) & 0xFF] >>>>>>> ^ byteTable4[crc >>> 24] >>>>>>> ^ byteTable3[secondHalf & 0xFF] >>>>>>> ^ byteTable2[(secondHalf >>> 8) & 0xFF] >>>>>>> ^ byteTable1[(secondHalf >>> 16) & 0xFF] >>>>>>> ^ byteTable0[secondHalf >>> 24]; >>>>>>> } else { // ByteOrder.BIG_ENDIAN >>>>>>> crc = byteTable0[secondHalf & 0xFF] >>>>>>> ^ byteTable1[(secondHalf >>> 8) & 0xFF] >>>>>>> ^ byteTable2[(secondHalf >>> 16) & 0xFF] >>>>>>> ^ byteTable3[secondHalf >>> 24] >>>>>>> ^ byteTable4[crc & 0xFF] >>>>>>> ^ byteTable5[(crc >>> 8) & 0xFF] >>>>>>> ^ byteTable6[(crc >>> 16) & 0xFF] >>>>>>> ^ byteTable7[crc >>> 24]; >>>>>>> } >>>>>>> } >>>>>>> >>>>>>> if (ByteOrder.nativeOrder() == ByteOrder.BIG_ENDIAN) { >>>>>>> crc = Integer.reverseBytes(crc); >>>>>>> } >>>>>>> } >>>>>>> >>>>>>> // Tail >>>>>>> for (; adr < end; adr++) { >>>>>>> crc = (crc >>> 8) >>>>>>> ^ byteTable[(crc ^ UNSAFE.getByte(adr)) & 0xFF]; >>>>>>> } >>>>>>> >>>>>>> return crc; >>>>>>> } >>>>>>> >>>>>>> >>>>>>> >>>>>>>> Also, in updateBytes, the usage of Unsafe.ARRAY_INT_INDEX_SCALE/ARRAY_LONG_INDEX_SCALE >>>>>>>> to index a byte array sounds a little scary. To be ultra portable you could >>>>>>>> check that ARRAY_BYTE_INDEX_SCALE == 1 first and refuse to use Unsafe for >>>>>>>> byte arrays if it is not 1. Then use Integer.BYTES/Long.BYTES to manipulate >>>>>>>> 'offsets' instead. In updateDirectByteBuffer it would be more appropriate >>>>>>>> to use Integer.BYTES/Long.BYTES too. >>>>>>>> >>>>>>> Good idea. Added a check in the initial if statement and it will get >>>>>>> automatically optimized away. >>>>>>> >>>>>>> 225 firstHalf = (int) (value & 0xFFFFFFFF); >>>>>>>> 226 secondHalf = (int) (value >>> 32); >>>>>>>> 227 } else { // ByteOrder.BIG_ENDIAN >>>>>>>> 228 firstHalf = (int) (value >>> 32); >>>>>>>> 229 secondHalf = (int) (value & >>>>>>>> 0xFFFFFFFF); >>>>>>>> >>>>>>>> firstHalf = (int) value; // this is equivalent for line 225 >>>>>>>> secondHalf = (int) value; // this is equivalent for line 229 >>>>>>>> >>>>>>> Done. >>>>>>> >>>>>>> Here is the latest webrev, http://cr.openjdk.java.net/~ >>>>>>> sfriberg/JDK-6321472/webrev.03 >>>>>>> >>>>>>> Cheers, >>>>>>> Staffan >>>>>>> >>>>>> >>>>>> >>>>> >>>> >>> >> > From andrej.golovnin at gmail.com Thu Nov 6 11:57:32 2014 From: andrej.golovnin at gmail.com (Andrej Golovnin) Date: Thu, 6 Nov 2014 12:57:32 +0100 Subject: RFR JDK-6321472: Add CRC-32C API In-Reply-To: <545B4AE7.4000208@oracle.com> References: <544073EA.3060304@oracle.com> <5440F803.2000406@oracle.com> <544166B8.8090103@oracle.com> <5441A192.8090602@gmail.com> <544697D2.6080301@oracle.com> <5446AAB0.3090304@oracle.com> <5446B551.7050605@oracle.com> <5446BDE1.9090709@oracle.com> <5446CB91.3050503@oracle.com> <5446DB24.4000507@oracle.com> <545B4AE7.4000208@oracle.com> Message-ID: Hi Staffan, I'm not a reviewer but I have small remarks to the code style: - the instance variable "crc" is declared bevor the class variables. I would move it closer to the constructor. - documentation comments should be used for the fields "crc" and "CRC32C_POLY", e.g. instead of // Calculated CRC-32C value private int crc = 0xFFFFFFFF; use this: /** * Calculated CRC-32C value */ private int crc = 0xFFFFFFFF; - the description of the implemented algorithm should use block comments and should be moved to the top of the class, e.g.: public final class CRC32C implements Checksum { /* * This CRC-32C implementation uses the 'slicing-by-8' algorithm described * in the paper "A Systematic Approach to Building High Performance * Software-Based CRC Generators" by Michael E. Kounavis and Frank L. Berry, * Intel Research and Development */ ... In other case I get the filling that the Usafe class implements the algorithm. :-) - why some of the fields declared as "private transient final static"? I would say "transient" is not needed here. Or am I missing something? And I would use "private static final". I think this is the preferred way in the JDK to declare a constant. 61 private transient final static Unsafe UNSAFE = Unsafe.getUnsafe(); 62 63 // Lookup tables 64 private transient final static int[][] byteTables = new int[8][256]; 65 private transient final static int[] byteTable; 66 private transient final static int[] byteTable0 = byteTables[0]; Best regards, Andrej Golovnin From igor.ignatyev at oracle.com Thu Nov 6 14:36:13 2014 From: igor.ignatyev at oracle.com (Igor Ignatyev) Date: Thu, 06 Nov 2014 17:36:13 +0300 Subject: [9] Review request : JDK-8062904: TEST_BUG: Tests java/lang/invoke/LFCaching fail when run with -Xcomp option In-Reply-To: <545B5127.5030903@oracle.com> References: <545B5127.5030903@oracle.com> Message-ID: <545B875D.1060405@oracle.com> Konstantin, please use 'java.vm.info' instead of iteration over flags. Igor On 11/06/2014 01:44 PM, Konstantin Shefov wrote: > Hello, > > Please review the test bug fix > https://bugs.openjdk.java.net/browse/JDK-8062904 > Webrev is http://cr.openjdk.java.net/~kshefov/8062904/webrev.00/ > > Thanks > > -Konstantin From staffan.friberg at oracle.com Thu Nov 6 15:03:50 2014 From: staffan.friberg at oracle.com (Staffan Friberg) Date: Thu, 06 Nov 2014 16:03:50 +0100 Subject: RFR JDK-6321472: Add CRC-32C API In-Reply-To: References: <544073EA.3060304@oracle.com> <5440F803.2000406@oracle.com> <544166B8.8090103@oracle.com> <5441A192.8090602@gmail.com> <544697D2.6080301@oracle.com> <5446AAB0.3090304@oracle.com> <5446B551.7050605@oracle.com> <5446BDE1.9090709@oracle.com> <5446CB91.3050503@oracle.com> <5446DB24.4000507@oracle.com> <545B4AE7.4000208@oracle.com> Message-ID: <545B8DD6.5030805@oracle.com> Hi Andrej, Thanks for your comments. New webrev, http://cr.openjdk.java.net/~sfriberg/JDK-6321472/webrev.08 Indeed more common :) jdk/src/java.base/share$ grep -R "private final static" *|wc -l 282 jdk/src/java.base/share$ grep -R "private static final" *|wc -l 3274 //Staffan On 11/06/2014 12:57 PM, Andrej Golovnin wrote: > Hi Staffan, > > I'm not a reviewer but I have small remarks to the code style: > > - the instance variable "crc" is declared bevor the class variables. I > would move it closer to the constructor. > > - documentation comments should be used for the fields "crc" and > "CRC32C_POLY", e.g. instead of > > // Calculated CRC-32C value > private int crc = 0xFFFFFFFF; > > use this: > > /** > * Calculated CRC-32C value > */ > private int crc = 0xFFFFFFFF; > > > - the description of the implemented algorithm should use block > comments and should be moved to the top of the class, e.g.: > > public final class CRC32C implements Checksum { > > /* > * This CRC-32C implementation uses the 'slicing-by-8' algorithm > described > * in the paper "A Systematic Approach to Building High Performance > * Software-Based CRC Generators" by Michael E. Kounavis and > Frank L. Berry, > * Intel Research and Development > */ > > ... > > In other case I get the filling that the Usafe class implements the > algorithm. :-) > > - why some of the fields declared as "private transient final static"? > I would say "transient" is not needed here. Or am I missing something? > And I would use "private static final". I think this is the preferred > way in the JDK to declare a constant. > > 61 private transient final static Unsafe UNSAFE = > Unsafe.getUnsafe(); > 62 > 63 // Lookup tables > 64 private transient final static int[][] byteTables = new > int[8][256]; > 65 private transient final static int[] byteTable; > 66 private transient final static int[] byteTable0 = byteTables[0]; > > > Best regards, > Andrej Golovnin From staffan.friberg at oracle.com Thu Nov 6 15:12:35 2014 From: staffan.friberg at oracle.com (Staffan Friberg) Date: Thu, 06 Nov 2014 16:12:35 +0100 Subject: RFR JDK-6321472: Add CRC-32C API In-Reply-To: References: <544073EA.3060304@oracle.com> <5440F803.2000406@oracle.com> <544166B8.8090103@oracle.com> <5441A192.8090602@gmail.com> <544697D2.6080301@oracle.com> <5446AAB0.3090304@oracle.com> <5446B551.7050605@oracle.com> <5446BDE1.9090709@oracle.com> <5446CB91.3050503@oracle.com> <5446DB24.4000507@oracle.com> <545B4AE7.4000208@oracle.com> Message-ID: <545B8FE3.5090902@oracle.com> Hi Stanimir, I believe this makes the code harder to follow as you now have to read another method to follow the initialization of the arrays: The shifting of crc and secondHalf is also different so it would still be some difference between the two if I'm not mistaken. Cheers, Staffan On 11/06/2014 12:26 PM, Stanimir Simeonoff wrote: > Hi Staffan, > Given the tables in the static class init : > 66 private transient final static int[] byteTable0 = byteTables[0]; > 67 private transient final static int[] byteTable1 = byteTables[1]; > 68 private transient final static int[] byteTable2 = byteTables[2]; > 69 private transient final static int[] byteTable3 = byteTables[3]; > 70 private transient final static int[] byteTable4 = byteTables[4]; > 71 private transient final static int[] byteTable5 = byteTables[5]; > 72 private transient final static int[] byteTable6 = byteTables[6]; > 73 private transient final static int[] byteTable7 = byteTables[7]; > > > I was wondering if assigning the tables in the static part in similar fashion would make the code much more readable: > 66 private transient final static int[] byteTable0 = byteTables[idx(0)]; > 67 private transient final static int[] byteTable1 = byteTables[idx(1)]; > 68 private transient final static int[] byteTable2 = byteTables[idx(2)]; > 69 private transient final static int[] byteTable3 = byteTables[idx(3)]; > 70 private transient final static int[] byteTable4 = byteTables[idx(4(]; > 71 private transient final static int[] byteTable5 = byteTables[idx(5)]; > 72 private transient final static int[] byteTable6 = byteTables[idx(6)]; > 73 private transient final static int[] byteTable7 = byteTables[idx(7)]; > private static int idx(int i){ return ByteOrder.nativeOrder() == ByteOrder.LITTLE_ENDIAN?7-i:i;}; > > > Then in the loops you can skip checking the condition, something like: > 296- if (ByteOrder.nativeOrder() == ByteOrder.LITTLE_ENDIAN) { > 297- crc = byteTable7[crc & 0xFF] > 298- ^ byteTable6[(crc >>> 8) & 0xFF] > 299- ^ byteTable5[(crc >>> 16) & 0xFF] > 300- ^ byteTable4[crc >>> 24] > 301- ^ byteTable3[secondHalf & 0xFF] > 302- ^ byteTable2[(secondHalf >>> 8) & 0xFF] > 303- ^ byteTable1[(secondHalf >>> 16) & 0xFF] > 304- ^ byteTable0[secondHalf >>> 24]; > 305- } else { // ByteOrder.BIG_ENDIAN > 306 crc = byteTable0[secondHalf & 0xFF] > 307 ^ byteTable1[(secondHalf >>> 8) & 0xFF] > 308 ^ byteTable2[(secondHalf >>> 16) & 0xFF] > 309 ^ byteTable3[secondHalf >>> 24] > 310 ^ byteTable4[crc & 0xFF] > 311 ^ byteTable5[(crc >>> 8) & 0xFF] > 312 ^ byteTable6[(crc >>> 16) & 0xFF] > 313 ^ byteTable7[crc >>> 24]; > 314 - } > Since the byteTabeleN are properly assigned during the class init based on the endianess of the systen. Of course, there won't be any performance benefits since hotspot should constant fold the code > but it'll be easier to maintain and read. > > Cheers > Stanimir > > On Thu, Nov 6, 2014 at 12:18 PM, Staffan Friberg > > wrote: > > Anyone have time to be the second reviewer? > > http://cr.openjdk.java.net/~sfriberg/JDK-6321472/webrev.07 > > > The CCC request has been approved for the new API. > > Regards; > Staffan > > > On 10/22/2014 12:16 AM, Staffan Friberg wrote: > > Thanks for the review. Updated the @implSpec. > > Also includes Peter's good catch. > > Webrev: > http://cr.openjdk.java.net/~sfriberg/JDK-6321472/webrev.06 > > > //Staffan > > On 10/21/2014 02:09 PM, Xueming Shen wrote: > > Staffan, > > Thanks for the package.html update. > > Just wonder if it would be better to use > > buffer.remaining(), instead of the buffer.limit() - > buffer.position() > > in Checksum.udpate(ByteBuffer)'s #implSpec > > The rest looks fine for me. > > -Sherman > > On 10/21/2014 01:11 PM, Staffan Friberg wrote: > > Converted. > > http://cr.openjdk.java.net/~sfriberg/JDK-6321472/webrev.05 > > > //Staffan > > On 10/21/2014 12:34 PM, Joe Darcy wrote: > > Hi Staffan, > > If you are updating package.html, please also hg > mv the file to be a package-info.java file with > the equivalent javadoc. > > Thanks, > > -Joe > > On 10/21/2014 11:49 AM, Staffan Friberg wrote: > > Hi, > > Got an offline comment that the package.html > should be update as well to cover CRC-32C. > > Otherwise there are no code changes in this > new webrev. > > http://cr.openjdk.java.net/~sfriberg/JDK-6321472/webrev.04 > > > //Staffan > > On 10/21/2014 10:28 AM, Staffan Friberg wrote: > > Hi Peter, > > Thanks for the comments.. > > > 217 if > (Unsafe.ADDRESS_SIZE == 4) { > 218 // On 32 bit > platforms read two ints instead of a > single 64bit long > > When you're reading from byte[] using > Unsafe (updateBytes), you have the > option of reading 64bit values on > 64bit platforms. When you're reading > from DirectByteBuffer memory > (updateDirectByteBuffer), you're only > using 32bit reads. > > I will add a comment in the code for this > decision. The reason is that read a long > results in slightly worse performance in > this case, in updateBytes it is faster. I > was able to get it to run slightly faster > by working directly with the address > instead of always adding address + off, > but this makes things worse in the 32bit > case since all calculation will now be > using long variables. So using the getInt > as in the current code feels like the best > solution as it strikes the best balance > between 32 and 64bit. Below is how > updateByteBuffer looked with the rewrite I > mentioned. > > > ong address = ((DirectBuffer) > buffer).address(); > crc = updateDirectByteBuffer(crc, address > + pos, address + limit); > > > private static int > updateDirectByteBuffer(int crc, long adr, > long end) { > > // Do only byte reads for arrays > so short they can't be aligned > if (end - adr >= 8) { > > // align on 8 bytes > int alignLength = (8 - (int) > (adr & 0x7)) & 0x7; > for (long alignEnd = adr + > alignLength; adr < alignEnd; adr++) { > crc = (crc >>> 8) > ^ byteTable[(crc ^ > UNSAFE.getByte(adr)) & 0xFF]; > } > > if (ByteOrder.nativeOrder() == > ByteOrder.BIG_ENDIAN) { > crc = > Integer.reverseBytes(crc); > } > > // slicing-by-8 > for (; adr < (end - > Long.BYTES); adr += Long.BYTES) { > int firstHalf; > int secondHalf; > if (Unsafe.ADDRESS_SIZE == > 4) { > // On 32 bit platforms > read two ints instead of a single 64bit long > firstHalf = > UNSAFE.getInt(adr); > secondHalf = > UNSAFE.getInt(adr + Integer.BYTES); > } else { > long value = > UNSAFE.getLong(adr); > if > (ByteOrder.nativeOrder() == > ByteOrder.LITTLE_ENDIAN) { > firstHalf = (int) > value; > secondHalf = (int) > (value >>> 32); > } else { // > ByteOrder.BIG_ENDIAN > firstHalf = (int) > (value >>> 32); > secondHalf = (int) > value; > } > } > crc ^= firstHalf; > if > (ByteOrder.nativeOrder() == > ByteOrder.LITTLE_ENDIAN) { > crc = byteTable7[crc & > 0xFF] > ^ > byteTable6[(crc >>> 8) & 0xFF] > ^ > byteTable5[(crc >>> 16) & 0xFF] > ^ > byteTable4[crc >>> 24] > ^ > byteTable3[secondHalf & 0xFF] > ^ > byteTable2[(secondHalf >>> 8) & 0xFF] > ^ > byteTable1[(secondHalf >>> 16) & 0xFF] > ^ > byteTable0[secondHalf >>> 24]; > } else { // > ByteOrder.BIG_ENDIAN > crc = > byteTable0[secondHalf & 0xFF] > ^ > byteTable1[(secondHalf >>> 8) & 0xFF] > ^ > byteTable2[(secondHalf >>> 16) & 0xFF] > ^ > byteTable3[secondHalf >>> 24] > ^ > byteTable4[crc & 0xFF] > ^ > byteTable5[(crc >>> 8) & 0xFF] > ^ > byteTable6[(crc >>> 16) & 0xFF] > ^ > byteTable7[crc >>> 24]; > } > } > > if (ByteOrder.nativeOrder() == > ByteOrder.BIG_ENDIAN) { > crc = > Integer.reverseBytes(crc); > } > } > > // Tail > for (; adr < end; adr++) { > crc = (crc >>> 8) > ^ byteTable[(crc ^ > UNSAFE.getByte(adr)) & 0xFF]; > } > > return crc; > } > > > > Also, in updateBytes, the usage of > Unsafe.ARRAY_INT_INDEX_SCALE/ARRAY_LONG_INDEX_SCALE > to index a byte array sounds a little > scary. To be ultra portable you could > check that ARRAY_BYTE_INDEX_SCALE == 1 > first and refuse to use Unsafe for > byte arrays if it is not 1. Then use > Integer.BYTES/Long.BYTES to manipulate > 'offsets' instead. In > updateDirectByteBuffer it would be > more appropriate to use > Integer.BYTES/Long.BYTES too. > > Good idea. Added a check in the initial if > statement and it will get automatically > optimized away. > > 225 firstHalf = (int) (value & > 0xFFFFFFFF); > 226 > secondHalf = (int) (value >>> 32); > 227 } else { // > ByteOrder.BIG_ENDIAN > 228 > firstHalf = (int) (value >>> 32); > 229 > secondHalf = (int) (value & 0xFFFFFFFF); > > firstHalf = (int) value; // this is > equivalent for line 225 > secondHalf = (int) value; // this is > equivalent for line 229 > > Done. > > Here is the latest webrev, > http://cr.openjdk.java.net/~sfriberg/JDK-6321472/webrev.03 > > > Cheers, > Staffan > > > > > > > > From konstantin.shefov at oracle.com Thu Nov 6 15:20:48 2014 From: konstantin.shefov at oracle.com (Konstantin Shefov) Date: Thu, 06 Nov 2014 18:20:48 +0300 Subject: [9] Review request : JDK-8062904: TEST_BUG: Tests java/lang/invoke/LFCaching fail when run with -Xcomp option In-Reply-To: <545B875D.1060405@oracle.com> References: <545B5127.5030903@oracle.com> <545B875D.1060405@oracle.com> Message-ID: <545B91D0.4030008@oracle.com> Igor, Thanks for your comment! Here is the updated webrev: http://cr.openjdk.java.net/~kshefov/8062904/webrev.01/ -Konstantin On 06.11.2014 17:36, Igor Ignatyev wrote: > Konstantin, > > please use 'java.vm.info' instead of iteration over flags. > > Igor > > On 11/06/2014 01:44 PM, Konstantin Shefov wrote: >> Hello, >> >> Please review the test bug fix >> https://bugs.openjdk.java.net/browse/JDK-8062904 >> Webrev is http://cr.openjdk.java.net/~kshefov/8062904/webrev.00/ >> >> Thanks >> >> -Konstantin From martinrb at google.com Thu Nov 6 18:50:38 2014 From: martinrb at google.com (Martin Buchholz) Date: Thu, 6 Nov 2014 10:50:38 -0800 Subject: RFR: 8063147: Class.getFields spec should state that fields are inherited from superinterfaces Message-ID: Hi Joe, I'd like you to do a code review. https://bugs.openjdk.java.net/browse/JDK-8063147 Webrev: http://cr.openjdk.java.net/~martin/webrevs/openjdk9/Class-getFields-spec/ getFields was embarrassingly undertested, so I added some. Please shepherd through CCC. From martinrb at google.com Thu Nov 6 19:05:16 2014 From: martinrb at google.com (Martin Buchholz) Date: Thu, 6 Nov 2014 11:05:16 -0800 Subject: RFR: 8062773: Clarifications for Class specification In-Reply-To: <5459B518.1080206@oracle.com> References: <7746DAA6-2F3E-4891-B269-FC716D352B81@oracle.com> <5459B518.1080206@oracle.com> Message-ID: I moved the change to getFields to another changeset, redid some wording as suggested, harmonized getInterfaces and getGenericInterfaces some more, and regenerated the webrev.http://cr.openjdk.java.net/~martin/webrevs/openjdk9/Class-spec/ On Tue, Nov 4, 2014 at 9:26 PM, joe darcy wrote: > Hi Martin, > > This description in getGenericInterfaces > > 876 *

If this object represents a class, the return value is an > 877 * array containing objects representing all interfaces > 878 * implemented by the class. The order of the interface objects in > > could also be direct-ized. > > In getGenericInterfaces, the new text > > 899 *

If this {@code Class} object represents an array type, the > 900 * interfaces {@code Cloneable} and {@code java.io.Serializable} > are > 901 * returned in that order. > > is redundant with the existing text > > 881 * represented by this object. In the case of an array class, the > 882 * interfaces {@code Cloneable} and {@code Serializable} are > 883 * returned in that order. > > Exactly one copy of the Cloneable & Serializable assertion should be removed > :-) > > With this change to getFields, > > 1540 * returns the public fields of the class and of all its > superclasses and > 1541 * superinterfaces. > > the bug will need a ccc request since the specification is changing (even if > it is changing to match long-standing behavior). > > Thanks, > > -Joe > > > On 11/4/2014 9:08 AM, Martin Buchholz wrote: >> >> [+core-libs-dev] >> >> I'm sorry I keep forgetting to add the mailing list to my review >> requests. I blame google's code review mail tool, which cc's the >> right mailing lists automagically. >> >> Webrev updated to add a few more "direct"s, as suggested by Paul. >> >> https://bugs.openjdk.java.net/browse/JDK-8062773 >> http://cr.openjdk.java.net/~martin/webrevs/openjdk9/Class-spec/ >> >> On Tue, Nov 4, 2014 at 3:02 AM, Paul Sandoz >> wrote: >>> >>> On Nov 3, 2014, at 11:35 PM, Martin Buchholz wrote: >>> >>> Hi Joe, Paul, >>> >>> I'd like you to do a code review. >>> >>> https://bugs.openjdk.java.net/browse/JDK-8062773 >>> http://cr.openjdk.java.net/~martin/webrevs/openjdk9/Class-spec/ >>> >>> >>> >>> Looks ok to me. >>> >>> 730 * @return the superclass of the class represented by this >>> object. >>> >>> You could tweak that to say "the direct superclass..." >>> >>> Paul. > > From martinrb at google.com Thu Nov 6 19:07:20 2014 From: martinrb at google.com (Martin Buchholz) Date: Thu, 6 Nov 2014 11:07:20 -0800 Subject: RFR: 8063147: Class.getFields spec should state that fields are inherited from superinterfaces In-Reply-To: References: Message-ID: Sorry, Paul, I missed your offer on the other thread to take this to CCC. It's yours. On Thu, Nov 6, 2014 at 10:50 AM, Martin Buchholz wrote: > Hi Joe, > > I'd like you to do a code review. > > https://bugs.openjdk.java.net/browse/JDK-8063147 > Webrev: http://cr.openjdk.java.net/~martin/webrevs/openjdk9/Class-getFields-spec/ > > getFields was embarrassingly undertested, so I added some. > > Please shepherd through CCC. From martinrb at google.com Thu Nov 6 19:51:02 2014 From: martinrb at google.com (Martin Buchholz) Date: Thu, 6 Nov 2014 11:51:02 -0800 Subject: RFR: 8062771: Core reflection should use final fields whenever possible In-Reply-To: References: <843282E1-DC2B-48B0-B0E4-A60B5078D75A@oracle.com> Message-ID: Hi Joel, On Thu, Nov 6, 2014 at 2:48 AM, Joel Borggr?n-Franck wrote: > Hi, > > I?m having a hard time following this thread, which webrev has been updated, for which release, fixing which issue? I am submitting the changeset for JDK-8062771, adding the finals and the (failed reproduction) test. Hopefully there will be followon changes to add even more thread safety. > Martin, as far as I can see you are the only one of us who has replied to this thread who is a jdk7u committer (or reviewer). I am entirely in favor of having more reviewers. We are not planning to push these into jdk7u or jdk8u ourselves, but we are willing to help anyone who wants to take on that task. Joel, are you volunteering? We believe all versions of stock openjdk still have (rarely seen) thread safety issues in core reflection. We have applied changes locally at Google to fix those. > I think the assumption is that generic repositories of different kinds are ok with multiple initialization, but maybe we need to revisit this? Whether or not having multiple initialization is a problem today, it would definitely be safer to ensure single initialization via volatile CAS of all fields that make up core reflection. From david.buck at oracle.com Fri Nov 7 03:08:39 2014 From: david.buck at oracle.com (david buck) Date: Fri, 07 Nov 2014 12:08:39 +0900 Subject: RFR: 8062608: BCEL corrupts debug data of methods that use generics Message-ID: <545C37B7.5090105@oracle.com> Hi! May I please have a review of this fix: BUGURL: https://bugs.openjdk.java.net/browse/JDK-8062608 WEBREV: http://cr.openjdk.java.net/~dbuck/8062608/webrev.00/ Cheers, -Buck From huizhe.wang at oracle.com Fri Nov 7 04:30:05 2014 From: huizhe.wang at oracle.com (huizhe wang) Date: Thu, 06 Nov 2014 20:30:05 -0800 Subject: RFR: 8062608: BCEL corrupts debug data of methods that use generics In-Reply-To: <545C37B7.5090105@oracle.com> References: <545C37B7.5090105@oracle.com> Message-ID: <545C4ACD.9070906@oracle.com> Looks good. Are you checking in the test (the one attached to the bug report)? Cheers, Joe On 11/6/2014 7:08 PM, david buck wrote: > Hi! > > May I please have a review of this fix: > > BUGURL: https://bugs.openjdk.java.net/browse/JDK-8062608 > WEBREV: http://cr.openjdk.java.net/~dbuck/8062608/webrev.00/ > > Cheers, > -Buck From david.buck at oracle.com Fri Nov 7 04:57:31 2014 From: david.buck at oracle.com (david.buck at oracle.com) Date: Fri, 7 Nov 2014 13:57:31 +0900 Subject: RFR: 8062608: BCEL corrupts debug data of methods that use generics In-Reply-To: <545C4ACD.9070906@oracle.com> References: <545C37B7.5090105@oracle.com> <545C4ACD.9070906@oracle.com> Message-ID: <444CFB85-9B70-4E53-A628-F429F5923B72@oracle.com> Hi Joe, Thank you for the review! > Are you checking in the test (the one attached to the bug report)? I was not planning on it as I do not believe we should be adding test cases to JAXP that only test completely internal implementation classes (i.e. white box testing) for code paths that aren't even used by JAXP itself. We would just be making the test suite more brittle with almost zero additional coverage of code actually used by JAXP. Cheers, -Buck 2014/11/07 13:30?huizhe wang ??????: > Looks good. > > Are you checking in the test (the one attached to the bug report)? > > Cheers, > Joe > >> On 11/6/2014 7:08 PM, david buck wrote: >> Hi! >> >> May I please have a review of this fix: >> >> BUGURL: https://bugs.openjdk.java.net/browse/JDK-8062608 >> WEBREV: http://cr.openjdk.java.net/~dbuck/8062608/webrev.00/ >> >> Cheers, >> -Buck > From huizhe.wang at oracle.com Fri Nov 7 05:40:30 2014 From: huizhe.wang at oracle.com (huizhe wang) Date: Thu, 06 Nov 2014 21:40:30 -0800 Subject: RFR: 8062608: BCEL corrupts debug data of methods that use generics In-Reply-To: <444CFB85-9B70-4E53-A628-F429F5923B72@oracle.com> References: <545C37B7.5090105@oracle.com> <545C4ACD.9070906@oracle.com> <444CFB85-9B70-4E53-A628-F429F5923B72@oracle.com> Message-ID: <545C5B4E.4010602@oracle.com> On 11/6/2014 8:57 PM, david.buck at oracle.com wrote: > Hi Joe, > > Thank you for the review! > >> Are you checking in the test (the one attached to the bug report)? > I was not planning on it as I do not believe we should be adding test cases to JAXP that only test completely internal implementation classes (i.e. white box testing) for code paths that aren't even used by JAXP itself. We would just be making the test suite more brittle with almost zero additional coverage of code actually used by JAXP. Ok. It's true that it doesn't affect JAXP. -Joe > > Cheers, > -Buck > > 2014/11/07 13:30?huizhe wang ??????: > >> Looks good. >> >> Are you checking in the test (the one attached to the bug report)? >> >> Cheers, >> Joe >> >>> On 11/6/2014 7:08 PM, david buck wrote: >>> Hi! >>> >>> May I please have a review of this fix: >>> >>> BUGURL: https://bugs.openjdk.java.net/browse/JDK-8062608 >>> WEBREV: http://cr.openjdk.java.net/~dbuck/8062608/webrev.00/ >>> >>> Cheers, >>> -Buck From mandy.chung at oracle.com Fri Nov 7 06:42:39 2014 From: mandy.chung at oracle.com (Mandy Chung) Date: Thu, 06 Nov 2014 22:42:39 -0800 Subject: RFR: 8060132: Handlers configured on abstract nodes in logging.properties are not always properly closed In-Reply-To: <545A4D0D.7090105@oracle.com> References: <5437F268.9020600@oracle.com> <5437FDCB.30803@oracle.com> <544A9B16.6030500@oracle.com> <544AC9B3.8090202@oracle.com> <54590368.5070309@oracle.com> <545955C7.4010909@oracle.com> <545A0E6E.5080700@oracle.com> <545A4D0D.7090105@oracle.com> Message-ID: <545C69DF.30502@oracle.com> On 11/5/2014 8:15 AM, Daniel Fuchs wrote: > Here is the new webrev: > http://cr.openjdk.java.net/~dfuchs/webrev_8060132/webrev.03/index.html Looks good. Minor nits and no need to generate new webrev. line 108: {@linkplain #reset} -- # is missing. line 918-921: nit: formatting looks a little inconsistent with the code around it. Perhaps better to wrap them in 2 lines: boolean ensureCloseOnReset = names.length > 0 && getBooleanProperty(handlersPropertyName + ".ensureCloseOnReset", true); Mandy From Alan.Bateman at oracle.com Fri Nov 7 09:52:47 2014 From: Alan.Bateman at oracle.com (Alan Bateman) Date: Fri, 07 Nov 2014 09:52:47 +0000 Subject: RFR: 8062194: java.util.jar.Attributes should use insertion-ordered iteration In-Reply-To: References: <544E96AC.90904@oracle.com> <544FF3ED.2020901@oracle.com> Message-ID: <545C966F.8080101@oracle.com> On 05/11/2014 00:06, Martin Buchholz wrote: > : > So ... there is the question whether we are tightening the spec for > Attributes and its subclasses or only for Attributes. > Subclasses of Attribute that would change the order are likely to be > very rare (is there a use case?) - I'm OK with those becoming > non-spec-compliant subclasses, especially given that they will > continue to work just fine in practice. So in my proposal the > tightening applies to subclasses as well. What you have is okay but I think if I were doing this then I would keep the existing wording in the constructor (the update to the class description is good). The reason is that the new wording might distract the reader into thinking about the iteration order of the given Attributes when it's just not interesting here. -Alan. From konstantin.shefov at oracle.com Fri Nov 7 10:22:40 2014 From: konstantin.shefov at oracle.com (Konstantin Shefov) Date: Fri, 07 Nov 2014 13:22:40 +0300 Subject: [9] Review request : JDK-8059070: [TESTBUG] java/lang/invoke/LFCaching/LFMultiThreadCachingTest.java failed - timeout In-Reply-To: <5450F8E4.2060808@oracle.com> References: <543D1DEE.8030206@oracle.com> <543F8520.9090408@oracle.com> <8F9DD322-1793-4739-8155-61CE7D2C5AAB@oracle.com> <543F9800.7040406@oracle.com> <543FC71F.8020504@oracle.com> <5440E39C.8070001@oracle.com> <5448E5A2.9030804@oracle.com> <544E1B65.1090400@oracle.com> <5450F8E4.2060808@oracle.com> Message-ID: <545C9D70.6060308@oracle.com> Gently reminder 29.10.2014 17:25, Konstantin Shefov ?????: > Please, review a test bug fix. > http://cr.openjdk.java.net/~kshefov/8059070/webrev.01/ > > -Konstantin > > On 27.10.2014 13:16, Konstantin Shefov wrote: >> Kindly reminder >> >> On 23.10.2014 19:04, Paul Sandoz wrote: >>> On Oct 23, 2014, at 1:25 PM, Konstantin Shefov >>> wrote: >>>> Gently reminder >>>> >>>> On 17.10.2014 13:38, Konstantin Shefov wrote: >>>>> Hi, >>>>> >>>>> I have updated the webrev: >>>>> http://cr.openjdk.java.net/~kshefov/8059070/webrev.01/ >>>>> >>> +1 >>> >>> Sorry for the delay, >>> Paul. >> > From konstantin.shefov at oracle.com Fri Nov 7 10:23:44 2014 From: konstantin.shefov at oracle.com (Konstantin Shefov) Date: Fri, 07 Nov 2014 13:23:44 +0300 Subject: [9] Review request : JDK-8062904: TEST_BUG: Tests java/lang/invoke/LFCaching fail when run with -Xcomp option In-Reply-To: <545B91D0.4030008@oracle.com> References: <545B5127.5030903@oracle.com> <545B875D.1060405@oracle.com> <545B91D0.4030008@oracle.com> Message-ID: <545C9DB0.3090604@oracle.com> Kindly reminder http://cr.openjdk.java.net/~kshefov/8062904/webrev.01/ 06.11.2014 18:20, Konstantin Shefov ?????: > Igor, > > Thanks for your comment! > Here is the updated webrev: > http://cr.openjdk.java.net/~kshefov/8062904/webrev.01/ > > -Konstantin > > On 06.11.2014 17:36, Igor Ignatyev wrote: >> Konstantin, >> >> please use 'java.vm.info' instead of iteration over flags. >> >> Igor >> >> On 11/06/2014 01:44 PM, Konstantin Shefov wrote: >>> Hello, >>> >>> Please review the test bug fix >>> https://bugs.openjdk.java.net/browse/JDK-8062904 >>> Webrev is http://cr.openjdk.java.net/~kshefov/8062904/webrev.00/ >>> >>> Thanks >>> >>> -Konstantin > From igor.ignatyev at oracle.com Fri Nov 7 10:28:08 2014 From: igor.ignatyev at oracle.com (Igor Ignatyev) Date: Fri, 07 Nov 2014 13:28:08 +0300 Subject: [9] Review request : JDK-8062904: TEST_BUG: Tests java/lang/invoke/LFCaching fail when run with -Xcomp option In-Reply-To: <545B91D0.4030008@oracle.com> References: <545B5127.5030903@oracle.com> <545B875D.1060405@oracle.com> <545B91D0.4030008@oracle.com> Message-ID: <545C9EB8.2090200@oracle.com> Ho Konstantin, Actually, I don't think that this should be fixed by reducing iterations, CompileCommand magic sounds more reasonable here. I'd even say I'm not sure that it's a test bug, it looks very similar to JDK-8046809. have you tried to run the tests on the build where JDK-8046809 is fixed? Igor On 11/06/2014 06:20 PM, Konstantin Shefov wrote: > Igor, > > Thanks for your comment! > Here is the updated webrev: > http://cr.openjdk.java.net/~kshefov/8062904/webrev.01/ > > -Konstantin > > On 06.11.2014 17:36, Igor Ignatyev wrote: >> Konstantin, >> >> please use 'java.vm.info' instead of iteration over flags. >> >> Igor >> >> On 11/06/2014 01:44 PM, Konstantin Shefov wrote: >>> Hello, >>> >>> Please review the test bug fix >>> https://bugs.openjdk.java.net/browse/JDK-8062904 >>> Webrev is http://cr.openjdk.java.net/~kshefov/8062904/webrev.00/ >>> >>> Thanks >>> >>> -Konstantin > From daniel.fuchs at oracle.com Fri Nov 7 11:19:46 2014 From: daniel.fuchs at oracle.com (Daniel Fuchs) Date: Fri, 07 Nov 2014 12:19:46 +0100 Subject: RFR: 8060132: Handlers configured on abstract nodes in logging.properties are not always properly closed In-Reply-To: References: <5437F268.9020600@oracle.com> <5437FDCB.30803@oracle.com> <544A9B16.6030500@oracle.com> <544AC9B3.8090202@oracle.com> <54590368.5070309@oracle.com> <545955C7.4010909@oracle.com> <545A0E6E.5080700@oracle.com> <545A4D0D.7090105@oracle.com> Message-ID: <545CAAD2.1070905@oracle.com> On 05/11/14 19:47, Stanimir Simeonoff wrote: > 945 } catch (Exception ex) { > 946 System.err.println("Can't load log handler \"" + type + "\""); > 947 System.err.println("" + ex); > 948 ex.printStackTrace(); > 949 } > > I'm not quite sure if handling Error (in particular any subclass of LinkageError) would make sense here (similar to the invocation of listeners). > Of course there are other case handling just Exception and using word instead of 'type' but that's matter of another clean up. Yes... LogManager should probably have some kind of ErrorManager - and not just for listener invocation... best regards, -- daniel > > Stanimir > > On Wed, Nov 5, 2014 at 6:15 PM, Daniel Fuchs> wrote: > > Hi Mandy, Stanimir, > > Here is the new webrev: > http://cr.openjdk.java.net/~__dfuchs/webrev_8060132/webrev.__03/index.html > > > best regards, > > -- daniel > > > On 05/11/14 12:47, Daniel Fuchs wrote: > > On 04/11/14 23:40, Mandy Chung wrote: > > On 11/4/14 8:48 AM, Daniel Fuchs wrote: > > Hi, > > Here is a revised patch that introduces a new > .handlers.__ensureCloseOnReset=true|false > property. > > You have explored several different options and as you have > found, > none of them is ideal. I think having a property to specify > ensureCloseOnReset would be reasonable to address the memory > leak > concern I have. > > Would it be better to name it .handlers.__closeOnReset? > > > Well - closeOnReset is the normal behavior - if the logger > is still around. So I believe ensureCloseOnReset is a better name. > > I am less sure if we need to have a global switch to revert > the default - do we expect the number of .handlers > is large? I would expect there would be small number. > > > I would expect the cases where you need to specify > ensureCloseOnReset will be rare. So maybe we should wait > until the case arises before implementing a global switch: > we can add that later if it's needed. The local switches > will provide a workable (even though possibly cumbersome) > work around. > > This property can be used as a fallback to turn the new > behavior > off in the improbable case where an application may need > to do > so. I have updated the LogManager class-level API > documentation. > > http://cr.openjdk.java.net/~__dfuchs/webrev_8060132/webrev.__02/ > > > line 104-114: s/"true"/{@code true} > {@code reset} should be {@link ...} to the reset method (the > first occurrance) > > > Will do. > > It doesn't seem necessary to specify "strongly referenced". > Maybe it's good to say "it will ensure the handlers associated > with the loggers will be closed at shutdown or at reset. > > > OK > > If "closeOnReset is false", application should ensure the > handlers be closed before the logger gets garbage collected." > > > I will try to find something. > > - something along that line. > > typos: > 186: explicitely -> explicitly > > > I keep on doing this same mistake again and again :-( > > Perhaps PersistentLogger can be renamed to CloseOnResetLogger > to make it clear what these loggers are kept for. Similarly > for the variable name "persistentLogger" may be renamed too. > > > OK. > > > 917: final might be adding noise > > I like final ;-) > > 940: persitentLoggers > > OK > > > line 917: you don't need (names.length == 0) to determine > the default, > right? if names.length == 0, it wouldn't add to the > persistentLoggers > list > as no iteration in the for loop. > > > I will use the syntax that Stanimir suggested. It's much simpler > than what I originally wrote. If names.length == 0 we don't need to > look up the ensureCloseOnReset property. > > Thanks Mandy, > > I will send a new webrev shortly - with both yours & Stanimir > comments > integrated (barring the global switch - as we could do that in a > followup revision if that's really needed). > > best regards, > > -- daniel > > > > Mandy > > > > From daniel.fuchs at oracle.com Fri Nov 7 11:20:10 2014 From: daniel.fuchs at oracle.com (Daniel Fuchs) Date: Fri, 07 Nov 2014 12:20:10 +0100 Subject: RFR: 8060132: Handlers configured on abstract nodes in logging.properties are not always properly closed In-Reply-To: <545C69DF.30502@oracle.com> References: <5437F268.9020600@oracle.com> <5437FDCB.30803@oracle.com> <544A9B16.6030500@oracle.com> <544AC9B3.8090202@oracle.com> <54590368.5070309@oracle.com> <545955C7.4010909@oracle.com> <545A0E6E.5080700@oracle.com> <545A4D0D.7090105@oracle.com> <545C69DF.30502@oracle.com> Message-ID: <545CAAEA.7040701@oracle.com> On 07/11/14 07:42, Mandy Chung wrote: > > On 11/5/2014 8:15 AM, Daniel Fuchs wrote: >> Here is the new webrev: >> http://cr.openjdk.java.net/~dfuchs/webrev_8060132/webrev.03/index.html > > Looks good. Minor nits and no need to generate new webrev. > > line 108: {@linkplain #reset} -- # is missing. > line 918-921: nit: formatting looks a little inconsistent with the code > around it. > Perhaps better to wrap them in 2 lines: > > boolean ensureCloseOnReset = names.length > 0 && > getBooleanProperty(handlersPropertyName + ".ensureCloseOnReset", > true); Thanks Mandy, Will do! -- daniel > > > Mandy From pavel.rappo at oracle.com Fri Nov 7 12:14:53 2014 From: pavel.rappo at oracle.com (Pavel Rappo) Date: Fri, 7 Nov 2014 12:14:53 +0000 Subject: RFR JDK-8059311: com/sun/jndi/ldap/LdapTimeoutTest.java fails with exit_code == 0 In-Reply-To: <1F65F028-5F02-4608-9221-EF10498CF8A7@oracle.com> References: <1F65F028-5F02-4608-9221-EF10498CF8A7@oracle.com> Message-ID: <55347142-43A1-4DC7-B3D3-931D165F4B9B@oracle.com> Hi everyone, Could you please review my change for JDK-8059311? http://cr.openjdk.java.net/~prappo/8059311/webrev.00/ ---------------------------------------------------------------- It looks like one of the "killSwitches" went off and executed System.exit(0). The problem is jtreg doesn't like System.exit and treats it as a test failure (http://openjdk.java.net/jtreg/faq.html#question7.1): "...Tests are not allowed to call System.exit because the test must have the ability to run in the same JVM as the harness. Calling System.exit while the test is running in this manner would cause the harness itself to exit..." Moreover, the purpose of the "killSwitch" is not clear. It seems to be ensuring the test won't hang by killing JVM after a certain timeout. First of all, if the test hangs -- it means we have a problem to investigate and the "killSwitch" is just masking it. If it doesn't hang -- there's no need for a "killSwitch". If the test hangs it will be noticed by jtreg (http://openjdk.java.net/jtreg/tag-spec.txt): "... /timeout= Specify the timeout value. The default timeout is two minutes. If an action does not finish before the timeout expires, it fails..." Thus if we remove the "killSwitch" we treat the problem of slow systems (where killSwitch timeout was too small) and at the same time will be notified if test hangs. Thanks -Pavel From eric.mccorkle at oracle.com Fri Nov 7 13:27:54 2014 From: eric.mccorkle at oracle.com (Eric McCorkle) Date: Fri, 07 Nov 2014 08:27:54 -0500 Subject: Review request for 8055063: Parameter#toString() fails w/ AIOOBE for ctr of inner class w/ generic type In-Reply-To: <545A9D46.9040002@oracle.com> References: <5453B58A.1090507@oracle.com> <55A63A8D-1D0E-4D48-8CC4-D9326A57586E@oracle.com> <545A9D46.9040002@oracle.com> Message-ID: <545CC8DA.5070104@oracle.com> Are there any more comments on this patch? This is an 8u40 issue, so it needs to be committed soon. Thanks, Eric On 11/05/14 16:57, Eric McCorkle wrote: > A word on the tests: I don't think there's a stable way to check the > types on getParameterizedType() and getAnnotatedType(). So I checked > the type on getType() only. > > The problem is, you have to detect and cast from ParameterizedType and > AnnotatedType respectively, but depending on the situation with generic > signatures and method parameters information (and type annotations), you > might not actually get back one of these. > > On 11/05/14 10:10, Joel Borggr?n-Franck wrote: >> Hi Eric, >> >> I think caching the real parameter type array on Executable is the wrong thing to do considering we can have a lot of instances of Executable but fairly few will be examined for parameters and the vast majority of those will never have this issue. >> >> Also, the tests are a bit minimal, I would expect you to check that the type of the parameter is correct. The tests should also have a @bug line. >> >> I think you are overly cautious here. I?m not sure we need to work around potential bugs in non-Java compilers. If language Y inserts synthetic/mandated parameters anywhere else than as a prefix they better generate a correct Signature (or non at all). This is a discussion we can defer however. >> >> If you flesh out the test and remove the caching I think this is a fine fix for both 8u and 9. >> >> cheers >> /Joel >> >> On 31 okt 2014, at 17:15, Eric McCorkle wrote: >> >>> Hello, >>> >>> Please review this patch which fixes issues that arise with >>> getGenericParameterTypes() and getAnnotatedParameterTypes() when there >>> are generic signatures and synthetic parameters. >>> >>> Please note that a complete fix is not possible for all cases. See >>> discussion on https://bugs.openjdk.java.net/browse/JDK-8062582 for details. >>> >>> This patch will cause Executable.getAnnotatedParameterTypes(), >>> Parameter.getAnnotatedType(), and Parameter.getParameterizedType() to >>> report the correct types in the following cases: >>> >>> * No generic signature is present. >>> * Both a generic signature and method parameters information are present >>> * A generic signature is present, but method parameters information is >>> not present, but the number of parameters in the generic signature and >>> the number of parameters in the method descriptor are the same. >>> >>> In the problematic case, where there is a generic signature, no method >>> parameters information, and the generic signature does not match the >>> method descriptor, these methods will return the correct /non/-generic >>> type, as there is no general way of associating parameters in the >>> generic signature with those in the method descriptor in this case. >>> >>> Please also note that there is currently a bug in javac which causes >>> type annotations' parameter indexes to be wrong when synthetic >>> parameters are generated: https://bugs.openjdk.java.net/browse/JDK-8029012. >>> >>> The bug report is here: >>> https://bugs.openjdk.java.net/browse/JDK-8055063 >>> >>> The webrev is here: >>> http://cr.openjdk.java.net/~emc/8055063/ >> From vincent.x.ryan at oracle.com Fri Nov 7 15:02:37 2014 From: vincent.x.ryan at oracle.com (Vincent Ryan) Date: Fri, 7 Nov 2014 15:02:37 +0000 Subject: RFR JDK-8059311: com/sun/jndi/ldap/LdapTimeoutTest.java fails with exit_code == 0 In-Reply-To: <55347142-43A1-4DC7-B3D3-931D165F4B9B@oracle.com> References: <1F65F028-5F02-4608-9221-EF10498CF8A7@oracle.com> <55347142-43A1-4DC7-B3D3-931D165F4B9B@oracle.com> Message-ID: All looks good. Thanks. On 7 Nov 2014, at 12:14, Pavel Rappo wrote: > Hi everyone, > > Could you please review my change for JDK-8059311? > > http://cr.openjdk.java.net/~prappo/8059311/webrev.00/ > > ---------------------------------------------------------------- > It looks like one of the "killSwitches" went off and executed System.exit(0). The problem is jtreg doesn't like System.exit and treats it as a test failure (http://openjdk.java.net/jtreg/faq.html#question7.1): > > "...Tests are not allowed to call System.exit because the test must have the ability to run in the same JVM as the harness. Calling System.exit while the test is running in this manner would cause the harness itself to exit..." > > Moreover, the purpose of the "killSwitch" is not clear. It seems to be ensuring the test won't hang by killing JVM after a certain timeout. > First of all, if the test hangs -- it means we have a problem to investigate and the "killSwitch" is just masking it. If it doesn't hang -- there's no need for a "killSwitch". > > If the test hangs it will be noticed by jtreg (http://openjdk.java.net/jtreg/tag-spec.txt): > > "... /timeout= > > Specify the timeout value. The default timeout is two minutes. If an action > does not finish before the timeout expires, it fails..." > > Thus if we remove the "killSwitch" we treat the problem of slow systems (where killSwitch timeout was too small) and at the same time will be notified if test hangs. > > Thanks > -Pavel From lance.andersen at oracle.com Fri Nov 7 16:25:12 2014 From: lance.andersen at oracle.com (Lance Andersen) Date: Fri, 7 Nov 2014 11:25:12 -0500 Subject: RFR 8064356: Add BaseRowSet, SQLInputImpl, and SQLOutputImpl tests Message-ID: <464340DD-483F-4414-AD86-3650BAF65A29@oracle.com> Hi all, Need a reviewer for the following tests for exercising BaseRowSet, SQlInputImpl, and SQLOutputImpl The webrev can be found at: http://cr.openjdk.java.net/~lancea/8064356/webrev.00/ Have a good weekend! Best, Lance Lance Andersen| Principal Member of Technical Staff | +1.781.442.2037 Oracle Java Engineering 1 Network Drive Burlington, MA 01803 Lance.Andersen at oracle.com From huizhe.wang at oracle.com Fri Nov 7 16:50:25 2014 From: huizhe.wang at oracle.com (huizhe wang) Date: Fri, 07 Nov 2014 08:50:25 -0800 Subject: RFR (JAXP) 8059327 : XML parser returns corrupt attribute value Message-ID: <545CF851.1070505@oracle.com> Hi, This is a regression caused by changeset: http://cr.openjdk.java.net/~joehw/jdk9/8027359/webrev/. The fix then is to recover those changes that modified the buffer. Please review. http://cr.openjdk.java.net/~joehw/jdk9/8059327/webrev/ https://bugs.openjdk.java.net/browse/JDK-8059327 Thanks, Joe From lance.andersen at oracle.com Fri Nov 7 17:30:30 2014 From: lance.andersen at oracle.com (Lance Andersen) Date: Fri, 7 Nov 2014 12:30:30 -0500 Subject: RFR (JAXP) 8059327 : XML parser returns corrupt attribute value In-Reply-To: <545CF851.1070505@oracle.com> References: <545CF851.1070505@oracle.com> Message-ID: <24806494-1601-4FF8-81B0-8DF1ADF4FCB1@oracle.com> Looks OK Joe. I went and looked at the previous version prior to the 1st fix. Best Lance On Nov 7, 2014, at 11:50 AM, huizhe wang wrote: > Hi, > > This is a regression caused by changeset: http://cr.openjdk.java.net/~joehw/jdk9/8027359/webrev/. The fix then is to recover those changes that modified the buffer. Please review. > > http://cr.openjdk.java.net/~joehw/jdk9/8059327/webrev/ > https://bugs.openjdk.java.net/browse/JDK-8059327 > > Thanks, > Joe Lance Andersen| Principal Member of Technical Staff | +1.781.442.2037 Oracle Java Engineering 1 Network Drive Burlington, MA 01803 Lance.Andersen at oracle.com From Roger.Riggs at Oracle.com Fri Nov 7 17:47:43 2014 From: Roger.Riggs at Oracle.com (Roger Riggs) Date: Fri, 07 Nov 2014 12:47:43 -0500 Subject: RFR 8064356: Add BaseRowSet, SQLInputImpl, and SQLOutputImpl tests In-Reply-To: <464340DD-483F-4414-AD86-3650BAF65A29@oracle.com> References: <464340DD-483F-4414-AD86-3650BAF65A29@oracle.com> Message-ID: <545D05BF.9010608@Oracle.com> Hi Lance, Looks ok. Roger On 11/7/14 11:25 AM, Lance Andersen wrote: > Hi all, > > Need a reviewer for the following tests for exercising BaseRowSet, SQlInputImpl, and SQLOutputImpl > > The webrev can be found at: http://cr.openjdk.java.net/~lancea/8064356/webrev.00/ > > Have a good weekend! > > Best, > Lance > > > Lance Andersen| Principal Member of Technical Staff | +1.781.442.2037 > Oracle Java Engineering > 1 Network Drive > Burlington, MA 01803 > Lance.Andersen at oracle.com > > > From huizhe.wang at oracle.com Fri Nov 7 20:03:27 2014 From: huizhe.wang at oracle.com (huizhe wang) Date: Fri, 07 Nov 2014 12:03:27 -0800 Subject: RFR 8064356: Add BaseRowSet, SQLInputImpl, and SQLOutputImpl tests In-Reply-To: <545D05BF.9010608@Oracle.com> References: <464340DD-483F-4414-AD86-3650BAF65A29@oracle.com> <545D05BF.9010608@Oracle.com> Message-ID: <545D258F.40500@oracle.com> +1, nice tests! Joe On 11/7/2014 9:47 AM, Roger Riggs wrote: > Hi Lance, > > Looks ok. > > Roger > > On 11/7/14 11:25 AM, Lance Andersen wrote: >> Hi all, >> >> Need a reviewer for the following tests for exercising BaseRowSet, >> SQlInputImpl, and SQLOutputImpl >> >> The webrev can be found at: >> http://cr.openjdk.java.net/~lancea/8064356/webrev.00/ >> >> Have a good weekend! >> >> Best, >> Lance >> >> >> Lance Andersen| Principal Member of Technical Staff | +1.781.442.2037 >> Oracle Java Engineering >> 1 Network Drive >> Burlington, MA 01803 >> Lance.Andersen at oracle.com >> >> >> > From martinrb at google.com Fri Nov 7 20:52:08 2014 From: martinrb at google.com (Martin Buchholz) Date: Fri, 7 Nov 2014 12:52:08 -0800 Subject: RFR: 8062194: java.util.jar.Attributes should use insertion-ordered iteration In-Reply-To: <545C966F.8080101@oracle.com> References: <544E96AC.90904@oracle.com> <544FF3ED.2020901@oracle.com> <545C966F.8080101@oracle.com> Message-ID: On Fri, Nov 7, 2014 at 1:52 AM, Alan Bateman wrote: > On 05/11/2014 00:06, Martin Buchholz wrote: >> >> : >> So ... there is the question whether we are tightening the spec for >> Attributes and its subclasses or only for Attributes. >> Subclasses of Attribute that would change the order are likely to be >> very rare (is there a use case?) - I'm OK with those becoming >> non-spec-compliant subclasses, especially given that they will >> continue to work just fine in practice. So in my proposal the >> tightening applies to subclasses as well. > > What you have is okay but I think if I were doing this then I would keep the > existing wording in the constructor (the update to the class description is > good). The reason is that the new wording might distract the reader into > thinking about the iteration order of the given Attributes when it's just > not interesting here. I don't much care either way, so the spec change to Attributes(Attributes) is reverted, as you wish. Webrev refreshed. From david.r.chase at oracle.com Fri Nov 7 21:14:38 2014 From: david.r.chase at oracle.com (David Chase) Date: Fri, 7 Nov 2014 16:14:38 -0500 Subject: [9] RFR(L) 8013267 : move MemberNameTable from native code to Java heap, use to intern MemberNames In-Reply-To: References: <594FE416-D445-426E-B7A9-C75F012ADE16@oracle.com> <0BDCD5DD-CBFB-4A3D-9BAD-7F9912E9237C@oracle.com> <5453C230.8010709@oracle.com> <9747A3A3-B7F4-407A-8E00-1E647D9DC2D1@oracle.com> <1D195ED3-8BD2-46CB-9D70-29CF809D9F5A@oracle.com> <5456FB59.60905@oracle.com> <632A5C98-B386-4625-BE12-355241581955@oracle.com> <5457AA75.8090103@gmail.com> <5457E0F9.8090004@gmail.com> <5458A57C.4060208@gmail.com> <260D49F5-6380-4FC3-A900-6CD9AB3ED6F7@oracle.com> <5459034E.8070809@gmail.com> Message-ID: <39826508-110B-4FCE-9A58-8C3D1B9FC7DE@oracle.com> New webrev: bug: https://bugs.openjdk.java.net/browse/JDK-8013267 webrevs: http://cr.openjdk.java.net/~drchase/8013267/jdk.06/ http://cr.openjdk.java.net/~drchase/8013267/hotspot.06/ Changes since last: 1) refactored to put ClassData under java.lang.invoke.MemberName 2) split the data structure into two parts; handshake with JVM uses a linked list, which makes for a simpler backout-if-race, and Java side continues to use the simple sorted array. This should allow easier use of (for example) fancier data structures (like ConcurrentHashMap) if this later proves necessary. 3) Cleaned up symbol references in the new hotspot code to go through vmSymbols. 4) renamed oldCapacity to oldSize 5) ran two different benchmarks and saw no change in performance. a) nashorn ScriptTest (see https://bugs.openjdk.java.net/browse/JDK-8014288 ) b) JMH microbenchmarks (see bug comments for details) And it continues to pass the previously-failing tests, as well as the new test which has been added to hotspot/test/compiler/jsr292 . David On 2014-11-04, at 3:54 PM, David Chase wrote: > I?m working on the initial benchmarking, and so far this arrangement (with synchronization > and binary search for lookup, lots of barriers and linear cost insertion) has not yet been any > slower. > > I am nonetheless tempted by the 2-tables solution, because I think the simpler JVM-side > interface that it allows is desirable. > > David > > On 2014-11-04, at 11:48 AM, Peter Levart wrote: > >> On 11/04/2014 04:19 PM, David Chase wrote: >>> On 2014-11-04, at 5:07 AM, Peter Levart wrote: >>>> Are you thinking of an IdentityHashMap type of hash table (no linked-list of elements for same bucket, just search for 1st free slot on insert)? The problem would be how to pre-size the array. Count declared members? >>> It can?t be an identityHashMap, because we are interning member names. >> >> I know it can't be IdentityHashMap - I just wondered if you were thinking of an IdentityHashMap-like data structure in contrast to standard HashMap-like. Not in terms of equality/hashCode used, but in terms of internal data structure. IdentityHashMap is just an array of elements (well pairs of them - key, value are placed in two consecutive array slots). Lookup searches for element linearly in the array starting from hashCode based index to the element if found or 1st empty array slot. It's very easy to implement if the only operations are get() and put() and could be used for interning and as a shared structure for VM to scan, but array has to be sized to at least 3/2 the number of elements for performance to not degrade. >> >>> In spite of my grumbling about benchmarking, I?m inclined to do that and try a couple of experiments. >>> One possibility would be to use two data structures, one for interning, the other for communication with the VM. >>> Because there?s no lookup in the VM data stucture it can just be an array that gets elements appended, >>> and the synchronization dance is much simpler. >>> >>> For interning, maybe I use a ConcurrentHashMap, and I try the following idiom: >>> >>> mn = resolve(args) >>> // deal with any errors >>> mn? = chm.get(mn) >>> if (mn? != null) return mn? // hoped-for-common-case >>> >>> synchronized (something) { >>> mn? = chm.get(mn) >>> if (mn? != null) return mn? >>> txn_class = mn.getDeclaringClass() >>> >>> while (true) { >>> redef_count = txn_class.redefCount() >>> mn = resolve(args) >>> >>> shared_array.add(mn); >>> // barrier, because we are a paranoid >>> if (redef_count = redef_count.redefCount()) { >>> chm.add(mn); // safe to publish to other Java threads. >>> return mn; >>> } >>> shared_array.drop_last(); // Try again >>> } >>> } >>> >>> (Idiom gets slightly revised for the one or two other intern use cases, but this is the basic idea). >> >> Yes, that's similar to what I suggested by using a linked-list of MemberName(s) instead of the "shared_array" (easier to reason about ordering of writes) and a sorted array of MemberName(s) instead of the "chm" in your scheme above. ConcurrentHashMap would certainly be the most performant solution in terms of lookup/insertion-time and concurrent throughput, but it will use more heap than a simple packed array of MemberNames. CHM is much better now in JDK8 though regarding heap use. >> >> A combination of the two approaches is also possible: >> >> - instead of maintaining a "shared_array" of MemberName(s), have them form a linked-list (you trade a slot in array for 'next' pointer in MemberName) >> - use ConcurrentHashMap for interning. >> >> Regards, Peter >> >>> >>> David >>> >>>>> And another way to view this is that we?re now quibbling about performance, when we still >>>>> have an existing correctness problem that this patch solves, so maybe we should just get this >>>>> done and then file an RFE. >>>> Perhaps, yes. But note that questions about JMM and ordering of writes to array elements are about correctness, not performance. >>>> >>>> Regards, Peter >>>> >>>>> David > From joel.franck at oracle.com Fri Nov 7 21:36:39 2014 From: joel.franck at oracle.com (=?windows-1252?Q?Joel_Borggr=E9n-Franck?=) Date: Fri, 7 Nov 2014 22:36:39 +0100 Subject: RFR: 8062771: Core reflection should use final fields whenever possible In-Reply-To: References: <843282E1-DC2B-48B0-B0E4-A60B5078D75A@oracle.com> Message-ID: <45A50B54-43A3-4134-9EB5-D1AEDA649587@oracle.com> Hi Martin, Thanks for the clarification. On 6 nov 2014, at 20:51, Martin Buchholz wrote: > Hi Joel, > > On Thu, Nov 6, 2014 at 2:48 AM, Joel Borggr?n-Franck > wrote: >> Hi, >> >> I?m having a hard time following this thread, which webrev has been updated, for which release, fixing which issue? > > I am submitting the changeset for JDK-8062771, adding the finals and > the (failed reproduction) test. > Hopefully there will be followon changes to add even more thread safety. > Great. Are we sure the accidental fix of making ClassRepository volatile is good on all platforms? I can?t establish the happens before relations that would prove the fix but then again I?m not an expert on concurrency. >> Martin, as far as I can see you are the only one of us who has replied to this thread who is a jdk7u committer (or reviewer). > > I am entirely in favor of having more reviewers. > > We are not planning to push these into jdk7u or jdk8u ourselves, but > we are willing to help anyone who wants to take on that task. Joel, > are you volunteering? > We believe all versions of stock openjdk still have (rarely seen) > thread safety issues in core reflection. We have applied changes > locally at Google to fix those. > I?ll take care of 8u, and I can hunt down someone to fix this in 7u if we can find a suitable reviewer (sigh). cheers /Joel From martinrb at google.com Fri Nov 7 22:48:35 2014 From: martinrb at google.com (Martin Buchholz) Date: Fri, 7 Nov 2014 14:48:35 -0800 Subject: RFR: 8062771: Core reflection should use final fields whenever possible In-Reply-To: <45A50B54-43A3-4134-9EB5-D1AEDA649587@oracle.com> References: <843282E1-DC2B-48B0-B0E4-A60B5078D75A@oracle.com> <45A50B54-43A3-4134-9EB5-D1AEDA649587@oracle.com> Message-ID: Hi Joel, Thanks for volunteering. I foisted all I have in https://bugs.openjdk.java.net/browse/JDK-8064391 I volunteer to be your reviewer for the backports. On Fri, Nov 7, 2014 at 1:36 PM, Joel Borggr?n-Franck wrote: > Hi Martin, > > Thanks for the clarification. > > On 6 nov 2014, at 20:51, Martin Buchholz wrote: > >> Hi Joel, >> >> On Thu, Nov 6, 2014 at 2:48 AM, Joel Borggr?n-Franck >> wrote: >>> Hi, >>> >>> I?m having a hard time following this thread, which webrev has been updated, for which release, fixing which issue? >> >> I am submitting the changeset for JDK-8062771, adding the finals and >> the (failed reproduction) test. >> Hopefully there will be followon changes to add even more thread safety. >> > > Great. > > Are we sure the accidental fix of making ClassRepository volatile is good on all platforms? I can?t establish the happens before relations that would prove the fix but then again I?m not an expert on concurrency. > >>> Martin, as far as I can see you are the only one of us who has replied to this thread who is a jdk7u committer (or reviewer). >> >> I am entirely in favor of having more reviewers. >> >> We are not planning to push these into jdk7u or jdk8u ourselves, but >> we are willing to help anyone who wants to take on that task. Joel, >> are you volunteering? >> We believe all versions of stock openjdk still have (rarely seen) >> thread safety issues in core reflection. We have applied changes >> locally at Google to fix those. >> > > I?ll take care of 8u, and I can hunt down someone to fix this in 7u if we can find a suitable reviewer (sigh). > > cheers > /Joel > > From Alan.Bateman at oracle.com Sat Nov 8 08:59:09 2014 From: Alan.Bateman at oracle.com (Alan Bateman) Date: Sat, 08 Nov 2014 08:59:09 +0000 Subject: RFR: 8062194: java.util.jar.Attributes should use insertion-ordered iteration In-Reply-To: References: <544E96AC.90904@oracle.com> <544FF3ED.2020901@oracle.com> <545C966F.8080101@oracle.com> Message-ID: <545DDB5D.4030401@oracle.com> On 07/11/2014 20:52, Martin Buchholz wrote: > : > I don't much care either way, so the spec change to > Attributes(Attributes) is reverted, as you wish. Webrev refreshed. Thanks, I think it's a better with just the update to the class description. -Alan From peter.levart at gmail.com Sat Nov 8 15:07:38 2014 From: peter.levart at gmail.com (Peter Levart) Date: Sat, 08 Nov 2014 16:07:38 +0100 Subject: [9] RFR(L) 8013267 : move MemberNameTable from native code to Java heap, use to intern MemberNames In-Reply-To: <39826508-110B-4FCE-9A58-8C3D1B9FC7DE@oracle.com> References: <594FE416-D445-426E-B7A9-C75F012ADE16@oracle.com> <0BDCD5DD-CBFB-4A3D-9BAD-7F9912E9237C@oracle.com> <5453C230.8010709@oracle.com> <9747A3A3-B7F4-407A-8E00-1E647D9DC2D1@oracle.com> <1D195ED3-8BD2-46CB-9D70-29CF809D9F5A@oracle.com> <5456FB59.60905@oracle.com> <632A5C98-B386-4625-BE12-355241581955@oracle.com> <5457AA75.8090103@gmail.com> <5457E0F9.8090004@gmail.com> <5458A57C.4060208@gmail.com> <260D49F5-6380-4FC3-A900-6CD9AB3ED6F7@oracle.com> <5459034E.8070809@gmail.com> <39826508-110B-4FCE-9A58-8C3D1B9FC7DE@oracle.com> Message-ID: <545E31BA.3070500@gmail.com> Hi David, As previously, I feel competent to comment only the Java side of the patch. Using linked list to publish to VM is a safer and easier way for the desired purpose and using a separate data structure for interning makes it easier to change it in the future should need arise. That need may just be around the corner as there are people (Martin Buchholz, Duncan MacGregor, for example) that use classes with huge number of methods... Here are few comments for the new webrev (MemberName): 78 @SuppressWarnings("rawtypes") //Comparable in next line 79 /*non-public*/ final class MemberName implements Member, Comparable, Cloneable { Since MemberName is final and can only be compared to itself, you could make it implement Comparable and eliminate the @SuppressWarnings more MemberName: 84 private volatile MemberName next; // used for a linked list of MemberNames known to VM ... 1375 private static class ClassData { 1376 /** 1377 * This needs to be a simple data structure because we need to access 1378 * and update its elements from the JVM. Note that the Java side controls 1379 * the allocation and order of elements in the array; the JVM modifies 1380 * fields of those elements during class redefinition. 1381 */ 1382 private volatile MemberName[] elementData; 1383 private volatile MemberName publishedToVM; 1384 private volatile int size; 1385 1386 /** 1387 * Interns a member name in the member name table. 1388 * Returns null if a race with the jvm occurred. Races are detected 1389 * by checking for changes in the class redefinition count that occur 1390 * before an intern is complete. 1391 * 1392 * @param klass class whose redefinition count is checked. 1393 * @param memberName member name to be interned 1394 * @param redefined_count the value of classRedefinedCount() observed before 1395 * creation of the MemberName that is being interned. 1396 * @return null if a race occurred, otherwise the interned MemberName. 1397 */ 1398 @SuppressWarnings({"unchecked","rawtypes"}) 1399 public MemberName intern(Class klass, MemberName memberName, int redefined_count) { 1400 if (elementData == null) { 1401 synchronized (this) { 1402 if (elementData == null) { 1403 elementData = new MemberName[1]; 1404 } 1405 } 1406 } 1407 synchronized (this) { // this == ClassData 1408 final int index = Arrays.binarySearch(elementData, 0, size, memberName); 1409 if (index >= 0) { 1410 return elementData[index]; 1411 } 1412 // Not found, add carefully. 1413 return add(klass, ~index, memberName, redefined_count); 1414 } 1415 } ... 1426 private MemberName add(Class klass, int index, MemberName e, int redefined_count) { 1427 // First attempt publication to JVM, if that succeeds, 1428 // then record internally. 1429 e.next = publishedToVM; 1430 publishedToVM = e; 1431 storeFence(); 1432 if (redefined_count != jla.getClassRedefinedCount(klass)) { 1433 // Lost a race, back out publication and report failure. 1434 publishedToVM = e.next; 1435 return null; 1436 } Since you now synchronize lookup/add *and* lazy elementData construction on the same object (the ClassData instance), you can merge two synchronized blocks and simplify code. You can make MemberName.next, ClassData.elementData and ClassData.size be non-volatile (just ClassData.publishedToVM needs to be volatile) and ClassData.intern() can look something like that: public synchronized MemberName intern(Class klass, MemberName memberName, int redefined_count) { final int index; if (elementData == null) { elementData = new MemberName[1]; index = ~0; } else { index = Arrays.binarySearch(elementData, 0, size, memberName); if (index >= 0) return elementData[index]; } // Not found, add carefully. return add(klass, ~index, memberName, redefined_count); } // Note: no need for additional storeFence() in add()... private MemberName add(Class klass, int index, MemberName e, int redefined_count) { // First attempt publication to JVM, if that succeeds, // then record internally. e.next = publishedToVM; // volatile read of publishedToVM, followed by normal write of e.next... publishedToVM = e; // ...which is ordered before volatile write of publishedToVM... if (redefined_count != jla.getClassRedefinedCount(klass)) { // ...which is ordered before volatile read of klass.classRedefinedCount. // Lost a race, back out publication and report failure. publishedToVM = e.next; return null; } ... Now let's take for example one of the MemberName.make() methods that return interned MemberNames: 206 public static MemberName make(Method m, boolean wantSpecial) { 207 // Unreflected member names are resolved so intern them here. 208 MemberName tmp0 = null; 209 InternTransaction tx = new InternTransaction(m.getDeclaringClass()); 210 while (tmp0 == null) { 211 MemberName tmp = new MemberName(m, wantSpecial); 212 tmp0 = tx.tryIntern(tmp); 213 } 214 return tmp0; 215 } I'm trying to understand the workings of InternTransaction helper class (and find an example that breaks it). You create an instance of it, passing Method's declaringClass. You then (in retry loop) create a resolved MemberName from the Method and wantSpecial flag. This MemberName's clazz can apparently differ from Method's declaringClass. I don't know when and why this happens, but apparently it can (super method?), so in InternTransaction.tryIntern() you do... 363 if (member_name.isResolved()) { 364 if (member_name.clazz != tx_class) { 365 Class prev_tx_class = tx_class; 366 int prev_txn_token = txn_token; 367 tx_class = member_name.clazz; 368 txn_token = internTxnToken(tx_class); 369 // Zero is a special case. 370 if (txn_token != 0 || 371 prev_txn_token != internTxnToken(prev_tx_class)) { 372 // Resolved class is different and at least one 373 // redef of it occurred, therefore repeat with 374 // proper class for race consistency checking. 375 return null; 376 } 377 } 378 member_name = member_name.intern(txn_token); 379 if (member_name == null) { 380 // Update the token for the next try. 381 txn_token = internTxnToken(tx_class); 382 } 383 } Now let's assume that the resolved member_name.clazz differs from Method's declaringClass. Let's assume also that either member_name.clazz has had at least one redefinition or Method's declaringClass has been redefined between creating InternTransaction and reading member_name.clazz's txn_token. You return 'null' in such case, concluding that not only the resolved member_name.clazz redefinition matters, but Method's declaringClass redefinition can also invalidate resolved MemberName am I right? It would be helpful if I could understand when and how Method's declaringClass redefinition can affect member_name. Can it affect which clazz is resolved for member_name? Anyway, you return null in such case from an updated InternTransaction (tx_class and txn_token are now updated to have values for resolved member_name.clazz). In next round the checks of newly constructed and resolved member_name are not performed against Method's declaringClass but against previous round's member_name.clazz. Is this what is intended? I can see there has to be a stop condition for loop to end, but shouldn't checks for Method's declaringClass redefinition be performed in every iteration (in addition to the check for member_name.clazz redefinition if it differs from Method's declaringClass)? Regards, Peter On 11/07/2014 10:14 PM, David Chase wrote: > New webrev: > > bug:https://bugs.openjdk.java.net/browse/JDK-8013267 > > webrevs: > http://cr.openjdk.java.net/~drchase/8013267/jdk.06/ > http://cr.openjdk.java.net/~drchase/8013267/hotspot.06/ > > Changes since last: > > 1) refactored to put ClassData under java.lang.invoke.MemberName > > 2) split the data structure into two parts; handshake with JVM uses a linked list, > which makes for a simpler backout-if-race, and Java side continues to use the > simple sorted array. This should allow easier use of (for example) fancier > data structures (like ConcurrentHashMap) if this later proves necessary. > > 3) Cleaned up symbol references in the new hotspot code to go through vmSymbols. > > 4) renamed oldCapacity to oldSize > > 5) ran two different benchmarks and saw no change in performance. > a) nashorn ScriptTest (seehttps://bugs.openjdk.java.net/browse/JDK-8014288 ) > b) JMH microbenchmarks > (see bug comments for details) > > And it continues to pass the previously-failing tests, as well as the new test > which has been added to hotspot/test/compiler/jsr292 . > > David > > On 2014-11-04, at 3:54 PM, David Chase wrote: > >> I?m working on the initial benchmarking, and so far this arrangement (with synchronization >> and binary search for lookup, lots of barriers and linear cost insertion) has not yet been any >> slower. >> >> I am nonetheless tempted by the 2-tables solution, because I think the simpler JVM-side >> interface that it allows is desirable. >> >> David >> >> On 2014-11-04, at 11:48 AM, Peter Levart wrote: >> >>> On 11/04/2014 04:19 PM, David Chase wrote: >>>> On 2014-11-04, at 5:07 AM, Peter Levart wrote: >>>>> Are you thinking of an IdentityHashMap type of hash table (no linked-list of elements for same bucket, just search for 1st free slot on insert)? The problem would be how to pre-size the array. Count declared members? >>>> It can?t be an identityHashMap, because we are interning member names. >>> I know it can't be IdentityHashMap - I just wondered if you were thinking of an IdentityHashMap-like data structure in contrast to standard HashMap-like. Not in terms of equality/hashCode used, but in terms of internal data structure. IdentityHashMap is just an array of elements (well pairs of them - key, value are placed in two consecutive array slots). Lookup searches for element linearly in the array starting from hashCode based index to the element if found or 1st empty array slot. It's very easy to implement if the only operations are get() and put() and could be used for interning and as a shared structure for VM to scan, but array has to be sized to at least 3/2 the number of elements for performance to not degrade. >>> >>>> In spite of my grumbling about benchmarking, I?m inclined to do that and try a couple of experiments. >>>> One possibility would be to use two data structures, one for interning, the other for communication with the VM. >>>> Because there?s no lookup in the VM data stucture it can just be an array that gets elements appended, >>>> and the synchronization dance is much simpler. >>>> >>>> For interning, maybe I use a ConcurrentHashMap, and I try the following idiom: >>>> >>>> mn = resolve(args) >>>> // deal with any errors >>>> mn? = chm.get(mn) >>>> if (mn? != null) return mn? // hoped-for-common-case >>>> >>>> synchronized (something) { >>>> mn? = chm.get(mn) >>>> if (mn? != null) return mn? >>>> txn_class = mn.getDeclaringClass() >>>> >>>> while (true) { >>>> redef_count = txn_class.redefCount() >>>> mn = resolve(args) >>>> >>>> shared_array.add(mn); >>>> // barrier, because we are a paranoid >>>> if (redef_count = redef_count.redefCount()) { >>>> chm.add(mn); // safe to publish to other Java threads. >>>> return mn; >>>> } >>>> shared_array.drop_last(); // Try again >>>> } >>>> } >>>> >>>> (Idiom gets slightly revised for the one or two other intern use cases, but this is the basic idea). >>> Yes, that's similar to what I suggested by using a linked-list of MemberName(s) instead of the "shared_array" (easier to reason about ordering of writes) and a sorted array of MemberName(s) instead of the "chm" in your scheme above. ConcurrentHashMap would certainly be the most performant solution in terms of lookup/insertion-time and concurrent throughput, but it will use more heap than a simple packed array of MemberNames. CHM is much better now in JDK8 though regarding heap use. >>> >>> A combination of the two approaches is also possible: >>> >>> - instead of maintaining a "shared_array" of MemberName(s), have them form a linked-list (you trade a slot in array for 'next' pointer in MemberName) >>> - use ConcurrentHashMap for interning. >>> >>> Regards, Peter >>> >>>> David >>>> >>>>>> And another way to view this is that we?re now quibbling about performance, when we still >>>>>> have an existing correctness problem that this patch solves, so maybe we should just get this >>>>>> done and then file an RFE. >>>>> Perhaps, yes. But note that questions about JMM and ordering of writes to array elements are about correctness, not performance. >>>>> >>>>> Regards, Peter >>>>> >>>>>> David From Ulf.Zibis at CoSoCo.de Sat Nov 8 17:46:43 2014 From: Ulf.Zibis at CoSoCo.de (Ulf Zibis) Date: Sat, 08 Nov 2014 18:46:43 +0100 Subject: Review request: JDK-8055723 Replace concat String to append in StringBuilder parameters In-Reply-To: References: <5445A836.8090005@CoSoCo.de> Message-ID: <545E5703.7060902@CoSoCo.de> Hi Ot?vio, in sun/tools/jstat/SyntaxException.java I see a possible enhencement (maybe applies to other places too): 65 public SyntaxException(int lineno, Set expected, Token found) { 66 StringBuilder msg = new StringBuilder(A + B * expected.size()); 67 68 msg.append("Syntax error at line ").append(lineno).append(": Expected one of \'"); 69 71 for (String keyWord : expected) { 72 msg.append(keyWord).append('|'); 73 } 74 // if (!expected.isEmpty()) // only needed if expected may be empty. 75 msg.setLength(msg.length() - 1); 76 81 message = msg.append("\', Found ").append(found.toMessage()).toString(); 83 } ***** Additionally at many places you could similarly introduce the foreach syntax. -Ulf Am 02.11.2014 um 15:45 schrieb Ot?vio Gon?alves de Santana: > Could another reviewer look these codes, please. > http://cr.openjdk.java.net/~weijun/8055723/webrev.00/ > > > On Fri, Oct 24, 2014 at 3:25 AM, Ot?vio Gon?alves de Santana > wrote: > > Thank you Ulf. > I removed the fix in toString method and in debug classes: > http://cr.openjdk.java.net/~weijun/8055723/webrev.00/ > > > On Mon, Oct 20, 2014 at 10:26 PM, Ulf Zibis > > wrote: > > > Am 21.10.2014 um 01:02 schrieb Ot?vio Gon?alves de Santana: > > BUGURL: https://bugs.openjdk.java.net/browse/JDK-8055723 > > > WEBREV: http://cr.openjdk.java.net/~weijun/8055723/client/webrev.02/ > > WEBREV: http://cr.openjdk.java.net/~weijun/8055723/core/webrev.03/ > > > > I did not look through all sources. > In Scanner.java I discovered: > 1307 sb.append("[delimiters=").append(delimPattern).append(']'); > 1308 sb.append("[position=").append(position).append(']'); > ... > Maybe better: > 1307 sb.append("[delimiters=").append(delimPattern); > 1308 sb.append("][position=").append(position); > ... > > -Ulf > > > > > -- > Ot?vio Gon?alves de Santana > > blog: http://otaviosantana.blogspot.com.br/ > twitter: http://twitter.com/otaviojava > site: _http://about.me/otaviojava_ > 55 (11) 98255-3513 > > > > > -- > Ot?vio Gon?alves de Santana > > blog: http://otaviosantana.blogspot.com.br/ > twitter: http://twitter.com/otaviojava > site: _http://about.me/otaviojava_ > 55 (11) 98255-3513 From ivan.gerasimov at oracle.com Sat Nov 8 22:00:51 2014 From: ivan.gerasimov at oracle.com (Ivan Gerasimov) Date: Sun, 09 Nov 2014 01:00:51 +0300 Subject: RFR: 8062771: Core reflection should use final fields whenever possible In-Reply-To: References: <843282E1-DC2B-48B0-B0E4-A60B5078D75A@oracle.com> <45A50B54-43A3-4134-9EB5-D1AEDA649587@oracle.com> Message-ID: <545E9293.5010203@oracle.com> Hi! I think I can handle the backport to jdk7, if you still need a volunteer. Sincerely yours, Ivan On 08.11.2014 1:48, Martin Buchholz wrote: > Hi Joel, > > Thanks for volunteering. I foisted all I have in > > https://bugs.openjdk.java.net/browse/JDK-8064391 > > I volunteer to be your reviewer for the backports. > > On Fri, Nov 7, 2014 at 1:36 PM, Joel Borggr?n-Franck > wrote: >> Hi Martin, >> >> Thanks for the clarification. >> >> On 6 nov 2014, at 20:51, Martin Buchholz wrote: >> >>> Hi Joel, >>> >>> On Thu, Nov 6, 2014 at 2:48 AM, Joel Borggr?n-Franck >>> wrote: >>>> Hi, >>>> >>>> I?m having a hard time following this thread, which webrev has been updated, for which release, fixing which issue? >>> I am submitting the changeset for JDK-8062771, adding the finals and >>> the (failed reproduction) test. >>> Hopefully there will be followon changes to add even more thread safety. >>> >> Great. >> >> Are we sure the accidental fix of making ClassRepository volatile is good on all platforms? I can?t establish the happens before relations that would prove the fix but then again I?m not an expert on concurrency. >> >>>> Martin, as far as I can see you are the only one of us who has replied to this thread who is a jdk7u committer (or reviewer). >>> I am entirely in favor of having more reviewers. >>> >>> We are not planning to push these into jdk7u or jdk8u ourselves, but >>> we are willing to help anyone who wants to take on that task. Joel, >>> are you volunteering? >>> We believe all versions of stock openjdk still have (rarely seen) >>> thread safety issues in core reflection. We have applied changes >>> locally at Google to fix those. >>> >> I?ll take care of 8u, and I can hunt down someone to fix this in 7u if we can find a suitable reviewer (sigh). >> >> cheers >> /Joel >> >> > From pbenedict at apache.org Sun Nov 9 06:01:06 2014 From: pbenedict at apache.org (Paul Benedict) Date: Sun, 9 Nov 2014 00:01:06 -0600 Subject: Properties::setProperty should document NPE? Message-ID: I accidentally passed a null to setProperty() and got an NPE. Okay. The javadoc says this method relies on Hashtable, which, in turn, is documented to throw an NPE when put() key/value is null. So, since setProperty() bubbles up the exception, I think this method should gain a @throws tag to make the exception explicit. Worth creating a JIRA for this? (I don't have a JIRA account) Cheers, Paul From peter.levart at gmail.com Sun Nov 9 12:55:10 2014 From: peter.levart at gmail.com (Peter Levart) Date: Sun, 09 Nov 2014 13:55:10 +0100 Subject: [9] RFR(L) 8013267 : move MemberNameTable from native code to Java heap, use to intern MemberNames In-Reply-To: <39826508-110B-4FCE-9A58-8C3D1B9FC7DE@oracle.com> References: <594FE416-D445-426E-B7A9-C75F012ADE16@oracle.com> <0BDCD5DD-CBFB-4A3D-9BAD-7F9912E9237C@oracle.com> <5453C230.8010709@oracle.com> <9747A3A3-B7F4-407A-8E00-1E647D9DC2D1@oracle.com> <1D195ED3-8BD2-46CB-9D70-29CF809D9F5A@oracle.com> <5456FB59.60905@oracle.com> <632A5C98-B386-4625-BE12-355241581955@oracle.com> <5457AA75.8090103@gmail.com> <5457E0F9.8090004@gmail.com> <5458A57C.4060208@gmail.com> <260D49F5-6380-4FC3-A900-6CD9AB3ED6F7@oracle.com> <5459034E.8070809@gmail.com> <39826508-110B-4FCE-9A58-8C3D1B9FC7DE@oracle.com> Message-ID: <545F642E.30205@gmail.com> Hi David, I played a little with the idea of having a hash table instead of packed sorted array for interning. Using ConcurrentHashMap would present quite some memory overhead. A more compact representation is possible in the form of a linear-scan hash table where elements of array are MemberNames themselves: http://cr.openjdk.java.net/~plevart/misc/MemberName.intern/jdk.06.diff/ This is a drop-in replacement for MemberName on top of your jdk.06 patch. If you have some time, you can run this with your performance tests to see if it presents any difference. If not, then perhaps this interning is not so performance critical after all. Regards, Peter On 11/07/2014 10:14 PM, David Chase wrote: > New webrev: > > bug: https://bugs.openjdk.java.net/browse/JDK-8013267 > > webrevs: > http://cr.openjdk.java.net/~drchase/8013267/jdk.06/ > http://cr.openjdk.java.net/~drchase/8013267/hotspot.06/ > > Changes since last: > > 1) refactored to put ClassData under java.lang.invoke.MemberName > > 2) split the data structure into two parts; handshake with JVM uses a linked list, > which makes for a simpler backout-if-race, and Java side continues to use the > simple sorted array. This should allow easier use of (for example) fancier > data structures (like ConcurrentHashMap) if this later proves necessary. > > 3) Cleaned up symbol references in the new hotspot code to go through vmSymbols. > > 4) renamed oldCapacity to oldSize > > 5) ran two different benchmarks and saw no change in performance. > a) nashorn ScriptTest (see https://bugs.openjdk.java.net/browse/JDK-8014288 ) > b) JMH microbenchmarks > (see bug comments for details) > > And it continues to pass the previously-failing tests, as well as the new test > which has been added to hotspot/test/compiler/jsr292 . > > David > > On 2014-11-04, at 3:54 PM, David Chase wrote: > >> I?m working on the initial benchmarking, and so far this arrangement (with synchronization >> and binary search for lookup, lots of barriers and linear cost insertion) has not yet been any >> slower. >> >> I am nonetheless tempted by the 2-tables solution, because I think the simpler JVM-side >> interface that it allows is desirable. >> >> David >> >> On 2014-11-04, at 11:48 AM, Peter Levart wrote: >> >>> On 11/04/2014 04:19 PM, David Chase wrote: >>>> On 2014-11-04, at 5:07 AM, Peter Levart wrote: >>>>> Are you thinking of an IdentityHashMap type of hash table (no linked-list of elements for same bucket, just search for 1st free slot on insert)? The problem would be how to pre-size the array. Count declared members? >>>> It can?t be an identityHashMap, because we are interning member names. >>> I know it can't be IdentityHashMap - I just wondered if you were thinking of an IdentityHashMap-like data structure in contrast to standard HashMap-like. Not in terms of equality/hashCode used, but in terms of internal data structure. IdentityHashMap is just an array of elements (well pairs of them - key, value are placed in two consecutive array slots). Lookup searches for element linearly in the array starting from hashCode based index to the element if found or 1st empty array slot. It's very easy to implement if the only operations are get() and put() and could be used for interning and as a shared structure for VM to scan, but array has to be sized to at least 3/2 the number of elements for performance to not degrade. >>> >>>> In spite of my grumbling about benchmarking, I?m inclined to do that and try a couple of experiments. >>>> One possibility would be to use two data structures, one for interning, the other for communication with the VM. >>>> Because there?s no lookup in the VM data stucture it can just be an array that gets elements appended, >>>> and the synchronization dance is much simpler. >>>> >>>> For interning, maybe I use a ConcurrentHashMap, and I try the following idiom: >>>> >>>> mn = resolve(args) >>>> // deal with any errors >>>> mn? = chm.get(mn) >>>> if (mn? != null) return mn? // hoped-for-common-case >>>> >>>> synchronized (something) { >>>> mn? = chm.get(mn) >>>> if (mn? != null) return mn? >>>> txn_class = mn.getDeclaringClass() >>>> >>>> while (true) { >>>> redef_count = txn_class.redefCount() >>>> mn = resolve(args) >>>> >>>> shared_array.add(mn); >>>> // barrier, because we are a paranoid >>>> if (redef_count = redef_count.redefCount()) { >>>> chm.add(mn); // safe to publish to other Java threads. >>>> return mn; >>>> } >>>> shared_array.drop_last(); // Try again >>>> } >>>> } >>>> >>>> (Idiom gets slightly revised for the one or two other intern use cases, but this is the basic idea). >>> Yes, that's similar to what I suggested by using a linked-list of MemberName(s) instead of the "shared_array" (easier to reason about ordering of writes) and a sorted array of MemberName(s) instead of the "chm" in your scheme above. ConcurrentHashMap would certainly be the most performant solution in terms of lookup/insertion-time and concurrent throughput, but it will use more heap than a simple packed array of MemberNames. CHM is much better now in JDK8 though regarding heap use. >>> >>> A combination of the two approaches is also possible: >>> >>> - instead of maintaining a "shared_array" of MemberName(s), have them form a linked-list (you trade a slot in array for 'next' pointer in MemberName) >>> - use ConcurrentHashMap for interning. >>> >>> Regards, Peter >>> >>>> David >>>> >>>>>> And another way to view this is that we?re now quibbling about performance, when we still >>>>>> have an existing correctness problem that this patch solves, so maybe we should just get this >>>>>> done and then file an RFE. >>>>> Perhaps, yes. But note that questions about JMM and ordering of writes to array elements are about correctness, not performance. >>>>> >>>>> Regards, Peter >>>>> >>>>>> David From aleksey.shipilev at oracle.com Sun Nov 9 18:45:35 2014 From: aleksey.shipilev at oracle.com (Aleksey Shipilev) Date: Sun, 09 Nov 2014 21:45:35 +0300 Subject: RFR (S) 8059677: Thread.getName() instantiates Strings Message-ID: <545FB64F.7090705@oracle.com> Hi, Thread.getName() returns String, and does new String instantiation every time, because the thread name is stored in char[]. Even though we use a private String constructor that shares the char[] array without copying it, this still hurts some use cases (think extra-fast logging). To the extent some people actually maintain Map to avoid it. https://bugs.openjdk.java.net/browse/JDK-8059677 Here's the attempt to maintain String instead of char[]: http://cr.openjdk.java.net/~shade/8059677/webrev.01.jdk/ http://cr.openjdk.java.net/~shade/8059677/webrev.01.hs/ JDK changes are trivial, but HS changes require some rewiring, since VM treats Thread.name specially. However, it turns out we can make a contained change, since the getter is used sparingly, and setter seems to be not used at all. Any trouble with this change? Testing: JPRT, manual tests, jdk/test/java/lang/Thread jtreg, hotspot/test/runtime/ jtreg, vm.quick.testlist, nsk.jvmti.testlist, svc.quick.testlist Thanks, -Aleksey. From joel.franck at oracle.com Mon Nov 10 09:00:15 2014 From: joel.franck at oracle.com (Joel =?utf-8?Q?Borggr=C3=A9n-Franck?=) Date: Mon, 10 Nov 2014 10:00:15 +0100 Subject: RFR: 8062771: Core reflection should use final fields whenever possible In-Reply-To: <545E9293.5010203@oracle.com> References: <843282E1-DC2B-48B0-B0E4-A60B5078D75A@oracle.com> <45A50B54-43A3-4134-9EB5-D1AEDA649587@oracle.com> <545E9293.5010203@oracle.com> Message-ID: <20141110090015.GA22573@oracle.com> That would be great cheers /Joel On 2014-11-09, Ivan Gerasimov wrote: > Hi! > > I think I can handle the backport to jdk7, if you still need a volunteer. > > Sincerely yours, > Ivan > > On 08.11.2014 1:48, Martin Buchholz wrote: > >Hi Joel, > > > >Thanks for volunteering. I foisted all I have in > > > >https://bugs.openjdk.java.net/browse/JDK-8064391 > > > >I volunteer to be your reviewer for the backports. > > > >On Fri, Nov 7, 2014 at 1:36 PM, Joel Borggr?n-Franck > > wrote: > >>Hi Martin, > >> > >>Thanks for the clarification. > >> > >>On 6 nov 2014, at 20:51, Martin Buchholz wrote: > >> > >>>Hi Joel, > >>> > >>>On Thu, Nov 6, 2014 at 2:48 AM, Joel Borggr?n-Franck > >>> wrote: > >>>>Hi, > >>>> > >>>>I?m having a hard time following this thread, which webrev has been updated, for which release, fixing which issue? > >>>I am submitting the changeset for JDK-8062771, adding the finals and > >>>the (failed reproduction) test. > >>>Hopefully there will be followon changes to add even more thread safety. > >>> > >>Great. > >> > >>Are we sure the accidental fix of making ClassRepository volatile is good on all platforms? I can?t establish the happens before relations that would prove the fix but then again I?m not an expert on concurrency. > >> > >>>>Martin, as far as I can see you are the only one of us who has replied to this thread who is a jdk7u committer (or reviewer). > >>>I am entirely in favor of having more reviewers. > >>> > >>>We are not planning to push these into jdk7u or jdk8u ourselves, but > >>>we are willing to help anyone who wants to take on that task. Joel, > >>>are you volunteering? > >>>We believe all versions of stock openjdk still have (rarely seen) > >>>thread safety issues in core reflection. We have applied changes > >>>locally at Google to fix those. > >>> > >>I?ll take care of 8u, and I can hunt down someone to fix this in 7u if we can find a suitable reviewer (sigh). > >> > >>cheers > >>/Joel > >> > >> > > > From paul.sandoz at oracle.com Mon Nov 10 09:43:17 2014 From: paul.sandoz at oracle.com (Paul Sandoz) Date: Mon, 10 Nov 2014 10:43:17 +0100 Subject: RFR: 8063147: Class.getFields spec should state that fields are inherited from superinterfaces In-Reply-To: References: Message-ID: <2CFBB9AC-6794-4799-B51D-48A179F8CE91@oracle.com> On Nov 6, 2014, at 8:07 PM, Martin Buchholz wrote: > Sorry, Paul, I missed your offer on the other thread to take this to > CCC. It's yours. > > On Thu, Nov 6, 2014 at 10:50 AM, Martin Buchholz wrote: >> Hi Joe, >> >> I'd like you to do a code review. >> >> https://bugs.openjdk.java.net/browse/JDK-8063147 >> Webrev: http://cr.openjdk.java.net/~martin/webrevs/openjdk9/Class-getFields-spec/ >> >> getFields was embarrassingly undertested, so I added some. >> Test looks good. >> Please shepherd through CCC. Kick started. Paul. From chris.hegarty at oracle.com Mon Nov 10 10:40:04 2014 From: chris.hegarty at oracle.com (Chris Hegarty) Date: Mon, 10 Nov 2014 10:40:04 +0000 Subject: RFR: 8063147: Class.getFields spec should state that fields are inherited from superinterfaces In-Reply-To: <2CFBB9AC-6794-4799-B51D-48A179F8CE91@oracle.com> References: <2CFBB9AC-6794-4799-B51D-48A179F8CE91@oracle.com> Message-ID: <54609604.2070403@oracle.com> On 10/11/14 09:43, Paul Sandoz wrote: > On Nov 6, 2014, at 8:07 PM, Martin Buchholz wrote: >> Sorry, Paul, I missed your offer on the other thread to take this to >> CCC. It's yours. >> >> On Thu, Nov 6, 2014 at 10:50 AM, Martin Buchholz wrote: >>> Hi Joe, >>> >>> I'd like you to do a code review. >>> >>> https://bugs.openjdk.java.net/browse/JDK-8063147 >>> Webrev: http://cr.openjdk.java.net/~martin/webrevs/openjdk9/Class-getFields-spec/ The change looks good to me. >>> getFields was embarrassingly undertested, so I added some. >>> > > Test looks good. Yes, very nice. >>> Please shepherd through CCC. > > Kick started. Reviewed. -Chris. > Paul. > From chris.hegarty at oracle.com Mon Nov 10 11:52:08 2014 From: chris.hegarty at oracle.com (Chris Hegarty) Date: Mon, 10 Nov 2014 11:52:08 +0000 Subject: RFR (S) 8059677: Thread.getName() instantiates Strings In-Reply-To: <545FB64F.7090705@oracle.com> References: <545FB64F.7090705@oracle.com> Message-ID: <5460A6E8.9050506@oracle.com> Aleksey, I have only looked at the libraries changes, and I think they make sense . As in, I can find no reason why the name cannot be changed to be a String. Trivially, after your changes will NPE be thrown if setName(null), as it is today ? -Chris. On 09/11/14 18:45, Aleksey Shipilev wrote: > Hi, > > Thread.getName() returns String, and does new String instantiation every > time, because the thread name is stored in char[]. Even though we use a > private String constructor that shares the char[] array without copying > it, this still hurts some use cases (think extra-fast logging). To the > extent some people actually maintain Map to avoid it. > https://bugs.openjdk.java.net/browse/JDK-8059677 > > Here's the attempt to maintain String instead of char[]: > http://cr.openjdk.java.net/~shade/8059677/webrev.01.jdk/ > http://cr.openjdk.java.net/~shade/8059677/webrev.01.hs/ > > JDK changes are trivial, but HS changes require some rewiring, since VM > treats Thread.name specially. However, it turns out we can make a > contained change, since the getter is used sparingly, and setter seems > to be not used at all. Any trouble with this change? > > Testing: JPRT, manual tests, jdk/test/java/lang/Thread jtreg, > hotspot/test/runtime/ jtreg, vm.quick.testlist, nsk.jvmti.testlist, > svc.quick.testlist > > Thanks, > -Aleksey. > From joel.franck at oracle.com Mon Nov 10 11:45:44 2014 From: joel.franck at oracle.com (Joel =?utf-8?Q?Borggr=C3=A9n-Franck?=) Date: Mon, 10 Nov 2014 12:45:44 +0100 Subject: Review request for 8055063: Parameter#toString() fails w/ AIOOBE for ctr of inner class w/ generic type In-Reply-To: <545A9C84.1060805@oracle.com> References: <5453B58A.1090507@oracle.com> <55A63A8D-1D0E-4D48-8CC4-D9326A57586E@oracle.com> <545A9C84.1060805@oracle.com> Message-ID: <20141110114544.GC22573@oracle.com> Hi Eric, The changes to Executable.java and Parameter.java looks good. The test could still need some cleaning up: @summary should be before "action tags", put it right after @bug I don't think you need a separate file for InnerClassToStringNoParameters.java , should be able to accomplish the same with jtreg tags, * @clean InnerClassToString * @compile -parameters InnerClassToString.java * @run main InnerClassToString * @clean InnerClassToString * @compile InnerClassToString.java * @run main InnerClassToString I think this is cleaner, YMMV. The main method suffers from a lot of code duplication (and an absence of line breaks), you can extract a test method since the test just operates on two arrays of Type[] and call it with the parameter type array and the answer array. cheers /Joel On 2014-11-05, Eric McCorkle wrote: > Thanks, Joel. I've applied your fixes and re-upped the webrev. > > On 11/05/14 10:10, Joel Borggr?n-Franck wrote: > > Hi Eric, > > > > I think caching the real parameter type array on Executable is the wrong thing to do considering we can have a lot of instances of Executable but fairly few will be examined for parameters and the vast majority of those will never have this issue. > > > > Also, the tests are a bit minimal, I would expect you to check that the type of the parameter is correct. The tests should also have a @bug line. > > > > I think you are overly cautious here. I?m not sure we need to work around potential bugs in non-Java compilers. If language Y inserts synthetic/mandated parameters anywhere else than as a prefix they better generate a correct Signature (or non at all). This is a discussion we can defer however. > > > > If you flesh out the test and remove the caching I think this is a fine fix for both 8u and 9. > > > > cheers > > /Joel > > > > On 31 okt 2014, at 17:15, Eric McCorkle wrote: > > > >> Hello, > >> > >> Please review this patch which fixes issues that arise with > >> getGenericParameterTypes() and getAnnotatedParameterTypes() when there > >> are generic signatures and synthetic parameters. > >> > >> Please note that a complete fix is not possible for all cases. See > >> discussion on https://bugs.openjdk.java.net/browse/JDK-8062582 for details. > >> > >> This patch will cause Executable.getAnnotatedParameterTypes(), > >> Parameter.getAnnotatedType(), and Parameter.getParameterizedType() to > >> report the correct types in the following cases: > >> > >> * No generic signature is present. > >> * Both a generic signature and method parameters information are present > >> * A generic signature is present, but method parameters information is > >> not present, but the number of parameters in the generic signature and > >> the number of parameters in the method descriptor are the same. > >> > >> In the problematic case, where there is a generic signature, no method > >> parameters information, and the generic signature does not match the > >> method descriptor, these methods will return the correct /non/-generic > >> type, as there is no general way of associating parameters in the > >> generic signature with those in the method descriptor in this case. > >> > >> Please also note that there is currently a bug in javac which causes > >> type annotations' parameter indexes to be wrong when synthetic > >> parameters are generated: https://bugs.openjdk.java.net/browse/JDK-8029012. > >> > >> The bug report is here: > >> https://bugs.openjdk.java.net/browse/JDK-8055063 > >> > >> The webrev is here: > >> http://cr.openjdk.java.net/~emc/8055063/ > > > begin:vcard > fn:Eric McCorkle > n:McCorkle;Eric > org:Oracle Corporation;Java Platform Group > adr:BUR02-2117;;35 Network Drive;Burlington;MA;01803;United States > email;internet:eric.mccorkle at oracle.com > title:Senior Member of Technical Staff > tel;work:1-781-442-1028 > tel;cell:404-644-6360 > version:2.1 > end:vcard > From vladimir.x.ivanov at oracle.com Mon Nov 10 11:33:50 2014 From: vladimir.x.ivanov at oracle.com (Vladimir Ivanov) Date: Mon, 10 Nov 2014 15:33:50 +0400 Subject: [9] Review request : JDK-8059070: [TESTBUG] java/lang/invoke/LFCaching/LFMultiThreadCachingTest.java failed - timeout In-Reply-To: <545C9D70.6060308@oracle.com> References: <543D1DEE.8030206@oracle.com> <543F8520.9090408@oracle.com> <8F9DD322-1793-4739-8155-61CE7D2C5AAB@oracle.com> <543F9800.7040406@oracle.com> <543FC71F.8020504@oracle.com> <5440E39C.8070001@oracle.com> <5448E5A2.9030804@oracle.com> <544E1B65.1090400@oracle.com> <5450F8E4.2060808@oracle.com> <545C9D70.6060308@oracle.com> Message-ID: <5460A29E.6070206@oracle.com> Kontantin, sorry for the late response. In general, the fix looks good. I'd move timeout computation logic to Utils: + private static final long TIMEOUT = Utils.adjustTimeout(TimeUnit.SECONDS.toMillis(120)); Default value (120s) isn't specific to the tests, but an implementation detail of jtreg. Best regards, Vladimir Ivanov On 11/7/14, 2:22 PM, Konstantin Shefov wrote: > Gently reminder > > 29.10.2014 17:25, Konstantin Shefov ?????: >> Please, review a test bug fix. >> http://cr.openjdk.java.net/~kshefov/8059070/webrev.01/ >> >> -Konstantin >> >> On 27.10.2014 13:16, Konstantin Shefov wrote: >>> Kindly reminder >>> >>> On 23.10.2014 19:04, Paul Sandoz wrote: >>>> On Oct 23, 2014, at 1:25 PM, Konstantin Shefov >>>> wrote: >>>>> Gently reminder >>>>> >>>>> On 17.10.2014 13:38, Konstantin Shefov wrote: >>>>>> Hi, >>>>>> >>>>>> I have updated the webrev: >>>>>> http://cr.openjdk.java.net/~kshefov/8059070/webrev.01/ >>>>>> >>>> +1 >>>> >>>> Sorry for the delay, >>>> Paul. >>> >> > From david.holmes at oracle.com Mon Nov 10 12:56:40 2014 From: david.holmes at oracle.com (David Holmes) Date: Mon, 10 Nov 2014 22:56:40 +1000 Subject: RFR (S) 8059677: Thread.getName() instantiates Strings In-Reply-To: <5460A6E8.9050506@oracle.com> References: <545FB64F.7090705@oracle.com> <5460A6E8.9050506@oracle.com> Message-ID: <5460B608.4050909@oracle.com> On 10/11/2014 9:52 PM, Chris Hegarty wrote: > Aleksey, > > I have only looked at the libraries changes, and I think they make sense > . As in, I can find no reason why the name cannot be changed to be a > String. Very quick response, but IIRC this has been examined in the past and there were reasons why it can't/shouldn't be done. Will try to dig out more details in the morning. If String construction is a bottleneck just cache it. David ----- > Trivially, after your changes will NPE be thrown if setName(null), as it > is today ? > > -Chris. > > On 09/11/14 18:45, Aleksey Shipilev wrote: >> Hi, >> >> Thread.getName() returns String, and does new String instantiation every >> time, because the thread name is stored in char[]. Even though we use a >> private String constructor that shares the char[] array without copying >> it, this still hurts some use cases (think extra-fast logging). To the >> extent some people actually maintain Map to avoid it. >> https://bugs.openjdk.java.net/browse/JDK-8059677 >> >> Here's the attempt to maintain String instead of char[]: >> http://cr.openjdk.java.net/~shade/8059677/webrev.01.jdk/ >> http://cr.openjdk.java.net/~shade/8059677/webrev.01.hs/ >> >> JDK changes are trivial, but HS changes require some rewiring, since VM >> treats Thread.name specially. However, it turns out we can make a >> contained change, since the getter is used sparingly, and setter seems >> to be not used at all. Any trouble with this change? >> >> Testing: JPRT, manual tests, jdk/test/java/lang/Thread jtreg, >> hotspot/test/runtime/ jtreg, vm.quick.testlist, nsk.jvmti.testlist, >> svc.quick.testlist >> >> Thanks, >> -Aleksey. >> From chris.hegarty at oracle.com Mon Nov 10 13:53:24 2014 From: chris.hegarty at oracle.com (Chris Hegarty) Date: Mon, 10 Nov 2014 13:53:24 +0000 Subject: RFR (S) 8059677: Thread.getName() instantiates Strings In-Reply-To: <5460B608.4050909@oracle.com> References: <545FB64F.7090705@oracle.com> <5460A6E8.9050506@oracle.com> <5460B608.4050909@oracle.com> Message-ID: <5460C354.5000605@oracle.com> On 10/11/14 12:56, David Holmes wrote: > On 10/11/2014 9:52 PM, Chris Hegarty wrote: >> Aleksey, >> >> I have only looked at the libraries changes, and I think they make sense >> . As in, I can find no reason why the name cannot be changed to be a >> String. > > Very quick response, but IIRC this has been examined in the past and > there were reasons why it can't/shouldn't be done. Will try to dig out > more details in the morning. If there was previous discussion on this, that revealed some substantial issue, that would be great, but I can't recall, or find, it now. Hotspot express, and the desire for hotspot to run with different library versions, would certainly cause complication, but I don't believe that is an issue now. Just on that, the library changes are minimal, and if this were to proceed then they can accompany the hotspot change, as they make their way into jdk9/dev. Anyway, this should await your reply. -Chris. > If String construction is a bottleneck just cache it. > > David > ----- > >> Trivially, after your changes will NPE be thrown if setName(null), as it >> is today ? >> >> -Chris. >> >> On 09/11/14 18:45, Aleksey Shipilev wrote: >>> Hi, >>> >>> Thread.getName() returns String, and does new String instantiation every >>> time, because the thread name is stored in char[]. Even though we use a >>> private String constructor that shares the char[] array without copying >>> it, this still hurts some use cases (think extra-fast logging). To the >>> extent some people actually maintain Map to avoid it. >>> https://bugs.openjdk.java.net/browse/JDK-8059677 >>> >>> Here's the attempt to maintain String instead of char[]: >>> http://cr.openjdk.java.net/~shade/8059677/webrev.01.jdk/ >>> http://cr.openjdk.java.net/~shade/8059677/webrev.01.hs/ >>> >>> JDK changes are trivial, but HS changes require some rewiring, since VM >>> treats Thread.name specially. However, it turns out we can make a >>> contained change, since the getter is used sparingly, and setter seems >>> to be not used at all. Any trouble with this change? >>> >>> Testing: JPRT, manual tests, jdk/test/java/lang/Thread jtreg, >>> hotspot/test/runtime/ jtreg, vm.quick.testlist, nsk.jvmti.testlist, >>> svc.quick.testlist >>> >>> Thanks, >>> -Aleksey. >>> From aleksey.shipilev at oracle.com Mon Nov 10 14:08:51 2014 From: aleksey.shipilev at oracle.com (Aleksey Shipilev) Date: Mon, 10 Nov 2014 17:08:51 +0300 Subject: RFR (S) 8059677: Thread.getName() instantiates Strings In-Reply-To: <5460A6E8.9050506@oracle.com> References: <545FB64F.7090705@oracle.com> <5460A6E8.9050506@oracle.com> Message-ID: <5460C6F3.1080201@oracle.com> Hi Chris, Thanks for taking a look! On 11/10/2014 02:52 PM, Chris Hegarty wrote: > Trivially, after your changes will NPE be thrown if setName(null), as it > is today ? There is no way it could throw NPE now, therefore the behavior is different. The spec says nothing about NPE though, but it feels wrong to pass the null String to setNativeName. I should add Objects.requireNonNull there. Will wait for more feedbacks, and update the webrev. -Aleksey. From aleksey.shipilev at oracle.com Mon Nov 10 14:19:12 2014 From: aleksey.shipilev at oracle.com (Aleksey Shipilev) Date: Mon, 10 Nov 2014 17:19:12 +0300 Subject: RFR (S) 8059677: Thread.getName() instantiates Strings In-Reply-To: <5460C354.5000605@oracle.com> References: <545FB64F.7090705@oracle.com> <5460A6E8.9050506@oracle.com> <5460B608.4050909@oracle.com> <5460C354.5000605@oracle.com> Message-ID: <5460C960.9080509@oracle.com> Hi David, Chris, On 11/10/2014 04:53 PM, Chris Hegarty wrote: > On 10/11/14 12:56, David Holmes wrote: >> On 10/11/2014 9:52 PM, Chris Hegarty wrote: >>> I have only looked at the libraries changes, and I think they make sense >>> . As in, I can find no reason why the name cannot be changed to be a >>> String. >> >> Very quick response, but IIRC this has been examined in the past and >> there were reasons why it can't/shouldn't be done. Will try to dig out >> more details in the morning. > > If there was previous discussion on this, that revealed some substantial > issue, that would be great, but I can't recall, or find, it now. > > Hotspot express, and the desire for hotspot to run with different > library versions, would certainly cause complication, but I don't > believe that is an issue now. > > Just on that, the library changes are minimal, and if this were to > proceed then they can accompany the hotspot change, as they make their > way into jdk9/dev. > > Anyway, this should await your reply. Alan was having the same concern, there is an issue with JNI/JVMTI and other power users that might break when exposed to under-constructed Thread, e.g: https://bugs.openjdk.java.net/browse/JDK-6412693 This is why I ran jvmti and serviceability tests for this change, yielding no failures. This reinforces my belief this patch does not break the important invariant: if there is a problem with "Thread.name = name.toCharArray()" anywhere in Thread code, then "Thread.name = name" does neither regress it further nor fixes it. Then I speculated that having char[] name would help VM initialize the name if we wanted to switch to complete VM-side initialization of Thread, but it seems we can do String oop instantiation in the similar vein. Caching the name feels like a band-aid, that will probably complicate the Thread initialization on VM side even more. Let's wait and see if David can come up with some horror issue we are overlooking. :) Thanks, -Aleksey. From Alan.Bateman at oracle.com Mon Nov 10 14:30:33 2014 From: Alan.Bateman at oracle.com (Alan Bateman) Date: Mon, 10 Nov 2014 14:30:33 +0000 Subject: RFR (S) 8059677: Thread.getName() instantiates Strings In-Reply-To: <5460C354.5000605@oracle.com> References: <545FB64F.7090705@oracle.com> <5460A6E8.9050506@oracle.com> <5460B608.4050909@oracle.com> <5460C354.5000605@oracle.com> Message-ID: <5460CC09.7030204@oracle.com> On 10/11/2014 13:53, Chris Hegarty wrote: > > If there was previous discussion on this, that revealed some > substantial issue, that would be great, but I can't recall, or find, > it now. > > Hotspot express, and the desire for hotspot to run with different > library versions, would certainly cause complication, but I don't > believe that is an issue now. > > Just on that, the library changes are minimal, and if this were to > proceed then they can accompany the hotspot change, as they make their > way into jdk9/dev. > I remember the previous discussion on this and at the time it was just too troublesome to try to coordinate the change to hotspot + jdk. So a jdk-only change was pushed to address the last issue in this area, the issue of changing it from char[] to String was kicked down the road. -Alan From roger.riggs at oracle.com Mon Nov 10 14:31:01 2014 From: roger.riggs at oracle.com (roger riggs) Date: Mon, 10 Nov 2014 09:31:01 -0500 Subject: RFR (S) 8059677: Thread.getName() instantiates Strings In-Reply-To: <5460C6F3.1080201@oracle.com> References: <545FB64F.7090705@oracle.com> <5460A6E8.9050506@oracle.com> <5460C6F3.1080201@oracle.com> Message-ID: <5460CC25.1000609@oracle.com> Hi Aleksey, 1) The Thread class javadoc says: " Unless otherwise noted, passing a {@code null} argument to a constructor * or method in this class will cause a {@link NullPointerException} to be * thrown." So, NPE is already specified for setThreadName(null) or any other method. I'm not infavor of adding the Objects.requireNonNull, the NPE will be thrown soon enough and it is just noise in the source code in most cases that creates larger bytecodes and extra work for the compiler /interpreter. 2) About not storing the name as a String, I have some vague recollection of the issue being related to exposing an object settable by the application that can be used with synchronize and allows communication and sync issues between threads. Just because some test doesn't fail, doesn't mean there isn't a design/implementation constraint. Roger On 11/10/2014 9:08 AM, Aleksey Shipilev wrote: > Hi Chris, > > Thanks for taking a look! > > On 11/10/2014 02:52 PM, Chris Hegarty wrote: >> Trivially, after your changes will NPE be thrown if setName(null), as it >> is today ? > There is no way it could throw NPE now, therefore the behavior is > different. The spec says nothing about NPE though, but it feels wrong to > pass the null String to setNativeName. I should add > Objects.requireNonNull there. Will wait for more feedbacks, and update > the webrev. > > -Aleksey. > > From konstantin.shefov at oracle.com Mon Nov 10 14:45:36 2014 From: konstantin.shefov at oracle.com (Konstantin Shefov) Date: Mon, 10 Nov 2014 17:45:36 +0300 Subject: [9] Review request : JDK-8059070: [TESTBUG] java/lang/invoke/LFCaching/LFMultiThreadCachingTest.java failed - timeout In-Reply-To: <5460A29E.6070206@oracle.com> References: <543D1DEE.8030206@oracle.com> <543F8520.9090408@oracle.com> <8F9DD322-1793-4739-8155-61CE7D2C5AAB@oracle.com> <543F9800.7040406@oracle.com> <543FC71F.8020504@oracle.com> <5440E39C.8070001@oracle.com> <5448E5A2.9030804@oracle.com> <544E1B65.1090400@oracle.com> <5450F8E4.2060808@oracle.com> <545C9D70.6060308@oracle.com> <5460A29E.6070206@oracle.com> Message-ID: <5460CF90.1050303@oracle.com> Vladimir, thanks for reviewing I have updated the webrev: http://cr.openjdk.java.net/~kshefov/8059070/webrev.02 I have added DEFAULT_TEST_TIMEOUT constant to Utils class. -Konstantin On 10.11.2014 14:33, Vladimir Ivanov wrote: > Kontantin, sorry for the late response. > > In general, the fix looks good. > > I'd move timeout computation logic to Utils: > + private static final long TIMEOUT = > Utils.adjustTimeout(TimeUnit.SECONDS.toMillis(120)); > > Default value (120s) isn't specific to the tests, but an > implementation detail of jtreg. > > Best regards, > Vladimir Ivanov > > On 11/7/14, 2:22 PM, Konstantin Shefov wrote: >> Gently reminder >> >> 29.10.2014 17:25, Konstantin Shefov ?????: >>> Please, review a test bug fix. >>> http://cr.openjdk.java.net/~kshefov/8059070/webrev.01/ >>> >>> -Konstantin >>> >>> On 27.10.2014 13:16, Konstantin Shefov wrote: >>>> Kindly reminder >>>> >>>> On 23.10.2014 19:04, Paul Sandoz wrote: >>>>> On Oct 23, 2014, at 1:25 PM, Konstantin Shefov >>>>> wrote: >>>>>> Gently reminder >>>>>> >>>>>> On 17.10.2014 13:38, Konstantin Shefov wrote: >>>>>>> Hi, >>>>>>> >>>>>>> I have updated the webrev: >>>>>>> http://cr.openjdk.java.net/~kshefov/8059070/webrev.01/ >>>>>>> >>>>> +1 >>>>> >>>>> Sorry for the delay, >>>>> Paul. >>>> >>> >> From staffan.larsen at oracle.com Mon Nov 10 14:51:13 2014 From: staffan.larsen at oracle.com (Staffan Larsen) Date: Mon, 10 Nov 2014 15:51:13 +0100 Subject: RFR (S) 8059677: Thread.getName() instantiates Strings In-Reply-To: <5460C960.9080509@oracle.com> References: <545FB64F.7090705@oracle.com> <5460A6E8.9050506@oracle.com> <5460B608.4050909@oracle.com> <5460C354.5000605@oracle.com> <5460C960.9080509@oracle.com> Message-ID: I?m afraid this change requires changes in the Serviceability Agent as well. See OopUtilities.threadOopGetName() for example. /Staffan > On 10 nov 2014, at 15:19, Aleksey Shipilev wrote: > > Hi David, Chris, > > On 11/10/2014 04:53 PM, Chris Hegarty wrote: >> On 10/11/14 12:56, David Holmes wrote: >>> On 10/11/2014 9:52 PM, Chris Hegarty wrote: >>>> I have only looked at the libraries changes, and I think they make sense >>>> . As in, I can find no reason why the name cannot be changed to be a >>>> String. >>> >>> Very quick response, but IIRC this has been examined in the past and >>> there were reasons why it can't/shouldn't be done. Will try to dig out >>> more details in the morning. >> >> If there was previous discussion on this, that revealed some substantial >> issue, that would be great, but I can't recall, or find, it now. >> >> Hotspot express, and the desire for hotspot to run with different >> library versions, would certainly cause complication, but I don't >> believe that is an issue now. >> >> Just on that, the library changes are minimal, and if this were to >> proceed then they can accompany the hotspot change, as they make their >> way into jdk9/dev. >> >> Anyway, this should await your reply. > > Alan was having the same concern, there is an issue with JNI/JVMTI and > other power users that might break when exposed to under-constructed > Thread, e.g: > https://bugs.openjdk.java.net/browse/JDK-6412693 > > This is why I ran jvmti and serviceability tests for this change, > yielding no failures. This reinforces my belief this patch does not > break the important invariant: if there is a problem with "Thread.name = > name.toCharArray()" anywhere in Thread code, then "Thread.name = name" > does neither regress it further nor fixes it. > > Then I speculated that having char[] name would help VM initialize the > name if we wanted to switch to complete VM-side initialization of > Thread, but it seems we can do String oop instantiation in the similar vein. > > Caching the name feels like a band-aid, that will probably complicate > the Thread initialization on VM side even more. Let's wait and see if > David can come up with some horror issue we are overlooking. :) > > Thanks, > -Aleksey. > From eric.mccorkle at oracle.com Mon Nov 10 14:52:12 2014 From: eric.mccorkle at oracle.com (Eric McCorkle) Date: Mon, 10 Nov 2014 09:52:12 -0500 Subject: Review request for 8055063: Parameter#toString() fails w/ AIOOBE for ctr of inner class w/ generic type In-Reply-To: <20141110114544.GC22573@oracle.com> References: <5453B58A.1090507@oracle.com> <55A63A8D-1D0E-4D48-8CC4-D9326A57586E@oracle.com> <545A9C84.1060805@oracle.com> <20141110114544.GC22573@oracle.com> Message-ID: <5460D11C.90008@oracle.com> Ok, the test has been revised as you suggested. A new version is posted (webrev.03). On 11/10/14 06:45, Joel Borggr?n-Franck wrote: > Hi Eric, > > The changes to Executable.java and Parameter.java looks good. The test > could still need some cleaning up: > > @summary should be before "action tags", put it right after @bug > > I don't think you need a separate file for InnerClassToStringNoParameters.java , should be able to accomplish the same with jtreg tags, > > * @clean InnerClassToString > * @compile -parameters InnerClassToString.java > * @run main InnerClassToString > * @clean InnerClassToString > * @compile InnerClassToString.java > * @run main InnerClassToString > > I think this is cleaner, YMMV. > > The main method suffers from a lot of code duplication (and an absence > of line breaks), you can extract a test method since the test just > operates on two arrays of Type[] and call it with the parameter type > array and the answer array. > > cheers > /Joel > > On 2014-11-05, Eric McCorkle wrote: >> Thanks, Joel. I've applied your fixes and re-upped the webrev. >> >> On 11/05/14 10:10, Joel Borggr?n-Franck wrote: >>> Hi Eric, >>> >>> I think caching the real parameter type array on Executable is the wrong thing to do considering we can have a lot of instances of Executable but fairly few will be examined for parameters and the vast majority of those will never have this issue. >>> >>> Also, the tests are a bit minimal, I would expect you to check that the type of the parameter is correct. The tests should also have a @bug line. >>> >>> I think you are overly cautious here. I?m not sure we need to work around potential bugs in non-Java compilers. If language Y inserts synthetic/mandated parameters anywhere else than as a prefix they better generate a correct Signature (or non at all). This is a discussion we can defer however. >>> >>> If you flesh out the test and remove the caching I think this is a fine fix for both 8u and 9. >>> >>> cheers >>> /Joel >>> >>> On 31 okt 2014, at 17:15, Eric McCorkle wrote: >>> >>>> Hello, >>>> >>>> Please review this patch which fixes issues that arise with >>>> getGenericParameterTypes() and getAnnotatedParameterTypes() when there >>>> are generic signatures and synthetic parameters. >>>> >>>> Please note that a complete fix is not possible for all cases. See >>>> discussion on https://bugs.openjdk.java.net/browse/JDK-8062582 for details. >>>> >>>> This patch will cause Executable.getAnnotatedParameterTypes(), >>>> Parameter.getAnnotatedType(), and Parameter.getParameterizedType() to >>>> report the correct types in the following cases: >>>> >>>> * No generic signature is present. >>>> * Both a generic signature and method parameters information are present >>>> * A generic signature is present, but method parameters information is >>>> not present, but the number of parameters in the generic signature and >>>> the number of parameters in the method descriptor are the same. >>>> >>>> In the problematic case, where there is a generic signature, no method >>>> parameters information, and the generic signature does not match the >>>> method descriptor, these methods will return the correct /non/-generic >>>> type, as there is no general way of associating parameters in the >>>> generic signature with those in the method descriptor in this case. >>>> >>>> Please also note that there is currently a bug in javac which causes >>>> type annotations' parameter indexes to be wrong when synthetic >>>> parameters are generated: https://bugs.openjdk.java.net/browse/JDK-8029012. >>>> >>>> The bug report is here: >>>> https://bugs.openjdk.java.net/browse/JDK-8055063 >>>> >>>> The webrev is here: >>>> http://cr.openjdk.java.net/~emc/8055063/ >>> > >> begin:vcard >> fn:Eric McCorkle >> n:McCorkle;Eric >> org:Oracle Corporation;Java Platform Group >> adr:BUR02-2117;;35 Network Drive;Burlington;MA;01803;United States >> email;internet:eric.mccorkle at oracle.com >> title:Senior Member of Technical Staff >> tel;work:1-781-442-1028 >> tel;cell:404-644-6360 >> version:2.1 >> end:vcard >> > From aleksey.shipilev at oracle.com Mon Nov 10 14:54:16 2014 From: aleksey.shipilev at oracle.com (Aleksey Shipilev) Date: Mon, 10 Nov 2014 17:54:16 +0300 Subject: RFR (S) 8059677: Thread.getName() instantiates Strings In-Reply-To: <5460CC25.1000609@oracle.com> References: <545FB64F.7090705@oracle.com> <5460A6E8.9050506@oracle.com> <5460C6F3.1080201@oracle.com> <5460CC25.1000609@oracle.com> Message-ID: <5460D198.4030905@oracle.com> Hi Roger, On 11/10/2014 05:31 PM, roger riggs wrote: > 1) The Thread class javadoc says: > " Unless otherwise noted, passing a {@code null} argument to a constructor > * or method in this class will cause a {@link NullPointerException} to be > * thrown." > > So, NPE is already specified for setThreadName(null) or any other method. Ah, thanks! It is odd to see this specified in a blanked fashion in the class Javadoc, oh well. So we need to restore the NP check. > I'm not infavor of adding the Objects.requireNonNull, the NPE will > be thrown soon enough and it is just noise in the source code in most > cases that creates larger bytecodes and extra work for the compiler > /interpreter. Sorry, I have a hard time understanding what you are saying. How would you guarantee NPE (as per Javadoc contract above) in the new version of Thread.setName otherwise? > 2) About not storing the name as a String, I have some vague > recollection of the issue being related to exposing an object > settable by the application that can be used with synchronize and > allows communication and sync issues between threads. Again, I don't quite understand. Is it about storing the reference to String as the thread name, that can potentially be used for external synchronization? If so, I have a hard time devising a sane test case that might fail with this change. Internal code does not synchronize on Thread.name. Anyone synchronizing on Thread.getName() result has broken synchronization with current code. Anyone synchronizing on Thread.getName() result after this patch will have that (ahem) fixed, plus a performance problem. > Just because some test doesn't fail, doesn't mean there isn't a > design/implementation constraint. I should have said, "I *also* run the jvmti and serviceability tests" to confirm the change in innocuous. See the HS code change itself -- it does seem contained. Thanks, -Aleksey. From aleksey.shipilev at oracle.com Mon Nov 10 14:55:22 2014 From: aleksey.shipilev at oracle.com (Aleksey Shipilev) Date: Mon, 10 Nov 2014 17:55:22 +0300 Subject: RFR (S) 8059677: Thread.getName() instantiates Strings In-Reply-To: References: <545FB64F.7090705@oracle.com> <5460A6E8.9050506@oracle.com> <5460B608.4050909@oracle.com> <5460C354.5000605@oracle.com> <5460C960.9080509@oracle.com> Message-ID: <5460D1DA.4050907@oracle.com> Hi Staffan, Ow, it seems very like it. So, what testlist have I missed to catch this? -Aleksey. On 11/10/2014 05:51 PM, Staffan Larsen wrote: > I?m afraid this change requires changes in the Serviceability Agent as well. See OopUtilities.threadOopGetName() for example. > > /Staffan > >> On 10 nov 2014, at 15:19, Aleksey Shipilev wrote: >> >> Hi David, Chris, >> >> On 11/10/2014 04:53 PM, Chris Hegarty wrote: >>> On 10/11/14 12:56, David Holmes wrote: >>>> On 10/11/2014 9:52 PM, Chris Hegarty wrote: >>>>> I have only looked at the libraries changes, and I think they make sense >>>>> . As in, I can find no reason why the name cannot be changed to be a >>>>> String. >>>> >>>> Very quick response, but IIRC this has been examined in the past and >>>> there were reasons why it can't/shouldn't be done. Will try to dig out >>>> more details in the morning. >>> >>> If there was previous discussion on this, that revealed some substantial >>> issue, that would be great, but I can't recall, or find, it now. >>> >>> Hotspot express, and the desire for hotspot to run with different >>> library versions, would certainly cause complication, but I don't >>> believe that is an issue now. >>> >>> Just on that, the library changes are minimal, and if this were to >>> proceed then they can accompany the hotspot change, as they make their >>> way into jdk9/dev. >>> >>> Anyway, this should await your reply. >> >> Alan was having the same concern, there is an issue with JNI/JVMTI and >> other power users that might break when exposed to under-constructed >> Thread, e.g: >> https://bugs.openjdk.java.net/browse/JDK-6412693 >> >> This is why I ran jvmti and serviceability tests for this change, >> yielding no failures. This reinforces my belief this patch does not >> break the important invariant: if there is a problem with "Thread.name = >> name.toCharArray()" anywhere in Thread code, then "Thread.name = name" >> does neither regress it further nor fixes it. >> >> Then I speculated that having char[] name would help VM initialize the >> name if we wanted to switch to complete VM-side initialization of >> Thread, but it seems we can do String oop instantiation in the similar vein. >> >> Caching the name feels like a band-aid, that will probably complicate >> the Thread initialization on VM side even more. Let's wait and see if >> David can come up with some horror issue we are overlooking. :) >> >> Thanks, >> -Aleksey. >> > From joel.franck at oracle.com Mon Nov 10 15:04:45 2014 From: joel.franck at oracle.com (Joel =?utf-8?Q?Borggr=C3=A9n-Franck?=) Date: Mon, 10 Nov 2014 16:04:45 +0100 Subject: Review request for 8055063: Parameter#toString() fails w/ AIOOBE for ctr of inner class w/ generic type In-Reply-To: <5460D11C.90008@oracle.com> References: <5453B58A.1090507@oracle.com> <55A63A8D-1D0E-4D48-8CC4-D9326A57586E@oracle.com> <545A9C84.1060805@oracle.com> <20141110114544.GC22573@oracle.com> <5460D11C.90008@oracle.com> Message-ID: <20141110150445.GA3812@oracle.com> Approved thanks Eric, cheers /Joel On 2014-11-10, Eric McCorkle wrote: > Ok, the test has been revised as you suggested. > > A new version is posted (webrev.03). > > On 11/10/14 06:45, Joel Borggr?n-Franck wrote: > > Hi Eric, > > > > The changes to Executable.java and Parameter.java looks good. The test > > could still need some cleaning up: > > > > @summary should be before "action tags", put it right after @bug > > > > I don't think you need a separate file for InnerClassToStringNoParameters.java , should be able to accomplish the same with jtreg tags, > > > > * @clean InnerClassToString > > * @compile -parameters InnerClassToString.java > > * @run main InnerClassToString > > * @clean InnerClassToString > > * @compile InnerClassToString.java > > * @run main InnerClassToString > > > > I think this is cleaner, YMMV. > > > > The main method suffers from a lot of code duplication (and an absence > > of line breaks), you can extract a test method since the test just > > operates on two arrays of Type[] and call it with the parameter type > > array and the answer array. > > > > cheers > > /Joel > > > > On 2014-11-05, Eric McCorkle wrote: > >> Thanks, Joel. I've applied your fixes and re-upped the webrev. > >> > >> On 11/05/14 10:10, Joel Borggr?n-Franck wrote: > >>> Hi Eric, > >>> > >>> I think caching the real parameter type array on Executable is the wrong thing to do considering we can have a lot of instances of Executable but fairly few will be examined for parameters and the vast majority of those will never have this issue. > >>> > >>> Also, the tests are a bit minimal, I would expect you to check that the type of the parameter is correct. The tests should also have a @bug line. > >>> > >>> I think you are overly cautious here. I?m not sure we need to work around potential bugs in non-Java compilers. If language Y inserts synthetic/mandated parameters anywhere else than as a prefix they better generate a correct Signature (or non at all). This is a discussion we can defer however. > >>> > >>> If you flesh out the test and remove the caching I think this is a fine fix for both 8u and 9. > >>> > >>> cheers > >>> /Joel > >>> > >>> On 31 okt 2014, at 17:15, Eric McCorkle wrote: > >>> > >>>> Hello, > >>>> > >>>> Please review this patch which fixes issues that arise with > >>>> getGenericParameterTypes() and getAnnotatedParameterTypes() when there > >>>> are generic signatures and synthetic parameters. > >>>> > >>>> Please note that a complete fix is not possible for all cases. See > >>>> discussion on https://bugs.openjdk.java.net/browse/JDK-8062582 for details. > >>>> > >>>> This patch will cause Executable.getAnnotatedParameterTypes(), > >>>> Parameter.getAnnotatedType(), and Parameter.getParameterizedType() to > >>>> report the correct types in the following cases: > >>>> > >>>> * No generic signature is present. > >>>> * Both a generic signature and method parameters information are present > >>>> * A generic signature is present, but method parameters information is > >>>> not present, but the number of parameters in the generic signature and > >>>> the number of parameters in the method descriptor are the same. > >>>> > >>>> In the problematic case, where there is a generic signature, no method > >>>> parameters information, and the generic signature does not match the > >>>> method descriptor, these methods will return the correct /non/-generic > >>>> type, as there is no general way of associating parameters in the > >>>> generic signature with those in the method descriptor in this case. > >>>> > >>>> Please also note that there is currently a bug in javac which causes > >>>> type annotations' parameter indexes to be wrong when synthetic > >>>> parameters are generated: https://bugs.openjdk.java.net/browse/JDK-8029012. > >>>> > >>>> The bug report is here: > >>>> https://bugs.openjdk.java.net/browse/JDK-8055063 > >>>> > >>>> The webrev is here: > >>>> http://cr.openjdk.java.net/~emc/8055063/ > >>> > > > >> begin:vcard > >> fn:Eric McCorkle > >> n:McCorkle;Eric > >> org:Oracle Corporation;Java Platform Group > >> adr:BUR02-2117;;35 Network Drive;Burlington;MA;01803;United States > >> email;internet:eric.mccorkle at oracle.com > >> title:Senior Member of Technical Staff > >> tel;work:1-781-442-1028 > >> tel;cell:404-644-6360 > >> version:2.1 > >> end:vcard > >> > > > begin:vcard > fn:Eric McCorkle > n:McCorkle;Eric > org:Oracle Corporation;Java Platform Group > adr:BUR02-2117;;35 Network Drive;Burlington;MA;01803;United States > email;internet:eric.mccorkle at oracle.com > title:Senior Member of Technical Staff > tel;work:1-781-442-1028 > tel;cell:404-644-6360 > version:2.1 > end:vcard > From roger.riggs at oracle.com Mon Nov 10 15:13:32 2014 From: roger.riggs at oracle.com (roger riggs) Date: Mon, 10 Nov 2014 10:13:32 -0500 Subject: RFR (S) 8059677: Thread.getName() instantiates Strings In-Reply-To: <5460D198.4030905@oracle.com> References: <545FB64F.7090705@oracle.com> <5460A6E8.9050506@oracle.com> <5460C6F3.1080201@oracle.com> <5460CC25.1000609@oracle.com> <5460D198.4030905@oracle.com> Message-ID: <5460D61C.8090507@oracle.com> Hi, On 11/10/2014 9:54 AM, Aleksey Shipilev wrote: > Hi Roger, > > On 11/10/2014 05:31 PM, roger riggs wrote: >> 1) The Thread class javadoc says: >> " Unless otherwise noted, passing a {@code null} argument to a constructor >> * or method in this class will cause a {@link NullPointerException} to be >> * thrown." >> >> So, NPE is already specified for setThreadName(null) or any other method. > Ah, thanks! It is odd to see this specified in a blanked fashion in the > class Javadoc, oh well. So we need to restore the NP check. Class or package level specification of the default NPE behavior is now preferred to avoid the extra bulk and repetitiveness of extra @throws... > > >> I'm not infavor of adding the Objects.requireNonNull, the NPE will >> be thrown soon enough and it is just noise in the source code in most >> cases that creates larger bytecodes and extra work for the compiler >> /interpreter. > Sorry, I have a hard time understanding what you are saying. How would > you guarantee NPE (as per Javadoc contract above) in the new version of > Thread.setName otherwise? My mistake, an explicit check for null is needed in setName(). >> 2) About not storing the name as a String, I have some vague >> recollection of the issue being related to exposing an object >> settable by the application that can be used with synchronize and >> allows communication and sync issues between threads. > Again, I don't quite understand. Is it about storing the reference to > String as the thread name, that can potentially be used for external > synchronization? > > If so, I have a hard time devising a sane test case that might fail with > this change. Internal code does not synchronize on Thread.name. Anyone > synchronizing on Thread.getName() result has broken synchronization with > current code. Anyone synchronizing on Thread.getName() result after this > patch will have that (ahem) fixed, plus a performance problem. > > Potentially, any change we make to the implementation can result in a change in application behavior, even changing from returning a unique String to an immutable but constant String. For 9, this will have time to uncover obscure application dependencies. I'd be more cautious about backporting to 8. Roger From aleksey.shipilev at oracle.com Mon Nov 10 15:24:21 2014 From: aleksey.shipilev at oracle.com (Aleksey Shipilev) Date: Mon, 10 Nov 2014 18:24:21 +0300 Subject: RFR (S) 8059677: Thread.getName() instantiates Strings In-Reply-To: <5460D61C.8090507@oracle.com> References: <545FB64F.7090705@oracle.com> <5460A6E8.9050506@oracle.com> <5460C6F3.1080201@oracle.com> <5460CC25.1000609@oracle.com> <5460D198.4030905@oracle.com> <5460D61C.8090507@oracle.com> Message-ID: <5460D8A5.6090804@oracle.com> On 11/10/2014 06:13 PM, roger riggs wrote: > On 11/10/2014 9:54 AM, Aleksey Shipilev wrote: >> Again, I don't quite understand. Is it about storing the reference to >> String as the thread name, that can potentially be used for external >> synchronization? >> >> If so, I have a hard time devising a sane test case that might fail with >> this change. Internal code does not synchronize on Thread.name. Anyone >> synchronizing on Thread.getName() result has broken synchronization with >> current code. Anyone synchronizing on Thread.getName() result after this >> patch will have that (ahem) fixed, plus a performance problem. >> >> > Potentially, any change we make to the implementation can result in > a change in application behavior, even changing from returning a > unique String to an immutable but constant String. For 9, this will > have time to uncover obscure application dependencies. I'd be more > cautious about backporting to 8. Understood. I argue users (futilely) synchronizing on Thread.getName() are very rare. -Aleksey. From peter.levart at gmail.com Mon Nov 10 16:13:08 2014 From: peter.levart at gmail.com (Peter Levart) Date: Mon, 10 Nov 2014 17:13:08 +0100 Subject: RFR: 8062771: Core reflection should use final fields whenever possible In-Reply-To: References: <843282E1-DC2B-48B0-B0E4-A60B5078D75A@oracle.com> <45A50B54-43A3-4134-9EB5-D1AEDA649587@oracle.com> Message-ID: <5460E414.7060004@gmail.com> On 11/07/2014 11:48 PM, Martin Buchholz wrote: > Hi Joel, > > Thanks for volunteering. I foisted all I have in > > https://bugs.openjdk.java.net/browse/JDK-8064391 > > I volunteer to be your reviewer for the backports. Hi Martin, Sorry I haven't checked this earlier, but there are still some data races left-behind: http://cr.openjdk.java.net/~plevart/jdk9-dev/GenericsReflectionRaces/webrev.01/ Mainly the fact that lazy initialization uses arrays which are published unsafely. Would this need a separate issue. For example: "Core reflection should use volatile fields whenever necessary" ? Regards, Peter > > On Fri, Nov 7, 2014 at 1:36 PM, Joel Borggr?n-Franck > wrote: >> Hi Martin, >> >> Thanks for the clarification. >> >> On 6 nov 2014, at 20:51, Martin Buchholz wrote: >> >>> Hi Joel, >>> >>> On Thu, Nov 6, 2014 at 2:48 AM, Joel Borggr?n-Franck >>> wrote: >>>> Hi, >>>> >>>> I?m having a hard time following this thread, which webrev has been updated, for which release, fixing which issue? >>> I am submitting the changeset for JDK-8062771, adding the finals and >>> the (failed reproduction) test. >>> Hopefully there will be followon changes to add even more thread safety. >>> >> Great. >> >> Are we sure the accidental fix of making ClassRepository volatile is good on all platforms? I can?t establish the happens before relations that would prove the fix but then again I?m not an expert on concurrency. >> >>>> Martin, as far as I can see you are the only one of us who has replied to this thread who is a jdk7u committer (or reviewer). >>> I am entirely in favor of having more reviewers. >>> >>> We are not planning to push these into jdk7u or jdk8u ourselves, but >>> we are willing to help anyone who wants to take on that task. Joel, >>> are you volunteering? >>> We believe all versions of stock openjdk still have (rarely seen) >>> thread safety issues in core reflection. We have applied changes >>> locally at Google to fix those. >>> >> I?ll take care of 8u, and I can hunt down someone to fix this in 7u if we can find a suitable reviewer (sigh). >> >> cheers >> /Joel >> >> From staffan.larsen at oracle.com Mon Nov 10 16:39:27 2014 From: staffan.larsen at oracle.com (Staffan Larsen) Date: Mon, 10 Nov 2014 17:39:27 +0100 Subject: RFR (S) 8059677: Thread.getName() instantiates Strings In-Reply-To: <5460D1DA.4050907@oracle.com> References: <545FB64F.7090705@oracle.com> <5460A6E8.9050506@oracle.com> <5460B608.4050909@oracle.com> <5460C354.5000605@oracle.com> <5460C960.9080509@oracle.com> <5460D1DA.4050907@oracle.com> Message-ID: > On 10 nov 2014, at 15:55, Aleksey Shipilev wrote: > > Hi Staffan, > > Ow, it seems very like it. > So, what testlist have I missed to catch this? Probably vm.tmtools.testlist and/or nsk.sajdi.testlist. Just a warning that these tests are far from stable. Sorry about that. /Staffan > > -Aleksey. > > On 11/10/2014 05:51 PM, Staffan Larsen wrote: >> I?m afraid this change requires changes in the Serviceability Agent as well. See OopUtilities.threadOopGetName() for example. >> >> /Staffan >> >>> On 10 nov 2014, at 15:19, Aleksey Shipilev wrote: >>> >>> Hi David, Chris, >>> >>> On 11/10/2014 04:53 PM, Chris Hegarty wrote: >>>> On 10/11/14 12:56, David Holmes wrote: >>>>> On 10/11/2014 9:52 PM, Chris Hegarty wrote: >>>>>> I have only looked at the libraries changes, and I think they make sense >>>>>> . As in, I can find no reason why the name cannot be changed to be a >>>>>> String. >>>>> >>>>> Very quick response, but IIRC this has been examined in the past and >>>>> there were reasons why it can't/shouldn't be done. Will try to dig out >>>>> more details in the morning. >>>> >>>> If there was previous discussion on this, that revealed some substantial >>>> issue, that would be great, but I can't recall, or find, it now. >>>> >>>> Hotspot express, and the desire for hotspot to run with different >>>> library versions, would certainly cause complication, but I don't >>>> believe that is an issue now. >>>> >>>> Just on that, the library changes are minimal, and if this were to >>>> proceed then they can accompany the hotspot change, as they make their >>>> way into jdk9/dev. >>>> >>>> Anyway, this should await your reply. >>> >>> Alan was having the same concern, there is an issue with JNI/JVMTI and >>> other power users that might break when exposed to under-constructed >>> Thread, e.g: >>> https://bugs.openjdk.java.net/browse/JDK-6412693 >>> >>> This is why I ran jvmti and serviceability tests for this change, >>> yielding no failures. This reinforces my belief this patch does not >>> break the important invariant: if there is a problem with "Thread.name = >>> name.toCharArray()" anywhere in Thread code, then "Thread.name = name" >>> does neither regress it further nor fixes it. >>> >>> Then I speculated that having char[] name would help VM initialize the >>> name if we wanted to switch to complete VM-side initialization of >>> Thread, but it seems we can do String oop instantiation in the similar vein. >>> >>> Caching the name feels like a band-aid, that will probably complicate >>> the Thread initialization on VM side even more. Let's wait and see if >>> David can come up with some horror issue we are overlooking. :) >>> >>> Thanks, >>> -Aleksey. >>> >> > > From mark.sheppard at oracle.com Mon Nov 10 17:37:36 2014 From: mark.sheppard at oracle.com (Mark Sheppard) Date: Mon, 10 Nov 2014 17:37:36 +0000 Subject: RFR: JDK-8037909 - JNI warnings in jdk/src/windows/native/java/nio/MappedByteBuffer.c Message-ID: <5460F7E0.6050403@oracle.com> Hi please oblige and review the follow change diff -r c0d1026bff6f src/java.base/windows/native/libnio/MappedByteBuffer.c --- a/src/java.base/windows/native/libnio/MappedByteBuffer.c Tue Nov 04 15:10:38 2014 +0000 +++ b/src/java.base/windows/native/libnio/MappedByteBuffer.c Mon Nov 10 17:32:51 2014 +0000 @@ -86,6 +86,7 @@ if (clazz == NULL) return; // exception thrown handle_fdID = (*env)->GetFieldID(env, clazz, "handle", "J"); + CHECK_NULL(handle_fdID); } h = jlong_to_ptr((*env)->GetLongField(env, fdo, handle_fdID)); result = FlushFileBuffers(h); to address the parfait issue https://bugs.openjdk.java.net/browse/JDK-8037909 regards Mark From wuwen.55 at gmail.com Fri Nov 7 08:22:44 2014 From: wuwen.55 at gmail.com (wuwen.55) Date: Fri, 7 Nov 2014 16:22:44 +0800 Subject: Java CountDownLatch.await(timeout) is influenced by changes to System time on Linux Message-ID: call .await(10, TimeUnit.SECONDS) and change the time one hours back. i.e. start at 13:00 call .await and at 13:05 change the time to 12:05 you will notice that the wait does not return. when change the time to the current time . change the time to 13:05. the wait still does not return. test code: CountDownLatch c = new CountDownLatch(1); ... c.await(10, TimeUnit.SECONDS); ... ---------------------------------- AbstractQueuedSynchronizer.doAcquireNanos long lastTime = System.nanoTime(); final Node node = addWaiter(Node.EXCLUSIVE); boolean failed = true; try { for (;;) { final Node p = node.predecessor(); if (p == head && tryAcquire(arg)) { setHead(node); p.next = null; // help GC failed = false; return true; } if (nanosTimeout <= 0) return false; if (shouldParkAfterFailedAcquire(p, node) && nanosTimeout > spinForTimeoutThreshold) LockSupport.parkNanos(this, nanosTimeout); long now = System.nanoTime(); nanosTimeout -= now - lastTime; lastTime = now; if (Thread.interrupted()) throw new InterruptedException(); } } finally { if (failed) cancelAcquire(node); } when change the time one hours back now - lastTime is negative number. ????????? From Alan.Bateman at oracle.com Mon Nov 10 17:59:32 2014 From: Alan.Bateman at oracle.com (Alan Bateman) Date: Mon, 10 Nov 2014 17:59:32 +0000 Subject: RFR: JDK-8037909 - JNI warnings in jdk/src/windows/native/java/nio/MappedByteBuffer.c In-Reply-To: <5460F7E0.6050403@oracle.com> References: <5460F7E0.6050403@oracle.com> Message-ID: <5460FD04.9090706@oracle.com> On 10/11/2014 17:37, Mark Sheppard wrote: > Hi > please oblige and review the follow change > > diff -r c0d1026bff6f > src/java.base/windows/native/libnio/MappedByteBuffer.c > --- a/src/java.base/windows/native/libnio/MappedByteBuffer.c Tue Nov > 04 15:10:38 2014 +0000 > +++ b/src/java.base/windows/native/libnio/MappedByteBuffer.c Mon Nov > 10 17:32:51 2014 +0000 > @@ -86,6 +86,7 @@ > if (clazz == NULL) > return; // exception thrown > handle_fdID = (*env)->GetFieldID(env, clazz, "handle", "J"); > + CHECK_NULL(handle_fdID); > } > h = jlong_to_ptr((*env)->GetLongField(env, fdo, handle_fdID)); > result = FlushFileBuffers(h); > This looks okay to me. For consistency then the FindClass just before it could use CHECK_NULL too. -Alan From aleksey.shipilev at oracle.com Mon Nov 10 18:09:05 2014 From: aleksey.shipilev at oracle.com (Aleksey Shipilev) Date: Mon, 10 Nov 2014 21:09:05 +0300 Subject: RFR (S) 8059677: Thread.getName() instantiates Strings In-Reply-To: References: <545FB64F.7090705@oracle.com> <5460A6E8.9050506@oracle.com> <5460B608.4050909@oracle.com> <5460C354.5000605@oracle.com> <5460C960.9080509@oracle.com> <5460D1DA.4050907@oracle.com> Message-ID: <5460FF41.90208@oracle.com> On 10.11.2014 19:39, Staffan Larsen wrote: >> On 10 nov 2014, at 15:55, Aleksey Shipilev wrote: >> Ow, it seems very like it. >> So, what testlist have I missed to catch this? > > Probably vm.tmtools.testlist and/or nsk.sajdi.testlist. Just a warning that these tests are far from stable. Sorry about that. Alas, both these testlists pass with current change without a hitch. That probably tells something about the test coverage. Any other ideas how to test for it? Maybe some manual way? Anyhow, there is a synonymous block in ThreadGroup handling, I can copy the relevant bits from there. Updated webrev follows soon. Still need to test if that change is safe. -Aleksey. From mark.sheppard at oracle.com Mon Nov 10 18:12:01 2014 From: mark.sheppard at oracle.com (Mark Sheppard) Date: Mon, 10 Nov 2014 18:12:01 +0000 Subject: RFR: JDK-8037909 - JNI warnings in jdk/src/windows/native/java/nio/MappedByteBuffer.c In-Reply-To: <5460FD04.9090706@oracle.com> References: <5460F7E0.6050403@oracle.com> <5460FD04.9090706@oracle.com> Message-ID: <5460FFF1.7090806@oracle.com> thanks Alan ... grand, I'll replace the "if (clazz == null) return; " with a CHECK_NULL(clazz); regards Mark On 10/11/2014 17:59, Alan Bateman wrote: > On 10/11/2014 17:37, Mark Sheppard wrote: >> Hi >> please oblige and review the follow change >> >> diff -r c0d1026bff6f >> src/java.base/windows/native/libnio/MappedByteBuffer.c >> --- a/src/java.base/windows/native/libnio/MappedByteBuffer.c Tue Nov >> 04 15:10:38 2014 +0000 >> +++ b/src/java.base/windows/native/libnio/MappedByteBuffer.c Mon Nov >> 10 17:32:51 2014 +0000 >> @@ -86,6 +86,7 @@ >> if (clazz == NULL) >> return; // exception thrown >> handle_fdID = (*env)->GetFieldID(env, clazz, "handle", >> "J"); >> + CHECK_NULL(handle_fdID); >> } >> h = jlong_to_ptr((*env)->GetLongField(env, fdo, handle_fdID)); >> result = FlushFileBuffers(h); >> > This looks okay to me. For consistency then the FindClass just before > it could use CHECK_NULL too. > > -Alan From mark.sheppard at oracle.com Mon Nov 10 19:09:54 2014 From: mark.sheppard at oracle.com (Mark Sheppard) Date: Mon, 10 Nov 2014 19:09:54 +0000 Subject: RFR: JDK-8051990 - Uninitialised memory in jdk/src/share/native/java/lang/fdlibm/src/k_standard.c Message-ID: <54610D82.7040107@oracle.com> Hi please oblige and review the follow minor change diff -r c0d1026bff6f src/java.base/share/native/libfdlibm/k_standard.c --- a/src/java.base/share/native/libfdlibm/k_standard.c Tue Nov 04 15:10:38 2014 +0000 +++ b/src/java.base/share/native/libfdlibm/k_standard.c Mon Nov 10 19:02:57 2014 +0000 @@ -1,6 +1,6 @@ /* - * Copyright (c) 1998, 2001, Oracle and/or its affiliates. All rights reserved. + * Copyright (c) 1998, 2014, Oracle and/or its affiliates. All rights reserved. * DO NOT ALTER OR REMOVE COPYRIGHT NOTICES OR THIS FILE HEADER. * * This code is free software; you can redistribute it and/or modify it @@ -106,6 +106,7 @@ #endif exc.arg1 = x; exc.arg2 = y; + exc.retval = zero; switch(type) { case 1: /* acos(|x|>1) */ which addresses the following parfait issue https://bugs.openjdk.java.net/browse/JDK-8051990 regards Mark From chris.hegarty at oracle.com Mon Nov 10 19:10:43 2014 From: chris.hegarty at oracle.com (Chris Hegarty) Date: Mon, 10 Nov 2014 19:10:43 +0000 Subject: RFR: JDK-8037909 - JNI warnings in jdk/src/windows/native/java/nio/MappedByteBuffer.c In-Reply-To: <5460F7E0.6050403@oracle.com> References: <5460F7E0.6050403@oracle.com> Message-ID: <86D8231E-1615-4DA5-B583-5D66CF6B3264@oracle.com> On 10 Nov 2014, at 17:37, Mark Sheppard wrote: > Hi > please oblige and review the follow change Looks find to me Mark. -Chris. > diff -r c0d1026bff6f src/java.base/windows/native/libnio/MappedByteBuffer.c > --- a/src/java.base/windows/native/libnio/MappedByteBuffer.c Tue Nov 04 15:10:38 2014 +0000 > +++ b/src/java.base/windows/native/libnio/MappedByteBuffer.c Mon Nov 10 17:32:51 2014 +0000 > @@ -86,6 +86,7 @@ > if (clazz == NULL) > return; // exception thrown > handle_fdID = (*env)->GetFieldID(env, clazz, "handle", "J"); > + CHECK_NULL(handle_fdID); > } > h = jlong_to_ptr((*env)->GetLongField(env, fdo, handle_fdID)); > result = FlushFileBuffers(h); > > to address the parfait issue > > https://bugs.openjdk.java.net/browse/JDK-8037909 > > regards > Mark From Alan.Bateman at oracle.com Mon Nov 10 19:39:29 2014 From: Alan.Bateman at oracle.com (Alan Bateman) Date: Mon, 10 Nov 2014 19:39:29 +0000 Subject: RFR: JDK-8051990 - Uninitialised memory in jdk/src/share/native/java/lang/fdlibm/src/k_standard.c In-Reply-To: <54610D82.7040107@oracle.com> References: <54610D82.7040107@oracle.com> Message-ID: <54611471.4010202@oracle.com> On 10/11/2014 19:09, Mark Sheppard wrote: > Hi > > please oblige and review the follow minor change > > diff -r c0d1026bff6f src/java.base/share/native/libfdlibm/k_standard.c > --- a/src/java.base/share/native/libfdlibm/k_standard.c Tue Nov 04 > 15:10:38 2014 +0000 > +++ b/src/java.base/share/native/libfdlibm/k_standard.c Mon Nov 10 > 19:02:57 2014 +0000 > @@ -1,6 +1,6 @@ > > /* > - * Copyright (c) 1998, 2001, Oracle and/or its affiliates. All rights > reserved. > + * Copyright (c) 1998, 2014, Oracle and/or its affiliates. All rights > reserved. > * DO NOT ALTER OR REMOVE COPYRIGHT NOTICES OR THIS FILE HEADER. > * > * This code is free software; you can redistribute it and/or modify it > @@ -106,6 +106,7 @@ > #endif > exc.arg1 = x; > exc.arg2 = y; > + exc.retval = zero; > switch(type) { > case 1: > /* acos(|x|>1) */ I wonder if we should be feeding changes to fdlibm upstream, Joe Darcy might know. So is the real issue here that the switch doesn't have a default statement to catch a possible mis-use of this function? I just wonder if that might be better than setting retval twice (although it might get optimized away sometimes). -Alan From ecki at zusammenkunft.net Mon Nov 10 20:59:47 2014 From: ecki at zusammenkunft.net (Bernd Eckenfels) Date: Mon, 10 Nov 2014 21:59:47 +0100 Subject: AW: Java CountDownLatch.await(timeout) is influenced by changes to Systemtime on Linux In-Reply-To: References: Message-ID: <54612742.b05f8c0a.b868.fffff277@mx.google.com> Hm, how did you change the time? (normal date command?) System.nanoTime is supposed to be monotonic. I think on linux it even uses a timer for exactly that. i think there was a open bug around park(), but the nanoTime should be find (and negative difference are sonewhat normal with it anyway) gruss bernd -- http://bernd.eckenfels.net ----- Urspr?ngliche Nachricht ----- Von: "wuwen.55" Gesendet: ?10.?11.?2014 19:54 An: "core-libs-dev at openjdk.java.net" Betreff: Java CountDownLatch.await(timeout) is influenced by changes to Systemtime on Linux call .await(10, TimeUnit.SECONDS) and change the time one hours back. i.e. start at 13:00 call .await and at 13:05 change the time to 12:05 you will notice that the wait does not return. when change the time to the current time . change the time to 13:05. the wait still does not return. test code: CountDownLatch c = new CountDownLatch(1); ... c.await(10, TimeUnit.SECONDS); ... ---------------------------------- AbstractQueuedSynchronizer.doAcquireNanos long lastTime = System.nanoTime(); final Node node = addWaiter(Node.EXCLUSIVE); boolean failed = true; try { for (;;) { final Node p = node.predecessor(); if (p == head && tryAcquire(arg)) { setHead(node); p.next = null; // help GC failed = false; return true; } if (nanosTimeout <= 0) return false; if (shouldParkAfterFailedAcquire(p, node) && nanosTimeout > spinForTimeoutThreshold) LockSupport.parkNanos(this, nanosTimeout); long now = System.nanoTime(); nanosTimeout -= now - lastTime; lastTime = now; if (Thread.interrupted()) throw new InterruptedException(); } } finally { if (failed) cancelAcquire(node); } when change the time one hours back now - lastTime is negative number. ????????? From david.buck at oracle.com Mon Nov 10 23:05:30 2014 From: david.buck at oracle.com (david buck) Date: Tue, 11 Nov 2014 08:05:30 +0900 Subject: RFR: 8064516: BCEL still corrupts generic methods if bytecode offsets are modified Message-ID: <546144BA.9060305@oracle.com> Hi! May I please have a review of this fix: BUGURL: https://bugs.openjdk.java.net/browse/JDK-8064516 WEBREV: http://cr.openjdk.java.net/~dbuck/8064516/webrev00/ Cheers, -Buck From huizhe.wang at oracle.com Mon Nov 10 23:56:15 2014 From: huizhe.wang at oracle.com (huizhe wang) Date: Mon, 10 Nov 2014 15:56:15 -0800 Subject: RFR: 8064516: BCEL still corrupts generic methods if bytecode offsets are modified In-Reply-To: <546144BA.9060305@oracle.com> References: <546144BA.9060305@oracle.com> Message-ID: <5461509F.6050005@oracle.com> Hi David, Looks good to me. Thanks, Joe On 11/10/2014 3:05 PM, david buck wrote: > Hi! > > May I please have a review of this fix: > > BUGURL: https://bugs.openjdk.java.net/browse/JDK-8064516 > WEBREV: http://cr.openjdk.java.net/~dbuck/8064516/webrev00/ > > Cheers, > -Buck From david.holmes at oracle.com Tue Nov 11 02:17:25 2014 From: david.holmes at oracle.com (David Holmes) Date: Tue, 11 Nov 2014 12:17:25 +1000 Subject: AW: Java CountDownLatch.await(timeout) is influenced by changes to Systemtime on Linux In-Reply-To: <54612742.b05f8c0a.b868.fffff277@mx.google.com> References: <54612742.b05f8c0a.b868.fffff277@mx.google.com> Message-ID: <546171B5.3050105@oracle.com> This is a known issue: https://bugs.openjdk.java.net/browse/JDK-6900441 fixed in 7u60, 8 and 9. What version were you running? David On 11/11/2014 6:59 AM, Bernd Eckenfels wrote: > Hm, how did you change the time? (normal date command?) System.nanoTime is supposed to be monotonic. I think on linux it even uses a timer for exactly that. > > i think there was a open bug around park(), but the nanoTime should be find (and negative difference are sonewhat normal with it anyway) > gruss > bernd > From david.holmes at oracle.com Tue Nov 11 02:25:44 2014 From: david.holmes at oracle.com (David Holmes) Date: Tue, 11 Nov 2014 12:25:44 +1000 Subject: Java CountDownLatch.await(timeout) is influenced by changes to System time on Linux In-Reply-To: References: Message-ID: <546173A8.9070002@oracle.com> Repeating my response due to the subject mangling in the other "thread": This is a known issue: https://bugs.openjdk.java.net/browse/JDK-6900441 fixed in 7u60, 8 and 9. What version were you running? David On 7/11/2014 6:22 PM, wuwen.55 wrote: > call .await(10, TimeUnit.SECONDS) and change the time one hours back. > > i.e. start at 13:00 call .await and at 13:05 change the time to 12:05 > > you will notice that the wait does not return. > > when change the time to the current time . change the time to 13:05. > > the wait still does not return. > > test code: > CountDownLatch c = new CountDownLatch(1); > ... > c.await(10, TimeUnit.SECONDS); > ... > ---------------------------------- > > AbstractQueuedSynchronizer.doAcquireNanos > > > long lastTime = System.nanoTime(); > final Node node = addWaiter(Node.EXCLUSIVE); > boolean failed = true; > try { > for (;;) { > final Node p = node.predecessor(); > if (p == head && tryAcquire(arg)) { > setHead(node); > p.next = null; // help GC > failed = false; > return true; > } > if (nanosTimeout <= 0) > return false; > if (shouldParkAfterFailedAcquire(p, node) && > nanosTimeout > spinForTimeoutThreshold) > LockSupport.parkNanos(this, nanosTimeout); > long now = System.nanoTime(); > nanosTimeout -= now - lastTime; > lastTime = now; > if (Thread.interrupted()) > throw new InterruptedException(); > } > } finally { > if (failed) > cancelAcquire(node); > } > > when change the time one hours back > > now - lastTime is negative number. > > ????????? > > From david.holmes at oracle.com Tue Nov 11 07:06:59 2014 From: david.holmes at oracle.com (David Holmes) Date: Tue, 11 Nov 2014 17:06:59 +1000 Subject: RFR (S) 8059677: Thread.getName() instantiates Strings In-Reply-To: <5460C960.9080509@oracle.com> References: <545FB64F.7090705@oracle.com> <5460A6E8.9050506@oracle.com> <5460B608.4050909@oracle.com> <5460C354.5000605@oracle.com> <5460C960.9080509@oracle.com> Message-ID: <5461B593.1000104@oracle.com> Hi Aleksey, On 11/11/2014 12:19 AM, Aleksey Shipilev wrote: > Hi David, Chris, > > On 11/10/2014 04:53 PM, Chris Hegarty wrote: >> On 10/11/14 12:56, David Holmes wrote: >>> On 10/11/2014 9:52 PM, Chris Hegarty wrote: >>>> I have only looked at the libraries changes, and I think they make sense >>>> . As in, I can find no reason why the name cannot be changed to be a >>>> String. >>> >>> Very quick response, but IIRC this has been examined in the past and >>> there were reasons why it can't/shouldn't be done. Will try to dig out >>> more details in the morning. >> >> If there was previous discussion on this, that revealed some substantial >> issue, that would be great, but I can't recall, or find, it now. >> >> Hotspot express, and the desire for hotspot to run with different >> library versions, would certainly cause complication, but I don't >> believe that is an issue now. >> >> Just on that, the library changes are minimal, and if this were to >> proceed then they can accompany the hotspot change, as they make their >> way into jdk9/dev. >> >> Anyway, this should await your reply. > > Alan was having the same concern, there is an issue with JNI/JVMTI and > other power users that might break when exposed to under-constructed > Thread, e.g: > https://bugs.openjdk.java.net/browse/JDK-6412693 > > This is why I ran jvmti and serviceability tests for this change, > yielding no failures. This reinforces my belief this patch does not > break the important invariant: if there is a problem with "Thread.name = > name.toCharArray()" anywhere in Thread code, then "Thread.name = name" > does neither regress it further nor fixes it. True. > Then I speculated that having char[] name would help VM initialize the > name if we wanted to switch to complete VM-side initialization of > Thread, but it seems we can do String oop instantiation in the similar vein. I think it really just came down to accessing the Thread name from things like JVMDI/PI (now JVM TI) - easier for C code to access a raw char[]. Maybe once upon a time (in a land not so far away) we even passed char[] to the Thread constructor? :) But having re-discovered past discussions etc there's really nothing to stop this from being a String (slight memory use increase per Thread object). > Caching the name feels like a band-aid, that will probably complicate > the Thread initialization on VM side even more. Let's wait and see if > David can come up with some horror issue we are overlooking. :) I don't see how a Java side cache affects anything on the VM initialization side - and as Strings can be published unsafely we don't even need sync/volatile to do so :) That aside I think it is as Alan commented - a number of small things (some logistical I think) that made this change not worth the effort. Maybe now it is worth the effort if getName is a bottleneck (but again caching is the common fix for that kind of problem :)). I was concerned about executing even more Java code at thread attach time, but we already create a String to pass to the Thread constructor, so no change there. So looking at your proposal ... some minor comments ... JDK change is okay - but "name" doesn't need to be volatile when it is a String reference. Hotspot side: src/share/vm/classfile/javaClasses.hpp This added assert seems overly cautious: 134 oop value = java_string->obj_field(value_offset); 135 assert((value->is_typeArray() && TypeArrayKlass::cast(value->klass())->element_type() == T_CHAR), "expect char[]"); you are basically checking that String.value is defined as a char[]. If warranted, this is a check needed once in the lifetime of a VM not every time this method is called. (Yes I see we had something similarly odd in java_lang_thread::name. :( ) --- src/share/vm/classfile/javaClasses.cpp ! oop java_lang_Thread::name(oop java_thread) { oop name = java_thread->obj_field(_name_offset); ! assert(name != NULL, "thread name is NULL"); I'm not confident this can never be called before the name has been set. The original assertion allowed for NULL as does the JVM TI code. --- src/share/vm/prims/jvmtiTrace.cpp Copyright year needs updating. :) --- Aside: I wonder if we've inadvertently fixed 6771287 now. :) That was a fun one to debug. Thanks, David ----- > Thanks, > -Aleksey. > From staffan.larsen at oracle.com Tue Nov 11 08:03:18 2014 From: staffan.larsen at oracle.com (Staffan Larsen) Date: Tue, 11 Nov 2014 09:03:18 +0100 Subject: RFR (S) 8059677: Thread.getName() instantiates Strings In-Reply-To: <5460FF41.90208@oracle.com> References: <545FB64F.7090705@oracle.com> <5460A6E8.9050506@oracle.com> <5460B608.4050909@oracle.com> <5460C354.5000605@oracle.com> <5460C960.9080509@oracle.com> <5460D1DA.4050907@oracle.com> <5460FF41.90208@oracle.com> Message-ID: <0A9ACBF1-F16F-444A-9CB4-5338C18F68E4@oracle.com> I was able to provoke the failure with a ?jstack -F?. I think this patch solves the problem: http://cr.openjdk.java.net/~sla/8059677-thread.name.sa.patch . Feel free to not include the changes in StackTrace.java if you don?t want to complicate your review. Thanks, /Staffan > On 10 nov 2014, at 19:09, Aleksey Shipilev wrote: > > On 10.11.2014 19:39, Staffan Larsen wrote: >>> On 10 nov 2014, at 15:55, Aleksey Shipilev wrote: >>> Ow, it seems very like it. >>> So, what testlist have I missed to catch this? >> >> Probably vm.tmtools.testlist and/or nsk.sajdi.testlist. Just a warning that these tests are far from stable. Sorry about that. > > Alas, both these testlists pass with current change without a hitch. > That probably tells something about the test coverage. Any other ideas > how to test for it? Maybe some manual way? > > Anyhow, there is a synonymous block in ThreadGroup handling, I can copy > the relevant bits from there. Updated webrev follows soon. Still need to > test if that change is safe. > > -Aleksey. > From aleksey.shipilev at oracle.com Tue Nov 11 09:26:01 2014 From: aleksey.shipilev at oracle.com (Aleksey Shipilev) Date: Tue, 11 Nov 2014 12:26:01 +0300 Subject: RFR (S) 8059677: Thread.getName() instantiates Strings In-Reply-To: <0A9ACBF1-F16F-444A-9CB4-5338C18F68E4@oracle.com> References: <545FB64F.7090705@oracle.com> <5460A6E8.9050506@oracle.com> <5460B608.4050909@oracle.com> <5460C354.5000605@oracle.com> <5460C960.9080509@oracle.com> <5460D1DA.4050907@oracle.com> <5460FF41.90208@oracle.com> <0A9ACBF1-F16F-444A-9CB4-5338C18F68E4@oracle.com> Message-ID: <5461D629.3010001@oracle.com> Thanks Staffan, your change is exactly what I (blindly) did in my updated webrev. I will get David's comments in, respin some tests and publish the update. -Aleksey. On 11.11.2014 11:03, Staffan Larsen wrote: > I was able to provoke the failure with a ?jstack -F?. I think this patch > solves the > problem: http://cr.openjdk.java.net/~sla/8059677-thread.name.sa.patch > . Feel > free to not include the changes in StackTrace.java if you don?t want to > complicate your review. > > Thanks, > /Staffan > > >> On 10 nov 2014, at 19:09, Aleksey Shipilev >> > wrote: >> >> On 10.11.2014 19:39, Staffan Larsen wrote: >>>> On 10 nov 2014, at 15:55, Aleksey Shipilev >>>> > >>>> wrote: >>>> Ow, it seems very like it. >>>> So, what testlist have I missed to catch this? >>> >>> Probably vm.tmtools.testlist and/or nsk.sajdi.testlist. Just a >>> warning that these tests are far from stable. Sorry about that. >> >> Alas, both these testlists pass with current change without a hitch. >> That probably tells something about the test coverage. Any other ideas >> how to test for it? Maybe some manual way? >> >> Anyhow, there is a synonymous block in ThreadGroup handling, I can copy >> the relevant bits from there. Updated webrev follows soon. Still need to >> test if that change is safe. >> >> -Aleksey. >> > From aleksey.shipilev at oracle.com Tue Nov 11 09:38:46 2014 From: aleksey.shipilev at oracle.com (Aleksey Shipilev) Date: Tue, 11 Nov 2014 12:38:46 +0300 Subject: RFR (S) 8059677: Thread.getName() instantiates Strings In-Reply-To: <5461B593.1000104@oracle.com> References: <545FB64F.7090705@oracle.com> <5460A6E8.9050506@oracle.com> <5460B608.4050909@oracle.com> <5460C354.5000605@oracle.com> <5460C960.9080509@oracle.com> <5461B593.1000104@oracle.com> Message-ID: <5461D926.6010008@oracle.com> Hi David, Updated webrevs will follow after I respin the tests. Meanwhile, some comments below: On 11.11.2014 10:06, David Holmes wrote: > On 11/11/2014 12:19 AM, Aleksey Shipilev wrote: >> Then I speculated that having char[] name would help VM initialize the >> name if we wanted to switch to complete VM-side initialization of >> Thread, but it seems we can do String oop instantiation in the similar >> vein. > > I think it really just came down to accessing the Thread name from > things like JVMDI/PI (now JVM TI) - easier for C code to access a raw > char[]. Maybe once upon a time (in a land not so far away) we even > passed char[] to the Thread constructor? :) But having re-discovered > past discussions etc there's really nothing to stop this from being a > String (slight memory use increase per Thread object). Yes. char[] does appear simpler from the native side, if not that pesky Unicode requirement that forces use to use Unicode routines within the VM to deal with char[] exposed to the Java side. Not so much an improvement comparing to String oop dance. > JDK change is okay - but "name" doesn't need to be volatile when it is a > String reference. I understand the memory model reasoning about the correctness, but I think users rightfully expect getName() to return the last "updated" Thread.name, even though this requirement is not spelled out specifically. Therefore, I believe "volatile" should stay. (I would be violently disappointed about the JDK if I realized my logging is garbled and the same thread "appears" under several names back and forth within a short time window -- because of data race on Thread.name) > Hotspot side: > > src/share/vm/classfile/javaClasses.hpp > > This added assert seems overly cautious: > > 134 oop value = java_string->obj_field(value_offset); > 135 assert((value->is_typeArray() && > TypeArrayKlass::cast(value->klass())->element_type() == T_CHAR), "expect > char[]"); > > you are basically checking that String.value is defined as a char[]. If > warranted, this is a check needed once in the lifetime of a VM not every > time this method is called. (Yes I see we had something similarly odd in > java_lang_thread::name. :( ) Agreed. Dropped the assert from here. I think we already check this for String.name field when we pre-compute the value_offset. > --- > > src/share/vm/classfile/javaClasses.cpp > > ! oop java_lang_Thread::name(oop java_thread) { > oop name = java_thread->obj_field(_name_offset); > ! assert(name != NULL, "thread name is NULL"); > > I'm not confident this can never be called before the name has been set. > The original assertion allowed for NULL as does the JVM TI code. Agreed. Dropped the assert altogether. > --- > > src/share/vm/prims/jvmtiTrace.cpp > > Copyright year needs updating. :) Done. > --- > > Aside: I wonder if we've inadvertently fixed 6771287 now. :) That was a > fun one to debug. Ouch. -Aleksey. From david.holmes at oracle.com Tue Nov 11 10:29:22 2014 From: david.holmes at oracle.com (David Holmes) Date: Tue, 11 Nov 2014 20:29:22 +1000 Subject: RFR (S) 8059677: Thread.getName() instantiates Strings In-Reply-To: <5461D926.6010008@oracle.com> References: <545FB64F.7090705@oracle.com> <5460A6E8.9050506@oracle.com> <5460B608.4050909@oracle.com> <5460C354.5000605@oracle.com> <5460C960.9080509@oracle.com> <5461B593.1000104@oracle.com> <5461D926.6010008@oracle.com> Message-ID: <5461E502.4070700@oracle.com> On 11/11/2014 7:38 PM, Aleksey Shipilev wrote: > Hi David, > > Updated webrevs will follow after I respin the tests. Meanwhile, some > comments below: > > On 11.11.2014 10:06, David Holmes wrote: >> On 11/11/2014 12:19 AM, Aleksey Shipilev wrote: >>> Then I speculated that having char[] name would help VM initialize the >>> name if we wanted to switch to complete VM-side initialization of >>> Thread, but it seems we can do String oop instantiation in the similar >>> vein. >> >> I think it really just came down to accessing the Thread name from >> things like JVMDI/PI (now JVM TI) - easier for C code to access a raw >> char[]. Maybe once upon a time (in a land not so far away) we even >> passed char[] to the Thread constructor? :) But having re-discovered >> past discussions etc there's really nothing to stop this from being a >> String (slight memory use increase per Thread object). > > Yes. char[] does appear simpler from the native side, if not that pesky > Unicode requirement that forces use to use Unicode routines within the > VM to deal with char[] exposed to the Java side. Not so much an > improvement comparing to String oop dance. > > >> JDK change is okay - but "name" doesn't need to be volatile when it is a >> String reference. > > I understand the memory model reasoning about the correctness, but I > think users rightfully expect getName() to return the last "updated" > Thread.name, even though this requirement is not spelled out > specifically. Therefore, I believe "volatile" should stay. > > (I would be violently disappointed about the JDK if I realized my > logging is garbled and the same thread "appears" under several names > back and forth within a short time window -- because of data race on > Thread.name) Yes - silly of me. I was thinking the name is only set once but of course it can be set many times. Cheers, David ------ >> Hotspot side: >> >> src/share/vm/classfile/javaClasses.hpp >> >> This added assert seems overly cautious: >> >> 134 oop value = java_string->obj_field(value_offset); >> 135 assert((value->is_typeArray() && >> TypeArrayKlass::cast(value->klass())->element_type() == T_CHAR), "expect >> char[]"); >> >> you are basically checking that String.value is defined as a char[]. If >> warranted, this is a check needed once in the lifetime of a VM not every >> time this method is called. (Yes I see we had something similarly odd in >> java_lang_thread::name. :( ) > > Agreed. Dropped the assert from here. I think we already check this for > String.name field when we pre-compute the value_offset. > > >> --- >> >> src/share/vm/classfile/javaClasses.cpp >> >> ! oop java_lang_Thread::name(oop java_thread) { >> oop name = java_thread->obj_field(_name_offset); >> ! assert(name != NULL, "thread name is NULL"); >> >> I'm not confident this can never be called before the name has been set. >> The original assertion allowed for NULL as does the JVM TI code. > > Agreed. Dropped the assert altogether. > > >> --- >> >> src/share/vm/prims/jvmtiTrace.cpp >> >> Copyright year needs updating. :) > > Done. > > >> --- >> >> Aside: I wonder if we've inadvertently fixed 6771287 now. :) That was a >> fun one to debug. > > Ouch. > > > -Aleksey. > > > From aleksey.shipilev at oracle.com Tue Nov 11 14:40:58 2014 From: aleksey.shipilev at oracle.com (Aleksey Shipilev) Date: Tue, 11 Nov 2014 17:40:58 +0300 Subject: RFR (S) 8059677: Thread.getName() instantiates Strings In-Reply-To: <545FB64F.7090705@oracle.com> References: <545FB64F.7090705@oracle.com> Message-ID: <54621FFA.2070503@oracle.com> Hi, On 11/09/2014 09:45 PM, Aleksey Shipilev wrote: > Thread.getName() returns String, and does new String instantiation every > time, because the thread name is stored in char[]. Even though we use a > private String constructor that shares the char[] array without copying > it, this still hurts some use cases (think extra-fast logging). To the > extent some people actually maintain Map to avoid it. > https://bugs.openjdk.java.net/browse/JDK-8059677 > > Here's the attempt to maintain String instead of char[]: > http://cr.openjdk.java.net/~shade/8059677/webrev.01.jdk/ > http://cr.openjdk.java.net/~shade/8059677/webrev.01.hs/ Updated webrevs: http://cr.openjdk.java.net/~shade/8059677/webrev.02.jdk/ http://cr.openjdk.java.net/~shade/8059677/webrev.02.hs/ This version incorporates feedbacks from Chris, Staffan and David. I think it is very close to what we would like to push. Opinions? Testing: JPRT, jdk/test/java/lang/Thread jtreg, hotspot/test/runtime/ jtreg, vm.quick.testlist, nsk.jvmti.testlist, svc.quick.testlist, vm.tmtools.testlist Thanks, -Aleksey. From chris.hegarty at oracle.com Tue Nov 11 16:10:35 2014 From: chris.hegarty at oracle.com (Chris Hegarty) Date: Tue, 11 Nov 2014 16:10:35 +0000 Subject: RFR (S) 8059677: Thread.getName() instantiates Strings In-Reply-To: <54621FFA.2070503@oracle.com> References: <545FB64F.7090705@oracle.com> <54621FFA.2070503@oracle.com> Message-ID: On 11 Nov 2014, at 14:40, Aleksey Shipilev wrote: > Hi, > > On 11/09/2014 09:45 PM, Aleksey Shipilev wrote: >> Thread.getName() returns String, and does new String instantiation every >> time, because the thread name is stored in char[]. Even though we use a >> private String constructor that shares the char[] array without copying >> it, this still hurts some use cases (think extra-fast logging). To the >> extent some people actually maintain Map to avoid it. >> https://bugs.openjdk.java.net/browse/JDK-8059677 >> >> Here's the attempt to maintain String instead of char[]: >> http://cr.openjdk.java.net/~shade/8059677/webrev.01.jdk/ >> http://cr.openjdk.java.net/~shade/8059677/webrev.01.hs/ > > Updated webrevs: > http://cr.openjdk.java.net/~shade/8059677/webrev.02.jdk/ Looks good. > http://cr.openjdk.java.net/~shade/8059677/webrev.02.hs/ I skimmed this webrev, and it also looks fine to me. -Chris. > This version incorporates feedbacks from Chris, Staffan and David. I > think it is very close to what we would like to push. Opinions? > > Testing: JPRT, jdk/test/java/lang/Thread jtreg, hotspot/test/runtime/ > jtreg, vm.quick.testlist, nsk.jvmti.testlist, svc.quick.testlist, > vm.tmtools.testlist > > Thanks, > -Aleksey. > > > > From david.r.chase at oracle.com Tue Nov 11 18:58:30 2014 From: david.r.chase at oracle.com (David Chase) Date: Tue, 11 Nov 2014 13:58:30 -0500 Subject: [9] RFR(L) 8013267 : move MemberNameTable from native code to Java heap, use to intern MemberNames In-Reply-To: <545E31BA.3070500@gmail.com> References: <594FE416-D445-426E-B7A9-C75F012ADE16@oracle.com> <0BDCD5DD-CBFB-4A3D-9BAD-7F9912E9237C@oracle.com> <5453C230.8010709@oracle.com> <9747A3A3-B7F4-407A-8E00-1E647D9DC2D1@oracle.com> <1D195ED3-8BD2-46CB-9D70-29CF809D9F5A@oracle.com> <5456FB59.60905@oracle.com> <632A5C98-B386-4625-BE12-355241581955@oracle.com> <5457AA75.8090103@gmail.com> <5457E0F9.8090004@gmail.com> <5458A57C.4060208@gmail.com> <260D49F5-6380-4FC3-A900-6CD9AB3ED6F7@oracle.com> <5459034E.8070809@gmail.com> <39826508-110B-4FCE-9A58-8C3D1B9FC7DE@oracle.com> <545E31BA.3070500@gmail.com> Message-ID: <1D96462F-87D3-4794-A818-27DD2EE10046@oracle.com> On 2014-11-08, at 10:07 AM, Peter Levart wrote: > > Now let's take for example one of the MemberName.make() methods that return interned MemberNames: > > 206 public static MemberName make(Method m, boolean wantSpecial) { > 207 // Unreflected member names are resolved so intern them here. > 208 MemberName tmp0 = null; > 209 InternTransaction tx = new InternTransaction(m.getDeclaringClass()); > 210 while (tmp0 == null) { > 211 MemberName tmp = new MemberName(m, wantSpecial); > 212 tmp0 = tx.tryIntern(tmp); > 213 } > 214 return tmp0; > 215 } > > I'm trying to understand the workings of InternTransaction helper class (and find an example that breaks it). You create an instance of it, passing Method's declaringClass. You then (in retry loop) create a resolved MemberName from the Method and wantSpecial flag. This MemberName's clazz can apparently differ from Method's declaringClass. I don't know when and why this happens, but apparently it can (super method?), so in InternTransaction.tryIntern() you do... > > 363 if (member_name.isResolved()) { > 364 if (member_name.clazz != tx_class) { > 365 Class prev_tx_class = tx_class; > 366 int prev_txn_token = txn_token; > 367 tx_class = member_name.clazz; > 368 txn_token = internTxnToken(tx_class); > 369 // Zero is a special case. > 370 if (txn_token != 0 || > 371 prev_txn_token != internTxnToken(prev_tx_class)) { > 372 // Resolved class is different and at least one > 373 // redef of it occurred, therefore repeat with > 374 // proper class for race consistency checking. > 375 return null; > 376 } > 377 } > 378 member_name = member_name.intern(txn_token); > 379 if (member_name == null) { > 380 // Update the token for the next try. > 381 txn_token = internTxnToken(tx_class); > 382 } > 383 } > > > Now let's assume that the resolved member_name.clazz differs from Method's declaringClass. Let's assume also that either member_name.clazz has had at least one redefinition or Method's declaringClass has been redefined between creating InternTransaction and reading member_name.clazz's txn_token. You return 'null' in such case, concluding that not only the resolved member_name.clazz redefinition matters, but Method's declaringClass redefinition can also invalidate resolved MemberName am I right? It would be helpful if I could understand when and how Method's declaringClass redefinition can affect member_name. Can it affect which clazz is resolved for member_name? If a declaring class is redefined before a MemberName is ?published? to the VM, then there is a risk that its secret fields will have gone stale because the referenced VM methods changed but were not updated. Therefore, the resolution must be retried to get a fresh resolution that is known not to be stale. There is sort of a glitch in the race-checking protocol; I don?t have certain knowledge which class will be resolved, so if I guessed wrong (and the common-case no redefinition at all check fails) then I am forced to retry and get a fresh, known-good resolution. However, based on my understanding of what is (not) allowed in class redefinition, what differs after redefinition is only the code of the method, and not the owner ? that is, if D.m resolved to B.m before redefinition of D, C, or B, then it will always resolve to B.m ? but the definition of B.m itself might have changed (from the test cases, it might print ?foo? instead of ?bar?). Or to put it differently, the methods change, but their hierarchy does not. > Anyway, you return null in such case from an updated InternTransaction (tx_class and txn_token are now updated to have values for resolved member_name.clazz). In next round the checks of newly constructed and resolved member_name are not performed against Method's declaringClass but against previous round's member_name.clazz. Is this what is intended? > I can see there has to be a stop condition for loop to end, but shouldn't checks for Method's declaringClass redefinition be performed in every iteration (in addition to the check for member_name.clazz redefinition if it differs from Method's declaringClass)? To the best of my understanding (see restrictions above) the tx_class ought to be wrong at most once; all subsequent resolutions including those that span a class redefinition should return the same class, so it suffices to detect redefinition of the method itself. I?ve incorporated your other changes (not yet the linear-scan hash table) and will be retesting. One thing I wonder about for both hash table and binary search is if the first try should be attempted with no lock to avoid the overhead of synchronization; I expect that looking will be more common than interning, which in turn will be (vastly) more common than class redefinition. David From peter.levart at gmail.com Tue Nov 11 19:30:10 2014 From: peter.levart at gmail.com (Peter Levart) Date: Tue, 11 Nov 2014 20:30:10 +0100 Subject: [9] RFR(L) 8013267 : move MemberNameTable from native code to Java heap, use to intern MemberNames In-Reply-To: <1D96462F-87D3-4794-A818-27DD2EE10046@oracle.com> References: <594FE416-D445-426E-B7A9-C75F012ADE16@oracle.com> <0BDCD5DD-CBFB-4A3D-9BAD-7F9912E9237C@oracle.com> <5453C230.8010709@oracle.com> <9747A3A3-B7F4-407A-8E00-1E647D9DC2D1@oracle.com> <1D195ED3-8BD2-46CB-9D70-29CF809D9F5A@oracle.com> <5456FB59.60905@oracle.com> <632A5C98-B386-4625-BE12-355241581955@oracle.com> <5457AA75.8090103@gmail.com> <5457E0F9.8090004@gmail.com> <5458A57C.4060208@gmail.com> <260D49F5-6380-4FC3-A900-6CD9AB3ED6F7@oracle.com> <5459034E.8070809@gmail.com> <39826508-110B-4FCE-9A58-8C3D1B9FC7DE@oracle.com> <545E31BA.3070500@gmail.com> <1D96462F-87D3-4794-A818-27DD2EE10046@oracle.com> Message-ID: <546263C2.1060508@gmail.com> On 11/11/2014 07:58 PM, David Chase wrote: > I?ve incorporated your other changes (not yet the linear-scan hash table) and will be retesting. > One thing I wonder about for both hash table and binary search is if the first try should be attempted with no lock to avoid the overhead of synchronization; I expect that looking will be more common than interning, which in turn will be (vastly) more common than class redefinition. Hi David, Yes, that's why I implemented the hash table in a way where lookups are lock-free. Binary-search would be trickier to implement without locking, but maybe not impossible. Surely not with Arrays.binarySearch() but perhaps with a separate implementation. The problem with Arrays.binarySearch is that it returns an index. By the time you retrieve the element at that index, it can already move. I'm also not sure that "careful" concurrent insertion of new element would not break the correctness of binary search. But there is another way I showed before: using StampedLock. It is a kind of optimistic/pessimistic read-write lock. Its beauty is in that optimistic read part is almost free (just a volatile read at start and a readFence followed by another volatile read at the end). You just have to be sure that the algorithm guarded by an optimistic read lock terminates normally (that it doesn't spin in an endless loop or throw exceptions) in the presence of arbitrary concurrent modifications of looked-up state. Well, binary search is such an algorithm. Regards, Peter > David From aleksey.shipilev at oracle.com Tue Nov 11 19:35:24 2014 From: aleksey.shipilev at oracle.com (Aleksey Shipilev) Date: Tue, 11 Nov 2014 22:35:24 +0300 Subject: RFR (S) 8059677: Thread.getName() instantiates Strings In-Reply-To: References: <545FB64F.7090705@oracle.com> <54621FFA.2070503@oracle.com> Message-ID: <546264FC.2060001@oracle.com> On 11/11/2014 07:10 PM, Chris Hegarty wrote: > On 11 Nov 2014, at 14:40, Aleksey Shipilev wrote: >> Updated webrevs: >> http://cr.openjdk.java.net/~shade/8059677/webrev.02.jdk/ > > Looks good. > >> http://cr.openjdk.java.net/~shade/8059677/webrev.02.hs/ > > I skimmed this webrev, and it also looks fine to me. > > -Chris. Thanks Chris! -Aleksey. From tristan.yan at oracle.com Tue Nov 11 20:26:07 2014 From: tristan.yan at oracle.com (Tristan Yan) Date: Tue, 11 Nov 2014 12:26:07 -0800 Subject: RFR 8047962: XML test colocation: AuctionPortal test for bid.com In-Reply-To: <97F180B9-DD34-4450-92B5-E9CBA369E8C8@oracle.com> References: <545815D5.5020009@oracle.com> <545AB159.9090905@oracle.com> <97F180B9-DD34-4450-92B5-E9CBA369E8C8@oracle.com> Message-ID: Hi Joe I changed movies.xml to a txt format for avoiding binary file is not supported issue. Also I?ve added the BOM header support for txt file in JAXPTestUtilities. Please see the latest webrev at http://cr.openjdk.java.net/~tyan/JDK-8047962/webrev.04/ I appreciate you can push it if you?re okay with this change. Thank you Tristan > On Nov 5, 2014, at 4:01 PM, Tristan Yan wrote: > > Thanks again > This makes more sense to me. Now they look clearer. > > http://cr.openjdk.java.net/~tyan/JDK-8047962/webrev.03/ > > Thank you for sponsoring this. > Tristan > >> On Nov 5, 2014, at 3:23 PM, huizhe wang wrote: >> >> Hi Tristan, >> >> It's good to see movie.xml is replaced with a literal string. >> >> Sorry if I wasn't clear on the directory structure. It would be nice if all files for AuctionPortal to be placed under AuctionPortal, that is: >> test/javax/xml/jaxp/functional/test/auctionportal/ >> content/ (xml and xsd) >> golden/ (gold files) >> >> >> Would that make sense? It's nice you have the paths in one place (HiBidConstants) that makes it easier to move files around. >> >> Yes, I'll sponsor that for you. >> >> Thanks, >> Joe >> >> On 11/5/2014 2:57 PM, Tristan Yan wrote: >>> Thank you. Joe. >>> Git plugin for mercurial works well for hg command but webrev script still doesn?t support the binary file. I did one small change to replace movies.xml with a Java string to suppress this error. Also I took your advice to move xml files into auction portal directory. Changed tests have been run with and without the Security Manager as usual. All the tests pass under both scenarios. >>> >>> http://cr.openjdk.java.net/~tyan/JDK-8047962/webrev.02/ >>> >>> Could you please sponsor this for me. >>> Thank you very much. >>> Tristan >>> >>>> On Nov 3, 2014, at 3:55 PM, huizhe wang > wrote: >>>> >>>> Hi Tristan, >>>> >>>> Looks good overall. Once again, it's great to see that you've made the tests a lot cleaner, getting rid of the old report system and etc. >>>> >>>> The only issue I see is with movies.xml. If I use patch to apply your patch to my workspace, I get no movies.xml that in turn causes AuctionController to fail. It's probably related how your webrev was generated (note the error: Unexpected Error occurred reading `diff -e /dev/null new/test/javax/xml/jaxp/functional/test/content/movies.xml`: $?=0, err= 1 ). hg diff works fine generally, but not for changes in binary files. Use the git option, "-git", can fix the problem. You may see the following in your configuration: >>>> >>>> [diff] >>>> git = true >>>> >>>> Thanks, >>>> Joe >>>> >>>> On 10/31/2014 12:16 PM, Tristan Yan wrote: >>>>> Hi Joe, Alan and all others >>>>> >>>>> Would you please help reviewing these tests? The intent is moving some JAXP tests from closed to open. The associated bug number is >>>>> https://bugs.openjdk.java.net/browse/JDK-8047962 . >>>>> >>>>> These tests have been ran with and without the Security Manager. All the tests pass under both scenarios. >>>>> >>>>> http://cr.openjdk.java.net/~tyan/JDK-8047962/webrev.01/ >>>>> Thank you. >>>>> Tristan >>>> >>> >> > From staffan.larsen at oracle.com Tue Nov 11 20:38:15 2014 From: staffan.larsen at oracle.com (Staffan Larsen) Date: Tue, 11 Nov 2014 21:38:15 +0100 Subject: RFR (S) 8059677: Thread.getName() instantiates Strings In-Reply-To: <54621FFA.2070503@oracle.com> References: <545FB64F.7090705@oracle.com> <54621FFA.2070503@oracle.com> Message-ID: SA changes look good. /Staffan > On 11 nov 2014, at 15:40, Aleksey Shipilev wrote: > > Hi, > > On 11/09/2014 09:45 PM, Aleksey Shipilev wrote: >> Thread.getName() returns String, and does new String instantiation every >> time, because the thread name is stored in char[]. Even though we use a >> private String constructor that shares the char[] array without copying >> it, this still hurts some use cases (think extra-fast logging). To the >> extent some people actually maintain Map to avoid it. >> https://bugs.openjdk.java.net/browse/JDK-8059677 >> >> Here's the attempt to maintain String instead of char[]: >> http://cr.openjdk.java.net/~shade/8059677/webrev.01.jdk/ >> http://cr.openjdk.java.net/~shade/8059677/webrev.01.hs/ > > Updated webrevs: > http://cr.openjdk.java.net/~shade/8059677/webrev.02.jdk/ > http://cr.openjdk.java.net/~shade/8059677/webrev.02.hs/ > > This version incorporates feedbacks from Chris, Staffan and David. I > think it is very close to what we would like to push. Opinions? > > Testing: JPRT, jdk/test/java/lang/Thread jtreg, hotspot/test/runtime/ > jtreg, vm.quick.testlist, nsk.jvmti.testlist, svc.quick.testlist, > vm.tmtools.testlist > > Thanks, > -Aleksey. > > > > From huizhe.wang at oracle.com Tue Nov 11 21:27:55 2014 From: huizhe.wang at oracle.com (huizhe wang) Date: Tue, 11 Nov 2014 13:27:55 -0800 Subject: RFR 8047962: XML test colocation: AuctionPortal test for bid.com In-Reply-To: References: <545815D5.5020009@oracle.com> <545AB159.9090905@oracle.com> <97F180B9-DD34-4450-92B5-E9CBA369E8C8@oracle.com> Message-ID: <54627F5B.70505@oracle.com> Hi Tristan, Did you mean to keep the original code you replaced with nio, that is, the ones you commented out and the read method that's no longer used? -Joe On 11/11/2014 12:26 PM, Tristan Yan wrote: > Hi Joe > I changed movies.xml to a txt format for avoiding binary file is not > supported issue. Also I?ve added the BOM header support for txt file > in JAXPTestUtilities. Please see the latest webrev at > http://cr.openjdk.java.net/~tyan/JDK-8047962/webrev.04/ > > I appreciate you can push it if you?re okay with this change. > Thank you > Tristan > >> On Nov 5, 2014, at 4:01 PM, Tristan Yan > > wrote: >> >> Thanks again >> This makes more sense to me. Now they look clearer. >> >> http://cr.openjdk.java.net/~tyan/JDK-8047962/webrev.03/ >> >> > > >> >> Thank you for sponsoring this. >> Tristan >> >>> On Nov 5, 2014, at 3:23 PM, huizhe wang >> > wrote: >>> >>> Hi Tristan, >>> >>> It's good to see movie.xml is replaced with a literal string. >>> >>> Sorry if I wasn't clear on the directory structure. It would be nice >>> if all files for AuctionPortal to be placed under AuctionPortal, >>> that is: >>> test/javax/xml/jaxp/functional/test/auctionportal/ >>> content/ (xml and xsd) >>> golden/ (gold files) >>> >>> >>> Would that make sense? It's nice you have the paths in one place >>> (HiBidConstants) that makes it easier to move files around. >>> >>> Yes, I'll sponsor that for you. >>> >>> Thanks, >>> Joe >>> >>> On 11/5/2014 2:57 PM, Tristan Yan wrote: >>>> Thank you. Joe. >>>> Git plugin for mercurial works well for hg command but webrev >>>> script still doesn?t support the binary file. I did one small >>>> change to replace movies.xml with a Java string to suppress this >>>> error. Also I took your advice to move xml files into auction >>>> portal directory. Changed tests have been run with and without the >>>> Security Manager as usual. All the tests pass under both scenarios. >>>> >>>> http://cr.openjdk.java.net/~tyan/JDK-8047962/webrev.02/ >>>> >>>> >>>> >>>> Could you please sponsor this for me. >>>> Thank you very much. >>>> Tristan >>>> >>>>> On Nov 3, 2014, at 3:55 PM, huizhe wang >>>> > >>>>> wrote: >>>>> >>>>> Hi Tristan, >>>>> >>>>> Looks good overall. Once again, it's great to see that you've made >>>>> the tests a lot cleaner, getting rid of the old report system and etc. >>>>> >>>>> The only issue I see is with movies.xml. If I use patch to apply >>>>> your patch to my workspace, I get no movies.xml that in turn >>>>> causes AuctionController to fail. It's probably related how your >>>>> webrev was generated (note the error: Unexpected Error occurred >>>>> reading `diff -e /dev/null >>>>> new/test/javax/xml/jaxp/functional/test/content/movies.xml`: $?=0, >>>>> err= 1 ). hg diff works fine generally, but not for changes in >>>>> binary files. Use the git option, "-git", can fix the problem. You >>>>> may see the following in your configuration: >>>>> >>>>> [diff] >>>>> git = true >>>>> >>>>> Thanks, >>>>> Joe >>>>> >>>>> On 10/31/2014 12:16 PM, Tristan Yan wrote: >>>>>> Hi Joe, Alan and all others >>>>>> >>>>>> Would you please help reviewing these tests? The intent is moving >>>>>> some JAXP tests from closed to open. The associated bug number is >>>>>> https://bugs.openjdk.java.net/browse/JDK-8047962 >>>>>> . >>>>>> >>>>>> These tests have been ran with and without the Security Manager. >>>>>> All the tests pass under both scenarios. >>>>>> >>>>>> http://cr.openjdk.java.net/~tyan/JDK-8047962/webrev.01/ >>>>>> >>>>>> >>>>>> Thank you. >>>>>> Tristan >>>>> >>>> >>> >> > From david.holmes at oracle.com Wed Nov 12 04:48:53 2014 From: david.holmes at oracle.com (David Holmes) Date: Wed, 12 Nov 2014 14:48:53 +1000 Subject: RFR (S) 8059677: Thread.getName() instantiates Strings In-Reply-To: <54621FFA.2070503@oracle.com> References: <545FB64F.7090705@oracle.com> <54621FFA.2070503@oracle.com> Message-ID: <5462E6B5.7080504@oracle.com> On 12/11/2014 12:40 AM, Aleksey Shipilev wrote: > Hi, > > On 11/09/2014 09:45 PM, Aleksey Shipilev wrote: >> Thread.getName() returns String, and does new String instantiation every >> time, because the thread name is stored in char[]. Even though we use a >> private String constructor that shares the char[] array without copying >> it, this still hurts some use cases (think extra-fast logging). To the >> extent some people actually maintain Map to avoid it. >> https://bugs.openjdk.java.net/browse/JDK-8059677 >> >> Here's the attempt to maintain String instead of char[]: >> http://cr.openjdk.java.net/~shade/8059677/webrev.01.jdk/ >> http://cr.openjdk.java.net/~shade/8059677/webrev.01.hs/ > > Updated webrevs: > http://cr.openjdk.java.net/~shade/8059677/webrev.02.jdk/ > http://cr.openjdk.java.net/~shade/8059677/webrev.02.hs/ > > This version incorporates feedbacks from Chris, Staffan and David. I > think it is very close to what we would like to push. Opinions? All looks good to me. But I also noticed this strange (to me) assertion in javaClasses.cpp void java_lang_Thread::set_name(oop java_thread, oop name) { assert(java_thread->obj_field(_name_offset) == NULL, "name should be NULL"); java_thread->obj_field_put(_name_offset, name); } and on investigation it seems like this is dead code - I couldn't locate a call to java_lang_Thread::set_name ?? It would only be usable on an attaching thread (else name can't be null) and we pass the name to the Thread constructor in that case. Cheers, David > Testing: JPRT, jdk/test/java/lang/Thread jtreg, hotspot/test/runtime/ > jtreg, vm.quick.testlist, nsk.jvmti.testlist, svc.quick.testlist, > vm.tmtools.testlist > > Thanks, > -Aleksey. > > > > From otaviojava at java.net Wed Nov 12 07:19:52 2014 From: otaviojava at java.net (=?UTF-8?Q?Ot=C3=A1vio_Gon=C3=A7alves_de_Santana?=) Date: Wed, 12 Nov 2014 05:19:52 -0200 Subject: Review request: JDK-8055723 Replace concat String to append in StringBuilder parameters In-Reply-To: <545E5703.7060902@CoSoCo.de> References: <5445A836.8090005@CoSoCo.de> <545E5703.7060902@CoSoCo.de> Message-ID: Thank you Ulf. http://cr.openjdk.java.net/~weijun/8055723/webrev.01/ On Sat, Nov 8, 2014 at 3:46 PM, Ulf Zibis wrote: > Hi Ot?vio, > in sun/tools/jstat/SyntaxException.java I see a possible enhencement > (maybe applies to other places too): > > 65 public SyntaxException(int lineno, Set expected, Token > found) { > 66 StringBuilder msg = new StringBuilder(A + B * > expected.size()); > 67 > 68 msg.append("Syntax error at line ").append(lineno).append(": > Expected one of \'"); > 69 > 71 for (String keyWord : expected) { > 72 msg.append(keyWord).append('|'); > 73 } > 74 // if (!expected.isEmpty()) // only needed if expected may be > empty. > 75 msg.setLength(msg.length() - 1); > 76 > 81 message = msg.append("\', Found ").append(found.toMessage()). > toString(); > 83 } > > ***** Additionally at many places you could similarly introduce the > foreach syntax. > > -Ulf > > > Am 02.11.2014 um 15:45 schrieb Ot?vio Gon?alves de Santana: > >> Could another reviewer look these codes, please. >> http://cr.openjdk.java.net/~weijun/8055723/webrev.00/ < >> http://cr.openjdk.java.net/%7Eweijun/8055723/webrev.00/> >> >> On Fri, Oct 24, 2014 at 3:25 AM, Ot?vio Gon?alves de Santana < >> otaviojava at java.net > wrote: >> >> Thank you Ulf. >> I removed the fix in toString method and in debug classes: >> http://cr.openjdk.java.net/~weijun/8055723/webrev.00/ >> >> >> On Mon, Oct 20, 2014 at 10:26 PM, Ulf Zibis > > >> wrote: >> >> >> Am 21.10.2014 um 01:02 schrieb Ot?vio Gon?alves de Santana: >> >> BUGURL: https://bugs.openjdk.java.net/browse/JDK-8055723 >> >> >> WEBREV: http://cr.openjdk.java.net/~ >> weijun/8055723/client/webrev.02/ >> > webrev.02/> >> WEBREV: http://cr.openjdk.java.net/~ >> weijun/8055723/core/webrev.03/ >> > > >> >> >> I did not look through all sources. >> In Scanner.java I discovered: >> 1307 sb.append("[delimiters=").append(delimPattern).append(']'); >> 1308 sb.append("[position=").append(position).append(']'); >> ... >> Maybe better: >> 1307 sb.append("[delimiters=").append(delimPattern); >> 1308 sb.append("][position=").append(position); >> ... >> >> -Ulf >> >> >> >> >> -- Ot?vio Gon?alves de Santana >> >> blog: http://otaviosantana.blogspot.com.br/ >> twitter: http://twitter.com/otaviojava >> site: _http://about.me/otaviojava_ >> 55 (11) 98255-3513 >> >> >> >> >> -- >> Ot?vio Gon?alves de Santana >> >> blog: http://otaviosantana.blogspot.com.br/ >> twitter: http://twitter.com/otaviojava >> site: _http://about.me/otaviojava_ >> 55 (11) 98255-3513 >> > > -- Ot?vio Gon?alves de Santana blog: http://otaviosantana.blogspot.com.br/ twitter: http://twitter.com/otaviojava site: *http://about.me/otaviojava * 55 (11) 98255-3513 From aleksey.shipilev at oracle.com Wed Nov 12 10:18:41 2014 From: aleksey.shipilev at oracle.com (Aleksey Shipilev) Date: Wed, 12 Nov 2014 13:18:41 +0300 Subject: RFR (S) 8059677: Thread.getName() instantiates Strings In-Reply-To: References: <545FB64F.7090705@oracle.com> <54621FFA.2070503@oracle.com> Message-ID: <54633401.6040208@oracle.com> Thanks Staffan! -Aleksey. On 11.11.2014 23:38, Staffan Larsen wrote: > SA changes look good. > > /Staffan > >> On 11 nov 2014, at 15:40, Aleksey Shipilev wrote: >> >> Hi, >> >> On 11/09/2014 09:45 PM, Aleksey Shipilev wrote: >>> Thread.getName() returns String, and does new String instantiation every >>> time, because the thread name is stored in char[]. Even though we use a >>> private String constructor that shares the char[] array without copying >>> it, this still hurts some use cases (think extra-fast logging). To the >>> extent some people actually maintain Map to avoid it. >>> https://bugs.openjdk.java.net/browse/JDK-8059677 >>> >>> Here's the attempt to maintain String instead of char[]: >>> http://cr.openjdk.java.net/~shade/8059677/webrev.01.jdk/ >>> http://cr.openjdk.java.net/~shade/8059677/webrev.01.hs/ >> >> Updated webrevs: >> http://cr.openjdk.java.net/~shade/8059677/webrev.02.jdk/ >> http://cr.openjdk.java.net/~shade/8059677/webrev.02.hs/ >> >> This version incorporates feedbacks from Chris, Staffan and David. I >> think it is very close to what we would like to push. Opinions? >> >> Testing: JPRT, jdk/test/java/lang/Thread jtreg, hotspot/test/runtime/ >> jtreg, vm.quick.testlist, nsk.jvmti.testlist, svc.quick.testlist, >> vm.tmtools.testlist >> >> Thanks, >> -Aleksey. >> >> >> >> > From aleksey.shipilev at oracle.com Wed Nov 12 10:23:32 2014 From: aleksey.shipilev at oracle.com (Aleksey Shipilev) Date: Wed, 12 Nov 2014 13:23:32 +0300 Subject: RFR (S) 8059677: Thread.getName() instantiates Strings In-Reply-To: <5462E6B5.7080504@oracle.com> References: <545FB64F.7090705@oracle.com> <54621FFA.2070503@oracle.com> <5462E6B5.7080504@oracle.com> Message-ID: <54633524.8000607@oracle.com> Hi David, On 12.11.2014 07:48, David Holmes wrote: > On 12/11/2014 12:40 AM, Aleksey Shipilev wrote: > All looks good to me. Thanks for the review! > But I also noticed this strange (to me) assertion in javaClasses.cpp > > void java_lang_Thread::set_name(oop java_thread, oop name) { > assert(java_thread->obj_field(_name_offset) == NULL, "name should be > NULL"); > java_thread->obj_field_put(_name_offset, name); > } > > and on investigation it seems like this is dead code - I couldn't locate > a call to java_lang_Thread::set_name ?? It would only be usable on an > attaching thread (else name can't be null) and we pass the name to the > Thread constructor in that case. set_name is not used, as I mentioned earlier -- that makes the change even more "safe". I was even tempted to drop the setter completely, but it would break the symmetry against other setters and getters. I dropped the assert at set_name in this update: http://cr.openjdk.java.net/~shade/8059677/webrev.03.hs/ http://cr.openjdk.java.net/~shade/8059677/webrev.03.jdk/ The only difference against the previous version is the dropped assert, so I haven't re-spinned the tests. Thanks, -Aleksey. From Ulf.Zibis at CoSoCo.de Wed Nov 12 10:36:29 2014 From: Ulf.Zibis at CoSoCo.de (Ulf Zibis) Date: Wed, 12 Nov 2014 11:36:29 +0100 Subject: Review request: JDK-8055723 Replace concat String to append in StringBuilder parameters In-Reply-To: References: <5445A836.8090005@CoSoCo.de> <545E5703.7060902@CoSoCo.de> Message-ID: <5463382D.20201@CoSoCo.de> Hi Ot?vio, I now think you could replace if (!expected.isEmpty()) with assert !expected.isEmpty(); If expected ever would be empty, the only thing which happens is, that a "'" is missing in a message which anyway doesn't make sense without arguments. -Ulf Am 12.11.2014 um 08:19 schrieb Ot?vio Gon?alves de Santana: > Thank you Ulf. > http://cr.openjdk.java.net/~weijun/8055723/webrev.01/ > > > On Sat, Nov 8, 2014 at 3:46 PM, Ulf Zibis > wrote: > > Hi Ot?vio, > in sun/tools/jstat/SyntaxException.java I see a possible enhencement (maybe applies to other > places too): > > 65 public SyntaxException(int lineno, Set expected, Token found) { > 66 StringBuilder msg = new StringBuilder(A + B * expected.size()); > 67 > 68 msg.append("Syntax error at line ").append(lineno).append(": Expected one of \'"); > 69 > 71 for (String keyWord : expected) { > 72 msg.append(keyWord).append('|'); > 73 } > 74 // if (!expected.isEmpty()) // only needed if expected may be empty. > 75 msg.setLength(msg.length() - 1); > 76 > 81 message = msg.append("\', Found ").append(found.toMessage()).toString(); > 83 } > > ***** Additionally at many places you could similarly introduce the foreach syntax. > > -Ulf > > > Am 02.11.2014 um 15:45 schrieb Ot?vio Gon?alves de Santana: > > Could another reviewer look these codes, please. > http://cr.openjdk.java.net/~weijun/8055723/webrev.00/ > > > > On Fri, Oct 24, 2014 at 3:25 AM, Ot?vio Gon?alves de Santana >> wrote: > > Thank you Ulf. > I removed the fix in toString method and in debug classes: > http://cr.openjdk.java.net/~weijun/8055723/webrev.00/ > > > > On Mon, Oct 20, 2014 at 10:26 PM, Ulf Zibis >> > wrote: > > > Am 21.10.2014 um 01:02 schrieb Ot?vio Gon?alves de Santana: > > BUGURL: https://bugs.openjdk.java.net/browse/JDK-8055723 > > > WEBREV: http://cr.openjdk.java.net/~weijun/8055723/client/webrev.02/ > > > WEBREV: http://cr.openjdk.java.net/~weijun/8055723/core/webrev.03/ > > > > > I did not look through all sources. > In Scanner.java I discovered: > 1307 sb.append("[delimiters=").append(delimPattern).append(']'); > 1308 sb.append("[position=").append(position).append(']'); > ... > Maybe better: > 1307 sb.append("[delimiters=").append(delimPattern); > 1308 sb.append("][position=").append(position); > ... > > -Ulf > > > > > -- Ot?vio Gon?alves de Santana > > blog: http://otaviosantana.blogspot.com.br/ > twitter: http://twitter.com/otaviojava > site: _http://about.me/otaviojava_ > 55 (11) 98255-3513 > > > > > -- > Ot?vio Gon?alves de Santana > > blog: http://otaviosantana.blogspot.com.br/ > twitter: http://twitter.com/otaviojava > site: _http://about.me/otaviojava_ > 55 (11) 98255-3513 > > > > > > -- > Ot?vio Gon?alves de Santana > > blog: http://otaviosantana.blogspot.com.br/ > twitter: http://twitter.com/otaviojava > site: _http://about.me/otaviojava_ > 55 (11) 98255-3513 From david.r.chase at oracle.com Wed Nov 12 17:03:11 2014 From: david.r.chase at oracle.com (David Chase) Date: Wed, 12 Nov 2014 12:03:11 -0500 Subject: [9] RFR(L) 8013267 : move MemberNameTable from native code to Java heap, use to intern MemberNames In-Reply-To: <545F642E.30205@gmail.com> References: <594FE416-D445-426E-B7A9-C75F012ADE16@oracle.com> <0BDCD5DD-CBFB-4A3D-9BAD-7F9912E9237C@oracle.com> <5453C230.8010709@oracle.com> <9747A3A3-B7F4-407A-8E00-1E647D9DC2D1@oracle.com> <1D195ED3-8BD2-46CB-9D70-29CF809D9F5A@oracle.com> <5456FB59.60905@oracle.com> <632A5C98-B386-4625-BE12-355241581955@oracle.com> <5457AA75.8090103@gmail.com> <5457E0F9.8090004@gmail.com> <5458A57C.4060208@gmail.com> <260D49F5-6380-4FC3-A900-6CD9AB3ED6F7@oracle.com> <5459034E.8070809@gmail.com> <39826508-110B-4FCE-9A58-8C3D1B9FC7DE@oracle.com> <545F642E.30205@gmail.com> Message-ID: Hello Peter, I was looking at this (thinking it would be a useful thing to benchmark, looking for possible improvements) and noticed that you rely on the hashed objects having a sensible value-dependent hashcode (as opposed to the default Object hashcode). Sadly, this seems not to be the case for MemberNames or for ?Types?. I am sorely tempted to repair this glitch, not sure if it fits in the scope of the original bug, but there?s a lot to be said for future-performance-proofing. David On 2014-11-09, at 7:55 AM, Peter Levart wrote: > Hi David, > > I played a little with the idea of having a hash table instead of packed sorted array for interning. Using ConcurrentHashMap would present quite some memory overhead. A more compact representation is possible in the form of a linear-scan hash table where elements of array are MemberNames themselves: > > http://cr.openjdk.java.net/~plevart/misc/MemberName.intern/jdk.06.diff/ > > This is a drop-in replacement for MemberName on top of your jdk.06 patch. If you have some time, you can run this with your performance tests to see if it presents any difference. If not, then perhaps this interning is not so performance critical after all. > > Regards, Peter From david.r.chase at oracle.com Wed Nov 12 18:27:33 2014 From: david.r.chase at oracle.com (David Chase) Date: Wed, 12 Nov 2014 13:27:33 -0500 Subject: [9] RFR(L) 8013267 : move MemberNameTable from native code to Java heap, use to intern MemberNames In-Reply-To: <545F642E.30205@gmail.com> References: <594FE416-D445-426E-B7A9-C75F012ADE16@oracle.com> <0BDCD5DD-CBFB-4A3D-9BAD-7F9912E9237C@oracle.com> <5453C230.8010709@oracle.com> <9747A3A3-B7F4-407A-8E00-1E647D9DC2D1@oracle.com> <1D195ED3-8BD2-46CB-9D70-29CF809D9F5A@oracle.com> <5456FB59.60905@oracle.com> <632A5C98-B386-4625-BE12-355241581955@oracle.com> <5457AA75.8090103@gmail.com> <5457E0F9.8090004@gmail.com> <5458A57C.4060208@gmail.com> <260D49F5-6380-4FC3-A900-6CD9AB3ED6F7@oracle.com> <5459034E.8070809@gmail.com> <39826508-110B-4FCE-9A58-8C3D1B9FC7DE@oracle.com> <545F642E.30205@gmail.com> Message-ID: Hello Peter, > Sadly, this seems not to be the case for MemberNames or for ?Types?. That statement is inoperative. Mistakes were made. It?s compareTo that they lack. David On 2014-11-09, at 7:55 AM, Peter Levart wrote: > Hi David, > > I played a little with the idea of having a hash table instead of packed sorted array for interning. Using ConcurrentHashMap would present quite some memory overhead. A more compact representation is possible in the form of a linear-scan hash table where elements of array are MemberNames themselves: > > http://cr.openjdk.java.net/~plevart/misc/MemberName.intern/jdk.06.diff/ > > This is a drop-in replacement for MemberName on top of your jdk.06 patch. If you have some time, you can run this with your performance tests to see if it presents any difference. If not, then perhaps this interning is not so performance critical after all. > > Regards, Peter From roger.riggs at oracle.com Wed Nov 12 19:34:27 2014 From: roger.riggs at oracle.com (roger riggs) Date: Wed, 12 Nov 2014 14:34:27 -0500 Subject: RFR 9 8043477: java/lang/ProcessBuilder/Basic.java failed with: java.lang.AssertionError: Some tests failed Message-ID: <5463B643.4050301@oracle.com> Please review test changes to ProcessBuilder Basic.java to add some debugging information. The test has been failing intermittently. The wait times have been extended to see if the systems are just slow. The failure criteria have not changed. Suggestions welcome. Webrev: http://cr.openjdk.java.net/~rriggs/webrev-basic-debug-8043477/ issue: 8043477: java/lang/ProcessBuilder/Basic.java failed with: java.lang.AssertionError: Some tests failed Thanks, Roger [1] https://bugs.openjdk.java.net/browse/JDK-8043477 From martinrb at google.com Wed Nov 12 19:44:58 2014 From: martinrb at google.com (Martin Buchholz) Date: Wed, 12 Nov 2014 11:44:58 -0800 Subject: RFR 9 8043477: java/lang/ProcessBuilder/Basic.java failed with: java.lang.AssertionError: Some tests failed In-Reply-To: <5463B643.4050301@oracle.com> References: <5463B643.4050301@oracle.com> Message-ID: The print statement below seems redundant with the assertion failure message. You could improve the assertion message instead if need be. Adding thousand separator underscores to 200000000L would help a little. I like my little helper static long millisElapsedSince(long startNanoTime) { return NANOSECONDS.toMillis(System.nanoTime() - startNanoTime); } + System.out.printf(" waitFor process: delta: %d%n",(end - start) ); + if ((end - start) > 200000000L * (AIX.is() ? 2 : 1)) fail("Test failed: waitFor took too long (" + (end - start) + "ns)"); 200 ms timeout for subprocesses to finish is just too damn low. In j.u.c. tests we switched to 10 seconds for most "long" timeouts (LONG_DELAY_MS) and are happy with the disappearance of rare flaky results. On Wed, Nov 12, 2014 at 11:34 AM, roger riggs wrote: > Please review test changes to ProcessBuilder Basic.java to add some > debugging information. > The test has been failing intermittently. The wait times have been extended > to see > if the systems are just slow. The failure criteria have not changed. > > Suggestions welcome. > > Webrev: > http://cr.openjdk.java.net/~rriggs/webrev-basic-debug-8043477/ > > issue: > 8043477: java/lang/ProcessBuilder/Basic.java failed with: > java.lang.AssertionError: Some tests failed > > Thanks, Roger > > [1] https://bugs.openjdk.java.net/browse/JDK-8043477 > From rob.mckenna at oracle.com Wed Nov 12 20:02:22 2014 From: rob.mckenna at oracle.com (Rob McKenna) Date: Wed, 12 Nov 2014 20:02:22 +0000 Subject: RFR 9 8043477: java/lang/ProcessBuilder/Basic.java failed with: java.lang.AssertionError: Some tests failed In-Reply-To: References: <5463B643.4050301@oracle.com> Message-ID: <5463BCCE.9070903@oracle.com> Eesh, Sorry Roger, I have something like this on my todo. Martin, my concern with the long delay approach is that it effectively nullifies the point of the test. Given that this test has been flakey the approach has been to simply bump the acceptable delta by another 100ms or so every time. Since we've had to do this repeatedly however, I'm beginning to question whether the test is more trouble than its worth. -Rob On 12/11/14 19:44, Martin Buchholz wrote: > The print statement below seems redundant with the assertion failure message. > You could improve the assertion message instead if need be. > Adding thousand separator underscores to 200000000L would help a little. > > I like my little helper > > static long millisElapsedSince(long startNanoTime) { > return NANOSECONDS.toMillis(System.nanoTime() - startNanoTime); > } > > > + System.out.printf(" waitFor process: delta: %d%n",(end - start) ); > + > if ((end - start) > 200000000L * (AIX.is() ? 2 : 1)) > fail("Test failed: waitFor took too long (" + (end - > start) + "ns)"); > > 200 ms timeout for subprocesses to finish is just too damn low. In > j.u.c. tests we switched to 10 seconds for most "long" timeouts > (LONG_DELAY_MS) and are happy with the disappearance of rare flaky > results. > > > On Wed, Nov 12, 2014 at 11:34 AM, roger riggs wrote: >> Please review test changes to ProcessBuilder Basic.java to add some >> debugging information. >> The test has been failing intermittently. The wait times have been extended >> to see >> if the systems are just slow. The failure criteria have not changed. >> >> Suggestions welcome. >> >> Webrev: >> http://cr.openjdk.java.net/~rriggs/webrev-basic-debug-8043477/ >> >> issue: >> 8043477: java/lang/ProcessBuilder/Basic.java failed with: >> java.lang.AssertionError: Some tests failed >> >> Thanks, Roger >> >> [1] https://bugs.openjdk.java.net/browse/JDK-8043477 >> From roger.riggs at oracle.com Wed Nov 12 20:16:16 2014 From: roger.riggs at oracle.com (roger riggs) Date: Wed, 12 Nov 2014 15:16:16 -0500 Subject: RFR 9 8043477: java/lang/ProcessBuilder/Basic.java failed with: java.lang.AssertionError: Some tests failed In-Reply-To: <5463BCCE.9070903@oracle.com> References: <5463B643.4050301@oracle.com> <5463BCCE.9070903@oracle.com> Message-ID: <5463C010.7000606@oracle.com> Hi Martin, Rob, I thought the point of the test was to verify the timeout logic in waitFor(timeout). Adding the prints without changing the test criteria was intended to gather data about the distribution. A long timeout would still catch a failure in the reaper/waitFor communication so maybe that's ok. As for printing the time, I would switch to the newer java.time.Instant/Duration which have a decent toString. Roger On 11/12/2014 3:02 PM, Rob McKenna wrote: > Eesh, Sorry Roger, I have something like this on my todo. > > Martin, my concern with the long delay approach is that it effectively > nullifies the point of the test. Given that this test has been flakey > the approach has been to simply bump the acceptable delta by another > 100ms or so every time. Since we've had to do this repeatedly however, > I'm beginning to question whether the test is more trouble than its > worth. > > -Rob > > On 12/11/14 19:44, Martin Buchholz wrote: >> The print statement below seems redundant with the assertion failure >> message. >> You could improve the assertion message instead if need be. >> Adding thousand separator underscores to 200000000L would help a little. >> >> I like my little helper >> >> static long millisElapsedSince(long startNanoTime) { >> return NANOSECONDS.toMillis(System.nanoTime() - startNanoTime); >> } >> >> >> + System.out.printf(" waitFor process: delta: %d%n",(end - >> start) ); >> + >> if ((end - start) > 200000000L * (AIX.is() ? 2 : 1)) >> fail("Test failed: waitFor took too long (" + (end - >> start) + "ns)"); >> >> 200 ms timeout for subprocesses to finish is just too damn low. In >> j.u.c. tests we switched to 10 seconds for most "long" timeouts >> (LONG_DELAY_MS) and are happy with the disappearance of rare flaky >> results. >> >> >> On Wed, Nov 12, 2014 at 11:34 AM, roger riggs >> wrote: >>> Please review test changes to ProcessBuilder Basic.java to add some >>> debugging information. >>> The test has been failing intermittently. The wait times have been >>> extended >>> to see >>> if the systems are just slow. The failure criteria have not changed. >>> >>> Suggestions welcome. >>> >>> Webrev: >>> http://cr.openjdk.java.net/~rriggs/webrev-basic-debug-8043477/ >>> >>> issue: >>> 8043477: java/lang/ProcessBuilder/Basic.java failed with: >>> java.lang.AssertionError: Some tests failed >>> >>> Thanks, Roger >>> >>> [1] https://bugs.openjdk.java.net/browse/JDK-8043477 >>> > From martinrb at google.com Wed Nov 12 21:45:27 2014 From: martinrb at google.com (Martin Buchholz) Date: Wed, 12 Nov 2014 13:45:27 -0800 Subject: RFR 9 8043477: java/lang/ProcessBuilder/Basic.java failed with: java.lang.AssertionError: Some tests failed In-Reply-To: <5463C010.7000606@oracle.com> References: <5463B643.4050301@oracle.com> <5463BCCE.9070903@oracle.com> <5463C010.7000606@oracle.com> Message-ID: Looking over the test again (I was the original author IIRC), I think this test still has lots of value with a longer timeout. Experience seems to show that 200ms is not a long enough timeout for "system" operations to use in non-flaky tests, including everything in Process handling. I say we ncrease the timeout by at least 10x and move on. On Wed, Nov 12, 2014 at 12:16 PM, roger riggs wrote: > Hi Martin, Rob, > > I thought the point of the test was to verify the timeout logic in > waitFor(timeout). > Adding the prints without changing the test criteria was intended to gather > data > about the distribution. > > A long timeout would still catch a failure in the reaper/waitFor > communication > so maybe that's ok. > > As for printing the time, I would switch to the newer > java.time.Instant/Duration > which have a decent toString. > > Roger > > > > On 11/12/2014 3:02 PM, Rob McKenna wrote: >> >> Eesh, Sorry Roger, I have something like this on my todo. >> >> Martin, my concern with the long delay approach is that it effectively >> nullifies the point of the test. Given that this test has been flakey the >> approach has been to simply bump the acceptable delta by another 100ms or so >> every time. Since we've had to do this repeatedly however, I'm beginning to >> question whether the test is more trouble than its worth. >> >> -Rob >> >> On 12/11/14 19:44, Martin Buchholz wrote: >>> >>> The print statement below seems redundant with the assertion failure >>> message. >>> You could improve the assertion message instead if need be. >>> Adding thousand separator underscores to 200000000L would help a little. >>> >>> I like my little helper >>> >>> static long millisElapsedSince(long startNanoTime) { >>> return NANOSECONDS.toMillis(System.nanoTime() - startNanoTime); >>> } >>> >>> >>> + System.out.printf(" waitFor process: delta: %d%n",(end - >>> start) ); >>> + >>> if ((end - start) > 200000000L * (AIX.is() ? 2 : 1)) >>> fail("Test failed: waitFor took too long (" + (end - >>> start) + "ns)"); >>> >>> 200 ms timeout for subprocesses to finish is just too damn low. In >>> j.u.c. tests we switched to 10 seconds for most "long" timeouts >>> (LONG_DELAY_MS) and are happy with the disappearance of rare flaky >>> results. >>> >>> >>> On Wed, Nov 12, 2014 at 11:34 AM, roger riggs >>> wrote: >>>> >>>> Please review test changes to ProcessBuilder Basic.java to add some >>>> debugging information. >>>> The test has been failing intermittently. The wait times have been >>>> extended >>>> to see >>>> if the systems are just slow. The failure criteria have not changed. >>>> >>>> Suggestions welcome. >>>> >>>> Webrev: >>>> http://cr.openjdk.java.net/~rriggs/webrev-basic-debug-8043477/ >>>> >>>> issue: >>>> 8043477: java/lang/ProcessBuilder/Basic.java failed with: >>>> java.lang.AssertionError: Some tests failed >>>> >>>> Thanks, Roger >>>> >>>> [1] https://bugs.openjdk.java.net/browse/JDK-8043477 >>>> >> > From roger.riggs at oracle.com Wed Nov 12 21:57:01 2014 From: roger.riggs at oracle.com (roger riggs) Date: Wed, 12 Nov 2014 16:57:01 -0500 Subject: RFR 9 8043477: java/lang/ProcessBuilder/Basic.java failed with: java.lang.AssertionError: Some tests failed In-Reply-To: References: <5463B643.4050301@oracle.com> <5463BCCE.9070903@oracle.com> <5463C010.7000606@oracle.com> Message-ID: <5463D7AD.9080600@oracle.com> Hi Martin, I updated the webrev to use long timeouts for the test in question; they can't be too long because the spawned sleep is only alive for 10 seconds. Take a look please: http://cr.openjdk.java.net/~rriggs/webrev-basic-debug-8043477/ Thanks, Roger On 11/12/2014 4:45 PM, Martin Buchholz wrote: > Looking over the test again (I was the original author IIRC), > I think this test still has lots of value with a longer timeout. > > Experience seems to show that 200ms is not a long enough timeout for > "system" operations to use in non-flaky tests, including everything in > Process handling. > I say we ncrease the timeout by at least 10x and move on. > > On Wed, Nov 12, 2014 at 12:16 PM, roger riggs wrote: >> Hi Martin, Rob, >> >> I thought the point of the test was to verify the timeout logic in >> waitFor(timeout). >> Adding the prints without changing the test criteria was intended to gather >> data >> about the distribution. >> >> A long timeout would still catch a failure in the reaper/waitFor >> communication >> so maybe that's ok. >> >> As for printing the time, I would switch to the newer >> java.time.Instant/Duration >> which have a decent toString. >> >> Roger >> >> >> >> On 11/12/2014 3:02 PM, Rob McKenna wrote: >>> Eesh, Sorry Roger, I have something like this on my todo. >>> >>> Martin, my concern with the long delay approach is that it effectively >>> nullifies the point of the test. Given that this test has been flakey the >>> approach has been to simply bump the acceptable delta by another 100ms or so >>> every time. Since we've had to do this repeatedly however, I'm beginning to >>> question whether the test is more trouble than its worth. >>> >>> -Rob >>> >>> On 12/11/14 19:44, Martin Buchholz wrote: >>>> The print statement below seems redundant with the assertion failure >>>> message. >>>> You could improve the assertion message instead if need be. >>>> Adding thousand separator underscores to 200000000L would help a little. >>>> >>>> I like my little helper >>>> >>>> static long millisElapsedSince(long startNanoTime) { >>>> return NANOSECONDS.toMillis(System.nanoTime() - startNanoTime); >>>> } >>>> >>>> >>>> + System.out.printf(" waitFor process: delta: %d%n",(end - >>>> start) ); >>>> + >>>> if ((end - start) > 200000000L * (AIX.is() ? 2 : 1)) >>>> fail("Test failed: waitFor took too long (" + (end - >>>> start) + "ns)"); >>>> >>>> 200 ms timeout for subprocesses to finish is just too damn low. In >>>> j.u.c. tests we switched to 10 seconds for most "long" timeouts >>>> (LONG_DELAY_MS) and are happy with the disappearance of rare flaky >>>> results. >>>> >>>> >>>> On Wed, Nov 12, 2014 at 11:34 AM, roger riggs >>>> wrote: >>>>> Please review test changes to ProcessBuilder Basic.java to add some >>>>> debugging information. >>>>> The test has been failing intermittently. The wait times have been >>>>> extended >>>>> to see >>>>> if the systems are just slow. The failure criteria have not changed. >>>>> >>>>> Suggestions welcome. >>>>> >>>>> Webrev: >>>>> http://cr.openjdk.java.net/~rriggs/webrev-basic-debug-8043477/ >>>>> >>>>> issue: >>>>> 8043477: java/lang/ProcessBuilder/Basic.java failed with: >>>>> java.lang.AssertionError: Some tests failed >>>>> >>>>> Thanks, Roger >>>>> >>>>> [1] https://bugs.openjdk.java.net/browse/JDK-8043477 >>>>> From david.holmes at oracle.com Wed Nov 12 22:45:16 2014 From: david.holmes at oracle.com (David Holmes) Date: Thu, 13 Nov 2014 08:45:16 +1000 Subject: RFR (S) 8059677: Thread.getName() instantiates Strings In-Reply-To: <54633524.8000607@oracle.com> References: <545FB64F.7090705@oracle.com> <54621FFA.2070503@oracle.com> <5462E6B5.7080504@oracle.com> <54633524.8000607@oracle.com> Message-ID: <5463E2FC.5010207@oracle.com> On 12/11/2014 8:23 PM, Aleksey Shipilev wrote: > Hi David, > > On 12.11.2014 07:48, David Holmes wrote: >> On 12/11/2014 12:40 AM, Aleksey Shipilev wrote: >> All looks good to me. > > Thanks for the review! > >> But I also noticed this strange (to me) assertion in javaClasses.cpp >> >> void java_lang_Thread::set_name(oop java_thread, oop name) { >> assert(java_thread->obj_field(_name_offset) == NULL, "name should be >> NULL"); >> java_thread->obj_field_put(_name_offset, name); >> } >> >> and on investigation it seems like this is dead code - I couldn't locate >> a call to java_lang_Thread::set_name ?? It would only be usable on an >> attaching thread (else name can't be null) and we pass the name to the >> Thread constructor in that case. > > set_name is not used, as I mentioned earlier -- that makes the change Sorry, I missed that comment. > even more "safe". I was even tempted to drop the setter completely, but > it would break the symmetry against other setters and getters. I dropped > the assert at set_name in this update: > http://cr.openjdk.java.net/~shade/8059677/webrev.03.hs/ > http://cr.openjdk.java.net/~shade/8059677/webrev.03.jdk/ > > The only difference against the previous version is the dropped assert, > so I haven't re-spinned the tests. OK. I'm more inclined to delete unused code but it is fine as is. Thanks, David > Thanks, > -Aleksey. > From aleksey.shipilev at oracle.com Wed Nov 12 23:01:39 2014 From: aleksey.shipilev at oracle.com (Aleksey Shipilev) Date: Thu, 13 Nov 2014 02:01:39 +0300 Subject: RFR (S) 8059677: Thread.getName() instantiates Strings In-Reply-To: <54621FFA.2070503@oracle.com> References: <545FB64F.7090705@oracle.com> <54621FFA.2070503@oracle.com> Message-ID: <5463E6D3.6030806@oracle.com> On 11.11.2014 17:40, Aleksey Shipilev wrote: > On 11/09/2014 09:45 PM, Aleksey Shipilev wrote: >> Thread.getName() returns String, and does new String instantiation every >> time, because the thread name is stored in char[]. Even though we use a >> private String constructor that shares the char[] array without copying >> it, this still hurts some use cases (think extra-fast logging). To the >> extent some people actually maintain Map to avoid it. >> https://bugs.openjdk.java.net/browse/JDK-8059677 >> >> Here's the attempt to maintain String instead of char[]: >> http://cr.openjdk.java.net/~shade/8059677/webrev.01.jdk/ >> http://cr.openjdk.java.net/~shade/8059677/webrev.01.hs/ > > Updated webrevs: > http://cr.openjdk.java.net/~shade/8059677/webrev.02.jdk/ > http://cr.openjdk.java.net/~shade/8059677/webrev.02.hs/ All right, third time a charm. All reviewers seem to be happy with these changes: http://cr.openjdk.java.net/~shade/8059677/webrev.03.jdk/ http://cr.openjdk.java.net/~shade/8059677/webrev.03.hs/ Coleen had volunteered to sponsor them (thanks!), here are the changesets: http://cr.openjdk.java.net/~shade/8059677/8059677-jdk.changeset http://cr.openjdk.java.net/~shade/8059677/8059677-hs.changeset Thanks, -Aleksey. From martinrb at google.com Wed Nov 12 23:32:29 2014 From: martinrb at google.com (Martin Buchholz) Date: Wed, 12 Nov 2014 15:32:29 -0800 Subject: RFR 9 8043477: java/lang/ProcessBuilder/Basic.java failed with: java.lang.AssertionError: Some tests failed In-Reply-To: <5463D7AD.9080600@oracle.com> References: <5463B643.4050301@oracle.com> <5463BCCE.9070903@oracle.com> <5463C010.7000606@oracle.com> <5463D7AD.9080600@oracle.com> Message-ID: Hi Roger, On Wed, Nov 12, 2014 at 1:57 PM, roger riggs wrote: > Hi Martin, > > I updated the webrev to use long timeouts for the test in question; they > can't be too long > because the spawned sleep is only alive for 10 seconds. I am looking at the "sleep" command invocation, but it's an indefinite sleep. I don't see 10 seconds anywhere. > Take a look please: > > http://cr.openjdk.java.net/~rriggs/webrev-basic-debug-8043477/ 2313 Integer exitValue = null; 2314 try { 2315 exitValue = p.exitValue(); 2316 } catch (IllegalStateException ise) { 2317 // no exitValue 2318 } 2319 if ((end - start) > 6 * 1_000_000_000) 2320 fail("Test failed: waitFor took too long on a dead process. (" + (end - start) + "ns)" 2321 + ", exitValue: " + Objects.requireNonNull(exitValue)); exitValue throws IllegalThreadStateException, not IllegalStateException. I would fail separately in case of ITSE. It looks instead like you'll fail with uninformative NPE. 2301 p.waitFor(1000, TimeUnit.MILLISECONDS); 2304 if ((end - start) < 500_000_000) I am surprised the test is not testing that the initial timed waitFor doesn't wait at least the specified time, instead of only half. I consider returning from waitFor early a bug. > > Thanks, Roger > > > On 11/12/2014 4:45 PM, Martin Buchholz wrote: >> >> Looking over the test again (I was the original author IIRC), >> I think this test still has lots of value with a longer timeout. >> >> Experience seems to show that 200ms is not a long enough timeout for >> "system" operations to use in non-flaky tests, including everything in >> Process handling. >> I say we ncrease the timeout by at least 10x and move on. >> >> On Wed, Nov 12, 2014 at 12:16 PM, roger riggs >> wrote: >>> >>> Hi Martin, Rob, >>> >>> I thought the point of the test was to verify the timeout logic in >>> waitFor(timeout). >>> Adding the prints without changing the test criteria was intended to >>> gather >>> data >>> about the distribution. >>> >>> A long timeout would still catch a failure in the reaper/waitFor >>> communication >>> so maybe that's ok. >>> >>> As for printing the time, I would switch to the newer >>> java.time.Instant/Duration >>> which have a decent toString. >>> >>> Roger >>> >>> >>> >>> On 11/12/2014 3:02 PM, Rob McKenna wrote: >>>> >>>> Eesh, Sorry Roger, I have something like this on my todo. >>>> >>>> Martin, my concern with the long delay approach is that it effectively >>>> nullifies the point of the test. Given that this test has been flakey >>>> the >>>> approach has been to simply bump the acceptable delta by another 100ms >>>> or so >>>> every time. Since we've had to do this repeatedly however, I'm beginning >>>> to >>>> question whether the test is more trouble than its worth. >>>> >>>> -Rob >>>> >>>> On 12/11/14 19:44, Martin Buchholz wrote: >>>>> >>>>> The print statement below seems redundant with the assertion failure >>>>> message. >>>>> You could improve the assertion message instead if need be. >>>>> Adding thousand separator underscores to 200000000L would help a >>>>> little. >>>>> >>>>> I like my little helper >>>>> >>>>> static long millisElapsedSince(long startNanoTime) { >>>>> return NANOSECONDS.toMillis(System.nanoTime() - >>>>> startNanoTime); >>>>> } >>>>> >>>>> >>>>> + System.out.printf(" waitFor process: delta: %d%n",(end - >>>>> start) ); >>>>> + >>>>> if ((end - start) > 200000000L * (AIX.is() ? 2 : 1)) >>>>> fail("Test failed: waitFor took too long (" + (end - >>>>> start) + "ns)"); >>>>> >>>>> 200 ms timeout for subprocesses to finish is just too damn low. In >>>>> j.u.c. tests we switched to 10 seconds for most "long" timeouts >>>>> (LONG_DELAY_MS) and are happy with the disappearance of rare flaky >>>>> results. >>>>> >>>>> >>>>> On Wed, Nov 12, 2014 at 11:34 AM, roger riggs >>>>> wrote: >>>>>> >>>>>> Please review test changes to ProcessBuilder Basic.java to add some >>>>>> debugging information. >>>>>> The test has been failing intermittently. The wait times have been >>>>>> extended >>>>>> to see >>>>>> if the systems are just slow. The failure criteria have not changed. >>>>>> >>>>>> Suggestions welcome. >>>>>> >>>>>> Webrev: >>>>>> http://cr.openjdk.java.net/~rriggs/webrev-basic-debug-8043477/ >>>>>> >>>>>> issue: >>>>>> 8043477: java/lang/ProcessBuilder/Basic.java failed with: >>>>>> java.lang.AssertionError: Some tests failed >>>>>> >>>>>> Thanks, Roger >>>>>> >>>>>> [1] https://bugs.openjdk.java.net/browse/JDK-8043477 >>>>>> > From vladimir.kozlov at oracle.com Thu Nov 13 00:14:05 2014 From: vladimir.kozlov at oracle.com (Vladimir Kozlov) Date: Wed, 12 Nov 2014 16:14:05 -0800 Subject: RFR: AARCH64: 8064594: Top-level JDK changes In-Reply-To: References: Message-ID: <5463F7CD.5080002@oracle.com> Oh. I just replied to the wrong email. Anyway, here it goes again: Maybe we should CC sound-dev (if that?s the correct list)? The new jvm.cfg files should only have a copyright year of 2014. Otherwise this looks good. > On Nov 12, 2014, at 3:48 AM, Andrew Haley wrote: > > The changes for the /jdk subdirectory. > > The missing files problem bit me again. > > New webrev: http://cr.openjdk.java.net/~aph/aarch64-JDK-8064594-1/ > > Apologies, > Andrew. From otaviojava at java.net Thu Nov 13 00:57:44 2014 From: otaviojava at java.net (=?UTF-8?Q?Ot=C3=A1vio_Gon=C3=A7alves_de_Santana?=) Date: Wed, 12 Nov 2014 22:57:44 -0200 Subject: Review request: JDK-8055723 Replace concat String to append in StringBuilder parameters In-Reply-To: <5463382D.20201@CoSoCo.de> References: <5445A836.8090005@CoSoCo.de> <545E5703.7060902@CoSoCo.de> <5463382D.20201@CoSoCo.de> Message-ID: But this class is an Exception, doesn't make sense an exception get another Exception. IMHO: I prefer this way On Wed, Nov 12, 2014 at 8:36 AM, Ulf Zibis wrote: > Hi Ot?vio, > I now think you could replace > if (!expected.isEmpty()) > with > assert !expected.isEmpty(); > > If expected ever would be empty, the only thing which happens is, that a > "'" is missing in a message which anyway doesn't make sense without > arguments. > > -Ulf > > > > Am 12.11.2014 um 08:19 schrieb Ot?vio Gon?alves de Santana: > >> Thank you Ulf. >> http://cr.openjdk.java.net/~weijun/8055723/webrev.01/ < >> http://cr.openjdk.java.net/%7Eweijun/8055723/webrev.01/> >> >> >> On Sat, Nov 8, 2014 at 3:46 PM, Ulf Zibis > Ulf.Zibis at cosoco.de>> wrote: >> >> Hi Ot?vio, >> in sun/tools/jstat/SyntaxException.java I see a possible enhencement >> (maybe applies to other >> places too): >> >> 65 public SyntaxException(int lineno, Set expected, >> Token found) { >> 66 StringBuilder msg = new StringBuilder(A + B * >> expected.size()); >> 67 >> 68 msg.append("Syntax error at line >> ").append(lineno).append(": Expected one of \'"); >> 69 >> 71 for (String keyWord : expected) { >> 72 msg.append(keyWord).append('|'); >> 73 } >> 74 // if (!expected.isEmpty()) // only needed if expected >> may be empty. >> 75 msg.setLength(msg.length() - 1); >> 76 >> 81 message = msg.append("\', Found >> ").append(found.toMessage()).toString(); >> 83 } >> >> ***** Additionally at many places you could similarly introduce the >> foreach syntax. >> >> -Ulf >> >> >> Am 02.11.2014 um 15:45 schrieb Ot?vio Gon?alves de Santana: >> >> Could another reviewer look these codes, please. >> http://cr.openjdk.java.net/~weijun/8055723/webrev.00/ >> >> >> >> On Fri, Oct 24, 2014 at 3:25 AM, Ot?vio Gon?alves de Santana < >> otaviojava at java.net >> > otaviojava at java.net>>> wrote: >> >> Thank you Ulf. >> I removed the fix in toString method and in debug classes: >> http://cr.openjdk.java.net/~weijun/8055723/webrev.00/ >> >> >> >> On Mon, Oct 20, 2014 at 10:26 PM, Ulf Zibis < >> Ulf.Zibis at cosoco.de >> > Ulf.Zibis at cosoco.de>>> >> wrote: >> >> >> Am 21.10.2014 um 01:02 schrieb Ot?vio Gon?alves de >> Santana: >> >> BUGURL: https://bugs.openjdk.java.net/ >> browse/JDK-8055723 >> >> >> WEBREV: http://cr.openjdk.java.net/~ >> weijun/8055723/client/webrev.02/ >> >> > webrev.02/> >> WEBREV: http://cr.openjdk.java.net/~ >> weijun/8055723/core/webrev.03/ >> >> > 7Eweijun/8055723/core/webrev.03/> >> >> >> I did not look through all sources. >> In Scanner.java I discovered: >> 1307 sb.append("[delimiters="). >> append(delimPattern).append(']'); >> 1308 sb.append("[position="). >> append(position).append(']'); >> ... >> Maybe better: >> 1307 sb.append("[delimiters=").append(delimPattern); >> 1308 sb.append("][position=").append(position); >> ... >> >> -Ulf >> >> >> >> >> -- Ot?vio Gon?alves de Santana >> >> blog: http://otaviosantana.blogspot.com.br/ >> twitter: http://twitter.com/otaviojava >> site: _http://about.me/otaviojava_ >> 55 (11) 98255-3513 >> >> >> >> >> >> -- Ot?vio Gon?alves de Santana >> >> blog: http://otaviosantana.blogspot.com.br/ >> twitter: http://twitter.com/otaviojava >> site: _http://about.me/otaviojava_ >> 55 (11) 98255-3513 >> >> >> >> >> >> >> -- >> Ot?vio Gon?alves de Santana >> >> blog: http://otaviosantana.blogspot.com.br/ >> twitter: http://twitter.com/otaviojava >> site: _http://about.me/otaviojava_ >> 55 (11) 98255-3513 >> > > -- Ot?vio Gon?alves de Santana blog: http://otaviosantana.blogspot.com.br/ twitter: http://twitter.com/otaviojava site: *http://about.me/otaviojava * 55 (11) 98255-3513 From weijun.wang at oracle.com Thu Nov 13 01:19:31 2014 From: weijun.wang at oracle.com (Wang Weijun) Date: Thu, 13 Nov 2014 09:19:31 +0800 Subject: Review request: JDK-8055723 Replace concat String to append in StringBuilder parameters In-Reply-To: References: <5445A836.8090005@CoSoCo.de> <545E5703.7060902@CoSoCo.de> <5463382D.20201@CoSoCo.de> Message-ID: <73B0315D-C66A-4133-A26A-2A71FAD654DE@oracle.com> I hope we can restrict the code change to what the bug description is about. IMHO this bug should only include cleanup and introduce no obvious behavior change. Any other fix can go to another bug. --Max > On Nov 13, 2014, at 08:57, Ot?vio Gon?alves de Santana wrote: > > But this class is an Exception, doesn't make sense an exception get another > Exception. > IMHO: I prefer this way > > On Wed, Nov 12, 2014 at 8:36 AM, Ulf Zibis wrote: > >> Hi Ot?vio, >> I now think you could replace >> if (!expected.isEmpty()) >> with >> assert !expected.isEmpty(); >> >> If expected ever would be empty, the only thing which happens is, that a >> "'" is missing in a message which anyway doesn't make sense without >> arguments. >> >> -Ulf >> >> ...... From otaviojava at java.net Thu Nov 13 01:33:01 2014 From: otaviojava at java.net (=?UTF-8?Q?Ot=C3=A1vio_Gon=C3=A7alves_de_Santana?=) Date: Wed, 12 Nov 2014 23:33:01 -0200 Subject: Review request: JDK-8055723 Replace concat String to append in StringBuilder parameters In-Reply-To: <73B0315D-C66A-4133-A26A-2A71FAD654DE@oracle.com> References: <5445A836.8090005@CoSoCo.de> <545E5703.7060902@CoSoCo.de> <5463382D.20201@CoSoCo.de> <73B0315D-C66A-4133-A26A-2A71FAD654DE@oracle.com> Message-ID: Ok, you're right. On Wed, Nov 12, 2014 at 11:19 PM, Wang Weijun wrote: > I hope we can restrict the code change to what the bug description is > about. IMHO this bug should only include cleanup and introduce no obvious > behavior change. > > Any other fix can go to another bug. > > --Max > > > On Nov 13, 2014, at 08:57, Ot?vio Gon?alves de Santana < > otaviojava at java.net> wrote: > > > > But this class is an Exception, doesn't make sense an exception get > another > > Exception. > > IMHO: I prefer this way > > > > On Wed, Nov 12, 2014 at 8:36 AM, Ulf Zibis wrote: > > > >> Hi Ot?vio, > >> I now think you could replace > >> if (!expected.isEmpty()) > >> with > >> assert !expected.isEmpty(); > >> > >> If expected ever would be empty, the only thing which happens is, that a > >> "'" is missing in a message which anyway doesn't make sense without > >> arguments. > >> > >> -Ulf > >> > >> > ...... -- Ot?vio Gon?alves de Santana blog: http://otaviosantana.blogspot.com.br/ twitter: http://twitter.com/otaviojava site: *http://about.me/otaviojava * 55 (11) 98255-3513 From roger.riggs at oracle.com Thu Nov 13 02:25:23 2014 From: roger.riggs at oracle.com (roger riggs) Date: Wed, 12 Nov 2014 21:25:23 -0500 Subject: RFR 9 8043477: java/lang/ProcessBuilder/Basic.java failed with: java.lang.AssertionError: Some tests failed In-Reply-To: References: <5463B643.4050301@oracle.com> <5463BCCE.9070903@oracle.com> <5463C010.7000606@oracle.com> <5463D7AD.9080600@oracle.com> Message-ID: <54641693.8020700@oracle.com> Hi Martin, I think that sleep is the version that spawns a JavaChild process and has its own comment interpreter. See line 309: Am I getting the sleeps mixed up? Roger On 11/12/2014 6:32 PM, Martin Buchholz wrote: > Hi Roger, > > > > On Wed, Nov 12, 2014 at 1:57 PM, roger riggs wrote: >> Hi Martin, >> >> I updated the webrev to use long timeouts for the test in question; they >> can't be too long >> because the spawned sleep is only alive for 10 seconds. > I am looking at the "sleep" command invocation, but it's an indefinite > sleep. I don't see 10 seconds anywhere. > >> Take a look please: >> >> http://cr.openjdk.java.net/~rriggs/webrev-basic-debug-8043477/ > 2313 Integer exitValue = null; > 2314 try { > 2315 exitValue = p.exitValue(); > 2316 } catch (IllegalStateException ise) { > 2317 // no exitValue > 2318 } > 2319 if ((end - start) > 6 * 1_000_000_000) > 2320 fail("Test failed: waitFor took too long on a > dead process. (" + (end - start) + "ns)" > 2321 + ", exitValue: " + Objects.requireNonNull(exitValue)); > > exitValue throws IllegalThreadStateException, not IllegalStateException. > > I would fail separately in case of ITSE. It looks instead like you'll > fail with uninformative NPE. > > 2301 p.waitFor(1000, TimeUnit.MILLISECONDS); > > 2304 if ((end - start) < 500_000_000) > > > I am surprised the test is not testing that the initial timed waitFor > doesn't wait at least the specified time, instead of only half. I > consider returning from waitFor early a bug. > > >> Thanks, Roger >> >> >> On 11/12/2014 4:45 PM, Martin Buchholz wrote: >>> Looking over the test again (I was the original author IIRC), >>> I think this test still has lots of value with a longer timeout. >>> >>> Experience seems to show that 200ms is not a long enough timeout for >>> "system" operations to use in non-flaky tests, including everything in >>> Process handling. >>> I say we ncrease the timeout by at least 10x and move on. >>> >>> On Wed, Nov 12, 2014 at 12:16 PM, roger riggs >>> wrote: >>>> Hi Martin, Rob, >>>> >>>> I thought the point of the test was to verify the timeout logic in >>>> waitFor(timeout). >>>> Adding the prints without changing the test criteria was intended to >>>> gather >>>> data >>>> about the distribution. >>>> >>>> A long timeout would still catch a failure in the reaper/waitFor >>>> communication >>>> so maybe that's ok. >>>> >>>> As for printing the time, I would switch to the newer >>>> java.time.Instant/Duration >>>> which have a decent toString. >>>> >>>> Roger >>>> >>>> >>>> >>>> On 11/12/2014 3:02 PM, Rob McKenna wrote: >>>>> Eesh, Sorry Roger, I have something like this on my todo. >>>>> >>>>> Martin, my concern with the long delay approach is that it effectively >>>>> nullifies the point of the test. Given that this test has been flakey >>>>> the >>>>> approach has been to simply bump the acceptable delta by another 100ms >>>>> or so >>>>> every time. Since we've had to do this repeatedly however, I'm beginning >>>>> to >>>>> question whether the test is more trouble than its worth. >>>>> >>>>> -Rob >>>>> >>>>> On 12/11/14 19:44, Martin Buchholz wrote: >>>>>> The print statement below seems redundant with the assertion failure >>>>>> message. >>>>>> You could improve the assertion message instead if need be. >>>>>> Adding thousand separator underscores to 200000000L would help a >>>>>> little. >>>>>> >>>>>> I like my little helper >>>>>> >>>>>> static long millisElapsedSince(long startNanoTime) { >>>>>> return NANOSECONDS.toMillis(System.nanoTime() - >>>>>> startNanoTime); >>>>>> } >>>>>> >>>>>> >>>>>> + System.out.printf(" waitFor process: delta: %d%n",(end - >>>>>> start) ); >>>>>> + >>>>>> if ((end - start) > 200000000L * (AIX.is() ? 2 : 1)) >>>>>> fail("Test failed: waitFor took too long (" + (end - >>>>>> start) + "ns)"); >>>>>> >>>>>> 200 ms timeout for subprocesses to finish is just too damn low. In >>>>>> j.u.c. tests we switched to 10 seconds for most "long" timeouts >>>>>> (LONG_DELAY_MS) and are happy with the disappearance of rare flaky >>>>>> results. >>>>>> >>>>>> >>>>>> On Wed, Nov 12, 2014 at 11:34 AM, roger riggs >>>>>> wrote: >>>>>>> Please review test changes to ProcessBuilder Basic.java to add some >>>>>>> debugging information. >>>>>>> The test has been failing intermittently. The wait times have been >>>>>>> extended >>>>>>> to see >>>>>>> if the systems are just slow. The failure criteria have not changed. >>>>>>> >>>>>>> Suggestions welcome. >>>>>>> >>>>>>> Webrev: >>>>>>> http://cr.openjdk.java.net/~rriggs/webrev-basic-debug-8043477/ >>>>>>> >>>>>>> issue: >>>>>>> 8043477: java/lang/ProcessBuilder/Basic.java failed with: >>>>>>> java.lang.AssertionError: Some tests failed >>>>>>> >>>>>>> Thanks, Roger >>>>>>> >>>>>>> [1] https://bugs.openjdk.java.net/browse/JDK-8043477 >>>>>>> From martinrb at google.com Thu Nov 13 03:42:24 2014 From: martinrb at google.com (Martin Buchholz) Date: Wed, 12 Nov 2014 19:42:24 -0800 Subject: RFR 9 8043477: java/lang/ProcessBuilder/Basic.java failed with: java.lang.AssertionError: Some tests failed In-Reply-To: <54641693.8020700@oracle.com> References: <5463B643.4050301@oracle.com> <5463BCCE.9070903@oracle.com> <5463C010.7000606@oracle.com> <5463D7AD.9080600@oracle.com> <54641693.8020700@oracle.com> Message-ID: On Wed, Nov 12, 2014 at 6:25 PM, roger riggs wrote: > Hi Martin, > > I think that sleep is the version that spawns a JavaChild process and > has its own comment interpreter. See line 309: Am I getting the sleeps mixed > up? Ah, good point - yes, you're right. Although we could always bump up the sleep of the JavaChild from 10 if we wanted to - it should be comfortably bigger than any individual test case's need. From peter.levart at gmail.com Thu Nov 13 08:24:53 2014 From: peter.levart at gmail.com (Peter Levart) Date: Thu, 13 Nov 2014 09:24:53 +0100 Subject: [9] RFR(L) 8013267 : move MemberNameTable from native code to Java heap, use to intern MemberNames In-Reply-To: References: <594FE416-D445-426E-B7A9-C75F012ADE16@oracle.com> <0BDCD5DD-CBFB-4A3D-9BAD-7F9912E9237C@oracle.com> <5453C230.8010709@oracle.com> <9747A3A3-B7F4-407A-8E00-1E647D9DC2D1@oracle.com> <1D195ED3-8BD2-46CB-9D70-29CF809D9F5A@oracle.com> <5456FB59.60905@oracle.com> <632A5C98-B386-4625-BE12-355241581955@oracle.com> <5457AA75.8090103@gmail.com> <5457E0F9.8090004@gmail.com> <5458A57C.4060208@gmail.com> <260D49F5-6380-4FC3-A900-6CD9AB3ED6F7@oracle.com> <5459034E.8070809@gmail.com> <39826508-110B-4FCE-9A58-8C3D1B9FC7DE@oracle.com> <545F642E.30205@gmail.com> Message-ID: <54646AD5.4000404@gmail.com> On 11/12/2014 07:27 PM, David Chase wrote: > Hello Peter, > >> Sadly, this seems not to be the case for MemberNames or for ?Types?. > That statement is inoperative. Mistakes were made. > It?s compareTo that they lack. Yes, I say your quite tricky implementation of MemberName.compareTo, based on hashCode(s), String representations, etc... The hash-table based interning does not need it though. Regards, Peter > David > > > On 2014-11-09, at 7:55 AM, Peter Levart wrote: > >> Hi David, >> >> I played a little with the idea of having a hash table instead of packed sorted array for interning. Using ConcurrentHashMap would present quite some memory overhead. A more compact representation is possible in the form of a linear-scan hash table where elements of array are MemberNames themselves: >> >> http://cr.openjdk.java.net/~plevart/misc/MemberName.intern/jdk.06.diff/ >> >> This is a drop-in replacement for MemberName on top of your jdk.06 patch. If you have some time, you can run this with your performance tests to see if it presents any difference. If not, then perhaps this interning is not so performance critical after all. >> >> Regards, Peter From joel.franck at oracle.com Thu Nov 13 09:39:58 2014 From: joel.franck at oracle.com (=?windows-1252?Q?Joel_Borggr=E9n-Franck?=) Date: Thu, 13 Nov 2014 10:39:58 +0100 Subject: RFR: 8061950: Class.getMethods() exhibits quadratic time complexity In-Reply-To: <545A572D.3050608@gmail.com> References: <900D2AE3-B3A6-4B1B-B25A-C564B10334AA@oracle.com> <54492255.3040108@gmail.com> <544D58D4.7020909@gmail.com> <544D7B74.5060607@gmail.com> <5450CEE9.7060707@gmail.com> <545120D2.5010301@gmail.com> <54526D10.5060101@gmail.com> <54551D2F.407@gmail.com> <545A572D.3050608@gmail.com> Message-ID: Hi Peter, As always, thanks for taking a look at this, This is quite big so in order to make this more approachable perhaps you can split the patch up into a series? If you start with creating the MethodTable interface, adding tests for how the interface should behave and refactored the current MethodArray into implementing that interface while also changing the lookup logic that would be easier to review. Then you could add different implementations of MethodTable (with additional unit tests) as follow up patches. I am a bit concerned about the size and scope of the implementations. In general I would prefer if you targeted these to the precise need of core reflection today. If you want to expand these to general purpose data structures (even internal ones) I think that is a larger effort. In general I think the changes to Class are sound, but there is a slight change in the default method pruning. The call to removeLessSpecifics was deliberately placed at the end, so that all default methods would be present (the algorithm is sensitive to the order of pair vise comparisons). Since we add methods in a deterministic order, I think consolidate() as you go should result in the same set of methods, but I haven?t 100% convinced myself of this just yet. Have you double checked that all methods returning root Method/Ctors are private? On 5 nov 2014, at 17:58, Peter Levart wrote: > Here's new webrev: > > http://cr.openjdk.java.net/~plevart/jdk9-dev/Class.getMethods/webrev.06/ > > > The optimizations made from webrev.05 are: > > - getMethod() skips construction of MethodTable if there are no (super)interfaces. > - getMethods() returns just declared public methods if there are no superclass and no (super)interfaces. > - comparing method parameter types is optimized by adding two methods to Method/LangReflectAccess/ReflectionFactory. > > New MethodTable implementation based on a linear-probe hash table is a space/garbage improvement. I took IdentityHashMap, removed unneeded stuff and modified it's API. The result is a HashArray. It's API is similar in function and form to java.util.Map, but doesn't use separate keys and values. An element of HashArray is a key and a value at the same time. Elements are always non-null, so the method return values are unambiguous. As HashArray is a linear-probe hash table and there are no Map.Entry objects involved, the underlying data structure is very simple and memory efficient. It is just a sparse array of elements with length that is always a power of two and larger than 3 * size / 2. It also features overriddable element equals/hashCode methods. I made it a separate generic class because I think it can find it's usage elsewhere (for example as a cannonicalizing cache). > > Since HashArray based MethodTable is more space-efficient I moved the line between simple array based and HashArray based MethodTable down to 20 elements to minimize the worst-case scenario effect. Calling getMethods() on all rt.jar classes now constructs about 3/4 simple array based and 1/4 HashArray based MethodTables. > HashArray.java: I was hoping for a decent set of unit tests for the new HashArray data structure. I think it is reasonable to test the corner cases/non-trivial areas of the table (closeDeletion(), rezise() etc). Perhaps also run these over the simple implementation. Also, please document thread safety (there is none IFAICT it should just be noted). Instead of using inheritance to change the behavior of equals() and hash() you could give it two lambdas at table creation time, a ToIntFunction for hash() and a BiPredicate for equals(). Might not give you the performance we need though. Note that the file doesn?t actually compile in jdk9/dev, you have two unchecked casts and we build with -Werror. MethodTable.java HashMapImpl is missing serialVersionUID, but it looks like this class won?t be needed at all. > Here's also Martin's ManyMethodsBenchmark: > > Original: > > Base class load time: 129.95 ms > getDeclaredMethods: 65521 methods, 36.58 ms total time, 0.0006 ms per method > getMethods : 65530 methods, 47.43 ms total time, 0.0007 ms per method > Derived class load time: 32216.09 ms > getDeclaredMethods: 65521 methods, 35.05 ms total time, 0.0005 ms per method > getMethods : 65530 methods, 8068.66 ms total time, 0.1231 ms per method > > > Patched (using HashArray based MethodTable): > > Base class load time: 126.00 ms > getDeclaredMethods: 65521 methods, 36.83 ms total time, 0.0006 ms per method > getMethods : 65530 methods, 45.08 ms total time, 0.0007 ms per method > Derived class load time: 31865.27 ms > getDeclaredMethods: 65521 methods, 35.01 ms total time, 0.0005 ms per method > getMethods : 65530 methods, 78.05 ms total time, 0.0012 ms per method > > > All 86 jtreg test in java.lang/Class/ and java/lang/reflect/ still pass. > I have seen discussion about allocation, should we measure and compare? You can probably use the Hotspot impl of ThreadMXBean to get the allocation in the tread. Also, it might be time to fix the boolean parameters: 2741 Method[] declaredMethods = privateGetDeclaredMethods(true); 2742 Class superclass = getSuperclass(); 2743 Class[] interfaces = getInterfaces(false); Perhaps just add boolean constants somewhere so that it is easier to decode. 2741 Method[] declaredMethods = privateGetDeclaredMethods(PUBLIC_METHOD_ONLY); 2742 Class superclass = getSuperclass(); 2743 Class[] interfaces = getInterfaces(NO_COPY_RESULT); or so. HashArray.java: 155 if (lookupObj == null) throw new NullPointerException(); use Objects.requreNonNull() ? cheers /Joel From frank.yuan at oracle.com Thu Nov 13 11:02:57 2014 From: frank.yuan at oracle.com (Frank Yuan) Date: Thu, 13 Nov 2014 19:02:57 +0800 Subject: Review request for XML JAXP unit test colocation In-Reply-To: <013701cff8d8$9acf3950$d06dabf0$@oracle.com> References: <013701cff8d8$9acf3950$d06dabf0$@oracle.com> Message-ID: <009601cfff31$631da000$2958e000$@oracle.com> Hi Joe and All I revised the code based on latest comments and put the webrev on http://cr.openjdk.java.net/~joehw/jdk9/test/Frank/8043090/webrev/ Best Regards Frank -----Original Message----- From: Frank Yuan [mailto:frank.yuan at oracle.com] Sent: Wednesday, November 05, 2014 5:12 PM To: 'huizhe wang'; 'Core-Libs-Dev' Cc: 'jibing chen'; 'Gustavo Galimberti'; sandeep.konchady at oracle.com Subject: Review request for XML JAXP unit test colocation Hi, Joe and All Thanks for your previous reviews and comments. Per your comments: I added description for every test, got them running with and without security manager and fixed the known test issues. Could you help review the changes for JAXP unittest co-location again? After your review, I will push the tests into JDK repo: jaxp/test. bug: https://bugs.openjdk.java.net/browse/JDK-8043090 webrev: Thanks, Frank From konstantin.shefov at oracle.com Thu Nov 13 12:48:16 2014 From: konstantin.shefov at oracle.com (Konstantin Shefov) Date: Thu, 13 Nov 2014 15:48:16 +0300 Subject: [9] Review request : JDK-8059070: [TESTBUG] java/lang/invoke/LFCaching/LFMultiThreadCachingTest.java failed - timeout In-Reply-To: <5460CF90.1050303@oracle.com> References: <543D1DEE.8030206@oracle.com> <543F8520.9090408@oracle.com> <8F9DD322-1793-4739-8155-61CE7D2C5AAB@oracle.com> <543F9800.7040406@oracle.com> <543FC71F.8020504@oracle.com> <5440E39C.8070001@oracle.com> <5448E5A2.9030804@oracle.com> <544E1B65.1090400@oracle.com> <5450F8E4.2060808@oracle.com> <545C9D70.6060308@oracle.com> <5460A29E.6070206@oracle.com> <5460CF90.1050303@oracle.com> Message-ID: <5464A890.6080404@oracle.com> Kindly reminder. On 10.11.2014 17:45, Konstantin Shefov wrote: > Vladimir, thanks for reviewing > > I have updated the webrev: > http://cr.openjdk.java.net/~kshefov/8059070/webrev.02 > I have added DEFAULT_TEST_TIMEOUT constant to Utils class. > > -Konstantin > > On 10.11.2014 14:33, Vladimir Ivanov wrote: >> Kontantin, sorry for the late response. >> >> In general, the fix looks good. >> >> I'd move timeout computation logic to Utils: >> + private static final long TIMEOUT = >> Utils.adjustTimeout(TimeUnit.SECONDS.toMillis(120)); >> >> Default value (120s) isn't specific to the tests, but an >> implementation detail of jtreg. >> >> Best regards, >> Vladimir Ivanov >> >> On 11/7/14, 2:22 PM, Konstantin Shefov wrote: >>> Gently reminder >>> >>> 29.10.2014 17:25, Konstantin Shefov ?????: >>>> Please, review a test bug fix. >>>> http://cr.openjdk.java.net/~kshefov/8059070/webrev.01/ >>>> >>>> -Konstantin >>>> >>>> On 27.10.2014 13:16, Konstantin Shefov wrote: >>>>> Kindly reminder >>>>> >>>>> On 23.10.2014 19:04, Paul Sandoz wrote: >>>>>> On Oct 23, 2014, at 1:25 PM, Konstantin Shefov >>>>>> wrote: >>>>>>> Gently reminder >>>>>>> >>>>>>> On 17.10.2014 13:38, Konstantin Shefov wrote: >>>>>>>> Hi, >>>>>>>> >>>>>>>> I have updated the webrev: >>>>>>>> http://cr.openjdk.java.net/~kshefov/8059070/webrev.01/ >>>>>>>> >>>>>> +1 >>>>>> >>>>>> Sorry for the delay, >>>>>> Paul. >>>>> >>>> >>> > From joel.franck at oracle.com Thu Nov 13 13:55:18 2014 From: joel.franck at oracle.com (Joel =?utf-8?Q?Borggr=C3=A9n-Franck?=) Date: Thu, 13 Nov 2014 14:55:18 +0100 Subject: RFR: 8062771: Core reflection should use final fields whenever possible In-Reply-To: References: <843282E1-DC2B-48B0-B0E4-A60B5078D75A@oracle.com> <45A50B54-43A3-4134-9EB5-D1AEDA649587@oracle.com> Message-ID: <20141113135518.GB16532@oracle.com> Hi Martin, Since you have already provided us with the patch here: http://cr.openjdk.java.net/~martin/webrevs/openjdk8/Class-thread-safety/ Lets do it the other way around. I think this is a very good fix for 8u, reviewed. I'll start the backporting approval with you as the provider of the fix and me as reviewer. cheers /Joel On 2014-11-07, Martin Buchholz wrote: > Hi Joel, > > Thanks for volunteering. I foisted all I have in > > https://bugs.openjdk.java.net/browse/JDK-8064391 > > I volunteer to be your reviewer for the backports. > > On Fri, Nov 7, 2014 at 1:36 PM, Joel Borggr?n-Franck > wrote: > > Hi Martin, > > > > Thanks for the clarification. > > > > On 6 nov 2014, at 20:51, Martin Buchholz wrote: > > > >> Hi Joel, > >> > >> On Thu, Nov 6, 2014 at 2:48 AM, Joel Borggr?n-Franck > >> wrote: > >>> Hi, > >>> > >>> I?m having a hard time following this thread, which webrev has been updated, for which release, fixing which issue? > >> > >> I am submitting the changeset for JDK-8062771, adding the finals and > >> the (failed reproduction) test. > >> Hopefully there will be followon changes to add even more thread safety. > >> > > > > Great. > > > > Are we sure the accidental fix of making ClassRepository volatile is good on all platforms? I can?t establish the happens before relations that would prove the fix but then again I?m not an expert on concurrency. > > > >>> Martin, as far as I can see you are the only one of us who has replied to this thread who is a jdk7u committer (or reviewer). > >> > >> I am entirely in favor of having more reviewers. > >> > >> We are not planning to push these into jdk7u or jdk8u ourselves, but > >> we are willing to help anyone who wants to take on that task. Joel, > >> are you volunteering? > >> We believe all versions of stock openjdk still have (rarely seen) > >> thread safety issues in core reflection. We have applied changes > >> locally at Google to fix those. > >> > > > > I?ll take care of 8u, and I can hunt down someone to fix this in 7u if we can find a suitable reviewer (sigh). > > > > cheers > > /Joel > > > > From joel.franck at oracle.com Thu Nov 13 14:35:47 2014 From: joel.franck at oracle.com (=?windows-1252?Q?Joel_Borggr=E9n-Franck?=) Date: Thu, 13 Nov 2014 15:35:47 +0100 Subject: RFR: 8062771: Core reflection should use final fields whenever possible In-Reply-To: <5460E414.7060004@gmail.com> References: <843282E1-DC2B-48B0-B0E4-A60B5078D75A@oracle.com> <45A50B54-43A3-4134-9EB5-D1AEDA649587@oracle.com> <5460E414.7060004@gmail.com> Message-ID: Hi Peter, Yes, please file a separate issue and a RFR. cheers /Joel On 10 nov 2014, at 17:13, Peter Levart wrote: > On 11/07/2014 11:48 PM, Martin Buchholz wrote: >> Hi Joel, >> >> Thanks for volunteering. I foisted all I have in >> >> https://bugs.openjdk.java.net/browse/JDK-8064391 >> >> I volunteer to be your reviewer for the backports. > > Hi Martin, > > Sorry I haven't checked this earlier, but there are still some data races left-behind: > > http://cr.openjdk.java.net/~plevart/jdk9-dev/GenericsReflectionRaces/webrev.01/ > > Mainly the fact that lazy initialization uses arrays which are published unsafely. > > Would this need a separate issue. For example: "Core reflection should use volatile fields whenever necessary" ? > > > Regards, Peter > >> >> On Fri, Nov 7, 2014 at 1:36 PM, Joel Borggr?n-Franck >> wrote: >>> Hi Martin, >>> >>> Thanks for the clarification. >>> >>> On 6 nov 2014, at 20:51, Martin Buchholz wrote: >>> >>>> Hi Joel, >>>> >>>> On Thu, Nov 6, 2014 at 2:48 AM, Joel Borggr?n-Franck >>>> wrote: >>>>> Hi, >>>>> >>>>> I?m having a hard time following this thread, which webrev has been updated, for which release, fixing which issue? >>>> I am submitting the changeset for JDK-8062771, adding the finals and >>>> the (failed reproduction) test. >>>> Hopefully there will be followon changes to add even more thread safety. >>>> >>> Great. >>> >>> Are we sure the accidental fix of making ClassRepository volatile is good on all platforms? I can?t establish the happens before relations that would prove the fix but then again I?m not an expert on concurrency. >>> >>>>> Martin, as far as I can see you are the only one of us who has replied to this thread who is a jdk7u committer (or reviewer). >>>> I am entirely in favor of having more reviewers. >>>> >>>> We are not planning to push these into jdk7u or jdk8u ourselves, but >>>> we are willing to help anyone who wants to take on that task. Joel, >>>> are you volunteering? >>>> We believe all versions of stock openjdk still have (rarely seen) >>>> thread safety issues in core reflection. We have applied changes >>>> locally at Google to fix those. >>>> >>> I?ll take care of 8u, and I can hunt down someone to fix this in 7u if we can find a suitable reviewer (sigh). >>> >>> cheers >>> /Joel >>> >>> > From roger.riggs at oracle.com Thu Nov 13 15:37:00 2014 From: roger.riggs at oracle.com (roger riggs) Date: Thu, 13 Nov 2014 10:37:00 -0500 Subject: RFR 9 8043477: java/lang/ProcessBuilder/Basic.java failed with: java.lang.AssertionError: Some tests failed In-Reply-To: References: <5463B643.4050301@oracle.com> <5463BCCE.9070903@oracle.com> <5463C010.7000606@oracle.com> <5463D7AD.9080600@oracle.com> <54641693.8020700@oracle.com> Message-ID: <5464D01C.7050600@oracle.com> Please re-review, Corrected the webrev, and a few changes to consistently use the same units, thanks for the review: The timeout time is extended to wait up to 7 seconds and the initial waitFor should last at least 1 second. http://cr.openjdk.java.net/~rriggs/webrev-basic-debug-8043477/ Roger On 11/12/2014 10:42 PM, Martin Buchholz wrote: > On Wed, Nov 12, 2014 at 6:25 PM, roger riggs wrote: >> Hi Martin, >> >> I think that sleep is the version that spawns a JavaChild process and >> has its own comment interpreter. See line 309: Am I getting the sleeps mixed >> up? > Ah, good point - yes, you're right. Although we could always bump up > the sleep of the JavaChild from 10 if we wanted to - it should be > comfortably bigger than any individual test case's need. From martinrb at google.com Thu Nov 13 16:47:16 2014 From: martinrb at google.com (Martin Buchholz) Date: Thu, 13 Nov 2014 08:47:16 -0800 Subject: RFR 9 8043477: java/lang/ProcessBuilder/Basic.java failed with: java.lang.AssertionError: Some tests failed In-Reply-To: <5464D01C.7050600@oracle.com> References: <5463B643.4050301@oracle.com> <5463BCCE.9070903@oracle.com> <5463C010.7000606@oracle.com> <5463D7AD.9080600@oracle.com> <54641693.8020700@oracle.com> <5464D01C.7050600@oracle.com> Message-ID: You should delete your import 36 import java.lang.IllegalStateException; --- I would bump up the time you're willing to wait here, by at least 10x. 2277 if ((end - start) > 200000000L * (AIX.is() ? 2 : 1)) 2278 fail("Test failed: waitFor took too long (" + (end - start) + "ns)"); --- When you are surely waiting, it's best to optimize by waiting as little as possible (although jtreg concurrency helps) 2301 p.waitFor(1, TimeUnit.SECONDS); I would set the timeout to only 10 millis here, and check that the operation took at least 10 millis and no more than your long timeout (you've settled on 7 seconds?) as in jsr166 tck tests --- 2315 exitValue = p.exitValue(); 2316 } catch (IllegalThreadStateException ise) { 2317 // no exitValue 2318 } If ITSE exception is thrown by exitValue, then the test should always fail but that is not currently the case. If the test does fail, it currently throws uninformative NPE in bjects.requireNonNull(exitValue), and we can do better. On Thu, Nov 13, 2014 at 7:37 AM, roger riggs wrote: > Please re-review, > > Corrected the webrev, and a few changes to consistently use the same > units, thanks for the review: > > The timeout time is extended to wait up to 7 seconds > and the initial waitFor should last at least 1 second. > > http://cr.openjdk.java.net/~rriggs/webrev-basic-debug-8043477/ > > Roger > > > > > On 11/12/2014 10:42 PM, Martin Buchholz wrote: >> >> On Wed, Nov 12, 2014 at 6:25 PM, roger riggs >> wrote: >>> >>> Hi Martin, >>> >>> I think that sleep is the version that spawns a JavaChild process and >>> has its own comment interpreter. See line 309: Am I getting the sleeps >>> mixed >>> up? >> >> Ah, good point - yes, you're right. Although we could always bump up >> the sleep of the JavaChild from 10 if we wanted to - it should be >> comfortably bigger than any individual test case's need. > > From roger.riggs at oracle.com Thu Nov 13 17:07:00 2014 From: roger.riggs at oracle.com (roger riggs) Date: Thu, 13 Nov 2014 12:07:00 -0500 Subject: RFR 9 8043477: java/lang/ProcessBuilder/Basic.java failed with: java.lang.AssertionError: Some tests failed In-Reply-To: References: <5463B643.4050301@oracle.com> <5463BCCE.9070903@oracle.com> <5463C010.7000606@oracle.com> <5463D7AD.9080600@oracle.com> <54641693.8020700@oracle.com> <5464D01C.7050600@oracle.com> Message-ID: <5464E534.6070804@oracle.com> Hi, Thank you for you patience. On 11/13/2014 11:47 AM, Martin Buchholz wrote: > You should delete your import > > 36 import java.lang.IllegalStateException; > > --- > > I would bump up the time you're willing to wait here, by at least 10x. > > 2277 if ((end - start) > 200000000L * (AIX.is() ? 2 : 1)) > 2278 fail("Test failed: waitFor took too long (" + > (end - start) + "ns)"); ok, then the special case of the AIX bump isn't necessary > > --- > > When you are surely waiting, it's best to optimize by waiting as > little as possible (although jtreg concurrency helps) > > 2301 p.waitFor(1, TimeUnit.SECONDS); > > I would set the timeout to only 10 millis here, and check that the > operation took at least 10 millis and no more than your long timeout > (you've settled on 7 seconds?) > as in jsr166 tck tests ok, I was thinking that too short and the spawned process would not have gotten started; but that's not significant in the test. > > --- > > 2315 exitValue = p.exitValue(); > 2316 } catch (IllegalThreadStateException ise) { > 2317 // no exitValue > 2318 } > > If ITSE exception is thrown by exitValue, then the test should always > fail but that is not currently the case. > If the test does fail, it currently throws uninformative NPE in > bjects.requireNonNull(exitValue), and we can do better. The exitValue was intended to help with further debugging. Objects.toString was intended. But it is as effective to leave the ITSE fall into the unexpected exception handler. Webrev updated. Thanks, Roger > > On Thu, Nov 13, 2014 at 7:37 AM, roger riggs wrote: >> Please re-review, >> >> Corrected the webrev, and a few changes to consistently use the same >> units, thanks for the review: >> >> The timeout time is extended to wait up to 7 seconds >> and the initial waitFor should last at least 1 second. >> >> http://cr.openjdk.java.net/~rriggs/webrev-basic-debug-8043477/ >> >> Roger >> >> >> >> >> On 11/12/2014 10:42 PM, Martin Buchholz wrote: >>> On Wed, Nov 12, 2014 at 6:25 PM, roger riggs >>> wrote: >>>> Hi Martin, >>>> >>>> I think that sleep is the version that spawns a JavaChild process and >>>> has its own comment interpreter. See line 309: Am I getting the sleeps >>>> mixed >>>> up? >>> Ah, good point - yes, you're right. Although we could always bump up >>> the sleep of the JavaChild from 10 if we wanted to - it should be >>> comfortably bigger than any individual test case's need. >> From martinrb at google.com Thu Nov 13 17:11:43 2014 From: martinrb at google.com (Martin Buchholz) Date: Thu, 13 Nov 2014 09:11:43 -0800 Subject: RFR 9 8043477: java/lang/ProcessBuilder/Basic.java failed with: java.lang.AssertionError: Some tests failed In-Reply-To: References: <5463B643.4050301@oracle.com> <5463BCCE.9070903@oracle.com> <5463C010.7000606@oracle.com> <5463D7AD.9080600@oracle.com> <54641693.8020700@oracle.com> <5464D01C.7050600@oracle.com> Message-ID: Looks good! On Thu, Nov 13, 2014 at 8:47 AM, Martin Buchholz wrote: > You should delete your import > > 36 import java.lang.IllegalStateException; > > --- > > I would bump up the time you're willing to wait here, by at least 10x. > > 2277 if ((end - start) > 200000000L * (AIX.is() ? 2 : 1)) > 2278 fail("Test failed: waitFor took too long (" + > (end - start) + "ns)"); > > --- > > When you are surely waiting, it's best to optimize by waiting as > little as possible (although jtreg concurrency helps) > > 2301 p.waitFor(1, TimeUnit.SECONDS); > > I would set the timeout to only 10 millis here, and check that the > operation took at least 10 millis and no more than your long timeout > (you've settled on 7 seconds?) > as in jsr166 tck tests > > --- > > 2315 exitValue = p.exitValue(); > 2316 } catch (IllegalThreadStateException ise) { > 2317 // no exitValue > 2318 } > > If ITSE exception is thrown by exitValue, then the test should always > fail but that is not currently the case. > If the test does fail, it currently throws uninformative NPE in > bjects.requireNonNull(exitValue), and we can do better. > > On Thu, Nov 13, 2014 at 7:37 AM, roger riggs wrote: >> Please re-review, >> >> Corrected the webrev, and a few changes to consistently use the same >> units, thanks for the review: >> >> The timeout time is extended to wait up to 7 seconds >> and the initial waitFor should last at least 1 second. >> >> http://cr.openjdk.java.net/~rriggs/webrev-basic-debug-8043477/ >> >> Roger >> >> >> >> >> On 11/12/2014 10:42 PM, Martin Buchholz wrote: >>> >>> On Wed, Nov 12, 2014 at 6:25 PM, roger riggs >>> wrote: >>>> >>>> Hi Martin, >>>> >>>> I think that sleep is the version that spawns a JavaChild process and >>>> has its own comment interpreter. See line 309: Am I getting the sleeps >>>> mixed >>>> up? >>> >>> Ah, good point - yes, you're right. Although we could always bump up >>> the sleep of the JavaChild from 10 if we wanted to - it should be >>> comfortably bigger than any individual test case's need. >> >> From patrick at reini.net Thu Nov 13 19:31:47 2014 From: patrick at reini.net (Patrick Reinhart) Date: Thu, 13 Nov 2014 20:31:47 +0100 Subject: Library enhancement proposal for copying data from/to Reader/Writer InputStream/OutputStream Message-ID: <566A428C-1A13-4DEE-940B-2EBE375F2A7A@reini.net> Hi there, In the followup of a BOF with Stephen Colebourne with his ideas of small library changes that may could get in JDK9. As of the fact that in the codebase of my company there are several locations where we copy from/to IO stream over and over again using either external libraries or do it by ourselves, I suggested to have some support for easy coping data between Input-/OutputStream or Reader/Writer without having to use to external libraries. My suggestion would look something like this: try (InputStream in = ? ; OutputStream out = ?) { in.copyTo(out); } Or from the other end: try (InputStream in = ? ; OutputStream out = ?) { out.copyFrom(in); } The same for character based streams: try (Reader in = ?; Writer out = ?) { in.copyTo(out); } or try (Reader in = ?; Writer out = ?) { out.copyFrom(in); } What do you think about this idea? I could imagine, that we could also pass a IntConsumer / LongConsumer in, that will be called with each amount of successfully copied amount of bytes/chars in case we need a total amount of data or displaying the progress of copy externally. I?m looking forward to your opinions upon this proposal. Cheers Patrick From stevenschlansker at gmail.com Thu Nov 13 19:37:26 2014 From: stevenschlansker at gmail.com (Steven Schlansker) Date: Thu, 13 Nov 2014 11:37:26 -0800 Subject: Library enhancement proposal for copying data from/to Reader/Writer InputStream/OutputStream In-Reply-To: <566A428C-1A13-4DEE-940B-2EBE375F2A7A@reini.net> References: <566A428C-1A13-4DEE-940B-2EBE375F2A7A@reini.net> Message-ID: <9252DA7C-C082-470A-A9D4-0A5AF0DCE5D7@gmail.com> On Nov 13, 2014, at 11:31 AM, Patrick Reinhart wrote: > Hi there, > > In the followup of a BOF with Stephen Colebourne with his ideas of small library changes that may could get in JDK9. As of the fact that in the codebase of my company there are several locations where we copy from/to IO stream over and over again using either external libraries or do it by ourselves, I suggested to have some support for easy coping data between Input-/OutputStream or Reader/Writer without having to use to external libraries. > > I?m looking forward to your opinions upon this proposal. +1, in my projects that I have open in Eclipse right now there are no fewer than six different implementations of this - Guava, commons-io, two homegrown that do things slightly differently, and on and on... Everyone needs to do it and it feels like a great addition. From Alan.Bateman at oracle.com Thu Nov 13 19:59:52 2014 From: Alan.Bateman at oracle.com (Alan Bateman) Date: Thu, 13 Nov 2014 19:59:52 +0000 Subject: Library enhancement proposal for copying data from/to Reader/Writer InputStream/OutputStream In-Reply-To: <566A428C-1A13-4DEE-940B-2EBE375F2A7A@reini.net> References: <566A428C-1A13-4DEE-940B-2EBE375F2A7A@reini.net> Message-ID: <54650DB8.6050801@oracle.com> On 13/11/2014 19:31, Patrick Reinhart wrote: > Hi there, > > In the followup of a BOF with Stephen Colebourne with his ideas of small library changes that may could get in JDK9. As of the fact that in the codebase of my company there are several locations where we copy from/to IO stream over and over again using either external libraries or do it by ourselves, I suggested to have some support for easy coping data between Input-/OutputStream or Reader/Writer without having to use to external libraries. > Long overdue. I remember we prototyped methods like this (and much more) during JDK 7 but didn't do enough at the time to actually get them in. We did include a bunch of easy to use methods in the Files class at the time, including copy between input/output streams and files, but we didn't introduce an IOUtils or such class in java.io. -Alan From vladimir.kozlov at oracle.com Thu Nov 13 23:06:28 2014 From: vladimir.kozlov at oracle.com (Vladimir Kozlov) Date: Thu, 13 Nov 2014 15:06:28 -0800 Subject: RFR: AARCH64: 8064594: Top-level JDK changes In-Reply-To: <5464FFB2.2050205@oracle.com> References: <546348F8.9060900@redhat.com> <54647DC5.9010102@redhat.com> <5464FFB2.2050205@oracle.com> Message-ID: <54653974.70804@oracle.com> Looks like the only comment we have is to change 2013 year to 2014 in the Copyright header in new files (and keep creation year from old file as Joe suggested). I can do that trivial change and push these changes into aarch64 staging repo. webrev: http://cr.openjdk.java.net/~aph/aarch64-JDK-8064594-1/ I included David's review from other thread to keep all reviews here: On 11/12/14 8:28 PM, David Holmes wrote: > On 13/11/2014 10:04 AM, Dean Long wrote: >> And adding build-infra-dev and jdk9-dev wouldn't hurt either. > > Let's not get carried away for what is quite a trivial copying of > existing platform specific patterns :) build-dev (not build-infra-dev) > would be okay. jdk9-dev isn't needed if already on hotspot-dev, > build-dev and sound-dev. > > These changes seem quite trivially fine to me. > > David H. Thanks, Vladimir On 11/13/14 11:00 AM, joe darcy wrote: > FWIW, if I were creating a new file by first copying an old file, I > would use a copyright range from the creation date of the old file to > the current year. > > -Joe > > On 11/13/2014 1:45 AM, Andrew Haley wrote: >> On 12/11/14 23:51, Christian Thalinger wrote: >>> The new jvm.cfg files should only have a copyright year of 2014. >> Why, exactly? They have been around for a while. >> >> Andrew. >> > From martinrb at google.com Fri Nov 14 00:03:59 2014 From: martinrb at google.com (Martin Buchholz) Date: Thu, 13 Nov 2014 16:03:59 -0800 Subject: RFR: 8064846: Lazy-init thread safety problems in core reflection Message-ID: Hi Joel, Joe, Paul, I'd like you to do a code review. http://cr.openjdk.java.net/~martin/webrevs/openjdk9/core-reflection-volatile/ https://bugs.openjdk.java.net/browse/JDK-8064846 From patrick at reini.net Fri Nov 14 06:14:47 2014 From: patrick at reini.net (Patrick Reinhart) Date: Fri, 14 Nov 2014 07:14:47 +0100 Subject: Library enhancement proposal for copying data from/to Reader/Writer InputStream/OutputStream In-Reply-To: <54650DB8.6050801@oracle.com> References: <566A428C-1A13-4DEE-940B-2EBE375F2A7A@reini.net> <54650DB8.6050801@oracle.com> Message-ID: <49F5819E-CAED-4564-880A-249967E70329@reini.net> Hi Alan, Where would you suggest to put such methods though? Should we aim for a ?IOUitls" class holding such methods in the first place and eventually place some simple convenience methods on Input-/OutputStream and Reader/Writer? I personally would do both, because I think it would make the usage more fluid from the usage perspective coming from a InputStream/Reader. Having said that, this leads me to the question: Are some other use cases where it would be good to have some sort of a callback in form of a function reference being called to notify about the amount of bytes/chars read and written to the target. This to have a hook for cases where we want to show either some sort of a progress or secondly sum of the total amount of transferred data. Also I think there should be may some possibility to stop the copy job on a ?feedback? event as follows: while ((read=input.read(buffer)>-1 && proceedCheck.read(read)) { output.write(buffer, 0, read); } Where proceedCheck would be a functional interface similar to the IntConsumer or LongConsumer but with a boolean return type signaling to keep on copying or terminate. (or there is a better approach using other functional interfaces, I missed so far) Other opinions / input? I would be glad to help improve the library in this direction to finally get rid of lots of duplicated code copying data? Cheers Patrick > Am 13.11.2014 um 20:59 schrieb Alan Bateman : > > On 13/11/2014 19:31, Patrick Reinhart wrote: >> Hi there, >> >> In the followup of a BOF with Stephen Colebourne with his ideas of small library changes that may could get in JDK9. As of the fact that in the codebase of my company there are several locations where we copy from/to IO stream over and over again using either external libraries or do it by ourselves, I suggested to have some support for easy coping data between Input-/OutputStream or Reader/Writer without having to use to external libraries. >> > Long overdue. I remember we prototyped methods like this (and much more) during JDK 7 but didn't do enough at the time to actually get them in. We did include a bunch of easy to use methods in the Files class at the time, including copy between input/output streams and files, but we didn't introduce an IOUtils or such class in java.io. > > -Alan > From aleksey.shipilev at oracle.com Fri Nov 14 08:03:41 2014 From: aleksey.shipilev at oracle.com (Aleksey Shipilev) Date: Fri, 14 Nov 2014 11:03:41 +0300 Subject: RFR: 8064846: Lazy-init thread safety problems in core reflection In-Reply-To: References: Message-ID: <5465B75D.4090703@oracle.com> Hi Martin, On 11/14/2014 03:03 AM, Martin Buchholz wrote: > I'd like you to do a code review. > > http://cr.openjdk.java.net/~martin/webrevs/openjdk9/core-reflection-volatile/ > https://bugs.openjdk.java.net/browse/JDK-8064846 Looks good and sane, thanks for taking care of it. Not a big fan of chained assignments though. It looks like you may reuse the same local variable "superinterfaces" here: 97 public Type[] getSuperInterfaces() { 98 Type[] superInterfaces = this.superInterfaces; 99 if (superInterfaces == null) { // lazily initialize super interfaces 100 // first, extract super interface subtree(s) from AST 101 TypeTree[] ts = getTree().getSuperInterfaces(); 102 // create array to store reified subtree(s) 103 superInterfaces = new Type[ts.length]; 104 // reify all subtrees 105 for (int i = 0; i < ts.length; i++) { 106 Reifier r = getReifier(); // obtain visitor 107 ts[i].accept(r);// reify subtree 108 // extract result from visitor and store it 109 superInterfaces[i] = r.getResult(); 110 } 111 this.superInterfaces = superInterfaces; 112 } 113 return superInterfaces.clone(); // return cached result 114 } Thanks, -Aleksey. From joel.franck at oracle.com Fri Nov 14 09:29:18 2014 From: joel.franck at oracle.com (Joel =?utf-8?Q?Borggr=C3=A9n-Franck?=) Date: Fri, 14 Nov 2014 10:29:18 +0100 Subject: RFR: 8064846: Lazy-init thread safety problems in core reflection In-Reply-To: References: Message-ID: <20141114092918.GE16532@oracle.com> On 2014-11-13, Martin Buchholz wrote: > Hi Joel, Joe, Paul, > > I'd like you to do a code review. > > http://cr.openjdk.java.net/~martin/webrevs/openjdk9/core-reflection-volatile/ > https://bugs.openjdk.java.net/browse/JDK-8064846 Looks good. I think Aleksey's version removing 'sis' is a bit neater. cheers /Joel From rafael.wth at gmail.com Fri Nov 14 09:51:53 2014 From: rafael.wth at gmail.com (Rafael Winterhalter) Date: Fri, 14 Nov 2014 10:51:53 +0100 Subject: AnnotationInvocationHandler - null check for hashCode / equals / toString computation? Message-ID: For reasons I am not going into, I am implementing my own AnnotationInvocationHandler and I want it to be compatible to the OpenJDK's annotation invocation handler with respect to the hash code and equals functionality, thus I just looked at the source in more detail. During implementing the handler, I realized that: 1. The invoke method applies a null check for an annotation value when looking it up for explicitly reading an annotation property and throws an IncompleteAnnotationException if no value and no default value is provided. 2. No such checks are applied when invoking the hashCode / equals / toString methods for the annotation which all have their own handler method. This would result in a NullPointerException for all of them. I did not find any formal specification for what is supposed to happen. Is it correct to throw these exceptions or should it rather also be an IncompleteAnnotationException? Thanks for the info! Rafael From sean.coffey at oracle.com Fri Nov 14 15:27:05 2014 From: sean.coffey at oracle.com (=?UTF-8?B?U2XDoW4gQ29mZmV5?=) Date: Fri, 14 Nov 2014 15:27:05 +0000 Subject: RFR : 8047340 - (process) Runtime.exec() fails in Turkish locale Message-ID: <54661F49.4010406@oracle.com> This issue was addressed in JDK8u and later via JDK-8000975. I'm planning on fixing this in JDK 7u by itself. webrev : http://cr.openjdk.java.net/~coffeys/webrev.8047340.7u/webrev/ bug report : https://bugs.openjdk.java.net/browse/JDK-8047340 regards, Sean. From paul.sandoz at oracle.com Fri Nov 14 15:53:26 2014 From: paul.sandoz at oracle.com (Paul Sandoz) Date: Fri, 14 Nov 2014 16:53:26 +0100 Subject: RFR: 8063147: Class.getFields spec should state that fields are inherited from superinterfaces In-Reply-To: <2CFBB9AC-6794-4799-B51D-48A179F8CE91@oracle.com> References: <2CFBB9AC-6794-4799-B51D-48A179F8CE91@oracle.com> Message-ID: <4F76087E-8EA4-4129-A56A-225BEEA5B577@oracle.com> On Nov 10, 2014, at 10:43 AM, Paul Sandoz wrote: >>> Please shepherd through CCC. > > Kick started. > CCC has been approved, Paul. From peter.levart at gmail.com Fri Nov 14 17:32:45 2014 From: peter.levart at gmail.com (Peter Levart) Date: Fri, 14 Nov 2014 18:32:45 +0100 Subject: RFR: 8064846: Lazy-init thread safety problems in core reflection In-Reply-To: References: Message-ID: Hi Martin, I dont know if you saw https://bugs.openjdk.java.net/browse/JDK-8064517 (a followup to your fix for final fields). It would be best to merge those fixes, what do you think? Otherwise I think that making all lazily initialized fields volatile is not necessary. Since you have made the Type implementations unsafe-publication-tolerable (by making their fields final), the only fields that need to be volatile are those that are used to lazily publish arrays. Am I right? Regards, Peter On Nov 14, 2014 1:03 AM, "Martin Buchholz" wrote: > Hi Joel, Joe, Paul, > > I'd like you to do a code review. > > > http://cr.openjdk.java.net/~martin/webrevs/openjdk9/core-reflection-volatile/ > https://bugs.openjdk.java.net/browse/JDK-8064846 > From martinrb at google.com Fri Nov 14 19:31:18 2014 From: martinrb at google.com (Martin Buchholz) Date: Fri, 14 Nov 2014 11:31:18 -0800 Subject: RFR: 8062194: java.util.jar.Attributes should use insertion-ordered iteration In-Reply-To: <545DDB5D.4030401@oracle.com> References: <544E96AC.90904@oracle.com> <544FF3ED.2020901@oracle.com> <545C966F.8080101@oracle.com> <545DDB5D.4030401@oracle.com> Message-ID: Committed as http://hg.openjdk.java.net/jdk9/dev/jdk/rev/3ff567ffe52a. On Sat, Nov 8, 2014 at 12:59 AM, Alan Bateman wrote: > On 07/11/2014 20:52, Martin Buchholz wrote: >> >> : >> I don't much care either way, so the spec change to >> Attributes(Attributes) is reverted, as you wish. Webrev refreshed. > > Thanks, I think it's a better with just the update to the class description. > > -Alan From roger.riggs at oracle.com Fri Nov 14 19:48:45 2014 From: roger.riggs at oracle.com (roger riggs) Date: Fri, 14 Nov 2014 14:48:45 -0500 Subject: RFR 9: 8064932: test java/lang/ProcessBuilder/Basic.java: waitFor didn't take long enough Message-ID: <54665C9D.1010903@oracle.com> Please review this test change to make the wait time in ProcessBuilder/Basic a bit more lenient. Webrev: http://cr.openjdk.java.net/~rriggs/webrev-basic-notenough-8064932/ Issue: 8064932: java/lang/ProcessBuilder/Basic.java: waitFor didn't take long enough Thanks, Roger From martinrb at google.com Fri Nov 14 19:57:19 2014 From: martinrb at google.com (Martin Buchholz) Date: Fri, 14 Nov 2014 11:57:19 -0800 Subject: RFR 9: 8064932: test java/lang/ProcessBuilder/Basic.java: waitFor didn't take long enough In-Reply-To: <54665C9D.1010903@oracle.com> References: <54665C9D.1010903@oracle.com> Message-ID: This sort of change may be a necessary concession to reality, but before we go there ... as I said in a previous review, this test may demonstrate a real bug in the jdk implementation. Is this system-dependent? Is it easy to reproduce? Can we fix the JDK? On Fri, Nov 14, 2014 at 11:48 AM, roger riggs wrote: > Please review this test change to make the wait time in ProcessBuilder/Basic > a bit more lenient. > > Webrev: > http://cr.openjdk.java.net/~rriggs/webrev-basic-notenough-8064932/ > > Issue: > 8064932: java/lang/ProcessBuilder/Basic.java: waitFor didn't take long > enough > > Thanks, Roger > From martinrb at google.com Fri Nov 14 20:17:27 2014 From: martinrb at google.com (Martin Buchholz) Date: Fri, 14 Nov 2014 12:17:27 -0800 Subject: RFR: 8063147: Class.getFields spec should state that fields are inherited from superinterfaces In-Reply-To: <4F76087E-8EA4-4129-A56A-225BEEA5B577@oracle.com> References: <2CFBB9AC-6794-4799-B51D-48A179F8CE91@oracle.com> <4F76087E-8EA4-4129-A56A-225BEEA5B577@oracle.com> Message-ID: Committed. On Fri, Nov 14, 2014 at 7:53 AM, Paul Sandoz wrote: > On Nov 10, 2014, at 10:43 AM, Paul Sandoz wrote: >>>> Please shepherd through CCC. >> >> Kick started. >> > > CCC has been approved, > Paul. > From martinrb at google.com Fri Nov 14 20:22:07 2014 From: martinrb at google.com (Martin Buchholz) Date: Fri, 14 Nov 2014 12:22:07 -0800 Subject: RFR: 8062773: Clarifications for Class specification In-Reply-To: References: <7746DAA6-2F3E-4891-B269-FC716D352B81@oracle.com> <5459B518.1080206@oracle.com> Message-ID: Companion change to getFields has been submitted. I plan on submitting this one soon if I don't hear any objections. On Thu, Nov 6, 2014 at 11:05 AM, Martin Buchholz wrote: > I moved the change to getFields to another changeset, redid some > wording as suggested, harmonized getInterfaces and > getGenericInterfaces some more, and regenerated the > webrev.http://cr.openjdk.java.net/~martin/webrevs/openjdk9/Class-spec/ > > On Tue, Nov 4, 2014 at 9:26 PM, joe darcy wrote: >> Hi Martin, >> >> This description in getGenericInterfaces >> >> 876 *

If this object represents a class, the return value is an >> 877 * array containing objects representing all interfaces >> 878 * implemented by the class. The order of the interface objects in >> >> could also be direct-ized. >> >> In getGenericInterfaces, the new text >> >> 899 *

If this {@code Class} object represents an array type, the >> 900 * interfaces {@code Cloneable} and {@code java.io.Serializable} >> are >> 901 * returned in that order. >> >> is redundant with the existing text >> >> 881 * represented by this object. In the case of an array class, the >> 882 * interfaces {@code Cloneable} and {@code Serializable} are >> 883 * returned in that order. >> >> Exactly one copy of the Cloneable & Serializable assertion should be removed >> :-) >> >> With this change to getFields, >> >> 1540 * returns the public fields of the class and of all its >> superclasses and >> 1541 * superinterfaces. >> >> the bug will need a ccc request since the specification is changing (even if >> it is changing to match long-standing behavior). >> >> Thanks, >> >> -Joe >> >> >> On 11/4/2014 9:08 AM, Martin Buchholz wrote: >>> >>> [+core-libs-dev] >>> >>> I'm sorry I keep forgetting to add the mailing list to my review >>> requests. I blame google's code review mail tool, which cc's the >>> right mailing lists automagically. >>> >>> Webrev updated to add a few more "direct"s, as suggested by Paul. >>> >>> https://bugs.openjdk.java.net/browse/JDK-8062773 >>> http://cr.openjdk.java.net/~martin/webrevs/openjdk9/Class-spec/ >>> >>> On Tue, Nov 4, 2014 at 3:02 AM, Paul Sandoz >>> wrote: >>>> >>>> On Nov 3, 2014, at 11:35 PM, Martin Buchholz wrote: >>>> >>>> Hi Joe, Paul, >>>> >>>> I'd like you to do a code review. >>>> >>>> https://bugs.openjdk.java.net/browse/JDK-8062773 >>>> http://cr.openjdk.java.net/~martin/webrevs/openjdk9/Class-spec/ >>>> >>>> >>>> >>>> Looks ok to me. >>>> >>>> 730 * @return the superclass of the class represented by this >>>> object. >>>> >>>> You could tweak that to say "the direct superclass..." >>>> >>>> Paul. >> >> From martinrb at google.com Fri Nov 14 21:38:10 2014 From: martinrb at google.com (Martin Buchholz) Date: Fri, 14 Nov 2014 13:38:10 -0800 Subject: RFR: 8064846: Lazy-init thread safety problems in core reflection In-Reply-To: <5465B75D.4090703@oracle.com> References: <5465B75D.4090703@oracle.com> Message-ID: Hi Aleksey, I've implemented your suggestions and regenerated the webrev. On Fri, Nov 14, 2014 at 12:03 AM, Aleksey Shipilev wrote: > Hi Martin, > > On 11/14/2014 03:03 AM, Martin Buchholz wrote: >> I'd like you to do a code review. >> >> http://cr.openjdk.java.net/~martin/webrevs/openjdk9/core-reflection-volatile/ >> https://bugs.openjdk.java.net/browse/JDK-8064846 > > Looks good and sane, thanks for taking care of it. > > Not a big fan of chained assignments though. It looks like you may reuse > the same local variable "superinterfaces" here: > > 97 public Type[] getSuperInterfaces() { > 98 Type[] superInterfaces = this.superInterfaces; > 99 if (superInterfaces == null) { // lazily initialize super > interfaces > 100 // first, extract super interface subtree(s) from AST > 101 TypeTree[] ts = getTree().getSuperInterfaces(); > 102 // create array to store reified subtree(s) > 103 superInterfaces = new Type[ts.length]; > 104 // reify all subtrees > 105 for (int i = 0; i < ts.length; i++) { > 106 Reifier r = getReifier(); // obtain visitor > 107 ts[i].accept(r);// reify subtree > 108 // extract result from visitor and store it > 109 superInterfaces[i] = r.getResult(); > 110 } > 111 this.superInterfaces = superInterfaces; > 112 } > 113 return superInterfaces.clone(); // return cached result > 114 } > > > Thanks, > -Aleksey. > > From martinrb at google.com Fri Nov 14 21:48:04 2014 From: martinrb at google.com (Martin Buchholz) Date: Fri, 14 Nov 2014 13:48:04 -0800 Subject: RFR: 8064846: Lazy-init thread safety problems in core reflection In-Reply-To: References: Message-ID: On Fri, Nov 14, 2014 at 9:32 AM, Peter Levart wrote: > Hi Martin, > > I dont know if you saw https://bugs.openjdk.java.net/browse/JDK-8064517 (a > followup to your fix for final fields). It would be best to merge those > fixes, what do you think? I've deliberately ignored those changes for now, so that this when this change goes in, all the jdk releases will have the same fixes applied. I agree there is more work to do. > Otherwise I think that making all lazily initialized fields volatile is not > necessary. Since you have made the Type implementations > unsafe-publication-tolerable (by making their fields final), the only fields > that need to be volatile are those that are used to lazily publish arrays. > > Am I right? You may be right, but it's risky - it's hard to tell whether every single Type implementation has exclusively final fields. We'd have to write yet another test that examines every single Class in the bootclasspath to be sure. (How does one do this in a jigsaw'ed world??) For safety's sake, I'd also like us to use CAS with our lazy-init fields. Perhaps use Atomic field updaters throughout the reflection codebase, for jdk9. I'm not (yet) claiming to be an OWNER of java/reflect. Who is the real OWNER? From roger.riggs at oracle.com Fri Nov 14 22:15:08 2014 From: roger.riggs at oracle.com (roger riggs) Date: Fri, 14 Nov 2014 17:15:08 -0500 Subject: RFR 9: 8064932: test java/lang/ProcessBuilder/Basic.java: waitFor didn't take long enough In-Reply-To: References: <54665C9D.1010903@oracle.com> Message-ID: <54667EEC.9060002@oracle.com> Hi Martin, The artifact is by-product of using System.nanoTime to measure the duration in UNIXProcess and the conversion to milliseconds to call Object.wait(): The low order bits of nanoseconds are truncated in the conversion. long timeoutAsNanos = unit.toNanos(timeout); long startTime = System.nanoTime(); long rem = timeoutAsNanos; while (!hasExited && (rem > 0)) { wait(Math.max(TimeUnit.NANOSECONDS.toMillis(rem), 1)); rem = timeoutAsNanos - (System.nanoTime() - startTime); } The issue may be further complicated by the test logic also doing the timing in nanoSeconds when the best Object.wait can do is milliseconds. If System.nanoTime is to be used, perhaps it should be modulo milliseconds. Roger On 11/14/2014 2:57 PM, Martin Buchholz wrote: > This sort of change may be a necessary concession to reality, but > before we go there ... as I said in a previous review, this test may > demonstrate a real bug in the jdk implementation. Is this > system-dependent? Is it easy to reproduce? Can we fix the JDK? > > On Fri, Nov 14, 2014 at 11:48 AM, roger riggs wrote: >> Please review this test change to make the wait time in ProcessBuilder/Basic >> a bit more lenient. >> >> Webrev: >> http://cr.openjdk.java.net/~rriggs/webrev-basic-notenough-8064932/ >> >> Issue: >> 8064932: java/lang/ProcessBuilder/Basic.java: waitFor didn't take long >> enough >> >> Thanks, Roger >> From martinrb at google.com Fri Nov 14 23:41:05 2014 From: martinrb at google.com (Martin Buchholz) Date: Fri, 14 Nov 2014 15:41:05 -0800 Subject: RFR : 8047340 - (process) Runtime.exec() fails in Turkish locale In-Reply-To: <54661F49.4010406@oracle.com> References: <54661F49.4010406@oracle.com> Message-ID: Another amusing "Turkish I bug"! Looks good to me! On Fri, Nov 14, 2014 at 7:27 AM, Se?n Coffey wrote: > This issue was addressed in JDK8u and later via JDK-8000975. I'm planning on > fixing this in JDK 7u by itself. > > webrev : http://cr.openjdk.java.net/~coffeys/webrev.8047340.7u/webrev/ > bug report : https://bugs.openjdk.java.net/browse/JDK-8047340 > > regards, > Sean. From martinrb at google.com Sat Nov 15 00:38:22 2014 From: martinrb at google.com (Martin Buchholz) Date: Fri, 14 Nov 2014 16:38:22 -0800 Subject: RFR 9: 8064932: test java/lang/ProcessBuilder/Basic.java: waitFor didn't take long enough In-Reply-To: <54667EEC.9060002@oracle.com> References: <54665C9D.1010903@oracle.com> <54667EEC.9060002@oracle.com> Message-ID: Hi Roger, I keep staring at the code in UNIXProcess.java and am having trouble imagining how waitFor could possibly return early - that loop can only terminate when elapsed time as measured by System.nanoTime exceeds timeoutAsNanos. It's true that we might have truncation when converting to millis, but then the wait should get called a second time with arg 1 (which is suboptimal, yes - we should improve waitFor (and very sadly, we should also improve Object.wait)). Is there a way for me to repro this test failure? The JDK tries hard to provide monotonic nanoTime(). On Fri, Nov 14, 2014 at 2:15 PM, roger riggs wrote: > Hi Martin, > > The artifact is by-product of using System.nanoTime to measure the duration > in UNIXProcess and the conversion to milliseconds to call Object.wait(): > > The low order bits of nanoseconds are truncated in the conversion. > > long timeoutAsNanos = unit.toNanos(timeout); > long startTime = System.nanoTime(); > long rem = timeoutAsNanos; > > while (!hasExited && (rem > 0)) { > wait(Math.max(TimeUnit.NANOSECONDS.toMillis(rem), 1)); > rem = timeoutAsNanos - (System.nanoTime() - startTime); > } > > The issue may be further complicated by the test logic also doing the > timing in nanoSeconds when the best Object.wait can do is milliseconds. > > If System.nanoTime is to be used, perhaps it should be modulo milliseconds. > > Roger > > > > On 11/14/2014 2:57 PM, Martin Buchholz wrote: >> >> This sort of change may be a necessary concession to reality, but >> before we go there ... as I said in a previous review, this test may >> demonstrate a real bug in the jdk implementation. Is this >> system-dependent? Is it easy to reproduce? Can we fix the JDK? >> >> On Fri, Nov 14, 2014 at 11:48 AM, roger riggs >> wrote: >>> >>> Please review this test change to make the wait time in >>> ProcessBuilder/Basic >>> a bit more lenient. >>> >>> Webrev: >>> http://cr.openjdk.java.net/~rriggs/webrev-basic-notenough-8064932/ >>> >>> Issue: >>> 8064932: java/lang/ProcessBuilder/Basic.java: waitFor didn't take >>> long >>> enough >>> >>> Thanks, Roger >>> > From richard.warburton at gmail.com Sat Nov 15 18:07:29 2014 From: richard.warburton at gmail.com (Richard Warburton) Date: Sat, 15 Nov 2014 18:07:29 +0000 Subject: Lower overhead String encoding/decoding In-Reply-To: <544FB0F7.8080909@univ-mlv.fr> References: <54200F07.4070604@oracle.com> <542441BB.600@oracle.com> <5444BB17.8080708@oracle.com> <544FADA5.7000403@oracle.com> <544FB0F7.8080909@univ-mlv.fr> Message-ID: Hi all, Thanks for taking the time to look at this - most appreciated. I've pushed >>> the latest iteration to http://cr.openjdk.java.net/~ >>> rwarburton/string-patch-webrev-8/ >> 7Erwarburton/string-patch-webrev-8/>. >>> >>> I think this is looking good. >> > Thanks - I've pushed a new iteration to http://cr.openjdk.java.net/~ rwarburton/string-patch-webrev-9. For the constructor then the words "decoding the specified byte buffer", it >> might be a bit clearer as "decoding the remaining bytes in the ...". >> >> For getBytes(ByteBuffer, Charset) then the position is advanced by the >> bytes written, no need to mention the number of chars read here. >> >> In the constructor then you make it clear that malformed/unmappable >> sequences use the default replacement. This is important to state in the >> getBytes methods too because the encoding can fail. > > I've made all these changes. Hi Richard, hi all, > Two comments, > You replace the nullcheck in getBytes() by a requireNonNull and at the > same time, you don"t use requireNonNull in String(ByteBuffer,Charset), > no very logical isn't it ? > Thanks for spotting this Remi - I've fixed this issue in my latest iteration. I think you need a supplementary constructor that takes a ByteBuffer and a > charset name as a String, > i may be wrong, but it seems that every constructor of String that takes a > Charset has a dual constructor that takes a Charset as a String. > As far as I remember, a constructor that takes a Charset as a String may > be faster because you can share the character decoder > instead of creating a new one. A good observation. I've added variants which take a String for the charset name as well the charset variants. Having said that - wouldn't it also be a good idea to replicate the caching on the charset versions as well as the charset name? I don't see any obvious reason why this isn't possible but perhaps there's something I'm missing here. Probably cleaner as a separate patch either way. regards, Richard Warburton http://insightfullogic.com @RichardWarburto From forax at univ-mlv.fr Sat Nov 15 19:19:22 2014 From: forax at univ-mlv.fr (Remi Forax) Date: Sat, 15 Nov 2014 20:19:22 +0100 Subject: Lower overhead String encoding/decoding In-Reply-To: References: <54200F07.4070604@oracle.com> <542441BB.600@oracle.com> <5444BB17.8080708@oracle.com> <544FADA5.7000403@oracle.com> <544FB0F7.8080909@univ-mlv.fr> Message-ID: <5467A73A.7050009@univ-mlv.fr> On 11/15/2014 07:07 PM, Richard Warburton wrote: > Hi all, > > [...] > > A good observation. I've added variants which take a String for the charset > name as well the charset variants. > > Having said that - wouldn't it also be a good idea to replicate the caching > on the charset versions as well as the charset name? I don't see any > obvious reason why this isn't possible but perhaps there's something I'm > missing here. Probably cleaner as a separate patch either way. > > regards, I think it will create a security hazard, if a rogue charset declare itself as UTF-8 and create a malicious charset decoder, you don't want this decoder to be shared and reused. R?mi > > Richard Warburton > > http://insightfullogic.com > @RichardWarburto From richard.warburton at gmail.com Sat Nov 15 21:55:47 2014 From: richard.warburton at gmail.com (Richard Warburton) Date: Sat, 15 Nov 2014 21:55:47 +0000 Subject: Lower overhead String encoding/decoding In-Reply-To: <5467A73A.7050009@univ-mlv.fr> References: <54200F07.4070604@oracle.com> <542441BB.600@oracle.com> <5444BB17.8080708@oracle.com> <544FADA5.7000403@oracle.com> <544FB0F7.8080909@univ-mlv.fr> <5467A73A.7050009@univ-mlv.fr> Message-ID: Hi, Having said that - wouldn't it also be a good idea to replicate the caching >> on the charset versions as well as the charset name? I don't see any >> obvious reason why this isn't possible but perhaps there's something I'm >> missing here. Probably cleaner as a separate patch either way. >> >> regards, >> > > I think it will create a security hazard, if a rogue charset declare > itself as UTF-8 and create a malicious charset decoder, you don't want this > decoder to be shared and reused. > When you're passing the name of the charset as a string, then you lookup the charset by name, but if you were using Charsets, then you could check that your cache has the same charset instance. Wouldn't that avoid this security issue? regards, Richard Warburton http://insightfullogic.com @RichardWarburto From joe.darcy at oracle.com Sun Nov 16 18:20:15 2014 From: joe.darcy at oracle.com (joe darcy) Date: Sun, 16 Nov 2014 10:20:15 -0800 Subject: AnnotationInvocationHandler - null check for hashCode / equals / toString computation? In-Reply-To: References: Message-ID: <5468EADF.9090907@oracle.com> Hello, The reference type members of annotations are all required to be non-null. Cheers, -Joe On 11/14/2014 1:51 AM, Rafael Winterhalter wrote: > For reasons I am not going into, I am implementing my own > AnnotationInvocationHandler and I want it to be compatible to the OpenJDK's > annotation invocation handler with respect to the hash code and equals > functionality, thus I just looked at the source in more detail. During > implementing the handler, I realized that: > > 1. The invoke method applies a null check for an annotation value when > looking it up for explicitly reading an annotation property and throws an > IncompleteAnnotationException if no value and no default value is provided. > > 2. No such checks are applied when invoking the hashCode / equals / > toString methods for the annotation which all have their own handler > method. This would result in a NullPointerException for all of them. > > I did not find any formal specification for what is supposed to happen. Is > it correct to throw these exceptions or should it rather also be an > IncompleteAnnotationException? > > Thanks for the info! Rafael From claes.redestad at oracle.com Sun Nov 16 23:09:53 2014 From: claes.redestad at oracle.com (Claes Redestad) Date: Mon, 17 Nov 2014 00:09:53 +0100 Subject: RFR (S): 8065070: (fmt) Avoid creating substrings when building FormatSpecifier Message-ID: <54692EC1.8010401@oracle.com> Hi, can I have a review for this small optimization to j.u.Formatter which removes a number of String allocationswhen parsing format specifiers: http://cr.openjdk.java.net/~redestad/8065070/webrev.00 https://bugs.openjdk.java.net/browse/JDK-8065070 Testing: jtreg Reasonable improvement (~5-20%) when microbenchmarking String.format expressions with index, precision and/or width specifiers. Example: @Benchmark public String formatIndexedDoubleExpr() { return String.format("%1$2.3f", pi); } before: 1082.564 18.943 ops/ms after: 1309.945 18.219 ops/ms # 1.21x Thanks! /Claes From aleksej.efimov at oracle.com Mon Nov 17 00:11:01 2014 From: aleksej.efimov at oracle.com (Aleksej Efimov) Date: Mon, 17 Nov 2014 03:11:01 +0300 Subject: RFR: 8064914: tzdb.dat compilation failure when using tzdata2014j Message-ID: <54693D15.5040204@oracle.com> Hello, During the latest tzdata (2014j) integration the tzdb.dat build failure was observed - details can be found in JBS [1]. The proposed [2] fix resolves time zones double link problem and JDK compilation is succedded. Can I ask a review for this fix, please? Thank you, Aleksej [1] Bug URL: https://bugs.openjdk.java.net/browse/JDK-8064914 [2] Webrev: http://cr.openjdk.java.net/~aefimov/8064914/9/webrev.00 From aleksej.efimov at oracle.com Mon Nov 17 00:13:23 2014 From: aleksej.efimov at oracle.com (Aleksej Efimov) Date: Mon, 17 Nov 2014 03:13:23 +0300 Subject: RFR: 8064560: (tz) Support tzdata2014j Message-ID: <54693DA3.30908@oracle.com> Hi, Please, review [1] tzdata2014j integration to JDK9 [2]. It is a straight-forward tzdata update, but new data caused a tzdb.dat build failure. It was reported under different bugId [3], there is a possible fix for that and request for review was sent [4]. Testing: JPRT: jdk_other jdk_util jdk_text jdk_time JTREG tests: test/sun/util/calendar test/java/util/Calendar test/sun/util/resources/TimeZone test/sun/util/calendar test/java/util/TimeZone test/java/time test/java/util/Formatter test/closed/java/util/Calendar test/closed/java/util/TimeZone test/closed/java/text/Format/DateFormat Thank you, Aleksej [1] Webrev: http://cr.openjdk.java.net/~aefimov/8064560/9/webrev.00/ [2] Bug URL: https://bugs.openjdk.java.net/browse/JDK-8064560 [3] Build failure bug: https://bugs.openjdk.java.net/browse/JDK-8064914 [4] Build failure RFR: http://mail.openjdk.java.net/pipermail/core-libs-dev/2014-November/029677.html From masayoshi.okutsu at oracle.com Mon Nov 17 02:45:19 2014 From: masayoshi.okutsu at oracle.com (Masayoshi Okutsu) Date: Mon, 17 Nov 2014 11:45:19 +0900 Subject: RFR: 8064560: (tz) Support tzdata2014j In-Reply-To: <54693DA3.30908@oracle.com> References: <54693DA3.30908@oracle.com> Message-ID: <5469613F.7000903@oracle.com> Looks good to me. Masayoshi On 11/17/2014 9:13 AM, Aleksej Efimov wrote: > Hi, > > Please, review [1] tzdata2014j integration to JDK9 [2]. It is a > straight-forward tzdata update, but new data caused a tzdb.dat build > failure. > It was reported under different bugId [3], there is a possible fix for > that and request for review was sent [4]. > Testing: > JPRT: jdk_other jdk_util jdk_text jdk_time > JTREG tests: test/sun/util/calendar test/java/util/Calendar > test/sun/util/resources/TimeZone test/sun/util/calendar > test/java/util/TimeZone test/java/time test/java/util/Formatter > test/closed/java/util/Calendar test/closed/java/util/TimeZone > test/closed/java/text/Format/DateFormat > > Thank you, > Aleksej > > [1] Webrev: http://cr.openjdk.java.net/~aefimov/8064560/9/webrev.00/ > [2] Bug URL: https://bugs.openjdk.java.net/browse/JDK-8064560 > [3] Build failure bug: https://bugs.openjdk.java.net/browse/JDK-8064914 > [4] Build failure RFR: > http://mail.openjdk.java.net/pipermail/core-libs-dev/2014-November/029677.html > From xueming.shen at oracle.com Mon Nov 17 04:29:19 2014 From: xueming.shen at oracle.com (Xueming Shen) Date: Sun, 16 Nov 2014 20:29:19 -0800 Subject: RFR: 8064914: tzdb.dat compilation failure when using tzdata2014j In-Reply-To: <54693D15.5040204@oracle.com> References: <54693D15.5040204@oracle.com> Message-ID: <5469799F.6050206@oracle.com> On 11/16/14 4:11 PM, Aleksej Efimov wrote: > Hello, > > During the latest tzdata (2014j) integration the tzdb.dat build > failure was observed - details can be found in JBS [1]. > The proposed [2] fix resolves time zones double link problem and JDK > compilation is succedded. > Can I ask a review for this fix, please? > > Thank you, > Aleksej > > [1] Bug URL: https://bugs.openjdk.java.net/browse/JDK-8064914 > [2] Webrev: http://cr.openjdk.java.net/~aefimov/8064914/9/webrev.00 looks good. From sean.coffey at oracle.com Mon Nov 17 09:39:32 2014 From: sean.coffey at oracle.com (=?UTF-8?B?U2XDoW4gQ29mZmV5?=) Date: Mon, 17 Nov 2014 09:39:32 +0000 Subject: RFR: 8064914: tzdb.dat compilation failure when using tzdata2014j In-Reply-To: <54693D15.5040204@oracle.com> References: <54693D15.5040204@oracle.com> Message-ID: <5469C254.30109@oracle.com> Looks fine. Thanks for handling. regards, Sean. On 17/11/2014 00:11, Aleksej Efimov wrote: > Hello, > > During the latest tzdata (2014j) integration the tzdb.dat build > failure was observed - details can be found in JBS [1]. > The proposed [2] fix resolves time zones double link problem and JDK > compilation is succedded. > Can I ask a review for this fix, please? > > Thank you, > Aleksej > > [1] Bug URL: https://bugs.openjdk.java.net/browse/JDK-8064914 > [2] Webrev: http://cr.openjdk.java.net/~aefimov/8064914/9/webrev.00 From aleksey.shipilev at oracle.com Mon Nov 17 09:52:33 2014 From: aleksey.shipilev at oracle.com (Aleksey Shipilev) Date: Mon, 17 Nov 2014 12:52:33 +0300 Subject: AnnotationInvocationHandler - null check for hashCode / equals / toString computation? In-Reply-To: <5468EADF.9090907@oracle.com> References: <5468EADF.9090907@oracle.com> Message-ID: <5469C561.2090702@oracle.com> Hi Joe, I am missing something. Of course, JLS requires the reference types members of annotations to be non-null. But IncompleteAnnotationEx is here to cover the case when the annotation is mis-constructed. Therefore, I think the original Rafael's question is still unanswered. I can see how the NPE can be produced while calling equals on mis-constructed annotation backed by AnnotationInvocationHandler: private Boolean equalsImpl(Object o) { ... for (Method memberMethod : getMemberMethods()) { // no value, returns null Object ourValue = memberValues.get(member); ... if (!memberValueEquals(ourValue, hisValue)) ... } ... } private static boolean memberValueEquals(Object v1, Object v2) { Class type = v1.getClass(); // NPE here ... } Like Rafael, I wonder if we can/should throw the IncompleteAnnotationException when accessing the annotation member implicitly with equals(), not with the named accessor. -Aleksey. On 11/16/2014 09:20 PM, joe darcy wrote: > Hello, > > The reference type members of annotations are all required to be non-null. > > Cheers, > > -Joe > > On 11/14/2014 1:51 AM, Rafael Winterhalter wrote: >> For reasons I am not going into, I am implementing my own >> AnnotationInvocationHandler and I want it to be compatible to the >> OpenJDK's >> annotation invocation handler with respect to the hash code and equals >> functionality, thus I just looked at the source in more detail. During >> implementing the handler, I realized that: >> >> 1. The invoke method applies a null check for an annotation value when >> looking it up for explicitly reading an annotation property and throws an >> IncompleteAnnotationException if no value and no default value is >> provided. >> >> 2. No such checks are applied when invoking the hashCode / equals / >> toString methods for the annotation which all have their own handler >> method. This would result in a NullPointerException for all of them. >> >> I did not find any formal specification for what is supposed to >> happen. Is >> it correct to throw these exceptions or should it rather also be an >> IncompleteAnnotationException? >> >> Thanks for the info! Rafael > From aleksey.shipilev at oracle.com Mon Nov 17 09:54:26 2014 From: aleksey.shipilev at oracle.com (Aleksey Shipilev) Date: Mon, 17 Nov 2014 12:54:26 +0300 Subject: AnnotationInvocationHandler - null check for hashCode / equals / toString computation? In-Reply-To: <5469C561.2090702@oracle.com> References: <5468EADF.9090907@oracle.com> <5469C561.2090702@oracle.com> Message-ID: <5469C5D2.1080709@oracle.com> On 11/17/2014 12:52 PM, Aleksey Shipilev wrote: > Like Rafael, I wonder if we can/should throw the > IncompleteAnnotationException when accessing the annotation member > implicitly with equals(), not with the named accessor. This should be "*as well as explicitly* with the named accessor". -Aleksey. From dawid.weiss at gmail.com Mon Nov 17 09:56:12 2014 From: dawid.weiss at gmail.com (Dawid Weiss) Date: Mon, 17 Nov 2014 10:56:12 +0100 Subject: Inconsistent access/ synchronization on Authenticator.theAuthenticator Message-ID: This is very minor, but java.net.Authenticator has the following static field for the default authenticator: private static Authenticator theAuthenticator; This field is set from a synchronized method setDefault(..), but accessed without proper sync from all other request* methods: Authenticator a = theAuthenticator; if (a == null) { return null; } D. From aleksey.shipilev at oracle.com Mon Nov 17 10:28:17 2014 From: aleksey.shipilev at oracle.com (Aleksey Shipilev) Date: Mon, 17 Nov 2014 13:28:17 +0300 Subject: Inconsistent access/ synchronization on Authenticator.theAuthenticator In-Reply-To: References: Message-ID: <5469CDC1.2090903@oracle.com> On 11/17/2014 12:56 PM, Dawid Weiss wrote: > This is very minor, but java.net.Authenticator has the following > static field for the default authenticator: > > private static Authenticator theAuthenticator; > > This field is set from a synchronized method setDefault(..), but > accessed without proper sync from all other request* methods: > > Authenticator a = theAuthenticator; > if (a == null) { > return null; > } Yes, theAuthenticator should be volatile. Filed: https://bugs.openjdk.java.net/browse/JDK-8065096 -Aleksey. From aleksey.shipilev at oracle.com Mon Nov 17 10:35:11 2014 From: aleksey.shipilev at oracle.com (Aleksey Shipilev) Date: Mon, 17 Nov 2014 13:35:11 +0300 Subject: RFR (XS) 8065096: java.net.Authenticator.theAuthenticator should be properly synchronized Message-ID: <5469CF5F.6010702@oracle.com> Hi, Please review and commit this trivial change: http://cr.openjdk.java.net/~shade/8065096/webrev.01/ https://bugs.openjdk.java.net/browse/JDK-8065096 "volatile" should nominally fix the the data race on theAuthenticator field. Thanks, -Aleksey. From aleksey.shipilev at oracle.com Mon Nov 17 10:45:04 2014 From: aleksey.shipilev at oracle.com (Aleksey Shipilev) Date: Mon, 17 Nov 2014 13:45:04 +0300 Subject: RFR (S): 8065070: (fmt) Avoid creating substrings when building FormatSpecifier In-Reply-To: <54692EC1.8010401@oracle.com> References: <54692EC1.8010401@oracle.com> Message-ID: <5469D1B0.1090706@oracle.com> On 11/17/2014 02:09 AM, Claes Redestad wrote: > http://cr.openjdk.java.net/~redestad/8065070/webrev.00 > https://bugs.openjdk.java.net/browse/JDK-8065070 (Not a Reviewer) The change looks very sane given the capture ranges are already available in Matcher. I wonder if you want to cache m.start() and m.end() in locals while you are at it -- I would think this amounts to moving tTStart, tTEnd a few lines before. > @Benchmark > public String formatIndexedDoubleExpr() { > return String.format("%1$2.3f", pi); > } > > before: 1082.564 18.943 ops/ms > after: 1309.945 18.219 ops/ms # 1.21x So this measures both the formatter setup costs and the actual formatting, right? 1.21x on that composite operation means even more speedup on formatter setup? Thanks, -Aleksey. From aleksey.shipilev at oracle.com Mon Nov 17 10:45:55 2014 From: aleksey.shipilev at oracle.com (Aleksey Shipilev) Date: Mon, 17 Nov 2014 13:45:55 +0300 Subject: RFR: 8064846: Lazy-init thread safety problems in core reflection In-Reply-To: References: <5465B75D.4090703@oracle.com> Message-ID: <5469D1E3.9020205@oracle.com> On 11/15/2014 12:38 AM, Martin Buchholz wrote: > I've implemented your suggestions and regenerated the webrev. Looks good, thanks! -Aleksey. From dawid.weiss at gmail.com Mon Nov 17 11:03:41 2014 From: dawid.weiss at gmail.com (Dawid Weiss) Date: Mon, 17 Nov 2014 12:03:41 +0100 Subject: Inconsistent access/ synchronization on Authenticator.theAuthenticator In-Reply-To: <5469CDC1.2090903@oracle.com> References: <5469CDC1.2090903@oracle.com> Message-ID: Thanks Aleksey. Can't file Jira issues myself, unfortunately; holding my word to Rory's promise that there will be a public Jira subscription access at some point -- it'd help a lot for those who wish to track changes on issues and currently have to do it manually. Dawid On Mon, Nov 17, 2014 at 11:28 AM, Aleksey Shipilev wrote: > On 11/17/2014 12:56 PM, Dawid Weiss wrote: >> This is very minor, but java.net.Authenticator has the following >> static field for the default authenticator: >> >> private static Authenticator theAuthenticator; >> >> This field is set from a synchronized method setDefault(..), but >> accessed without proper sync from all other request* methods: >> >> Authenticator a = theAuthenticator; >> if (a == null) { >> return null; >> } > > Yes, theAuthenticator should be volatile. > > Filed: > https://bugs.openjdk.java.net/browse/JDK-8065096 > > -Aleksey. > > From claes.redestad at oracle.com Mon Nov 17 11:49:19 2014 From: claes.redestad at oracle.com (Claes Redestad) Date: Mon, 17 Nov 2014 12:49:19 +0100 Subject: RFR (S): 8065070: (fmt) Avoid creating substrings when building FormatSpecifier In-Reply-To: <5469D1B0.1090706@oracle.com> References: <54692EC1.8010401@oracle.com> <5469D1B0.1090706@oracle.com> Message-ID: <5469E0BF.1050506@oracle.com> On 2014-11-17 11:45, Aleksey Shipilev wrote: > On 11/17/2014 02:09 AM, Claes Redestad wrote: >> http://cr.openjdk.java.net/~redestad/8065070/webrev.00 >> https://bugs.openjdk.java.net/browse/JDK-8065070 > (Not a Reviewer) The change looks very sane given the capture ranges are > already available in Matcher.I wonder if you want to cache m.start() > and m.end() in locals while you are at it -- I would think this amounts > to moving tTStart, tTEnd a few lines before. Thanks for taking a look at this! Not sure how you mean locally caching m.start(idx) would improve things, as we never reuse the same start/end values. Perhaps rewriting to something like this would make the code cleaner: index(s, m.start(1), m.end(1)); flags(s, m.start(2), m.end(2)); width(s, m.start(3), m.end(3)); precision(s, m.start(4), m.end(4)); int tTStart = m.start(5); if (tTStart >= 0) { dt = true; if (s.charAt(tTStart) == 'T') { f.add(Flags.UPPERCASE); } } conversion(s.charAt(m.start(6))); (I noticed that getting and checking tTEnd is basically redundant, since the formatSpecifier regex guarantees either one char or nothing is matched for that group) > >> @Benchmark >> public String formatIndexedDoubleExpr() { >> return String.format("%1$2.3f", pi); >> } >> >> before: 1082.564 18.943 ops/ms >> after: 1309.945 18.219 ops/ms # 1.21x > So this measures both the formatter setup costs and the actual > formatting, right? 1.21x on that composite operation means even more > speedup on formatter setup? Yes and yes (by deduction, since formatting code is unchanged). It's not possible to isolate setup from formatting using the public API, since formatter string parse is done in the private Formatter.format method. I guess copying the setup code into the benchmark and running it in isolation is feasible, but since this setup cost is paid on every format operation even if you break out and use a standalone Formatter, it currently makes sense to focus on the composite cost. /Claes > > Thanks, > -Aleksey. > > > From aleksey.shipilev at oracle.com Mon Nov 17 11:54:03 2014 From: aleksey.shipilev at oracle.com (Aleksey Shipilev) Date: Mon, 17 Nov 2014 14:54:03 +0300 Subject: RFR (S): 8065070: (fmt) Avoid creating substrings when building FormatSpecifier In-Reply-To: <5469E0BF.1050506@oracle.com> References: <54692EC1.8010401@oracle.com> <5469D1B0.1090706@oracle.com> <5469E0BF.1050506@oracle.com> Message-ID: <5469E1DB.9070608@oracle.com> On 11/17/2014 02:49 PM, Claes Redestad wrote: > On 2014-11-17 11:45, Aleksey Shipilev wrote: >> On 11/17/2014 02:09 AM, Claes Redestad wrote: >>> http://cr.openjdk.java.net/~redestad/8065070/webrev.00 >>> https://bugs.openjdk.java.net/browse/JDK-8065070 >> (Not a Reviewer) The change looks very sane given the capture ranges are >> already available in Matcher.I wonder if you want to cache m.start() >> and m.end() in locals while you are at it -- I would think this amounts >> to moving tTStart, tTEnd a few lines before. > > Thanks for taking a look at this! > > Not sure how you mean locally caching m.start(idx) would > improve things, as we never reuse the same start/end values. Ah, right. Curse those inline side effects. > Perhaps rewriting to something like this would make the code > cleaner: > > index(s, m.start(1), m.end(1)); > flags(s, m.start(2), m.end(2)); > width(s, m.start(3), m.end(3)); > precision(s, m.start(4), m.end(4)); > > int tTStart = m.start(5); > if (tTStart >= 0) { > dt = true; > if (s.charAt(tTStart) == 'T') { > f.add(Flags.UPPERCASE); > } > } > conversion(s.charAt(m.start(6))); Yes please. > (I noticed that getting and checking tTEnd is basically redundant, > since the formatSpecifier regex guarantees either one char or > nothing is matched for that group) That makes sense. -Aleksey. From chris.hegarty at oracle.com Mon Nov 17 12:05:20 2014 From: chris.hegarty at oracle.com (Chris Hegarty) Date: Mon, 17 Nov 2014 12:05:20 +0000 Subject: RFR (XS) 8065096: java.net.Authenticator.theAuthenticator should be properly synchronized In-Reply-To: <5469CF5F.6010702@oracle.com> References: <5469CF5F.6010702@oracle.com> Message-ID: Thanks for submitting the review Aleksey. Reviewed. -Chris. On 17 Nov 2014, at 10:35, Aleksey Shipilev wrote: > Hi, > > Please review and commit this trivial change: > http://cr.openjdk.java.net/~shade/8065096/webrev.01/ > https://bugs.openjdk.java.net/browse/JDK-8065096 > > "volatile" should nominally fix the the data race on theAuthenticator field. > > Thanks, > -Aleksey. > > From lance.andersen at oracle.com Mon Nov 17 12:15:33 2014 From: lance.andersen at oracle.com (Lance @ Oracle) Date: Mon, 17 Nov 2014 07:15:33 -0500 Subject: RFR (XS) 8065096: java.net.Authenticator.theAuthenticator should be properly synchronized In-Reply-To: <5469CF5F.6010702@oracle.com> References: <5469CF5F.6010702@oracle.com> Message-ID: <7352FCEC-14A8-4111-AF28-91197C020056@oracle.com> Looks fine aleksey Lance Andersen| Principal Member of Technical Staff | +1.781.442.2037 Oracle Java Engineering 1 Network Drive Burlington, MA 01803 Lance.Andersen at oracle.com Sent from my iPad > On Nov 17, 2014, at 5:35 AM, Aleksey Shipilev wrote: > > Hi, > > Please review and commit this trivial change: > http://cr.openjdk.java.net/~shade/8065096/webrev.01/ > https://bugs.openjdk.java.net/browse/JDK-8065096 > > "volatile" should nominally fix the the data race on theAuthenticator field. > > Thanks, > -Aleksey. > > From claes.redestad at oracle.com Mon Nov 17 12:23:38 2014 From: claes.redestad at oracle.com (Claes Redestad) Date: Mon, 17 Nov 2014 13:23:38 +0100 Subject: RFR (S): 8065070: (fmt) Avoid creating substrings when building FormatSpecifier In-Reply-To: <5469E1DB.9070608@oracle.com> References: <54692EC1.8010401@oracle.com> <5469D1B0.1090706@oracle.com> <5469E0BF.1050506@oracle.com> <5469E1DB.9070608@oracle.com> Message-ID: <5469E8CA.40500@oracle.com> On 2014-11-17 12:54, Aleksey Shipilev wrote: >> Perhaps rewriting to something like this would make the code >> cleaner: >> >> index(s, m.start(1), m.end(1)); >> flags(s, m.start(2), m.end(2)); >> width(s, m.start(3), m.end(3)); >> precision(s, m.start(4), m.end(4)); >> >> int tTStart = m.start(5); >> if (tTStart >= 0) { >> dt = true; >> if (s.charAt(tTStart) == 'T') { >> f.add(Flags.UPPERCASE); >> } >> } >> conversion(s.charAt(m.start(6))); > Yes please. > >> (I noticed that getting and checking tTEnd is basically redundant, >> since the formatSpecifier regex guarantees either one char or >> nothing is matched for that group) > That makes sense. New webrev including the above cleanup: http://cr.openjdk.java.net/~redestad/8065070/webrev.01 /Claes > > -Aleksey. > > From aleksey.shipilev at oracle.com Mon Nov 17 12:29:11 2014 From: aleksey.shipilev at oracle.com (Aleksey Shipilev) Date: Mon, 17 Nov 2014 15:29:11 +0300 Subject: RFR (S): 8065070: (fmt) Avoid creating substrings when building FormatSpecifier In-Reply-To: <5469E8CA.40500@oracle.com> References: <54692EC1.8010401@oracle.com> <5469D1B0.1090706@oracle.com> <5469E0BF.1050506@oracle.com> <5469E1DB.9070608@oracle.com> <5469E8CA.40500@oracle.com> Message-ID: <5469EA17.4010706@oracle.com> On 11/17/2014 03:23 PM, Claes Redestad wrote: > > On 2014-11-17 12:54, Aleksey Shipilev wrote: >>> Perhaps rewriting to something like this would make the code >>> cleaner: >>> >>> index(s, m.start(1), m.end(1)); >>> flags(s, m.start(2), m.end(2)); >>> width(s, m.start(3), m.end(3)); >>> precision(s, m.start(4), m.end(4)); >>> int tTStart = m.start(5); >>> if (tTStart >= 0) { >>> dt = true; >>> if (s.charAt(tTStart) == 'T') { >>> f.add(Flags.UPPERCASE); >>> } >>> } >>> conversion(s.charAt(m.start(6))); >> Yes please. >> >>> (I noticed that getting and checking tTEnd is basically redundant, >>> since the formatSpecifier regex guarantees either one char or >>> nothing is matched for that group) >> That makes sense. > > New webrev including the above cleanup: > > http://cr.openjdk.java.net/~redestad/8065070/webrev.01 Good, thank you! -Aleksey. From aleksey.shipilev at oracle.com Mon Nov 17 12:31:44 2014 From: aleksey.shipilev at oracle.com (Aleksey Shipilev) Date: Mon, 17 Nov 2014 15:31:44 +0300 Subject: RFR (XS) 8065096: java.net.Authenticator.theAuthenticator should be properly synchronized In-Reply-To: <7352FCEC-14A8-4111-AF28-91197C020056@oracle.com> References: <5469CF5F.6010702@oracle.com> <7352FCEC-14A8-4111-AF28-91197C020056@oracle.com> Message-ID: <5469EAB0.8000405@oracle.com> Thanks for reviews! Please push: http://cr.openjdk.java.net/~shade/8065096/8065096.changeset -Aleksey. On 11/17/2014 03:15 PM, Lance @ Oracle wrote: > Looks fine aleksey > > > Lance Andersen| > Principal Member of Technical Staff | +1.781.442.2037 > Oracle Java Engineering > 1 Network Drive > Burlington, MA 01803 > Lance.Andersen at oracle.com > Sent from my iPad > > On Nov 17, 2014, at 5:35 AM, Aleksey Shipilev > > wrote: > >> Hi, >> >> Please review and commit this trivial change: >> http://cr.openjdk.java.net/~shade/8065096/webrev.01/ >> >> https://bugs.openjdk.java.net/browse/JDK-8065096 >> >> "volatile" should nominally fix the the data race on theAuthenticator >> field. >> >> Thanks, >> -Aleksey. >> >> From rafael.wth at gmail.com Mon Nov 17 12:42:42 2014 From: rafael.wth at gmail.com (Rafael Winterhalter) Date: Mon, 17 Nov 2014 13:42:42 +0100 Subject: AnnotationInvocationHandler - null check for hashCode / equals / toString computation? In-Reply-To: <5469C561.2090702@oracle.com> References: <5468EADF.9090907@oracle.com> <5469C561.2090702@oracle.com> Message-ID: Thanks for clearing this up Aleksey, I should have posted an example right away so here is some code that reproduces the NullPointerException where I would either expect: (a) an IncompleteAnnotationException or (b) a return value of false. I guess returning false makes more sense as one expects to being able to call the equals (and hashCode / toString) methods even for incomplete annotations: https://gist.github.com/raphw/baeb31d81322d3dba482 By the javadoc documentation of the memberValueEquals method in the AnnotationInvocationHandler class, I would argue that at least the resulting NullPointerException is not desired. Thanks for the clarification, Rafael 2014-11-17 10:52 GMT+01:00 Aleksey Shipilev : > Hi Joe, > > I am missing something. Of course, JLS requires the reference types > members of annotations to be non-null. But IncompleteAnnotationEx is > here to cover the case when the annotation is mis-constructed. > > Therefore, I think the original Rafael's question is still unanswered. I > can see how the NPE can be produced while calling equals on > mis-constructed annotation backed by AnnotationInvocationHandler: > > private Boolean equalsImpl(Object o) { > ... > for (Method memberMethod : getMemberMethods()) { > // no value, returns null > Object ourValue = memberValues.get(member); > ... > if (!memberValueEquals(ourValue, hisValue)) > ... > } > ... > } > > private static boolean memberValueEquals(Object v1, Object v2) { > Class type = v1.getClass(); // NPE here > ... > } > > Like Rafael, I wonder if we can/should throw the > IncompleteAnnotationException when accessing the annotation member > implicitly with equals(), not with the named accessor. > > -Aleksey. > > On 11/16/2014 09:20 PM, joe darcy wrote: > > Hello, > > > > The reference type members of annotations are all required to be > non-null. > > > > Cheers, > > > > -Joe > > > > On 11/14/2014 1:51 AM, Rafael Winterhalter wrote: > >> For reasons I am not going into, I am implementing my own > >> AnnotationInvocationHandler and I want it to be compatible to the > >> OpenJDK's > >> annotation invocation handler with respect to the hash code and equals > >> functionality, thus I just looked at the source in more detail. During > >> implementing the handler, I realized that: > >> > >> 1. The invoke method applies a null check for an annotation value when > >> looking it up for explicitly reading an annotation property and throws > an > >> IncompleteAnnotationException if no value and no default value is > >> provided. > >> > >> 2. No such checks are applied when invoking the hashCode / equals / > >> toString methods for the annotation which all have their own handler > >> method. This would result in a NullPointerException for all of them. > >> > >> I did not find any formal specification for what is supposed to > >> happen. Is > >> it correct to throw these exceptions or should it rather also be an > >> IncompleteAnnotationException? > >> > >> Thanks for the info! Rafael > > > > > From rafael.wth at gmail.com Mon Nov 17 14:02:58 2014 From: rafael.wth at gmail.com (Rafael Winterhalter) Date: Mon, 17 Nov 2014 15:02:58 +0100 Subject: AnnotationInvocationHandler - null check for hashCode / equals / toString computation? In-Reply-To: <5469C5D2.1080709@oracle.com> References: <5468EADF.9090907@oracle.com> <5469C561.2090702@oracle.com> <5469C5D2.1080709@oracle.com> Message-ID: Hello again, I looked a little closer at the implementation of the AnnotationInvocationHandler and I want to extend my question beyond the equals method: When I define an annotation @Foo{ String value() } and instantiate two such annotations as @Foo("bar") class First {} and @Foo("bar") class Second {} these annotations for First and Second have identical return values for both their toString and hashCode methods and they are equal to one another. If I do however change the annotation propery's type to for example int without recompiling the classes that declare the two annotation instances, I get different hash codes for both annotations and the toString method prints something like @Foo(value="sun.reflect.annotation.AnnotationTypeMismatchExceptionProxy at 1540e19d ") where the proxy is of course of different object identity for both annotations. The hash code of the annotation is dependent on the hash code of this very same exception proxy instance. Obviously, this is difficult to reproduce for my third-party annotation invocation handler. As far as it is my understanding, the exceptions related to annotations (except the ClassNotFoundException) are triggered lazily on requesting a property in order to allow the usage of an annotation value even if one of its properties is "broken" as long as this property is not used within an annotation. The reasoning for this distinction lies on the assumption that annotations are rather meta data instead of an integral part of the program. I argue that this implies that that the annotations for First and Second should be equal. For example, I observed the use of annotations as keys in maps where one implicitly relies on the functioning for the equals and the hashCode methods. If I did not use the Foo::value property directly, I would never encounter an exception but I would miss values that were looked up in this very same map due to the hash code misses. On a side note, the AnnotationTypeMismatchExceptionProxy promises to be serializable but has a non-volatile field referencing a Method instance. Would this not cause a serialization error? Thanks for the information and best regards, Rafael 2014-11-17 10:54 GMT+01:00 Aleksey Shipilev : > On 11/17/2014 12:52 PM, Aleksey Shipilev wrote: > > Like Rafael, I wonder if we can/should throw the > > IncompleteAnnotationException when accessing the annotation member > > implicitly with equals(), not with the named accessor. > > This should be "*as well as explicitly* with the named accessor". > > -Aleksey. > > From claes.redestad at oracle.com Mon Nov 17 16:26:41 2014 From: claes.redestad at oracle.com (Claes Redestad) Date: Mon, 17 Nov 2014 17:26:41 +0100 Subject: RFR (XS) 8065096: java.net.Authenticator.theAuthenticator should be properly synchronized In-Reply-To: <5469EAB0.8000405@oracle.com> References: <5469CF5F.6010702@oracle.com> <7352FCEC-14A8-4111-AF28-91197C020056@oracle.com> <5469EAB0.8000405@oracle.com> Message-ID: <546A21C1.5070107@oracle.com> On 11/17/2014 01:31 PM, Aleksey Shipilev wrote: > Thanks for reviews! > > Please push: > http://cr.openjdk.java.net/~shade/8065096/8065096.changeset > > -Aleksey. I'll be happy to sponsor and push this for you. /Claes From mandy.chung at oracle.com Mon Nov 17 17:05:58 2014 From: mandy.chung at oracle.com (Mandy Chung) Date: Mon, 17 Nov 2014 09:05:58 -0800 Subject: [9] Review Request: 8059753 Fix for java/security/Security/ClassLoaderDeadlock/Deadlock2.sh fails with exit code 1 In-Reply-To: <545A3694.1090506@oracle.com> References: <5433C6BA.3000609@oracle.com> <545A3694.1090506@oracle.com> Message-ID: <546A2AF6.8040402@oracle.com> Hi Pooja, This test will need to be examined and updated (see JDK-8062758 [1]). The deadlock bug this test verifies is not specific the extension class loader. Although your patch is orthogonal to JDK-8062758, I wonder if you or Rajan can help look into updating this bug to verify JDK-6440846. Mandy [1] https://bugs.openjdk.java.net/browse/JDK-8062758 On 11/5/14 6:39 AM, pooja chopra wrote: > Hi All, > > A gentle reminder . Please review below fix. > > Regards, > Pooja > On 10/7/2014 4:25 PM, pooja chopra wrote: >> Hello, >> Please review a fix for the issue: >> 8059753 [TEST_BUG] Test >> java/security/Security/ClassLoaderDeadlock/Deadlock2.sh fails with >> exit code 1 >> Test bug fix. >> https://bugs.openjdk.java.net/browse/JDK-8059753 >> The webrev is: http://cr.openjdk.java.net/~kshefov/8059753/webrev.00/ >> >> Thanks >> Pooja > From martinrb at google.com Mon Nov 17 17:31:36 2014 From: martinrb at google.com (Martin Buchholz) Date: Mon, 17 Nov 2014 09:31:36 -0800 Subject: RFR (S): 8065070: (fmt) Avoid creating substrings when building FormatSpecifier In-Reply-To: <5469EA17.4010706@oracle.com> References: <54692EC1.8010401@oracle.com> <5469D1B0.1090706@oracle.com> <5469E0BF.1050506@oracle.com> <5469E1DB.9070608@oracle.com> <5469E8CA.40500@oracle.com> <5469EA17.4010706@oracle.com> Message-ID: I'll add my reviewer-bit (let's give more good people reviewer bits!). Looks good! (but also demonstrates deep existing problems with substring change) Not a defect with this change, but it looks to me like those assert(false) will trigger if e.g. a number is specified that is greater than Integer.MAX_VALUE. 2632 } catch (NumberFormatException x) { 2633 assert(false); On Mon, Nov 17, 2014 at 4:29 AM, Aleksey Shipilev wrote: > On 11/17/2014 03:23 PM, Claes Redestad wrote: >> >> On 2014-11-17 12:54, Aleksey Shipilev wrote: >>>> Perhaps rewriting to something like this would make the code >>>> cleaner: >>>> >>>> index(s, m.start(1), m.end(1)); >>>> flags(s, m.start(2), m.end(2)); >>>> width(s, m.start(3), m.end(3)); >>>> precision(s, m.start(4), m.end(4)); >>>> int tTStart = m.start(5); >>>> if (tTStart >= 0) { >>>> dt = true; >>>> if (s.charAt(tTStart) == 'T') { >>>> f.add(Flags.UPPERCASE); >>>> } >>>> } >>>> conversion(s.charAt(m.start(6))); >>> Yes please. >>> >>>> (I noticed that getting and checking tTEnd is basically redundant, >>>> since the formatSpecifier regex guarantees either one char or >>>> nothing is matched for that group) >>> That makes sense. >> >> New webrev including the above cleanup: >> >> http://cr.openjdk.java.net/~redestad/8065070/webrev.01 > > Good, thank you! > > -Aleksey. > > From dawid.weiss at gmail.com Mon Nov 17 18:38:39 2014 From: dawid.weiss at gmail.com (Dawid Weiss) Date: Mon, 17 Nov 2014 19:38:39 +0100 Subject: RFR (XS) 8065096: java.net.Authenticator.theAuthenticator should be properly synchronized In-Reply-To: <546A21C1.5070107@oracle.com> References: <5469CF5F.6010702@oracle.com> <7352FCEC-14A8-4111-AF28-91197C020056@oracle.com> <5469EAB0.8000405@oracle.com> <546A21C1.5070107@oracle.com> Message-ID: Hey, all of you guys... this must be the fastest turnaround ever! Thanks a lot, very much appreciated (even if it wasn't that crucial, only something I accidentally spotted in the code). Dawid On Mon, Nov 17, 2014 at 5:26 PM, Claes Redestad wrote: > On 11/17/2014 01:31 PM, Aleksey Shipilev wrote: >> >> Thanks for reviews! >> >> Please push: >> http://cr.openjdk.java.net/~shade/8065096/8065096.changeset >> >> -Aleksey. > > > I'll be happy to sponsor and push this for you. > > /Claes From roger.riggs at oracle.com Mon Nov 17 19:27:38 2014 From: roger.riggs at oracle.com (roger riggs) Date: Mon, 17 Nov 2014 14:27:38 -0500 Subject: RFR 9: 8064932: test java/lang/ProcessBuilder/Basic.java: waitFor didn't take long enough In-Reply-To: References: <54665C9D.1010903@oracle.com> <54667EEC.9060002@oracle.com> Message-ID: <546A4C2A.4070208@oracle.com> Hi, I need to go back and identify the platform of the failure; it is failing on Windows so the correct code is in ProcessImpl_md.c in waitForTimeoutInterruptibly in which the timeout is in milliseconds. If waitForMultipleObjects can return 'early' then the same kind of loop used on Linux for testing nanoTime may be needed. Roger On 11/14/2014 7:38 PM, Martin Buchholz wrote: > Hi Roger, > > I keep staring at the code in UNIXProcess.java and am having trouble > imagining how waitFor could possibly return early - that loop can only > terminate when elapsed time as measured by System.nanoTime exceeds > timeoutAsNanos. It's true that we might have truncation when > converting to millis, but then the wait should get called a second > time with arg 1 (which is suboptimal, yes - we should improve waitFor > (and very sadly, we should also improve Object.wait)). > > Is there a way for me to repro this test failure? The JDK tries hard > to provide monotonic nanoTime(). > > On Fri, Nov 14, 2014 at 2:15 PM, roger riggs wrote: >> Hi Martin, >> >> The artifact is by-product of using System.nanoTime to measure the duration >> in UNIXProcess and the conversion to milliseconds to call Object.wait(): >> >> The low order bits of nanoseconds are truncated in the conversion. >> >> long timeoutAsNanos = unit.toNanos(timeout); >> long startTime = System.nanoTime(); >> long rem = timeoutAsNanos; >> >> while (!hasExited && (rem > 0)) { >> wait(Math.max(TimeUnit.NANOSECONDS.toMillis(rem), 1)); >> rem = timeoutAsNanos - (System.nanoTime() - startTime); >> } >> >> The issue may be further complicated by the test logic also doing the >> timing in nanoSeconds when the best Object.wait can do is milliseconds. >> >> If System.nanoTime is to be used, perhaps it should be modulo milliseconds. >> >> Roger >> >> >> >> On 11/14/2014 2:57 PM, Martin Buchholz wrote: >>> This sort of change may be a necessary concession to reality, but >>> before we go there ... as I said in a previous review, this test may >>> demonstrate a real bug in the jdk implementation. Is this >>> system-dependent? Is it easy to reproduce? Can we fix the JDK? >>> >>> On Fri, Nov 14, 2014 at 11:48 AM, roger riggs >>> wrote: >>>> Please review this test change to make the wait time in >>>> ProcessBuilder/Basic >>>> a bit more lenient. >>>> >>>> Webrev: >>>> http://cr.openjdk.java.net/~rriggs/webrev-basic-notenough-8064932/ >>>> >>>> Issue: >>>> 8064932: java/lang/ProcessBuilder/Basic.java: waitFor didn't take >>>> long >>>> enough >>>> >>>> Thanks, Roger >>>> From martinrb at google.com Mon Nov 17 19:54:55 2014 From: martinrb at google.com (Martin Buchholz) Date: Mon, 17 Nov 2014 11:54:55 -0800 Subject: RFR 9: 8064932: test java/lang/ProcessBuilder/Basic.java: waitFor didn't take long enough In-Reply-To: <546A4C2A.4070208@oracle.com> References: <54665C9D.1010903@oracle.com> <54667EEC.9060002@oracle.com> <546A4C2A.4070208@oracle.com> Message-ID: Returning early is EVIL. ("""What part of 'wait for NNN nanoseconds' did you not understand??""") Unfortunately, Object.wait may do so. And perhaps also waitForMultipleObjects. HIgher level libraries need to be paranoid and compensate. On Mon, Nov 17, 2014 at 11:27 AM, roger riggs wrote: > Hi, > > I need to go back and identify the platform of the failure; it is failing > on Windows > so the correct code is in ProcessImpl_md.c in waitForTimeoutInterruptibly > in which the timeout is in milliseconds. > > If waitForMultipleObjects can return 'early' then the same kind of loop > used on Linux for testing nanoTime may be needed. > > Roger > > > > > On 11/14/2014 7:38 PM, Martin Buchholz wrote: >> >> Hi Roger, >> >> I keep staring at the code in UNIXProcess.java and am having trouble >> imagining how waitFor could possibly return early - that loop can only >> terminate when elapsed time as measured by System.nanoTime exceeds >> timeoutAsNanos. It's true that we might have truncation when >> converting to millis, but then the wait should get called a second >> time with arg 1 (which is suboptimal, yes - we should improve waitFor >> (and very sadly, we should also improve Object.wait)). >> >> Is there a way for me to repro this test failure? The JDK tries hard >> to provide monotonic nanoTime(). >> >> On Fri, Nov 14, 2014 at 2:15 PM, roger riggs >> wrote: >>> >>> Hi Martin, >>> >>> The artifact is by-product of using System.nanoTime to measure the >>> duration >>> in UNIXProcess and the conversion to milliseconds to call Object.wait(): >>> >>> The low order bits of nanoseconds are truncated in the conversion. >>> >>> long timeoutAsNanos = unit.toNanos(timeout); >>> long startTime = System.nanoTime(); >>> long rem = timeoutAsNanos; >>> >>> while (!hasExited && (rem > 0)) { >>> wait(Math.max(TimeUnit.NANOSECONDS.toMillis(rem), 1)); >>> rem = timeoutAsNanos - (System.nanoTime() - startTime); >>> } >>> >>> The issue may be further complicated by the test logic also doing the >>> timing in nanoSeconds when the best Object.wait can do is milliseconds. >>> >>> If System.nanoTime is to be used, perhaps it should be modulo >>> milliseconds. >>> >>> Roger >>> >>> >>> >>> On 11/14/2014 2:57 PM, Martin Buchholz wrote: >>>> >>>> This sort of change may be a necessary concession to reality, but >>>> before we go there ... as I said in a previous review, this test may >>>> demonstrate a real bug in the jdk implementation. Is this >>>> system-dependent? Is it easy to reproduce? Can we fix the JDK? >>>> >>>> On Fri, Nov 14, 2014 at 11:48 AM, roger riggs >>>> wrote: >>>>> >>>>> Please review this test change to make the wait time in >>>>> ProcessBuilder/Basic >>>>> a bit more lenient. >>>>> >>>>> Webrev: >>>>> http://cr.openjdk.java.net/~rriggs/webrev-basic-notenough-8064932/ >>>>> >>>>> Issue: >>>>> 8064932: java/lang/ProcessBuilder/Basic.java: waitFor didn't take >>>>> long >>>>> enough >>>>> >>>>> Thanks, Roger >>>>> > From joel.franck at oracle.com Mon Nov 17 21:15:06 2014 From: joel.franck at oracle.com (=?utf-8?Q?Joel_Borggr=C3=A9n-Franck?=) Date: Mon, 17 Nov 2014 22:15:06 +0100 Subject: RFR: 8064846: Lazy-init thread safety problems in core reflection In-Reply-To: References: Message-ID: <1C7B00E1-6411-4380-AFEE-9EB553D3FDC7@oracle.com> Hi, > On 14 Nov 2014, at 22:48, Martin Buchholz wrote: > > On Fri, Nov 14, 2014 at 9:32 AM, Peter Levart wrote: >> Hi Martin, >> >> I dont know if you saw https://bugs.openjdk.java.net/browse/JDK-8064517 (a >> followup to your fix for final fields). It would be best to merge those >> fixes, what do you think? > > I've deliberately ignored those changes for now, so that this when > this change goes in, all the jdk releases will have the same fixes > applied. > > I agree there is more work to do. > >> Otherwise I think that making all lazily initialized fields volatile is not >> necessary. Since you have made the Type implementations >> unsafe-publication-tolerable (by making their fields final), the only fields >> that need to be volatile are those that are used to lazily publish arrays. >> >> Am I right? > > You may be right, but it's risky - it's hard to tell whether every > single Type implementation has exclusively final fields. We'd have to > write yet another test that examines every single Class in the > bootclasspath to be sure. (How does one do this in a jigsaw'ed > world??) IIRC The jdk contains 4 subtypes of Type. I think Peter is right here, but on the other hand aren't uncontended volatile reads kind of cheap? Unless someone comes back with reports of measurable slowdown I think safe publication is ok. We can always revisit this later. > For safety's sake, I'd also like us to use CAS with our lazy-init > fields. Perhaps use Atomic field updaters throughout the reflection > codebase, for jdk9. > What is the problem you would be solving? > I'm not (yet) claiming to be an OWNER of java/reflect. Who is the real OWNER? Define OWNER. Joe I think, but most of the work is done by Mandy, Peter (!) or I depending on where you look. cheers /Joel From martinrb at google.com Mon Nov 17 21:39:39 2014 From: martinrb at google.com (Martin Buchholz) Date: Mon, 17 Nov 2014 13:39:39 -0800 Subject: RFR: 8062773: Clarifications for Class specification In-Reply-To: References: <7746DAA6-2F3E-4891-B269-FC716D352B81@oracle.com> <5459B518.1080206@oracle.com> Message-ID: Submitted. On Fri, Nov 14, 2014 at 12:22 PM, Martin Buchholz wrote: > Companion change to getFields has been submitted. I plan on > submitting this one soon if I don't hear any objections. > > On Thu, Nov 6, 2014 at 11:05 AM, Martin Buchholz wrote: >> I moved the change to getFields to another changeset, redid some >> wording as suggested, harmonized getInterfaces and >> getGenericInterfaces some more, and regenerated the >> webrev.http://cr.openjdk.java.net/~martin/webrevs/openjdk9/Class-spec/ >> >> On Tue, Nov 4, 2014 at 9:26 PM, joe darcy wrote: >>> Hi Martin, >>> >>> This description in getGenericInterfaces >>> >>> 876 *

If this object represents a class, the return value is an >>> 877 * array containing objects representing all interfaces >>> 878 * implemented by the class. The order of the interface objects in >>> >>> could also be direct-ized. >>> >>> In getGenericInterfaces, the new text >>> >>> 899 *

If this {@code Class} object represents an array type, the >>> 900 * interfaces {@code Cloneable} and {@code java.io.Serializable} >>> are >>> 901 * returned in that order. >>> >>> is redundant with the existing text >>> >>> 881 * represented by this object. In the case of an array class, the >>> 882 * interfaces {@code Cloneable} and {@code Serializable} are >>> 883 * returned in that order. >>> >>> Exactly one copy of the Cloneable & Serializable assertion should be removed >>> :-) >>> >>> With this change to getFields, >>> >>> 1540 * returns the public fields of the class and of all its >>> superclasses and >>> 1541 * superinterfaces. >>> >>> the bug will need a ccc request since the specification is changing (even if >>> it is changing to match long-standing behavior). >>> >>> Thanks, >>> >>> -Joe >>> >>> >>> On 11/4/2014 9:08 AM, Martin Buchholz wrote: >>>> >>>> [+core-libs-dev] >>>> >>>> I'm sorry I keep forgetting to add the mailing list to my review >>>> requests. I blame google's code review mail tool, which cc's the >>>> right mailing lists automagically. >>>> >>>> Webrev updated to add a few more "direct"s, as suggested by Paul. >>>> >>>> https://bugs.openjdk.java.net/browse/JDK-8062773 >>>> http://cr.openjdk.java.net/~martin/webrevs/openjdk9/Class-spec/ >>>> >>>> On Tue, Nov 4, 2014 at 3:02 AM, Paul Sandoz >>>> wrote: >>>>> >>>>> On Nov 3, 2014, at 11:35 PM, Martin Buchholz wrote: >>>>> >>>>> Hi Joe, Paul, >>>>> >>>>> I'd like you to do a code review. >>>>> >>>>> https://bugs.openjdk.java.net/browse/JDK-8062773 >>>>> http://cr.openjdk.java.net/~martin/webrevs/openjdk9/Class-spec/ >>>>> >>>>> >>>>> >>>>> Looks ok to me. >>>>> >>>>> 730 * @return the superclass of the class represented by this >>>>> object. >>>>> >>>>> You could tweak that to say "the direct superclass..." >>>>> >>>>> Paul. >>> >>> From roger.riggs at oracle.com Mon Nov 17 22:02:42 2014 From: roger.riggs at oracle.com (roger riggs) Date: Mon, 17 Nov 2014 17:02:42 -0500 Subject: RFR 9: 8064932: test java/lang/ProcessBuilder/Basic.java: waitFor didn't take long enough In-Reply-To: References: <54665C9D.1010903@oracle.com> <54667EEC.9060002@oracle.com> <546A4C2A.4070208@oracle.com> Message-ID: <546A7082.5060302@oracle.com> Hi, The technique used in the Linux version of Process.waitFor() is applied to the Windows version. The duration of the native wait is measured using nanoTime and the wait is repeated as necessary. For most uses, some jitter is expected due to workload, clock resolution, etc. Webrev: http://cr.openjdk.java.net/~rriggs/webrev-basic-notenough-8064932/ Thanks, Roger On 11/17/2014 2:54 PM, Martin Buchholz wrote: > Returning early is EVIL. > ("""What part of 'wait for NNN nanoseconds' did you not understand??""") > Unfortunately, Object.wait may do so. And perhaps also waitForMultipleObjects. > HIgher level libraries need to be paranoid and compensate. > > On Mon, Nov 17, 2014 at 11:27 AM, roger riggs wrote: >> Hi, >> >> I need to go back and identify the platform of the failure; it is failing >> on Windows >> so the correct code is in ProcessImpl_md.c in waitForTimeoutInterruptibly >> in which the timeout is in milliseconds. >> >> If waitForMultipleObjects can return 'early' then the same kind of loop >> used on Linux for testing nanoTime may be needed. >> >> Roger >> >> >> >> >> On 11/14/2014 7:38 PM, Martin Buchholz wrote: >>> Hi Roger, >>> >>> I keep staring at the code in UNIXProcess.java and am having trouble >>> imagining how waitFor could possibly return early - that loop can only >>> terminate when elapsed time as measured by System.nanoTime exceeds >>> timeoutAsNanos. It's true that we might have truncation when >>> converting to millis, but then the wait should get called a second >>> time with arg 1 (which is suboptimal, yes - we should improve waitFor >>> (and very sadly, we should also improve Object.wait)). >>> >>> Is there a way for me to repro this test failure? The JDK tries hard >>> to provide monotonic nanoTime(). >>> >>> On Fri, Nov 14, 2014 at 2:15 PM, roger riggs >>> wrote: >>>> Hi Martin, >>>> >>>> The artifact is by-product of using System.nanoTime to measure the >>>> duration >>>> in UNIXProcess and the conversion to milliseconds to call Object.wait(): >>>> >>>> The low order bits of nanoseconds are truncated in the conversion. >>>> >>>> long timeoutAsNanos = unit.toNanos(timeout); >>>> long startTime = System.nanoTime(); >>>> long rem = timeoutAsNanos; >>>> >>>> while (!hasExited && (rem > 0)) { >>>> wait(Math.max(TimeUnit.NANOSECONDS.toMillis(rem), 1)); >>>> rem = timeoutAsNanos - (System.nanoTime() - startTime); >>>> } >>>> >>>> The issue may be further complicated by the test logic also doing the >>>> timing in nanoSeconds when the best Object.wait can do is milliseconds. >>>> >>>> If System.nanoTime is to be used, perhaps it should be modulo >>>> milliseconds. >>>> >>>> Roger >>>> >>>> >>>> >>>> On 11/14/2014 2:57 PM, Martin Buchholz wrote: >>>>> This sort of change may be a necessary concession to reality, but >>>>> before we go there ... as I said in a previous review, this test may >>>>> demonstrate a real bug in the jdk implementation. Is this >>>>> system-dependent? Is it easy to reproduce? Can we fix the JDK? >>>>> >>>>> On Fri, Nov 14, 2014 at 11:48 AM, roger riggs >>>>> wrote: >>>>>> Please review this test change to make the wait time in >>>>>> ProcessBuilder/Basic >>>>>> a bit more lenient. >>>>>> >>>>>> Webrev: >>>>>> http://cr.openjdk.java.net/~rriggs/webrev-basic-notenough-8064932/ >>>>>> >>>>>> Issue: >>>>>> 8064932: java/lang/ProcessBuilder/Basic.java: waitFor didn't take >>>>>> long >>>>>> enough >>>>>> >>>>>> Thanks, Roger >>>>>> From martinrb at google.com Mon Nov 17 22:24:47 2014 From: martinrb at google.com (Martin Buchholz) Date: Mon, 17 Nov 2014 14:24:47 -0800 Subject: RFR: 8064846: Lazy-init thread safety problems in core reflection In-Reply-To: <1C7B00E1-6411-4380-AFEE-9EB553D3FDC7@oracle.com> References: <1C7B00E1-6411-4380-AFEE-9EB553D3FDC7@oracle.com> Message-ID: On Mon, Nov 17, 2014 at 1:15 PM, Joel Borggr?n-Franck wrote: > IIRC The jdk contains 4 subtypes of Type. I think Peter is right here, but on the other hand aren't uncontended volatile reads kind of cheap? Unless someone comes back with reports of measurable slowdown I think safe publication is ok. We can always revisit this later. I am OK with lazy non-volatile publication of immutable objects as long as we have a (reflective) test that proves we've kept all implementations correct. >> For safety's sake, I'd also like us to use CAS with our lazy-init >> fields. Perhaps use Atomic field updaters throughout the reflection >> codebase, for jdk9. >> > > What is the problem you would be solving? It's a safety thing. It's "probably" OK if a very rare race causes 2 invocations of an API to return different objects when normally the same object is returned. But it's simply safer to not do so, by CAS-ing from null to the newly computed value instead of unconditionally storing the new value. As we've seen, reflection is under-maintained and under-tested. From claes.redestad at oracle.com Mon Nov 17 22:54:45 2014 From: claes.redestad at oracle.com (Claes Redestad) Date: Mon, 17 Nov 2014 23:54:45 +0100 Subject: RFR (S): 8065070: (fmt) Avoid creating substrings when building FormatSpecifier In-Reply-To: References: <54692EC1.8010401@oracle.com> <5469D1B0.1090706@oracle.com> <5469E0BF.1050506@oracle.com> <5469E1DB.9070608@oracle.com> <5469E8CA.40500@oracle.com> <5469EA17.4010706@oracle.com> Message-ID: <546A7CB5.30700@oracle.com> On 2014-11-17 18:31, Martin Buchholz wrote: > I'll add my reviewer-bit (let's give more good people reviewer bits!). > Looks good! Thanks for reviewing! /Claes From pavel.rappo at oracle.com Mon Nov 17 23:27:15 2014 From: pavel.rappo at oracle.com (Pavel Rappo) Date: Mon, 17 Nov 2014 23:27:15 +0000 Subject: Several issues with java.net.URLConnection Message-ID: Hi everyone, I've been working with the URLConnection class for some time now and have noticed several interesting things. 1. Unsafe publication of java.net.URLConnection.factory static field (it looks very similar to [1]) 2. Benign race condition in java.net.URLConnection.getContentHandler on java.net.URLConnection.handlers static field 3. Handlers produced by factory are not cached in the handlers field (I don't know whether it was done deliberately or not, but I couldn't find any explanation on it in the javadoc. Moreover the behaviour of the factory in the URL class is different. It caches whatever is produced by the factory there). So which of these do you think are real issues worth fixing? ------------------------------------------------------------ [1] The same thing was noticed by Peter Levart here: http://mail.openjdk.java.net/pipermail/core-libs-dev/2014-July/027534.html -Pavel From mandy.chung at oracle.com Tue Nov 18 02:02:58 2014 From: mandy.chung at oracle.com (Mandy Chung) Date: Mon, 17 Nov 2014 18:02:58 -0800 Subject: [8u40] Review request 8064667: Provide support to help identify use of endorsed standards and extension mechanism In-Reply-To: <546A28F3.1010802@oracle.com> References: <546A28F3.1010802@oracle.com> Message-ID: <546AA8D2.1050600@oracle.com> Updated webrev: http://cr.openjdk.java.net/~mchung/jdk8u/webrevs/8064667/webrev.01/ This addresses Calvin's comment. It now keeps a list of the jar files shipped with jre/lib/ext and determine if jre/lib/ext has any other non-JDK jar files installed. Mandy On 11/17/2014 8:57 AM, Mandy Chung wrote: > This requests both code review and 8u40 approval for: > https://bugs.openjdk.java.net/browse/JDK-8064667 > > Webrev: > http://cr.openjdk.java.net/~mchung/jdk8u/webrevs/8064667/webrev.00/ > > JEP 220 [1] proposes to remove the endorsed standards override > mechanism and extension mechanism. This patch adds a VM flag in 8u40 > to help identify any existing uses of these mechanisms so that users > can turn on the VM flag to help identify if they depend on the > endorsed standards override mechanism and extension mechanism and can > plan to prepare for the migration to a newer JDK release early on. > When -XX:+CheckEndorsedAndExtDirs is set, the VM will exit if the > system property -Djava.endorsed.dirs or -Djava.ext.dirs is set, or if > ${java.home}/lib/endorsed or ${java.home}/lib/ext exists, or any > system extension directory contains JAR files. > > Thanks > Mandy > [1] http://openjdk.java.net/jeps/220 > > > From david.holmes at oracle.com Tue Nov 18 02:11:14 2014 From: david.holmes at oracle.com (David Holmes) Date: Tue, 18 Nov 2014 12:11:14 +1000 Subject: RFR 9: 8064932: test java/lang/ProcessBuilder/Basic.java: waitFor didn't take long enough In-Reply-To: <546A7082.5060302@oracle.com> References: <54665C9D.1010903@oracle.com> <54667EEC.9060002@oracle.com> <546A4C2A.4070208@oracle.com> <546A7082.5060302@oracle.com> Message-ID: <546AAAC2.6080009@oracle.com> Hi Roger, On 18/11/2014 8:02 AM, roger riggs wrote: > Hi, > > The technique used in the Linux version of Process.waitFor() is applied to > the Windows version. The duration of the native wait is measured > using nanoTime and the wait is repeated as necessary. > For most uses, some jitter is expected due to workload, clock > resolution, etc. > > Webrev: > http://cr.openjdk.java.net/~rriggs/webrev-basic-notenough-8064932/ This looks good to me. As can be seen here: http://msdn.microsoft.com/en-us/library/ms687069%28VS.85%29.aspx The win32 wait functions can return early. > Thanks, Roger > > On 11/17/2014 2:54 PM, Martin Buchholz wrote: >> Returning early is EVIL. Yet every OS seems to allow it Cheers, David >> ("""What part of 'wait for NNN nanoseconds' did you not understand??""") >> Unfortunately, Object.wait may do so. And perhaps also >> waitForMultipleObjects. >> HIgher level libraries need to be paranoid and compensate. >> >> On Mon, Nov 17, 2014 at 11:27 AM, roger riggs >> wrote: >>> Hi, >>> >>> I need to go back and identify the platform of the failure; it is >>> failing >>> on Windows >>> so the correct code is in ProcessImpl_md.c in >>> waitForTimeoutInterruptibly >>> in which the timeout is in milliseconds. >>> >>> If waitForMultipleObjects can return 'early' then the same kind of loop >>> used on Linux for testing nanoTime may be needed. >>> >>> Roger >>> >>> >>> >>> >>> On 11/14/2014 7:38 PM, Martin Buchholz wrote: >>>> Hi Roger, >>>> >>>> I keep staring at the code in UNIXProcess.java and am having trouble >>>> imagining how waitFor could possibly return early - that loop can only >>>> terminate when elapsed time as measured by System.nanoTime exceeds >>>> timeoutAsNanos. It's true that we might have truncation when >>>> converting to millis, but then the wait should get called a second >>>> time with arg 1 (which is suboptimal, yes - we should improve waitFor >>>> (and very sadly, we should also improve Object.wait)). >>>> >>>> Is there a way for me to repro this test failure? The JDK tries hard >>>> to provide monotonic nanoTime(). >>>> >>>> On Fri, Nov 14, 2014 at 2:15 PM, roger riggs >>>> wrote: >>>>> Hi Martin, >>>>> >>>>> The artifact is by-product of using System.nanoTime to measure the >>>>> duration >>>>> in UNIXProcess and the conversion to milliseconds to call >>>>> Object.wait(): >>>>> >>>>> The low order bits of nanoseconds are truncated in the conversion. >>>>> >>>>> long timeoutAsNanos = unit.toNanos(timeout); >>>>> long startTime = System.nanoTime(); >>>>> long rem = timeoutAsNanos; >>>>> >>>>> while (!hasExited && (rem > 0)) { >>>>> wait(Math.max(TimeUnit.NANOSECONDS.toMillis(rem), 1)); >>>>> rem = timeoutAsNanos - (System.nanoTime() - startTime); >>>>> } >>>>> >>>>> The issue may be further complicated by the test logic also doing the >>>>> timing in nanoSeconds when the best Object.wait can do is >>>>> milliseconds. >>>>> >>>>> If System.nanoTime is to be used, perhaps it should be modulo >>>>> milliseconds. >>>>> >>>>> Roger >>>>> >>>>> >>>>> >>>>> On 11/14/2014 2:57 PM, Martin Buchholz wrote: >>>>>> This sort of change may be a necessary concession to reality, but >>>>>> before we go there ... as I said in a previous review, this test may >>>>>> demonstrate a real bug in the jdk implementation. Is this >>>>>> system-dependent? Is it easy to reproduce? Can we fix the JDK? >>>>>> >>>>>> On Fri, Nov 14, 2014 at 11:48 AM, roger riggs >>>>>> >>>>>> wrote: >>>>>>> Please review this test change to make the wait time in >>>>>>> ProcessBuilder/Basic >>>>>>> a bit more lenient. >>>>>>> >>>>>>> Webrev: >>>>>>> http://cr.openjdk.java.net/~rriggs/webrev-basic-notenough-8064932/ >>>>>>> >>>>>>> Issue: >>>>>>> 8064932: java/lang/ProcessBuilder/Basic.java: waitFor >>>>>>> didn't take >>>>>>> long >>>>>>> enough >>>>>>> >>>>>>> Thanks, Roger >>>>>>> > From martinrb at google.com Tue Nov 18 02:28:31 2014 From: martinrb at google.com (Martin Buchholz) Date: Mon, 17 Nov 2014 18:28:31 -0800 Subject: RFR 9: 8064932: test java/lang/ProcessBuilder/Basic.java: waitFor didn't take long enough In-Reply-To: <546A7082.5060302@oracle.com> References: <54665C9D.1010903@oracle.com> <54667EEC.9060002@oracle.com> <546A4C2A.4070208@oracle.com> <546A7082.5060302@oracle.com> Message-ID: I was staring at that old process code I wrote many years ago, and I think it can be improved. I'll post a patch later. On Mon, Nov 17, 2014 at 2:02 PM, roger riggs wrote: > Hi, > > The technique used in the Linux version of Process.waitFor() is applied to > the Windows version. The duration of the native wait is measured > using nanoTime and the wait is repeated as necessary. > For most uses, some jitter is expected due to workload, clock resolution, > etc. > > Webrev: > http://cr.openjdk.java.net/~rriggs/webrev-basic-notenough-8064932/ > > Thanks, Roger > > > > > On 11/17/2014 2:54 PM, Martin Buchholz wrote: >> >> Returning early is EVIL. >> ("""What part of 'wait for NNN nanoseconds' did you not understand??""") >> Unfortunately, Object.wait may do so. And perhaps also >> waitForMultipleObjects. >> HIgher level libraries need to be paranoid and compensate. >> >> On Mon, Nov 17, 2014 at 11:27 AM, roger riggs >> wrote: >>> >>> Hi, >>> >>> I need to go back and identify the platform of the failure; it is >>> failing >>> on Windows >>> so the correct code is in ProcessImpl_md.c in waitForTimeoutInterruptibly >>> in which the timeout is in milliseconds. >>> >>> If waitForMultipleObjects can return 'early' then the same kind of loop >>> used on Linux for testing nanoTime may be needed. >>> >>> Roger >>> >>> >>> >>> >>> On 11/14/2014 7:38 PM, Martin Buchholz wrote: >>>> >>>> Hi Roger, >>>> >>>> I keep staring at the code in UNIXProcess.java and am having trouble >>>> imagining how waitFor could possibly return early - that loop can only >>>> terminate when elapsed time as measured by System.nanoTime exceeds >>>> timeoutAsNanos. It's true that we might have truncation when >>>> converting to millis, but then the wait should get called a second >>>> time with arg 1 (which is suboptimal, yes - we should improve waitFor >>>> (and very sadly, we should also improve Object.wait)). >>>> >>>> Is there a way for me to repro this test failure? The JDK tries hard >>>> to provide monotonic nanoTime(). >>>> >>>> On Fri, Nov 14, 2014 at 2:15 PM, roger riggs >>>> wrote: >>>>> >>>>> Hi Martin, >>>>> >>>>> The artifact is by-product of using System.nanoTime to measure the >>>>> duration >>>>> in UNIXProcess and the conversion to milliseconds to call >>>>> Object.wait(): >>>>> >>>>> The low order bits of nanoseconds are truncated in the conversion. >>>>> >>>>> long timeoutAsNanos = unit.toNanos(timeout); >>>>> long startTime = System.nanoTime(); >>>>> long rem = timeoutAsNanos; >>>>> >>>>> while (!hasExited && (rem > 0)) { >>>>> wait(Math.max(TimeUnit.NANOSECONDS.toMillis(rem), 1)); >>>>> rem = timeoutAsNanos - (System.nanoTime() - startTime); >>>>> } >>>>> >>>>> The issue may be further complicated by the test logic also doing the >>>>> timing in nanoSeconds when the best Object.wait can do is milliseconds. >>>>> >>>>> If System.nanoTime is to be used, perhaps it should be modulo >>>>> milliseconds. >>>>> >>>>> Roger >>>>> >>>>> >>>>> >>>>> On 11/14/2014 2:57 PM, Martin Buchholz wrote: >>>>>> >>>>>> This sort of change may be a necessary concession to reality, but >>>>>> before we go there ... as I said in a previous review, this test may >>>>>> demonstrate a real bug in the jdk implementation. Is this >>>>>> system-dependent? Is it easy to reproduce? Can we fix the JDK? >>>>>> >>>>>> On Fri, Nov 14, 2014 at 11:48 AM, roger riggs >>>>>> wrote: >>>>>>> >>>>>>> Please review this test change to make the wait time in >>>>>>> ProcessBuilder/Basic >>>>>>> a bit more lenient. >>>>>>> >>>>>>> Webrev: >>>>>>> http://cr.openjdk.java.net/~rriggs/webrev-basic-notenough-8064932/ >>>>>>> >>>>>>> Issue: >>>>>>> 8064932: java/lang/ProcessBuilder/Basic.java: waitFor didn't >>>>>>> take >>>>>>> long >>>>>>> enough >>>>>>> >>>>>>> Thanks, Roger >>>>>>> > From martinrb at google.com Tue Nov 18 04:43:37 2014 From: martinrb at google.com (Martin Buchholz) Date: Mon, 17 Nov 2014 20:43:37 -0800 Subject: RFR 9: 8064932: test java/lang/ProcessBuilder/Basic.java: waitFor didn't take long enough In-Reply-To: References: <54665C9D.1010903@oracle.com> <54667EEC.9060002@oracle.com> <546A4C2A.4070208@oracle.com> <546A7082.5060302@oracle.com> Message-ID: Proposed sibling change http://cr.openjdk.java.net/~martin/webrevs/openjdk9/UNIXProcess.waitFor/ - don't unconditionally call nanoTime when the wait ends - use the millis/nanos form of Object.wait in case sub-millisecond waits are ever supported. On Mon, Nov 17, 2014 at 6:28 PM, Martin Buchholz wrote: > I was staring at that old process code I wrote many years ago, and I > think it can be improved. I'll post a patch later. > > On Mon, Nov 17, 2014 at 2:02 PM, roger riggs wrote: >> Hi, >> >> The technique used in the Linux version of Process.waitFor() is applied to >> the Windows version. The duration of the native wait is measured >> using nanoTime and the wait is repeated as necessary. >> For most uses, some jitter is expected due to workload, clock resolution, >> etc. >> >> Webrev: >> http://cr.openjdk.java.net/~rriggs/webrev-basic-notenough-8064932/ >> >> Thanks, Roger >> >> >> >> >> On 11/17/2014 2:54 PM, Martin Buchholz wrote: >>> >>> Returning early is EVIL. >>> ("""What part of 'wait for NNN nanoseconds' did you not understand??""") >>> Unfortunately, Object.wait may do so. And perhaps also >>> waitForMultipleObjects. >>> HIgher level libraries need to be paranoid and compensate. >>> >>> On Mon, Nov 17, 2014 at 11:27 AM, roger riggs >>> wrote: >>>> >>>> Hi, >>>> >>>> I need to go back and identify the platform of the failure; it is >>>> failing >>>> on Windows >>>> so the correct code is in ProcessImpl_md.c in waitForTimeoutInterruptibly >>>> in which the timeout is in milliseconds. >>>> >>>> If waitForMultipleObjects can return 'early' then the same kind of loop >>>> used on Linux for testing nanoTime may be needed. >>>> >>>> Roger >>>> >>>> >>>> >>>> >>>> On 11/14/2014 7:38 PM, Martin Buchholz wrote: >>>>> >>>>> Hi Roger, >>>>> >>>>> I keep staring at the code in UNIXProcess.java and am having trouble >>>>> imagining how waitFor could possibly return early - that loop can only >>>>> terminate when elapsed time as measured by System.nanoTime exceeds >>>>> timeoutAsNanos. It's true that we might have truncation when >>>>> converting to millis, but then the wait should get called a second >>>>> time with arg 1 (which is suboptimal, yes - we should improve waitFor >>>>> (and very sadly, we should also improve Object.wait)). >>>>> >>>>> Is there a way for me to repro this test failure? The JDK tries hard >>>>> to provide monotonic nanoTime(). >>>>> >>>>> On Fri, Nov 14, 2014 at 2:15 PM, roger riggs >>>>> wrote: >>>>>> >>>>>> Hi Martin, >>>>>> >>>>>> The artifact is by-product of using System.nanoTime to measure the >>>>>> duration >>>>>> in UNIXProcess and the conversion to milliseconds to call >>>>>> Object.wait(): >>>>>> >>>>>> The low order bits of nanoseconds are truncated in the conversion. >>>>>> >>>>>> long timeoutAsNanos = unit.toNanos(timeout); >>>>>> long startTime = System.nanoTime(); >>>>>> long rem = timeoutAsNanos; >>>>>> >>>>>> while (!hasExited && (rem > 0)) { >>>>>> wait(Math.max(TimeUnit.NANOSECONDS.toMillis(rem), 1)); >>>>>> rem = timeoutAsNanos - (System.nanoTime() - startTime); >>>>>> } >>>>>> >>>>>> The issue may be further complicated by the test logic also doing the >>>>>> timing in nanoSeconds when the best Object.wait can do is milliseconds. >>>>>> >>>>>> If System.nanoTime is to be used, perhaps it should be modulo >>>>>> milliseconds. >>>>>> >>>>>> Roger >>>>>> >>>>>> >>>>>> >>>>>> On 11/14/2014 2:57 PM, Martin Buchholz wrote: >>>>>>> >>>>>>> This sort of change may be a necessary concession to reality, but >>>>>>> before we go there ... as I said in a previous review, this test may >>>>>>> demonstrate a real bug in the jdk implementation. Is this >>>>>>> system-dependent? Is it easy to reproduce? Can we fix the JDK? >>>>>>> >>>>>>> On Fri, Nov 14, 2014 at 11:48 AM, roger riggs >>>>>>> wrote: >>>>>>>> >>>>>>>> Please review this test change to make the wait time in >>>>>>>> ProcessBuilder/Basic >>>>>>>> a bit more lenient. >>>>>>>> >>>>>>>> Webrev: >>>>>>>> http://cr.openjdk.java.net/~rriggs/webrev-basic-notenough-8064932/ >>>>>>>> >>>>>>>> Issue: >>>>>>>> 8064932: java/lang/ProcessBuilder/Basic.java: waitFor didn't >>>>>>>> take >>>>>>>> long >>>>>>>> enough >>>>>>>> >>>>>>>> Thanks, Roger >>>>>>>> >> From martinrb at google.com Tue Nov 18 04:49:00 2014 From: martinrb at google.com (Martin Buchholz) Date: Mon, 17 Nov 2014 20:49:00 -0800 Subject: RFR 9: 8064932: test java/lang/ProcessBuilder/Basic.java: waitFor didn't take long enough In-Reply-To: <546AAAC2.6080009@oracle.com> References: <54665C9D.1010903@oracle.com> <54667EEC.9060002@oracle.com> <546A4C2A.4070208@oracle.com> <546A7082.5060302@oracle.com> <546AAAC2.6080009@oracle.com> Message-ID: Hi David, I'm still hoping to persuade you (and others) to fix the long-standing return-early bug in Object.wait. As new support for my position I offer the "at least" in JLS https://docs.oracle.com/javase/specs/jls/se8/html/jls-17.html#jls-17.2.1 """ If this is a timed wait, an internal action removing t from m's wait set that occurs after at least millisecs milliseconds plusnanosecs nanoseconds elapse since the beginning of this wait action. """ --- a/src/java.base/share/classes/java/lang/Object.java +++ b/src/java.base/share/classes/java/lang/Object.java @@ -444,16 +444,17 @@ * this exception is thrown. */ public final void wait(long timeout, int nanos) throws InterruptedException { - if (timeout < 0) { - throw new IllegalArgumentException("timeout value is negative"); - } + if (nanos != 0) { + if (timeout < 0) { + throw new IllegalArgumentException + ("timeout value is negative"); + } - if (nanos < 0 || nanos > 999999) { - throw new IllegalArgumentException( - "nanosecond timeout value out of range"); - } + if (nanos < 0 || nanos > 999999) { + throw new IllegalArgumentException + ("nanosecond timeout value out of range"); + } - if (nanos >= 500000 || (nanos != 0 && timeout == 0)) { timeout++; } On Mon, Nov 17, 2014 at 6:11 PM, David Holmes wrote: >> On 11/17/2014 2:54 PM, Martin Buchholz wrote: >>> >>> Returning early is EVIL. > > > Yet every OS seems to allow it From david.holmes at oracle.com Tue Nov 18 04:54:10 2014 From: david.holmes at oracle.com (David Holmes) Date: Tue, 18 Nov 2014 14:54:10 +1000 Subject: RFR 9: 8064932: test java/lang/ProcessBuilder/Basic.java: waitFor didn't take long enough In-Reply-To: References: <54665C9D.1010903@oracle.com> <54667EEC.9060002@oracle.com> <546A4C2A.4070208@oracle.com> <546A7082.5060302@oracle.com> Message-ID: <546AD0F2.9040002@oracle.com> On 18/11/2014 2:43 PM, Martin Buchholz wrote: > Proposed sibling change > http://cr.openjdk.java.net/~martin/webrevs/openjdk9/UNIXProcess.waitFor/ > - don't unconditionally call nanoTime when the wait ends > - use the millis/nanos form of Object.wait in case sub-millisecond > waits are ever supported. I don't really see the point of adding the extra math, plus an extra call (the two arg wait will call the one arg version) for no actual gain. If you want to add a fast exit path that's fine but the rest seems superfluous to me. David > On Mon, Nov 17, 2014 at 6:28 PM, Martin Buchholz wrote: >> I was staring at that old process code I wrote many years ago, and I >> think it can be improved. I'll post a patch later. >> >> On Mon, Nov 17, 2014 at 2:02 PM, roger riggs wrote: >>> Hi, >>> >>> The technique used in the Linux version of Process.waitFor() is applied to >>> the Windows version. The duration of the native wait is measured >>> using nanoTime and the wait is repeated as necessary. >>> For most uses, some jitter is expected due to workload, clock resolution, >>> etc. >>> >>> Webrev: >>> http://cr.openjdk.java.net/~rriggs/webrev-basic-notenough-8064932/ >>> >>> Thanks, Roger >>> >>> >>> >>> >>> On 11/17/2014 2:54 PM, Martin Buchholz wrote: >>>> >>>> Returning early is EVIL. >>>> ("""What part of 'wait for NNN nanoseconds' did you not understand??""") >>>> Unfortunately, Object.wait may do so. And perhaps also >>>> waitForMultipleObjects. >>>> HIgher level libraries need to be paranoid and compensate. >>>> >>>> On Mon, Nov 17, 2014 at 11:27 AM, roger riggs >>>> wrote: >>>>> >>>>> Hi, >>>>> >>>>> I need to go back and identify the platform of the failure; it is >>>>> failing >>>>> on Windows >>>>> so the correct code is in ProcessImpl_md.c in waitForTimeoutInterruptibly >>>>> in which the timeout is in milliseconds. >>>>> >>>>> If waitForMultipleObjects can return 'early' then the same kind of loop >>>>> used on Linux for testing nanoTime may be needed. >>>>> >>>>> Roger >>>>> >>>>> >>>>> >>>>> >>>>> On 11/14/2014 7:38 PM, Martin Buchholz wrote: >>>>>> >>>>>> Hi Roger, >>>>>> >>>>>> I keep staring at the code in UNIXProcess.java and am having trouble >>>>>> imagining how waitFor could possibly return early - that loop can only >>>>>> terminate when elapsed time as measured by System.nanoTime exceeds >>>>>> timeoutAsNanos. It's true that we might have truncation when >>>>>> converting to millis, but then the wait should get called a second >>>>>> time with arg 1 (which is suboptimal, yes - we should improve waitFor >>>>>> (and very sadly, we should also improve Object.wait)). >>>>>> >>>>>> Is there a way for me to repro this test failure? The JDK tries hard >>>>>> to provide monotonic nanoTime(). >>>>>> >>>>>> On Fri, Nov 14, 2014 at 2:15 PM, roger riggs >>>>>> wrote: >>>>>>> >>>>>>> Hi Martin, >>>>>>> >>>>>>> The artifact is by-product of using System.nanoTime to measure the >>>>>>> duration >>>>>>> in UNIXProcess and the conversion to milliseconds to call >>>>>>> Object.wait(): >>>>>>> >>>>>>> The low order bits of nanoseconds are truncated in the conversion. >>>>>>> >>>>>>> long timeoutAsNanos = unit.toNanos(timeout); >>>>>>> long startTime = System.nanoTime(); >>>>>>> long rem = timeoutAsNanos; >>>>>>> >>>>>>> while (!hasExited && (rem > 0)) { >>>>>>> wait(Math.max(TimeUnit.NANOSECONDS.toMillis(rem), 1)); >>>>>>> rem = timeoutAsNanos - (System.nanoTime() - startTime); >>>>>>> } >>>>>>> >>>>>>> The issue may be further complicated by the test logic also doing the >>>>>>> timing in nanoSeconds when the best Object.wait can do is milliseconds. >>>>>>> >>>>>>> If System.nanoTime is to be used, perhaps it should be modulo >>>>>>> milliseconds. >>>>>>> >>>>>>> Roger >>>>>>> >>>>>>> >>>>>>> >>>>>>> On 11/14/2014 2:57 PM, Martin Buchholz wrote: >>>>>>>> >>>>>>>> This sort of change may be a necessary concession to reality, but >>>>>>>> before we go there ... as I said in a previous review, this test may >>>>>>>> demonstrate a real bug in the jdk implementation. Is this >>>>>>>> system-dependent? Is it easy to reproduce? Can we fix the JDK? >>>>>>>> >>>>>>>> On Fri, Nov 14, 2014 at 11:48 AM, roger riggs >>>>>>>> wrote: >>>>>>>>> >>>>>>>>> Please review this test change to make the wait time in >>>>>>>>> ProcessBuilder/Basic >>>>>>>>> a bit more lenient. >>>>>>>>> >>>>>>>>> Webrev: >>>>>>>>> http://cr.openjdk.java.net/~rriggs/webrev-basic-notenough-8064932/ >>>>>>>>> >>>>>>>>> Issue: >>>>>>>>> 8064932: java/lang/ProcessBuilder/Basic.java: waitFor didn't >>>>>>>>> take >>>>>>>>> long >>>>>>>>> enough >>>>>>>>> >>>>>>>>> Thanks, Roger >>>>>>>>> >>> From david.holmes at oracle.com Tue Nov 18 05:02:05 2014 From: david.holmes at oracle.com (David Holmes) Date: Tue, 18 Nov 2014 15:02:05 +1000 Subject: RFR 9: 8064932: test java/lang/ProcessBuilder/Basic.java: waitFor didn't take long enough In-Reply-To: References: <54665C9D.1010903@oracle.com> <54667EEC.9060002@oracle.com> <546A4C2A.4070208@oracle.com> <546A7082.5060302@oracle.com> <546AAAC2.6080009@oracle.com> Message-ID: <546AD2CD.90300@oracle.com> Hi Martin, On 18/11/2014 2:49 PM, Martin Buchholz wrote: > Hi David, > > I'm still hoping to persuade you (and others) to fix the long-standing > return-early bug in Object.wait. Are you referring to the low level mechanics or specifically to the nanosecond version below? > As new support for my position I offer the "at least" in JLS > https://docs.oracle.com/javase/specs/jls/se8/html/jls-17.html#jls-17.2.1 > > """ > > If this is a timed wait, an internal action removing t from m's wait > set that occurs after at least millisecs milliseconds plusnanosecs > nanoseconds elapse since the beginning of this wait action. > > """ > > --- a/src/java.base/share/classes/java/lang/Object.java > +++ b/src/java.base/share/classes/java/lang/Object.java > @@ -444,16 +444,17 @@ > * this exception is thrown. > */ > public final void wait(long timeout, int nanos) throws > InterruptedException { > - if (timeout < 0) { > - throw new IllegalArgumentException("timeout value is negative"); > - } > + if (nanos != 0) { > + if (timeout < 0) { > + throw new IllegalArgumentException > + ("timeout value is negative"); > + } > > - if (nanos < 0 || nanos > 999999) { > - throw new IllegalArgumentException( > - "nanosecond timeout value out of range"); > - } > + if (nanos < 0 || nanos > 999999) { > + throw new IllegalArgumentException > + ("nanosecond timeout value out of range"); > + } > > - if (nanos >= 500000 || (nanos != 0 && timeout == 0)) { > timeout++; > } That seems overly complex - after checking for valid values of timeout and nanos you simply need: if (nanos > 0) timeout++; to ensure the >= requirement. But seriously we should just deprecate this version as we're not likely to get any underlying OS mechanisms for doing nanosecond resolution timed blocking actions. David ----- > > > On Mon, Nov 17, 2014 at 6:11 PM, David Holmes wrote: > >>> On 11/17/2014 2:54 PM, Martin Buchholz wrote: >>>> >>>> Returning early is EVIL. >> >> >> Yet every OS seems to allow it From ecki at zusammenkunft.net Tue Nov 18 05:11:08 2014 From: ecki at zusammenkunft.net (Bernd Eckenfels) Date: Tue, 18 Nov 2014 06:11:08 +0100 Subject: ThreadLocal with null initial values - avoid create map and entry? Message-ID: <20141118061108.0000746a.ecki@zusammenkunft.net> Hello, while debugging and optimizing some application ThreadLocal code I noticed, that if I not use the initialValue() method for filling a start value, I could often use the null value to shortcut processing. I.e. I would add a object with some state to clean up, but if there is no object, I know I dont have to clean up anything (and I dont need to create one). Unfortunatelly the ThreadLocal#get() will call #setInitialValue() in all cases, and this will in turn call createMap(). setInitialValue() could avoid to call createMap() when value is null. This would reduce the number of created thread specific maps and entries (and also weak references to the thread). I think not setting the entry is not a problem, as it happens always single threaded. Bucket conflicts should also be detected in the same way. I also think that the concurrently used createMap() is already racing with remove(), so it does not hurt. What do you think? (code from OpenJDK 8:) 179 private T setInitialValue() { 180 T value = initialValue(); 181 Thread t = Thread.currentThread(); 182 ThreadLocalMap map = getMap(t); 183 if (map != null) 184 map.set(this, value); 185 else +++ if (value != null) 186 createMap(t, value); 187 return value; 188 } I am not sure if InheritableThreadLocal would need to overwrite this and skip the optimization? Gruss Bernd From chris.hegarty at oracle.com Tue Nov 18 11:38:20 2014 From: chris.hegarty at oracle.com (Chris Hegarty) Date: Tue, 18 Nov 2014 11:38:20 +0000 Subject: Several issues with java.net.URLConnection In-Reply-To: References: Message-ID: <546B2FAC.8060209@oracle.com> On 17/11/14 23:27, Pavel Rappo wrote: > Hi everyone, > > I've been working with the URLConnection class for some time now and have noticed several interesting things. > > 1. Unsafe publication of java.net.URLConnection.factory static field (it looks very similar to [1]) > 2. Benign race condition in java.net.URLConnection.getContentHandler on java.net.URLConnection.handlers static field > 3. Handlers produced by factory are not cached in the handlers field (I don't know whether it was done deliberately or not, but I couldn't find any explanation on it in the javadoc. Moreover the behaviour of the factory in the URL class is different. It caches whatever is produced by the factory there). > > So which of these do you think are real issues worth fixing? 1 and 2 : Yes. 3 needs a little more consideration, but my gut feeling is that we should do it unless there is a good reason not to. -Chris. > ------------------------------------------------------------ > [1] The same thing was noticed by Peter Levart here: http://mail.openjdk.java.net/pipermail/core-libs-dev/2014-July/027534.html > > -Pavel > From tom.hawtin at oracle.com Tue Nov 18 11:37:34 2014 From: tom.hawtin at oracle.com (Tom Hawtin) Date: Tue, 18 Nov 2014 11:37:34 +0000 Subject: ThreadLocal with null initial values - avoid create map and entry? In-Reply-To: <20141118061108.0000746a.ecki@zusammenkunft.net> References: <20141118061108.0000746a.ecki@zusammenkunft.net> Message-ID: <546B2F7E.10206@oracle.com> On 18/11/2014 05:11, Bernd Eckenfels wrote: > Unfortunatelly the ThreadLocal#get() will call #setInitialValue() in > all cases, and this will in turn call createMap(). setInitialValue() > could avoid to call createMap() when value is null. This would reduce > the number of created thread specific maps and entries (and also weak > references to the thread). There's no guarantee that the execution of initialValue was trivial even if it returns null. I don't know whether or not mixing up the execution paths through get would harm performance. Tom From scolebourne at joda.org Tue Nov 18 12:13:07 2014 From: scolebourne at joda.org (Stephen Colebourne) Date: Tue, 18 Nov 2014 12:13:07 +0000 Subject: Time-zone distribution service Message-ID: For those of you not aware, an IETF group is currently defining a time zone distribution service "tzdist": http://tools.ietf.org/html/draft-ietf-tzdist-service-03 It will be worth considering whether this might be of use to Java to provide a mechanism to update time zone data. Readers of this list may wish to review the spec and ensure it meets needs. thanks Stephen From ecki at zusammenkunft.net Tue Nov 18 14:59:19 2014 From: ecki at zusammenkunft.net (Bernd Eckenfels) Date: Tue, 18 Nov 2014 15:59:19 +0100 Subject: ThreadLocal with null initial values - avoid create map and entry? In-Reply-To: <546B2F7E.10206@oracle.com> References: <20141118061108.0000746a.ecki@zusammenkunft.net> <546B2F7E.10206@oracle.com> Message-ID: <20141118155919.000000ff.ecki@zusammenkunft.net> Am Tue, 18 Nov 2014 11:37:34 +0000 schrieb Tom Hawtin : > On 18/11/2014 05:11, Bernd Eckenfels wrote: > > > Unfortunatelly the ThreadLocal#get() will call #setInitialValue() in > > all cases, and this will in turn call createMap(). setInitialValue() > > could avoid to call createMap() when value is null. This would > > reduce the number of created thread specific maps and entries (and > > also weak references to the thread). > > There's no guarantee that the execution of initialValue was trivial > even if it returns null. Yes, but I dont think it has to be. As long as the initial value is null it can be represented without any thread local map or entry. And as there is no "hasEntry()" a application can only call get to probe for existence (and this will in the current implementation always created it). So another option (but more complicated as it adds a new API function) would be to have a hasEntry() (or similiar named) function which does the same as get without calling the initialValue(). This would in this case really differ for non-trivial initialValue() implementations, thats why I would prefer to do it in get(). It needs to be checked if get() gets too big for inlining, but the new if is not in a hot path. Gruss Bernd > > I don't know whether or not mixing up the execution paths through get > would harm performance. > > Tom From ecki at zusammenkunft.net Tue Nov 18 15:09:11 2014 From: ecki at zusammenkunft.net (Bernd Eckenfels) Date: Tue, 18 Nov 2014 16:09:11 +0100 Subject: ThreadLocal with null initial values - avoid create map and entry? In-Reply-To: <20141118155919.000000ff.ecki@zusammenkunft.net> References: <20141118061108.0000746a.ecki@zusammenkunft.net> <546B2F7E.10206@oracle.com> <20141118155919.000000ff.ecki@zusammenkunft.net> Message-ID: <20141118160911.000045b1.ecki@zusammenkunft.net> Hello, after eating something I understood what you mean :) Yes you are right, with the proposed change the initialValue would be called much more often if it returns (after longish compilation or in the trivial case) null. Not sure if this is a loss compared to not constructing the entry. Just some background, in the legacy code I have seen was a non-extended ThreadLocal() object bound to a instance variable. So this makes the situation even worse. Constructing ThreadLocal and not using get() is however not a problem as it does not register the map. So besides rewriting the code (to use a static) one approch of mine was to delay using set(). But as mentioned below, thats not possible when get() constructs the whole shebang. Gruss Bernd Am Tue, 18 Nov 2014 15:59:19 +0100 schrieb Bernd Eckenfels : > Am Tue, 18 Nov 2014 11:37:34 +0000 > schrieb Tom Hawtin : > > > On 18/11/2014 05:11, Bernd Eckenfels wrote: > > > > > Unfortunatelly the ThreadLocal#get() will call #setInitialValue() > > > in all cases, and this will in turn call createMap(). > > > setInitialValue() could avoid to call createMap() when value is > > > null. This would reduce the number of created thread specific > > > maps and entries (and also weak references to the thread). > > > > There's no guarantee that the execution of initialValue was trivial > > even if it returns null. > > Yes, but I dont think it has to be. As long as the initial value is > null it can be represented without any thread local map or entry. And > as there is no "hasEntry()" a application can only call get to probe > for existence (and this will in the current implementation always > created it). > > So another option (but more complicated as it adds a new API function) > would be to have a hasEntry() (or similiar named) function which does > the same as get without calling the initialValue(). This would in this > case really differ for non-trivial initialValue() implementations, > thats why I would prefer to do it in get(). It needs to be checked if > get() gets too big for inlining, but the new if is not in a hot path. > > Gruss > Bernd > > > > > > I don't know whether or not mixing up the execution paths through > > get would harm performance. > > > > Tom > From chris.hegarty at oracle.com Tue Nov 18 15:12:46 2014 From: chris.hegarty at oracle.com (Chris Hegarty) Date: Tue, 18 Nov 2014 15:12:46 +0000 Subject: ThreadLocal with null initial values - avoid create map and entry? In-Reply-To: <20141118061108.0000746a.ecki@zusammenkunft.net> References: <20141118061108.0000746a.ecki@zusammenkunft.net> Message-ID: <546B61EE.3050206@oracle.com> On 18/11/14 05:11, Bernd Eckenfels wrote: > .... > What do you think? (code from OpenJDK 8:) > > 179 private T setInitialValue() { > 180 T value = initialValue(); > 181 Thread t = Thread.currentThread(); > 182 ThreadLocalMap map = getMap(t); > 183 if (map != null) > 184 map.set(this, value); > 185 else > +++ if (value != null) > 186 createMap(t, value); > 187 return value; > 188 } Interesting idea. If this was to be the case woultn't multiple get() invocations cause multiple initialValue() invocations? I think storing the null is important to obey the initialValue / remove contract, i.e. initialValue method is invoked at most once per thread, but it may be invoked again in case of subsequent invocations of remove() followed by get(). -Chris. From ecki at zusammenkunft.net Tue Nov 18 15:14:04 2014 From: ecki at zusammenkunft.net (Bernd Eckenfels) Date: Tue, 18 Nov 2014 16:14:04 +0100 Subject: ThreadLocal with null initial values - avoid create map and entry? In-Reply-To: <546B61EE.3050206@oracle.com> References: <20141118061108.0000746a.ecki@zusammenkunft.net> <546B61EE.3050206@oracle.com> Message-ID: <20141118161404.00003b0e.ecki@zusammenkunft.net> Am Tue, 18 Nov 2014 15:12:46 +0000 schrieb Chris Hegarty : > Interesting idea. If this was to be the case woultn't multiple get() > invocations cause multiple initialValue() invocations? > > I think storing the null is important to obey the initialValue / > remove contract, i.e. initialValue method is invoked at most once per > thread, but it may be invoked again in case of subsequent invocations > of remove() followed by get(). Yes, either the contract has to change, or the empty default implementation of initial value is changed to not return null but INITIAL. final static protected Object INITIAL = new Object(); initialValue() { return INITIAL; } ... if (value != INITIAL) createMap(); This way a overwritten initialValue() is still called only after remove() and only for a unaltered ThreadLocal the optimization applies. Gruss Bernd From chris.hegarty at oracle.com Tue Nov 18 15:36:09 2014 From: chris.hegarty at oracle.com (Chris Hegarty) Date: Tue, 18 Nov 2014 15:36:09 +0000 Subject: ThreadLocal with null initial values - avoid create map and entry? In-Reply-To: <20141118161404.00003b0e.ecki@zusammenkunft.net> References: <20141118061108.0000746a.ecki@zusammenkunft.net> <546B61EE.3050206@oracle.com> <20141118161404.00003b0e.ecki@zusammenkunft.net> Message-ID: <546B6769.7060009@oracle.com> On 18/11/14 15:14, Bernd Eckenfels wrote: > Am Tue, 18 Nov 2014 15:12:46 +0000 > schrieb Chris Hegarty : > >> Interesting idea. If this was to be the case woultn't multiple get() >> invocations cause multiple initialValue() invocations? >> >> I think storing the null is important to obey the initialValue / >> remove contract, i.e. initialValue method is invoked at most once per >> thread, but it may be invoked again in case of subsequent invocations >> of remove() followed by get(). > > Yes, either the contract has to change, or the empty > default implementation of initial value is changed to not return null > but INITIAL. Yes, that was my initial thought too, but unfortunately initialValue specifies that the default implementation returns null :-( So it would require a specification change too, and it might be surprising to a number of existing applications that rely on this default behavior. -Chris. > final static protected Object INITIAL = new Object(); > initialValue() { return INITIAL; } > ... > if (value != INITIAL) > createMap(); > > This way a overwritten initialValue() is still called only after > remove() and only for a unaltered ThreadLocal the optimization applies. > > Gruss > Bernd > From martinrb at google.com Tue Nov 18 16:02:55 2014 From: martinrb at google.com (Martin Buchholz) Date: Tue, 18 Nov 2014 08:02:55 -0800 Subject: RFR 9: 8064932: test java/lang/ProcessBuilder/Basic.java: waitFor didn't take long enough In-Reply-To: <546AD2CD.90300@oracle.com> References: <54665C9D.1010903@oracle.com> <54667EEC.9060002@oracle.com> <546A4C2A.4070208@oracle.com> <546A7082.5060302@oracle.com> <546AAAC2.6080009@oracle.com> <546AD2CD.90300@oracle.com> Message-ID: On Mon, Nov 17, 2014 at 9:02 PM, David Holmes wrote: > Hi Martin, > > On 18/11/2014 2:49 PM, Martin Buchholz wrote: >> >> Hi David, >> >> I'm still hoping to persuade you (and others) to fix the long-standing >> return-early bug in Object.wait. > > > Are you referring to the low level mechanics or specifically to the > nanosecond version below? Here I'm just trying to fix the problem that Object.wait(millis,nanos) trivially returns early. > That seems overly complex - after checking for valid values of timeout and > nanos you simply need: > > if (nanos > 0) timeout++; > > to ensure the >= requirement. Sure, the miminal diff is: diff --git a/src/java.base/share/classes/java/lang/Object.java b/src/java.base/share/classes/java/lang/Object.java --- a/src/java.base/share/classes/java/lang/Object.java +++ b/src/java.base/share/classes/java/lang/Object.java @@ -453,7 +453,7 @@ "nanosecond timeout value out of range"); } - if (nanos >= 500000 || (nanos != 0 && timeout == 0)) { + if (nanos != 0) { timeout++; } > But seriously we should just deprecate this version as we're not likely to > get any underlying OS mechanisms for doing nanosecond resolution timed > blocking actions. Object.wait(millis, nanos) is an ugly API (I wish for both a public and hotspot internal API that simply operated on nanoseconds), but it's not totally awful. It's not obvious to me that computing advances over the lifetime of the java platform won't allow sub-millisecond scheduling, and other APIs that do operate on nanoseconds (like Process.waitFor) should call this API. From ecki at zusammenkunft.net Tue Nov 18 16:07:06 2014 From: ecki at zusammenkunft.net (Bernd Eckenfels) Date: Tue, 18 Nov 2014 17:07:06 +0100 Subject: ThreadLocal with null initial values - avoid create map and entry? In-Reply-To: <546B6769.7060009@oracle.com> References: <20141118061108.0000746a.ecki@zusammenkunft.net> <546B61EE.3050206@oracle.com> <20141118161404.00003b0e.ecki@zusammenkunft.net> <546B6769.7060009@oracle.com> Message-ID: <20141118170706.000007d1.ecki@zusammenkunft.net> Am Tue, 18 Nov 2014 15:36:09 +0000 schrieb Chris Hegarty : > > Yes, either the contract has to change, or the empty > > default implementation of initial value is changed to not return > > null but INITIAL. > > Yes, that was my initial thought too, but unfortunately initialValue > specifies that the default implementation returns null :-( So it > would require a specification change too, and it might be surprising > to a number of existing applications that rely on this default > behavior. Ok, this would makes the hasEntry() more attractive (meaning, only new and aware applications will have the benefit of saving): /** * Does the current thread local variable exist. *

* This returns false if the thread local was never set * or @{link #remove() removed}. The next @{link#get()} * would call @{link #initialValue()} or the and create a map * and entry. *

* If true is retruned the entry was initialized or set, * but the value might be null. *

* This can be used before calling @{link #get} to further * delay initialisation (especially when working with null as initial * value). */ public boolean hasEntry() { Thread t = Thread.currentThread(); ThreadLocalMap map = getMap(t); if (map == null) return false; ThreadLocalMap.Entry e = map.getEntry(this); return (e != null); } (Name is u for discussion, I prefer hasEntry over hasValue as it makes it clearer that an entry itself might exist but have a null value. Another common name is I guess exists() with the same issue. Gruss Bernd From martinrb at google.com Tue Nov 18 16:08:33 2014 From: martinrb at google.com (Martin Buchholz) Date: Tue, 18 Nov 2014 08:08:33 -0800 Subject: RFR 9: 8064932: test java/lang/ProcessBuilder/Basic.java: waitFor didn't take long enough In-Reply-To: <546AD0F2.9040002@oracle.com> References: <54665C9D.1010903@oracle.com> <54667EEC.9060002@oracle.com> <546A4C2A.4070208@oracle.com> <546A7082.5060302@oracle.com> <546AD0F2.9040002@oracle.com> Message-ID: On Mon, Nov 17, 2014 at 8:54 PM, David Holmes wrote: > On 18/11/2014 2:43 PM, Martin Buchholz wrote: >> >> Proposed sibling change >> http://cr.openjdk.java.net/~martin/webrevs/openjdk9/UNIXProcess.waitFor/ >> - don't unconditionally call nanoTime when the wait ends >> - use the millis/nanos form of Object.wait in case sub-millisecond >> waits are ever supported. > > > I don't really see the point of adding the extra math, plus an extra call > (the two arg wait will call the one arg version) for no actual gain. The idea was to code to the Object.wait API, not its current implementation with only millisecond resolution. Even if we code to the current implementation, we should round UP not down, to avoid the problem of needing to call wait twice in case of early return. That would also be progress. > If you want to add a fast exit path that's fine but the rest seems > superfluous to me. From huizhe.wang at oracle.com Tue Nov 18 19:05:26 2014 From: huizhe.wang at oracle.com (huizhe wang) Date: Tue, 18 Nov 2014 11:05:26 -0800 Subject: Review request for XML JAXP unit test colocation In-Reply-To: <009601cfff31$631da000$2958e000$@oracle.com> References: <013701cff8d8$9acf3950$d06dabf0$@oracle.com> <009601cfff31$631da000$2958e000$@oracle.com> Message-ID: <546B9876.1070206@oracle.com> Hi Frank, The webrev is updated and looks good now. Thanks for the great work! Joe On 11/13/2014 3:02 AM, Frank Yuan wrote: > Hi Joe and All > > I revised the code based on latest comments and put the webrev on > http://cr.openjdk.java.net/~joehw/jdk9/test/Frank/8043090/webrev/ > > Best Regards > Frank > > -----Original Message----- > From: Frank Yuan [mailto:frank.yuan at oracle.com] > Sent: Wednesday, November 05, 2014 5:12 PM > To: 'huizhe wang'; 'Core-Libs-Dev' > Cc: 'jibing chen'; 'Gustavo Galimberti'; sandeep.konchady at oracle.com > Subject: Review request for XML JAXP unit test colocation > > Hi, Joe and All > > Thanks for your previous reviews and comments. Per your comments: I added > description for every test, got them running with and without security > manager and fixed the known test issues. Could you help review the changes > for JAXP unittest co-location again? After your review, I will push the > tests into JDK repo: jaxp/test. > > bug: https://bugs.openjdk.java.net/browse/JDK-8043090 > webrev: > > > Thanks, > > Frank > From peter.levart at gmail.com Tue Nov 18 19:15:37 2014 From: peter.levart at gmail.com (Peter Levart) Date: Tue, 18 Nov 2014 20:15:37 +0100 Subject: ThreadLocal with null initial values - avoid create map and entry? In-Reply-To: <20141118160911.000045b1.ecki@zusammenkunft.net> References: <20141118061108.0000746a.ecki@zusammenkunft.net> <546B2F7E.10206@oracle.com> <20141118155919.000000ff.ecki@zusammenkunft.net> <20141118160911.000045b1.ecki@zusammenkunft.net> Message-ID: <546B9AD9.3000307@gmail.com> Hi Bernd, If you can swap the default ThreadLocal with your own implementation (a subclass of ThreadLocal), you can make it behave like you would like it to (It's a hack, I know ;-) public class XThreadLocal extends ThreadLocal { private static class Null extends RuntimeException { static final Null INSTANCE = new Null(); private Null() { super(null, null, false, false); } } @Override protected T initialValue() { throw Null.INSTANCE; } @Override public T get() { try { return super.get(); } catch (Null n) { return null; } } } Regards, Peter On 11/18/2014 04:09 PM, Bernd Eckenfels wrote: > Hello, > > > after eating something I understood what you mean :) Yes you are right, > with the proposed change the initialValue would be called much more > often if it returns (after longish compilation or in the trivial case) > null. > > Not sure if this is a loss compared to not constructing the entry. > > Just some background, in the legacy code I have seen was a non-extended > ThreadLocal() object bound to a instance variable. So this makes the > situation even worse. Constructing ThreadLocal and not using get() is > however not a problem as it does not register the map. So besides > rewriting the code (to use a static) one approch of mine was to delay > using set(). But as mentioned below, thats not possible when get() > constructs the whole shebang. > > Gruss > Bernd > > > Am Tue, 18 Nov 2014 > 15:59:19 +0100 schrieb Bernd Eckenfels : > >> Am Tue, 18 Nov 2014 11:37:34 +0000 >> schrieb Tom Hawtin : >> >>> On 18/11/2014 05:11, Bernd Eckenfels wrote: >>> >>>> Unfortunatelly the ThreadLocal#get() will call #setInitialValue() >>>> in all cases, and this will in turn call createMap(). >>>> setInitialValue() could avoid to call createMap() when value is >>>> null. This would reduce the number of created thread specific >>>> maps and entries (and also weak references to the thread). >>> There's no guarantee that the execution of initialValue was trivial >>> even if it returns null. >> Yes, but I dont think it has to be. As long as the initial value is >> null it can be represented without any thread local map or entry. And >> as there is no "hasEntry()" a application can only call get to probe >> for existence (and this will in the current implementation always >> created it). >> >> So another option (but more complicated as it adds a new API function) >> would be to have a hasEntry() (or similiar named) function which does >> the same as get without calling the initialValue(). This would in this >> case really differ for non-trivial initialValue() implementations, >> thats why I would prefer to do it in get(). It needs to be checked if >> get() gets too big for inlining, but the new if is not in a hot path. >> >> Gruss >> Bernd >> >> >>> I don't know whether or not mixing up the execution paths through >>> get would harm performance. >>> >>> Tom From roger.riggs at oracle.com Tue Nov 18 20:09:34 2014 From: roger.riggs at oracle.com (roger riggs) Date: Tue, 18 Nov 2014 15:09:34 -0500 Subject: RFR 9: 8064932: test java/lang/ProcessBuilder/Basic.java: waitFor didn't take long enough In-Reply-To: References: <54665C9D.1010903@oracle.com> <54667EEC.9060002@oracle.com> <546A4C2A.4070208@oracle.com> <546A7082.5060302@oracle.com> <546AD0F2.9040002@oracle.com> Message-ID: <546BA77E.4030505@oracle.com> Hi, Work on Object.wait is an issue to be taken up separately. I agree that the timeout values should not be truncated to milliseconds, likely causing an additional cycle through the while loop and waiting. The Process.waitFor methods are expected to wait for the specified time to elapse. The webrev has been updated for both Windows and Unix to use a ceiling function on milliseconds and ensure that at least the requested time has elapsed. Please review: http://cr.openjdk.java.net/~rriggs/webrev-basic-notenough-8064932/ Thanks, Roger On 11/18/2014 11:08 AM, Martin Buchholz wrote: > On Mon, Nov 17, 2014 at 8:54 PM, David Holmes wrote: >> On 18/11/2014 2:43 PM, Martin Buchholz wrote: >>> Proposed sibling change >>> http://cr.openjdk.java.net/~martin/webrevs/openjdk9/UNIXProcess.waitFor/ >>> - don't unconditionally call nanoTime when the wait ends >>> - use the millis/nanos form of Object.wait in case sub-millisecond >>> waits are ever supported. >> >> I don't really see the point of adding the extra math, plus an extra call >> (the two arg wait will call the one arg version) for no actual gain. > The idea was to code to the Object.wait API, not its current > implementation with only millisecond resolution. > Even if we code to the current implementation, we should round UP not down, > to avoid the problem of needing to call wait twice in case of early return. > That would also be progress. > >> If you want to add a fast exit path that's fine but the rest seems >> superfluous to me. From vladimir.x.ivanov at oracle.com Tue Nov 18 20:23:05 2014 From: vladimir.x.ivanov at oracle.com (Vladimir Ivanov) Date: Wed, 19 Nov 2014 00:23:05 +0400 Subject: [9, 8u40] RFR (M): 8063135: Enable full LF sharing by default Message-ID: <546BAAA9.8090009@oracle.com> http://cr.openjdk.java.net/~vlivanov/8063135/webrev.00/ https://bugs.openjdk.java.net/browse/JDK-8063135 Turn on full LambdaForm sharing and get rid of custom LambdaForm shapes. 8059877 [1] eliminated most of the regressions caused by LF sharing. The rest will be addressed separately [2]. Testing: jck (api/java_lang/invoke), jdk/java/lang/invoke, jdk/java/util/streams, octane Thanks! Best regards, Vladimir Ivanov [1] https://bugs.openjdk.java.net/browse/JDK-8059877 [2] https://bugs.openjdk.java.net/browse/JDK-8063137 From martinrb at google.com Tue Nov 18 21:58:13 2014 From: martinrb at google.com (Martin Buchholz) Date: Tue, 18 Nov 2014 13:58:13 -0800 Subject: RFR 9: 8064932: test java/lang/ProcessBuilder/Basic.java: waitFor didn't take long enough In-Reply-To: <546BA77E.4030505@oracle.com> References: <54665C9D.1010903@oracle.com> <54667EEC.9060002@oracle.com> <546A4C2A.4070208@oracle.com> <546A7082.5060302@oracle.com> <546AD0F2.9040002@oracle.com> <546BA77E.4030505@oracle.com> Message-ID: I think we're all 3 in agreement on the general direction, even if we disagree on the details. Once we properly round to millis, it is no longer necessary to use Math.max - just wait(NANOSECONDS.toMillis(rem + 999_999L)); As in my proposed change, I would like every call to wait immediately followed by a check if (hasExited) return true; to avoid the extra call to nanoTime. On Tue, Nov 18, 2014 at 12:09 PM, roger riggs wrote: > Hi, > > Work on Object.wait is an issue to be taken up separately. > > I agree that the timeout values should not be truncated to milliseconds, > likely > causing an additional cycle through the while loop and waiting. > The Process.waitFor methods are expected to wait for the specified time to > elapse. > The webrev has been updated for both Windows and Unix to use a ceiling > function > on milliseconds and ensure that at least the requested time has elapsed. > > Please review: > http://cr.openjdk.java.net/~rriggs/webrev-basic-notenough-8064932/ > > Thanks, Roger > > > > > On 11/18/2014 11:08 AM, Martin Buchholz wrote: >> >> On Mon, Nov 17, 2014 at 8:54 PM, David Holmes >> wrote: >>> >>> On 18/11/2014 2:43 PM, Martin Buchholz wrote: >>>> >>>> Proposed sibling change >>>> http://cr.openjdk.java.net/~martin/webrevs/openjdk9/UNIXProcess.waitFor/ >>>> - don't unconditionally call nanoTime when the wait ends >>>> - use the millis/nanos form of Object.wait in case sub-millisecond >>>> waits are ever supported. >>> >>> >>> I don't really see the point of adding the extra math, plus an extra call >>> (the two arg wait will call the one arg version) for no actual gain. >> >> The idea was to code to the Object.wait API, not its current >> implementation with only millisecond resolution. >> Even if we code to the current implementation, we should round UP not >> down, >> to avoid the problem of needing to call wait twice in case of early >> return. >> That would also be progress. >> >>> If you want to add a fast exit path that's fine but the rest seems >>> superfluous to me. > > From vladimir.x.ivanov at oracle.com Tue Nov 18 21:01:54 2014 From: vladimir.x.ivanov at oracle.com (Vladimir Ivanov) Date: Wed, 19 Nov 2014 01:01:54 +0400 Subject: [9, 8u40] RFR (XXS): 8059880: Get rid of LambdaForm interpretation Message-ID: <546BB3C2.60202@oracle.com> http://cr.openjdk.java.net/~vlivanov/8059880/webrev.00/ https://bugs.openjdk.java.net/browse/JDK-8059880 Once full LambdaForm sharing is on by default, amount of instantiated LambdaForms drastically reduces. On Octane it varies from 1k to 3k depending on the subbenchmark (was up to 300k before LF sharing-related changes). It allows to skip interpretation step without inducing slowdown at startup. Every LambdaForm is precompiled once it is instantiated. Avoiding LF interpretation significantly reduces worst-case stack usage (4-5 stack frames per interpreted LF to 1 stack frame per compiled LF). The fix is to set COMPILE_THRESHOLD down to 0. It triggers LF precompilation for every LF. Testing: jck (api/java_lang/invoke), jdk/java/lang/invoke, jdk/java/util/streams, octane Thanks! Best regards, Vladimir Ivanov From roger.riggs at oracle.com Tue Nov 18 22:28:37 2014 From: roger.riggs at oracle.com (roger riggs) Date: Tue, 18 Nov 2014 17:28:37 -0500 Subject: RFR 9: 8064932: test java/lang/ProcessBuilder/Basic.java: waitFor didn't take long enough In-Reply-To: References: <54665C9D.1010903@oracle.com> <54667EEC.9060002@oracle.com> <546A4C2A.4070208@oracle.com> <546A7082.5060302@oracle.com> <546AD0F2.9040002@oracle.com> <546BA77E.4030505@oracle.com> Message-ID: <546BC815.10107@oracle.com> Hi, Done, added the test and immediate return after the wait. On Windows, the Thread.interrupted check needs to be done first to be consistent with Unix. Refactored a bit to remove a local variable and extra arithmetic computing the remainingNanos. Updated: http://cr.openjdk.java.net/~rriggs/webrev-basic-notenough-8064932/ Roger On 11/18/2014 4:58 PM, Martin Buchholz wrote: > I think we're all 3 in agreement on the general direction, even if we > disagree on the details. > > Once we properly round to millis, it is no longer necessary to use > Math.max - just > > wait(NANOSECONDS.toMillis(rem + 999_999L)); > > As in my proposed change, I would like every call to wait immediately > followed by a check > if (hasExited) > return true; > > to avoid the extra call to nanoTime. > > On Tue, Nov 18, 2014 at 12:09 PM, roger riggs wrote: >> Hi, >> >> Work on Object.wait is an issue to be taken up separately. >> >> I agree that the timeout values should not be truncated to milliseconds, >> likely >> causing an additional cycle through the while loop and waiting. >> The Process.waitFor methods are expected to wait for the specified time to >> elapse. >> The webrev has been updated for both Windows and Unix to use a ceiling >> function >> on milliseconds and ensure that at least the requested time has elapsed. >> >> Please review: >> http://cr.openjdk.java.net/~rriggs/webrev-basic-notenough-8064932/ >> >> Thanks, Roger >> >> >> >> >> On 11/18/2014 11:08 AM, Martin Buchholz wrote: >>> On Mon, Nov 17, 2014 at 8:54 PM, David Holmes >>> wrote: >>>> On 18/11/2014 2:43 PM, Martin Buchholz wrote: >>>>> Proposed sibling change >>>>> http://cr.openjdk.java.net/~martin/webrevs/openjdk9/UNIXProcess.waitFor/ >>>>> - don't unconditionally call nanoTime when the wait ends >>>>> - use the millis/nanos form of Object.wait in case sub-millisecond >>>>> waits are ever supported. >>>> >>>> I don't really see the point of adding the extra math, plus an extra call >>>> (the two arg wait will call the one arg version) for no actual gain. >>> The idea was to code to the Object.wait API, not its current >>> implementation with only millisecond resolution. >>> Even if we code to the current implementation, we should round UP not >>> down, >>> to avoid the problem of needing to call wait twice in case of early >>> return. >>> That would also be progress. >>> >>>> If you want to add a fast exit path that's fine but the rest seems >>>> superfluous to me. >> From aleksey.shipilev at oracle.com Tue Nov 18 23:33:53 2014 From: aleksey.shipilev at oracle.com (Aleksey Shipilev) Date: Wed, 19 Nov 2014 02:33:53 +0300 Subject: [9, 8u40] RFR (XXS): 8059880: Get rid of LambdaForm interpretation In-Reply-To: <546BB3C2.60202@oracle.com> References: <546BB3C2.60202@oracle.com> Message-ID: <546BD761.6030108@oracle.com> On 11/19/2014 12:01 AM, Vladimir Ivanov wrote: > http://cr.openjdk.java.net/~vlivanov/8059880/webrev.00/ > https://bugs.openjdk.java.net/browse/JDK-8059880 Yes, for the love of God, GO FOR IT. -Aleksey. From aleksey.shipilev at oracle.com Tue Nov 18 23:38:00 2014 From: aleksey.shipilev at oracle.com (Aleksey Shipilev) Date: Wed, 19 Nov 2014 02:38:00 +0300 Subject: [9, 8u40] RFR (M): 8063135: Enable full LF sharing by default In-Reply-To: <546BAAA9.8090009@oracle.com> References: <546BAAA9.8090009@oracle.com> Message-ID: <546BD858.5040206@oracle.com> On 11/18/2014 11:23 PM, Vladimir Ivanov wrote: > http://cr.openjdk.java.net/~vlivanov/8063135/webrev.00/ > https://bugs.openjdk.java.net/browse/JDK-8063135 Broken array index here: 69 TRACE_METHOD_LINKAGE = (Boolean) values[3]; 70 COMPILE_THRESHOLD = (Integer) values[5]; Also, how does it pass the tests? It should fail with AIOBE during MethodHandleStatics::, since 53 final Object[] values = new Object[7]; ... 72 PROFILE_LEVEL = (Integer) values[7]; Thanks, -Aleksey. From martinrb at google.com Wed Nov 19 00:42:10 2014 From: martinrb at google.com (Martin Buchholz) Date: Tue, 18 Nov 2014 16:42:10 -0800 Subject: RFR: 8065172: More core reflection final and volatile annotations Message-ID: Hi Joe, Peter, Paul This is the followup on thread safety I promised Peter. As I said before, I'm willing to take the risk to use fields written via a race if we have a reflective test proving it correct, and the bug contains the 0.1 version of such a test. https://bugs.openjdk.java.net/browse/JDK-8065172 http://cr.openjdk.java.net/~martin/webrevs/openjdk9/core-reflection-more-safety/ From david.holmes at oracle.com Wed Nov 19 02:26:44 2014 From: david.holmes at oracle.com (David Holmes) Date: Wed, 19 Nov 2014 12:26:44 +1000 Subject: RFR 9: 8064932: test java/lang/ProcessBuilder/Basic.java: waitFor didn't take long enough In-Reply-To: <546BC815.10107@oracle.com> References: <54665C9D.1010903@oracle.com> <54667EEC.9060002@oracle.com> <546A4C2A.4070208@oracle.com> <546A7082.5060302@oracle.com> <546AD0F2.9040002@oracle.com> <546BA77E.4030505@oracle.com> <546BC815.10107@oracle.com> Message-ID: <546BFFE4.40000@oracle.com> Hi Roger, On 19/11/2014 8:28 AM, roger riggs wrote: > Hi, > > Done, added the test and immediate return after the wait. > On Windows, the Thread.interrupted check needs to be done first to be > consistent with Unix. > Refactored a bit to remove a local variable and extra arithmetic > computing the remainingNanos. > > Updated: > http://cr.openjdk.java.net/~rriggs/webrev-basic-notenough-8064932/ Can you add a comment before all TimeUnit.NANOSECONDS.toMillis(remainingNanos + 999_999L) to indicate it is rounding up the millis value. Thanks, David > Roger > > > > On 11/18/2014 4:58 PM, Martin Buchholz wrote: >> I think we're all 3 in agreement on the general direction, even if we >> disagree on the details. >> >> Once we properly round to millis, it is no longer necessary to use >> Math.max - just >> >> wait(NANOSECONDS.toMillis(rem + 999_999L)); >> >> As in my proposed change, I would like every call to wait immediately >> followed by a check >> if (hasExited) >> return true; >> >> to avoid the extra call to nanoTime. >> >> On Tue, Nov 18, 2014 at 12:09 PM, roger riggs >> wrote: >>> Hi, >>> >>> Work on Object.wait is an issue to be taken up separately. >>> >>> I agree that the timeout values should not be truncated to milliseconds, >>> likely >>> causing an additional cycle through the while loop and waiting. >>> The Process.waitFor methods are expected to wait for the specified >>> time to >>> elapse. >>> The webrev has been updated for both Windows and Unix to use a ceiling >>> function >>> on milliseconds and ensure that at least the requested time has elapsed. >>> >>> Please review: >>> http://cr.openjdk.java.net/~rriggs/webrev-basic-notenough-8064932/ >>> >>> Thanks, Roger >>> >>> >>> >>> >>> On 11/18/2014 11:08 AM, Martin Buchholz wrote: >>>> On Mon, Nov 17, 2014 at 8:54 PM, David Holmes >>>> wrote: >>>>> On 18/11/2014 2:43 PM, Martin Buchholz wrote: >>>>>> Proposed sibling change >>>>>> http://cr.openjdk.java.net/~martin/webrevs/openjdk9/UNIXProcess.waitFor/ >>>>>> >>>>>> - don't unconditionally call nanoTime when the wait ends >>>>>> - use the millis/nanos form of Object.wait in case sub-millisecond >>>>>> waits are ever supported. >>>>> >>>>> I don't really see the point of adding the extra math, plus an >>>>> extra call >>>>> (the two arg wait will call the one arg version) for no actual gain. >>>> The idea was to code to the Object.wait API, not its current >>>> implementation with only millisecond resolution. >>>> Even if we code to the current implementation, we should round UP not >>>> down, >>>> to avoid the problem of needing to call wait twice in case of early >>>> return. >>>> That would also be progress. >>>> >>>>> If you want to add a fast exit path that's fine but the rest seems >>>>> superfluous to me. >>> > From david.holmes at oracle.com Wed Nov 19 02:29:16 2014 From: david.holmes at oracle.com (David Holmes) Date: Wed, 19 Nov 2014 12:29:16 +1000 Subject: RFR 9: 8064932: test java/lang/ProcessBuilder/Basic.java: waitFor didn't take long enough In-Reply-To: References: <54665C9D.1010903@oracle.com> <54667EEC.9060002@oracle.com> <546A4C2A.4070208@oracle.com> <546A7082.5060302@oracle.com> <546AAAC2.6080009@oracle.com> <546AD2CD.90300@oracle.com> Message-ID: <546C007C.7050301@oracle.com> On 19/11/2014 2:02 AM, Martin Buchholz wrote: > On Mon, Nov 17, 2014 at 9:02 PM, David Holmes wrote: >> Hi Martin, >> >> On 18/11/2014 2:49 PM, Martin Buchholz wrote: >>> >>> Hi David, >>> >>> I'm still hoping to persuade you (and others) to fix the long-standing >>> return-early bug in Object.wait. >> >> >> Are you referring to the low level mechanics or specifically to the >> nanosecond version below? > > Here I'm just trying to fix the problem that Object.wait(millis,nanos) > trivially returns early. > >> That seems overly complex - after checking for valid values of timeout and >> nanos you simply need: >> >> if (nanos > 0) timeout++; >> >> to ensure the >= requirement. > > Sure, the miminal diff is: > > diff --git a/src/java.base/share/classes/java/lang/Object.java > b/src/java.base/share/classes/java/lang/Object.java > --- a/src/java.base/share/classes/java/lang/Object.java > +++ b/src/java.base/share/classes/java/lang/Object.java > @@ -453,7 +453,7 @@ > "nanosecond timeout value out of range"); > } > > - if (nanos >= 500000 || (nanos != 0 && timeout == 0)) { > + if (nanos != 0) { > timeout++; > } I can live with that. Though given spurious wakeups it doesn't add much. >> But seriously we should just deprecate this version as we're not likely to >> get any underlying OS mechanisms for doing nanosecond resolution timed >> blocking actions. > > Object.wait(millis, nanos) is an ugly API (I wish for both a public > and hotspot internal API that simply operated on nanoseconds), but > it's not totally awful. It's not obvious to me that computing > advances over the lifetime of the java platform won't allow > sub-millisecond scheduling, and other APIs that do operate on > nanoseconds (like Process.waitFor) should call this API. I'm sure the same thing was said 20 years ago, yet the OS seem stuck at microsecond resolution. Maybe one day ... David From stanimir at riflexo.com Wed Nov 19 07:36:19 2014 From: stanimir at riflexo.com (Stanimir Simeonoff) Date: Wed, 19 Nov 2014 09:36:19 +0200 Subject: RFR: 8065172: More core reflection final and volatile annotations In-Reply-To: References: Message-ID: Hi, Signature hashCode (the static class in MethodUtil) should be eagerly calculated (i.e. a final field too) since the only usage is being a key in the hashmap. I will add bit more later. Stanimir On Wed, Nov 19, 2014 at 2:42 AM, Martin Buchholz wrote: > Hi Joe, Peter, Paul > > This is the followup on thread safety I promised Peter. > As I said before, I'm willing to take the risk to use fields written > via a race if we have a reflective test proving it correct, and the > bug contains the 0.1 version of such a test. > > https://bugs.openjdk.java.net/browse/JDK-8065172 > > http://cr.openjdk.java.net/~martin/webrevs/openjdk9/core-reflection-more-safety/ > From chris.plummer at oracle.com Wed Nov 19 08:49:10 2014 From: chris.plummer at oracle.com (Chris Plummer) Date: Wed, 19 Nov 2014 00:49:10 -0800 Subject: [9] RFR (S) 6762191: Setting stack size to 16K causes segmentation fault In-Reply-To: <546BC35E.4070402@oracle.com> References: <545D939D.2030308@oracle.com> <5463B896.10801@oracle.com> <54641ADE.8030504@oracle.com> <546BC35E.4070402@oracle.com> Message-ID: <546C5986.6010500@oracle.com> I've update the webrev to add STACK_SIZE_MINIMUM in place of the 32k references, and also moved the test from hotspot/test/runtime to jdk/test/tools/launcher as David requested. That required some adjustments to the test script, since test_env.sh does not exist in jdk/test, so I had to pull in the bits I needed into the script. http://cr.openjdk.java.net/~cjplummer/6762191/webrev.01/ I still need to rerun through JPRT. I'll do so once there are no more suggested changes. thanks, Chris On 11/18/14 2:08 PM, Chris Plummer wrote: > Adding core-libs-dev at openjdk.java.net, since one of the changes is in > java.c. > > Chris > > On 11/12/14 6:43 PM, David Holmes wrote: >> Hi Chris, >> >> Sorry for the delay. >> >> On 13/11/2014 5:44 AM, Chris Plummer wrote: >>> Hi, >>> >>> I'm still looking for reviewers. >> >> As the change is to the launcher it needs to be reviewed by the >> launcher owner - which I think is serviceability (though also cc'd >> Kumar :) ). >> >> Launcher change, and your rationale, seems okay to me. I'd probably >> put the test in to jdk/test/tools/launcher/ though. >> >> Thanks, >> David >> >>> thanks, >>> >>> Chris >>> >>> On 11/7/14 7:53 PM, Chris Plummer wrote: >>>> This is an initial review for 6762191. I'm guessing there will be >>>> recommendations to fix in a different way, but thought this would be a >>>> good time to start the discussion. >>>> >>>> https://bugs.openjdk.java.net/browse/JDK-6762191 >>>> http://cr.openjdk.java.net/~cjplummer/6762191/webrev.00.jdk/ >>>> http://cr.openjdk.java.net/~cjplummer/6762191/webrev.00.hotspot/ >>>> >>>> The bug is that if the -Xss size is set to something very small (like >>>> 16k), on linux there will be a crash due to overwriting the end of the >>>> stack. This happens before hotspot can compute its stack needs and >>>> verify that the stack is big enough. >>>> >>>> It didn't seem viable to move the hotspot stack size check earlier. It >>>> depends on too much other work done before that point, and the changes >>>> would have been disruptive. The stack size check is currently done in >>>> os::init_2(). >>>> >>>> What is needed is a check before the thread is created. That way we >>>> can create a thread with a big enough stack to handle all needs up to >>>> the point of the check in os::init_2(). This initial check does not >>>> need to be the final check. It just needs to confirm that we have >>>> enough stack to get us to the check in os::init_2(). >>>> >>>> I decided to check in java.c if the -Xss size is too small, and set it >>>> to a larger size if it is. I hard coded this size to 32k (I'll explain >>>> why 32k later). I suspect this is the part that will result in some >>>> debate. If you have better suggestions let me know. If it does stay >>>> here, then probably the 32k needs to be a #define, and maybe even an >>>> OS porting interface, but I'm not sure where to put it. >>>> >>>> The reason I chose 32k is because this is big enough for all platforms >>>> to get to the stack size check in os::init_2(). It is also smaller >>>> than the actual minimum stack size allowed on any platform. 32-bit >>>> windows has the smallest requirement at 64k. I add some printfs to >>>> print the minimum stack requirement, and then ran a simple JTReg test >>>> with every JPRT supported platform to get the results. >>>> >>>> The TooSmallStackSize.sh will run "java -version" with -Xss16k, >>>> -Xss32k, and -XXss, where is the size from the >>>> error message produced by the JVM, such as in the following: >>>> >>>> $ java -Xss32k -version >>>> The stack size specified is too small, Specify at least 100k >>>> Error: Could not create the Java Virtual Machine. >>>> Error: A fatal exception has occurred. Program will exit. >>>> >>>> I ran this test through JPRT on all platforms, and they all pass. >>>> >>>> One thing to point out is that Windows behaves a bit different than >>>> the other platforms. It always rounds the stack size up to a multiple >>>> of 64k , so even if you specify -Xss16k, you get a 64k stack. On >>>> 32-bit Windows with C1, 64k is also the minimum requirement, so there >>>> is no error produced in this case. However, on 32-bit Windows with C2, >>>> 68k is the minimum, so an error is produced since the stack will only >>>> be 64k. There is no bug here. It's just a bit confusing. >>>> >>>> thanks, >>>> >>>> Chris >>> > From serguei.spitsyn at oracle.com Wed Nov 19 08:54:48 2014 From: serguei.spitsyn at oracle.com (serguei.spitsyn at oracle.com) Date: Wed, 19 Nov 2014 00:54:48 -0800 Subject: [9] RFR (S) 6762191: Setting stack size to 16K causes segmentation fault In-Reply-To: <546C5986.6010500@oracle.com> References: <545D939D.2030308@oracle.com> <5463B896.10801@oracle.com> <54641ADE.8030504@oracle.com> <546BC35E.4070402@oracle.com> <546C5986.6010500@oracle.com> Message-ID: <546C5AD8.2090701@oracle.com> Reviewed Thanks, Serguei On 11/19/14 12:49 AM, Chris Plummer wrote: > I've update the webrev to add STACK_SIZE_MINIMUM in place of the 32k > references, and also moved the test from hotspot/test/runtime to > jdk/test/tools/launcher as David requested. That required some > adjustments to the test script, since test_env.sh does not exist in > jdk/test, so I had to pull in the bits I needed into the script. > > http://cr.openjdk.java.net/~cjplummer/6762191/webrev.01/ > > I still need to rerun through JPRT. I'll do so once there are no more > suggested changes. > > thanks, > > Chris > > On 11/18/14 2:08 PM, Chris Plummer wrote: >> Adding core-libs-dev at openjdk.java.net, since one of the changes is in >> java.c. >> >> Chris >> >> On 11/12/14 6:43 PM, David Holmes wrote: >>> Hi Chris, >>> >>> Sorry for the delay. >>> >>> On 13/11/2014 5:44 AM, Chris Plummer wrote: >>>> Hi, >>>> >>>> I'm still looking for reviewers. >>> >>> As the change is to the launcher it needs to be reviewed by the >>> launcher owner - which I think is serviceability (though also cc'd >>> Kumar :) ). >>> >>> Launcher change, and your rationale, seems okay to me. I'd probably >>> put the test in to jdk/test/tools/launcher/ though. >>> >>> Thanks, >>> David >>> >>>> thanks, >>>> >>>> Chris >>>> >>>> On 11/7/14 7:53 PM, Chris Plummer wrote: >>>>> This is an initial review for 6762191. I'm guessing there will be >>>>> recommendations to fix in a different way, but thought this would >>>>> be a >>>>> good time to start the discussion. >>>>> >>>>> https://bugs.openjdk.java.net/browse/JDK-6762191 >>>>> http://cr.openjdk.java.net/~cjplummer/6762191/webrev.00.jdk/ >>>>> http://cr.openjdk.java.net/~cjplummer/6762191/webrev.00.hotspot/ >>>>> >>>>> The bug is that if the -Xss size is set to something very small (like >>>>> 16k), on linux there will be a crash due to overwriting the end of >>>>> the >>>>> stack. This happens before hotspot can compute its stack needs and >>>>> verify that the stack is big enough. >>>>> >>>>> It didn't seem viable to move the hotspot stack size check >>>>> earlier. It >>>>> depends on too much other work done before that point, and the >>>>> changes >>>>> would have been disruptive. The stack size check is currently done in >>>>> os::init_2(). >>>>> >>>>> What is needed is a check before the thread is created. That way we >>>>> can create a thread with a big enough stack to handle all needs up to >>>>> the point of the check in os::init_2(). This initial check does not >>>>> need to be the final check. It just needs to confirm that we have >>>>> enough stack to get us to the check in os::init_2(). >>>>> >>>>> I decided to check in java.c if the -Xss size is too small, and >>>>> set it >>>>> to a larger size if it is. I hard coded this size to 32k (I'll >>>>> explain >>>>> why 32k later). I suspect this is the part that will result in some >>>>> debate. If you have better suggestions let me know. If it does stay >>>>> here, then probably the 32k needs to be a #define, and maybe even an >>>>> OS porting interface, but I'm not sure where to put it. >>>>> >>>>> The reason I chose 32k is because this is big enough for all >>>>> platforms >>>>> to get to the stack size check in os::init_2(). It is also smaller >>>>> than the actual minimum stack size allowed on any platform. 32-bit >>>>> windows has the smallest requirement at 64k. I add some printfs to >>>>> print the minimum stack requirement, and then ran a simple JTReg test >>>>> with every JPRT supported platform to get the results. >>>>> >>>>> The TooSmallStackSize.sh will run "java -version" with -Xss16k, >>>>> -Xss32k, and -XXss, where is the size from the >>>>> error message produced by the JVM, such as in the following: >>>>> >>>>> $ java -Xss32k -version >>>>> The stack size specified is too small, Specify at least 100k >>>>> Error: Could not create the Java Virtual Machine. >>>>> Error: A fatal exception has occurred. Program will exit. >>>>> >>>>> I ran this test through JPRT on all platforms, and they all pass. >>>>> >>>>> One thing to point out is that Windows behaves a bit different than >>>>> the other platforms. It always rounds the stack size up to a multiple >>>>> of 64k , so even if you specify -Xss16k, you get a 64k stack. On >>>>> 32-bit Windows with C1, 64k is also the minimum requirement, so there >>>>> is no error produced in this case. However, on 32-bit Windows with >>>>> C2, >>>>> 68k is the minimum, so an error is produced since the stack will only >>>>> be 64k. There is no bug here. It's just a bit confusing. >>>>> >>>>> thanks, >>>>> >>>>> Chris >>>> >> > From david.holmes at oracle.com Wed Nov 19 10:12:42 2014 From: david.holmes at oracle.com (David Holmes) Date: Wed, 19 Nov 2014 20:12:42 +1000 Subject: [9] RFR (S) 6762191: Setting stack size to 16K causes segmentation fault In-Reply-To: <546C5986.6010500@oracle.com> References: <545D939D.2030308@oracle.com> <5463B896.10801@oracle.com> <54641ADE.8030504@oracle.com> <546BC35E.4070402@oracle.com> <546C5986.6010500@oracle.com> Message-ID: <546C6D1A.8050903@oracle.com> On 19/11/2014 6:49 PM, Chris Plummer wrote: > I've update the webrev to add STACK_SIZE_MINIMUM in place of the 32k > references, and also moved the test from hotspot/test/runtime to > jdk/test/tools/launcher as David requested. That required some > adjustments to the test script, since test_env.sh does not exist in > jdk/test, so I had to pull in the bits I needed into the script. Is there a reason this needs a shell script instead of using the testlibrary tools to launch the VM and check the output? Sorry that should have been mentioned much earlier. David > http://cr.openjdk.java.net/~cjplummer/6762191/webrev.01/ > > I still need to rerun through JPRT. I'll do so once there are no more > suggested changes. > > thanks, > > Chris > > On 11/18/14 2:08 PM, Chris Plummer wrote: >> Adding core-libs-dev at openjdk.java.net, since one of the changes is in >> java.c. >> >> Chris >> >> On 11/12/14 6:43 PM, David Holmes wrote: >>> Hi Chris, >>> >>> Sorry for the delay. >>> >>> On 13/11/2014 5:44 AM, Chris Plummer wrote: >>>> Hi, >>>> >>>> I'm still looking for reviewers. >>> >>> As the change is to the launcher it needs to be reviewed by the >>> launcher owner - which I think is serviceability (though also cc'd >>> Kumar :) ). >>> >>> Launcher change, and your rationale, seems okay to me. I'd probably >>> put the test in to jdk/test/tools/launcher/ though. >>> >>> Thanks, >>> David >>> >>>> thanks, >>>> >>>> Chris >>>> >>>> On 11/7/14 7:53 PM, Chris Plummer wrote: >>>>> This is an initial review for 6762191. I'm guessing there will be >>>>> recommendations to fix in a different way, but thought this would be a >>>>> good time to start the discussion. >>>>> >>>>> https://bugs.openjdk.java.net/browse/JDK-6762191 >>>>> http://cr.openjdk.java.net/~cjplummer/6762191/webrev.00.jdk/ >>>>> http://cr.openjdk.java.net/~cjplummer/6762191/webrev.00.hotspot/ >>>>> >>>>> The bug is that if the -Xss size is set to something very small (like >>>>> 16k), on linux there will be a crash due to overwriting the end of the >>>>> stack. This happens before hotspot can compute its stack needs and >>>>> verify that the stack is big enough. >>>>> >>>>> It didn't seem viable to move the hotspot stack size check earlier. It >>>>> depends on too much other work done before that point, and the changes >>>>> would have been disruptive. The stack size check is currently done in >>>>> os::init_2(). >>>>> >>>>> What is needed is a check before the thread is created. That way we >>>>> can create a thread with a big enough stack to handle all needs up to >>>>> the point of the check in os::init_2(). This initial check does not >>>>> need to be the final check. It just needs to confirm that we have >>>>> enough stack to get us to the check in os::init_2(). >>>>> >>>>> I decided to check in java.c if the -Xss size is too small, and set it >>>>> to a larger size if it is. I hard coded this size to 32k (I'll explain >>>>> why 32k later). I suspect this is the part that will result in some >>>>> debate. If you have better suggestions let me know. If it does stay >>>>> here, then probably the 32k needs to be a #define, and maybe even an >>>>> OS porting interface, but I'm not sure where to put it. >>>>> >>>>> The reason I chose 32k is because this is big enough for all platforms >>>>> to get to the stack size check in os::init_2(). It is also smaller >>>>> than the actual minimum stack size allowed on any platform. 32-bit >>>>> windows has the smallest requirement at 64k. I add some printfs to >>>>> print the minimum stack requirement, and then ran a simple JTReg test >>>>> with every JPRT supported platform to get the results. >>>>> >>>>> The TooSmallStackSize.sh will run "java -version" with -Xss16k, >>>>> -Xss32k, and -XXss, where is the size from the >>>>> error message produced by the JVM, such as in the following: >>>>> >>>>> $ java -Xss32k -version >>>>> The stack size specified is too small, Specify at least 100k >>>>> Error: Could not create the Java Virtual Machine. >>>>> Error: A fatal exception has occurred. Program will exit. >>>>> >>>>> I ran this test through JPRT on all platforms, and they all pass. >>>>> >>>>> One thing to point out is that Windows behaves a bit different than >>>>> the other platforms. It always rounds the stack size up to a multiple >>>>> of 64k , so even if you specify -Xss16k, you get a 64k stack. On >>>>> 32-bit Windows with C1, 64k is also the minimum requirement, so there >>>>> is no error produced in this case. However, on 32-bit Windows with C2, >>>>> 68k is the minimum, so an error is produced since the stack will only >>>>> be 64k. There is no bug here. It's just a bit confusing. >>>>> >>>>> thanks, >>>>> >>>>> Chris >>>> >> > From konstantin.shefov at oracle.com Wed Nov 19 11:12:08 2014 From: konstantin.shefov at oracle.com (Konstantin Shefov) Date: Wed, 19 Nov 2014 14:12:08 +0300 Subject: [9] Review request : JDK-8059070: [TESTBUG] java/lang/invoke/LFCaching/LFMultiThreadCachingTest.java failed - timeout In-Reply-To: <5464A890.6080404@oracle.com> References: <543D1DEE.8030206@oracle.com> <543F8520.9090408@oracle.com> <8F9DD322-1793-4739-8155-61CE7D2C5AAB@oracle.com> <543F9800.7040406@oracle.com> <543FC71F.8020504@oracle.com> <5440E39C.8070001@oracle.com> <5448E5A2.9030804@oracle.com> <544E1B65.1090400@oracle.com> <5450F8E4.2060808@oracle.com> <545C9D70.6060308@oracle.com> <5460A29E.6070206@oracle.com> <5460CF90.1050303@oracle.com> <5464A890.6080404@oracle.com> Message-ID: <546C7B08.3070700@oracle.com> Hello, Vladimir I have updated the Webrev http://cr.openjdk.java.net/~kshefov/8059070/webrev.02 I have added DEFAULT_TEST_TIMEOUT constant to Utils class. -Konstantin On 13.11.2014 15:48, Konstantin Shefov wrote: > Kindly reminder. > > On 10.11.2014 17:45, Konstantin Shefov wrote: >> Vladimir, thanks for reviewing >> >> I have updated the webrev: >> http://cr.openjdk.java.net/~kshefov/8059070/webrev.02 >> I have added DEFAULT_TEST_TIMEOUT constant to Utils class. >> >> -Konstantin >> >> On 10.11.2014 14:33, Vladimir Ivanov wrote: >>> Kontantin, sorry for the late response. >>> >>> In general, the fix looks good. >>> >>> I'd move timeout computation logic to Utils: >>> + private static final long TIMEOUT = >>> Utils.adjustTimeout(TimeUnit.SECONDS.toMillis(120)); >>> >>> Default value (120s) isn't specific to the tests, but an >>> implementation detail of jtreg. >>> >>> Best regards, >>> Vladimir Ivanov >>> >>> On 11/7/14, 2:22 PM, Konstantin Shefov wrote: >>>> Gently reminder >>>> >>>> 29.10.2014 17:25, Konstantin Shefov ?????: >>>>> Please, review a test bug fix. >>>>> http://cr.openjdk.java.net/~kshefov/8059070/webrev.01/ >>>>> >>>>> -Konstantin >>>>> >>>>> On 27.10.2014 13:16, Konstantin Shefov wrote: >>>>>> Kindly reminder >>>>>> >>>>>> On 23.10.2014 19:04, Paul Sandoz wrote: >>>>>>> On Oct 23, 2014, at 1:25 PM, Konstantin Shefov >>>>>>> wrote: >>>>>>>> Gently reminder >>>>>>>> >>>>>>>> On 17.10.2014 13:38, Konstantin Shefov wrote: >>>>>>>>> Hi, >>>>>>>>> >>>>>>>>> I have updated the webrev: >>>>>>>>> http://cr.openjdk.java.net/~kshefov/8059070/webrev.01/ >>>>>>>>> >>>>>>> +1 >>>>>>> >>>>>>> Sorry for the delay, >>>>>>> Paul. >>>>>> >>>>> >>>> >> > From daniel.fuchs at oracle.com Wed Nov 19 11:17:01 2014 From: daniel.fuchs at oracle.com (Daniel Fuchs) Date: Wed, 19 Nov 2014 12:17:01 +0100 Subject: RFR: 8065138 - Encodings.isRecognizedEnconding sometimes fails to recognize 'UTF8' Message-ID: <546C7C2D.3060805@oracle.com> Hi, Please find below a trivial fix for 8065138: Encodings.isRecognizedEnconding sometimes fails to recognize 'UTF8' https://bugs.openjdk.java.net/browse/JDK-8065138 webrev: http://cr.openjdk.java.net/~dfuchs/webrev_8065138/webrev.00/ The root of the issue is with jaxp/src/java.xml/share/classes/com/sun/org/apache/xml/internal/serializer/Encodings.properties It contains a special character '?' which confuses the build system on Mavericks. The Encodings.properties file ends up truncated in resources.jar - it contains only one line (the line before the special character was encountered). The fix is to replace the special character '?' by its unicode representation \u00e5. best regards, -- daniel From vladimir.x.ivanov at oracle.com Wed Nov 19 10:24:36 2014 From: vladimir.x.ivanov at oracle.com (Vladimir Ivanov) Date: Wed, 19 Nov 2014 14:24:36 +0400 Subject: [9, 8u40] RFR (XXS): 8059880: Get rid of LambdaForm interpretation In-Reply-To: References: <546BB3C2.60202@oracle.com> <546BD761.6030108@oracle.com> Message-ID: <546C6FE4.60008@oracle.com> Aleksey, Duncan, thanks for the review and the confirmation that it doesn't break stuff for you. Any Reviews, please? :-) Best regards, Vladimir Ivanov On 11/19/14, 2:23 PM, MacGregor, Duncan (GE Energy Management) wrote: > On 18/11/2014 23:33, "Aleksey Shipilev" > wrote: >> On 11/19/2014 12:01 AM, Vladimir Ivanov wrote: >>> http://cr.openjdk.java.net/~vlivanov/8059880/webrev.00/ >>> https://bugs.openjdk.java.net/browse/JDK-8059880 >> >> Yes, for the love of God, GO FOR IT. > > Seconded. Startup of our stuff seems fine now with a compile threshold of > zero, and it will make stacks so much easier to read in the debugger. :-) > > _______________________________________________ > mlvm-dev mailing list > mlvm-dev at openjdk.java.net > http://mail.openjdk.java.net/mailman/listinfo/mlvm-dev > From vladimir.x.ivanov at oracle.com Wed Nov 19 10:30:30 2014 From: vladimir.x.ivanov at oracle.com (Vladimir Ivanov) Date: Wed, 19 Nov 2014 14:30:30 +0400 Subject: [9, 8u40] RFR (M): 8063135: Enable full LF sharing by default In-Reply-To: <546BD858.5040206@oracle.com> References: <546BAAA9.8090009@oracle.com> <546BD858.5040206@oracle.com> Message-ID: <546C7146.2030901@oracle.com> Hm, I remember I fixed that long time ago... Seems like I chose a stale patch. Sorry for that. Updated webrev in place. Best regards, Vladimir Ivanov On 11/19/14, 3:38 AM, Aleksey Shipilev wrote: > On 11/18/2014 11:23 PM, Vladimir Ivanov wrote: >> http://cr.openjdk.java.net/~vlivanov/8063135/webrev.00/ >> https://bugs.openjdk.java.net/browse/JDK-8063135 > > Broken array index here: > > 69 TRACE_METHOD_LINKAGE = (Boolean) values[3]; > 70 COMPILE_THRESHOLD = (Integer) values[5]; > > Also, how does it pass the tests? It should fail with AIOBE during > MethodHandleStatics::, since > > 53 final Object[] values = new Object[7]; > ... > 72 PROFILE_LEVEL = (Integer) values[7]; > > Thanks, > -Aleksey. > > > > _______________________________________________ > mlvm-dev mailing list > mlvm-dev at openjdk.java.net > http://mail.openjdk.java.net/mailman/listinfo/mlvm-dev > From aleksey.shipilev at oracle.com Wed Nov 19 12:02:19 2014 From: aleksey.shipilev at oracle.com (Aleksey Shipilev) Date: Wed, 19 Nov 2014 15:02:19 +0300 Subject: RFR: 8065138 - Encodings.isRecognizedEnconding sometimes fails to recognize 'UTF8' In-Reply-To: <546C7C2D.3060805@oracle.com> References: <546C7C2D.3060805@oracle.com> Message-ID: <546C86CB.3060101@oracle.com> On 11/19/2014 02:17 PM, Daniel Fuchs wrote: > webrev: http://cr.openjdk.java.net/~dfuchs/webrev_8065138/webrev.00/ Looks good. (not a Reviewer) > The root of the issue is with > jaxp/src/java.xml/share/classes/com/sun/org/apache/xml/internal/serializer/Encodings.properties > > It contains a special character '?' which confuses the build system > on Mavericks. The Encodings.properties file ends up truncated in > resources.jar - it contains only one line (the line before the > special character was encountered). ?_?... (??????? ??? > The fix is to replace the special character '?' by its unicode representation \u00e5. Can you also look if these files are corrupted in your build as well? $ grep --include=\*.java --include=\*.properties -R --color='auto' -n -v -e "^[[:alnum:][:punct:][:space:]]*$" I see more hits in jdk/ within java.desktop and java.xml.crypto. Thanks, -Aleksey. From aleksey.shipilev at oracle.com Wed Nov 19 12:04:12 2014 From: aleksey.shipilev at oracle.com (Aleksey Shipilev) Date: Wed, 19 Nov 2014 15:04:12 +0300 Subject: [9, 8u40] RFR (M): 8063135: Enable full LF sharing by default In-Reply-To: <546C7146.2030901@oracle.com> References: <546BAAA9.8090009@oracle.com> <546BD858.5040206@oracle.com> <546C7146.2030901@oracle.com> Message-ID: <546C873C.9010205@oracle.com> Looks good now. -Aleksey. On 11/19/2014 01:30 PM, Vladimir Ivanov wrote: > Hm, I remember I fixed that long time ago... Seems like I chose a stale > patch. Sorry for that. Updated webrev in place. > > Best regards, > Vladimir Ivanov > > On 11/19/14, 3:38 AM, Aleksey Shipilev wrote: >> On 11/18/2014 11:23 PM, Vladimir Ivanov wrote: >>> http://cr.openjdk.java.net/~vlivanov/8063135/webrev.00/ >>> https://bugs.openjdk.java.net/browse/JDK-8063135 >> >> Broken array index here: >> >> 69 TRACE_METHOD_LINKAGE = (Boolean) values[3]; >> 70 COMPILE_THRESHOLD = (Integer) values[5]; >> >> Also, how does it pass the tests? It should fail with AIOBE during >> MethodHandleStatics::, since >> >> 53 final Object[] values = new Object[7]; >> ... >> 72 PROFILE_LEVEL = (Integer) values[7]; >> >> Thanks, >> -Aleksey. >> >> >> >> _______________________________________________ >> mlvm-dev mailing list >> mlvm-dev at openjdk.java.net >> http://mail.openjdk.java.net/mailman/listinfo/mlvm-dev >> From daniel.fuchs at oracle.com Wed Nov 19 13:44:17 2014 From: daniel.fuchs at oracle.com (Daniel Fuchs) Date: Wed, 19 Nov 2014 14:44:17 +0100 Subject: RFR: 8065138 - Encodings.isRecognizedEnconding sometimes fails to recognize 'UTF8' In-Reply-To: <546C86CB.3060101@oracle.com> References: <546C7C2D.3060805@oracle.com> <546C86CB.3060101@oracle.com> Message-ID: <546C9EB1.7000603@oracle.com> Hi Aleksey, On 19/11/14 13:02, Aleksey Shipilev wrote: > Can you also look if these files are corrupted in your build as well? > > $ grep --include=\*.java --include=\*.properties -R --color='auto' -n -v > -e "^[[:alnum:][:punct:][:space:]]*$" > > I see more hits in jdk/ within java.desktop and java.xml.crypto. > .java file should not be an issue - and I suspect message catalogs should not be either (they are compiled into .class files - they don't end up in resources.jar) So I ran your script taking into account properties file only, and compared with the properties found in resources.jar. It seems that jdk/src/java.xml.crypto/share/classes/com/sun/org/apache/xml/internal/security/resource/xmlsecurity_de.properties suffers from the same issue. I will log a bug for that. There is also another outstanding issue with the build process (concerning '#' character embedded in property values) which seems to be affecting more than JAXP: JDK-8043146 Maybe we should find a owner for that one too. Thanks for the suggestion, -- daniel From roger.riggs at oracle.com Wed Nov 19 15:21:22 2014 From: roger.riggs at oracle.com (roger riggs) Date: Wed, 19 Nov 2014 10:21:22 -0500 Subject: RFR 9: 8065372: Object.wait(ms, ns) timeout returns early In-Reply-To: References: <54665C9D.1010903@oracle.com> <54667EEC.9060002@oracle.com> <546A4C2A.4070208@oracle.com> <546A7082.5060302@oracle.com> <546AAAC2.6080009@oracle.com> <546AD2CD.90300@oracle.com> Message-ID: <546CB572.90607@oracle.com> Hi, Created a new issue and webrev for this correction: Please review: http://cr.openjdk.java.net/~rriggs/webrev-wait-8065372/ Roger On 11/18/2014 11:02 AM, Martin Buchholz wrote: > Here I'm just trying to fix the problem that Object.wait(millis,nanos) > trivially returns early. >> That seems overly complex - after checking for valid values of timeout and >> nanos you simply need: >> >> if (nanos > 0) timeout++; >> >> to ensure the >= requirement. > Sure, the miminal diff is: > > diff --git a/src/java.base/share/classes/java/lang/Object.java > b/src/java.base/share/classes/java/lang/Object.java > --- a/src/java.base/share/classes/java/lang/Object.java > +++ b/src/java.base/share/classes/java/lang/Object.java > @@ -453,7 +453,7 @@ > "nanosecond timeout value out of range"); > } > > - if (nanos >= 500000 || (nanos != 0 && timeout == 0)) { > + if (nanos != 0) { > timeout++; > } > >> But seriously we should just deprecate this version as we're not likely to >> get any underlying OS mechanisms for doing nanosecond resolution timed >> blocking actions. > Object.wait(millis, nanos) is an ugly API (I wish for both a public > and hotspot internal API that simply operated on nanoseconds), but > it's not totally awful. It's not obvious to me that computing > advances over the lifetime of the java platform won't allow > sub-millisecond scheduling, and other APIs that do operate on > nanoseconds (like Process.waitFor) should call this API. From chris.plummer at oracle.com Wed Nov 19 15:52:11 2014 From: chris.plummer at oracle.com (Chris Plummer) Date: Wed, 19 Nov 2014 07:52:11 -0800 Subject: [9] RFR (S) 6762191: Setting stack size to 16K causes segmentation fault In-Reply-To: <546C6D1A.8050903@oracle.com> References: <545D939D.2030308@oracle.com> <5463B896.10801@oracle.com> <54641ADE.8030504@oracle.com> <546BC35E.4070402@oracle.com> <546C5986.6010500@oracle.com> <546C6D1A.8050903@oracle.com> Message-ID: <546CBCAB.7040101@oracle.com> On 11/19/14 2:12 AM, David Holmes wrote: > On 19/11/2014 6:49 PM, Chris Plummer wrote: >> I've update the webrev to add STACK_SIZE_MINIMUM in place of the 32k >> references, and also moved the test from hotspot/test/runtime to >> jdk/test/tools/launcher as David requested. That required some >> adjustments to the test script, since test_env.sh does not exist in >> jdk/test, so I had to pull in the bits I needed into the script. > > Is there a reason this needs a shell script instead of using the > testlibrary tools to launch the VM and check the output? Not that I'm aware of. I guess I just really didn't look at what it would take to make it all in java. I'll have a look at java examples and convert it. Chris > > Sorry that should have been mentioned much earlier. > > David > > >> http://cr.openjdk.java.net/~cjplummer/6762191/webrev.01/ >> >> I still need to rerun through JPRT. I'll do so once there are no more >> suggested changes. >> >> thanks, >> >> Chris >> >> On 11/18/14 2:08 PM, Chris Plummer wrote: >>> Adding core-libs-dev at openjdk.java.net, since one of the changes is in >>> java.c. >>> >>> Chris >>> >>> On 11/12/14 6:43 PM, David Holmes wrote: >>>> Hi Chris, >>>> >>>> Sorry for the delay. >>>> >>>> On 13/11/2014 5:44 AM, Chris Plummer wrote: >>>>> Hi, >>>>> >>>>> I'm still looking for reviewers. >>>> >>>> As the change is to the launcher it needs to be reviewed by the >>>> launcher owner - which I think is serviceability (though also cc'd >>>> Kumar :) ). >>>> >>>> Launcher change, and your rationale, seems okay to me. I'd probably >>>> put the test in to jdk/test/tools/launcher/ though. >>>> >>>> Thanks, >>>> David >>>> >>>>> thanks, >>>>> >>>>> Chris >>>>> >>>>> On 11/7/14 7:53 PM, Chris Plummer wrote: >>>>>> This is an initial review for 6762191. I'm guessing there will be >>>>>> recommendations to fix in a different way, but thought this would >>>>>> be a >>>>>> good time to start the discussion. >>>>>> >>>>>> https://bugs.openjdk.java.net/browse/JDK-6762191 >>>>>> http://cr.openjdk.java.net/~cjplummer/6762191/webrev.00.jdk/ >>>>>> http://cr.openjdk.java.net/~cjplummer/6762191/webrev.00.hotspot/ >>>>>> >>>>>> The bug is that if the -Xss size is set to something very small >>>>>> (like >>>>>> 16k), on linux there will be a crash due to overwriting the end >>>>>> of the >>>>>> stack. This happens before hotspot can compute its stack needs and >>>>>> verify that the stack is big enough. >>>>>> >>>>>> It didn't seem viable to move the hotspot stack size check >>>>>> earlier. It >>>>>> depends on too much other work done before that point, and the >>>>>> changes >>>>>> would have been disruptive. The stack size check is currently >>>>>> done in >>>>>> os::init_2(). >>>>>> >>>>>> What is needed is a check before the thread is created. That way we >>>>>> can create a thread with a big enough stack to handle all needs >>>>>> up to >>>>>> the point of the check in os::init_2(). This initial check does not >>>>>> need to be the final check. It just needs to confirm that we have >>>>>> enough stack to get us to the check in os::init_2(). >>>>>> >>>>>> I decided to check in java.c if the -Xss size is too small, and >>>>>> set it >>>>>> to a larger size if it is. I hard coded this size to 32k (I'll >>>>>> explain >>>>>> why 32k later). I suspect this is the part that will result in some >>>>>> debate. If you have better suggestions let me know. If it does stay >>>>>> here, then probably the 32k needs to be a #define, and maybe even an >>>>>> OS porting interface, but I'm not sure where to put it. >>>>>> >>>>>> The reason I chose 32k is because this is big enough for all >>>>>> platforms >>>>>> to get to the stack size check in os::init_2(). It is also smaller >>>>>> than the actual minimum stack size allowed on any platform. 32-bit >>>>>> windows has the smallest requirement at 64k. I add some printfs to >>>>>> print the minimum stack requirement, and then ran a simple JTReg >>>>>> test >>>>>> with every JPRT supported platform to get the results. >>>>>> >>>>>> The TooSmallStackSize.sh will run "java -version" with -Xss16k, >>>>>> -Xss32k, and -XXss, where is the size from the >>>>>> error message produced by the JVM, such as in the following: >>>>>> >>>>>> $ java -Xss32k -version >>>>>> The stack size specified is too small, Specify at least 100k >>>>>> Error: Could not create the Java Virtual Machine. >>>>>> Error: A fatal exception has occurred. Program will exit. >>>>>> >>>>>> I ran this test through JPRT on all platforms, and they all pass. >>>>>> >>>>>> One thing to point out is that Windows behaves a bit different than >>>>>> the other platforms. It always rounds the stack size up to a >>>>>> multiple >>>>>> of 64k , so even if you specify -Xss16k, you get a 64k stack. On >>>>>> 32-bit Windows with C1, 64k is also the minimum requirement, so >>>>>> there >>>>>> is no error produced in this case. However, on 32-bit Windows >>>>>> with C2, >>>>>> 68k is the minimum, so an error is produced since the stack will >>>>>> only >>>>>> be 64k. There is no bug here. It's just a bit confusing. >>>>>> >>>>>> thanks, >>>>>> >>>>>> Chris >>>>> >>> >> From martinrb at google.com Wed Nov 19 16:41:33 2014 From: martinrb at google.com (Martin Buchholz) Date: Wed, 19 Nov 2014 08:41:33 -0800 Subject: RFR 9: 8065372: Object.wait(ms, ns) timeout returns early In-Reply-To: <546CB572.90607@oracle.com> References: <54665C9D.1010903@oracle.com> <54667EEC.9060002@oracle.com> <546A4C2A.4070208@oracle.com> <546A7082.5060302@oracle.com> <546AAAC2.6080009@oracle.com> <546AD2CD.90300@oracle.com> <546CB572.90607@oracle.com> Message-ID: Thanks, looks good to me! On Wed, Nov 19, 2014 at 7:21 AM, roger riggs wrote: > Hi, > > Created a new issue and webrev for this correction: > > Please review: > > http://cr.openjdk.java.net/~rriggs/webrev-wait-8065372/ > > Roger > > > On 11/18/2014 11:02 AM, Martin Buchholz wrote: >> >> Here I'm just trying to fix the problem that Object.wait(millis,nanos) >> trivially returns early. >>> >>> That seems overly complex - after checking for valid values of timeout >>> and >>> nanos you simply need: >>> >>> if (nanos > 0) timeout++; >>> >>> to ensure the >= requirement. >> >> Sure, the miminal diff is: >> >> diff --git a/src/java.base/share/classes/java/lang/Object.java >> b/src/java.base/share/classes/java/lang/Object.java >> --- a/src/java.base/share/classes/java/lang/Object.java >> +++ b/src/java.base/share/classes/java/lang/Object.java >> @@ -453,7 +453,7 @@ >> "nanosecond timeout value out of >> range"); >> } >> >> - if (nanos >= 500000 || (nanos != 0 && timeout == 0)) { >> + if (nanos != 0) { >> timeout++; >> } >> >>> But seriously we should just deprecate this version as we're not likely >>> to >>> get any underlying OS mechanisms for doing nanosecond resolution timed >>> blocking actions. >> >> Object.wait(millis, nanos) is an ugly API (I wish for both a public >> and hotspot internal API that simply operated on nanoseconds), but >> it's not totally awful. It's not obvious to me that computing >> advances over the lifetime of the java platform won't allow >> sub-millisecond scheduling, and other APIs that do operate on >> nanoseconds (like Process.waitFor) should call this API. > > From martinrb at google.com Wed Nov 19 17:01:42 2014 From: martinrb at google.com (Martin Buchholz) Date: Wed, 19 Nov 2014 09:01:42 -0800 Subject: RFR: 8065138 - Encodings.isRecognizedEnconding sometimes fails to recognize 'UTF8' In-Reply-To: <546C7C2D.3060805@oracle.com> References: <546C7C2D.3060805@oracle.com> Message-ID: On Wed, Nov 19, 2014 at 3:17 AM, Daniel Fuchs wrote: > Hi, > > Please find below a trivial fix for > > 8065138: Encodings.isRecognizedEnconding sometimes fails to > recognize 'UTF8' > https://bugs.openjdk.java.net/browse/JDK-8065138 > > webrev: http://cr.openjdk.java.net/~dfuchs/webrev_8065138/webrev.00/ > > The root of the issue is with > jaxp/src/java.xml/share/classes/com/sun/org/apache/xml/internal/serializer/Encodings.properties > It contains a special character '?' which confuses the build > system on Mavericks. Isn't that a bug in the build system that really ought to be fixed? If properties files are to be stored as resources in jar files, they should either be incorporated byte-for-byte identical, or they should be decoded using ISO-8859-1 (as specified). It may be best to leave non-ASCII characters in the source files, as a "test" of the build system and the jdk itself. > The Encodings.properties file ends up truncated in resources.jar - it > contains only one line (the line before the special character was > encountered). > The fix is to replace the special character '?' by its unicode > representation \u00e5. > > best regards, > > -- daniel > From vladimir.kozlov at oracle.com Wed Nov 19 17:20:51 2014 From: vladimir.kozlov at oracle.com (Vladimir Kozlov) Date: Wed, 19 Nov 2014 09:20:51 -0800 Subject: [9, 8u40] RFR (XXS): 8059880: Get rid of LambdaForm interpretation In-Reply-To: <546C6FE4.60008@oracle.com> References: <546BB3C2.60202@oracle.com> <546BD761.6030108@oracle.com> <546C6FE4.60008@oracle.com> Message-ID: <546CD173.6010502@oracle.com> As far as I can guess :) this change looks good. Reviewed. Thanks, Vladimir On 11/19/14 2:24 AM, Vladimir Ivanov wrote: > Aleksey, Duncan, thanks for the review and the confirmation that it doesn't break stuff for you. > > Any Reviews, please? :-) > > Best regards, > Vladimir Ivanov > > On 11/19/14, 2:23 PM, MacGregor, Duncan (GE Energy Management) wrote: >> On 18/11/2014 23:33, "Aleksey Shipilev" >> wrote: >>> On 11/19/2014 12:01 AM, Vladimir Ivanov wrote: >>>> http://cr.openjdk.java.net/~vlivanov/8059880/webrev.00/ >>>> https://bugs.openjdk.java.net/browse/JDK-8059880 >>> >>> Yes, for the love of God, GO FOR IT. >> >> Seconded. Startup of our stuff seems fine now with a compile threshold of >> zero, and it will make stacks so much easier to read in the debugger. :-) >> >> _______________________________________________ >> mlvm-dev mailing list >> mlvm-dev at openjdk.java.net >> http://mail.openjdk.java.net/mailman/listinfo/mlvm-dev >> > _______________________________________________ > mlvm-dev mailing list > mlvm-dev at openjdk.java.net > http://mail.openjdk.java.net/mailman/listinfo/mlvm-dev From martinrb at google.com Wed Nov 19 17:23:04 2014 From: martinrb at google.com (Martin Buchholz) Date: Wed, 19 Nov 2014 09:23:04 -0800 Subject: RFR 9: 8064932: test java/lang/ProcessBuilder/Basic.java: waitFor didn't take long enough In-Reply-To: <546BC815.10107@oracle.com> References: <54665C9D.1010903@oracle.com> <54667EEC.9060002@oracle.com> <546A4C2A.4070208@oracle.com> <546A7082.5060302@oracle.com> <546AD0F2.9040002@oracle.com> <546BA77E.4030505@oracle.com> <546BC815.10107@oracle.com> Message-ID: The latest version looks good, but here are some more nitpicks: I'd prefer the name "deadline" to "endTime" since we already use that convention in j.u.c., e.g final long deadline = System.nanoTime() + nanosTimeout; With the "deadline" style of checking, variable remainingNanos becomes unnecessary, and you can just do do { ... } while (deadline - System.nanoTime() > 0) On Tue, Nov 18, 2014 at 2:28 PM, roger riggs wrote: > Hi, > > Done, added the test and immediate return after the wait. > On Windows, the Thread.interrupted check needs to be done first to be > consistent with Unix. > Refactored a bit to remove a local variable and extra arithmetic computing > the remainingNanos. > > Updated: > http://cr.openjdk.java.net/~rriggs/webrev-basic-notenough-8064932/ > > Roger > > > > > On 11/18/2014 4:58 PM, Martin Buchholz wrote: >> >> I think we're all 3 in agreement on the general direction, even if we >> disagree on the details. >> >> Once we properly round to millis, it is no longer necessary to use >> Math.max - just >> >> wait(NANOSECONDS.toMillis(rem + 999_999L)); >> >> As in my proposed change, I would like every call to wait immediately >> followed by a check >> if (hasExited) >> return true; >> >> to avoid the extra call to nanoTime. >> >> On Tue, Nov 18, 2014 at 12:09 PM, roger riggs >> wrote: >>> >>> Hi, >>> >>> Work on Object.wait is an issue to be taken up separately. >>> >>> I agree that the timeout values should not be truncated to milliseconds, >>> likely >>> causing an additional cycle through the while loop and waiting. >>> The Process.waitFor methods are expected to wait for the specified time >>> to >>> elapse. >>> The webrev has been updated for both Windows and Unix to use a ceiling >>> function >>> on milliseconds and ensure that at least the requested time has elapsed. >>> >>> Please review: >>> http://cr.openjdk.java.net/~rriggs/webrev-basic-notenough-8064932/ >>> >>> Thanks, Roger >>> >>> >>> >>> >>> On 11/18/2014 11:08 AM, Martin Buchholz wrote: >>>> >>>> On Mon, Nov 17, 2014 at 8:54 PM, David Holmes >>>> wrote: >>>>> >>>>> On 18/11/2014 2:43 PM, Martin Buchholz wrote: >>>>>> >>>>>> Proposed sibling change >>>>>> >>>>>> http://cr.openjdk.java.net/~martin/webrevs/openjdk9/UNIXProcess.waitFor/ >>>>>> - don't unconditionally call nanoTime when the wait ends >>>>>> - use the millis/nanos form of Object.wait in case sub-millisecond >>>>>> waits are ever supported. >>>>> >>>>> >>>>> I don't really see the point of adding the extra math, plus an extra >>>>> call >>>>> (the two arg wait will call the one arg version) for no actual gain. >>>> >>>> The idea was to code to the Object.wait API, not its current >>>> implementation with only millisecond resolution. >>>> Even if we code to the current implementation, we should round UP not >>>> down, >>>> to avoid the problem of needing to call wait twice in case of early >>>> return. >>>> That would also be progress. >>>> >>>>> If you want to add a fast exit path that's fine but the rest seems >>>>> superfluous to me. >>> >>> > From roger.riggs at oracle.com Wed Nov 19 18:04:07 2014 From: roger.riggs at oracle.com (roger riggs) Date: Wed, 19 Nov 2014 13:04:07 -0500 Subject: RFR 9: 8064932: test java/lang/ProcessBuilder/Basic.java: waitFor didn't take long enough In-Reply-To: References: <54665C9D.1010903@oracle.com> <54667EEC.9060002@oracle.com> <546A4C2A.4070208@oracle.com> <546A7082.5060302@oracle.com> <546AD0F2.9040002@oracle.com> <546BA77E.4030505@oracle.com> <546BC815.10107@oracle.com> Message-ID: <546CDB97.8020601@oracle.com> Hi Martin, On 11/19/2014 12:23 PM, Martin Buchholz wrote: > The latest version looks good, but here are some more nitpicks: > > I'd prefer the name "deadline" to "endTime" since we already use that > convention in j.u.c., e.g Ok, applied to the webrev: > > final long deadline = System.nanoTime() + nanosTimeout; > > With the "deadline" style of checking, variable remainingNanos becomes > unnecessary, and you can just do > > do { ... } while (deadline - System.nanoTime() > 0) No really expendable, the remainingNanos value is needed for the wait() call on the retry. Roger > > On Tue, Nov 18, 2014 at 2:28 PM, roger riggs wrote: >> Hi, >> >> Done, added the test and immediate return after the wait. >> On Windows, the Thread.interrupted check needs to be done first to be >> consistent with Unix. >> Refactored a bit to remove a local variable and extra arithmetic computing >> the remainingNanos. >> >> Updated: >> http://cr.openjdk.java.net/~rriggs/webrev-basic-notenough-8064932/ >> >> Roger >> >> >> >> >> On 11/18/2014 4:58 PM, Martin Buchholz wrote: >>> I think we're all 3 in agreement on the general direction, even if we >>> disagree on the details. >>> >>> Once we properly round to millis, it is no longer necessary to use >>> Math.max - just >>> >>> wait(NANOSECONDS.toMillis(rem + 999_999L)); >>> >>> As in my proposed change, I would like every call to wait immediately >>> followed by a check >>> if (hasExited) >>> return true; >>> >>> to avoid the extra call to nanoTime. >>> >>> On Tue, Nov 18, 2014 at 12:09 PM, roger riggs >>> wrote: >>>> Hi, >>>> >>>> Work on Object.wait is an issue to be taken up separately. >>>> >>>> I agree that the timeout values should not be truncated to milliseconds, >>>> likely >>>> causing an additional cycle through the while loop and waiting. >>>> The Process.waitFor methods are expected to wait for the specified time >>>> to >>>> elapse. >>>> The webrev has been updated for both Windows and Unix to use a ceiling >>>> function >>>> on milliseconds and ensure that at least the requested time has elapsed. >>>> >>>> Please review: >>>> http://cr.openjdk.java.net/~rriggs/webrev-basic-notenough-8064932/ >>>> >>>> Thanks, Roger >>>> >>>> >>>> >>>> >>>> On 11/18/2014 11:08 AM, Martin Buchholz wrote: >>>>> On Mon, Nov 17, 2014 at 8:54 PM, David Holmes >>>>> wrote: >>>>>> On 18/11/2014 2:43 PM, Martin Buchholz wrote: >>>>>>> Proposed sibling change >>>>>>> >>>>>>> http://cr.openjdk.java.net/~martin/webrevs/openjdk9/UNIXProcess.waitFor/ >>>>>>> - don't unconditionally call nanoTime when the wait ends >>>>>>> - use the millis/nanos form of Object.wait in case sub-millisecond >>>>>>> waits are ever supported. >>>>>> >>>>>> I don't really see the point of adding the extra math, plus an extra >>>>>> call >>>>>> (the two arg wait will call the one arg version) for no actual gain. >>>>> The idea was to code to the Object.wait API, not its current >>>>> implementation with only millisecond resolution. >>>>> Even if we code to the current implementation, we should round UP not >>>>> down, >>>>> to avoid the problem of needing to call wait twice in case of early >>>>> return. >>>>> That would also be progress. >>>>> >>>>>> If you want to add a fast exit path that's fine but the rest seems >>>>>> superfluous to me. >>>> From daniel.fuchs at oracle.com Wed Nov 19 18:15:12 2014 From: daniel.fuchs at oracle.com (Daniel Fuchs) Date: Wed, 19 Nov 2014 19:15:12 +0100 Subject: RFR: 8065138 - Encodings.isRecognizedEnconding sometimes fails to recognize 'UTF8' In-Reply-To: References: <546C7C2D.3060805@oracle.com> Message-ID: <546CDE30.6030907@oracle.com> On 19/11/14 18:01, Martin Buchholz wrote: > On Wed, Nov 19, 2014 at 3:17 AM, Daniel Fuchs wrote: >> Hi, >> >> Please find below a trivial fix for >> >> 8065138: Encodings.isRecognizedEnconding sometimes fails to >> recognize 'UTF8' >> https://bugs.openjdk.java.net/browse/JDK-8065138 >> >> webrev: http://cr.openjdk.java.net/~dfuchs/webrev_8065138/webrev.00/ >> >> The root of the issue is with >> jaxp/src/java.xml/share/classes/com/sun/org/apache/xml/internal/serializer/Encodings.properties >> It contains a special character '?' which confuses the build >> system on Mavericks. > > Isn't that a bug in the build system that really ought to be fixed? > > If properties files are to be stored as resources in jar files, they > should either be incorporated byte-for-byte identical, or they should > be decoded using ISO-8859-1 (as specified). It may be best to leave > non-ASCII characters in the source files, as a "test" of the build > system and the jdk itself. Hmmm. If the character is indeed legal then you're right, fixing the build is probably a better idea. However the issue seems to be with using 'sed' over property files: If I simply do: cat jaxp/src/java.xml/share/classes/com/sun/org/apache/xml/internal/serializer/Encodings.properties | sed 's,x,x,g' on my machine, it balks with: sed: RE error: illegal byte sequence -- daniel > >> The Encodings.properties file ends up truncated in resources.jar - it >> contains only one line (the line before the special character was >> encountered). >> The fix is to replace the special character '?' by its unicode >> representation \u00e5. >> >> best regards, >> >> -- daniel >> From martinrb at google.com Wed Nov 19 18:15:43 2014 From: martinrb at google.com (Martin Buchholz) Date: Wed, 19 Nov 2014 10:15:43 -0800 Subject: RFR 9: 8064932: test java/lang/ProcessBuilder/Basic.java: waitFor didn't take long enough In-Reply-To: <546CDB97.8020601@oracle.com> References: <54665C9D.1010903@oracle.com> <54667EEC.9060002@oracle.com> <546A4C2A.4070208@oracle.com> <546A7082.5060302@oracle.com> <546AD0F2.9040002@oracle.com> <546BA77E.4030505@oracle.com> <546BC815.10107@oracle.com> <546CDB97.8020601@oracle.com> Message-ID: Now I'm reduced to super-nitpicks: There are a couple of extraneous SPACES here: + long msTimeout = TimeUnit.NANOSECONDS.toMillis(remainingNanos + 999_999L); + } while (remainingNanos > 0); On Wed, Nov 19, 2014 at 10:04 AM, roger riggs wrote: >> With the "deadline" style of checking, variable remainingNanos becomes >> unnecessary, and you can just do >> >> do { ... } while (deadline - System.nanoTime() > 0) > > No really expendable, the remainingNanos value is needed for the wait() call > on the retry. Ohh .... right you are! From chris.plummer at oracle.com Tue Nov 18 22:08:30 2014 From: chris.plummer at oracle.com (Chris Plummer) Date: Tue, 18 Nov 2014 14:08:30 -0800 Subject: [9] RFR (S) 6762191: Setting stack size to 16K causes segmentation fault In-Reply-To: <54641ADE.8030504@oracle.com> References: <545D939D.2030308@oracle.com> <5463B896.10801@oracle.com> <54641ADE.8030504@oracle.com> Message-ID: <546BC35E.4070402@oracle.com> Adding core-libs-dev at openjdk.java.net, since one of the changes is in java.c. Chris On 11/12/14 6:43 PM, David Holmes wrote: > Hi Chris, > > Sorry for the delay. > > On 13/11/2014 5:44 AM, Chris Plummer wrote: >> Hi, >> >> I'm still looking for reviewers. > > As the change is to the launcher it needs to be reviewed by the > launcher owner - which I think is serviceability (though also cc'd > Kumar :) ). > > Launcher change, and your rationale, seems okay to me. I'd probably > put the test in to jdk/test/tools/launcher/ though. > > Thanks, > David > >> thanks, >> >> Chris >> >> On 11/7/14 7:53 PM, Chris Plummer wrote: >>> This is an initial review for 6762191. I'm guessing there will be >>> recommendations to fix in a different way, but thought this would be a >>> good time to start the discussion. >>> >>> https://bugs.openjdk.java.net/browse/JDK-6762191 >>> http://cr.openjdk.java.net/~cjplummer/6762191/webrev.00.jdk/ >>> http://cr.openjdk.java.net/~cjplummer/6762191/webrev.00.hotspot/ >>> >>> The bug is that if the -Xss size is set to something very small (like >>> 16k), on linux there will be a crash due to overwriting the end of the >>> stack. This happens before hotspot can compute its stack needs and >>> verify that the stack is big enough. >>> >>> It didn't seem viable to move the hotspot stack size check earlier. It >>> depends on too much other work done before that point, and the changes >>> would have been disruptive. The stack size check is currently done in >>> os::init_2(). >>> >>> What is needed is a check before the thread is created. That way we >>> can create a thread with a big enough stack to handle all needs up to >>> the point of the check in os::init_2(). This initial check does not >>> need to be the final check. It just needs to confirm that we have >>> enough stack to get us to the check in os::init_2(). >>> >>> I decided to check in java.c if the -Xss size is too small, and set it >>> to a larger size if it is. I hard coded this size to 32k (I'll explain >>> why 32k later). I suspect this is the part that will result in some >>> debate. If you have better suggestions let me know. If it does stay >>> here, then probably the 32k needs to be a #define, and maybe even an >>> OS porting interface, but I'm not sure where to put it. >>> >>> The reason I chose 32k is because this is big enough for all platforms >>> to get to the stack size check in os::init_2(). It is also smaller >>> than the actual minimum stack size allowed on any platform. 32-bit >>> windows has the smallest requirement at 64k. I add some printfs to >>> print the minimum stack requirement, and then ran a simple JTReg test >>> with every JPRT supported platform to get the results. >>> >>> The TooSmallStackSize.sh will run "java -version" with -Xss16k, >>> -Xss32k, and -XXss, where is the size from the >>> error message produced by the JVM, such as in the following: >>> >>> $ java -Xss32k -version >>> The stack size specified is too small, Specify at least 100k >>> Error: Could not create the Java Virtual Machine. >>> Error: A fatal exception has occurred. Program will exit. >>> >>> I ran this test through JPRT on all platforms, and they all pass. >>> >>> One thing to point out is that Windows behaves a bit different than >>> the other platforms. It always rounds the stack size up to a multiple >>> of 64k , so even if you specify -Xss16k, you get a 64k stack. On >>> 32-bit Windows with C1, 64k is also the minimum requirement, so there >>> is no error produced in this case. However, on 32-bit Windows with C2, >>> 68k is the minimum, so an error is produced since the stack will only >>> be 64k. There is no bug here. It's just a bit confusing. >>> >>> thanks, >>> >>> Chris >> From patrick at reini.net Wed Nov 19 18:44:50 2014 From: patrick at reini.net (Patrick Reinhart) Date: Wed, 19 Nov 2014 19:44:50 +0100 Subject: Library enhancement proposal for copying data from/to Reader/Writer InputStream/OutputStream In-Reply-To: <54650DB8.6050801@oracle.com> References: <566A428C-1A13-4DEE-940B-2EBE375F2A7A@reini.net> <54650DB8.6050801@oracle.com> Message-ID: <8093A82A-E79A-4BB9-A729-9C458C23ABAB@reini.net> > Am 13.11.2014 um 20:59 schrieb Alan Bateman : > > On 13/11/2014 19:31, Patrick Reinhart wrote: >> Hi there, >> >> In the followup of a BOF with Stephen Colebourne with his ideas of small library changes that may could get in JDK9. As of the fact that in the codebase of my company there are several locations where we copy from/to IO stream over and over again using either external libraries or do it by ourselves, I suggested to have some support for easy coping data between Input-/OutputStream or Reader/Writer without having to use to external libraries. >> > Long overdue. I remember we prototyped methods like this (and much more) during JDK 7 but didn't do enough at the time to actually get them in. We did include a bunch of easy to use methods in the Files class at the time, including copy between input/output streams and files, but we didn't introduce an IOUtils or such class in java.io. I did some implementation of a IOUtils class for the basic copy task already. Should I post a patch with the work I already done? Best regards, Patrick From roger.riggs at oracle.com Wed Nov 19 18:47:08 2014 From: roger.riggs at oracle.com (roger riggs) Date: Wed, 19 Nov 2014 13:47:08 -0500 Subject: RFR 9: 8064932: test java/lang/ProcessBuilder/Basic.java: waitFor didn't take long enough In-Reply-To: References: <54665C9D.1010903@oracle.com> <54667EEC.9060002@oracle.com> <546A4C2A.4070208@oracle.com> <546A7082.5060302@oracle.com> <546AD0F2.9040002@oracle.com> <546BA77E.4030505@oracle.com> <546BC815.10107@oracle.com> <546CDB97.8020601@oracle.com> Message-ID: <546CE5AC.5020105@oracle.com> Hi Martin, I'll take that as an approval. We've got to work on the efficiency of this collaborative pair-programming. ;-) Thanks, Roger On 11/19/2014 1:15 PM, Martin Buchholz wrote: > Now I'm reduced to super-nitpicks: There are a couple of extraneous SPACES here: > > + long msTimeout = > TimeUnit.NANOSECONDS.toMillis(remainingNanos + 999_999L); > + } while (remainingNanos > 0); > > > On Wed, Nov 19, 2014 at 10:04 AM, roger riggs wrote: >>> With the "deadline" style of checking, variable remainingNanos becomes >>> unnecessary, and you can just do >>> >>> do { ... } while (deadline - System.nanoTime() > 0) >> No really expendable, the remainingNanos value is needed for the wait() call >> on the retry. > Ohh .... right you are! From vladimir.x.ivanov at oracle.com Wed Nov 19 19:27:14 2014 From: vladimir.x.ivanov at oracle.com (Vladimir Ivanov) Date: Wed, 19 Nov 2014 23:27:14 +0400 Subject: [9, 8u40] RFR (XXS): 8059880: Get rid of LambdaForm interpretation In-Reply-To: <546CD173.6010502@oracle.com> References: <546BB3C2.60202@oracle.com> <546BD761.6030108@oracle.com> <546C6FE4.60008@oracle.com> <546CD173.6010502@oracle.com> Message-ID: <546CEF12.4000904@oracle.com> Thanks, Vladimir! Best regards, Vladimir Ivanov On 11/19/14, 9:20 PM, Vladimir Kozlov wrote: > As far as I can guess :) this change looks good. Reviewed. > > Thanks, > Vladimir > > On 11/19/14 2:24 AM, Vladimir Ivanov wrote: >> Aleksey, Duncan, thanks for the review and the confirmation that it >> doesn't break stuff for you. >> >> Any Reviews, please? :-) >> >> Best regards, >> Vladimir Ivanov >> >> On 11/19/14, 2:23 PM, MacGregor, Duncan (GE Energy Management) wrote: >>> On 18/11/2014 23:33, "Aleksey Shipilev" >>> wrote: >>>> On 11/19/2014 12:01 AM, Vladimir Ivanov wrote: >>>>> http://cr.openjdk.java.net/~vlivanov/8059880/webrev.00/ >>>>> https://bugs.openjdk.java.net/browse/JDK-8059880 >>>> >>>> Yes, for the love of God, GO FOR IT. >>> >>> Seconded. Startup of our stuff seems fine now with a compile >>> threshold of >>> zero, and it will make stacks so much easier to read in the debugger. >>> :-) >>> >>> _______________________________________________ >>> mlvm-dev mailing list >>> mlvm-dev at openjdk.java.net >>> http://mail.openjdk.java.net/mailman/listinfo/mlvm-dev >>> >> _______________________________________________ >> mlvm-dev mailing list >> mlvm-dev at openjdk.java.net >> http://mail.openjdk.java.net/mailman/listinfo/mlvm-dev From martinrb at google.com Wed Nov 19 20:30:54 2014 From: martinrb at google.com (Martin Buchholz) Date: Wed, 19 Nov 2014 12:30:54 -0800 Subject: RFR: 8065172: More core reflection final and volatile annotations In-Reply-To: References: Message-ID: On Tue, Nov 18, 2014 at 11:36 PM, Stanimir Simeonoff wrote: > Hi, > > Signature hashCode (the static class in MethodUtil) should be eagerly > calculated (i.e. a final field too) since the only usage is being a key in > the hashmap. > I will add bit more later. Thanks, Stanimir. That's a good point. (I was making all these changes with mostly local reasoning) Signature.hashCode is now final and has much simpler init code. From mandy.chung at oracle.com Wed Nov 19 20:36:12 2014 From: mandy.chung at oracle.com (Mandy Chung) Date: Wed, 19 Nov 2014 12:36:12 -0800 Subject: RFR: 8065138 - Encodings.isRecognizedEnconding sometimes fails to recognize 'UTF8' In-Reply-To: <546CDE30.6030907@oracle.com> References: <546C7C2D.3060805@oracle.com> <546CDE30.6030907@oracle.com> Message-ID: <546CFF3C.2030904@oracle.com> resources.jar will be gone when we move to the modular runtime image (JEP 220 [1]). JDK-8065138 and JDK-8065365 will become non-issue in JDK 9. Mandy [1] http://openjdk.java.net/jeps/220 On 11/19/2014 10:15 AM, Daniel Fuchs wrote: >> Isn't that a bug in the build system that really ought to be fixed? >> >> If properties files are to be stored as resources in jar files, they >> should either be incorporated byte-for-byte identical, or they should >> be decoded using ISO-8859-1 (as specified). It may be best to leave >> non-ASCII characters in the source files, as a "test" of the build >> system and the jdk itself. > > Hmmm. If the character is indeed legal then you're right, fixing > the build is probably a better idea. > > However the issue seems to be with using 'sed' over property files: > > If I simply do: > > cat > jaxp/src/java.xml/share/classes/com/sun/org/apache/xml/internal/serializer/Encodings.properties > | sed 's,x,x,g' > > on my machine, it balks with: > > sed: RE error: illegal byte sequence From daniel.fuchs at oracle.com Wed Nov 19 20:50:13 2014 From: daniel.fuchs at oracle.com (Daniel Fuchs) Date: Wed, 19 Nov 2014 21:50:13 +0100 Subject: RFR: 8065138 - Encodings.isRecognizedEnconding sometimes fails to recognize 'UTF8' In-Reply-To: <546CFF3C.2030904@oracle.com> References: <546C7C2D.3060805@oracle.com> <546CDE30.6030907@oracle.com> <546CFF3C.2030904@oracle.com> Message-ID: <546D0285.3090504@oracle.com> On 11/19/14 9:36 PM, Mandy Chung wrote: > resources.jar will be gone when we move to the modular runtime image > (JEP 220 [1]). > JDK-8065138 and JDK-8065365 will become non-issue in JDK 9. Do you mean that the property files will no longer be stripped of their comments? This is what is causing the issue. 'sed' doesn't like the special character. -- daniel > > Mandy > [1] http://openjdk.java.net/jeps/220 > > On 11/19/2014 10:15 AM, Daniel Fuchs wrote: >>> Isn't that a bug in the build system that really ought to be fixed? >>> >>> If properties files are to be stored as resources in jar files, they >>> should either be incorporated byte-for-byte identical, or they should >>> be decoded using ISO-8859-1 (as specified). It may be best to leave >>> non-ASCII characters in the source files, as a "test" of the build >>> system and the jdk itself. >> >> Hmmm. If the character is indeed legal then you're right, fixing >> the build is probably a better idea. >> >> However the issue seems to be with using 'sed' over property files: >> >> If I simply do: >> >> cat >> jaxp/src/java.xml/share/classes/com/sun/org/apache/xml/internal/serializer/Encodings.properties >> | sed 's,x,x,g' >> >> on my machine, it balks with: >> >> sed: RE error: illegal byte sequence > From daniel.smith at oracle.com Wed Nov 19 21:54:28 2014 From: daniel.smith at oracle.com (Dan Smith) Date: Wed, 19 Nov 2014 14:54:28 -0700 Subject: Lambda-fied pattern matching Message-ID: <2F21F150-F3D1-4F7A-A2EB-38F26DE28169@oracle.com> Working with the pattern matching API, I noticed that it could be made a lot less clumsy with some lambdafication. Here's the status quo: Pattern p = Pattern.compile("(\w)*, (\d)*, (\w)*"); for (String s : lines) { Matcher m = p.matcher(str); if (m.match(s)) { System.out.println(m.group(1)); } } With a lambda-friendly API: Pattern p = Pattern.compile("\d*, \d*, \d*"); for (String s : lines) { p.match(str, r -> System.out.println(r.group(1))); } The 'match' is declared as 'match(String, Consumer)'. You could argue that the functional interface should be a Function rather than a Consumer; whatever. Could also do 'matchFirst', 'matchAll' -- the latter eliminates even more boilerplate. If considered useful, this could be added to String too: str.match("\d*, \d*, \d*", r -> System.out.println(r.group(1))); Is this something that has been considered? Should I file an RFE? ?Dan From mark.reinhold at oracle.com Wed Nov 19 22:25:29 2014 From: mark.reinhold at oracle.com (mark.reinhold at oracle.com) Date: Wed, 19 Nov 2014 14:25:29 -0800 (PST) Subject: JEP 231: Remove Launch-Time JRE Version Selection Message-ID: <20141119222529.D08BB4326F@eggemoggin.niobe.net> New JEP Candidate: http://openjdk.java.net/jeps/231 - Mark From martinrb at google.com Wed Nov 19 23:00:53 2014 From: martinrb at google.com (Martin Buchholz) Date: Wed, 19 Nov 2014 15:00:53 -0800 Subject: RFR: 8065159: AttributedString has quadratic resize algorithm Message-ID: Hey Naoto and Masayoshi, I haven't sent you a friendly code review in a while. http://cr.openjdk.java.net/~martin/webrevs/openjdk9/AttributedString-optimization/ https://bugs.openjdk.java.net/browse/JDK-8065159 (AttributedString could also independently see some testing and code hygiene love) From forax at univ-mlv.fr Wed Nov 19 23:10:07 2014 From: forax at univ-mlv.fr (Remi Forax) Date: Thu, 20 Nov 2014 00:10:07 +0100 Subject: Lambda-fied pattern matching In-Reply-To: <2F21F150-F3D1-4F7A-A2EB-38F26DE28169@oracle.com> References: <2F21F150-F3D1-4F7A-A2EB-38F26DE28169@oracle.com> Message-ID: <546D234F.1090004@univ-mlv.fr> On 11/19/2014 10:54 PM, Dan Smith wrote: > Working with the pattern matching API, I noticed that it could be made a lot less clumsy with some lambdafication. > > Here's the status quo: > > Pattern p = Pattern.compile("(\w)*, (\d)*, (\w)*"); > for (String s : lines) { > Matcher m = p.matcher(str); > if (m.match(s)) { > System.out.println(m.group(1)); > } > } > > With a lambda-friendly API: > > Pattern p = Pattern.compile("\d*, \d*, \d*"); > for (String s : lines) { > p.match(str, r -> System.out.println(r.group(1))); > } > > The 'match' is declared as 'match(String, Consumer)'. You could argue that the functional interface should be a Function rather than a Consumer; whatever. > > Could also do 'matchFirst', 'matchAll' -- the latter eliminates even more boilerplate. > > If considered useful, this could be added to String too: > > str.match("\d*, \d*, \d*", r -> System.out.println(r.group(1))); > > Is this something that has been considered? Should I file an RFE? > > ?Dan While I agree that we could have a more lambda-ish API, I prefer having a method Pattern.matchAsStream that returns a Stream of MatchResult because its more flexible that the API you propose. It's also more coherent with the fact that there is already a method splitAsStream(). Pattern p = Pattern.compile("(\d)*, (\d)*, (\d)*"); for (String s : lines) { p.matchAsStream(line).forEach(r -> System.out.println(r.group(1))); } or written using flatMap() Pattern p = Pattern.compile("(\d)*, (\d)*, (\d)*"); lines.stream().flatMap(line -> p.matchAsStream(line)).map(r -> r.group(1)).forEach(System.out::println); and yes, please log a RFE. R?mi From martinrb at google.com Wed Nov 19 23:11:21 2014 From: martinrb at google.com (Martin Buchholz) Date: Wed, 19 Nov 2014 15:11:21 -0800 Subject: RFR: 8065138 - Encodings.isRecognizedEnconding sometimes fails to recognize 'UTF8' In-Reply-To: <546C7C2D.3060805@oracle.com> References: <546C7C2D.3060805@oracle.com> Message-ID: On Wed, Nov 19, 2014 at 3:17 AM, Daniel Fuchs wrote: > The Encodings.properties file ends up truncated in resources.jar - it > contains only one line (the line before the special character was > encountered). If there's an error in the build resulting in a truncated file, then the build should fail. Isn't the return code being checked? So, first fix this bug by breaking the build! > The fix is to replace the special character '?' by its unicode > representation \u00e5. From martinrb at google.com Wed Nov 19 23:14:50 2014 From: martinrb at google.com (Martin Buchholz) Date: Wed, 19 Nov 2014 15:14:50 -0800 Subject: RFR: 8065138 - Encodings.isRecognizedEnconding sometimes fails to recognize 'UTF8' In-Reply-To: <546CDE30.6030907@oracle.com> References: <546C7C2D.3060805@oracle.com> <546CDE30.6030907@oracle.com> Message-ID: It's certainly annoying to write a 20-line pure Java program to replace the sed one-liner, but we've had success doing this for other build tools. Write a java program that explicitly reads in the input in ISO-8859-1 to strip the comments, and it will work correctly forever. The lazy alternative fix is to ensure that all calls to sed and similar system tools are run with LC_ALL=C which will also fix the problem. On Wed, Nov 19, 2014 at 10:15 AM, Daniel Fuchs wrote: > On 19/11/14 18:01, Martin Buchholz wrote: >> >> On Wed, Nov 19, 2014 at 3:17 AM, Daniel Fuchs >> wrote: >>> >>> Hi, >>> >>> Please find below a trivial fix for >>> >>> 8065138: Encodings.isRecognizedEnconding sometimes fails to >>> recognize 'UTF8' >>> https://bugs.openjdk.java.net/browse/JDK-8065138 >>> >>> webrev: http://cr.openjdk.java.net/~dfuchs/webrev_8065138/webrev.00/ >>> >>> The root of the issue is with >>> >>> jaxp/src/java.xml/share/classes/com/sun/org/apache/xml/internal/serializer/Encodings.properties >>> It contains a special character '?' which confuses the build >>> system on Mavericks. >> >> >> Isn't that a bug in the build system that really ought to be fixed? >> >> If properties files are to be stored as resources in jar files, they >> should either be incorporated byte-for-byte identical, or they should >> be decoded using ISO-8859-1 (as specified). It may be best to leave >> non-ASCII characters in the source files, as a "test" of the build >> system and the jdk itself. > > > Hmmm. If the character is indeed legal then you're right, fixing > the build is probably a better idea. > > However the issue seems to be with using 'sed' over property files: > > If I simply do: > > cat > jaxp/src/java.xml/share/classes/com/sun/org/apache/xml/internal/serializer/Encodings.properties > | sed 's,x,x,g' > > on my machine, it balks with: > > sed: RE error: illegal byte sequence > > -- daniel > > >> >>> The Encodings.properties file ends up truncated in resources.jar - it >>> contains only one line (the line before the special character was >>> encountered). >>> The fix is to replace the special character '?' by its unicode >>> representation \u00e5. >>> >>> best regards, >>> >>> -- daniel >>> > From daniel.fuchs at oracle.com Wed Nov 19 23:16:56 2014 From: daniel.fuchs at oracle.com (Daniel Fuchs) Date: Thu, 20 Nov 2014 00:16:56 +0100 Subject: RFR: 8065138 - Encodings.isRecognizedEnconding sometimes fails to recognize 'UTF8' In-Reply-To: References: <546C7C2D.3060805@oracle.com> Message-ID: <546D24E8.10904@oracle.com> On 11/20/14 12:11 AM, Martin Buchholz wrote: > On Wed, Nov 19, 2014 at 3:17 AM, Daniel Fuchs wrote: >> The Encodings.properties file ends up truncated in resources.jar - it >> contains only one line (the line before the special character was >> encountered). > If there's an error in the build resulting in a truncated file, then > the build should fail. Isn't the return code being checked? > > So, first fix this bug by breaking the build! I'm not really interested on breaking the build on my dev machine ;-) In previous versions of the JDK the Makefile was using 'nawk' rather than 'sed'. There was a comment implying that nawk was used to avoid using sed... cheers, -- daniel > >> The fix is to replace the special character '?' by its unicode >> representation \u00e5. From david.holmes at oracle.com Wed Nov 19 23:41:40 2014 From: david.holmes at oracle.com (David Holmes) Date: Thu, 20 Nov 2014 09:41:40 +1000 Subject: RFR 9: 8065372: Object.wait(ms, ns) timeout returns early In-Reply-To: <546CB572.90607@oracle.com> References: <54665C9D.1010903@oracle.com> <54667EEC.9060002@oracle.com> <546A4C2A.4070208@oracle.com> <546A7082.5060302@oracle.com> <546AAAC2.6080009@oracle.com> <546AD2CD.90300@oracle.com> <546CB572.90607@oracle.com> Message-ID: <546D2AB4.3010209@oracle.com> On 20/11/2014 1:21 AM, roger riggs wrote: > Hi, > > Created a new issue and webrev for this correction: > > Please review: > > http://cr.openjdk.java.net/~rriggs/webrev-wait-8065372/ Fine by me. Thanks, David > Roger > > > On 11/18/2014 11:02 AM, Martin Buchholz wrote: >> Here I'm just trying to fix the problem that Object.wait(millis,nanos) >> trivially returns early. >>> That seems overly complex - after checking for valid values of >>> timeout and >>> nanos you simply need: >>> >>> if (nanos > 0) timeout++; >>> >>> to ensure the >= requirement. >> Sure, the miminal diff is: >> >> diff --git a/src/java.base/share/classes/java/lang/Object.java >> b/src/java.base/share/classes/java/lang/Object.java >> --- a/src/java.base/share/classes/java/lang/Object.java >> +++ b/src/java.base/share/classes/java/lang/Object.java >> @@ -453,7 +453,7 @@ >> "nanosecond timeout value out of >> range"); >> } >> >> - if (nanos >= 500000 || (nanos != 0 && timeout == 0)) { >> + if (nanos != 0) { >> timeout++; >> } >> >>> But seriously we should just deprecate this version as we're not >>> likely to >>> get any underlying OS mechanisms for doing nanosecond resolution timed >>> blocking actions. >> Object.wait(millis, nanos) is an ugly API (I wish for both a public >> and hotspot internal API that simply operated on nanoseconds), but >> it's not totally awful. It's not obvious to me that computing >> advances over the lifetime of the java platform won't allow >> sub-millisecond scheduling, and other APIs that do operate on >> nanoseconds (like Process.waitFor) should call this API. > From naoto.sato at oracle.com Wed Nov 19 23:41:52 2014 From: naoto.sato at oracle.com (Naoto Sato) Date: Wed, 19 Nov 2014 15:41:52 -0800 Subject: RFR: 8065159: AttributedString has quadratic resize algorithm In-Reply-To: References: Message-ID: <546D2AC0.7040801@oracle.com> Hi Martin, The fix looks good to me. Although it is not inherently related to your fix, there are two separate declarations of "newRunAttributes" and "newRunAttributeValues" in ensureRunBreak() method and their usages are different! It would be desirable to correct it. Naoto On 11/19/14, 3:00 PM, Martin Buchholz wrote: > Hey Naoto and Masayoshi, > > I haven't sent you a friendly code review in a while. > > http://cr.openjdk.java.net/~martin/webrevs/openjdk9/AttributedString-optimization/ > https://bugs.openjdk.java.net/browse/JDK-8065159 > > (AttributedString could also independently see some testing and code > hygiene love) > From mandy.chung at oracle.com Wed Nov 19 23:49:16 2014 From: mandy.chung at oracle.com (Mandy Chung) Date: Wed, 19 Nov 2014 15:49:16 -0800 Subject: RFR: 8065138 - Encodings.isRecognizedEnconding sometimes fails to recognize 'UTF8' In-Reply-To: <546D0285.3090504@oracle.com> References: <546C7C2D.3060805@oracle.com> <546CDE30.6030907@oracle.com> <546CFF3C.2030904@oracle.com> <546D0285.3090504@oracle.com> Message-ID: <546D2C7C.5020103@oracle.com> On 11/19/2014 12:50 PM, Daniel Fuchs wrote: > On 11/19/14 9:36 PM, Mandy Chung wrote: >> resources.jar will be gone when we move to the modular runtime image >> (JEP 220 [1]). >> JDK-8065138 and JDK-8065365 will become non-issue in JDK 9. > Do you mean that the property files will no longer be stripped of > their comments? (sorry for my delay in reply as I was trying to get the number of the resources in the modular image vs resources.jar but got distracted.) The current version copies all bytes when generating the modular image. This is a good question whether we should strip off the comments when writing to the modular runtime image. I think we should look at the footprint and any performance saving and determine if we should do the same in JDK 9. Mandy > This is what is causing the issue. 'sed' doesn't like the special > character. > > -- daniel > >> >> Mandy >> [1] http://openjdk.java.net/jeps/220 >> >> On 11/19/2014 10:15 AM, Daniel Fuchs wrote: >>>> Isn't that a bug in the build system that really ought to be fixed? >>>> >>>> If properties files are to be stored as resources in jar files, they >>>> should either be incorporated byte-for-byte identical, or they should >>>> be decoded using ISO-8859-1 (as specified). It may be best to leave >>>> non-ASCII characters in the source files, as a "test" of the build >>>> system and the jdk itself. >>> >>> Hmmm. If the character is indeed legal then you're right, fixing >>> the build is probably a better idea. >>> >>> However the issue seems to be with using 'sed' over property files: >>> >>> If I simply do: >>> >>> cat >>> jaxp/src/java.xml/share/classes/com/sun/org/apache/xml/internal/serializer/Encodings.properties >>> | sed 's,x,x,g' >>> >>> on my machine, it balks with: >>> >>> sed: RE error: illegal byte sequence >> > From mandy.chung at oracle.com Thu Nov 20 00:09:10 2014 From: mandy.chung at oracle.com (Mandy Chung) Date: Wed, 19 Nov 2014 16:09:10 -0800 Subject: RFR: 8065138 - Encodings.isRecognizedEnconding sometimes fails to recognize 'UTF8' In-Reply-To: <546D2C7C.5020103@oracle.com> References: <546C7C2D.3060805@oracle.com> <546CDE30.6030907@oracle.com> <546CFF3C.2030904@oracle.com> <546D0285.3090504@oracle.com> <546D2C7C.5020103@oracle.com> Message-ID: <546D3126.5010301@oracle.com> On 11/19/2014 3:49 PM, Mandy Chung wrote: > > On 11/19/2014 12:50 PM, Daniel Fuchs wrote: >> On 11/19/14 9:36 PM, Mandy Chung wrote: >>> resources.jar will be gone when we move to the modular runtime image >>> (JEP 220 [1]). >>> JDK-8065138 and JDK-8065365 will become non-issue in JDK 9. >> Do you mean that the property files will no longer be stripped of >> their comments? > > (sorry for my delay in reply as I was trying to get the number of the > resources in the modular image vs resources.jar but got distracted.) > > The current version copies all bytes when generating the modular > image. This is a good question whether we should strip off the > comments when writing to the modular runtime image. I think we > should look at the footprint and any performance saving and determine > if we should do the same in JDK 9. > I looked at the exploded image under BUILD_OUTPUTDIR/jdk/modules/java.xml and found that the comments of Encodings.properties are stripped. I was confused with the mention of resources.jar that I assume it was a step stripping the comments before writing to resources.jar. This is still an issue in jigsaw m2 I believe. Where does the build strip the comments? Mandy From martinrb at google.com Thu Nov 20 00:37:42 2014 From: martinrb at google.com (Martin Buchholz) Date: Wed, 19 Nov 2014 16:37:42 -0800 Subject: RFR: 8065159: AttributedString has quadratic resize algorithm In-Reply-To: <546D2AC0.7040801@oracle.com> References: <546D2AC0.7040801@oracle.com> Message-ID: Thanks Naoto! Yeah, I noticed that too, but I'm not comfortable enough with this code to suggest a really good naming scheme. There are 3 levels of pluralization here - Attribute Attributes Attributeses [sic]!? On Wed, Nov 19, 2014 at 3:41 PM, Naoto Sato wrote: > Hi Martin, > > The fix looks good to me. Although it is not inherently related to your fix, > there are two separate declarations of "newRunAttributes" and > "newRunAttributeValues" in ensureRunBreak() method and their usages are > different! It would be desirable to correct it. > > Naoto > > > On 11/19/14, 3:00 PM, Martin Buchholz wrote: >> >> Hey Naoto and Masayoshi, >> >> I haven't sent you a friendly code review in a while. >> >> >> http://cr.openjdk.java.net/~martin/webrevs/openjdk9/AttributedString-optimization/ >> https://bugs.openjdk.java.net/browse/JDK-8065159 >> >> (AttributedString could also independently see some testing and code >> hygiene love) >> > From huizhe.wang at oracle.com Thu Nov 20 01:07:26 2014 From: huizhe.wang at oracle.com (huizhe wang) Date: Wed, 19 Nov 2014 17:07:26 -0800 Subject: RFR: 8065138 - Encodings.isRecognizedEnconding sometimes fails to recognize 'UTF8' In-Reply-To: <546D3126.5010301@oracle.com> References: <546C7C2D.3060805@oracle.com> <546CDE30.6030907@oracle.com> <546CFF3C.2030904@oracle.com> <546D0285.3090504@oracle.com> <546D2C7C.5020103@oracle.com> <546D3126.5010301@oracle.com> Message-ID: <546D3ECE.2040500@oracle.com> On 11/19/2014 4:09 PM, Mandy Chung wrote: > > On 11/19/2014 3:49 PM, Mandy Chung wrote: >> >> On 11/19/2014 12:50 PM, Daniel Fuchs wrote: >>> On 11/19/14 9:36 PM, Mandy Chung wrote: >>>> resources.jar will be gone when we move to the modular runtime >>>> image (JEP 220 [1]). >>>> JDK-8065138 and JDK-8065365 will become non-issue in JDK 9. >>> Do you mean that the property files will no longer be stripped of >>> their comments? >> >> (sorry for my delay in reply as I was trying to get the number of the >> resources in the modular image vs resources.jar but got distracted.) >> >> The current version copies all bytes when generating the modular >> image. This is a good question whether we should strip off the >> comments when writing to the modular runtime image. I think we >> should look at the footprint and any performance saving and determine >> if we should do the same in JDK 9. >> > > I looked at the exploded image under BUILD_OUTPUTDIR/jdk/modules/java.xml > and found that the comments of Encodings.properties are stripped. I was > confused with the mention of resources.jar that I assume it was a step > stripping the comments before writing to resources.jar. This is still > an issue in jigsaw m2 I believe. > > Where does the build strip the comments? A previous issue was that the build process strips off anything after '#' when copying properties files. In JDK8: jaxp/make/BuildJaxp.gmk: $(JAXP_OUTPUTDIR)/classes/%.properties: $(JAXP_TOPDIR)/src/%.properties $(MKDIR) -p $(@D) $(RM) $@ $@.tmp $(CAT) $< | LANG=C $(NAWK) '{ sub(/#.*$$/,"#"); print }' > $@.tmp $(MV) $@.tmp $@ This was fixed in JDK 9. The per-repo process was removed. It looks like the properties processing process is now consolidated into make/common/JavaCompilation.gmk. So the issue Daniel found is new (in terms of stripping). Search 'properties files' to see the macro. Joe > > Mandy > From martinrb at google.com Thu Nov 20 01:18:38 2014 From: martinrb at google.com (Martin Buchholz) Date: Wed, 19 Nov 2014 17:18:38 -0800 Subject: RFR: 8065138 - Encodings.isRecognizedEnconding sometimes fails to recognize 'UTF8' In-Reply-To: <546D3ECE.2040500@oracle.com> References: <546C7C2D.3060805@oracle.com> <546CDE30.6030907@oracle.com> <546CFF3C.2030904@oracle.com> <546D0285.3090504@oracle.com> <546D2C7C.5020103@oracle.com> <546D3126.5010301@oracle.com> <546D3ECE.2040500@oracle.com> Message-ID: [+ build-dev] I think I see the problem. By default, a Unix pipeline sadly fails only when the last command fails. In bash, you can change this to a more sensible default via set -o pipefail but that's not portable enough for openjdk. $(CAT) $$< | $(SED) -e 's/\([^\\]\):/\1\\:/g' -e 's/\([^\\]\)=/\1\\=/g' \ -e 's/\([^\\]\)!/\1\\!/g' -e 's/#.*/#/g' \ | $(SED) -f "$(SRC_ROOT)/make/common/support/unicode2x.sed" \ | $(SED) -e '/^#/d' -e '/^$$$$/d' \ -e :a -e '/\\$$$$/N; s/\\\n//; ta' \ -e 's/^[ ]*//;s/[ ]*$$$$//' \ -e 's/\\=/=/' | LC_ALL=C $(SORT) > $$@ On Wed, Nov 19, 2014 at 5:07 PM, huizhe wang wrote: > > On 11/19/2014 4:09 PM, Mandy Chung wrote: >> >> >> On 11/19/2014 3:49 PM, Mandy Chung wrote: >>> >>> >>> On 11/19/2014 12:50 PM, Daniel Fuchs wrote: >>>> >>>> On 11/19/14 9:36 PM, Mandy Chung wrote: >>>>> >>>>> resources.jar will be gone when we move to the modular runtime image >>>>> (JEP 220 [1]). >>>>> JDK-8065138 and JDK-8065365 will become non-issue in JDK 9. >>>> >>>> Do you mean that the property files will no longer be stripped of their >>>> comments? >>> >>> >>> (sorry for my delay in reply as I was trying to get the number of the >>> resources in the modular image vs resources.jar but got distracted.) >>> >>> The current version copies all bytes when generating the modular image. >>> This is a good question whether we should strip off the comments when >>> writing to the modular runtime image. I think we should look at the >>> footprint and any performance saving and determine if we should do the same >>> in JDK 9. >>> >> >> I looked at the exploded image under BUILD_OUTPUTDIR/jdk/modules/java.xml >> and found that the comments of Encodings.properties are stripped. I was >> confused with the mention of resources.jar that I assume it was a step >> stripping the comments before writing to resources.jar. This is still >> an issue in jigsaw m2 I believe. >> >> Where does the build strip the comments? > > > A previous issue was that the build process strips off anything after '#' > when copying properties files. In JDK8: > jaxp/make/BuildJaxp.gmk: > $(JAXP_OUTPUTDIR)/classes/%.properties: $(JAXP_TOPDIR)/src/%.properties > $(MKDIR) -p $(@D) > $(RM) $@ $@.tmp > $(CAT) $< | LANG=C $(NAWK) '{ sub(/#.*$$/,"#"); print }' > $@.tmp > $(MV) $@.tmp $@ > > This was fixed in JDK 9. The per-repo process was removed. It looks like > the properties processing process is now consolidated into > make/common/JavaCompilation.gmk. So the issue Daniel found is new (in terms > of stripping). Search 'properties files' to see the macro. > > Joe > >> >> Mandy >> > From martinrb at google.com Thu Nov 20 01:20:45 2014 From: martinrb at google.com (Martin Buchholz) Date: Wed, 19 Nov 2014 17:20:45 -0800 Subject: RFR: 8065138 - Encodings.isRecognizedEnconding sometimes fails to recognize 'UTF8' In-Reply-To: References: <546C7C2D.3060805@oracle.com> <546CDE30.6030907@oracle.com> <546CFF3C.2030904@oracle.com> <546D0285.3090504@oracle.com> <546D2C7C.5020103@oracle.com> <546D3126.5010301@oracle.com> <546D3ECE.2040500@oracle.com> Message-ID: Amusingly, the $(SORT) has an LC_ALL=C carefully placed before it, but the $(SED)s need it too! On Wed, Nov 19, 2014 at 5:18 PM, Martin Buchholz wrote: > [+ build-dev] > > I think I see the problem. By default, a Unix pipeline sadly fails > only when the last command fails. In bash, you can change this to a > more sensible default via > set -o pipefail > but that's not portable enough for openjdk. > > $(CAT) $$< | $(SED) -e 's/\([^\\]\):/\1\\:/g' -e 's/\([^\\]\)=/\1\\=/g' \ > -e 's/\([^\\]\)!/\1\\!/g' -e 's/#.*/#/g' \ > | $(SED) -f "$(SRC_ROOT)/make/common/support/unicode2x.sed" \ > | $(SED) -e '/^#/d' -e '/^$$$$/d' \ > -e :a -e '/\\$$$$/N; s/\\\n//; ta' \ > -e 's/^[ ]*//;s/[ ]*$$$$//' \ > -e 's/\\=/=/' | LC_ALL=C $(SORT) > $$@ > > On Wed, Nov 19, 2014 at 5:07 PM, huizhe wang wrote: >> >> On 11/19/2014 4:09 PM, Mandy Chung wrote: >>> >>> >>> On 11/19/2014 3:49 PM, Mandy Chung wrote: >>>> >>>> >>>> On 11/19/2014 12:50 PM, Daniel Fuchs wrote: >>>>> >>>>> On 11/19/14 9:36 PM, Mandy Chung wrote: >>>>>> >>>>>> resources.jar will be gone when we move to the modular runtime image >>>>>> (JEP 220 [1]). >>>>>> JDK-8065138 and JDK-8065365 will become non-issue in JDK 9. >>>>> >>>>> Do you mean that the property files will no longer be stripped of their >>>>> comments? >>>> >>>> >>>> (sorry for my delay in reply as I was trying to get the number of the >>>> resources in the modular image vs resources.jar but got distracted.) >>>> >>>> The current version copies all bytes when generating the modular image. >>>> This is a good question whether we should strip off the comments when >>>> writing to the modular runtime image. I think we should look at the >>>> footprint and any performance saving and determine if we should do the same >>>> in JDK 9. >>>> >>> >>> I looked at the exploded image under BUILD_OUTPUTDIR/jdk/modules/java.xml >>> and found that the comments of Encodings.properties are stripped. I was >>> confused with the mention of resources.jar that I assume it was a step >>> stripping the comments before writing to resources.jar. This is still >>> an issue in jigsaw m2 I believe. >>> >>> Where does the build strip the comments? >> >> >> A previous issue was that the build process strips off anything after '#' >> when copying properties files. In JDK8: >> jaxp/make/BuildJaxp.gmk: >> $(JAXP_OUTPUTDIR)/classes/%.properties: $(JAXP_TOPDIR)/src/%.properties >> $(MKDIR) -p $(@D) >> $(RM) $@ $@.tmp >> $(CAT) $< | LANG=C $(NAWK) '{ sub(/#.*$$/,"#"); print }' > $@.tmp >> $(MV) $@.tmp $@ >> >> This was fixed in JDK 9. The per-repo process was removed. It looks like >> the properties processing process is now consolidated into >> make/common/JavaCompilation.gmk. So the issue Daniel found is new (in terms >> of stripping). Search 'properties files' to see the macro. >> >> Joe >> >>> >>> Mandy >>> >> From naoto.sato at oracle.com Thu Nov 20 02:20:50 2014 From: naoto.sato at oracle.com (Naoto Sato) Date: Wed, 19 Nov 2014 18:20:50 -0800 Subject: RFR: 8065159: AttributedString has quadratic resize algorithm In-Reply-To: References: <546D2AC0.7040801@oracle.com> Message-ID: <546D5002.30006@oracle.com> OK, fine by me. Naoto On 11/19/14 4:37 PM, Martin Buchholz wrote: > Thanks Naoto! > > Yeah, I noticed that too, but I'm not comfortable enough with this > code to suggest a really good naming scheme. > There are 3 levels of pluralization here - Attribute Attributes > Attributeses [sic]!? > > On Wed, Nov 19, 2014 at 3:41 PM, Naoto Sato wrote: >> Hi Martin, >> >> The fix looks good to me. Although it is not inherently related to your fix, >> there are two separate declarations of "newRunAttributes" and >> "newRunAttributeValues" in ensureRunBreak() method and their usages are >> different! It would be desirable to correct it. >> >> Naoto >> >> >> On 11/19/14, 3:00 PM, Martin Buchholz wrote: >>> >>> Hey Naoto and Masayoshi, >>> >>> I haven't sent you a friendly code review in a while. >>> >>> >>> http://cr.openjdk.java.net/~martin/webrevs/openjdk9/AttributedString-optimization/ >>> https://bugs.openjdk.java.net/browse/JDK-8065159 >>> >>> (AttributedString could also independently see some testing and code >>> hygiene love) >>> >> From masayoshi.okutsu at oracle.com Thu Nov 20 05:18:01 2014 From: masayoshi.okutsu at oracle.com (Masayoshi Okutsu) Date: Thu, 20 Nov 2014 14:18:01 +0900 Subject: RFR: 8065159: AttributedString has quadratic resize algorithm In-Reply-To: <546D5002.30006@oracle.com> References: <546D2AC0.7040801@oracle.com> <546D5002.30006@oracle.com> Message-ID: <546D7989.2080307@oracle.com> Looks good to me. Masayoshi On 11/20/2014 11:20 AM, Naoto Sato wrote: > OK, fine by me. > > Naoto > > On 11/19/14 4:37 PM, Martin Buchholz wrote: >> Thanks Naoto! >> >> Yeah, I noticed that too, but I'm not comfortable enough with this >> code to suggest a really good naming scheme. >> There are 3 levels of pluralization here - Attribute Attributes >> Attributeses [sic]!? >> >> On Wed, Nov 19, 2014 at 3:41 PM, Naoto Sato >> wrote: >>> Hi Martin, >>> >>> The fix looks good to me. Although it is not inherently related to >>> your fix, >>> there are two separate declarations of "newRunAttributes" and >>> "newRunAttributeValues" in ensureRunBreak() method and their usages are >>> different! It would be desirable to correct it. >>> >>> Naoto >>> >>> >>> On 11/19/14, 3:00 PM, Martin Buchholz wrote: >>>> >>>> Hey Naoto and Masayoshi, >>>> >>>> I haven't sent you a friendly code review in a while. >>>> >>>> >>>> http://cr.openjdk.java.net/~martin/webrevs/openjdk9/AttributedString-optimization/ >>>> >>>> https://bugs.openjdk.java.net/browse/JDK-8065159 >>>> >>>> (AttributedString could also independently see some testing and code >>>> hygiene love) >>>> >>> From pavel.rappo at oracle.com Thu Nov 20 09:22:39 2014 From: pavel.rappo at oracle.com (Pavel Rappo) Date: Thu, 20 Nov 2014 09:22:39 +0000 Subject: Library enhancement proposal for copying data from/to Reader/Writer InputStream/OutputStream In-Reply-To: <566A428C-1A13-4DEE-940B-2EBE375F2A7A@reini.net> References: <566A428C-1A13-4DEE-940B-2EBE375F2A7A@reini.net> Message-ID: <51329A99-55E5-4A42-9180-D872366B4DEE@oracle.com> Hi Patrick, There is at least one method in the JDK with similar characteristics: java.nio.file.Files#copy(java.io.InputStream, java.io.OutputStream) But, (1) it has a private access (2) even if we made it public I doubt java.nio.file.Files would be a good place for it P.S. The thing that confuses me though, is the progress consumer. I believe this feature is rarely needed (at least in a way you described it). If you want to do something like this, you should probably go asynchronous with full blown solution like what they have in javax.swing.SwingWorker. -Pavel > On 13 Nov 2014, at 19:31, Patrick Reinhart wrote: > > Hi there, > > In the followup of a BOF with Stephen Colebourne with his ideas of small library changes that may could get in JDK9. As of the fact that in the codebase of my company there are several locations where we copy from/to IO stream over and over again using either external libraries or do it by ourselves, I suggested to have some support for easy coping data between Input-/OutputStream or Reader/Writer without having to use to external libraries. > > My suggestion would look something like this: > > try (InputStream in = ? ; OutputStream out = ?) { > in.copyTo(out); > } > > Or from the other end: > > try (InputStream in = ? ; OutputStream out = ?) { > out.copyFrom(in); > } > > The same for character based streams: > > try (Reader in = ?; Writer out = ?) { > in.copyTo(out); > } > > or > > try (Reader in = ?; Writer out = ?) { > out.copyFrom(in); > } > > > What do you think about this idea? I could imagine, that we could also pass a IntConsumer / LongConsumer in, that will be called with each amount of successfully copied amount of bytes/chars in case we need a total amount of data or displaying the progress of copy externally. > > I?m looking forward to your opinions upon this proposal. > > Cheers > > Patrick From erik.joelsson at oracle.com Thu Nov 20 09:26:19 2014 From: erik.joelsson at oracle.com (Erik Joelsson) Date: Thu, 20 Nov 2014 10:26:19 +0100 Subject: RFR: 8065138 - Encodings.isRecognizedEnconding sometimes fails to recognize 'UTF8' In-Reply-To: References: <546C7C2D.3060805@oracle.com> <546CDE30.6030907@oracle.com> <546CFF3C.2030904@oracle.com> <546D0285.3090504@oracle.com> <546D2C7C.5020103@oracle.com> <546D3126.5010301@oracle.com> <546D3ECE.2040500@oracle.com> Message-ID: <546DB3BB.7020901@oracle.com> Hello, On 2014-11-20 02:20, Martin Buchholz wrote: > Amusingly, the $(SORT) has an LC_ALL=C carefully placed before it, but > the $(SED)s need it too! Yes, I think that's the correct fix in this case. > On Wed, Nov 19, 2014 at 5:18 PM, Martin Buchholz wrote: >> [+ build-dev] >> >> I think I see the problem. By default, a Unix pipeline sadly fails >> only when the last command fails. In bash, you can change this to a >> more sensible default via >> set -o pipefail >> but that's not portable enough for openjdk. This is interesting and something I had missed. I will experiment with enabling pipefail if configure finds support for it. This will include setting the SHELL to bash. We really should fail instead of silently generating broken builds. Daniel, I can take over this bug if you want and work on a proper build fix. /Erik >> $(CAT) $$< | $(SED) -e 's/\([^\\]\):/\1\\:/g' -e 's/\([^\\]\)=/\1\\=/g' \ >> -e 's/\([^\\]\)!/\1\\!/g' -e 's/#.*/#/g' \ >> | $(SED) -f "$(SRC_ROOT)/make/common/support/unicode2x.sed" \ >> | $(SED) -e '/^#/d' -e '/^$$$$/d' \ >> -e :a -e '/\\$$$$/N; s/\\\n//; ta' \ >> -e 's/^[ ]*//;s/[ ]*$$$$//' \ >> -e 's/\\=/=/' | LC_ALL=C $(SORT) > $$@ >> >> On Wed, Nov 19, 2014 at 5:07 PM, huizhe wang wrote: >>> On 11/19/2014 4:09 PM, Mandy Chung wrote: >>>> >>>> On 11/19/2014 3:49 PM, Mandy Chung wrote: >>>>> >>>>> On 11/19/2014 12:50 PM, Daniel Fuchs wrote: >>>>>> On 11/19/14 9:36 PM, Mandy Chung wrote: >>>>>>> resources.jar will be gone when we move to the modular runtime image >>>>>>> (JEP 220 [1]). >>>>>>> JDK-8065138 and JDK-8065365 will become non-issue in JDK 9. >>>>>> Do you mean that the property files will no longer be stripped of their >>>>>> comments? >>>>> >>>>> (sorry for my delay in reply as I was trying to get the number of the >>>>> resources in the modular image vs resources.jar but got distracted.) >>>>> >>>>> The current version copies all bytes when generating the modular image. >>>>> This is a good question whether we should strip off the comments when >>>>> writing to the modular runtime image. I think we should look at the >>>>> footprint and any performance saving and determine if we should do the same >>>>> in JDK 9. >>>>> >>>> I looked at the exploded image under BUILD_OUTPUTDIR/jdk/modules/java.xml >>>> and found that the comments of Encodings.properties are stripped. I was >>>> confused with the mention of resources.jar that I assume it was a step >>>> stripping the comments before writing to resources.jar. This is still >>>> an issue in jigsaw m2 I believe. >>>> >>>> Where does the build strip the comments? >>> >>> A previous issue was that the build process strips off anything after '#' >>> when copying properties files. In JDK8: >>> jaxp/make/BuildJaxp.gmk: >>> $(JAXP_OUTPUTDIR)/classes/%.properties: $(JAXP_TOPDIR)/src/%.properties >>> $(MKDIR) -p $(@D) >>> $(RM) $@ $@.tmp >>> $(CAT) $< | LANG=C $(NAWK) '{ sub(/#.*$$/,"#"); print }' > $@.tmp >>> $(MV) $@.tmp $@ >>> >>> This was fixed in JDK 9. The per-repo process was removed. It looks like >>> the properties processing process is now consolidated into >>> make/common/JavaCompilation.gmk. So the issue Daniel found is new (in terms >>> of stripping). Search 'properties files' to see the macro. >>> >>> Joe >>> >>>> Mandy >>>> From daniel.fuchs at oracle.com Thu Nov 20 09:34:36 2014 From: daniel.fuchs at oracle.com (Daniel Fuchs) Date: Thu, 20 Nov 2014 10:34:36 +0100 Subject: RFR: 8065138 - Encodings.isRecognizedEnconding sometimes fails to recognize 'UTF8' In-Reply-To: <546DB3BB.7020901@oracle.com> References: <546C7C2D.3060805@oracle.com> <546CDE30.6030907@oracle.com> <546CFF3C.2030904@oracle.com> <546D0285.3090504@oracle.com> <546D2C7C.5020103@oracle.com> <546D3126.5010301@oracle.com> <546D3ECE.2040500@oracle.com> <546DB3BB.7020901@oracle.com> Message-ID: <546DB5AC.60401@oracle.com> On 11/20/14 10:26 AM, Erik Joelsson wrote: > Hello, > > On 2014-11-20 02:20, Martin Buchholz wrote: >> Amusingly, the $(SORT) has an LC_ALL=C carefully placed before it, but >> the $(SED)s need it too! > Yes, I think that's the correct fix in this case. >> On Wed, Nov 19, 2014 at 5:18 PM, Martin Buchholz >> wrote: >>> [+ build-dev] >>> >>> I think I see the problem. By default, a Unix pipeline sadly fails >>> only when the last command fails. In bash, you can change this to a >>> more sensible default via >>> set -o pipefail >>> but that's not portable enough for openjdk. > This is interesting and something I had missed. I will experiment with > enabling pipefail if configure finds support for it. This will include > setting the SHELL to bash. We really should fail instead of silently > generating broken builds. > > Daniel, I can take over this bug if you want and work on a proper > build fix. Thanks Erik! You are welcome! So the whole issue seems to be that my default setting is LC_ALL=en_US.UTF-8 whereas sed requires LC_ALL=C to work properly on these property files... When the test first failed I tried to rerun the test with LC_ALL=C - with no success of course. But I never thought of rebuilding with LC_ALL=C :-( My apologies for the red herring :-( best regards -- daniel > > /Erik >>> $(CAT) $$< | $(SED) -e 's/\([^\\]\):/\1\\:/g' -e >>> 's/\([^\\]\)=/\1\\=/g' \ >>> -e 's/\([^\\]\)!/\1\\!/g' -e 's/#.*/#/g' \ >>> | $(SED) -f "$(SRC_ROOT)/make/common/support/unicode2x.sed" \ >>> | $(SED) -e '/^#/d' -e '/^$$$$/d' \ >>> -e :a -e '/\\$$$$/N; s/\\\n//; ta' \ >>> -e 's/^[ ]*//;s/[ ]*$$$$//' \ >>> -e 's/\\=/=/' | LC_ALL=C $(SORT) > $$@ >>> >>> On Wed, Nov 19, 2014 at 5:07 PM, huizhe wang >>> wrote: >>>> On 11/19/2014 4:09 PM, Mandy Chung wrote: >>>>> >>>>> On 11/19/2014 3:49 PM, Mandy Chung wrote: >>>>>> >>>>>> On 11/19/2014 12:50 PM, Daniel Fuchs wrote: >>>>>>> On 11/19/14 9:36 PM, Mandy Chung wrote: >>>>>>>> resources.jar will be gone when we move to the modular runtime >>>>>>>> image >>>>>>>> (JEP 220 [1]). >>>>>>>> JDK-8065138 and JDK-8065365 will become non-issue in JDK 9. >>>>>>> Do you mean that the property files will no longer be stripped >>>>>>> of their >>>>>>> comments? >>>>>> >>>>>> (sorry for my delay in reply as I was trying to get the number of >>>>>> the >>>>>> resources in the modular image vs resources.jar but got distracted.) >>>>>> >>>>>> The current version copies all bytes when generating the modular >>>>>> image. >>>>>> This is a good question whether we should strip off the comments >>>>>> when >>>>>> writing to the modular runtime image. I think we should look at >>>>>> the >>>>>> footprint and any performance saving and determine if we should >>>>>> do the same >>>>>> in JDK 9. >>>>>> >>>>> I looked at the exploded image under >>>>> BUILD_OUTPUTDIR/jdk/modules/java.xml >>>>> and found that the comments of Encodings.properties are stripped. >>>>> I was >>>>> confused with the mention of resources.jar that I assume it was a >>>>> step >>>>> stripping the comments before writing to resources.jar. This is still >>>>> an issue in jigsaw m2 I believe. >>>>> >>>>> Where does the build strip the comments? >>>> >>>> A previous issue was that the build process strips off anything >>>> after '#' >>>> when copying properties files. In JDK8: >>>> jaxp/make/BuildJaxp.gmk: >>>> $(JAXP_OUTPUTDIR)/classes/%.properties: >>>> $(JAXP_TOPDIR)/src/%.properties >>>> $(MKDIR) -p $(@D) >>>> $(RM) $@ $@.tmp >>>> $(CAT) $< | LANG=C $(NAWK) '{ sub(/#.*$$/,"#"); print }' > >>>> $@.tmp >>>> $(MV) $@.tmp $@ >>>> >>>> This was fixed in JDK 9. The per-repo process was removed. It >>>> looks like >>>> the properties processing process is now consolidated into >>>> make/common/JavaCompilation.gmk. So the issue Daniel found is new >>>> (in terms >>>> of stripping). Search 'properties files' to see the macro. >>>> >>>> Joe >>>> >>>>> Mandy >>>>> > From staffan.friberg at oracle.com Thu Nov 20 12:45:12 2014 From: staffan.friberg at oracle.com (Staffan Friberg) Date: Thu, 20 Nov 2014 13:45:12 +0100 Subject: RFR JDK-6321472: Add CRC-32C API In-Reply-To: <545B8DD6.5030805@oracle.com> References: <544073EA.3060304@oracle.com> <5440F803.2000406@oracle.com> <544166B8.8090103@oracle.com> <5441A192.8090602@gmail.com> <544697D2.6080301@oracle.com> <5446AAB0.3090304@oracle.com> <5446B551.7050605@oracle.com> <5446BDE1.9090709@oracle.com> <5446CB91.3050503@oracle.com> <5446DB24.4000507@oracle.com> <545B4AE7.4000208@oracle.com> <545B8DD6.5030805@oracle.com> Message-ID: <546DE258.6030600@oracle.com> Hi, Anyone who can be the second Reviewer? Thanks, Staffan On 11/06/2014 04:03 PM, Staffan Friberg wrote: > Hi Andrej, > > Thanks for your comments. New webrev, > http://cr.openjdk.java.net/~sfriberg/JDK-6321472/webrev.08 > > Indeed more common :) > > jdk/src/java.base/share$ grep -R "private final static" *|wc -l > 282 > jdk/src/java.base/share$ grep -R "private static final" *|wc -l > 3274 > > //Staffan > > On 11/06/2014 12:57 PM, Andrej Golovnin wrote: >> Hi Staffan, >> >> I'm not a reviewer but I have small remarks to the code style: >> >> - the instance variable "crc" is declared bevor the class variables. >> I would move it closer to the constructor. >> >> - documentation comments should be used for the fields "crc" and >> "CRC32C_POLY", e.g. instead of >> >> // Calculated CRC-32C value >> private int crc = 0xFFFFFFFF; >> >> use this: >> >> /** >> * Calculated CRC-32C value >> */ >> private int crc = 0xFFFFFFFF; >> >> >> - the description of the implemented algorithm should use block >> comments and should be moved to the top of the class, e.g.: >> >> public final class CRC32C implements Checksum { >> >> /* >> * This CRC-32C implementation uses the 'slicing-by-8' algorithm >> described >> * in the paper "A Systematic Approach to Building High Performance >> * Software-Based CRC Generators" by Michael E. Kounavis and >> Frank L. Berry, >> * Intel Research and Development >> */ >> >> ... >> >> In other case I get the filling that the Usafe class implements the >> algorithm. :-) >> >> - why some of the fields declared as "private transient final >> static"? I would say "transient" is not needed here. Or am I missing >> something? And I would use "private static final". I think this is >> the preferred way in the JDK to declare a constant. >> >> 61 private transient final static Unsafe UNSAFE = >> Unsafe.getUnsafe(); >> 62 >> 63 // Lookup tables >> 64 private transient final static int[][] byteTables = new >> int[8][256]; >> 65 private transient final static int[] byteTable; >> 66 private transient final static int[] byteTable0 = >> byteTables[0]; >> >> >> Best regards, >> Andrej Golovnin > From erik.joelsson at oracle.com Thu Nov 20 13:36:48 2014 From: erik.joelsson at oracle.com (Erik Joelsson) Date: Thu, 20 Nov 2014 14:36:48 +0100 Subject: RFR: 8065138 - Encodings.isRecognizedEnconding sometimes fails to recognize 'UTF8' In-Reply-To: <546DB5AC.60401@oracle.com> References: <546C7C2D.3060805@oracle.com> <546CDE30.6030907@oracle.com> <546CFF3C.2030904@oracle.com> <546D0285.3090504@oracle.com> <546D2C7C.5020103@oracle.com> <546D3126.5010301@oracle.com> <546D3ECE.2040500@oracle.com> <546DB3BB.7020901@oracle.com> <546DB5AC.60401@oracle.com> Message-ID: <546DEE70.6020306@oracle.com> Here is my proposal for fixing the particular issue of generating the correct properties files. I'm simply adding LC_ALL=C to the whole command line instead of just the sort at the end. It seems to require using "export" to get picked up. Bug: https://bugs.openjdk.java.net/browse/JDK-8065138 Patch: diff --git a/make/common/JavaCompilation.gmk b/make/common/JavaCompilation.gmk --- a/make/common/JavaCompilation.gmk +++ b/make/common/JavaCompilation.gmk @@ -400,13 +400,15 @@ # Now we can setup the depency that will trigger the copying. $$($1_BIN)$$($2_TARGET) : $2 $(MKDIR) -p $$(@D) - $(CAT) $$< | $(SED) -e 's/\([^\\]\):/\1\\:/g' -e 's/\([^\\]\)=/\1\\=/g' \ + export LC_ALL=C ; $(CAT) $$< \ + | $(SED) -e 's/\([^\\]\):/\1\\:/g' -e 's/\([^\\]\)=/\1\\=/g' \ -e 's/\([^\\]\)!/\1\\!/g' -e 's/#.*/#/g' \ | $(SED) -f "$(SRC_ROOT)/make/common/support/unicode2x.sed" \ | $(SED) -e '/^#/d' -e '/^$$$$/d' \ -e :a -e '/\\$$$$/N; s/\\\n//; ta' \ -e 's/^[ ]*//;s/[ ]*$$$$//' \ - -e 's/\\=/=/' | LC_ALL=C $(SORT) > $$@ + -e 's/\\=/=/' \ + | $(SORT) > $$@ $(CHMOD) -f ug+w $$@ # And do not forget this target I filed a separate issue [1] for investigating the use of pipefail. /Erik [1] https://bugs.openjdk.java.net/browse/JDK-8065576 On 2014-11-20 10:34, Daniel Fuchs wrote: > On 11/20/14 10:26 AM, Erik Joelsson wrote: >> Hello, >> >> On 2014-11-20 02:20, Martin Buchholz wrote: >>> Amusingly, the $(SORT) has an LC_ALL=C carefully placed before it, but >>> the $(SED)s need it too! >> Yes, I think that's the correct fix in this case. >>> On Wed, Nov 19, 2014 at 5:18 PM, Martin Buchholz >>> wrote: >>>> [+ build-dev] >>>> >>>> I think I see the problem. By default, a Unix pipeline sadly fails >>>> only when the last command fails. In bash, you can change this to a >>>> more sensible default via >>>> set -o pipefail >>>> but that's not portable enough for openjdk. >> This is interesting and something I had missed. I will experiment >> with enabling pipefail if configure finds support for it. This will >> include setting the SHELL to bash. We really should fail instead of >> silently generating broken builds. >> >> Daniel, I can take over this bug if you want and work on a proper >> build fix. > > Thanks Erik! You are welcome! > So the whole issue seems to be that my default setting is > LC_ALL=en_US.UTF-8 > whereas sed requires LC_ALL=C to work properly on these property files... > > When the test first failed I tried to rerun the test with LC_ALL=C - > with no success > of course. But I never thought of rebuilding with LC_ALL=C :-( > > My apologies for the red herring :-( > > best regards > > -- daniel > >> >> /Erik >>>> $(CAT) $$< | $(SED) -e 's/\([^\\]\):/\1\\:/g' -e >>>> 's/\([^\\]\)=/\1\\=/g' \ >>>> -e 's/\([^\\]\)!/\1\\!/g' -e 's/#.*/#/g' \ >>>> | $(SED) -f "$(SRC_ROOT)/make/common/support/unicode2x.sed" \ >>>> | $(SED) -e '/^#/d' -e '/^$$$$/d' \ >>>> -e :a -e '/\\$$$$/N; s/\\\n//; ta' \ >>>> -e 's/^[ ]*//;s/[ ]*$$$$//' \ >>>> -e 's/\\=/=/' | LC_ALL=C $(SORT) > $$@ >>>> >>>> On Wed, Nov 19, 2014 at 5:07 PM, huizhe wang >>>> wrote: >>>>> On 11/19/2014 4:09 PM, Mandy Chung wrote: >>>>>> >>>>>> On 11/19/2014 3:49 PM, Mandy Chung wrote: >>>>>>> >>>>>>> On 11/19/2014 12:50 PM, Daniel Fuchs wrote: >>>>>>>> On 11/19/14 9:36 PM, Mandy Chung wrote: >>>>>>>>> resources.jar will be gone when we move to the modular runtime >>>>>>>>> image >>>>>>>>> (JEP 220 [1]). >>>>>>>>> JDK-8065138 and JDK-8065365 will become non-issue in JDK 9. >>>>>>>> Do you mean that the property files will no longer be stripped >>>>>>>> of their >>>>>>>> comments? >>>>>>> >>>>>>> (sorry for my delay in reply as I was trying to get the number >>>>>>> of the >>>>>>> resources in the modular image vs resources.jar but got >>>>>>> distracted.) >>>>>>> >>>>>>> The current version copies all bytes when generating the modular >>>>>>> image. >>>>>>> This is a good question whether we should strip off the comments >>>>>>> when >>>>>>> writing to the modular runtime image. I think we should look >>>>>>> at the >>>>>>> footprint and any performance saving and determine if we should >>>>>>> do the same >>>>>>> in JDK 9. >>>>>>> >>>>>> I looked at the exploded image under >>>>>> BUILD_OUTPUTDIR/jdk/modules/java.xml >>>>>> and found that the comments of Encodings.properties are stripped. >>>>>> I was >>>>>> confused with the mention of resources.jar that I assume it was a >>>>>> step >>>>>> stripping the comments before writing to resources.jar. This is >>>>>> still >>>>>> an issue in jigsaw m2 I believe. >>>>>> >>>>>> Where does the build strip the comments? >>>>> >>>>> A previous issue was that the build process strips off anything >>>>> after '#' >>>>> when copying properties files. In JDK8: >>>>> jaxp/make/BuildJaxp.gmk: >>>>> $(JAXP_OUTPUTDIR)/classes/%.properties: >>>>> $(JAXP_TOPDIR)/src/%.properties >>>>> $(MKDIR) -p $(@D) >>>>> $(RM) $@ $@.tmp >>>>> $(CAT) $< | LANG=C $(NAWK) '{ sub(/#.*$$/,"#"); print }' >>>>> > $@.tmp >>>>> $(MV) $@.tmp $@ >>>>> >>>>> This was fixed in JDK 9. The per-repo process was removed. It >>>>> looks like >>>>> the properties processing process is now consolidated into >>>>> make/common/JavaCompilation.gmk. So the issue Daniel found is new >>>>> (in terms >>>>> of stripping). Search 'properties files' to see the macro. >>>>> >>>>> Joe >>>>> >>>>>> Mandy >>>>>> >> > From peter.levart at gmail.com Thu Nov 20 15:33:42 2014 From: peter.levart at gmail.com (Peter Levart) Date: Thu, 20 Nov 2014 16:33:42 +0100 Subject: RFR: 8065172: More core reflection final and volatile annotations In-Reply-To: References: Message-ID: <546E09D6.9040800@gmail.com> Hi Martin, On 11/19/2014 01:42 AM, Martin Buchholz wrote: > Hi Joe, Peter, Paul > > This is the followup on thread safety I promised Peter. Looks good. I made the WildcardTypeImpl.[upperBoundASTs, lowerBoundASTs] and TypeVariableImpl.boundASTs fields volatile in my version of patch (instead of final): http://cr.openjdk.java.net/~plevart/jdk9-dev/GenericsReflectionRaces/webrev.01/ ...so that after the structure they point to has been parsed into bound types, they can be thrown away. The comments indicate that possibility already, but the implementor was afraid to do it because of possible races. I think I got it right here. > As I said before, I'm willing to take the risk to use fields written > via a race if we have a reflective test proving it correct, and the > bug contains the 0.1 version of such a test. > > https://bugs.openjdk.java.net/browse/JDK-8065172 > http://cr.openjdk.java.net/~martin/webrevs/openjdk9/core-reflection-more-safety/ Inspired by your 0.1 version of a test, I created version 0.2: http://cr.openjdk.java.net/~plevart/misc/SunReflectDataRaces/ ...which uses some of your code, but rewritten with heavy use of Stream(s). When run with your proposed patch applied (and all previous patches that are already commited), it finds 20 possible data races. I haven't yet studied any of them. Regards, Peter From paul.sandoz at oracle.com Thu Nov 20 16:23:20 2014 From: paul.sandoz at oracle.com (Paul Sandoz) Date: Thu, 20 Nov 2014 16:23:20 +0000 Subject: [9, 8u40] RFR (M): 8063135: Enable full LF sharing by default In-Reply-To: <546C7146.2030901@oracle.com> References: <546BAAA9.8090009@oracle.com> <546BD858.5040206@oracle.com> <546C7146.2030901@oracle.com> Message-ID: On Nov 19, 2014, at 10:30 AM, Vladimir Ivanov wrote: > Hm, I remember I fixed that long time ago... Seems like I chose a stale patch. Sorry for that. Updated webrev in place. > +1 Paul. From vladimir.x.ivanov at oracle.com Thu Nov 20 15:29:31 2014 From: vladimir.x.ivanov at oracle.com (Vladimir Ivanov) Date: Thu, 20 Nov 2014 19:29:31 +0400 Subject: [9, 8u40] RFR (M): 8063135: Enable full LF sharing by default In-Reply-To: References: <546BAAA9.8090009@oracle.com> <546BD858.5040206@oracle.com> <546C7146.2030901@oracle.com> Message-ID: <546E08DB.709@oracle.com> Paul, Aleksey, thanks for review! Best regards, Vladimir Ivanov On 11/20/14, 8:23 PM, Paul Sandoz wrote: > > On Nov 19, 2014, at 10:30 AM, Vladimir Ivanov wrote: > >> Hm, I remember I fixed that long time ago... Seems like I chose a stale patch. Sorry for that. Updated webrev in place. >> > > +1 > > Paul. > From daniel.fuchs at oracle.com Thu Nov 20 17:25:03 2014 From: daniel.fuchs at oracle.com (Daniel Fuchs) Date: Thu, 20 Nov 2014 18:25:03 +0100 Subject: RFR: 8065138 - Encodings.isRecognizedEnconding sometimes fails to recognize 'UTF8' In-Reply-To: <546DEE70.6020306@oracle.com> References: <546C7C2D.3060805@oracle.com> <546CDE30.6030907@oracle.com> <546CFF3C.2030904@oracle.com> <546D0285.3090504@oracle.com> <546D2C7C.5020103@oracle.com> <546D3126.5010301@oracle.com> <546D3ECE.2040500@oracle.com> <546DB3BB.7020901@oracle.com> <546DB5AC.60401@oracle.com> <546DEE70.6020306@oracle.com> Message-ID: <546E23EF.4000506@oracle.com> On 20/11/14 14:36, Erik Joelsson wrote: > Here is my proposal for fixing the particular issue of generating the > correct properties files. I'm simply adding LC_ALL=C to the whole > command line instead of just the sort at the end. It seems to require > using "export" to get picked up. Hi Erik, Looks good to me. I have applied your patch (manually, because the copy/paste seems to have eaten the tab characters away, which caused patch to reject the diff) - and I confirm that the issue has disappeared. Thanks for solving this! Do you think it would be worth to commit this test modification later on, in a followup Bug/RFE? http://cr.openjdk.java.net/~dfuchs/webrev_8065138/webrev.00/jdk/test/javax/xml/jaxp/Encodings/CheckEncodingPropertiesFile.java.frames.html If so I will take care of it. best regards, -- daniel > Bug: https://bugs.openjdk.java.net/browse/JDK-8065138 > Patch: > diff --git a/make/common/JavaCompilation.gmk > b/make/common/JavaCompilation.gmk > --- a/make/common/JavaCompilation.gmk > +++ b/make/common/JavaCompilation.gmk > @@ -400,13 +400,15 @@ > # Now we can setup the depency that will trigger the copying. > $$($1_BIN)$$($2_TARGET) : $2 > $(MKDIR) -p $$(@D) > - $(CAT) $$< | $(SED) -e 's/\([^\\]\):/\1\\:/g' -e > 's/\([^\\]\)=/\1\\=/g' \ > + export LC_ALL=C ; $(CAT) $$< \ > + | $(SED) -e 's/\([^\\]\):/\1\\:/g' -e 's/\([^\\]\)=/\1\\=/g' \ > -e 's/\([^\\]\)!/\1\\!/g' -e 's/#.*/#/g' \ > | $(SED) -f "$(SRC_ROOT)/make/common/support/unicode2x.sed" \ > | $(SED) -e '/^#/d' -e '/^$$$$/d' \ > -e :a -e '/\\$$$$/N; s/\\\n//; ta' \ > -e 's/^[ ]*//;s/[ ]*$$$$//' \ > - -e 's/\\=/=/' | LC_ALL=C $(SORT) > $$@ > + -e 's/\\=/=/' \ > + | $(SORT) > $$@ > $(CHMOD) -f ug+w $$@ > > # And do not forget this target > > > I filed a separate issue [1] for investigating the use of pipefail. > > /Erik > > [1] https://bugs.openjdk.java.net/browse/JDK-8065576 > > On 2014-11-20 10:34, Daniel Fuchs wrote: >> On 11/20/14 10:26 AM, Erik Joelsson wrote: >>> Hello, >>> >>> On 2014-11-20 02:20, Martin Buchholz wrote: >>>> Amusingly, the $(SORT) has an LC_ALL=C carefully placed before it, but >>>> the $(SED)s need it too! >>> Yes, I think that's the correct fix in this case. >>>> On Wed, Nov 19, 2014 at 5:18 PM, Martin Buchholz >>>> wrote: >>>>> [+ build-dev] >>>>> >>>>> I think I see the problem. By default, a Unix pipeline sadly fails >>>>> only when the last command fails. In bash, you can change this to a >>>>> more sensible default via >>>>> set -o pipefail >>>>> but that's not portable enough for openjdk. >>> This is interesting and something I had missed. I will experiment >>> with enabling pipefail if configure finds support for it. This will >>> include setting the SHELL to bash. We really should fail instead of >>> silently generating broken builds. >>> >>> Daniel, I can take over this bug if you want and work on a proper >>> build fix. >> >> Thanks Erik! You are welcome! >> So the whole issue seems to be that my default setting is >> LC_ALL=en_US.UTF-8 >> whereas sed requires LC_ALL=C to work properly on these property files... >> >> When the test first failed I tried to rerun the test with LC_ALL=C - >> with no success >> of course. But I never thought of rebuilding with LC_ALL=C :-( >> >> My apologies for the red herring :-( >> >> best regards >> >> -- daniel >> >>> >>> /Erik >>>>> $(CAT) $$< | $(SED) -e 's/\([^\\]\):/\1\\:/g' -e >>>>> 's/\([^\\]\)=/\1\\=/g' \ >>>>> -e 's/\([^\\]\)!/\1\\!/g' -e 's/#.*/#/g' \ >>>>> | $(SED) -f "$(SRC_ROOT)/make/common/support/unicode2x.sed" \ >>>>> | $(SED) -e '/^#/d' -e '/^$$$$/d' \ >>>>> -e :a -e '/\\$$$$/N; s/\\\n//; ta' \ >>>>> -e 's/^[ ]*//;s/[ ]*$$$$//' \ >>>>> -e 's/\\=/=/' | LC_ALL=C $(SORT) > $$@ >>>>> >>>>> On Wed, Nov 19, 2014 at 5:07 PM, huizhe wang >>>>> wrote: >>>>>> On 11/19/2014 4:09 PM, Mandy Chung wrote: >>>>>>> >>>>>>> On 11/19/2014 3:49 PM, Mandy Chung wrote: >>>>>>>> >>>>>>>> On 11/19/2014 12:50 PM, Daniel Fuchs wrote: >>>>>>>>> On 11/19/14 9:36 PM, Mandy Chung wrote: >>>>>>>>>> resources.jar will be gone when we move to the modular runtime >>>>>>>>>> image >>>>>>>>>> (JEP 220 [1]). >>>>>>>>>> JDK-8065138 and JDK-8065365 will become non-issue in JDK 9. >>>>>>>>> Do you mean that the property files will no longer be stripped >>>>>>>>> of their >>>>>>>>> comments? >>>>>>>> >>>>>>>> (sorry for my delay in reply as I was trying to get the number >>>>>>>> of the >>>>>>>> resources in the modular image vs resources.jar but got >>>>>>>> distracted.) >>>>>>>> >>>>>>>> The current version copies all bytes when generating the modular >>>>>>>> image. >>>>>>>> This is a good question whether we should strip off the comments >>>>>>>> when >>>>>>>> writing to the modular runtime image. I think we should look >>>>>>>> at the >>>>>>>> footprint and any performance saving and determine if we should >>>>>>>> do the same >>>>>>>> in JDK 9. >>>>>>>> >>>>>>> I looked at the exploded image under >>>>>>> BUILD_OUTPUTDIR/jdk/modules/java.xml >>>>>>> and found that the comments of Encodings.properties are stripped. >>>>>>> I was >>>>>>> confused with the mention of resources.jar that I assume it was a >>>>>>> step >>>>>>> stripping the comments before writing to resources.jar. This is >>>>>>> still >>>>>>> an issue in jigsaw m2 I believe. >>>>>>> >>>>>>> Where does the build strip the comments? >>>>>> >>>>>> A previous issue was that the build process strips off anything >>>>>> after '#' >>>>>> when copying properties files. In JDK8: >>>>>> jaxp/make/BuildJaxp.gmk: >>>>>> $(JAXP_OUTPUTDIR)/classes/%.properties: >>>>>> $(JAXP_TOPDIR)/src/%.properties >>>>>> $(MKDIR) -p $(@D) >>>>>> $(RM) $@ $@.tmp >>>>>> $(CAT) $< | LANG=C $(NAWK) '{ sub(/#.*$$/,"#"); print }' >>>>>> > $@.tmp >>>>>> $(MV) $@.tmp $@ >>>>>> >>>>>> This was fixed in JDK 9. The per-repo process was removed. It >>>>>> looks like >>>>>> the properties processing process is now consolidated into >>>>>> make/common/JavaCompilation.gmk. So the issue Daniel found is new >>>>>> (in terms >>>>>> of stripping). Search 'properties files' to see the macro. >>>>>> >>>>>> Joe >>>>>> >>>>>>> Mandy >>>>>>> >>> >> > From huizhe.wang at oracle.com Thu Nov 20 17:57:36 2014 From: huizhe.wang at oracle.com (huizhe wang) Date: Thu, 20 Nov 2014 09:57:36 -0800 Subject: RFR: 8065138 - Encodings.isRecognizedEnconding sometimes fails to recognize 'UTF8' In-Reply-To: <546E23EF.4000506@oracle.com> References: <546C7C2D.3060805@oracle.com> <546CDE30.6030907@oracle.com> <546CFF3C.2030904@oracle.com> <546D0285.3090504@oracle.com> <546D2C7C.5020103@oracle.com> <546D3126.5010301@oracle.com> <546D3ECE.2040500@oracle.com> <546DB3BB.7020901@oracle.com> <546DB5AC.60401@oracle.com> <546DEE70.6020306@oracle.com> <546E23EF.4000506@oracle.com> Message-ID: <546E2B90.2050503@oracle.com> Awesome! Glad to see the fix in the build process. It's a relief to know we don't have to worry about individual properties files. Daniel, your test looks good to me. That's how jaxp loads Encodings.properties. Thanks, Joe On 11/20/2014 9:25 AM, Daniel Fuchs wrote: > On 20/11/14 14:36, Erik Joelsson wrote: >> Here is my proposal for fixing the particular issue of generating the >> correct properties files. I'm simply adding LC_ALL=C to the whole >> command line instead of just the sort at the end. It seems to require >> using "export" to get picked up. > > Hi Erik, > > Looks good to me. > > I have applied your patch (manually, because the copy/paste > seems to have eaten the tab characters away, which caused patch to > reject the diff) - and I confirm that the issue has disappeared. > > Thanks for solving this! > > Do you think it would be worth to commit this test modification > later on, in a followup Bug/RFE? > > http://cr.openjdk.java.net/~dfuchs/webrev_8065138/webrev.00/jdk/test/javax/xml/jaxp/Encodings/CheckEncodingPropertiesFile.java.frames.html > > > If so I will take care of it. > > best regards, > > -- daniel > > >> Bug: https://bugs.openjdk.java.net/browse/JDK-8065138 >> Patch: >> diff --git a/make/common/JavaCompilation.gmk >> b/make/common/JavaCompilation.gmk >> --- a/make/common/JavaCompilation.gmk >> +++ b/make/common/JavaCompilation.gmk >> @@ -400,13 +400,15 @@ >> # Now we can setup the depency that will trigger the copying. >> $$($1_BIN)$$($2_TARGET) : $2 >> $(MKDIR) -p $$(@D) >> - $(CAT) $$< | $(SED) -e 's/\([^\\]\):/\1\\:/g' -e >> 's/\([^\\]\)=/\1\\=/g' \ >> + export LC_ALL=C ; $(CAT) $$< \ >> + | $(SED) -e 's/\([^\\]\):/\1\\:/g' -e 's/\([^\\]\)=/\1\\=/g' \ >> -e 's/\([^\\]\)!/\1\\!/g' -e 's/#.*/#/g' \ >> | $(SED) -f "$(SRC_ROOT)/make/common/support/unicode2x.sed" \ >> | $(SED) -e '/^#/d' -e '/^$$$$/d' \ >> -e :a -e '/\\$$$$/N; s/\\\n//; ta' \ >> -e 's/^[ ]*//;s/[ ]*$$$$//' \ >> - -e 's/\\=/=/' | LC_ALL=C $(SORT) > $$@ >> + -e 's/\\=/=/' \ >> + | $(SORT) > $$@ >> $(CHMOD) -f ug+w $$@ >> >> # And do not forget this target >> >> >> I filed a separate issue [1] for investigating the use of pipefail. >> >> /Erik >> >> [1] https://bugs.openjdk.java.net/browse/JDK-8065576 >> >> On 2014-11-20 10:34, Daniel Fuchs wrote: >>> On 11/20/14 10:26 AM, Erik Joelsson wrote: >>>> Hello, >>>> >>>> On 2014-11-20 02:20, Martin Buchholz wrote: >>>>> Amusingly, the $(SORT) has an LC_ALL=C carefully placed before it, >>>>> but >>>>> the $(SED)s need it too! >>>> Yes, I think that's the correct fix in this case. >>>>> On Wed, Nov 19, 2014 at 5:18 PM, Martin Buchholz >>>>> wrote: >>>>>> [+ build-dev] >>>>>> >>>>>> I think I see the problem. By default, a Unix pipeline sadly fails >>>>>> only when the last command fails. In bash, you can change this to a >>>>>> more sensible default via >>>>>> set -o pipefail >>>>>> but that's not portable enough for openjdk. >>>> This is interesting and something I had missed. I will experiment >>>> with enabling pipefail if configure finds support for it. This will >>>> include setting the SHELL to bash. We really should fail instead of >>>> silently generating broken builds. >>>> >>>> Daniel, I can take over this bug if you want and work on a proper >>>> build fix. >>> >>> Thanks Erik! You are welcome! >>> So the whole issue seems to be that my default setting is >>> LC_ALL=en_US.UTF-8 >>> whereas sed requires LC_ALL=C to work properly on these property >>> files... >>> >>> When the test first failed I tried to rerun the test with LC_ALL=C - >>> with no success >>> of course. But I never thought of rebuilding with LC_ALL=C :-( >>> >>> My apologies for the red herring :-( >>> >>> best regards >>> >>> -- daniel >>> >>>> >>>> /Erik >>>>>> $(CAT) $$< | $(SED) -e 's/\([^\\]\):/\1\\:/g' -e >>>>>> 's/\([^\\]\)=/\1\\=/g' \ >>>>>> -e 's/\([^\\]\)!/\1\\!/g' -e 's/#.*/#/g' \ >>>>>> | $(SED) -f "$(SRC_ROOT)/make/common/support/unicode2x.sed" \ >>>>>> | $(SED) -e '/^#/d' -e '/^$$$$/d' \ >>>>>> -e :a -e '/\\$$$$/N; s/\\\n//; ta' \ >>>>>> -e 's/^[ ]*//;s/[ ]*$$$$//' \ >>>>>> -e 's/\\=/=/' | LC_ALL=C $(SORT) > $$@ >>>>>> >>>>>> On Wed, Nov 19, 2014 at 5:07 PM, huizhe wang >>>>>> wrote: >>>>>>> On 11/19/2014 4:09 PM, Mandy Chung wrote: >>>>>>>> >>>>>>>> On 11/19/2014 3:49 PM, Mandy Chung wrote: >>>>>>>>> >>>>>>>>> On 11/19/2014 12:50 PM, Daniel Fuchs wrote: >>>>>>>>>> On 11/19/14 9:36 PM, Mandy Chung wrote: >>>>>>>>>>> resources.jar will be gone when we move to the modular runtime >>>>>>>>>>> image >>>>>>>>>>> (JEP 220 [1]). >>>>>>>>>>> JDK-8065138 and JDK-8065365 will become non-issue in JDK 9. >>>>>>>>>> Do you mean that the property files will no longer be stripped >>>>>>>>>> of their >>>>>>>>>> comments? >>>>>>>>> >>>>>>>>> (sorry for my delay in reply as I was trying to get the number >>>>>>>>> of the >>>>>>>>> resources in the modular image vs resources.jar but got >>>>>>>>> distracted.) >>>>>>>>> >>>>>>>>> The current version copies all bytes when generating the modular >>>>>>>>> image. >>>>>>>>> This is a good question whether we should strip off the comments >>>>>>>>> when >>>>>>>>> writing to the modular runtime image. I think we should look >>>>>>>>> at the >>>>>>>>> footprint and any performance saving and determine if we should >>>>>>>>> do the same >>>>>>>>> in JDK 9. >>>>>>>>> >>>>>>>> I looked at the exploded image under >>>>>>>> BUILD_OUTPUTDIR/jdk/modules/java.xml >>>>>>>> and found that the comments of Encodings.properties are stripped. >>>>>>>> I was >>>>>>>> confused with the mention of resources.jar that I assume it was a >>>>>>>> step >>>>>>>> stripping the comments before writing to resources.jar. This is >>>>>>>> still >>>>>>>> an issue in jigsaw m2 I believe. >>>>>>>> >>>>>>>> Where does the build strip the comments? >>>>>>> >>>>>>> A previous issue was that the build process strips off anything >>>>>>> after '#' >>>>>>> when copying properties files. In JDK8: >>>>>>> jaxp/make/BuildJaxp.gmk: >>>>>>> $(JAXP_OUTPUTDIR)/classes/%.properties: >>>>>>> $(JAXP_TOPDIR)/src/%.properties >>>>>>> $(MKDIR) -p $(@D) >>>>>>> $(RM) $@ $@.tmp >>>>>>> $(CAT) $< | LANG=C $(NAWK) '{ sub(/#.*$$/,"#"); print }' >>>>>>> > $@.tmp >>>>>>> $(MV) $@.tmp $@ >>>>>>> >>>>>>> This was fixed in JDK 9. The per-repo process was removed. It >>>>>>> looks like >>>>>>> the properties processing process is now consolidated into >>>>>>> make/common/JavaCompilation.gmk. So the issue Daniel found is new >>>>>>> (in terms >>>>>>> of stripping). Search 'properties files' to see the macro. >>>>>>> >>>>>>> Joe >>>>>>> >>>>>>>> Mandy >>>>>>>> >>>> >>> >> > From patrick at reini.net Thu Nov 20 18:30:00 2014 From: patrick at reini.net (Patrick Reinhart) Date: Thu, 20 Nov 2014 19:30:00 +0100 Subject: Library enhancement proposal for copying data from/to Reader/Writer InputStream/OutputStream In-Reply-To: <51329A99-55E5-4A42-9180-D872366B4DEE@oracle.com> References: <566A428C-1A13-4DEE-940B-2EBE375F2A7A@reini.net> <51329A99-55E5-4A42-9180-D872366B4DEE@oracle.com> Message-ID: Hi Pavel, > Am 20.11.2014 um 10:22 schrieb Pavel Rappo : > > There is at least one method in the JDK with similar characteristics: java.nio.file.Files#copy(java.io.InputStream, java.io.OutputStream) > But, (1) it has a private access (2) even if we made it public I doubt java.nio.file.Files would be a good place for it > I would suggest to introduce a separate IOUtil class holding such utility methods. Additionally to copyTo() and copyFrom() method could be added later for more intuitive usage. Also the copy method within the Files method could be replaced with a static reference to the IOUtil class. > P.S. The thing that confuses me though, is the progress consumer. I believe this feature is rarely needed (at least in a way you described it). > If you want to do something like this, you should probably go asynchronous with full blown solution like what they have in javax.swing.SwingWorker. > > -Pavel The method having a IntConsumer I have already discussed with some other colleagues and they suggested to better use a IntPredicate in order to have the possibility to interrupt the copy process, without having to interrupt any threads. Additionally there is still the possibility to use such a Predicate for counting the total or using the information for other possibilities. Here?s implementation example for Input-/OutputStream (similar code would apply for Reader/Writer): /** * Reads all bytes from an input stream and writes them to an output stream. * * @param source the input stream to read from * @param target the path to the file * * @return the number of bytes read and successfully written * * @throws IOException if an I/O error occurs when reading or writing */ public static long copy(InputStream source, OutputStream target) throws IOException { LongAdder totalread = new LongAdder(); copy(source, target, nread -> { totalread.add(nread); return true; }); return totalread.sum(); } /** * Reads all bytes from an input stream and writes them to an output stream. * While doing so, the given predicate is called with the amount of bytes * being read, to decide whenever they should be written to the output * stream. If the predicate returns false the copy operation is * stopped and no more data is written to the output stream. * * @param source the input stream to read from * @param target the path to the file * @param predicate the predicate tests if the copy operation should proceed * * @throws IOException if an I/O error occurs when reading or writing */ public static void copy(InputStream source, OutputStream target, IntPredicate predicate) throws IOException { byte[] buf = new byte[BUFFER_SIZE]; int n; while ((n = source.read(buf)) > 0 && predicate.test(n)) { target.write(buf, 0, n); } } Cheers Patrick From david.r.chase at oracle.com Thu Nov 20 19:41:23 2014 From: david.r.chase at oracle.com (David Chase) Date: Thu, 20 Nov 2014 14:41:23 -0500 Subject: RFR JDK-6321472: Add CRC-32C API In-Reply-To: <546DE258.6030600@oracle.com> References: <544073EA.3060304@oracle.com> <5440F803.2000406@oracle.com> <544166B8.8090103@oracle.com> <5441A192.8090602@gmail.com> <544697D2.6080301@oracle.com> <5446AAB0.3090304@oracle.com> <5446B551.7050605@oracle.com> <5446BDE1.9090709@oracle.com> <5446CB91.3050503@oracle.com> <5446DB24.4000507@oracle.com> <545B4AE7.4000208@oracle.com> <545B8DD6.5030805@oracle.com> <546DE258.6030600@oracle.com> Message-ID: <83BE5F55-075E-4702-BC7D-319077148428@oracle.com> On 2014-11-20, at 7:45 AM, Staffan Friberg wrote: > Hi, > > Anyone who can be the second Reviewer? > > Thanks, > Staffan I can review, but I am not a Reviewer. CRC32C.java: 56 /** 57 * CRC-32C Polynom 58 */ ?Polynomial?, perhaps? You did test this on both big- and little-endian boxes, right? It looks right to me, but this stuff is brain-hurty on mirror-image dyslexia and fenceposts. It might not hurt to mention that byteTable is for byte-at-a-time calculations and is always the same, whereas the other 8 tables are for bulk operations and will have contents dependent on the endianness of the platform. It would be a good idea to mention that the high-order terms of polynomials are stored in the low-order bits of the integers in the tables. You could get the bit reversal out of the init code if you bit-reversed CRC32C_POLY instead and wrote it 78 int r = index; // High-order polynomial term stored in LSB of r. 79 for (int i = 0; i < Byte.SIZE; i++) { 80 if ((r & 1) != 0) { 81 r = (r >>> 1) ^ REVERSED_CRC32C_POLY; 82 } else { 83 r >>>= 1; 84 } 85 } 86 byteTables[0][index] = r; at least, it seems that way to me. David From joe.darcy at oracle.com Thu Nov 20 20:34:29 2014 From: joe.darcy at oracle.com (joe darcy) Date: Thu, 20 Nov 2014 12:34:29 -0800 Subject: RFR JDK-6321472: Add CRC-32C API In-Reply-To: <546DE258.6030600@oracle.com> References: <544073EA.3060304@oracle.com> <5440F803.2000406@oracle.com> <544166B8.8090103@oracle.com> <5441A192.8090602@gmail.com> <544697D2.6080301@oracle.com> <5446AAB0.3090304@oracle.com> <5446B551.7050605@oracle.com> <5446BDE1.9090709@oracle.com> <5446CB91.3050503@oracle.com> <5446DB24.4000507@oracle.com> <545B4AE7.4000208@oracle.com> <545B8DD6.5030805@oracle.com> <546DE258.6030600@oracle.com> Message-ID: <546E5055.6090806@oracle.com> Hello, Core libs changes (at this stage of JDK 9) only require a single reviewer. Cheers, -Joe On 11/20/2014 4:45 AM, Staffan Friberg wrote: > Hi, > > Anyone who can be the second Reviewer? > > Thanks, > Staffan > > On 11/06/2014 04:03 PM, Staffan Friberg wrote: >> Hi Andrej, >> >> Thanks for your comments. New webrev, >> http://cr.openjdk.java.net/~sfriberg/JDK-6321472/webrev.08 >> >> Indeed more common :) >> >> jdk/src/java.base/share$ grep -R "private final static" *|wc -l >> 282 >> jdk/src/java.base/share$ grep -R "private static final" *|wc -l >> 3274 >> >> //Staffan >> >> On 11/06/2014 12:57 PM, Andrej Golovnin wrote: >>> Hi Staffan, >>> >>> I'm not a reviewer but I have small remarks to the code style: >>> >>> - the instance variable "crc" is declared bevor the class variables. >>> I would move it closer to the constructor. >>> >>> - documentation comments should be used for the fields "crc" and >>> "CRC32C_POLY", e.g. instead of >>> >>> // Calculated CRC-32C value >>> private int crc = 0xFFFFFFFF; >>> >>> use this: >>> >>> /** >>> * Calculated CRC-32C value >>> */ >>> private int crc = 0xFFFFFFFF; >>> >>> >>> - the description of the implemented algorithm should use block >>> comments and should be moved to the top of the class, e.g.: >>> >>> public final class CRC32C implements Checksum { >>> >>> /* >>> * This CRC-32C implementation uses the 'slicing-by-8' >>> algorithm described >>> * in the paper "A Systematic Approach to Building High >>> Performance >>> * Software-Based CRC Generators" by Michael E. Kounavis and >>> Frank L. Berry, >>> * Intel Research and Development >>> */ >>> >>> ... >>> >>> In other case I get the filling that the Usafe class implements the >>> algorithm. :-) >>> >>> - why some of the fields declared as "private transient final >>> static"? I would say "transient" is not needed here. Or am I missing >>> something? And I would use "private static final". I think this is >>> the preferred way in the JDK to declare a constant. >>> >>> 61 private transient final static Unsafe UNSAFE = >>> Unsafe.getUnsafe(); >>> 62 >>> 63 // Lookup tables >>> 64 private transient final static int[][] byteTables = new >>> int[8][256]; >>> 65 private transient final static int[] byteTable; >>> 66 private transient final static int[] byteTable0 = >>> byteTables[0]; >>> >>> >>> Best regards, >>> Andrej Golovnin >> > From mandy.chung at oracle.com Thu Nov 20 22:07:08 2014 From: mandy.chung at oracle.com (Mandy Chung) Date: Thu, 20 Nov 2014 14:07:08 -0800 Subject: RFR: 8065138 - Encodings.isRecognizedEnconding sometimes fails to recognize 'UTF8' In-Reply-To: <546E23EF.4000506@oracle.com> References: <546C7C2D.3060805@oracle.com> <546CDE30.6030907@oracle.com> <546CFF3C.2030904@oracle.com> <546D0285.3090504@oracle.com> <546D2C7C.5020103@oracle.com> <546D3126.5010301@oracle.com> <546D3ECE.2040500@oracle.com> <546DB3BB.7020901@oracle.com> <546DB5AC.60401@oracle.com> <546DEE70.6020306@oracle.com> <546E23EF.4000506@oracle.com> Message-ID: <546E660C.1060909@oracle.com> Daniel, Your test change looks fine and good to push this change that should catch if similar regression occurs in the future. I was tempted to suggest to have a dedicated test for the build-time modification to the properties file. Maybe a low priority RFE. Mandy On 11/20/14 9:25 AM, Daniel Fuchs wrote: > On 20/11/14 14:36, Erik Joelsson wrote: >> Here is my proposal for fixing the particular issue of generating the >> correct properties files. I'm simply adding LC_ALL=C to the whole >> command line instead of just the sort at the end. It seems to require >> using "export" to get picked up. > > Hi Erik, > > Looks good to me. > > I have applied your patch (manually, because the copy/paste > seems to have eaten the tab characters away, which caused patch to > reject the diff) - and I confirm that the issue has disappeared. > > Thanks for solving this! > > Do you think it would be worth to commit this test modification > later on, in a followup Bug/RFE? > > http://cr.openjdk.java.net/~dfuchs/webrev_8065138/webrev.00/jdk/test/javax/xml/jaxp/Encodings/CheckEncodingPropertiesFile.java.frames.html > > > If so I will take care of it. > > best regards, > > -- daniel > > >> Bug: https://bugs.openjdk.java.net/browse/JDK-8065138 >> Patch: >> diff --git a/make/common/JavaCompilation.gmk >> b/make/common/JavaCompilation.gmk >> --- a/make/common/JavaCompilation.gmk >> +++ b/make/common/JavaCompilation.gmk >> @@ -400,13 +400,15 @@ >> # Now we can setup the depency that will trigger the copying. >> $$($1_BIN)$$($2_TARGET) : $2 >> $(MKDIR) -p $$(@D) >> - $(CAT) $$< | $(SED) -e 's/\([^\\]\):/\1\\:/g' -e >> 's/\([^\\]\)=/\1\\=/g' \ >> + export LC_ALL=C ; $(CAT) $$< \ >> + | $(SED) -e 's/\([^\\]\):/\1\\:/g' -e 's/\([^\\]\)=/\1\\=/g' \ >> -e 's/\([^\\]\)!/\1\\!/g' -e 's/#.*/#/g' \ >> | $(SED) -f "$(SRC_ROOT)/make/common/support/unicode2x.sed" \ >> | $(SED) -e '/^#/d' -e '/^$$$$/d' \ >> -e :a -e '/\\$$$$/N; s/\\\n//; ta' \ >> -e 's/^[ ]*//;s/[ ]*$$$$//' \ >> - -e 's/\\=/=/' | LC_ALL=C $(SORT) > $$@ >> + -e 's/\\=/=/' \ >> + | $(SORT) > $$@ >> $(CHMOD) -f ug+w $$@ >> >> # And do not forget this target >> >> >> I filed a separate issue [1] for investigating the use of pipefail. >> >> /Erik >> >> [1] https://bugs.openjdk.java.net/browse/JDK-8065576 >> >> On 2014-11-20 10:34, Daniel Fuchs wrote: >>> On 11/20/14 10:26 AM, Erik Joelsson wrote: >>>> Hello, >>>> >>>> On 2014-11-20 02:20, Martin Buchholz wrote: >>>>> Amusingly, the $(SORT) has an LC_ALL=C carefully placed before it, >>>>> but >>>>> the $(SED)s need it too! >>>> Yes, I think that's the correct fix in this case. >>>>> On Wed, Nov 19, 2014 at 5:18 PM, Martin Buchholz >>>>> wrote: >>>>>> [+ build-dev] >>>>>> >>>>>> I think I see the problem. By default, a Unix pipeline sadly fails >>>>>> only when the last command fails. In bash, you can change this to a >>>>>> more sensible default via >>>>>> set -o pipefail >>>>>> but that's not portable enough for openjdk. >>>> This is interesting and something I had missed. I will experiment >>>> with enabling pipefail if configure finds support for it. This will >>>> include setting the SHELL to bash. We really should fail instead of >>>> silently generating broken builds. >>>> >>>> Daniel, I can take over this bug if you want and work on a proper >>>> build fix. >>> >>> Thanks Erik! You are welcome! >>> So the whole issue seems to be that my default setting is >>> LC_ALL=en_US.UTF-8 >>> whereas sed requires LC_ALL=C to work properly on these property >>> files... >>> >>> When the test first failed I tried to rerun the test with LC_ALL=C - >>> with no success >>> of course. But I never thought of rebuilding with LC_ALL=C :-( >>> >>> My apologies for the red herring :-( >>> >>> best regards >>> >>> -- daniel >>> >>>> >>>> /Erik >>>>>> $(CAT) $$< | $(SED) -e 's/\([^\\]\):/\1\\:/g' -e >>>>>> 's/\([^\\]\)=/\1\\=/g' \ >>>>>> -e 's/\([^\\]\)!/\1\\!/g' -e 's/#.*/#/g' \ >>>>>> | $(SED) -f "$(SRC_ROOT)/make/common/support/unicode2x.sed" \ >>>>>> | $(SED) -e '/^#/d' -e '/^$$$$/d' \ >>>>>> -e :a -e '/\\$$$$/N; s/\\\n//; ta' \ >>>>>> -e 's/^[ ]*//;s/[ ]*$$$$//' \ >>>>>> -e 's/\\=/=/' | LC_ALL=C $(SORT) > $$@ >>>>>> >>>>>> On Wed, Nov 19, 2014 at 5:07 PM, huizhe wang >>>>>> wrote: >>>>>>> On 11/19/2014 4:09 PM, Mandy Chung wrote: >>>>>>>> >>>>>>>> On 11/19/2014 3:49 PM, Mandy Chung wrote: >>>>>>>>> >>>>>>>>> On 11/19/2014 12:50 PM, Daniel Fuchs wrote: >>>>>>>>>> On 11/19/14 9:36 PM, Mandy Chung wrote: >>>>>>>>>>> resources.jar will be gone when we move to the modular runtime >>>>>>>>>>> image >>>>>>>>>>> (JEP 220 [1]). >>>>>>>>>>> JDK-8065138 and JDK-8065365 will become non-issue in JDK 9. >>>>>>>>>> Do you mean that the property files will no longer be stripped >>>>>>>>>> of their >>>>>>>>>> comments? >>>>>>>>> >>>>>>>>> (sorry for my delay in reply as I was trying to get the number >>>>>>>>> of the >>>>>>>>> resources in the modular image vs resources.jar but got >>>>>>>>> distracted.) >>>>>>>>> >>>>>>>>> The current version copies all bytes when generating the modular >>>>>>>>> image. >>>>>>>>> This is a good question whether we should strip off the comments >>>>>>>>> when >>>>>>>>> writing to the modular runtime image. I think we should look >>>>>>>>> at the >>>>>>>>> footprint and any performance saving and determine if we should >>>>>>>>> do the same >>>>>>>>> in JDK 9. >>>>>>>>> >>>>>>>> I looked at the exploded image under >>>>>>>> BUILD_OUTPUTDIR/jdk/modules/java.xml >>>>>>>> and found that the comments of Encodings.properties are stripped. >>>>>>>> I was >>>>>>>> confused with the mention of resources.jar that I assume it was a >>>>>>>> step >>>>>>>> stripping the comments before writing to resources.jar. This is >>>>>>>> still >>>>>>>> an issue in jigsaw m2 I believe. >>>>>>>> >>>>>>>> Where does the build strip the comments? >>>>>>> >>>>>>> A previous issue was that the build process strips off anything >>>>>>> after '#' >>>>>>> when copying properties files. In JDK8: >>>>>>> jaxp/make/BuildJaxp.gmk: >>>>>>> $(JAXP_OUTPUTDIR)/classes/%.properties: >>>>>>> $(JAXP_TOPDIR)/src/%.properties >>>>>>> $(MKDIR) -p $(@D) >>>>>>> $(RM) $@ $@.tmp >>>>>>> $(CAT) $< | LANG=C $(NAWK) '{ sub(/#.*$$/,"#"); print }' >>>>>>> > $@.tmp >>>>>>> $(MV) $@.tmp $@ >>>>>>> >>>>>>> This was fixed in JDK 9. The per-repo process was removed. It >>>>>>> looks like >>>>>>> the properties processing process is now consolidated into >>>>>>> make/common/JavaCompilation.gmk. So the issue Daniel found is new >>>>>>> (in terms >>>>>>> of stripping). Search 'properties files' to see the macro. >>>>>>> >>>>>>> Joe >>>>>>> >>>>>>>> Mandy >>>>>>>> >>>> >>> >> > From wuwen.55 at gmail.com Tue Nov 11 06:35:01 2014 From: wuwen.55 at gmail.com (wuwen.55) Date: Tue, 11 Nov 2014 14:35:01 +0800 Subject: Java CountDownLatch.await(timeout) is influenced by changes to System time on Linux In-Reply-To: <546173A8.9070002@oracle.com> References: <546173A8.9070002@oracle.com> Message-ID: <2AB028B8-1555-4023-8E44-3D7D97837E9D@gmail.com> Thank you? I'm version is 7u55. and upgrade 7u60 test is ok. jdk 1.7.0_55 test result time:Tue Nov 11 13:24:01 CST 2014 nanoTime:18093680000 currentTime:18093 ------------------------- time:Tue Nov 11 13:24:02 CST 2014 nanoTime:19097878000 currentTime:19098 ------------------------- time:Tue Nov 11 12:24:03 CST 2014 nanoTime:-3580426662000 (negative difference) currentTime:-3580427 ------------------------- time:Tue Nov 11 12:24:04 CST 2014 nanoTime:-3579424553000 currentTime:-3579425 jdk 1.7.0_60 test result time:Tue Nov 11 12:28:51 CST 2014 nanoTime:8034263027 currentTime:8034 ------------------------- time:Tue Nov 11 12:28:52 CST 2014 nanoTime:9034780087 currentTime:9035 ------------------------- time:Tue Nov 11 11:28:50 CST 2014 nanoTime:10035322113 (is ok) currentTime:-3593155 ------------------------- time:Tue Nov 11 11:28:51 CST 2014 nanoTime:11035884267 currentTime:-3592154 > ? 2014?11?11????10:25?David Holmes ??? > > Repeating my response due to the subject mangling in the other "thread": > > This is a known issue: > > https://bugs.openjdk.java.net/browse/JDK-6900441 > > fixed in 7u60, 8 and 9. > > What version were you running? > > David > > On 7/11/2014 6:22 PM, wuwen.55 wrote: >> call .await(10, TimeUnit.SECONDS) and change the time one hours back. >> >> i.e. start at 13:00 call .await and at 13:05 change the time to 12:05 >> >> you will notice that the wait does not return. >> >> when change the time to the current time . change the time to 13:05. >> >> the wait still does not return. >> >> test code: >> CountDownLatch c = new CountDownLatch(1); >> ... >> c.await(10, TimeUnit.SECONDS); >> ... >> ---------------------------------- >> >> AbstractQueuedSynchronizer.doAcquireNanos >> >> >> long lastTime = System.nanoTime(); >> final Node node = addWaiter(Node.EXCLUSIVE); >> boolean failed = true; >> try { >> for (;;) { >> final Node p = node.predecessor(); >> if (p == head && tryAcquire(arg)) { >> setHead(node); >> p.next = null; // help GC >> failed = false; >> return true; >> } >> if (nanosTimeout <= 0) >> return false; >> if (shouldParkAfterFailedAcquire(p, node) && >> nanosTimeout > spinForTimeoutThreshold) >> LockSupport.parkNanos(this, nanosTimeout); >> long now = System.nanoTime(); >> nanosTimeout -= now - lastTime; >> lastTime = now; >> if (Thread.interrupted()) >> throw new InterruptedException(); >> } >> } finally { >> if (failed) >> cancelAcquire(node); >> } >> >> when change the time one hours back >> >> now - lastTime is negative number. >> >> ????????? >> >> From duncan.macgregor at ge.com Wed Nov 19 10:23:29 2014 From: duncan.macgregor at ge.com (MacGregor, Duncan (GE Energy Management)) Date: Wed, 19 Nov 2014 10:23:29 +0000 Subject: [9, 8u40] RFR (XXS): 8059880: Get rid of LambdaForm interpretation In-Reply-To: <546BD761.6030108@oracle.com> References: <546BB3C2.60202@oracle.com> <546BD761.6030108@oracle.com> Message-ID: On 18/11/2014 23:33, "Aleksey Shipilev" wrote: >On 11/19/2014 12:01 AM, Vladimir Ivanov wrote: >> http://cr.openjdk.java.net/~vlivanov/8059880/webrev.00/ >> https://bugs.openjdk.java.net/browse/JDK-8059880 > >Yes, for the love of God, GO FOR IT. Seconded. Startup of our stuff seems fine now with a compile threshold of zero, and it will make stacks so much easier to read in the debugger. :-) From stuart.marks at oracle.com Thu Nov 20 23:51:40 2014 From: stuart.marks at oracle.com (Stuart Marks) Date: Thu, 20 Nov 2014 15:51:40 -0800 Subject: RFR(s): 8056313: TEST_BUG: java/util/Timer/NameConstructors.java fails intermittently Message-ID: <546E7E8C.7080006@oracle.com> Hi all, Here's a small test cleanup fix for review. This test fails rarely but often enough to be annoying. The cause is one or more of the following: a memory visibility issue (unsafe read/write to non-volatile fields), a race condition (fixed Thread.sleep(500) wait time), and potential failure to cancel a timer thread if an exception is thrown. This change fixes all of these probems. Webrev: http://cr.openjdk.java.net/~smarks/reviews/8056313/webrev.0/ Bug: https://bugs.openjdk.java.net/browse/JDK-8056313 Thanks, s'marks From lance.andersen at oracle.com Fri Nov 21 00:36:32 2014 From: lance.andersen at oracle.com (Lance @ Oracle) Date: Thu, 20 Nov 2014 19:36:32 -0500 Subject: RFR(s): 8056313: TEST_BUG: java/util/Timer/NameConstructors.java fails intermittently In-Reply-To: <546E7E8C.7080006@oracle.com> References: <546E7E8C.7080006@oracle.com> Message-ID: <2C8CE44D-37AD-4BA6-A553-6D90951F71F3@oracle.com> Look fine Stuart Best Lance Lance Andersen| Principal Member of Technical Staff | +1.781.442.2037 Oracle Java Engineering 1 Network Drive Burlington, MA 01803 Lance.Andersen at oracle.com Sent from my iPad > On Nov 20, 2014, at 6:51 PM, Stuart Marks wrote: > > Hi all, > > Here's a small test cleanup fix for review. This test fails rarely but often enough to be annoying. The cause is one or more of the following: a memory visibility issue (unsafe read/write to non-volatile fields), a race condition (fixed Thread.sleep(500) wait time), and potential failure to cancel a timer thread if an exception is thrown. This change fixes all of these probems. > > Webrev: > > http://cr.openjdk.java.net/~smarks/reviews/8056313/webrev.0/ > > Bug: > > https://bugs.openjdk.java.net/browse/JDK-8056313 > > Thanks, > > s'marks > From roger.riggs at oracle.com Fri Nov 21 01:28:43 2014 From: roger.riggs at oracle.com (roger riggs) Date: Thu, 20 Nov 2014 20:28:43 -0500 Subject: RFR(s): 8056313: TEST_BUG: java/util/Timer/NameConstructors.java fails intermittently In-Reply-To: <546E7E8C.7080006@oracle.com> References: <546E7E8C.7080006@oracle.com> Message-ID: <546E954B.4080601@oracle.com> Hi Stuart, Looks fine. Roger On 11/20/2014 6:51 PM, Stuart Marks wrote: > Hi all, > > Here's a small test cleanup fix for review. This test fails rarely but > often enough to be annoying. The cause is one or more of the > following: a memory visibility issue (unsafe read/write to > non-volatile fields), a race condition (fixed Thread.sleep(500) wait > time), and potential failure to cancel a timer thread if an exception > is thrown. This change fixes all of these probems. > > Webrev: > > http://cr.openjdk.java.net/~smarks/reviews/8056313/webrev.0/ > > Bug: > > https://bugs.openjdk.java.net/browse/JDK-8056313 > > Thanks, > > s'marks > From stuart.marks at oracle.com Fri Nov 21 03:05:17 2014 From: stuart.marks at oracle.com (Stuart Marks) Date: Thu, 20 Nov 2014 19:05:17 -0800 Subject: Library enhancement proposal for copying data from/to Reader/Writer InputStream/OutputStream In-Reply-To: References: <566A428C-1A13-4DEE-940B-2EBE375F2A7A@reini.net> <51329A99-55E5-4A42-9180-D872366B4DEE@oracle.com> Message-ID: <546EABED.9030301@oracle.com> Hi Patrick, Good to meet you in Antwerp! On 11/20/14 10:30 AM, Patrick Reinhart wrote: >> Am 20.11.2014 um 10:22 schrieb Pavel Rappo : >> >> There is at least one method in the JDK with similar characteristics: java.nio.file.Files#copy(java.io.InputStream, java.io.OutputStream) >> But, (1) it has a private access (2) even if we made it public I doubt java.nio.file.Files would be a good place for it >> > > I would suggest to introduce a separate IOUtil class holding such utility methods. Additionally to copyTo() and copyFrom() method could be added later for more intuitive usage. Also the copy method within the Files method could be replaced with a static reference to the IOUtil class. Thanks to Pavel for pointing out the existing copy operations in the nio Files class. I think there's a case for the InputStream => OutputStream copy method to be placed there too, although I admit it is somewhat unusual in that it doesn't actually have anything to do with files. At my first encounter with the nio.Files class some years ago I saw the following copying methods: copy(istream, targetfile) copy(sourcefile, ostream) copy(sourcefile, targetfile) and I immediately thought, "Where is copy(istream, ostream)?" So to me at least, it makes more sense to be in the Files class than in some newly created IOUtils class. (I'd like to hear further from Alan on this.) As Pavel pointed out, the logic is also already there in the Files class. Probably the way to proceed would be to rename the existing (private) method to be something like copyInternal() and then create a new, public copy(istream, ostream) method that does argument checking before calling copyInternal(). >> P.S. The thing that confuses me though, is the progress consumer. I believe this feature is rarely needed (at least in a way you described it). >> If you want to do something like this, you should probably go asynchronous with full blown solution like what they have in javax.swing.SwingWorker. >> >> -Pavel > > The method having a IntConsumer I have already discussed with some other colleagues and they suggested to better use a IntPredicate in order to have the possibility to interrupt the copy process, without having to interrupt any threads. Additionally there is still the possibility to use such a Predicate for counting the total or using the information for other possibilities. I'd suggest starting simple with a copy(istream, ostream) operation and considering some kind of interruptible, progress-reporting operation separately. It would seem quite limiting if the *only* progress-reporting operation in the system were the stream copier. We'd want a way to apply such a mechanism to other long-running operations. I think the progress update reports also need to be decoupled from the actual I/O operations. For example, the current buffer size in nio.Files is 8192 bytes. If the streams are connected to a fast network connection, this will result in thousands of calls to the predicate per second. On the other hand, if the buffer size were raised, and the streams are connected to a slow network connection -- like my home internet connection :-) -- that might result in too few callbacks per second. How to report progress from a running operation is an interesting problem and it's worthy of considering, but a copying utility doesn't seem like quite the right place for it. s'marks From ecki at zusammenkunft.net Fri Nov 21 06:05:35 2014 From: ecki at zusammenkunft.net (Bernd Eckenfels) Date: Fri, 21 Nov 2014 07:05:35 +0100 Subject: Java CountDownLatch.await(timeout) is influenced by changes to System time on Linux In-Reply-To: <2AB028B8-1555-4023-8E44-3D7D97837E9D@gmail.com> References: <546173A8.9070002@oracle.com> <2AB028B8-1555-4023-8E44-3D7D97837E9D@gmail.com> Message-ID: <20141121070535.00006057.ecki@zusammenkunft.net> Am Tue, 11 Nov 2014 14:35:01 +0800 schrieb "wuwen.55" : > Thank you? > I'm version is 7u55. and upgrade 7u60 test is ok. Cools results! Can you give a few details, what OS is this (Linux?) and how did you set the clock back (date?)? Did you also try to change the (system) timezone (I think this is not possible on Linux to affect running java?) or wait for a DST change (i.e. set date short before DST/ST or vice versa?) You did not test with ntp/adjusttime, right? Gruss Bernd From forax at univ-mlv.fr Fri Nov 21 07:09:39 2014 From: forax at univ-mlv.fr (Remi Forax) Date: Fri, 21 Nov 2014 08:09:39 +0100 Subject: Library enhancement proposal for copying data from/to Reader/Writer InputStream/OutputStream In-Reply-To: <546EABED.9030301@oracle.com> References: <566A428C-1A13-4DEE-940B-2EBE375F2A7A@reini.net> <51329A99-55E5-4A42-9180-D872366B4DEE@oracle.com> <546EABED.9030301@oracle.com> Message-ID: <546EE533.6070509@univ-mlv.fr> On 11/21/2014 04:05 AM, Stuart Marks wrote: > Hi Patrick, > > Good to meet you in Antwerp! > > On 11/20/14 10:30 AM, Patrick Reinhart wrote: >>> Am 20.11.2014 um 10:22 schrieb Pavel Rappo : >>> >>> There is at least one method in the JDK with similar >>> characteristics: java.nio.file.Files#copy(java.io.InputStream, >>> java.io.OutputStream) >>> But, (1) it has a private access (2) even if we made it public I >>> doubt java.nio.file.Files would be a good place for it >>> >> >> I would suggest to introduce a separate IOUtil class holding such >> utility methods. Additionally to copyTo() and copyFrom() method could >> be added later for more intuitive usage. Also the copy method within >> the Files method could be replaced with a static reference to the >> IOUtil class. > > Thanks to Pavel for pointing out the existing copy operations in the > nio Files class. I think there's a case for the InputStream => > OutputStream copy method to be placed there too, although I admit it > is somewhat unusual in that it doesn't actually have anything to do > with files. > > At my first encounter with the nio.Files class some years ago I saw > the following copying methods: > > copy(istream, targetfile) > copy(sourcefile, ostream) > copy(sourcefile, targetfile) > > and I immediately thought, "Where is copy(istream, ostream)?" So to me > at least, it makes more sense to be in the Files class than in some > newly created IOUtils class. (I'd like to hear further from Alan on > this.) > > As Pavel pointed out, the logic is also already there in the Files > class. Probably the way to proceed would be to rename the existing > (private) method to be something like copyInternal() and then create a > new, public copy(istream, ostream) method that does argument checking > before calling copyInternal(). > >>> P.S. The thing that confuses me though, is the progress consumer. I >>> believe this feature is rarely needed (at least in a way you >>> described it). >>> If you want to do something like this, you should probably go >>> asynchronous with full blown solution like what they have in >>> javax.swing.SwingWorker. >>> >>> -Pavel >> >> The method having a IntConsumer I have already discussed with some >> other colleagues and they suggested to better use a IntPredicate in >> order to have the possibility to interrupt the copy process, without >> having to interrupt any threads. Additionally there is still the >> possibility to use such a Predicate for counting the total or using >> the information for other possibilities. > > I'd suggest starting simple with a copy(istream, ostream) operation > and considering some kind of interruptible, progress-reporting > operation separately. It would seem quite limiting if the *only* > progress-reporting operation in the system were the stream copier. > We'd want a way to apply such a mechanism to other long-running > operations. > > I think the progress update reports also need to be decoupled from the > actual I/O operations. For example, the current buffer size in > nio.Files is 8192 bytes. If the streams are connected to a fast > network connection, this will result in thousands of calls to the > predicate per second. On the other hand, if the buffer size were > raised, and the streams are connected to a slow network connection -- > like my home internet connection :-) -- that might result in too few > callbacks per second. > > How to report progress from a running operation is an interesting > problem and it's worthy of considering, but a copying utility doesn't > seem like quite the right place for it. I also think you don't need a specific progress API given that you can write an InputStream that act as a proxy on top of the real inputStream and delegate the number of read bytes to a specific lambda. > > s'marks > cheers, R?mi From patrick at reini.net Fri Nov 21 07:29:13 2014 From: patrick at reini.net (Patrick Reinhart) Date: Fri, 21 Nov 2014 08:29:13 +0100 Subject: Library enhancement proposal for copying data from/to Reader/Writer InputStream/OutputStream In-Reply-To: <546EABED.9030301@oracle.com> References: <566A428C-1A13-4DEE-940B-2EBE375F2A7A@reini.net> <51329A99-55E5-4A42-9180-D872366B4DEE@oracle.com> <546EABED.9030301@oracle.com> Message-ID: <4AC3B648-EDD5-4016-9090-CDE236BF8BD6@reini.net> Hi Stuart, > Am 21.11.2014 um 04:05 schrieb Stuart Marks : > > Hi Patrick, > > Good to meet you in Antwerp! The pleasure was all mine :-) > On 11/20/14 10:30 AM, Patrick Reinhart wrote: >>> Am 20.11.2014 um 10:22 schrieb Pavel Rappo : >>> >>> There is at least one method in the JDK with similar characteristics: java.nio.file.Files#copy(java.io.InputStream, java.io.OutputStream) >>> But, (1) it has a private access (2) even if we made it public I doubt java.nio.file.Files would be a good place for it >>> >> >> I would suggest to introduce a separate IOUtil class holding such utility methods. Additionally to copyTo() and copyFrom() method could be added later for more intuitive usage. Also the copy method within the Files method could be replaced with a static reference to the IOUtil class. > > Thanks to Pavel for pointing out the existing copy operations in the nio Files class. I think there's a case for the InputStream => OutputStream copy method to be placed there too, although I admit it is somewhat unusual in that it doesn't actually have anything to do with files. > > At my first encounter with the nio.Files class some years ago I saw the following copying methods: > > copy(istream, targetfile) > copy(sourcefile, ostream) > copy(sourcefile, targetfile) > > and I immediately thought, "Where is copy(istream, ostream)?" So to me at least, it makes more sense to be in the Files class than in some newly created IOUtils class. (I?d like to hear further from Alan on this.) > The reason to not put those copy operations in the Files class in the first place, is the fact that this operation is not specific to files. If I would like to copy some database blob stream to the servlet output stream for example, there is no relation to a file at all. > As Pavel pointed out, the logic is also already there in the Files class. Probably the way to proceed would be to rename the existing (private) method to be something like copyInternal() and then create a new, public copy(istream, ostream) method that does argument checking before calling copyInternal(). > My first idea was to even in fact, to some sort of copy function on the InputStream like ?copyFrom(OutputStream)? and on the OutputStream something like ?copyTo(InputStream)?. Those methods would use the "copyInternal? method on Files if the one would be made default access. The same would the also apply to Reader/Writer of course, for which there would be the need to have a similar copy method. >>> P.S. The thing that confuses me though, is the progress consumer. I believe this feature is rarely needed (at least in a way you described it). >>> If you want to do something like this, you should probably go asynchronous with full blown solution like what they have in javax.swing.SwingWorker. >>> >>> -Pavel >> >> The method having a IntConsumer I have already discussed with some other colleagues and they suggested to better use a IntPredicate in order to have the possibility to interrupt the copy process, without having to interrupt any threads. Additionally there is still the possibility to use such a Predicate for counting the total or using the information for other possibilities. > > I'd suggest starting simple with a copy(istream, ostream) operation and considering some kind of interruptible, progress-reporting operation separately. It would seem quite limiting if the *only* progress-reporting operation in the system were the stream copier. We?d want a way to apply such a mechanism to other long-running operations. Agreed. I could implement some Input-/OutputFilterStream that could implement such a behavior, without having to do that on the API. > I think the progress update reports also need to be decoupled from the actual I/O operations. For example, the current buffer size in nio.Files is 8192 bytes. If the streams are connected to a fast network connection, this will result in thousands of calls to the predicate per second. On the other hand, if the buffer size were raised, and the streams are connected to a slow network connection -- like my home internet connection :-) -- that might result in too few callbacks per second. I see your point for decoupling there. So it would be more practical to have the actual buffer size being optionally specified? - by the way: I also don?t have a real fast internet connection ;-) > How to report progress from a running operation is an interesting problem and it?s worthy of considering, but a copying utility doesn't seem like quite the right place for it. I see your point there. Patrick From Alan.Bateman at oracle.com Fri Nov 21 08:36:28 2014 From: Alan.Bateman at oracle.com (Alan Bateman) Date: Fri, 21 Nov 2014 08:36:28 +0000 Subject: Library enhancement proposal for copying data from/to Reader/Writer InputStream/OutputStream In-Reply-To: <546EABED.9030301@oracle.com> References: <566A428C-1A13-4DEE-940B-2EBE375F2A7A@reini.net> <51329A99-55E5-4A42-9180-D872366B4DEE@oracle.com> <546EABED.9030301@oracle.com> Message-ID: <546EF98C.40704@oracle.com> On 21/11/2014 03:05, Stuart Marks wrote: > > Thanks to Pavel for pointing out the existing copy operations in the > nio Files class. I think there's a case for the InputStream => > OutputStream copy method to be placed there too, although I admit it > is somewhat unusual in that it doesn't actually have anything to do > with files. > > At my first encounter with the nio.Files class some years ago I saw > the following copying methods: > > copy(istream, targetfile) > copy(sourcefile, ostream) > copy(sourcefile, targetfile) > > and I immediately thought, "Where is copy(istream, ostream)?" So to me > at least, it makes more sense to be in the Files class than in some > newly created IOUtils class. (I'd like to hear further from Alan on > this.) > > As Pavel pointed out, the logic is also already there in the Files > class. Probably the way to proceed would be to rename the existing > (private) method to be something like copyInternal() and then create a > new, public copy(istream, ostream) method that does argument checking > before calling copyInternal(). The Files class works on files, it's not the right place for a general purpose copy(InputStream, OutputStream). When we prototyped a copy(InputStream, OutputStream) and many other I/O methods a few years ago then the intention was to put it java.io. One option on the table was a Collections-like utility class with static methods. Another option was to add methods to InputStream, Reader, etc. A concern with the latter was whether the new methods would cause problems for existing InputStream/etc. implementations, similar to concerns about adding default methods to the collection types in 8. I think that discussion needs to happen again and having a few prototypes to get experience with the various approaches would be good too. -Alan From otaviopolianasantana at gmail.com Fri Nov 21 13:35:13 2014 From: otaviopolianasantana at gmail.com (=?UTF-8?Q?Ot=C3=A1vio_Gon=C3=A7alves_de_Santana?=) Date: Fri, 21 Nov 2014 11:35:13 -0200 Subject: New JEP: Adding JSR 354 Money & Currency API Message-ID: Hi everyone, Could any commiter help us in this JSR to JEP? If there is any question, please tell me. --------- Forwarded message ---------- From: Date: 2014-11-05 20:16 GMT+01:00 Subject: Re: New JEP: Adding JSR 354 Money & Currency API To: Anatole Tresch Cc: jep-submit at openjdk.java.net 2014/10/30 8:11 -0700, atsticks at gmail.com: > Dear all > find attached out JEP for adding JSR 354 Money & Currency to the Java > platform version 9. > In case of any questions, just drop me a mail ;) Thanks for your submission, but we no longer accept JEP submissions via e-mail. We're transitioning to the JEP 2.0 Process [1], in which JEPs are created and maintained as a custom issue type in the JDK Bug System (JBS) [2]. The JEP Process is open to OpenJDK Contributors; to become one you'll need to submit an Oracle Contributor Agreement [3]. Since you're not already an OpenJDK Committer you'll need to find someone who is and is willing to draft and submit this JEP on your behalf. - Mark [1] http://cr.openjdk.java.net/~mr/jep/jep-2.0-02.html [2] https://bugs.openjdk.java.net/ [3] http://openjdk.java.net/legal/ -- Ot?vio Gon?alves de Santana twitter: http://twitter.com/otaviojava site: *http://about.me/otaviojava * 55 (11) 98255-3513 From patrick at reini.net Fri Nov 21 14:12:15 2014 From: patrick at reini.net (Patrick Reinhart) Date: Fri, 21 Nov 2014 15:12:15 +0100 Subject: Library enhancement proposal for copying data from/to Reader/Writer InputStream/OutputStream In-Reply-To: <546EF98C.40704@oracle.com> References: <566A428C-1A13-4DEE-940B-2EBE375F2A7A@reini.net> <51329A99-55E5-4A42-9180-D872366B4DEE@oracle.com> <546EABED.9030301@oracle.com> <546EF98C.40704@oracle.com> Message-ID: > Am 21.11.2014 um 09:36 schrieb Alan Bateman : > > On 21/11/2014 03:05, Stuart Marks wrote: >> >> Thanks to Pavel for pointing out the existing copy operations in the nio Files class. I think there's a case for the InputStream => OutputStream copy method to be placed there too, although I admit it is somewhat unusual in that it doesn't actually have anything to do with files. >> >> At my first encounter with the nio.Files class some years ago I saw the following copying methods: >> >> copy(istream, targetfile) >> copy(sourcefile, ostream) >> copy(sourcefile, targetfile) >> >> and I immediately thought, "Where is copy(istream, ostream)?" So to me at least, it makes more sense to be in the Files class than in some newly created IOUtils class. (I'd like to hear further from Alan on this.) >> >> As Pavel pointed out, the logic is also already there in the Files class. Probably the way to proceed would be to rename the existing (private) method to be something like copyInternal() and then create a new, public copy(istream, ostream) method that does argument checking before calling copyInternal(). > > The Files class works on files, it's not the right place for a general purpose copy(InputStream, OutputStream). That was the reason in my proposal to not put those methods on the Files class. I also would not try to find such methods there. Personally I tried to look for such method on ?Streams? ;-) > When we prototyped a copy(InputStream, OutputStream) and many other I/O methods a few years ago then the intention was to put it java.io. One option on the table was a Collections-like utility class with static methods. Another option was to add methods to InputStream, Reader, etc. A concern with the latter was whether the new methods would cause problems for existing InputStream/etc. implementations, similar to concerns about adding default methods to the collection types in 8. I think that discussion needs to happen again and having a few prototypes to get experience with the various approaches would be good too. > > -Alan It would be good to have such methods in the next release and I would love to help here. Patrick From staffan.friberg at oracle.com Fri Nov 21 14:50:16 2014 From: staffan.friberg at oracle.com (Staffan Friberg) Date: Fri, 21 Nov 2014 15:50:16 +0100 Subject: RFR JDK-6321472: Add CRC-32C API In-Reply-To: <83BE5F55-075E-4702-BC7D-319077148428@oracle.com> References: <544073EA.3060304@oracle.com> <5440F803.2000406@oracle.com> <544166B8.8090103@oracle.com> <5441A192.8090602@gmail.com> <544697D2.6080301@oracle.com> <5446AAB0.3090304@oracle.com> <5446B551.7050605@oracle.com> <5446BDE1.9090709@oracle.com> <5446CB91.3050503@oracle.com> <5446DB24.4000507@oracle.com> <545B4AE7.4000208@oracle.com> <545B8DD6.5030805@oracle.com> <546DE258.6030600@oracle.com> <83BE5F55-075E-4702-BC7D-319077148428@oracle.com> Message-ID: <546F5128.4060302@oracle.com> Hi David, Thanks for reviewing. > CRC32C.java: > 56 /** > 57 * CRC-32C Polynom > 58 */ > > ?Polynomial?, perhaps? Done > You did test this on both big- and little-endian boxes, right? > It looks right to me, but this stuff is brain-hurty on mirror-image dyslexia and fenceposts. Yes, as part of the push I added a tests for both CRC32 and CRC32C and they pass on all platforms (and were really helpful when implementing). > It might not hurt to mention that byteTable is for byte-at-a-time calculations and is always > the same, whereas the other 8 tables are for bulk operations and will have contents > dependent on the endianness of the platform. Did a slight reorder to separate the single table and the bulk tables and added comments according to your suggestion. > It would be a good idea to mention that the high-order terms of polynomials > are stored in the low-order bits of the integers in the tables. > > You could get the bit reversal out of the init code if you bit-reversed CRC32C_POLY instead and wrote it > > 78 int r = index; > // High-order polynomial term stored in LSB of r. > 79 for (int i = 0; i < Byte.SIZE; i++) { > 80 if ((r & 1) != 0) { > 81 r = (r >>> 1) ^ REVERSED_CRC32C_POLY; > 82 } else { > 83 r >>>= 1; > 84 } > 85 } > 86 byteTables[0][index] = r; > > at least, it seems that way to me. Agree, and so does the tests... :) Here is the last webrev with the above changes, http://cr.openjdk.java.net/~sfriberg/JDK-6321472/webrev.09 I will ask Sherman to sponsor and push the change, since Joe mentioned only a single Reviewer is currently OK in JDK 9. Thanks, Staffan On 11/20/2014 08:41 PM, David Chase wrote: > On 2014-11-20, at 7:45 AM, Staffan Friberg wrote: > >> Hi, >> >> Anyone who can be the second Reviewer? >> >> Thanks, >> Staffan > I can review, but I am not a Reviewer. > > CRC32C.java: > 56 /** > 57 * CRC-32C Polynom > 58 */ > > ?Polynomial?, perhaps? > > You did test this on both big- and little-endian boxes, right? > It looks right to me, but this stuff is brain-hurty on mirror-image dyslexia and fenceposts. > > It might not hurt to mention that byteTable is for byte-at-a-time calculations and is always > the same, whereas the other 8 tables are for bulk operations and will have contents > dependent on the endianness of the platform. > > It would be a good idea to mention that the high-order terms of polynomials > are stored in the low-order bits of the integers in the tables. > > You could get the bit reversal out of the init code if you bit-reversed CRC32C_POLY instead and wrote it > > 78 int r = index; > // High-order polynomial term stored in LSB of r. > 79 for (int i = 0; i < Byte.SIZE; i++) { > 80 if ((r & 1) != 0) { > 81 r = (r >>> 1) ^ REVERSED_CRC32C_POLY; > 82 } else { > 83 r >>>= 1; > 84 } > 85 } > 86 byteTables[0][index] = r; > > at least, it seems that way to me. > > David > From ecki at zusammenkunft.net Fri Nov 21 18:56:34 2014 From: ecki at zusammenkunft.net (Bernd Eckenfels) Date: Fri, 21 Nov 2014 19:56:34 +0100 Subject: RFR JDK-6321472: Add CRC-32C API In-Reply-To: <546F5128.4060302@oracle.com> References: <544073EA.3060304@oracle.com> <5440F803.2000406@oracle.com> <544166B8.8090103@oracle.com> <5441A192.8090602@gmail.com> <544697D2.6080301@oracle.com> <5446AAB0.3090304@oracle.com> <5446B551.7050605@oracle.com> <5446BDE1.9090709@oracle.com> <5446CB91.3050503@oracle.com> <5446DB24.4000507@oracle.com> <545B4AE7.4000208@oracle.com> <545B8DD6.5030805@oracle.com> <546DE258.6030600@oracle.com> <83BE5F55-075E-4702-BC7D-319077148428@oracle.com> <546F5128.4060302@oracle.com> Message-ID: <20141121195634.000065d9.ecki@zusammenkunft.net> Hello Staffan, short question, the patch currently only adds the algorithm to be used, is it planned/possible to add it as a supported algorithm to any of the compressors or packers or is there no specification for that? Besides that, the bound buffer now looks good. Gruss Bernd Am Fri, 21 Nov 2014 15:50:16 +0100 schrieb Staffan Friberg : > Here is the last webrev with the above changes, > http://cr.openjdk.java.net/~sfriberg/JDK-6321472/webrev.09 From martinrb at google.com Fri Nov 21 21:40:07 2014 From: martinrb at google.com (Martin Buchholz) Date: Fri, 21 Nov 2014 13:40:07 -0800 Subject: RFR: 8065138 - Encodings.isRecognizedEnconding sometimes fails to recognize 'UTF8' In-Reply-To: <546E660C.1060909@oracle.com> References: <546C7C2D.3060805@oracle.com> <546CDE30.6030907@oracle.com> <546CFF3C.2030904@oracle.com> <546D0285.3090504@oracle.com> <546D2C7C.5020103@oracle.com> <546D3126.5010301@oracle.com> <546D3ECE.2040500@oracle.com> <546DB3BB.7020901@oracle.com> <546DB5AC.60401@oracle.com> <546DEE70.6020306@oracle.com> <546E23EF.4000506@oracle.com> <546E660C.1060909@oracle.com> Message-ID: A high-level followup ... Running most text-based OS tools, including sed and sort, is risky because the user's encoding may be different from the encoding of source files in the JDK (of course, this is particularly problematic with properties files, which must be ISO-8859-1). These tools will probably work better when run with LC_ALL=C. We should consider changing the definition of SED and SORT to something like $(ENV) LC_ALL=C sed As for set -o pipefail, it is sad that every single pipeline in the makefiles is vulnerable to failure to detect broken builds. The workaround in http://stackoverflow.com/questions/23079651/equivalent-of-pipefail-in-gnu-make is unaesthetic. It's enough to make you rewrite all your build tools in java! On Thu, Nov 20, 2014 at 2:07 PM, Mandy Chung wrote: > Daniel, > > Your test change looks fine and good to push this change that should catch > if similar regression occurs in the future. I was tempted to suggest to > have a dedicated test for the build-time modification to the properties > file. Maybe a low priority RFE. > > Mandy > > > On 11/20/14 9:25 AM, Daniel Fuchs wrote: >> >> On 20/11/14 14:36, Erik Joelsson wrote: >>> >>> Here is my proposal for fixing the particular issue of generating the >>> correct properties files. I'm simply adding LC_ALL=C to the whole >>> command line instead of just the sort at the end. It seems to require >>> using "export" to get picked up. >> >> >> Hi Erik, >> >> Looks good to me. >> >> I have applied your patch (manually, because the copy/paste >> seems to have eaten the tab characters away, which caused patch to >> reject the diff) - and I confirm that the issue has disappeared. >> >> Thanks for solving this! >> >> Do you think it would be worth to commit this test modification >> later on, in a followup Bug/RFE? >> >> >> http://cr.openjdk.java.net/~dfuchs/webrev_8065138/webrev.00/jdk/test/javax/xml/jaxp/Encodings/CheckEncodingPropertiesFile.java.frames.html >> >> If so I will take care of it. >> >> best regards, >> >> -- daniel >> >> >>> Bug: https://bugs.openjdk.java.net/browse/JDK-8065138 >>> Patch: >>> diff --git a/make/common/JavaCompilation.gmk >>> b/make/common/JavaCompilation.gmk >>> --- a/make/common/JavaCompilation.gmk >>> +++ b/make/common/JavaCompilation.gmk >>> @@ -400,13 +400,15 @@ >>> # Now we can setup the depency that will trigger the copying. >>> $$($1_BIN)$$($2_TARGET) : $2 >>> $(MKDIR) -p $$(@D) >>> - $(CAT) $$< | $(SED) -e 's/\([^\\]\):/\1\\:/g' -e >>> 's/\([^\\]\)=/\1\\=/g' \ >>> + export LC_ALL=C ; $(CAT) $$< \ >>> + | $(SED) -e 's/\([^\\]\):/\1\\:/g' -e 's/\([^\\]\)=/\1\\=/g' \ >>> -e 's/\([^\\]\)!/\1\\!/g' -e 's/#.*/#/g' \ >>> | $(SED) -f "$(SRC_ROOT)/make/common/support/unicode2x.sed" \ >>> | $(SED) -e '/^#/d' -e '/^$$$$/d' \ >>> -e :a -e '/\\$$$$/N; s/\\\n//; ta' \ >>> -e 's/^[ ]*//;s/[ ]*$$$$//' \ >>> - -e 's/\\=/=/' | LC_ALL=C $(SORT) > $$@ >>> + -e 's/\\=/=/' \ >>> + | $(SORT) > $$@ >>> $(CHMOD) -f ug+w $$@ >>> >>> # And do not forget this target >>> >>> >>> I filed a separate issue [1] for investigating the use of pipefail. >>> >>> /Erik >>> >>> [1] https://bugs.openjdk.java.net/browse/JDK-8065576 >>> >>> On 2014-11-20 10:34, Daniel Fuchs wrote: >>>> >>>> On 11/20/14 10:26 AM, Erik Joelsson wrote: >>>>> >>>>> Hello, >>>>> >>>>> On 2014-11-20 02:20, Martin Buchholz wrote: >>>>>> >>>>>> Amusingly, the $(SORT) has an LC_ALL=C carefully placed before it, but >>>>>> the $(SED)s need it too! >>>>> >>>>> Yes, I think that's the correct fix in this case. >>>>>> >>>>>> On Wed, Nov 19, 2014 at 5:18 PM, Martin Buchholz >>>>>> wrote: >>>>>>> >>>>>>> [+ build-dev] >>>>>>> >>>>>>> I think I see the problem. By default, a Unix pipeline sadly fails >>>>>>> only when the last command fails. In bash, you can change this to a >>>>>>> more sensible default via >>>>>>> set -o pipefail >>>>>>> but that's not portable enough for openjdk. >>>>> >>>>> This is interesting and something I had missed. I will experiment >>>>> with enabling pipefail if configure finds support for it. This will >>>>> include setting the SHELL to bash. We really should fail instead of >>>>> silently generating broken builds. >>>>> >>>>> Daniel, I can take over this bug if you want and work on a proper >>>>> build fix. >>>> >>>> >>>> Thanks Erik! You are welcome! >>>> So the whole issue seems to be that my default setting is >>>> LC_ALL=en_US.UTF-8 >>>> whereas sed requires LC_ALL=C to work properly on these property >>>> files... >>>> >>>> When the test first failed I tried to rerun the test with LC_ALL=C - >>>> with no success >>>> of course. But I never thought of rebuilding with LC_ALL=C :-( >>>> >>>> My apologies for the red herring :-( >>>> >>>> best regards >>>> >>>> -- daniel >>>> >>>>> >>>>> /Erik >>>>>>> >>>>>>> $(CAT) $$< | $(SED) -e 's/\([^\\]\):/\1\\:/g' -e >>>>>>> 's/\([^\\]\)=/\1\\=/g' \ >>>>>>> -e 's/\([^\\]\)!/\1\\!/g' -e 's/#.*/#/g' \ >>>>>>> | $(SED) -f "$(SRC_ROOT)/make/common/support/unicode2x.sed" \ >>>>>>> | $(SED) -e '/^#/d' -e '/^$$$$/d' \ >>>>>>> -e :a -e '/\\$$$$/N; s/\\\n//; ta' \ >>>>>>> -e 's/^[ ]*//;s/[ ]*$$$$//' \ >>>>>>> -e 's/\\=/=/' | LC_ALL=C $(SORT) > $$@ >>>>>>> >>>>>>> On Wed, Nov 19, 2014 at 5:07 PM, huizhe wang >>>>>>> wrote: >>>>>>>> >>>>>>>> On 11/19/2014 4:09 PM, Mandy Chung wrote: >>>>>>>>> >>>>>>>>> >>>>>>>>> On 11/19/2014 3:49 PM, Mandy Chung wrote: >>>>>>>>>> >>>>>>>>>> >>>>>>>>>> On 11/19/2014 12:50 PM, Daniel Fuchs wrote: >>>>>>>>>>> >>>>>>>>>>> On 11/19/14 9:36 PM, Mandy Chung wrote: >>>>>>>>>>>> >>>>>>>>>>>> resources.jar will be gone when we move to the modular runtime >>>>>>>>>>>> image >>>>>>>>>>>> (JEP 220 [1]). >>>>>>>>>>>> JDK-8065138 and JDK-8065365 will become non-issue in JDK 9. >>>>>>>>>>> >>>>>>>>>>> Do you mean that the property files will no longer be stripped >>>>>>>>>>> of their >>>>>>>>>>> comments? >>>>>>>>>> >>>>>>>>>> >>>>>>>>>> (sorry for my delay in reply as I was trying to get the number >>>>>>>>>> of the >>>>>>>>>> resources in the modular image vs resources.jar but got >>>>>>>>>> distracted.) >>>>>>>>>> >>>>>>>>>> The current version copies all bytes when generating the modular >>>>>>>>>> image. >>>>>>>>>> This is a good question whether we should strip off the comments >>>>>>>>>> when >>>>>>>>>> writing to the modular runtime image. I think we should look >>>>>>>>>> at the >>>>>>>>>> footprint and any performance saving and determine if we should >>>>>>>>>> do the same >>>>>>>>>> in JDK 9. >>>>>>>>>> >>>>>>>>> I looked at the exploded image under >>>>>>>>> BUILD_OUTPUTDIR/jdk/modules/java.xml >>>>>>>>> and found that the comments of Encodings.properties are stripped. >>>>>>>>> I was >>>>>>>>> confused with the mention of resources.jar that I assume it was a >>>>>>>>> step >>>>>>>>> stripping the comments before writing to resources.jar. This is >>>>>>>>> still >>>>>>>>> an issue in jigsaw m2 I believe. >>>>>>>>> >>>>>>>>> Where does the build strip the comments? >>>>>>>> >>>>>>>> >>>>>>>> A previous issue was that the build process strips off anything >>>>>>>> after '#' >>>>>>>> when copying properties files. In JDK8: >>>>>>>> jaxp/make/BuildJaxp.gmk: >>>>>>>> $(JAXP_OUTPUTDIR)/classes/%.properties: >>>>>>>> $(JAXP_TOPDIR)/src/%.properties >>>>>>>> $(MKDIR) -p $(@D) >>>>>>>> $(RM) $@ $@.tmp >>>>>>>> $(CAT) $< | LANG=C $(NAWK) '{ sub(/#.*$$/,"#"); print }' >>>>>>>> > $@.tmp >>>>>>>> $(MV) $@.tmp $@ >>>>>>>> >>>>>>>> This was fixed in JDK 9. The per-repo process was removed. It >>>>>>>> looks like >>>>>>>> the properties processing process is now consolidated into >>>>>>>> make/common/JavaCompilation.gmk. So the issue Daniel found is new >>>>>>>> (in terms >>>>>>>> of stripping). Search 'properties files' to see the macro. >>>>>>>> >>>>>>>> Joe >>>>>>>> >>>>>>>>> Mandy >>>>>>>>> >>>>> >>>> >>> >> > From jandam at crcdata.cz Mon Nov 24 09:19:11 2014 From: jandam at crcdata.cz (Janda Martin) Date: Mon, 24 Nov 2014 10:19:11 +0100 (CET) Subject: Reg: NoClassDefFoundError: sun/management/ExtendedPlatformComponent Message-ID: <398791035.231.1416820751496.JavaMail.zimbra@crcdata.cz> Hi, I found regression in latest Java 8u40 b15 linux x64. There is missing internal class in JDK. Submitted as bug with Review ID: 9048260. Martin StackTrace: java.lang.NoClassDefFoundError: sun/management/ExtendedPlatformComponent at java.lang.management.ManagementFactory.getPlatformMBeanServer(ManagementFactory.java:494) at org.apache.derby.impl.services.jmx.JMXManagementService$1.run(Unknown Source) at org.apache.derby.impl.services.jmx.JMXManagementService$1.run(Unknown Source) at java.security.AccessController.doPrivileged(Native Method) at org.apache.derby.impl.services.jmx.JMXManagementService.findServer(Unknown Source) at org.apache.derby.impl.services.jmx.JMXManagementService.boot(Unknown Source) at org.apache.derby.impl.services.monitor.BaseMonitor.boot(Unknown Source) at org.apache.derby.impl.services.monitor.TopService.bootModule(Unknown Source) at org.apache.derby.impl.services.monitor.BaseMonitor.startModule(Unknown Source) at org.apache.derby.iapi.services.monitor.Monitor.startSystemModule(Unknown Source) at org.apache.derby.impl.services.monitor.BaseMonitor.runWithState(Unknown Source) at org.apache.derby.impl.services.monitor.FileMonitor.(Unknown Source) at org.apache.derby.iapi.services.monitor.Monitor.startMonitor(Unknown Source) at org.apache.derby.iapi.jdbc.JDBCBoot.boot(Unknown Source) at org.apache.derby.jdbc.EmbeddedDriver.boot(Unknown Source) at org.apache.derby.jdbc.EmbeddedDriver.(Unknown Source) at java.lang.Class.forName0(Native Method) at java.lang.Class.forName(Class.java:264) From Alan.Bateman at oracle.com Mon Nov 24 09:29:13 2014 From: Alan.Bateman at oracle.com (Alan Bateman) Date: Mon, 24 Nov 2014 09:29:13 +0000 Subject: Reg: NoClassDefFoundError: sun/management/ExtendedPlatformComponent In-Reply-To: <398791035.231.1416820751496.JavaMail.zimbra@crcdata.cz> References: <398791035.231.1416820751496.JavaMail.zimbra@crcdata.cz> Message-ID: <5472FA69.6040303@oracle.com> Thanks. From what I can tell, this one was specific to Oracle JDK builds and has already been fixed for an upcoming 8u40 build. -Alan On 24/11/2014 09:19, Janda Martin wrote: > Hi, > > I found regression in latest Java 8u40 b15 linux x64. There is missing internal class in JDK. Submitted as bug with Review ID: 9048260. > > Martin > > StackTrace: > > java.lang.NoClassDefFoundError: sun/management/ExtendedPlatformComponent > at java.lang.management.ManagementFactory.getPlatformMBeanServer(ManagementFactory.java:494) > at org.apache.derby.impl.services.jmx.JMXManagementService$1.run(Unknown Source) > at org.apache.derby.impl.services.jmx.JMXManagementService$1.run(Unknown Source) > at java.security.AccessController.doPrivileged(Native Method) > at org.apache.derby.impl.services.jmx.JMXManagementService.findServer(Unknown Source) > at org.apache.derby.impl.services.jmx.JMXManagementService.boot(Unknown Source) > at org.apache.derby.impl.services.monitor.BaseMonitor.boot(Unknown Source) > at org.apache.derby.impl.services.monitor.TopService.bootModule(Unknown Source) > at org.apache.derby.impl.services.monitor.BaseMonitor.startModule(Unknown Source) > at org.apache.derby.iapi.services.monitor.Monitor.startSystemModule(Unknown Source) > at org.apache.derby.impl.services.monitor.BaseMonitor.runWithState(Unknown Source) > at org.apache.derby.impl.services.monitor.FileMonitor.(Unknown Source) > at org.apache.derby.iapi.services.monitor.Monitor.startMonitor(Unknown Source) > at org.apache.derby.iapi.jdbc.JDBCBoot.boot(Unknown Source) > at org.apache.derby.jdbc.EmbeddedDriver.boot(Unknown Source) > at org.apache.derby.jdbc.EmbeddedDriver.(Unknown Source) > at java.lang.Class.forName0(Native Method) > at java.lang.Class.forName(Class.java:264) > From daniel.fuchs at oracle.com Mon Nov 24 09:32:48 2014 From: daniel.fuchs at oracle.com (Daniel Fuchs) Date: Mon, 24 Nov 2014 10:32:48 +0100 Subject: Reg: NoClassDefFoundError: sun/management/ExtendedPlatformComponent In-Reply-To: <398791035.231.1416820751496.JavaMail.zimbra@crcdata.cz> References: <398791035.231.1416820751496.JavaMail.zimbra@crcdata.cz> Message-ID: <5472FB40.5050705@oracle.com> On 24/11/14 10:19, Janda Martin wrote: > Hi, > > I found regression in latest Java 8u40 b15 linux x64. There is missing internal class in JDK. Submitted as bug with Review ID: 9048260. Hi Martin, This should now have been fixed in the jdk8u-dev forest by https://bugs.openjdk.java.net/browse/JDK-8065397 best regards, -- daniel > > Martin > > StackTrace: > > java.lang.NoClassDefFoundError: sun/management/ExtendedPlatformComponent > at java.lang.management.ManagementFactory.getPlatformMBeanServer(ManagementFactory.java:494) > at org.apache.derby.impl.services.jmx.JMXManagementService$1.run(Unknown Source) > at org.apache.derby.impl.services.jmx.JMXManagementService$1.run(Unknown Source) > at java.security.AccessController.doPrivileged(Native Method) > at org.apache.derby.impl.services.jmx.JMXManagementService.findServer(Unknown Source) > at org.apache.derby.impl.services.jmx.JMXManagementService.boot(Unknown Source) > at org.apache.derby.impl.services.monitor.BaseMonitor.boot(Unknown Source) > at org.apache.derby.impl.services.monitor.TopService.bootModule(Unknown Source) > at org.apache.derby.impl.services.monitor.BaseMonitor.startModule(Unknown Source) > at org.apache.derby.iapi.services.monitor.Monitor.startSystemModule(Unknown Source) > at org.apache.derby.impl.services.monitor.BaseMonitor.runWithState(Unknown Source) > at org.apache.derby.impl.services.monitor.FileMonitor.(Unknown Source) > at org.apache.derby.iapi.services.monitor.Monitor.startMonitor(Unknown Source) > at org.apache.derby.iapi.jdbc.JDBCBoot.boot(Unknown Source) > at org.apache.derby.jdbc.EmbeddedDriver.boot(Unknown Source) > at org.apache.derby.jdbc.EmbeddedDriver.(Unknown Source) > at java.lang.Class.forName0(Native Method) > at java.lang.Class.forName(Class.java:264) > From jandam at crcdata.cz Mon Nov 24 09:45:13 2014 From: jandam at crcdata.cz (Janda Martin) Date: Mon, 24 Nov 2014 10:45:13 +0100 (CET) Subject: Reg: NoClassDefFoundError: sun/management/ExtendedPlatformComponent In-Reply-To: <686888130.251.1416822311477.JavaMail.zimbra@crcdata.cz> References: <398791035.231.1416820751496.JavaMail.zimbra@crcdata.cz> <5472FB40.5050705@oracle.com> Message-ID: <481007129.252.1416822313407.JavaMail.zimbra@crcdata.cz> Thank you all for fast reply, I didn't check properly bug tracking system for reported/fixed issues. I'm sorry. Martin ----- Original Message ----- From: "Daniel Fuchs" To: "core-libs-dev" Sent: Monday, November 24, 2014 10:32:48 AM Subject: Re: Reg: NoClassDefFoundError: sun/management/ExtendedPlatformComponent On 24/11/14 10:19, Janda Martin wrote: > Hi, > > I found regression in latest Java 8u40 b15 linux x64. There is missing internal class in JDK. Submitted as bug with Review ID: 9048260. Hi Martin, This should now have been fixed in the jdk8u-dev forest by https://bugs.openjdk.java.net/browse/JDK-8065397 best regards, -- daniel > > Martin > > StackTrace: > > java.lang.NoClassDefFoundError: sun/management/ExtendedPlatformComponent > at java.lang.management.ManagementFactory.getPlatformMBeanServer(ManagementFactory.java:494) > at org.apache.derby.impl.services.jmx.JMXManagementService$1.run(Unknown Source) > at org.apache.derby.impl.services.jmx.JMXManagementService$1.run(Unknown Source) > at java.security.AccessController.doPrivileged(Native Method) > at org.apache.derby.impl.services.jmx.JMXManagementService.findServer(Unknown Source) > at org.apache.derby.impl.services.jmx.JMXManagementService.boot(Unknown Source) > at org.apache.derby.impl.services.monitor.BaseMonitor.boot(Unknown Source) > at org.apache.derby.impl.services.monitor.TopService.bootModule(Unknown Source) > at org.apache.derby.impl.services.monitor.BaseMonitor.startModule(Unknown Source) > at org.apache.derby.iapi.services.monitor.Monitor.startSystemModule(Unknown Source) > at org.apache.derby.impl.services.monitor.BaseMonitor.runWithState(Unknown Source) > at org.apache.derby.impl.services.monitor.FileMonitor.(Unknown Source) > at org.apache.derby.iapi.services.monitor.Monitor.startMonitor(Unknown Source) > at org.apache.derby.iapi.jdbc.JDBCBoot.boot(Unknown Source) > at org.apache.derby.jdbc.EmbeddedDriver.boot(Unknown Source) > at org.apache.derby.jdbc.EmbeddedDriver.(Unknown Source) > at java.lang.Class.forName0(Native Method) > at java.lang.Class.forName(Class.java:264) > From amy.lu at oracle.com Mon Nov 24 09:47:19 2014 From: amy.lu at oracle.com (Amy Lu) Date: Mon, 24 Nov 2014 17:47:19 +0800 Subject: RFR 8060026: Update test/tools/launcher tests to eliminate dependency on sun.tools.jar.Main Message-ID: <5472FEA7.1060405@oracle.com> Some test/tools/launcher tests have dependency on sun.tools.jar.Main The dependency actually comes from TestHelper.java This fix is to remove above internal JDK API dependency from the tests. bug: https://bugs.openjdk.java.net/browse/JDK-8060026 webrev: http://cr.openjdk.java.net/~weijun/8060026/webrev.00/ Thanks, Amy From erik.joelsson at oracle.com Mon Nov 24 10:44:47 2014 From: erik.joelsson at oracle.com (Erik Joelsson) Date: Mon, 24 Nov 2014 11:44:47 +0100 Subject: RFR: 8065138 - Encodings.isRecognizedEnconding sometimes fails to recognize 'UTF8' In-Reply-To: <546E23EF.4000506@oracle.com> References: <546C7C2D.3060805@oracle.com> <546CDE30.6030907@oracle.com> <546CFF3C.2030904@oracle.com> <546D0285.3090504@oracle.com> <546D2C7C.5020103@oracle.com> <546D3126.5010301@oracle.com> <546D3ECE.2040500@oracle.com> <546DB3BB.7020901@oracle.com> <546DB5AC.60401@oracle.com> <546DEE70.6020306@oracle.com> <546E23EF.4000506@oracle.com> Message-ID: <54730C1F.10101@oracle.com> Hello Daniel, Test seems like a great idea. Thanks! /Erik On 2014-11-20 18:25, Daniel Fuchs wrote: > On 20/11/14 14:36, Erik Joelsson wrote: >> Here is my proposal for fixing the particular issue of generating the >> correct properties files. I'm simply adding LC_ALL=C to the whole >> command line instead of just the sort at the end. It seems to require >> using "export" to get picked up. > > Hi Erik, > > Looks good to me. > > I have applied your patch (manually, because the copy/paste > seems to have eaten the tab characters away, which caused patch to > reject the diff) - and I confirm that the issue has disappeared. > > Thanks for solving this! > > Do you think it would be worth to commit this test modification > later on, in a followup Bug/RFE? > > http://cr.openjdk.java.net/~dfuchs/webrev_8065138/webrev.00/jdk/test/javax/xml/jaxp/Encodings/CheckEncodingPropertiesFile.java.frames.html > > > If so I will take care of it. > > best regards, > > -- daniel > > >> Bug: https://bugs.openjdk.java.net/browse/JDK-8065138 >> Patch: >> diff --git a/make/common/JavaCompilation.gmk >> b/make/common/JavaCompilation.gmk >> --- a/make/common/JavaCompilation.gmk >> +++ b/make/common/JavaCompilation.gmk >> @@ -400,13 +400,15 @@ >> # Now we can setup the depency that will trigger the copying. >> $$($1_BIN)$$($2_TARGET) : $2 >> $(MKDIR) -p $$(@D) >> - $(CAT) $$< | $(SED) -e 's/\([^\\]\):/\1\\:/g' -e >> 's/\([^\\]\)=/\1\\=/g' \ >> + export LC_ALL=C ; $(CAT) $$< \ >> + | $(SED) -e 's/\([^\\]\):/\1\\:/g' -e 's/\([^\\]\)=/\1\\=/g' \ >> -e 's/\([^\\]\)!/\1\\!/g' -e 's/#.*/#/g' \ >> | $(SED) -f "$(SRC_ROOT)/make/common/support/unicode2x.sed" \ >> | $(SED) -e '/^#/d' -e '/^$$$$/d' \ >> -e :a -e '/\\$$$$/N; s/\\\n//; ta' \ >> -e 's/^[ ]*//;s/[ ]*$$$$//' \ >> - -e 's/\\=/=/' | LC_ALL=C $(SORT) > $$@ >> + -e 's/\\=/=/' \ >> + | $(SORT) > $$@ >> $(CHMOD) -f ug+w $$@ >> >> # And do not forget this target >> >> >> I filed a separate issue [1] for investigating the use of pipefail. >> >> /Erik >> >> [1] https://bugs.openjdk.java.net/browse/JDK-8065576 >> >> On 2014-11-20 10:34, Daniel Fuchs wrote: >>> On 11/20/14 10:26 AM, Erik Joelsson wrote: >>>> Hello, >>>> >>>> On 2014-11-20 02:20, Martin Buchholz wrote: >>>>> Amusingly, the $(SORT) has an LC_ALL=C carefully placed before it, >>>>> but >>>>> the $(SED)s need it too! >>>> Yes, I think that's the correct fix in this case. >>>>> On Wed, Nov 19, 2014 at 5:18 PM, Martin Buchholz >>>>> wrote: >>>>>> [+ build-dev] >>>>>> >>>>>> I think I see the problem. By default, a Unix pipeline sadly fails >>>>>> only when the last command fails. In bash, you can change this to a >>>>>> more sensible default via >>>>>> set -o pipefail >>>>>> but that's not portable enough for openjdk. >>>> This is interesting and something I had missed. I will experiment >>>> with enabling pipefail if configure finds support for it. This will >>>> include setting the SHELL to bash. We really should fail instead of >>>> silently generating broken builds. >>>> >>>> Daniel, I can take over this bug if you want and work on a proper >>>> build fix. >>> >>> Thanks Erik! You are welcome! >>> So the whole issue seems to be that my default setting is >>> LC_ALL=en_US.UTF-8 >>> whereas sed requires LC_ALL=C to work properly on these property >>> files... >>> >>> When the test first failed I tried to rerun the test with LC_ALL=C - >>> with no success >>> of course. But I never thought of rebuilding with LC_ALL=C :-( >>> >>> My apologies for the red herring :-( >>> >>> best regards >>> >>> -- daniel >>> >>>> >>>> /Erik >>>>>> $(CAT) $$< | $(SED) -e 's/\([^\\]\):/\1\\:/g' -e >>>>>> 's/\([^\\]\)=/\1\\=/g' \ >>>>>> -e 's/\([^\\]\)!/\1\\!/g' -e 's/#.*/#/g' \ >>>>>> | $(SED) -f "$(SRC_ROOT)/make/common/support/unicode2x.sed" \ >>>>>> | $(SED) -e '/^#/d' -e '/^$$$$/d' \ >>>>>> -e :a -e '/\\$$$$/N; s/\\\n//; ta' \ >>>>>> -e 's/^[ ]*//;s/[ ]*$$$$//' \ >>>>>> -e 's/\\=/=/' | LC_ALL=C $(SORT) > $$@ >>>>>> >>>>>> On Wed, Nov 19, 2014 at 5:07 PM, huizhe wang >>>>>> wrote: >>>>>>> On 11/19/2014 4:09 PM, Mandy Chung wrote: >>>>>>>> >>>>>>>> On 11/19/2014 3:49 PM, Mandy Chung wrote: >>>>>>>>> >>>>>>>>> On 11/19/2014 12:50 PM, Daniel Fuchs wrote: >>>>>>>>>> On 11/19/14 9:36 PM, Mandy Chung wrote: >>>>>>>>>>> resources.jar will be gone when we move to the modular runtime >>>>>>>>>>> image >>>>>>>>>>> (JEP 220 [1]). >>>>>>>>>>> JDK-8065138 and JDK-8065365 will become non-issue in JDK 9. >>>>>>>>>> Do you mean that the property files will no longer be stripped >>>>>>>>>> of their >>>>>>>>>> comments? >>>>>>>>> >>>>>>>>> (sorry for my delay in reply as I was trying to get the number >>>>>>>>> of the >>>>>>>>> resources in the modular image vs resources.jar but got >>>>>>>>> distracted.) >>>>>>>>> >>>>>>>>> The current version copies all bytes when generating the modular >>>>>>>>> image. >>>>>>>>> This is a good question whether we should strip off the comments >>>>>>>>> when >>>>>>>>> writing to the modular runtime image. I think we should look >>>>>>>>> at the >>>>>>>>> footprint and any performance saving and determine if we should >>>>>>>>> do the same >>>>>>>>> in JDK 9. >>>>>>>>> >>>>>>>> I looked at the exploded image under >>>>>>>> BUILD_OUTPUTDIR/jdk/modules/java.xml >>>>>>>> and found that the comments of Encodings.properties are stripped. >>>>>>>> I was >>>>>>>> confused with the mention of resources.jar that I assume it was a >>>>>>>> step >>>>>>>> stripping the comments before writing to resources.jar. This is >>>>>>>> still >>>>>>>> an issue in jigsaw m2 I believe. >>>>>>>> >>>>>>>> Where does the build strip the comments? >>>>>>> >>>>>>> A previous issue was that the build process strips off anything >>>>>>> after '#' >>>>>>> when copying properties files. In JDK8: >>>>>>> jaxp/make/BuildJaxp.gmk: >>>>>>> $(JAXP_OUTPUTDIR)/classes/%.properties: >>>>>>> $(JAXP_TOPDIR)/src/%.properties >>>>>>> $(MKDIR) -p $(@D) >>>>>>> $(RM) $@ $@.tmp >>>>>>> $(CAT) $< | LANG=C $(NAWK) '{ sub(/#.*$$/,"#"); print }' >>>>>>> > $@.tmp >>>>>>> $(MV) $@.tmp $@ >>>>>>> >>>>>>> This was fixed in JDK 9. The per-repo process was removed. It >>>>>>> looks like >>>>>>> the properties processing process is now consolidated into >>>>>>> make/common/JavaCompilation.gmk. So the issue Daniel found is new >>>>>>> (in terms >>>>>>> of stripping). Search 'properties files' to see the macro. >>>>>>> >>>>>>> Joe >>>>>>> >>>>>>>> Mandy >>>>>>>> >>>> >>> >> > From paul.sandoz at oracle.com Mon Nov 24 10:55:29 2014 From: paul.sandoz at oracle.com (Paul Sandoz) Date: Mon, 24 Nov 2014 11:55:29 +0100 Subject: RFR: 8065172: More core reflection final and volatile annotations In-Reply-To: <546E09D6.9040800@gmail.com> References: <546E09D6.9040800@gmail.com> Message-ID: On Nov 20, 2014, at 4:33 PM, Peter Levart wrote: > Hi Martin, > > On 11/19/2014 01:42 AM, Martin Buchholz wrote: >> Hi Joe, Peter, Paul >> >> This is the followup on thread safety I promised Peter. > > Looks good. > Same here. > I made the WildcardTypeImpl.[upperBoundASTs, lowerBoundASTs] and TypeVariableImpl.boundASTs fields volatile in my version of patch (instead of final): > > http://cr.openjdk.java.net/~plevart/jdk9-dev/GenericsReflectionRaces/webrev.01/ > > ...so that after the structure they point to has been parsed into bound types, they can be thrown away. The comments indicate that possibility already, but the implementor was afraid to do it because of possible races. I think I got it right here. > I think so too. Paul. From daniel.fuchs at oracle.com Mon Nov 24 11:00:59 2014 From: daniel.fuchs at oracle.com (Daniel Fuchs) Date: Mon, 24 Nov 2014 12:00:59 +0100 Subject: RFR: 8065138 - Encodings.isRecognizedEnconding sometimes fails to recognize 'UTF8' In-Reply-To: <54730C1F.10101@oracle.com> References: <546C7C2D.3060805@oracle.com> <546CDE30.6030907@oracle.com> <546CFF3C.2030904@oracle.com> <546D0285.3090504@oracle.com> <546D2C7C.5020103@oracle.com> <546D3126.5010301@oracle.com> <546D3ECE.2040500@oracle.com> <546DB3BB.7020901@oracle.com> <546DB5AC.60401@oracle.com> <546DEE70.6020306@oracle.com> <546E23EF.4000506@oracle.com> <54730C1F.10101@oracle.com> Message-ID: <54730FEB.9020909@oracle.com> On 24/11/14 11:44, Erik Joelsson wrote: > Hello Daniel, > > Test seems like a great idea. Thanks! OK - I have logged JDK-8065748 https://bugs.openjdk.java.net/browse/JDK-8065748 I'll send a patch for review when your fix is in :-) Thanks! -- daniel > > /Erik > > On 2014-11-20 18:25, Daniel Fuchs wrote: >> On 20/11/14 14:36, Erik Joelsson wrote: >>> Here is my proposal for fixing the particular issue of generating the >>> correct properties files. I'm simply adding LC_ALL=C to the whole >>> command line instead of just the sort at the end. It seems to require >>> using "export" to get picked up. >> >> Hi Erik, >> >> Looks good to me. >> >> I have applied your patch (manually, because the copy/paste >> seems to have eaten the tab characters away, which caused patch to >> reject the diff) - and I confirm that the issue has disappeared. >> >> Thanks for solving this! >> >> Do you think it would be worth to commit this test modification >> later on, in a followup Bug/RFE? >> >> http://cr.openjdk.java.net/~dfuchs/webrev_8065138/webrev.00/jdk/test/javax/xml/jaxp/Encodings/CheckEncodingPropertiesFile.java.frames.html >> >> >> If so I will take care of it. >> >> best regards, >> >> -- daniel >> >> >>> Bug: https://bugs.openjdk.java.net/browse/JDK-8065138 >>> Patch: >>> diff --git a/make/common/JavaCompilation.gmk >>> b/make/common/JavaCompilation.gmk >>> --- a/make/common/JavaCompilation.gmk >>> +++ b/make/common/JavaCompilation.gmk >>> @@ -400,13 +400,15 @@ >>> # Now we can setup the depency that will trigger the copying. >>> $$($1_BIN)$$($2_TARGET) : $2 >>> $(MKDIR) -p $$(@D) >>> - $(CAT) $$< | $(SED) -e 's/\([^\\]\):/\1\\:/g' -e >>> 's/\([^\\]\)=/\1\\=/g' \ >>> + export LC_ALL=C ; $(CAT) $$< \ >>> + | $(SED) -e 's/\([^\\]\):/\1\\:/g' -e 's/\([^\\]\)=/\1\\=/g' \ >>> -e 's/\([^\\]\)!/\1\\!/g' -e 's/#.*/#/g' \ >>> | $(SED) -f "$(SRC_ROOT)/make/common/support/unicode2x.sed" \ >>> | $(SED) -e '/^#/d' -e '/^$$$$/d' \ >>> -e :a -e '/\\$$$$/N; s/\\\n//; ta' \ >>> -e 's/^[ ]*//;s/[ ]*$$$$//' \ >>> - -e 's/\\=/=/' | LC_ALL=C $(SORT) > $$@ >>> + -e 's/\\=/=/' \ >>> + | $(SORT) > $$@ >>> $(CHMOD) -f ug+w $$@ >>> >>> # And do not forget this target >>> >>> >>> I filed a separate issue [1] for investigating the use of pipefail. >>> >>> /Erik >>> >>> [1] https://bugs.openjdk.java.net/browse/JDK-8065576 >>> >>> On 2014-11-20 10:34, Daniel Fuchs wrote: >>>> On 11/20/14 10:26 AM, Erik Joelsson wrote: >>>>> Hello, >>>>> >>>>> On 2014-11-20 02:20, Martin Buchholz wrote: >>>>>> Amusingly, the $(SORT) has an LC_ALL=C carefully placed before it, >>>>>> but >>>>>> the $(SED)s need it too! >>>>> Yes, I think that's the correct fix in this case. >>>>>> On Wed, Nov 19, 2014 at 5:18 PM, Martin Buchholz >>>>>> wrote: >>>>>>> [+ build-dev] >>>>>>> >>>>>>> I think I see the problem. By default, a Unix pipeline sadly fails >>>>>>> only when the last command fails. In bash, you can change this to a >>>>>>> more sensible default via >>>>>>> set -o pipefail >>>>>>> but that's not portable enough for openjdk. >>>>> This is interesting and something I had missed. I will experiment >>>>> with enabling pipefail if configure finds support for it. This will >>>>> include setting the SHELL to bash. We really should fail instead of >>>>> silently generating broken builds. >>>>> >>>>> Daniel, I can take over this bug if you want and work on a proper >>>>> build fix. >>>> >>>> Thanks Erik! You are welcome! >>>> So the whole issue seems to be that my default setting is >>>> LC_ALL=en_US.UTF-8 >>>> whereas sed requires LC_ALL=C to work properly on these property >>>> files... >>>> >>>> When the test first failed I tried to rerun the test with LC_ALL=C - >>>> with no success >>>> of course. But I never thought of rebuilding with LC_ALL=C :-( >>>> >>>> My apologies for the red herring :-( >>>> >>>> best regards >>>> >>>> -- daniel >>>> >>>>> >>>>> /Erik >>>>>>> $(CAT) $$< | $(SED) -e 's/\([^\\]\):/\1\\:/g' -e >>>>>>> 's/\([^\\]\)=/\1\\=/g' \ >>>>>>> -e 's/\([^\\]\)!/\1\\!/g' -e 's/#.*/#/g' \ >>>>>>> | $(SED) -f "$(SRC_ROOT)/make/common/support/unicode2x.sed" \ >>>>>>> | $(SED) -e '/^#/d' -e '/^$$$$/d' \ >>>>>>> -e :a -e '/\\$$$$/N; s/\\\n//; ta' \ >>>>>>> -e 's/^[ ]*//;s/[ ]*$$$$//' \ >>>>>>> -e 's/\\=/=/' | LC_ALL=C $(SORT) > $$@ >>>>>>> >>>>>>> On Wed, Nov 19, 2014 at 5:07 PM, huizhe wang >>>>>>> wrote: >>>>>>>> On 11/19/2014 4:09 PM, Mandy Chung wrote: >>>>>>>>> >>>>>>>>> On 11/19/2014 3:49 PM, Mandy Chung wrote: >>>>>>>>>> >>>>>>>>>> On 11/19/2014 12:50 PM, Daniel Fuchs wrote: >>>>>>>>>>> On 11/19/14 9:36 PM, Mandy Chung wrote: >>>>>>>>>>>> resources.jar will be gone when we move to the modular runtime >>>>>>>>>>>> image >>>>>>>>>>>> (JEP 220 [1]). >>>>>>>>>>>> JDK-8065138 and JDK-8065365 will become non-issue in JDK 9. >>>>>>>>>>> Do you mean that the property files will no longer be stripped >>>>>>>>>>> of their >>>>>>>>>>> comments? >>>>>>>>>> >>>>>>>>>> (sorry for my delay in reply as I was trying to get the number >>>>>>>>>> of the >>>>>>>>>> resources in the modular image vs resources.jar but got >>>>>>>>>> distracted.) >>>>>>>>>> >>>>>>>>>> The current version copies all bytes when generating the modular >>>>>>>>>> image. >>>>>>>>>> This is a good question whether we should strip off the comments >>>>>>>>>> when >>>>>>>>>> writing to the modular runtime image. I think we should look >>>>>>>>>> at the >>>>>>>>>> footprint and any performance saving and determine if we should >>>>>>>>>> do the same >>>>>>>>>> in JDK 9. >>>>>>>>>> >>>>>>>>> I looked at the exploded image under >>>>>>>>> BUILD_OUTPUTDIR/jdk/modules/java.xml >>>>>>>>> and found that the comments of Encodings.properties are stripped. >>>>>>>>> I was >>>>>>>>> confused with the mention of resources.jar that I assume it was a >>>>>>>>> step >>>>>>>>> stripping the comments before writing to resources.jar. This is >>>>>>>>> still >>>>>>>>> an issue in jigsaw m2 I believe. >>>>>>>>> >>>>>>>>> Where does the build strip the comments? >>>>>>>> >>>>>>>> A previous issue was that the build process strips off anything >>>>>>>> after '#' >>>>>>>> when copying properties files. In JDK8: >>>>>>>> jaxp/make/BuildJaxp.gmk: >>>>>>>> $(JAXP_OUTPUTDIR)/classes/%.properties: >>>>>>>> $(JAXP_TOPDIR)/src/%.properties >>>>>>>> $(MKDIR) -p $(@D) >>>>>>>> $(RM) $@ $@.tmp >>>>>>>> $(CAT) $< | LANG=C $(NAWK) '{ sub(/#.*$$/,"#"); print }' >>>>>>>> > $@.tmp >>>>>>>> $(MV) $@.tmp $@ >>>>>>>> >>>>>>>> This was fixed in JDK 9. The per-repo process was removed. It >>>>>>>> looks like >>>>>>>> the properties processing process is now consolidated into >>>>>>>> make/common/JavaCompilation.gmk. So the issue Daniel found is new >>>>>>>> (in terms >>>>>>>> of stripping). Search 'properties files' to see the macro. >>>>>>>> >>>>>>>> Joe >>>>>>>> >>>>>>>>> Mandy >>>>>>>>> >>>>> >>>> >>> >> > From paul.sandoz at oracle.com Mon Nov 24 11:05:33 2014 From: paul.sandoz at oracle.com (Paul Sandoz) Date: Mon, 24 Nov 2014 12:05:33 +0100 Subject: [9, 8u40] RFR (XXS): 8059880: Get rid of LambdaForm interpretation In-Reply-To: <546C6FE4.60008@oracle.com> References: <546BB3C2.60202@oracle.com> <546BD761.6030108@oracle.com> <546C6FE4.60008@oracle.com> Message-ID: <33A36142-CDBA-43A8-866D-149D1756A565@oracle.com> On Nov 19, 2014, at 11:24 AM, Vladimir Ivanov wrote: > Aleksey, Duncan, thanks for the review and the confirmation that it doesn't break stuff for you. > > Any Reviews, please? :-) > Looks good. > Best regards, > Vladimir Ivanov > > On 11/19/14, 2:23 PM, MacGregor, Duncan (GE Energy Management) wrote: >> On 18/11/2014 23:33, "Aleksey Shipilev" >> wrote: >>> On 11/19/2014 12:01 AM, Vladimir Ivanov wrote: >>>> http://cr.openjdk.java.net/~vlivanov/8059880/webrev.00/ >>>> https://bugs.openjdk.java.net/browse/JDK-8059880 >>> >>> Yes, for the love of God, GO FOR IT. >> Indeed! Paul. >> Seconded. Startup of our stuff seems fine now with a compile threshold of >> zero, and it will make stacks so much easier to read in the debugger. :-) >> >> _______________________________________________ >> mlvm-dev mailing list >> mlvm-dev at openjdk.java.net >> http://mail.openjdk.java.net/mailman/listinfo/mlvm-dev >> > _______________________________________________ > mlvm-dev mailing list > mlvm-dev at openjdk.java.net > http://mail.openjdk.java.net/mailman/listinfo/mlvm-dev From paul.sandoz at oracle.com Mon Nov 24 11:09:20 2014 From: paul.sandoz at oracle.com (Paul Sandoz) Date: Mon, 24 Nov 2014 12:09:20 +0100 Subject: RFR 8060026: Update test/tools/launcher tests to eliminate dependency on sun.tools.jar.Main In-Reply-To: <5472FEA7.1060405@oracle.com> References: <5472FEA7.1060405@oracle.com> Message-ID: On Nov 24, 2014, at 10:47 AM, Amy Lu wrote: > Some test/tools/launcher tests have dependency on sun.tools.jar.Main > The dependency actually comes from TestHelper.java > This fix is to remove above internal JDK API dependency from the tests. > > bug: https://bugs.openjdk.java.net/browse/JDK-8060026 > webrev: http://cr.openjdk.java.net/~weijun/8060026/webrev.00/ > Looks ok to me, Paul. From daniel.fuchs at oracle.com Mon Nov 24 11:30:41 2014 From: daniel.fuchs at oracle.com (Daniel Fuchs) Date: Mon, 24 Nov 2014 12:30:41 +0100 Subject: RFR: 8065748 - Add a test to verify that non ascii characters in Encodings.properties do not cause issues Message-ID: <547316E1.6060601@oracle.com> Hi, Please find below a small webrev that adds a trivial sanity test to CheckEncodingPropertiesFile.java http://cr.openjdk.java.net/~dfuchs/webrev_8065748/webrev.00 This is as a followup of Joel's fix for https://bugs.openjdk.java.net/browse/JDK-8065138 best regards, -- daniel From Alan.Bateman at oracle.com Mon Nov 24 11:33:42 2014 From: Alan.Bateman at oracle.com (Alan Bateman) Date: Mon, 24 Nov 2014 11:33:42 +0000 Subject: RFR 8060026: Update test/tools/launcher tests to eliminate dependency on sun.tools.jar.Main In-Reply-To: <5472FEA7.1060405@oracle.com> References: <5472FEA7.1060405@oracle.com> Message-ID: <54731796.305@oracle.com> On 24/11/2014 09:47, Amy Lu wrote: > Some test/tools/launcher tests have dependency on sun.tools.jar.Main > The dependency actually comes from TestHelper.java > This fix is to remove above internal JDK API dependency from the tests. > > bug: https://bugs.openjdk.java.net/browse/JDK-8060026 > webrev: http://cr.openjdk.java.net/~weijun/8060026/webrev.00/ > This looks okay to me, I'm sure Kumar will want to look at this too. -Alan From paul.sandoz at oracle.com Mon Nov 24 14:32:06 2014 From: paul.sandoz at oracle.com (Paul Sandoz) Date: Mon, 24 Nov 2014 15:32:06 +0100 Subject: Library enhancement proposal for copying data from/to Reader/Writer InputStream/OutputStream In-Reply-To: References: <566A428C-1A13-4DEE-940B-2EBE375F2A7A@reini.net> <51329A99-55E5-4A42-9180-D872366B4DEE@oracle.com> <546EABED.9030301@oracle.com> <546EF98C.40704@oracle.com> Message-ID: <1D392C1F-E6CC-494D-97F8-C08C5BA36392@oracle.com> Hi Patrick, To speed things along for now i recommend creating a new class say IOUtils or preferably ByteStreams. You should keep things simple for an initial iteration and just add one static method :-) which is essentially a refined copy of the private method that Pavel pointed out. Then write some tests for that method [1]. Then consider any nio related classes for an equivalent copy method. Some other candidates to consider are on Guava's ByteStreams class: http://docs.guava-libraries.googlecode.com/git/javadoc/com/google/common/io/ByteStreams.html It's very easy to get carried away and over-engineer such utility methods thinking they need to be all things to all people so we need to consider them carefully and i think the ones you included with size and reporting are not required. Simply just one static method copying input to output will go a long way (i have lost count of how many times i have reimplemented that!). Hth, Paul. [1] You can download jtreg from here: https://adopt-openjdk.ci.cloudbees.com/view/OpenJDK/job/jtreg/ for executing JDK tests. Depending on how you want to write your tests you can also make 'em run standalone too, but sometimes it is easier convenient to use testng and execute via jtreg. On Nov 21, 2014, at 3:12 PM, Patrick Reinhart wrote: > >> Am 21.11.2014 um 09:36 schrieb Alan Bateman : >> >> On 21/11/2014 03:05, Stuart Marks wrote: >>> >>> Thanks to Pavel for pointing out the existing copy operations in the nio Files class. I think there's a case for the InputStream => OutputStream copy method to be placed there too, although I admit it is somewhat unusual in that it doesn't actually have anything to do with files. >>> >>> At my first encounter with the nio.Files class some years ago I saw the following copying methods: >>> >>> copy(istream, targetfile) >>> copy(sourcefile, ostream) >>> copy(sourcefile, targetfile) >>> >>> and I immediately thought, "Where is copy(istream, ostream)?" So to me at least, it makes more sense to be in the Files class than in some newly created IOUtils class. (I'd like to hear further from Alan on this.) >>> >>> As Pavel pointed out, the logic is also already there in the Files class. Probably the way to proceed would be to rename the existing (private) method to be something like copyInternal() and then create a new, public copy(istream, ostream) method that does argument checking before calling copyInternal(). >> >> The Files class works on files, it's not the right place for a general purpose copy(InputStream, OutputStream). > > That was the reason in my proposal to not put those methods on the Files class. I also would not try to find such methods there. Personally I tried to look for such method on ?Streams? ;-) > >> When we prototyped a copy(InputStream, OutputStream) and many other I/O methods a few years ago then the intention was to put it java.io. One option on the table was a Collections-like utility class with static methods. Another option was to add methods to InputStream, Reader, etc. A concern with the latter was whether the new methods would cause problems for existing InputStream/etc. implementations, similar to concerns about adding default methods to the collection types in 8. I think that discussion needs to happen again and having a few prototypes to get experience with the various approaches would be good too. >> >> -Alan > > It would be good to have such methods in the next release and I would love to help here. > > Patrick From joel.franck at oracle.com Mon Nov 24 15:04:29 2014 From: joel.franck at oracle.com (=?utf-8?Q?Joel_Borggr=C3=A9n-Franck?=) Date: Mon, 24 Nov 2014 16:04:29 +0100 Subject: RFR: 8065172: More core reflection final and volatile annotations In-Reply-To: <546E09D6.9040800@gmail.com> References: <546E09D6.9040800@gmail.com> Message-ID: <6B93C453-C558-4C0E-B7FA-4255507B154D@oracle.com> Hi, > On 20 Nov 2014, at 16:33, Peter Levart wrote: > > Hi Martin, > > On 11/19/2014 01:42 AM, Martin Buchholz wrote: >> Hi Joe, Peter, Paul >> >> This is the followup on thread safety I promised Peter. > > Looks good. +1 > > I made the WildcardTypeImpl.[upperBoundASTs, lowerBoundASTs] and TypeVariableImpl.boundASTs fields volatile in my version of patch (instead of final): > > http://cr.openjdk.java.net/~plevart/jdk9-dev/GenericsReflectionRaces/webrev.01/ > > ...so that after the structure they point to has been parsed into bound types, they can be thrown away. The comments indicate that possibility already, but the implementor was afraid to do it because of possible races. I think I got it right here. > As a cleanup and given the current state of testing here, I slightly favour Martin?s version because it should just make the current behaviour a bit safer. Btw, has anyone seen the assert for upper/lower bounds == null fail in the wild? private FieldTypeSignature[] getUpperBoundASTs() { // check that upper bounds were not evaluated yet assert(upperBounds == null); return upperBoundASTs; } shouldn?t it happen once in a while: public Type[] getUpperBounds() { // lazily initialize bounds if necessary if (upperBounds == null) { // thread gets preempted here, other thread completes init, upperBounds are != null FieldTypeSignature[] fts = getUpperBoundASTs(); // get AST Is the current code only working because most run without esa? cheers /Joel From peter.levart at gmail.com Mon Nov 24 15:36:33 2014 From: peter.levart at gmail.com (Peter Levart) Date: Mon, 24 Nov 2014 16:36:33 +0100 Subject: RFR: 8065172: More core reflection final and volatile annotations In-Reply-To: <6B93C453-C558-4C0E-B7FA-4255507B154D@oracle.com> References: <546E09D6.9040800@gmail.com> <6B93C453-C558-4C0E-B7FA-4255507B154D@oracle.com> Message-ID: <54735081.5050201@gmail.com> On 11/24/2014 04:04 PM, Joel Borggr?n-Franck wrote: > Hi, > > >> On 20 Nov 2014, at 16:33, Peter Levart wrote: >> >> Hi Martin, >> >> On 11/19/2014 01:42 AM, Martin Buchholz wrote: >>> Hi Joe, Peter, Paul >>> >>> This is the followup on thread safety I promised Peter. >> Looks good. > +1 > >> I made the WildcardTypeImpl.[upperBoundASTs, lowerBoundASTs] and TypeVariableImpl.boundASTs fields volatile in my version of patch (instead of final): >> >> http://cr.openjdk.java.net/~plevart/jdk9-dev/GenericsReflectionRaces/webrev.01/ >> >> ...so that after the structure they point to has been parsed into bound types, they can be thrown away. The comments indicate that possibility already, but the implementor was afraid to do it because of possible races. I think I got it right here. >> > > As a cleanup and given the current state of testing here, I slightly favour Martin?s version because it should just make the current behaviour a bit safer. > > Btw, has anyone seen the assert for upper/lower bounds == null fail in the wild? > > private FieldTypeSignature[] getUpperBoundASTs() { > // check that upper bounds were not evaluated yet > assert(upperBounds == null); > return upperBoundASTs; > } > > shouldn?t it happen once in a while: > > public Type[] getUpperBounds() { > // lazily initialize bounds if necessary > if (upperBounds == null) { > > // thread gets preempted here, other thread completes init, upperBounds are != null > > FieldTypeSignature[] fts = getUpperBoundASTs(); // get AST > > > Is the current code only working because most run without esa? ...or because races that would trigger this are very rare, since two consecutive "loads" of upperBounds happen one after the other, so the 2nd load and check might even be optimized away. The private getter for a private field is meaningless in my opinion. That's why I removed it in my version of the patch. Regards, Peter > > cheers > /Joel From pavel.rappo at oracle.com Mon Nov 24 16:22:07 2014 From: pavel.rappo at oracle.com (Pavel Rappo) Date: Mon, 24 Nov 2014 16:22:07 +0000 Subject: Library enhancement proposal for copying data from/to Reader/Writer InputStream/OutputStream In-Reply-To: <1D392C1F-E6CC-494D-97F8-C08C5BA36392@oracle.com> References: <566A428C-1A13-4DEE-940B-2EBE375F2A7A@reini.net> <51329A99-55E5-4A42-9180-D872366B4DEE@oracle.com> <546EABED.9030301@oracle.com> <546EF98C.40704@oracle.com> <1D392C1F-E6CC-494D-97F8-C08C5BA36392@oracle.com> Message-ID: Paul, I wouldn't name it 'ByteStreams' because of the possible subjective associations (and confusion) with java.util.stream.IntStream, java.util.stream.LongStream, etc. IOStreams? -Pavel > On 24 Nov 2014, at 14:32, Paul Sandoz wrote: > > i recommend creating a new class say IOUtils or preferably ByteStreams. From vladimir.x.ivanov at oracle.com Mon Nov 24 15:23:48 2014 From: vladimir.x.ivanov at oracle.com (Vladimir Ivanov) Date: Mon, 24 Nov 2014 19:23:48 +0400 Subject: [9] Review request : JDK-8059070: [TESTBUG] java/lang/invoke/LFCaching/LFMultiThreadCachingTest.java failed - timeout In-Reply-To: <546C7B08.3070700@oracle.com> References: <543D1DEE.8030206@oracle.com> <543F8520.9090408@oracle.com> <8F9DD322-1793-4739-8155-61CE7D2C5AAB@oracle.com> <543F9800.7040406@oracle.com> <543FC71F.8020504@oracle.com> <5440E39C.8070001@oracle.com> <5448E5A2.9030804@oracle.com> <544E1B65.1090400@oracle.com> <5450F8E4.2060808@oracle.com> <545C9D70.6060308@oracle.com> <5460A29E.6070206@oracle.com> <5460CF90.1050303@oracle.com> <5464A890.6080404@oracle.com> <546C7B08.3070700@oracle.com> Message-ID: <54734D84.1040307@oracle.com> Looks good. Best regards, Vladimir Ivanov On 11/19/14, 3:12 PM, Konstantin Shefov wrote: > Hello, Vladimir > > I have updated the Webrev > http://cr.openjdk.java.net/~kshefov/8059070/webrev.02 > I have added DEFAULT_TEST_TIMEOUT constant to Utils class. > > -Konstantin > > On 13.11.2014 15:48, Konstantin Shefov wrote: >> Kindly reminder. >> >> On 10.11.2014 17:45, Konstantin Shefov wrote: >>> Vladimir, thanks for reviewing >>> >>> I have updated the webrev: >>> http://cr.openjdk.java.net/~kshefov/8059070/webrev.02 >>> I have added DEFAULT_TEST_TIMEOUT constant to Utils class. >>> >>> -Konstantin >>> >>> On 10.11.2014 14:33, Vladimir Ivanov wrote: >>>> Kontantin, sorry for the late response. >>>> >>>> In general, the fix looks good. >>>> >>>> I'd move timeout computation logic to Utils: >>>> + private static final long TIMEOUT = >>>> Utils.adjustTimeout(TimeUnit.SECONDS.toMillis(120)); >>>> >>>> Default value (120s) isn't specific to the tests, but an >>>> implementation detail of jtreg. >>>> >>>> Best regards, >>>> Vladimir Ivanov >>>> >>>> On 11/7/14, 2:22 PM, Konstantin Shefov wrote: >>>>> Gently reminder >>>>> >>>>> 29.10.2014 17:25, Konstantin Shefov ?????: >>>>>> Please, review a test bug fix. >>>>>> http://cr.openjdk.java.net/~kshefov/8059070/webrev.01/ >>>>>> >>>>>> -Konstantin >>>>>> >>>>>> On 27.10.2014 13:16, Konstantin Shefov wrote: >>>>>>> Kindly reminder >>>>>>> >>>>>>> On 23.10.2014 19:04, Paul Sandoz wrote: >>>>>>>> On Oct 23, 2014, at 1:25 PM, Konstantin Shefov >>>>>>>> wrote: >>>>>>>>> Gently reminder >>>>>>>>> >>>>>>>>> On 17.10.2014 13:38, Konstantin Shefov wrote: >>>>>>>>>> Hi, >>>>>>>>>> >>>>>>>>>> I have updated the webrev: >>>>>>>>>> http://cr.openjdk.java.net/~kshefov/8059070/webrev.01/ >>>>>>>>>> >>>>>>>> +1 >>>>>>>> >>>>>>>> Sorry for the delay, >>>>>>>> Paul. >>>>>>> >>>>>> >>>>> >>> >> > From patrick at reini.net Mon Nov 24 16:34:23 2014 From: patrick at reini.net (Patrick Reinhart) Date: Mon, 24 Nov 2014 17:34:23 +0100 Subject: Library enhancement proposal for copying data from/to Reader/Writer InputStream/OutputStream In-Reply-To: <1D392C1F-E6CC-494D-97F8-C08C5BA36392@oracle.com> References: <566A428C-1A13-4DEE-940B-2EBE375F2A7A@reini.net> <51329A99-55E5-4A42-9180-D872366B4DEE@oracle.com> <546EABED.9030301@oracle.com> <546EF98C.40704@oracle.com> <1D392C1F-E6CC-494D-97F8-C08C5BA36392@oracle.com> Message-ID: Hi Paul, > Am 24.11.2014 um 15:32 schrieb Paul Sandoz : > > Hi Patrick, > > To speed things along for now i recommend creating a new class say IOUtils or preferably ByteStreams. Hmm, in the case of Reader/Writer ByteStreams seams to be a bit confusing for me, should then the reader copy stuff go to also a separate class like ?CharStreams?. Locally I created an implementation named ?IOUtils? containing a copy method for InputStream to OutputStream. > You should keep things simple for an initial iteration and just add one static method :-) which is essentially a refined copy of the private method that Pavel pointed out. Then write some tests for that method [1]. Then consider any nio related classes for an equivalent copy method. Some other candidates to consider are on Guava's ByteStreams class: > > http://docs.guava-libraries.googlecode.com/git/javadoc/com/google/common/io/ByteStreams.html I will also take a look at this > It's very easy to get carried away and over-engineer such utility methods thinking they need to be all things to all people so we need to consider them carefully and i think the ones you included with size and reporting are not required. Simply just one static method copying input to output will go a long way (i have lost count of how many times i have reimplemented that!). For now a single copy(from, to) for Input-/OutputStream and Readable/Appendable would be perfect for now. > Hth, > Paul. > > [1] You can download jtreg from here: > > https://adopt-openjdk.ci.cloudbees.com/view/OpenJDK/job/jtreg/ > > for executing JDK tests. Depending on how you want to write your tests you can also make ?em run standalone too, but sometimes it is easier convenient to use testng and execute via jtreg. The environment I will update in the next. Patrick From richard.warburton at gmail.com Mon Nov 24 16:45:21 2014 From: richard.warburton at gmail.com (Richard Warburton) Date: Mon, 24 Nov 2014 16:45:21 +0000 Subject: Lower overhead String encoding/decoding In-Reply-To: References: <54200F07.4070604@oracle.com> <542441BB.600@oracle.com> <5444BB17.8080708@oracle.com> <544FADA5.7000403@oracle.com> <544FB0F7.8080909@univ-mlv.fr> Message-ID: Hi all, I'm sure everyone is very busy, but is there any chance that someone take a look at the latest iteration of this patch? Thanks for taking the time to look at this - most appreciated. I've pushed >>>> the latest iteration to http://cr.openjdk.java.net/~ >>>> rwarburton/string-patch-webrev-8/ >>> 7Erwarburton/string-patch-webrev-8/>. >>>> >>>> I think this is looking good. >>> >> > Thanks - I've pushed a new iteration to http://cr.openjdk.java.net/~ > rwarburton/string-patch-webrev-9. > > For the constructor then the words "decoding the specified byte buffer", >>> it might be a bit clearer as "decoding the remaining bytes in the ...". >>> >>> For getBytes(ByteBuffer, Charset) then the position is advanced by the >>> bytes written, no need to mention the number of chars read here. >>> >>> In the constructor then you make it clear that malformed/unmappable >>> sequences use the default replacement. This is important to state in the >>> getBytes methods too because the encoding can fail. >> >> > I've made all these changes. > > Hi Richard, hi all, >> Two comments, >> You replace the nullcheck in getBytes() by a requireNonNull and at the >> same time, you don"t use requireNonNull in String(ByteBuffer,Charset), >> no very logical isn't it ? >> > > Thanks for spotting this Remi - I've fixed this issue in my latest > iteration. > > I think you need a supplementary constructor that takes a ByteBuffer and a >> charset name as a String, >> i may be wrong, but it seems that every constructor of String that takes >> a Charset has a dual constructor that takes a Charset as a String. >> As far as I remember, a constructor that takes a Charset as a String may >> be faster because you can share the character decoder >> instead of creating a new one. > > > A good observation. I've added variants which take a String for the > charset name as well the charset variants. > > Having said that - wouldn't it also be a good idea to replicate the > caching on the charset versions as well as the charset name? I don't see > any obvious reason why this isn't possible but perhaps there's something > I'm missing here. Probably cleaner as a separate patch either way. > regards, Richard Warburton http://insightfullogic.com @RichardWarburto From paul.sandoz at oracle.com Mon Nov 24 16:48:11 2014 From: paul.sandoz at oracle.com (Paul Sandoz) Date: Mon, 24 Nov 2014 17:48:11 +0100 Subject: Library enhancement proposal for copying data from/to Reader/Writer InputStream/OutputStream In-Reply-To: References: <566A428C-1A13-4DEE-940B-2EBE375F2A7A@reini.net> <51329A99-55E5-4A42-9180-D872366B4DEE@oracle.com> <546EABED.9030301@oracle.com> <546EF98C.40704@oracle.com> <1D392C1F-E6CC-494D-97F8-C08C5BA36392@oracle.com> Message-ID: On Nov 24, 2014, at 5:34 PM, Patrick Reinhart wrote: > Hi Paul, > >> Am 24.11.2014 um 15:32 schrieb Paul Sandoz : >> >> Hi Patrick, >> >> To speed things along for now i recommend creating a new class say IOUtils or preferably ByteStreams. > > Hmm, in the case of Reader/Writer ByteStreams seams to be a bit confusing for me, should then the reader copy stuff go to also a separate class like ?CharStreams?. Locally I created an implementation named ?IOUtils? containing a copy method for InputStream to OutputStream. > Pick the one you prefer so we don't rat hole down the naming abyss :-) Pavel, your point about the overload with j.u.stream.Stream is a reasonable one (I am not too concerned by it myself). I hope one day in the future we can have Stream :-) Paul. From patrick at reini.net Mon Nov 24 17:03:06 2014 From: patrick at reini.net (Patrick Reinhart) Date: Mon, 24 Nov 2014 18:03:06 +0100 Subject: Library enhancement proposal for copying data from/to Reader/Writer InputStream/OutputStream In-Reply-To: References: <566A428C-1A13-4DEE-940B-2EBE375F2A7A@reini.net> <51329A99-55E5-4A42-9180-D872366B4DEE@oracle.com> <546EABED.9030301@oracle.com> <546EF98C.40704@oracle.com> <1D392C1F-E6CC-494D-97F8-C08C5BA36392@oracle.com> Message-ID: Well I will then suggest using IOUtils with two simple methods: public static long copy(InputStream source, OutputStream target) public static long copy(Readable source, Appendable target) >>> >>> To speed things along for now i recommend creating a new class say IOUtils or preferably ByteStreams. >> >> Hmm, in the case of Reader/Writer ByteStreams seams to be a bit confusing for me, should then the reader copy stuff go to also a separate class like ?CharStreams?. Locally I created an implementation named ?IOUtils? containing a copy method for InputStream to OutputStream. >> > > Pick the one you prefer so we don't rat hole down the naming abyss :-) > > Pavel, your point about the overload with j.u.stream.Stream is a reasonable one (I am not too concerned by it myself). I hope one day in the future we can have Stream :-) > > Paul. Patrick From huizhe.wang at oracle.com Mon Nov 24 17:04:31 2014 From: huizhe.wang at oracle.com (huizhe wang) Date: Mon, 24 Nov 2014 09:04:31 -0800 Subject: RFR: 8065748 - Add a test to verify that non ascii characters in Encodings.properties do not cause issues In-Reply-To: <547316E1.6060601@oracle.com> References: <547316E1.6060601@oracle.com> Message-ID: <5473651F.8040607@oracle.com> Hi Daniel, Would you want to add 8065748 to the @bug tag? Otherwise, the test looks good to me. Best, Joe On 11/24/2014 3:30 AM, Daniel Fuchs wrote: > Hi, > > Please find below a small webrev that adds a trivial sanity > test to CheckEncodingPropertiesFile.java > > http://cr.openjdk.java.net/~dfuchs/webrev_8065748/webrev.00 > > This is as a followup of Joel's fix for > https://bugs.openjdk.java.net/browse/JDK-8065138 > > best regards, > > -- daniel From daniel.fuchs at oracle.com Mon Nov 24 17:12:13 2014 From: daniel.fuchs at oracle.com (Daniel Fuchs) Date: Mon, 24 Nov 2014 18:12:13 +0100 Subject: RFR: 8065748 - Add a test to verify that non ascii characters in Encodings.properties do not cause issues In-Reply-To: <5473651F.8040607@oracle.com> References: <547316E1.6060601@oracle.com> <5473651F.8040607@oracle.com> Message-ID: <547366ED.3060704@oracle.com> On 24/11/14 18:04, huizhe wang wrote: > Hi Daniel, > > Would you want to add 8065748 to the @bug tag? Otherwise, the test > looks good to me. Hi Joe, I don't think adding 8065748 would be a good idea - since the test doesn't test 8065748 (8065748 is about adding the test :-)). Or am I mistaken on the meaning of @bug ? best regards, -- daniel > > Best, > Joe > > On 11/24/2014 3:30 AM, Daniel Fuchs wrote: >> Hi, >> >> Please find below a small webrev that adds a trivial sanity >> test to CheckEncodingPropertiesFile.java >> >> http://cr.openjdk.java.net/~dfuchs/webrev_8065748/webrev.00 >> >> This is as a followup of Joel's fix for >> https://bugs.openjdk.java.net/browse/JDK-8065138 >> >> best regards, >> >> -- daniel > From xueming.shen at oracle.com Mon Nov 24 18:41:07 2014 From: xueming.shen at oracle.com (Xueming Shen) Date: Mon, 24 Nov 2014 10:41:07 -0800 Subject: Lower overhead String encoding/decoding In-Reply-To: References: <54200F07.4070604@oracle.com> <542441BB.600@oracle.com> <5444BB17.8080708@oracle.com> <544FADA5.7000403@oracle.com> <544FB0F7.8080909@univ-mlv.fr> Message-ID: <54737BC3.9010508@oracle.com> Hi Richard, Here are some comments regarding the updated webrev. (1) String(ByteBuffer, String) needs "@throw UEE". (2) It should be "default replacement byte array" not "replace string" for all getBytes() methods when malformed/unmappable (3) for decoding (new String) from ByteBuffer, since it is guaranteed that all bytes in the input ByteBuffer will be decoded, it might be desirable to clearly specify that the position of the buffer will be "advanced to its limit" ? (4) ln#1137 has an extra "*" (5) StringCoding.decode(cs, bb), why do you want to allocate a direct buffer here for "untrusted"? Basically the output buffer "ca" will always be a wrapper of a char[], all our charset implementation will have better performance if both input and output are "array based" (decode on array directly, instead of the "slow" ByteBuffer) -Sherman On 11/24/2014 08:45 AM, Richard Warburton wrote: > Hi all, > > I'm sure everyone is very busy, but is there any chance that someone take a > look at the latest iteration of this patch? > > Thanks for taking the time to look at this - most appreciated. I've pushed >>>>> the latest iteration to http://cr.openjdk.java.net/~ >>>>> rwarburton/string-patch-webrev-8/>>>> 7Erwarburton/string-patch-webrev-8/>. >>>>> >>>>> I think this is looking good. >> Thanks - I've pushed a new iteration to http://cr.openjdk.java.net/~ >> rwarburton/string-patch-webrev-9. >> >> For the constructor then the words "decoding the specified byte buffer", >>>> it might be a bit clearer as "decoding the remaining bytes in the ...". >>>> >>>> For getBytes(ByteBuffer, Charset) then the position is advanced by the >>>> bytes written, no need to mention the number of chars read here. >>>> >>>> In the constructor then you make it clear that malformed/unmappable >>>> sequences use the default replacement. This is important to state in the >>>> getBytes methods too because the encoding can fail. >>> >> I've made all these changes. >> >> Hi Richard, hi all, >>> Two comments, >>> You replace the nullcheck in getBytes() by a requireNonNull and at the >>> same time, you don"t use requireNonNull in String(ByteBuffer,Charset), >>> no very logical isn't it ? >>> >> Thanks for spotting this Remi - I've fixed this issue in my latest >> iteration. >> >> I think you need a supplementary constructor that takes a ByteBuffer and a >>> charset name as a String, >>> i may be wrong, but it seems that every constructor of String that takes >>> a Charset has a dual constructor that takes a Charset as a String. >>> As far as I remember, a constructor that takes a Charset as a String may >>> be faster because you can share the character decoder >>> instead of creating a new one. >> >> A good observation. I've added variants which take a String for the >> charset name as well the charset variants. >> >> Having said that - wouldn't it also be a good idea to replicate the >> caching on the charset versions as well as the charset name? I don't see >> any obvious reason why this isn't possible but perhaps there's something >> I'm missing here. Probably cleaner as a separate patch either way. >> > regards, > > Richard Warburton > > http://insightfullogic.com > @RichardWarburto From huizhe.wang at oracle.com Mon Nov 24 18:53:07 2014 From: huizhe.wang at oracle.com (huizhe wang) Date: Mon, 24 Nov 2014 10:53:07 -0800 Subject: RFR: 8065748 - Add a test to verify that non ascii characters in Encodings.properties do not cause issues In-Reply-To: <547366ED.3060704@oracle.com> References: <547316E1.6060601@oracle.com> <5473651F.8040607@oracle.com> <547366ED.3060704@oracle.com> Message-ID: <54737E93.5080504@oracle.com> On 11/24/2014 9:12 AM, Daniel Fuchs wrote: > On 24/11/14 18:04, huizhe wang wrote: >> Hi Daniel, >> >> Would you want to add 8065748 to the @bug tag? Otherwise, the test >> looks good to me. > > Hi Joe, > > I don't think adding 8065748 would be a good idea - since the test > doesn't test 8065748 (8065748 is about adding the test :-)). > > Or am I mistaken on the meaning of @bug ? These tags (bug, summary, author and etc.) are informational tags. For me, I use them to locate tests or corresponding bug reports. In this case, since there's a reference to 8065138, it's okay if you don't want to add it. Best, Joe > > best regards, > > -- daniel > >> >> Best, >> Joe >> >> On 11/24/2014 3:30 AM, Daniel Fuchs wrote: >>> Hi, >>> >>> Please find below a small webrev that adds a trivial sanity >>> test to CheckEncodingPropertiesFile.java >>> >>> http://cr.openjdk.java.net/~dfuchs/webrev_8065748/webrev.00 >>> >>> This is as a followup of Joel's fix for >>> https://bugs.openjdk.java.net/browse/JDK-8065138 >>> >>> best regards, >>> >>> -- daniel >> > From joel.franck at oracle.com Mon Nov 24 20:25:59 2014 From: joel.franck at oracle.com (=?utf-8?Q?Joel_Borggr=C3=A9n-Franck?=) Date: Mon, 24 Nov 2014 21:25:59 +0100 Subject: RFR: 8065172: More core reflection final and volatile annotations In-Reply-To: <54735081.5050201@gmail.com> References: <546E09D6.9040800@gmail.com> <6B93C453-C558-4C0E-B7FA-4255507B154D@oracle.com> <54735081.5050201@gmail.com> Message-ID: > On 24 Nov 2014, at 16:36, Peter Levart wrote: > > On 11/24/2014 04:04 PM, Joel Borggr?n-Franck wrote: >> >> Btw, has anyone seen the assert for upper/lower bounds == null fail in the wild? >> >> private FieldTypeSignature[] getUpperBoundASTs() { >> // check that upper bounds were not evaluated yet >> assert(upperBounds == null); >> return upperBoundASTs; >> } >> >> shouldn?t it happen once in a while: >> >> public Type[] getUpperBounds() { >> // lazily initialize bounds if necessary >> if (upperBounds == null) { >> >> // thread gets preempted here, other thread completes init, upperBounds are != null >> >> FieldTypeSignature[] fts = getUpperBoundASTs(); // get AST >> >> >> Is the current code only working because most run without esa? > > ...or because races that would trigger this are very rare, since two consecutive "loads" of upperBounds happen one after the other, so the 2nd load and check might even be optimized away. > I was thinking along the lines of warming up in the interpreter + large search engine infrastructure scale. cheers /Joel From roger.riggs at oracle.com Mon Nov 24 20:46:42 2014 From: roger.riggs at oracle.com (roger riggs) Date: Mon, 24 Nov 2014 15:46:42 -0500 Subject: Initializing Secure Random (Reprise) Message-ID: <54739932.3070506@oracle.com> Hi, This topic has languished for a bit and could use a bit of expertise from Windows developers. The improvements in entropy for initializing Secure Random in JDK 8 have some negative attributes that affect maintainability, robustness and performance[1]. The dependency on networking, can in some OS's and configurations lead to increased startup times and issues with bootstrapping the Java runtime. Martin has proposed[2] an alternative for Linux based on /dev/urandom with a fallback to a simpler algorithm if /dev/urandom is not available. Since /dev/urandom is not native to Windows, it seems prudent to identify a corresponding source of entropy data. What are the recommended ways on Windows to get seeds for random? Please suggest one or more ways to initialize SecureRandom Thanks, Roger p.s. Sorry to be covering old ground but I don't have all the context. [1] 8060435 SecureRandom initialization latency on Windows https://bugs.openjdk.java.net/browse/JDK-8060435 [2] http://cr.openjdk.java.net/~martin/webrevs/openjdk9/ThreadLocalRandom-system-entropy/ From martinrb at google.com Mon Nov 24 20:56:15 2014 From: martinrb at google.com (Martin Buchholz) Date: Mon, 24 Nov 2014 12:56:15 -0800 Subject: RFR: 8065804: JEP 171: Clarifications/corrections for fence intrinsics Message-ID: Hi folks, Review carefully - I am trying to learn about fences by explaining them! I have borrowed some wording from my reviewers! https://bugs.openjdk.java.net/browse/JDK-8065804 http://cr.openjdk.java.net/~martin/webrevs/openjdk9/fence-intrinsics/ From Alan.Bateman at oracle.com Mon Nov 24 21:21:43 2014 From: Alan.Bateman at oracle.com (Alan Bateman) Date: Mon, 24 Nov 2014 21:21:43 +0000 Subject: Lower overhead String encoding/decoding In-Reply-To: <54737BC3.9010508@oracle.com> References: <54200F07.4070604@oracle.com> <542441BB.600@oracle.com> <5444BB17.8080708@oracle.com> <544FADA5.7000403@oracle.com> <544FB0F7.8080909@univ-mlv.fr> <54737BC3.9010508@oracle.com> Message-ID: <5473A167.5040703@oracle.com> On 24/11/2014 18:41, Xueming Shen wrote: > Hi Richard, > > Here are some comments regarding the updated webrev. > > (1) String(ByteBuffer, String) needs "@throw UEE". > (2) It should be "default replacement byte array" not "replace string" > for all > getBytes() methods when malformed/unmappable > (3) for decoding (new String) from ByteBuffer, since it is guaranteed > that all > bytes in the input ByteBuffer will be decoded, it might be > desirable to clearly > specify that the position of the buffer will be "advanced to its > limit" ? > (4) ln#1137 has an extra "*" > (5) StringCoding.decode(cs, bb), why do you want to allocate a direct > buffer > here for "untrusted"? Basically the output buffer "ca" will > always be a wrapper > of a char[], all our charset implementation will have better > performance if > both input and output are "array based" (decode on array > directly, instead of > the "slow" ByteBuffer) Overall I think this is looking quite good and I think Sherman has captured the main issues. On #3 then wording such as ".. the position will be updated" isn't clear enough to allow the method be tested, it needs to make it clear that the position is advanced by the number of bytes that were decoded. Sherman - are you going to sponsor this for Richard? -Alan From aleksey.shipilev at oracle.com Mon Nov 24 21:29:40 2014 From: aleksey.shipilev at oracle.com (Aleksey Shipilev) Date: Tue, 25 Nov 2014 00:29:40 +0300 Subject: RFR: 8065804: JEP 171: Clarifications/corrections for fence intrinsics In-Reply-To: References: Message-ID: <5473A344.8090302@oracle.com> Hi Martin, On 11/24/2014 11:56 PM, Martin Buchholz wrote: > Review carefully - I am trying to learn about fences by explaining them! > I have borrowed some wording from my reviewers! > > https://bugs.openjdk.java.net/browse/JDK-8065804 > http://cr.openjdk.java.net/~martin/webrevs/openjdk9/fence-intrinsics/ I think "implies the effect of C++11" is too strong wording. "related" might be more appropriate. See also comments here for connection with "volatiles": https://bugs.openjdk.java.net/browse/JDK-8038978 Take note the Hans' correction that fences generally imply more than volatile load/store, but since you are listing the related things in the docs, I think the "native" Java example is good to have. -Aleksey. From xueming.shen at oracle.com Mon Nov 24 21:42:37 2014 From: xueming.shen at oracle.com (Xueming Shen) Date: Mon, 24 Nov 2014 13:42:37 -0800 Subject: Lower overhead String encoding/decoding In-Reply-To: <5473A167.5040703@oracle.com> References: <54200F07.4070604@oracle.com> <542441BB.600@oracle.com> <5444BB17.8080708@oracle.com> <544FADA5.7000403@oracle.com> <544FB0F7.8080909@univ-mlv.fr> <54737BC3.9010508@oracle.com> <5473A167.5040703@oracle.com> Message-ID: <5473A64D.4070001@oracle.com> On 11/24/2014 01:21 PM, Alan Bateman wrote: > On 24/11/2014 18:41, Xueming Shen wrote: >> Hi Richard, >> >> Here are some comments regarding the updated webrev. >> >> (1) String(ByteBuffer, String) needs "@throw UEE". >> (2) It should be "default replacement byte array" not "replace string" for all >> getBytes() methods when malformed/unmappable >> (3) for decoding (new String) from ByteBuffer, since it is guaranteed that all >> bytes in the input ByteBuffer will be decoded, it might be desirable to clearly >> specify that the position of the buffer will be "advanced to its limit" ? >> (4) ln#1137 has an extra "*" >> (5) StringCoding.decode(cs, bb), why do you want to allocate a direct buffer >> here for "untrusted"? Basically the output buffer "ca" will always be a wrapper >> of a char[], all our charset implementation will have better performance if >> both input and output are "array based" (decode on array directly, instead of >> the "slow" ByteBuffer) > Overall I think this is looking quite good and I think Sherman has captured the main issues. On #3 then wording such as ".. the position will be updated" isn't clear enough to allow the method be tested, it needs to make it clear that the position is advanced by the number of bytes that were decoded. > > Sherman - are you going to sponsor this for Richard? > Yes, I will sponsor this RFE. -Sherman From ecki at zusammenkunft.net Mon Nov 24 22:36:01 2014 From: ecki at zusammenkunft.net (Bernd Eckenfels) Date: Mon, 24 Nov 2014 23:36:01 +0100 Subject: Initializing Secure Random (Reprise) In-Reply-To: <54739932.3070506@oracle.com> References: <54739932.3070506@oracle.com> Message-ID: <20141124233601.00000410.ecki@zusammenkunft.net> Hello, Crypto API (as used by NativeRandom on windows with SubMSCAPI) would be the right thing to use for a secure high entropy source (and this is actually what is used if you ask SecureRandom for seed bytes). But I guess this is not at all expected/needed for TLR. Having all platform launchers generate a random process startup uuid would be so usefull, not only be the seeder for the TLR. Maybe there should be some bootstrap code to do this once (and without too much subsystem dependencies). This could even uses stuff like libuuid on Linux or UuidCreate from rpcrt.dll on Windows. Mixing this "JVM instance UUID" with PID and Timestamp -> good enough TLR. BTW: It was probably not a good idea to make SecureRandom.getSeed() so easily accessible... if you think TLR needs a good random source, you should at least allow the SecureRandom instance to use urandom and SHA1 mixing. Both is present with getInstance().getBytes() (but its probably also not faster in the bug-specific test as the SecureRandom will be seeded once at this point). BTW2: I am not sure, it this seeded per thread? then using getSeed() is probably even worse. From martinrb at google.com Mon Nov 24 23:01:38 2014 From: martinrb at google.com (Martin Buchholz) Date: Mon, 24 Nov 2014 15:01:38 -0800 Subject: Initializing Secure Random (Reprise) In-Reply-To: <20141124233601.00000410.ecki@zusammenkunft.net> References: <54739932.3070506@oracle.com> <20141124233601.00000410.ecki@zusammenkunft.net> Message-ID: On Mon, Nov 24, 2014 at 2:36 PM, Bernd Eckenfels wrote: > Hello, > BTW2: I am not sure, it this seeded per thread? then using getSeed() > is probably even worse. No. ThreadLocalRandom and SplittableRandom each need 8 seed bytes per JVM startup. From kumar.x.srinivasan at oracle.com Mon Nov 24 23:46:14 2014 From: kumar.x.srinivasan at oracle.com (Kumar Srinivasan) Date: Mon, 24 Nov 2014 15:46:14 -0800 Subject: RFR 8060026: Update test/tools/launcher tests to eliminate dependency on sun.tools.jar.Main In-Reply-To: <5472FEA7.1060405@oracle.com> References: <5472FEA7.1060405@oracle.com> Message-ID: <5473C346.1010409@oracle.com> Looks good. Kumar On 11/24/2014 1:47 AM, Amy Lu wrote: > Some test/tools/launcher tests have dependency on sun.tools.jar.Main > The dependency actually comes from TestHelper.java > This fix is to remove above internal JDK API dependency from the tests. > > bug: https://bugs.openjdk.java.net/browse/JDK-8060026 > webrev: http://cr.openjdk.java.net/~weijun/8060026/webrev.00/ > > Thanks, > Amy From martinrb at google.com Tue Nov 25 00:47:19 2014 From: martinrb at google.com (Martin Buchholz) Date: Mon, 24 Nov 2014 16:47:19 -0800 Subject: RFR: 8065804: JEP 171: Clarifications/corrections for fence intrinsics In-Reply-To: <5473A344.8090302@oracle.com> References: <5473A344.8090302@oracle.com> Message-ID: OK, I worked in some wording for comparison with volatiles. I believe you when you say that the semantics of the corresponding C++ fences are slightly different, but it's rather subtle - can we say anything more than "closely related to"? On Mon, Nov 24, 2014 at 1:29 PM, Aleksey Shipilev wrote: > Hi Martin, > > On 11/24/2014 11:56 PM, Martin Buchholz wrote: >> Review carefully - I am trying to learn about fences by explaining them! >> I have borrowed some wording from my reviewers! >> >> https://bugs.openjdk.java.net/browse/JDK-8065804 >> http://cr.openjdk.java.net/~martin/webrevs/openjdk9/fence-intrinsics/ > > I think "implies the effect of C++11" is too strong wording. "related" > might be more appropriate. > > See also comments here for connection with "volatiles": > https://bugs.openjdk.java.net/browse/JDK-8038978 > > Take note the Hans' correction that fences generally imply more than > volatile load/store, but since you are listing the related things in the > docs, I think the "native" Java example is good to have. > > -Aleksey. > > From martinrb at google.com Tue Nov 25 02:06:43 2014 From: martinrb at google.com (Martin Buchholz) Date: Mon, 24 Nov 2014 18:06:43 -0800 Subject: RFR: 8065172: More core reflection final and volatile annotations In-Reply-To: References: <546E09D6.9040800@gmail.com> <6B93C453-C558-4C0E-B7FA-4255507B154D@oracle.com> <54735081.5050201@gmail.com> Message-ID: I tried to address all the known problems in sun/reflect (except for the performance ones), including the ones in Peter's webrev (although it now looks quite different). I broke down and switched to using AtomicReferenceFieldUpdaters for all lazily initialized operations, like I had been thinking of doing. For the weird classes where we need to lazily switch the implementation of bounds, I store in an Object[] and cas to update. Now that we're using atomic updaters, it's hard to have any relaxed operations unless we also introduce Unsafe. My current thinking is this code is not performance critical enough to do that sort of brittle thing. Volatile reads are either already very cheap or are likely to get cheaper over time. The code using updaters seems robust, efficient, and deadlock free, unlike some other code in the JDK. Using updaters means there will never be an extremely rare bug due to different objects being returned from a reflection method. http://cr.openjdk.java.net/~martin/webrevs/openjdk9/core-reflection-more-safety/ On Mon, Nov 24, 2014 at 12:25 PM, Joel Borggr?n-Franck wrote: > >> On 24 Nov 2014, at 16:36, Peter Levart wrote: >> >> On 11/24/2014 04:04 PM, Joel Borggr?n-Franck wrote: >>> >>> Btw, has anyone seen the assert for upper/lower bounds == null fail in the wild? >>> >>> private FieldTypeSignature[] getUpperBoundASTs() { >>> // check that upper bounds were not evaluated yet >>> assert(upperBounds == null); >>> return upperBoundASTs; >>> } >>> >>> shouldn?t it happen once in a while: >>> >>> public Type[] getUpperBounds() { >>> // lazily initialize bounds if necessary >>> if (upperBounds == null) { >>> >>> // thread gets preempted here, other thread completes init, upperBounds are != null >>> >>> FieldTypeSignature[] fts = getUpperBoundASTs(); // get AST >>> >>> >>> Is the current code only working because most run without esa? >> >> ...or because races that would trigger this are very rare, since two consecutive "loads" of upperBounds happen one after the other, so the 2nd load and check might even be optimized away. >> > > I was thinking along the lines of warming up in the interpreter + large search engine infrastructure scale. > > cheers > /Joel From stuart.marks at oracle.com Tue Nov 25 02:26:25 2014 From: stuart.marks at oracle.com (Stuart Marks) Date: Mon, 24 Nov 2014 18:26:25 -0800 Subject: RFR(s): 8035000: TEST_BUG: remove ActivationLibrary.DestroyThread and have callers call rmid.destroy() instead Message-ID: <5473E8D1.8040606@oracle.com> Hi all, Here's another test cleanup fix. Basically this normalizes the shutdown/destroy policy for rmid processes that are started by RMI's test library in support of a few dozen of the RMI activation tests. The previous implementation was, well, confused, and it had a redundant timing loop that can be subsumed by other code in RMI's test library (JavaVM.waitFor). I've listed this as a "small" changeset even though lots of files have changed. In all but three or so of the files, the only change is from calling "ActivationLibrary.rmidCleanup(rmid)" to calling "rmid.cleanup()" which is a refactoring enabled by the cleanup. The real action is in the files test/java/rmi/testlibrary/ActivationLibrary.java and RMID.java. Webrev: http://cr.openjdk.java.net/~smarks/reviews/8035000/webrev.0/ Bug: https://bugs.openjdk.java.net/browse/JDK-8035000 Thanks, s'marks From amy.lu at oracle.com Tue Nov 25 08:57:31 2014 From: amy.lu at oracle.com (Amy Lu) Date: Tue, 25 Nov 2014 16:57:31 +0800 Subject: RFR 8060026: Update test/tools/launcher tests to eliminate dependency on sun.tools.jar.Main In-Reply-To: <5473C346.1010409@oracle.com> References: <5472FEA7.1060405@oracle.com> <5473C346.1010409@oracle.com> Message-ID: <5474447B.6000008@oracle.com> Thank you for your review. I also need your help to push this change. Thanks, Amy On 11/25/14 7:46 AM, Kumar Srinivasan wrote: > Looks good. > > Kumar > > On 11/24/2014 1:47 AM, Amy Lu wrote: >> Some test/tools/launcher tests have dependency on sun.tools.jar.Main >> The dependency actually comes from TestHelper.java >> This fix is to remove above internal JDK API dependency from the tests. >> >> bug: https://bugs.openjdk.java.net/browse/JDK-8060026 >> webrev: http://cr.openjdk.java.net/~weijun/8060026/webrev.00/ >> >> Thanks, >> Amy > From paul.sandoz at oracle.com Tue Nov 25 14:04:28 2014 From: paul.sandoz at oracle.com (Paul Sandoz) Date: Tue, 25 Nov 2014 15:04:28 +0100 Subject: [concurrency-interest] RFR: 8065804: JEP 171: Clarifications/corrections for fence intrinsics In-Reply-To: References: <5473A344.8090302@oracle.com> Message-ID: Hi Martin, Thanks for looking into this. 1141 * Currently hotspot's implementation of a Java language-level volatile 1142 * store has the same effect as a storeFence followed by a relaxed store, 1143 * although that may be a little stronger than needed. IIUC to emulate hotpot's volatile store you will need to say that a fullFence immediately follows the relaxed store. The bit that always confuses me about release and acquire is ordering is restricted to one direction, as talked about in orderAccess.hpp [1]. So for a release, accesses prior to the release cannot move below it, but accesses succeeding the release can move above it. And that seems to apply to Unsafe.storeFence [2] (acting like a monitor exit). Is that contrary to C++ release fences where ordering is restricted both to prior and succeeding accesses? [3] So what about the following? a = r1; // Cannot move below the fence Unsafe.storeFence(); b = r2; // Can move above the fence? Paul. [1] In orderAccess.hpp // Execution by a processor of release makes the effect of all memory // accesses issued by it previous to the release visible to all // processors *before* the release completes. The effect of subsequent // memory accesses issued by it *may* be made visible *before* the // release. I.e., subsequent memory accesses may float above the // release, but prior ones may not float below it. [2] In memnode.hpp // "Release" - no earlier ref can move after (but later refs can move // up, like a speculative pipelined cache-hitting Load). Requires // multi-cpu visibility. Inserted independent of any store, as required // for intrinsic sun.misc.Unsafe.storeFence(). class StoreFenceNode: public MemBarNode { public: StoreFenceNode(Compile* C, int alias_idx, Node* precedent) : MemBarNode(C, alias_idx, precedent) {} virtual int Opcode() const; }; [3] http://preshing.com/20131125/acquire-and-release-fences-dont-work-the-way-youd-expect/ On Nov 25, 2014, at 1:47 AM, Martin Buchholz wrote: > OK, I worked in some wording for comparison with volatiles. > I believe you when you say that the semantics of the corresponding C++ > fences are slightly different, but it's rather subtle - can we say > anything more than "closely related to"? > > On Mon, Nov 24, 2014 at 1:29 PM, Aleksey Shipilev > wrote: >> Hi Martin, >> >> On 11/24/2014 11:56 PM, Martin Buchholz wrote: >>> Review carefully - I am trying to learn about fences by explaining them! >>> I have borrowed some wording from my reviewers! >>> >>> https://bugs.openjdk.java.net/browse/JDK-8065804 >>> http://cr.openjdk.java.net/~martin/webrevs/openjdk9/fence-intrinsics/ >> >> I think "implies the effect of C++11" is too strong wording. "related" >> might be more appropriate. >> >> See also comments here for connection with "volatiles": >> https://bugs.openjdk.java.net/browse/JDK-8038978 >> >> Take note the Hans' correction that fences generally imply more than >> volatile load/store, but since you are listing the related things in the >> docs, I think the "native" Java example is good to have. >> >> -Aleksey. >> >> > _______________________________________________ > Concurrency-interest mailing list > Concurrency-interest at cs.oswego.edu > http://cs.oswego.edu/mailman/listinfo/concurrency-interest From aph at redhat.com Tue Nov 25 21:41:46 2014 From: aph at redhat.com (Andrew Haley) Date: Tue, 25 Nov 2014 21:41:46 +0000 Subject: [concurrency-interest] RFR: 8065804: JEP 171: Clarifications/corrections for fence intrinsics In-Reply-To: References: Message-ID: <5474F79A.7000607@redhat.com> On 11/24/2014 08:56 PM, Martin Buchholz wrote: > Hi folks, > > Review carefully - I am trying to learn about fences by explaining them! > I have borrowed some wording from my reviewers! + * Currently hotspot's implementation of a Java language-level volatile + * store has the same effect as a storeFence followed by a relaxed store, + * although that may be a little stronger than needed. While this may be true today, I'm hopefully about to commit an AArch64 OpenJDK port that uses the ARMv8 stlr instruction. I don't think that what you've written here is terribly misleading, but bear in mind that it may be there for some time. Andrew. From peter.levart at gmail.com Tue Nov 25 23:15:11 2014 From: peter.levart at gmail.com (Peter Levart) Date: Wed, 26 Nov 2014 00:15:11 +0100 Subject: Initializing Secure Random (Reprise) In-Reply-To: <54739932.3070506@oracle.com> References: <54739932.3070506@oracle.com> Message-ID: <54750D7F.7040409@gmail.com> On 11/24/2014 09:46 PM, roger riggs wrote: > Hi, > > This topic has languished for a bit and could use a bit of expertise from > Windows developers. > > The improvements in entropy for initializing Secure Random in JDK 8 have > some negative attributes that affect maintainability, robustness and > performance[1]. > The dependency on networking, can in some OS's and configurations lead > to increased startup times and issues with bootstrapping the Java > runtime. > > Martin has proposed[2] an alternative for Linux based on /dev/urandom > with a fallback to a simpler algorithm if /dev/urandom is not available. > > Since /dev/urandom is not native to Windows, it seems prudent to identify > a corresponding source of entropy data. > > What are the recommended ways on Windows to get seeds for random? > Please suggest one or more ways to initialize SecureRandom Hi Roger, Do you mean SecureRandom or ThreadLocalRandom/SplittbleRandom, since the 8060435 talks about the later two ? As Bernd identified, the Windows equivalent to /dev/urandom is MSCAPI. 5 months ago I made an attempt to expose a part of contained internal java security API to get access to /dev/urandom based and MSCAPI based seed generators. Here's the thread with the discussion: http://mail.openjdk.java.net/pipermail/core-libs-dev/2014-June/027256.html The minimal patch was the following: http://cr.openjdk.java.net/~plevart/jdk9-dev/TLR_SR_SeedGenerator/webrev.01/ Regards, Peter > > Thanks, Roger > > p.s. Sorry to be covering old ground but I don't have all the context. > > > [1] 8060435 SecureRandom initialization latency on Windows > https://bugs.openjdk.java.net/browse/JDK-8060435 > > [2] > http://cr.openjdk.java.net/~martin/webrevs/openjdk9/ThreadLocalRandom-system-entropy/ From peter.levart at gmail.com Tue Nov 25 23:37:28 2014 From: peter.levart at gmail.com (Peter Levart) Date: Wed, 26 Nov 2014 00:37:28 +0100 Subject: Initializing Secure Random (Reprise) In-Reply-To: <54750D7F.7040409@gmail.com> References: <54739932.3070506@oracle.com> <54750D7F.7040409@gmail.com> Message-ID: <547512B8.1010701@gmail.com> Hi Roger, I see the 8060435 bug already talks about that possible solution too. I agree with Paul that a public static method on SecureRandom would be better for JDK9. Or maybe even some new class in java.util, which would implement just the interface to native /dev/urandom on Unixes and MSCAPI on Windows (without any SHA1 mixing or such) and make java security SeedGenerator depend on that rather than the other way around (where TLR/SplittableRandom depend on java security). I think this is better from modularization standpoint too. In case of a new java.util class (what about a method on java.lang.System?) the code (together with native part on Windows) could just be copied from (Native)SeedGenerator where it is already tried and tested. Then NativeSeedGenerator would just become a facade for invoking java.util/java.lang.System method. Regards, Peter On 11/26/2014 12:15 AM, Peter Levart wrote: > > On 11/24/2014 09:46 PM, roger riggs wrote: >> Hi, >> >> This topic has languished for a bit and could use a bit of expertise >> from >> Windows developers. >> >> The improvements in entropy for initializing Secure Random in JDK 8 have >> some negative attributes that affect maintainability, robustness and >> performance[1]. >> The dependency on networking, can in some OS's and configurations lead >> to increased startup times and issues with bootstrapping the Java >> runtime. >> >> Martin has proposed[2] an alternative for Linux based on /dev/urandom >> with a fallback to a simpler algorithm if /dev/urandom is not available. >> >> Since /dev/urandom is not native to Windows, it seems prudent to >> identify >> a corresponding source of entropy data. >> >> What are the recommended ways on Windows to get seeds for random? >> Please suggest one or more ways to initialize SecureRandom > > Hi Roger, > > Do you mean SecureRandom or ThreadLocalRandom/SplittbleRandom, since > the 8060435 talks about the later two ? > > As Bernd identified, the Windows equivalent to /dev/urandom is MSCAPI. > > 5 months ago I made an attempt to expose a part of contained internal > java security API to get access to /dev/urandom based and MSCAPI based > seed generators. Here's the thread with the discussion: > > http://mail.openjdk.java.net/pipermail/core-libs-dev/2014-June/027256.html > > The minimal patch was the following: > > http://cr.openjdk.java.net/~plevart/jdk9-dev/TLR_SR_SeedGenerator/webrev.01/ > > Regards, Peter > >> >> Thanks, Roger >> >> p.s. Sorry to be covering old ground but I don't have all the context. >> >> >> [1] 8060435 SecureRandom initialization latency on Windows >> https://bugs.openjdk.java.net/browse/JDK-8060435 >> >> [2] >> http://cr.openjdk.java.net/~martin/webrevs/openjdk9/ThreadLocalRandom-system-entropy/ > From davidcholmes at aapt.net.au Tue Nov 25 23:41:35 2014 From: davidcholmes at aapt.net.au (David Holmes) Date: Wed, 26 Nov 2014 09:41:35 +1000 Subject: [concurrency-interest] RFR: 8065804: JEP 171:Clarifications/corrections for fence intrinsics In-Reply-To: <2936699.i7HuM0iO2v@d-allen> Message-ID: Stephan Diestelhorst writes: > > Am Dienstag, 25. November 2014, 11:15:36 schrieb Hans Boehm: > > I'm no hardware architect, but fundamentally it seems to me that > > > > load x > > acquire_fence > > > > imposes a much more stringent constraint than > > > > load_acquire x > > > > Consider the case in which the load from x is an L1 hit, but a preceding > > load (from say y) is a long-latency miss. If we enforce > ordering by just > > waiting for completion of prior operation, the former has to > wait for the > > load from y to complete; while the latter doesn't. I find it hard to > > believe that this doesn't leave an appreciable amount of > performance on the > > table, at least for some interesting microarchitectures. > > I agree, Hans, that this is a reasonable assumption. Load_acquire x > does allow roach motel, whereas the acquire fence does not. > > > In addition, for better or worse, fencing requirements on at least > > Power are actually driven as much by store atomicity issues, as by > > the ordering issues discussed in the cookbook. This was not > > understood in 2005, and unfortunately doesn't seem to be amenable to > > the kind of straightforward explanation as in Doug's cookbook. > > Coming from a strongly ordered architecture to a weakly ordered one > myself, I also needed some mental adjustment about store (multi-copy) > atomicity. I can imagine others will be unaware of this difference, > too, even in 2014. Sorry I'm missing the connection between fences and multi-copy atomicity. David > Stephan > > _______________________________________________ > Concurrency-interest mailing list > Concurrency-interest at cs.oswego.edu > http://cs.oswego.edu/mailman/listinfo/concurrency-interest From davidcholmes at aapt.net.au Wed Nov 26 00:36:12 2014 From: davidcholmes at aapt.net.au (David Holmes) Date: Wed, 26 Nov 2014 10:36:12 +1000 Subject: [concurrency-interest] RFR: 8065804: JEP171:Clarifications/corrections for fence intrinsics In-Reply-To: <2458774.qkf0xEMZGz@d-allen> Message-ID: Stephan Diestelhorst writes: > > David Holmes wrote: > > Stephan Diestelhorst writes: > > > Am Dienstag, 25. November 2014, 11:15:36 schrieb Hans Boehm: > > > > I'm no hardware architect, but fundamentally it seems to me that > > > > > > > > load x > > > > acquire_fence > > > > > > > > imposes a much more stringent constraint than > > > > > > > > load_acquire x > > > > > > > > Consider the case in which the load from x is an L1 hit, but a > > > > preceding load (from say y) is a long-latency miss. If we enforce > > > > ordering by just waiting for completion of prior operation, the > > > > former has to wait for the load from y to complete; while the > > > > latter doesn't. I find it hard to believe that this doesn't leave > > > > an appreciable amount of performance on the table, at least for > > > > some interesting microarchitectures. > > > > > > I agree, Hans, that this is a reasonable assumption. Load_acquire x > > > does allow roach motel, whereas the acquire fence does not. > > > > > > > In addition, for better or worse, fencing requirements on at least > > > > Power are actually driven as much by store atomicity issues, as by > > > > the ordering issues discussed in the cookbook. This was not > > > > understood in 2005, and unfortunately doesn't seem to be > amenable to > > > > the kind of straightforward explanation as in Doug's cookbook. > > > > > > Coming from a strongly ordered architecture to a weakly ordered one > > > myself, I also needed some mental adjustment about store (multi-copy) > > > atomicity. I can imagine others will be unaware of this difference, > > > too, even in 2014. > > > > Sorry I'm missing the connection between fences and multi-copy > atomicity. > > One example is the classic IRIW. With non-multi copy atomic stores, but > ordered (say through a dependency) loads in the following example: > > Memory: foo = bar = 0 > _T1_ _T2_ _T3_ _T4_ > st (foo),1 st (bar),1 ld r1, (bar) ld r3,(foo) > > ld r2, (foo) ld r4, (bar) > > You may observe r1 = 1, r2 = 0, r3 = 1, r4 = 0 on non-multi-copy atomic > machines. On TSO boxes, this is not possible. That means that the > memory fence that will prevent such a behaviour (DMB on ARM) needs to > carry some additional oomph in ensuring multi-copy atomicity, or rather > prevent you from seeing it (which is the same thing). I take it as given that any code for which you may have ordering constraints, must first have basic atomicity properties for loads and stores. I would not expect any kind of fence to add multi-copy-atomicity where there was none. David > Stephan > > _______________________________________________ > Concurrency-interest mailing list > Concurrency-interest at cs.oswego.edu > http://cs.oswego.edu/mailman/listinfo/concurrency-interest From davidcholmes at aapt.net.au Wed Nov 26 02:10:58 2014 From: davidcholmes at aapt.net.au (David Holmes) Date: Wed, 26 Nov 2014 12:10:58 +1000 Subject: [concurrency-interest] RFR: 8065804: JEP171:Clarifications/corrections for fence intrinsics In-Reply-To: Message-ID: Hi Hans, Given IRIW is a thorn in everyone's side and has no known useful benefit, and can hopefully be killed off in the future, lets not get bogged down in IRIW. But none of what you say below relates to multi-copy-atomicity. Cheers, David -----Original Message----- From: hjkhboehm at gmail.com [mailto:hjkhboehm at gmail.com]On Behalf Of Hans Boehm Sent: Wednesday, 26 November 2014 12:04 PM To: dholmes at ieee.org Cc: Stephan Diestelhorst; concurrency-interest at cs.oswego.edu; core-libs-dev Subject: Re: [concurrency-interest] RFR: 8065804: JEP171:Clarifications/corrections for fence intrinsics To be concrete here, on Power, loads can normally be ordered by an address dependency or light-weight fence (lwsync). However, neither is enough to prevent the questionable outcome for IRIW, since it doesn't ensure that the stores in T1 and T2 will be made visible to other threads in a consistent order. That outcome can be prevented by using heavyweight fences (sync) instructions between the loads instead. Peter Sewell's group concluded that to enforce correct volatile behavior on Power, you essentially need a a heavyweight fence between every pair of volatile operations on Power. That cannot be understood based on simple ordering constraints. As Stephan pointed out, there are similar issues on ARM, but they're less commonly encountered in a Java implementation. If you're lucky, you can get to the right implementation recipe by looking at only reordering, I think. On Tue, Nov 25, 2014 at 4:36 PM, David Holmes wrote: Stephan Diestelhorst writes: > > David Holmes wrote: > > Stephan Diestelhorst writes: > > > Am Dienstag, 25. November 2014, 11:15:36 schrieb Hans Boehm: > > > > I'm no hardware architect, but fundamentally it seems to me that > > > > > > > > load x > > > > acquire_fence > > > > > > > > imposes a much more stringent constraint than > > > > > > > > load_acquire x > > > > > > > > Consider the case in which the load from x is an L1 hit, but a > > > > preceding load (from say y) is a long-latency miss. If we enforce > > > > ordering by just waiting for completion of prior operation, the > > > > former has to wait for the load from y to complete; while the > > > > latter doesn't. I find it hard to believe that this doesn't leave > > > > an appreciable amount of performance on the table, at least for > > > > some interesting microarchitectures. > > > > > > I agree, Hans, that this is a reasonable assumption. Load_acquire x > > > does allow roach motel, whereas the acquire fence does not. > > > > > > > In addition, for better or worse, fencing requirements on at least > > > > Power are actually driven as much by store atomicity issues, as by > > > > the ordering issues discussed in the cookbook. This was not > > > > understood in 2005, and unfortunately doesn't seem to be > amenable to > > > > the kind of straightforward explanation as in Doug's cookbook. > > > > > > Coming from a strongly ordered architecture to a weakly ordered one > > > myself, I also needed some mental adjustment about store (multi-copy) > > > atomicity. I can imagine others will be unaware of this difference, > > > too, even in 2014. > > > > Sorry I'm missing the connection between fences and multi-copy > atomicity. > > One example is the classic IRIW. With non-multi copy atomic stores, but > ordered (say through a dependency) loads in the following example: > > Memory: foo = bar = 0 > _T1_ _T2_ _T3_ _T4_ > st (foo),1 st (bar),1 ld r1, (bar) ld r3,(foo) > > ld r2, (foo) ld r4, (bar) > > You may observe r1 = 1, r2 = 0, r3 = 1, r4 = 0 on non-multi-copy atomic > machines. On TSO boxes, this is not possible. That means that the > memory fence that will prevent such a behaviour (DMB on ARM) needs to > carry some additional oomph in ensuring multi-copy atomicity, or rather > prevent you from seeing it (which is the same thing). I take it as given that any code for which you may have ordering constraints, must first have basic atomicity properties for loads and stores. I would not expect any kind of fence to add multi-copy-atomicity where there was none. David > Stephan > > _______________________________________________ > Concurrency-interest mailing list > Concurrency-interest at cs.oswego.edu > http://cs.oswego.edu/mailman/listinfo/concurrency-interest _______________________________________________ Concurrency-interest mailing list Concurrency-interest at cs.oswego.edu http://cs.oswego.edu/mailman/listinfo/concurrency-interest From davidcholmes at aapt.net.au Wed Nov 26 08:54:08 2014 From: davidcholmes at aapt.net.au (David Holmes) Date: Wed, 26 Nov 2014 18:54:08 +1000 Subject: [concurrency-interest] RFR: 8065804:JEP171:Clarifications/corrections for fence intrinsics In-Reply-To: Message-ID: Can you expand on that please. All previous discussion of IRIW I have seen indicated that the property, while a consequence of existing JMM rules, had no practical use. Thanks, David -----Original Message----- From: Roman Elizarov [mailto:elizarov at devexperts.com] Sent: Wednesday, 26 November 2014 6:49 PM To: dholmes at ieee.org; Hans Boehm Cc: concurrency-interest at cs.oswego.edu; core-libs-dev Subject: RE: [concurrency-interest] RFR: 8065804:JEP171:Clarifications/corrections for fence intrinsics There is no conceivable way to kill IRIW consistency requirement while retaining ability to prove correctness of large software systems. If IRIW of volatile variables are not consistent, then volatile reads and writes are not linearizable, which breaks linearizabiliy of all higher-level primitives build on top of them and makes formal reasoning about behavior of concurrent systems practically impossible. There are many fields where this is not acceptable. /Roman From: concurrency-interest-bounces at cs.oswego.edu [mailto:concurrency-interest-bounces at cs.oswego.edu] On Behalf Of David Holmes Sent: Wednesday, November 26, 2014 5:11 AM To: Hans Boehm Cc: concurrency-interest at cs.oswego.edu; core-libs-dev Subject: Re: [concurrency-interest] RFR: 8065804: JEP171:Clarifications/corrections for fence intrinsics Hi Hans, Given IRIW is a thorn in everyone's side and has no known useful benefit, and can hopefully be killed off in the future, lets not get bogged down in IRIW. But none of what you say below relates to multi-copy-atomicity. Cheers, David -----Original Message----- From: hjkhboehm at gmail.com [mailto:hjkhboehm at gmail.com]On Behalf Of Hans Boehm Sent: Wednesday, 26 November 2014 12:04 PM To: dholmes at ieee.org Cc: Stephan Diestelhorst; concurrency-interest at cs.oswego.edu; core-libs-dev Subject: Re: [concurrency-interest] RFR: 8065804: JEP171:Clarifications/corrections for fence intrinsics To be concrete here, on Power, loads can normally be ordered by an address dependency or light-weight fence (lwsync). However, neither is enough to prevent the questionable outcome for IRIW, since it doesn't ensure that the stores in T1 and T2 will be made visible to other threads in a consistent order. That outcome can be prevented by using heavyweight fences (sync) instructions between the loads instead. Peter Sewell's group concluded that to enforce correct volatile behavior on Power, you essentially need a a heavyweight fence between every pair of volatile operations on Power. That cannot be understood based on simple ordering constraints. As Stephan pointed out, there are similar issues on ARM, but they're less commonly encountered in a Java implementation. If you're lucky, you can get to the right implementation recipe by looking at only reordering, I think. On Tue, Nov 25, 2014 at 4:36 PM, David Holmes wrote: Stephan Diestelhorst writes: > > David Holmes wrote: > > Stephan Diestelhorst writes: > > > Am Dienstag, 25. November 2014, 11:15:36 schrieb Hans Boehm: > > > > I'm no hardware architect, but fundamentally it seems to me that > > > > > > > > load x > > > > acquire_fence > > > > > > > > imposes a much more stringent constraint than > > > > > > > > load_acquire x > > > > > > > > Consider the case in which the load from x is an L1 hit, but a > > > > preceding load (from say y) is a long-latency miss. If we enforce > > > > ordering by just waiting for completion of prior operation, the > > > > former has to wait for the load from y to complete; while the > > > > latter doesn't. I find it hard to believe that this doesn't leave > > > > an appreciable amount of performance on the table, at least for > > > > some interesting microarchitectures. > > > > > > I agree, Hans, that this is a reasonable assumption. Load_acquire x > > > does allow roach motel, whereas the acquire fence does not. > > > > > > > In addition, for better or worse, fencing requirements on at least > > > > Power are actually driven as much by store atomicity issues, as by > > > > the ordering issues discussed in the cookbook. This was not > > > > understood in 2005, and unfortunately doesn't seem to be > amenable to > > > > the kind of straightforward explanation as in Doug's cookbook. > > > > > > Coming from a strongly ordered architecture to a weakly ordered one > > > myself, I also needed some mental adjustment about store (multi-copy) > > > atomicity. I can imagine others will be unaware of this difference, > > > too, even in 2014. > > > > Sorry I'm missing the connection between fences and multi-copy > atomicity. > > One example is the classic IRIW. With non-multi copy atomic stores, but > ordered (say through a dependency) loads in the following example: > > Memory: foo = bar = 0 > _T1_ _T2_ _T3_ _T4_ > st (foo),1 st (bar),1 ld r1, (bar) ld r3,(foo) > > ld r2, (foo) ld r4, (bar) > > You may observe r1 = 1, r2 = 0, r3 = 1, r4 = 0 on non-multi-copy atomic > machines. On TSO boxes, this is not possible. That means that the > memory fence that will prevent such a behaviour (DMB on ARM) needs to > carry some additional oomph in ensuring multi-copy atomicity, or rather > prevent you from seeing it (which is the same thing). I take it as given that any code for which you may have ordering constraints, must first have basic atomicity properties for loads and stores. I would not expect any kind of fence to add multi-copy-atomicity where there was none. David > Stephan > > _______________________________________________ > Concurrency-interest mailing list > Concurrency-interest at cs.oswego.edu > http://cs.oswego.edu/mailman/listinfo/concurrency-interest _______________________________________________ Concurrency-interest mailing list Concurrency-interest at cs.oswego.edu http://cs.oswego.edu/mailman/listinfo/concurrency-interest From ivan.gerasimov at oracle.com Wed Nov 26 11:36:00 2014 From: ivan.gerasimov at oracle.com (Ivan Gerasimov) Date: Wed, 26 Nov 2014 14:36:00 +0300 Subject: [7u-dev] Request for Review + Request for Approval to Backport: 8064391: More thread safety problems in core reflection Message-ID: <5475BB20.4000506@oracle.com> Hello! Would you please approve the (almost) direct backport from jdk 8u to 7u? The difference, comparing to 8u changeset, is due to change in Class.java, (which is already there in 8) and in GenericDeclRepository.java due to genericification. The patch is essentially the same as the one proposed by Martin in the bug summary. Bug: https://bugs.openjdk.java.net/browse/JDK-8064391 Webrev: http://cr.openjdk.java.net/~igerasim/8064391/0/webrev/ Sincerely yours, Ivan From sean.coffey at oracle.com Wed Nov 26 11:59:56 2014 From: sean.coffey at oracle.com (=?UTF-8?B?U2XDoW4gQ29mZmV5?=) Date: Wed, 26 Nov 2014 11:59:56 +0000 Subject: [7u-dev] Request for Review + Request for Approval to Backport: 8064391: More thread safety problems in core reflection In-Reply-To: <5475BB20.4000506@oracle.com> References: <5475BB20.4000506@oracle.com> Message-ID: <5475C0BC.5050009@oracle.com> Thanks for handling this Ivan. Looks good to me. Hopefully you can get another reviewer. Approved. regards, Sean. On 26/11/14 11:36, Ivan Gerasimov wrote: > Hello! > > Would you please approve the (almost) direct backport from jdk 8u to 7u? > > The difference, comparing to 8u changeset, is due to change in > Class.java, (which is already there in 8) and in > GenericDeclRepository.java due to genericification. > > The patch is essentially the same as the one proposed by Martin in the > bug summary. > > Bug: https://bugs.openjdk.java.net/browse/JDK-8064391 > Webrev: http://cr.openjdk.java.net/~igerasim/8064391/0/webrev/ > > Sincerely yours, > Ivan > From kumar.x.srinivasan at oracle.com Wed Nov 26 13:30:35 2014 From: kumar.x.srinivasan at oracle.com (Kumar Srinivasan) Date: Wed, 26 Nov 2014 05:30:35 -0800 Subject: RFR 8060026: Update test/tools/launcher tests to eliminate dependency on sun.tools.jar.Main In-Reply-To: <5474447B.6000008@oracle.com> References: <5472FEA7.1060405@oracle.com> <5473C346.1010409@oracle.com> <5474447B.6000008@oracle.com> Message-ID: <5475D5FB.9070907@oracle.com> Ok I will get this pushed. Kumar On 11/25/2014 12:57 AM, Amy Lu wrote: > Thank you for your review. > > I also need your help to push this change. > > Thanks, > Amy > > On 11/25/14 7:46 AM, Kumar Srinivasan wrote: >> Looks good. >> >> Kumar >> >> On 11/24/2014 1:47 AM, Amy Lu wrote: >>> Some test/tools/launcher tests have dependency on sun.tools.jar.Main >>> The dependency actually comes from TestHelper.java >>> This fix is to remove above internal JDK API dependency from the tests. >>> >>> bug: https://bugs.openjdk.java.net/browse/JDK-8060026 >>> webrev: http://cr.openjdk.java.net/~weijun/8060026/webrev.00/ >>> >>> Thanks, >>> Amy >> > From roger.riggs at oracle.com Wed Nov 26 14:50:24 2014 From: roger.riggs at oracle.com (roger riggs) Date: Wed, 26 Nov 2014 09:50:24 -0500 Subject: Initializing Secure Random (Reprise) In-Reply-To: <547512B8.1010701@gmail.com> References: <54739932.3070506@oracle.com> <54750D7F.7040409@gmail.com> <547512B8.1010701@gmail.com> Message-ID: <5475E8B0.7030502@oracle.com> Hi Peter, Thanks for the update, I'm just getting up to speed on this and had help correcting the bug description and linking to the history from the earlier discussions in June. The static initializations of SecureRandom contribute to the large number of classes loaded even for the trivial program so it may be useful to delay them so the most trivial default use of SecureRandom does not load them immediately. Roger On 11/25/2014 6:37 PM, Peter Levart wrote: > Hi Roger, > > I see the 8060435 bug already talks about that possible solution too. > > I agree with Paul that a public static method on SecureRandom would be > better for JDK9. Or maybe even some new class in java.util, which > would implement just the interface to native /dev/urandom on Unixes > and MSCAPI on Windows (without any SHA1 mixing or such) and make java > security SeedGenerator depend on that rather than the other way around > (where TLR/SplittableRandom depend on java security). I think this is > better from modularization standpoint too. > > In case of a new java.util class (what about a method on > java.lang.System?) the code (together with native part on Windows) > could just be copied from (Native)SeedGenerator where it is already > tried and tested. Then NativeSeedGenerator would just become a facade > for invoking java.util/java.lang.System method. > > Regards, Peter > > On 11/26/2014 12:15 AM, Peter Levart wrote: >> >> On 11/24/2014 09:46 PM, roger riggs wrote: >>> Hi, >>> >>> This topic has languished for a bit and could use a bit of expertise >>> from >>> Windows developers. >>> >>> The improvements in entropy for initializing Secure Random in JDK 8 >>> have >>> some negative attributes that affect maintainability, robustness and >>> performance[1]. >>> The dependency on networking, can in some OS's and configurations lead >>> to increased startup times and issues with bootstrapping the Java >>> runtime. >>> >>> Martin has proposed[2] an alternative for Linux based on /dev/urandom >>> with a fallback to a simpler algorithm if /dev/urandom is not >>> available. >>> >>> Since /dev/urandom is not native to Windows, it seems prudent to >>> identify >>> a corresponding source of entropy data. >>> >>> What are the recommended ways on Windows to get seeds for random? >>> Please suggest one or more ways to initialize SecureRandom >> >> Hi Roger, >> >> Do you mean SecureRandom or ThreadLocalRandom/SplittbleRandom, since >> the 8060435 talks about the later two ? >> >> As Bernd identified, the Windows equivalent to /dev/urandom is MSCAPI. >> >> 5 months ago I made an attempt to expose a part of contained internal >> java security API to get access to /dev/urandom based and MSCAPI >> based seed generators. Here's the thread with the discussion: >> >> http://mail.openjdk.java.net/pipermail/core-libs-dev/2014-June/027256.html >> >> The minimal patch was the following: >> >> http://cr.openjdk.java.net/~plevart/jdk9-dev/TLR_SR_SeedGenerator/webrev.01/ >> >> Regards, Peter >> >>> >>> Thanks, Roger >>> >>> p.s. Sorry to be covering old ground but I don't have all the context. >>> >>> >>> [1] 8060435 SecureRandom initialization latency on Windows >>> https://bugs.openjdk.java.net/browse/JDK-8060435 >>> >>> [2] >>> http://cr.openjdk.java.net/~martin/webrevs/openjdk9/ThreadLocalRandom-system-entropy/ >> > From martinrb at google.com Wed Nov 26 15:41:38 2014 From: martinrb at google.com (Martin Buchholz) Date: Wed, 26 Nov 2014 07:41:38 -0800 Subject: [7u-dev] Request for Review + Request for Approval to Backport: 8064391: More thread safety problems in core reflection In-Reply-To: <5475BB20.4000506@oracle.com> References: <5475BB20.4000506@oracle.com> Message-ID: Approved! On Wed, Nov 26, 2014 at 3:36 AM, Ivan Gerasimov wrote: > Hello! > > Would you please approve the (almost) direct backport from jdk 8u to 7u? > > The difference, comparing to 8u changeset, is due to change in Class.java, > (which is already there in 8) and in GenericDeclRepository.java due to > genericification. > > The patch is essentially the same as the one proposed by Martin in the bug > summary. > > Bug: https://bugs.openjdk.java.net/browse/JDK-8064391 > Webrev: http://cr.openjdk.java.net/~igerasim/8064391/0/webrev/ > > Sincerely yours, > Ivan > From daniel.fuchs at oracle.com Wed Nov 26 17:11:21 2014 From: daniel.fuchs at oracle.com (Daniel Fuchs) Date: Wed, 26 Nov 2014 18:11:21 +0100 Subject: RFR: 8065991: LogManager unecessarily calls JavaAWTAccess from within a critical section Message-ID: <547609B9.3080206@oracle.com> Hi, Please find below a patch for: 8065991: LogManager unecessarily calls JavaAWTAccess from within a critical section https://bugs.openjdk.java.net/browse/JDK-8065991 Webrev: http://cr.openjdk.java.net/~dfuchs/webrev_8065991/webrev.00/ Note: https://bugs.openjdk.java.net/browse/JDK-8065709 shows a thread dump where this is causing a deadlock between AWT and LogManager. However - fixing this particular deadlock may require more than modifying LogManager. The fix proposed here should hopefully make JDK-8065709 less likely - but will not guarantee that the issue discribed in JDK-8065709 is fully fixed. I have included a testcase that replaces the implementation of JavaAWTAccess and emulates AppContext.getAppContext(). The test fails in deadlock without the change, and passes with it (though only because LogManager is pre-initilized in main()). best regards, -- daniel From jonathan.gibbons at oracle.com Wed Nov 26 22:35:27 2014 From: jonathan.gibbons at oracle.com (Jonathan Gibbons) Date: Wed, 26 Nov 2014 14:35:27 -0800 Subject: NIO question: Path.toFile() Message-ID: <547655AF.6020805@oracle.com> NIO folk, java.nio.file.Path.toFile() is specified to throw UnsupportedOperationException if this Path is not associated with the default provider. UOE seems a somewhat strange choice here. IllegalStateException or a custom exception might be seen as more obvious alternatives. (It's also somewhat weird that UOE is explicitly documented as being a member of the Java Collections Framework.) So, if there any background rationale of why UOE was chosen here? I'm not looking to change the spec for Path.toFile() in any way, but I am looking to design a higher level API that may encounter the same underlying problem, that a Path may not be associated with the default provider. At my higher level, I need to decide whether to go with UOE, just because that is what Path.toFile() does, or whether to convert it into some other exception. -- Jon From richard.warburton at gmail.com Thu Nov 27 00:01:59 2014 From: richard.warburton at gmail.com (Richard Warburton) Date: Thu, 27 Nov 2014 00:01:59 +0000 Subject: Lower overhead String encoding/decoding In-Reply-To: <5473A167.5040703@oracle.com> References: <54200F07.4070604@oracle.com> <542441BB.600@oracle.com> <5444BB17.8080708@oracle.com> <544FADA5.7000403@oracle.com> <544FB0F7.8080909@univ-mlv.fr> <54737BC3.9010508@oracle.com> <5473A167.5040703@oracle.com> Message-ID: Hi gents, Here are some comments regarding the updated webrev. >> >> (1) String(ByteBuffer, String) needs "@throw UEE". >> (2) It should be "default replacement byte array" not "replace string" >> for all >> getBytes() methods when malformed/unmappable >> (3) for decoding (new String) from ByteBuffer, since it is guaranteed >> that all >> bytes in the input ByteBuffer will be decoded, it might be desirable >> to clearly >> specify that the position of the buffer will be "advanced to its >> limit" ? >> (4) ln#1137 has an extra "*" >> (5) StringCoding.decode(cs, bb), why do you want to allocate a direct >> buffer >> here for "untrusted"? Basically the output buffer "ca" will always >> be a wrapper >> of a char[], all our charset implementation will have better >> performance if >> both input and output are "array based" (decode on array directly, >> instead of >> the "slow" ByteBuffer) >> > Overall I think this is looking quite good and I think Sherman has > captured the main issues. On #3 then wording such as ".. the position will > be updated" isn't clear enough to allow the method be tested, it needs to > make it clear that the position is advanced by the number of bytes that > were decoded. Thanks for looking at this patch and agreeing to sponsor - I've pushed fixes for these issues at: http://cr.openjdk.java.net/~rwarburton/string-patch-webrev-10/ regards, Richard Warburton http://insightfullogic.com @RichardWarburto From david.holmes at oracle.com Thu Nov 27 01:08:06 2014 From: david.holmes at oracle.com (David Holmes) Date: Thu, 27 Nov 2014 11:08:06 +1000 Subject: RFR: 8065804: JEP 171: Clarifications/corrections for fence intrinsics In-Reply-To: References: Message-ID: <54767976.70303@oracle.com> Martin, On 25/11/2014 6:56 AM, Martin Buchholz wrote: > Hi folks, > > Review carefully - I am trying to learn about fences by explaining them! > I have borrowed some wording from my reviewers! > > > https://bugs.openjdk.java.net/browse/JDK-8065804 > http://cr.openjdk.java.net/~martin/webrevs/openjdk9/fence-intrinsics/ Please explain why you have changed the defined semantics for storeFence. You have completely reversed the direction of the barrier. Thanks, David From martinrb at google.com Thu Nov 27 02:15:13 2014 From: martinrb at google.com (Martin Buchholz) Date: Wed, 26 Nov 2014 18:15:13 -0800 Subject: RFR: 8065804: JEP 171: Clarifications/corrections for fence intrinsics In-Reply-To: <54767976.70303@oracle.com> References: <54767976.70303@oracle.com> Message-ID: On Wed, Nov 26, 2014 at 5:08 PM, David Holmes wrote: > Please explain why you have changed the defined semantics for storeFence. > You have completely reversed the direction of the barrier. Yes. I believe the current spec of storeFence was a copy-paste typo, and it seems others feel likewise. From martinrb at google.com Thu Nov 27 02:21:40 2014 From: martinrb at google.com (Martin Buchholz) Date: Wed, 26 Nov 2014 18:21:40 -0800 Subject: [concurrency-interest] RFR: 8065804: JEP 171: Clarifications/corrections for fence intrinsics In-Reply-To: References: <5473A344.8090302@oracle.com> Message-ID: On Tue, Nov 25, 2014 at 6:04 AM, Paul Sandoz wrote: > Hi Martin, > > Thanks for looking into this. > > 1141 * Currently hotspot's implementation of a Java language-level volatile > 1142 * store has the same effect as a storeFence followed by a relaxed store, > 1143 * although that may be a little stronger than needed. > > IIUC to emulate hotpot's volatile store you will need to say that a fullFence immediately follows the relaxed store. Right - I've been groking that. > The bit that always confuses me about release and acquire is ordering is restricted to one direction, as talked about in orderAccess.hpp [1]. So for a release, accesses prior to the release cannot move below it, but accesses succeeding the release can move above it. And that seems to apply to Unsafe.storeFence [2] (acting like a monitor exit). Is that contrary to C++ release fences where ordering is restricted both to prior and succeeding accesses? [3] > > So what about the following? > > a = r1; // Cannot move below the fence > Unsafe.storeFence(); > b = r2; // Can move above the fence? I think the hotspot docs need to be more precise about when they're talking about movement of stores and when about loads. > // release. I.e., subsequent memory accesses may float above the > // release, but prior ones may not float below it. As I've said elsewhere, the above makes no sense without restricting the type of access. From martinrb at google.com Thu Nov 27 02:29:06 2014 From: martinrb at google.com (Martin Buchholz) Date: Wed, 26 Nov 2014 18:29:06 -0800 Subject: [concurrency-interest] RFR: 8065804: JEP 171: Clarifications/corrections for fence intrinsics In-Reply-To: <5474F79A.7000607@redhat.com> References: <5474F79A.7000607@redhat.com> Message-ID: On Tue, Nov 25, 2014 at 1:41 PM, Andrew Haley wrote: > On 11/24/2014 08:56 PM, Martin Buchholz wrote: > + * Currently hotspot's implementation of a Java language-level volatile > + * store has the same effect as a storeFence followed by a relaxed store, > + * although that may be a little stronger than needed. > > While this may be true today No - it was very wrong, since it doesn't give you sequential consistency! , I'm hopefully about to commit an > AArch64 OpenJDK port that uses the ARMv8 stlr instruction. I > don't think that what you've written here is terribly misleading, > but bear in mind that it may be there for some time. From davidcholmes at aapt.net.au Thu Nov 27 02:56:23 2014 From: davidcholmes at aapt.net.au (David Holmes) Date: Thu, 27 Nov 2014 12:56:23 +1000 Subject: [concurrency-interest] RFR: 8065804: JEP 171:Clarifications/corrections for fence intrinsics In-Reply-To: Message-ID: Martin Buchholz writes: > On Wed, Nov 26, 2014 at 5:08 PM, David Holmes > wrote: > > Please explain why you have changed the defined semantics for > storeFence. > > You have completely reversed the direction of the barrier. > > Yes. I believe the current spec of storeFence was a copy-paste typo, > and it seems others feel likewise. Can whomever wrote that original spec please confirm that. Thanks, David From davidcholmes at aapt.net.au Thu Nov 27 03:00:34 2014 From: davidcholmes at aapt.net.au (David Holmes) Date: Thu, 27 Nov 2014 13:00:34 +1000 Subject: [concurrency-interest] RFR: 8065804: JEP 171: Clarifications/corrections for fence intrinsics In-Reply-To: Message-ID: Can I make an observation about acquire() and release() - to me they are meaningless when considered in isolation. Given their definitions they allow anything to move into a region bounded by acquire() and release(), then you can effectively move the whole program into the region and thus the acquire() and release() do not constrain any reorderings. acquire() and release() only make sense when their own movement is constrained with respect to something else - such as lock acquisition/release, or when combined with specific load/store actions. David Martin Buchholz writes: > > On Tue, Nov 25, 2014 at 6:04 AM, Paul Sandoz > wrote: > > Hi Martin, > > > > Thanks for looking into this. > > > > 1141 * Currently hotspot's implementation of a Java > language-level volatile > > 1142 * store has the same effect as a storeFence followed > by a relaxed store, > > 1143 * although that may be a little stronger than needed. > > > > IIUC to emulate hotpot's volatile store you will need to say > that a fullFence immediately follows the relaxed store. > > Right - I've been groking that. > > > The bit that always confuses me about release and acquire is > ordering is restricted to one direction, as talked about in > orderAccess.hpp [1]. So for a release, accesses prior to the > release cannot move below it, but accesses succeeding the release > can move above it. And that seems to apply to Unsafe.storeFence > [2] (acting like a monitor exit). Is that contrary to C++ release > fences where ordering is restricted both to prior and succeeding > accesses? [3] > > > > So what about the following? > > > > a = r1; // Cannot move below the fence > > Unsafe.storeFence(); > > b = r2; // Can move above the fence? > > I think the hotspot docs need to be more precise about when they're > talking about movement of stores and when about loads. > > > // release. I.e., subsequent memory accesses may float above the > > // release, but prior ones may not float below it. > > As I've said elsewhere, the above makes no sense without restricting > the type of access. > > _______________________________________________ > Concurrency-interest mailing list > Concurrency-interest at cs.oswego.edu > http://cs.oswego.edu/mailman/listinfo/concurrency-interest From martinrb at google.com Thu Nov 27 03:06:13 2014 From: martinrb at google.com (Martin Buchholz) Date: Wed, 26 Nov 2014 19:06:13 -0800 Subject: [concurrency-interest] RFR: 8065804: JEP 171: Clarifications/corrections for fence intrinsics In-Reply-To: References: Message-ID: On Wed, Nov 26, 2014 at 7:00 PM, David Holmes wrote: > Can I make an observation about acquire() and release() - to me they are > meaningless when considered in isolation. Given their definitions they allow > anything to move into a region bounded by acquire() and release(), then you > can effectively move the whole program into the region and thus the > acquire() and release() do not constrain any reorderings. acquire() and > release() only make sense when their own movement is constrained with > respect to something else - such as lock acquisition/release, or when > combined with specific load/store actions. David, it seems you are agreeing with my argument below. The definitions in the hotspot sources should be fixed, in the same sort of way that I'm trying to make the specs for Unsafe loads clearer and more precise. > David > > Martin Buchholz writes: >> I think the hotspot docs need to be more precise about when they're >> talking about movement of stores and when about loads. >> >> > // release. I.e., subsequent memory accesses may float above the >> > // release, but prior ones may not float below it. >> >> As I've said elsewhere, the above makes no sense without restricting >> the type of access. From davidcholmes at aapt.net.au Thu Nov 27 03:27:18 2014 From: davidcholmes at aapt.net.au (David Holmes) Date: Thu, 27 Nov 2014 13:27:18 +1000 Subject: [concurrency-interest] RFR: 8065804: JEP 171: Clarifications/corrections for fence intrinsics In-Reply-To: Message-ID: Martin writes: > On Wed, Nov 26, 2014 at 7:00 PM, David Holmes > wrote: > > Can I make an observation about acquire() and release() - to me they are > > meaningless when considered in isolation. Given their > definitions they allow > > anything to move into a region bounded by acquire() and > release(), then you > > can effectively move the whole program into the region and thus the > > acquire() and release() do not constrain any reorderings. acquire() and > > release() only make sense when their own movement is constrained with > > respect to something else - such as lock acquisition/release, or when > > combined with specific load/store actions. > > David, it seems you are agreeing with my argument below. The > definitions in the hotspot sources should be fixed, in the same sort > of way that I'm trying to make the specs for Unsafe loads clearer and > more precise. Please see: https://bugs.openjdk.java.net/browse/JDK-7143664 Though I'm not sure my ramblings there reflect my current thoughts on all this. I really think acquire/release are too confusingly used to be useful - by which I mean that the names do not reflect their actions so you will always have to remember/look-up exactly what *release* and *acquire* mean in that context, and hence talking about "acquire semantics" and "release semantics" becomes meaningless. In contrast the loadload|loadstore etc barriers are completely straight-forward to understand from their names. However it seems they are too strong compared to what recent hardware provides. Hotspot implementations in orderAccess are confusing - barriers with different semantics have been defined in terms of the other, but the low-level implementations provide a barrier that is stronger than the required semantics, so the high-level APIs are "satisfied" correctly, even if not implemented in a way that makes sense if you reason about what each barrier theoretically allows. David > > David > > > > Martin Buchholz writes: > >> I think the hotspot docs need to be more precise about when they're > >> talking about movement of stores and when about loads. > >> > >> > // release. I.e., subsequent memory accesses may float above the > >> > // release, but prior ones may not float below it. > >> > >> As I've said elsewhere, the above makes no sense without restricting > >> the type of access. > From ivan.gerasimov at oracle.com Thu Nov 27 06:59:54 2014 From: ivan.gerasimov at oracle.com (Ivan Gerasimov) Date: Thu, 27 Nov 2014 09:59:54 +0300 Subject: RFR: 8062771: Core reflection should use final fields whenever possible In-Reply-To: <20141110090015.GA22573@oracle.com> References: <843282E1-DC2B-48B0-B0E4-A60B5078D75A@oracle.com> <45A50B54-43A3-4134-9EB5-D1AEDA649587@oracle.com> <545E9293.5010203@oracle.com> <20141110090015.GA22573@oracle.com> Message-ID: <5476CBEA.3060503@oracle.com> Hi! I can take care of jdk8 backport too, if you want. I've checked it, and the port seemed quite straight-forward (modulo unshuffling). Only changes to a few files [1] had to be skipped, as the changes appear to already be there in jdk8u. Here's the webrev of jdk8u backport for JDK-8062771: WEBREV: http://cr.openjdk.java.net/~igerasim/8062771/0/webrev/ [1] And here's the list of the files that needed to be excluded from the changeset: src/share/classes/sun/reflect/generics/factory/CoreReflectionFactory.java src/share/classes/sun/reflect/generics/reflectiveObjects/LazyReflectiveObjectGenerator.java src/share/classes/sun/reflect/generics/repository/AbstractRepository.java src/share/classes/sun/reflect/generics/scope/AbstractScope.java src/share/classes/sun/reflect/generics/tree/ClassSignature.java Sincerely yours, Ivan On 10.11.2014 12:00, Joel Borggr?n-Franck wrote: > That would be great > > cheers > /Joel > > On 2014-11-09, Ivan Gerasimov wrote: >> Hi! >> >> I think I can handle the backport to jdk7, if you still need a volunteer. >> >> Sincerely yours, >> Ivan >> >> On 08.11.2014 1:48, Martin Buchholz wrote: >>> Hi Joel, >>> >>> Thanks for volunteering. I foisted all I have in >>> >>> https://bugs.openjdk.java.net/browse/JDK-8064391 >>> >>> I volunteer to be your reviewer for the backports. >>> >>> On Fri, Nov 7, 2014 at 1:36 PM, Joel Borggr?n-Franck >>> wrote: >>>> Hi Martin, >>>> >>>> Thanks for the clarification. >>>> >>>> On 6 nov 2014, at 20:51, Martin Buchholz wrote: >>>> >>>>> Hi Joel, >>>>> >>>>> On Thu, Nov 6, 2014 at 2:48 AM, Joel Borggr?n-Franck >>>>> wrote: >>>>>> Hi, >>>>>> >>>>>> I?m having a hard time following this thread, which webrev has been updated, for which release, fixing which issue? >>>>> I am submitting the changeset for JDK-8062771, adding the finals and >>>>> the (failed reproduction) test. >>>>> Hopefully there will be followon changes to add even more thread safety. >>>>> >>>> Great. >>>> >>>> Are we sure the accidental fix of making ClassRepository volatile is good on all platforms? I can?t establish the happens before relations that would prove the fix but then again I?m not an expert on concurrency. >>>> >>>>>> Martin, as far as I can see you are the only one of us who has replied to this thread who is a jdk7u committer (or reviewer). >>>>> I am entirely in favor of having more reviewers. >>>>> >>>>> We are not planning to push these into jdk7u or jdk8u ourselves, but >>>>> we are willing to help anyone who wants to take on that task. Joel, >>>>> are you volunteering? >>>>> We believe all versions of stock openjdk still have (rarely seen) >>>>> thread safety issues in core reflection. We have applied changes >>>>> locally at Google to fix those. >>>>> >>>> I?ll take care of 8u, and I can hunt down someone to fix this in 7u if we can find a suitable reviewer (sigh). >>>> >>>> cheers >>>> /Joel >>>> >>>> > From peter.levart at gmail.com Thu Nov 27 07:48:33 2014 From: peter.levart at gmail.com (Peter Levart) Date: Thu, 27 Nov 2014 08:48:33 +0100 Subject: [concurrency-interest] RFR: 8065804: JEP 171: Clarifications/corrections for fence intrinsics In-Reply-To: References: Message-ID: <5476D751.5040407@gmail.com> On 11/27/2014 04:00 AM, David Holmes wrote: > Can I make an observation about acquire() and release() - to me they are > meaningless when considered in isolation. Given their definitions they allow > anything to move into a region bounded by acquire() and release(), then you > can effectively move the whole program into the region and thus the > acquire() and release() do not constrain any reorderings. > acquire() and > release() only make sense when their own movement is constrained with > respect to something else - such as lock acquisition/release, or when > combined with specific load/store actions. ...or another acquire/release region? Regards, Peter > > David > > Martin Buchholz writes: >> On Tue, Nov 25, 2014 at 6:04 AM, Paul Sandoz >> wrote: >>> Hi Martin, >>> >>> Thanks for looking into this. >>> >>> 1141 * Currently hotspot's implementation of a Java >> language-level volatile >>> 1142 * store has the same effect as a storeFence followed >> by a relaxed store, >>> 1143 * although that may be a little stronger than needed. >>> >>> IIUC to emulate hotpot's volatile store you will need to say >> that a fullFence immediately follows the relaxed store. >> >> Right - I've been groking that. >> >>> The bit that always confuses me about release and acquire is >> ordering is restricted to one direction, as talked about in >> orderAccess.hpp [1]. So for a release, accesses prior to the >> release cannot move below it, but accesses succeeding the release >> can move above it. And that seems to apply to Unsafe.storeFence >> [2] (acting like a monitor exit). Is that contrary to C++ release >> fences where ordering is restricted both to prior and succeeding >> accesses? [3] >>> So what about the following? >>> >>> a = r1; // Cannot move below the fence >>> Unsafe.storeFence(); >>> b = r2; // Can move above the fence? >> I think the hotspot docs need to be more precise about when they're >> talking about movement of stores and when about loads. >> >>> // release. I.e., subsequent memory accesses may float above the >>> // release, but prior ones may not float below it. >> As I've said elsewhere, the above makes no sense without restricting >> the type of access. >> >> _______________________________________________ >> Concurrency-interest mailing list >> Concurrency-interest at cs.oswego.edu >> http://cs.oswego.edu/mailman/listinfo/concurrency-interest From Alan.Bateman at oracle.com Thu Nov 27 08:05:53 2014 From: Alan.Bateman at oracle.com (Alan Bateman) Date: Thu, 27 Nov 2014 08:05:53 +0000 Subject: NIO question: Path.toFile() In-Reply-To: <547655AF.6020805@oracle.com> References: <547655AF.6020805@oracle.com> Message-ID: <5476DB61.8070307@oracle.com> On 26/11/2014 22:35, Jonathan Gibbons wrote: > NIO folk, > > java.nio.file.Path.toFile() is specified to throw > UnsupportedOperationException if this Path is not associated with the > default provider. > > UOE seems a somewhat strange choice here. IllegalStateException or a > custom exception might be seen as more obvious alternatives. (It's > also somewhat weird that UOE is explicitly documented as being a > member of the Java Collections Framework.) > > So, if there any background rationale of why UOE was chosen here? I'm > not looking to change the spec for Path.toFile() in any way, but I am > looking to design a higher level API that may encounter the same > underlying problem, that a Path may not be associated with the default > provider. At my higher level, I need to decide whether to go with > UOE, just because that is what Path.toFile() does, or whether to > convert it into some other exception. I think UOE is right because it must fail for every instance of an implementation type that is not associated with the default provider. I don't think IllegalStateException would have worked there aren't any states where this method could succeed. A blog that I've re-read a few things is Kevin Bourrillion's take on unchecked exceptions: http://smallwig.blogspot.co.uk/2008/06/common-java-unchecked-exception-types.html It's probably an area where more guidelines need to be written down, also probably a topic where it's impossible to get broad agreement. -Alan. From ecki at zusammenkunft.net Thu Nov 27 09:18:06 2014 From: ecki at zusammenkunft.net (Bernd) Date: Thu, 27 Nov 2014 10:18:06 +0100 Subject: NIO question: Path.toFile() In-Reply-To: <5476DB61.8070307@oracle.com> References: <547655AF.6020805@oracle.com> <5476DB61.8070307@oracle.com> Message-ID: Hello, I think if you do not think in terms of "is it the default provider" but in "does it allow to convert to file" the exception is better understood (and more generically usefull). However for a specified API feedback its rather strange (and I would agree with IllegalState, since the state in this case is which provider was associated with it). Greetings Bernd Am 27.11.2014 10:06 schrieb "Alan Bateman" : > On 26/11/2014 22:35, Jonathan Gibbons wrote: > >> NIO folk, >> >> java.nio.file.Path.toFile() is specified to throw >> UnsupportedOperationException if this Path is not associated with the >> default provider. >> >> UOE seems a somewhat strange choice here. IllegalStateException or a >> custom exception might be seen as more obvious alternatives. (It's also >> somewhat weird that UOE is explicitly documented as being a member of the >> Java Collections Framework.) >> >> So, if there any background rationale of why UOE was chosen here? I'm not >> looking to change the spec for Path.toFile() in any way, but I am looking >> to design a higher level API that may encounter the same underlying >> problem, that a Path may not be associated with the default provider. At >> my higher level, I need to decide whether to go with UOE, just because that >> is what Path.toFile() does, or whether to convert it into some other >> exception. >> > > I think UOE is right because it must fail for every instance of an > implementation type that is not associated with the default provider. I > don't think IllegalStateException would have worked there aren't any states > where this method could succeed. > > A blog that I've re-read a few things is Kevin Bourrillion's take on > unchecked exceptions: > http://smallwig.blogspot.co.uk/2008/06/common-java- > unchecked-exception-types.html > > It's probably an area where more guidelines need to be written down, also > probably a topic where it's impossible to get broad agreement. > > -Alan. > > > From patrick at reini.net Thu Nov 27 09:40:32 2014 From: patrick at reini.net (Patrick Reinhart) Date: Thu, 27 Nov 2014 10:40:32 +0100 Subject: Library enhancement proposal for copying data from/to Reader/Writer InputStream/OutputStream In-Reply-To: References: <566A428C-1A13-4DEE-940B-2EBE375F2A7A@reini.net> <51329A99-55E5-4A42-9180-D872366B4DEE@oracle.com> <546EABED.9030301@oracle.com> <546EF98C.40704@oracle.com> <1D392C1F-E6CC-494D-97F8-C08C5BA36392@oracle.com> Message-ID: Hi Paul, Attached a patch containing my complete proposed code including tests. Patrick > Am 24.11.2014 um 18:03 schrieb Patrick Reinhart : > > Well I will then suggest using IOUtils with two simple methods: > > public static long copy(InputStream source, OutputStream target) > > public static long copy(Readable source, Appendable target) > > >>>> >>>> To speed things along for now i recommend creating a new class say IOUtils or preferably ByteStreams. >>> >>> Hmm, in the case of Reader/Writer ByteStreams seams to be a bit confusing for me, should then the reader copy stuff go to also a separate class like ?CharStreams?. Locally I created an implementation named ?IOUtils? containing a copy method for InputStream to OutputStream. >>> >> >> Pick the one you prefer so we don't rat hole down the naming abyss :-) >> >> Pavel, your point about the overload with j.u.stream.Stream is a reasonable one (I am not too concerned by it myself). I hope one day in the future we can have Stream :-) >> >> Paul. > > Patrick From patrick at reini.net Thu Nov 27 09:48:21 2014 From: patrick at reini.net (Patrick Reinhart) Date: Thu, 27 Nov 2014 10:48:21 +0100 Subject: Library enhancement proposal for copying data from/to Reader/Writer InputStream/OutputStream In-Reply-To: References: <566A428C-1A13-4DEE-940B-2EBE375F2A7A@reini.net> <51329A99-55E5-4A42-9180-D872366B4DEE@oracle.com> <546EABED.9030301@oracle.com> <546EF98C.40704@oracle.com> <1D392C1F-E6CC-494D-97F8-C08C5BA36392@oracle.com> Message-ID: Hi Paul, Seems not to work. See inline patch? Patrick > Hi Paul, > > Attached a patch containing my complete proposed code including tests. > > Patrick --- /dev/null 2014-11-24 08:00:19.282694686 +0100 +++ new/src/java.base/share/classes/java/io/IOUtil.java 2014-11-27 10:33:39.955791833 +0100 @@ -0,0 +1,86 @@ +/* + * Copyright (c) 1996, 2014, Oracle and/or its affiliates. All rights reserved. + * DO NOT ALTER OR REMOVE COPYRIGHT NOTICES OR THIS FILE HEADER. + * + * This code is free software; you can redistribute it and/or modify it + * under the terms of the GNU General Public License version 2 only, as + * published by the Free Software Foundation. Oracle designates this + * particular file as subject to the "Classpath" exception as provided + * by Oracle in the LICENSE file that accompanied this code. + * + * This code is distributed in the hope that it will be useful, but WITHOUT + * ANY WARRANTY; without even the implied warranty of MERCHANTABILITY or + * FITNESS FOR A PARTICULAR PURPOSE. See the GNU General Public License + * version 2 for more details (a copy is included in the LICENSE file that + * accompanied this code). + * + * You should have received a copy of the GNU General Public License version + * 2 along with this work; if not, write to the Free Software Foundation, + * Inc., 51 Franklin St, Fifth Floor, Boston, MA 02110-1301 USA. + * + * Please contact Oracle, 500 Oracle Parkway, Redwood Shores, CA 94065 USA + * or visit www.oracle.com if you need additional information or have any + * questions. + */ + +package java.io; + +import java.nio.CharBuffer; + +/** + * Utility methods for copying data between a {@link InputStream} and + * {@link OutputStream} or between a {@link Readable} and a {@link Appendable}. + */ +public final class IOUtil { + + // buffer size used for reading and writing + private static final int BUFFER_SIZE = 8192; + + private IOUtil() { + throw new Error("no instances"); + } + + /** + * Reads all bytes from an input stream and writes them to an output stream. + * + * @param source the input stream to read from + * @param target the output stream to write to + * + * @return the number of bytes successfully read and written + * + * @throws IOException if an I/O error occurs when reading or writing + */ + public static long copy(InputStream source, OutputStream target) + throws IOException { + long totalRead = 0L; + byte[] buffer = new byte[BUFFER_SIZE]; + int read; + while ((read = source.read(buffer)) > -1) { + target.write(buffer, 0, read); + totalRead += read; + } + return totalRead; + } + + /** + * Reads all characters from an readable and writes them to an appendable. + * + * @param source the readable to read from + * @param target the appendable to write to + * + * @return the number of characters successfully read and written + * + * @throws IOException if an I/O error occurs when reading or writing + */ + public static long copy(Readable source, Appendable target) throws IOException { + long totalRead = 0L; + CharBuffer buffer = CharBuffer.allocate(BUFFER_SIZE); + int read; + while ((read = source.read(buffer)) > -1) { + buffer.flip(); + target.append(buffer, 0, read); + totalRead += read; + } + return totalRead; + } +} --- /dev/null 2014-11-24 08:00:19.282694686 +0100 +++ new/test/java/io/IOUtil/CopyInputStreamOutputStream.java 2014-11-27 10:33:40.352791912 +0100 @@ -0,0 +1,53 @@ +/* + * Copyright (c) 1998, 2014, Oracle and/or its affiliates. All rights reserved. + * DO NOT ALTER OR REMOVE COPYRIGHT NOTICES OR THIS FILE HEADER. + * + * This code is free software; you can redistribute it and/or modify it + * under the terms of the GNU General Public License version 2 only, as + * published by the Free Software Foundation. + * + * This code is distributed in the hope that it will be useful, but WITHOUT + * ANY WARRANTY; without even the implied warranty of MERCHANTABILITY or + * FITNESS FOR A PARTICULAR PURPOSE. See the GNU General Public License + * version 2 for more details (a copy is included in the LICENSE file that + * accompanied this code). + * + * You should have received a copy of the GNU General Public License version + * 2 along with this work; if not, write to the Free Software Foundation, + * Inc., 51 Franklin St, Fifth Floor, Boston, MA 02110-1301 USA. + * + * Please contact Oracle, 500 Oracle Parkway, Redwood Shores, CA 94065 USA + * or visit www.oracle.com if you need additional information or have any + * questions. + */ + +import java.io.ByteArrayInputStream; +import java.io.ByteArrayOutputStream; +import java.io.IOUtil; +import java.util.Arrays; + +/* @test + @summary Test copy a InputStream to a OutputStream reporting the amount + of bytes copied in total. + */ +public class CopyInputStreamOutputStream { + + public static void main(String[] args) throws Exception { + new CopyInputStreamOutputStream().go(); + } + + + private void go() throws Exception { + byte[] data = LoremIpsum.getBytes(); + ByteArrayInputStream in = new ByteArrayInputStream(data); + ByteArrayOutputStream out = new ByteArrayOutputStream(); + + if (IOUtil.copy(in, out) != data.length) { + throw new Exception("expected amount of copied bytes wrong"); + } + + if (!Arrays.equals(data, out.toByteArray())) { + throw new Exception("copied data wrong"); + } + } +} --- /dev/null 2014-11-24 08:00:19.282694686 +0100 +++ new/test/java/io/IOUtil/CopyReadableAppendable.java 2014-11-27 10:33:40.722791986 +0100 @@ -0,0 +1,51 @@ +/* + * Copyright (c) 1998, 2014, Oracle and/or its affiliates. All rights reserved. + * DO NOT ALTER OR REMOVE COPYRIGHT NOTICES OR THIS FILE HEADER. + * + * This code is free software; you can redistribute it and/or modify it + * under the terms of the GNU General Public License version 2 only, as + * published by the Free Software Foundation. + * + * This code is distributed in the hope that it will be useful, but WITHOUT + * ANY WARRANTY; without even the implied warranty of MERCHANTABILITY or + * FITNESS FOR A PARTICULAR PURPOSE. See the GNU General Public License + * version 2 for more details (a copy is included in the LICENSE file that + * accompanied this code). + * + * You should have received a copy of the GNU General Public License version + * 2 along with this work; if not, write to the Free Software Foundation, + * Inc., 51 Franklin St, Fifth Floor, Boston, MA 02110-1301 USA. + * + * Please contact Oracle, 500 Oracle Parkway, Redwood Shores, CA 94065 USA + * or visit www.oracle.com if you need additional information or have any + * questions. + */ + +import java.io.IOUtil; +import java.io.StringReader; + +/* @test + @summary Test copy a Readable to a Appendable reporting the amount + of characters copied in total. + */ +public class CopyReadableAppendable { + + public static void main(String[] args) throws Exception { + new CopyReadableAppendable().go(); + } + + + private void go() throws Exception { + String data = LoremIpsum.getString(); + StringReader in = new StringReader(data); + StringBuilder out = new StringBuilder(); + + if (IOUtil.copy(in, out) != data.length()) { + throw new Exception("expected amount of copied characters wrong"); + } + + if (!data.equals(out.toString())) { + throw new Exception("copied data wrong"); + } + } +} --- /dev/null 2014-11-24 08:00:19.282694686 +0100 +++ new/test/java/io/IOUtil/LoremIpsum.java 2014-11-27 10:33:41.070792056 +0100 @@ -0,0 +1,219 @@ +/* + * Copyright (c) 1998, 2014, Oracle and/or its affiliates. All rights reserved. + * DO NOT ALTER OR REMOVE COPYRIGHT NOTICES OR THIS FILE HEADER. + * + * This code is free software; you can redistribute it and/or modify it + * under the terms of the GNU General Public License version 2 only, as + * published by the Free Software Foundation. + * + * This code is distributed in the hope that it will be useful, but WITHOUT + * ANY WARRANTY; without even the implied warranty of MERCHANTABILITY or + * FITNESS FOR A PARTICULAR PURPOSE. See the GNU General Public License + * version 2 for more details (a copy is included in the LICENSE file that + * accompanied this code). + * + * You should have received a copy of the GNU General Public License version + * 2 along with this work; if not, write to the Free Software Foundation, + * Inc., 51 Franklin St, Fifth Floor, Boston, MA 02110-1301 USA. + * + * Please contact Oracle, 500 Oracle Parkway, Redwood Shores, CA 94065 USA + * or visit www.oracle.com if you need additional information or have any + * questions. + */ + +public class LoremIpsum { + private static String LOREM_IPSUM = "Lorem ipsum dolor sit amet, " + + "consectetur adipiscing elit. Phasellus semper mi at ex " + + "aliquet, vitae dapibus tortor egestas. Duis convallis " + + "nunc non lorem faucibus, non tincidunt lacus elementum. " + + "Suspendisse suscipit porttitor enim, ac volutpat felis " + + "efficitur nec. Donec lacinia quam id dictum blandit. Sed " + + "urna odio, commodo laoreet convallis non, molestie vel " + + "tortor. Pellentesque maximus accumsan lectus nec aliquet. " + + "Nullam ut vestibulum elit. Integer laoreet leo in velit " + + "lobortis, non hendrerit est mollis. Etiam sed mauris et " + + "lacus viverra posuere eget sed mi. Nulla turpis dui, semper " + + "sit amet vehicula vitae, scelerisque imperdiet nisi. Morbi " + + "ac porttitor quam. Nullam id diam lectus. In erat mi, " + + "imperdiet in elit at, ultricies sollicitudin leo. Maecenas " + + "sit amet eros placerat, posuere purus vel, semper mi. " + + "Vivamus ornare viverra velit vel congue." + + "\n\n" + + "Nam sapien enim, ullamcorper non urna a, lacinia placerat " + + "libero. Donec gravida metus eget justo consequat pretium. " + + "Mauris venenatis, massa non ultricies suscipit, tortor " + + "tellus pharetra massa, at luctus tortor quam tincidunt " + + "mauris. Maecenas fermentum, ex eget volutpat sagittis, " + + "nibh risus molestie mauris, quis mattis ex lacus nec erat. " + + "Vestibulum vel iaculis tortor, eget tempor lorem. " + + "Suspendisse sodales lacinia ligula nec commodo. Morbi " + + "elementum ultricies arcu, sed sagittis mi sollicitudin sed. " + + "Curabitur hendrerit ante vitae nunc convallis, a feugiat " + + "nisi pulvinar." + + "\n\n" + + "Nam sollicitudin ultricies elit at tincidunt. Vestibulum " + + "ut elementum ipsum, sit amet tincidunt arcu. Interdum et " + + "malesuada fames ac ante ipsum primis in faucibus. Mauris " + + "ut tempor ipsum. Suspendisse vehicula justo sit amet ultricies " + + "gravida. Fusce pharetra lorem eu elit fringilla, tincidunt " + + "gravida velit malesuada. Suspendisse euismod odio ligula, et " + + "pharetra arcu feugiat id. Pellentesque dignissim quam eu " + + "urna hendrerit lobortis. Maecenas vulputate fringilla augue, " + + "et sollicitudin urna molestie ac. Maecenas eget sollicitudin " + + "mi. Aenean justo arcu, condimentum et nisl id, faucibus " + + "pulvinar orci. Praesent in tristique eros, non consequat nisl. " + + "In cursus orci et pellentesque facilisis. Vivamus scelerisque " + + "nisl sit amet velit accumsan dictum. Proin vestibulum nulla " + + "vel elit molestie, quis molestie sapien porttitor. Proin " + + "suscipit mollis enim, in convallis dui commodo eu." + + "\n\n" + + "Nam in elit placerat, vulputate lacus sed, tempor mi. Cras " + + "hendrerit nibh enim, sed viverra orci gravida eu. Proin " + + "egestas massa vitae ornare iaculis. In sed erat orci. " + + "Praesent a massa vehicula, mattis nunc eu, blandit nulla. " + + "Morbi luctus justo sit amet est volutpat facilisis. " + + "Aliquam vehicula justo non erat tempor, vitae sollicitudin " + + "risus rhoncus. Suspendisse non sem lacus. Duis tellus leo, " + + "tincidunt quis lorem ac, elementum pharetra tortor. Praesent " + + "sit amet tempus orci, non tempus est. Fusce auctor leo et " + + "cursus venenatis. Nunc dapibus ut ipsum id consectetur. " + + "Curabitur faucibus metus quis leo tempus condimentum." + + "\n\n" + + "Sed sodales elit nec laoreet pellentesque. Sed condimentum " + + "risus quis lectus condimentum, varius suscipit turpis " + + "sagittis. Phasellus consequat metus eget odio fringilla " + + "bibendum. Vivamus fermentum est et elit congue, vitae varius " + + "dui egestas. Nunc sed nunc quis felis lobortis faucibus. " + + "Integer at purus quis urna posuere pulvinar at at erat. " + + "Etiam finibus erat quis nisi fermentum interdum. Sed non " + + "velit nec justo ultrices ultrices. Praesent vitae eros id " + + "dolor commodo viverra et in lorem. Duis non lacus quis nulla " + + "ultricies aliquet ut id nibh. Aliquam erat volutpat. Proin " + + "tincidunt est in feugiat sagittis. Phasellus quis maximus " + + "felis, vel volutpat enim. Aenean convallis ipsum eu orci " + + "posuere aliquam. Donec et tristique nibh, id ultricies eros." + + "\n\n" + + "Duis semper nulla nec pharetra hendrerit. Proin nibh lorem, " + + "condimentum sed sem at, malesuada sollicitudin urna. Etiam " + + "id augue sit amet lectus volutpat ullamcorper finibus a " + + "velit. Sed lacinia enim quis sem sollicitudin, vel faucibus " + + "leo elementum. Etiam convallis felis sed tortor iaculis " + + "tristique. Nulla at est in ligula pulvinar porta vel " + + "interdum felis. Vivamus nec mattis metus, quis imperdiet " + + "ligula. Aliquam sed arcu arcu. Curabitur at consectetur sem. " + + "Morbi volutpat non eros ac ultrices. Nulla nunc urna, " + + "dignissim eget vestibulum at, molestie vitae justo. Mauris " + + "et enim quis est rhoncus laoreet eu nec ex." + + "\n\n" + + "In lobortis sodales quam sed fermentum. In quis fermentum " + + "arcu. Integer tincidunt, nisi at consequat laoreet, turpis " + + "arcu egestas metus, a fringilla nisl justo id urna. " + + "Suspendisse iaculis, urna eu eleifend ultrices, justo dolor " + + "tempus sapien, eu fermentum leo dolor a erat. Integer vitae " + + "egestas risus. Donec lacinia ac neque at fermentum. Class " + + "aptent taciti sociosqu ad litora torquent per conubia " + + "nostra, per inceptos himenaeos. Maecenas blandit ex enim, " + + "vel commodo lectus gravida id. Pellentesque aliquet dapibus " + + "consectetur." + + "\n\n" + + " Vestibulum maximus vitae nisi non tempus. Mauris rutrum " + + "rhoncus rutrum. Donec placerat, eros ut lacinia gravida, " + + "nunc lectus laoreet est, a ultricies dolor tellus non nulla. " + + "Aenean nibh eros, porttitor nec tortor in, vulputate " + + "porttitor elit. Nulla facilisi. Praesent a porta urna, non " + + "aliquam diam. Vestibulum ultricies turpis ut lectus " + + "vulputate, eu condimentum lorem consectetur. Lorem ipsum " + + "dolor sit amet, consectetur adipiscing elit. Quisque " + + "posuere nibh sed augue venenatis posuere. Cras pretium " + + "neque nec tristique tincidunt." + + "\n\n" + + "In tortor tellus, eleifend eget mattis ut, pellentesque " + + "ac ex. Phasellus rhoncus consectetur leo non aliquam. " + + "Suspendisse pellentesque purus quis consectetur congue. " + + "Fusce posuere arcu nisi. Nunc non lectus in lacus molestie " + + "maximus sed ut lacus. Integer tincidunt purus quis arcu " + + "pretium, at sodales erat fermentum. Praesent ornare " + + "sagittis euismod. Aliquam suscipit elit ut condimentum " + + "accumsan." + + "\n\n" + + "Mauris dignissim quam ut odio vehicula interdum eget " + + "eget enim. Vivamus rutrum ut felis vel volutpat. Duis " + + "aliquam dapibus maximus. Suspendisse fringilla blandit " + + "urna, ut finibus lacus tincidunt sed. Proin vel accumsan " + + "ipsum. Nullam dictum metus vel accumsan varius. Ut " + + "accumsan volutpat justo id gravida." + + "\n\n" + + "Proin lectus purus, tempus nec nibh in, convallis " + + "tempor magna. Quisque et scelerisque mauris. Quisque " + + "pharetra a arcu quis viverra. Quisque sed felis nisi. " + + "Interdum et malesuada fames ac ante ipsum primis in " + + "faucibus. Duis iaculis, elit vel dignissim congue, " + + "magna nisi tristique sapien, semper porta eros velit " + + "efficitur elit. Ut eget congue nisi, at luctus ipsum. " + + "Etiam ac elit sit amet ex tempor congue." + + "\n\n" + + "Aenean dictum ullamcorper mi, sed facilisis orci " + + "porttitor sit amet. Aliquam venenatis sapien vel leo " + + "pellentesque, iaculis aliquet augue tincidunt. Aenean " + + "posuere aliquet bibendum. Cum sociis natoque penatibus " + + "et magnis dis parturient montes, nascetur ridiculus " + + "mus. Vivamus gravida elit ac convallis lobortis. In " + + "hac habitasse platea dictumst. Sed ullamcorper mauris " + + "vitae ante sagittis finibus. Integer scelerisque orci " + + "sit amet nulla imperdiet, ac placerat lectus feugiat. " + + "Vestibulum sed dui tortor. Donec vitae dictum tortor." + + "\n\n" + + "Etiam porttitor dolor neque, ut ullamcorper tortor " + + "vulputate porta. Quisque vel lectus vel tellus " + + "hendrerit tincidunt. Nulla et luctus ligula. In vitae " + + "nulla non ex lobortis condimentum quis vitae tellus. " + + "Donec at arcu sit amet velit dictum euismod. Donec " + + "bibendum vel erat eget tincidunt. Etiam efficitur in " + + "odio ac aliquam. Nulla sodales vestibulum suscipit. " + + "Aliquam in venenatis ex. Maecenas rhoncus eleifend " + + "lobortis. Phasellus pharetra ligula eget pretium " + + "viverra. Cras ut pretium lorem." + + "\n\n" + + "Pellentesque eu tellus malesuada, varius leo quis, " + + "eleifend tortor. Mauris aliquet, massa in tincidunt " + + "egestas, neque orci luctus nisl, sit amet viverra " + + "risus nisl quis eros. Donec quis posuere justo. " + + "Curabitur nunc enim, sagittis at ligula eu, placerat " + + "pellentesque turpis. Quisque gravida nisl sed purus " + + "pulvinar, et ultricies massa tempor. Suspendisse " + + "interdum porttitor neque, pretium sagittis felis " + + "efficitur et. Quisque mollis placerat purus vel " + + "elementum. Quisque finibus, lectus aliquam rutrum " + + "efficitur, nisl sapien elementum purus, at pretium " + + "tellus quam vel mi. Nunc non dictum tortor. In sit " + + "amet sem sit amet erat gravida fringilla at in ipsum. " + + "In pretium massa vulputate imperdiet vulputate. " + + "Vivamus placerat, velit sed facilisis posuere, felis " + + "felis bibendum velit, ut gravida purus mauris non " + + "diam. Integer elementum libero augue, in pharetra " + + "felis scelerisque nec." + + "\n\n" + + " Etiam eu tellus ac lectus vulputate aliquet nec id " + + "mi. Donec eget neque vel ex lobortis sollicitudin. " + + "Curabitur quis ante eget sem cursus dignissim sed " + + "at ligula. Suspendisse mollis eget erat sit amet " + + "ornare. Donec laoreet convallis elementum. Aliquam " + + "mauris diam, porttitor eget ligula consequat, " + + "rhoncus hendrerit tellus. Sed ante tellus, tempor " + + "a ipsum nec, tempus auctor odio. Proin facilisis " + + "at urna eget maximus. Nulla vitae tempor nisi. " + + "Fusce a odio cursus, vestibulum nulla id, accumsan " + + "nisl." + + "\n\n" + + "Fusce sed varius massa, nec blandit velit. Integer " + + "vitae lacinia ligula. Mauris vestibulum sem ut " + + "blandit cras amet."; + + public static byte[] getBytes() { + return LOREM_IPSUM.getBytes(); + } + + public static String getString() { + return LOREM_IPSUM; + } +} From miroslav.kos at oracle.com Thu Nov 27 11:27:50 2014 From: miroslav.kos at oracle.com (Miroslav Kos) Date: Thu, 27 Nov 2014 12:27:50 +0100 Subject: RFR: 8065870 Update JAX-WS RI integration to latest version (2.2.11-b141124.1933) Message-ID: <54770AB6.6050806@oracle.com> Hi, there is a bulk update of JAX-B/WS from upstream projects - webrev: http://cr.openjdk.java.net/~mkos/8065870/jaxws.00/ more details in issue desc: https://bugs.openjdk.java.net/browse/JDK-8065870 Could I ask for a review? It seems quite big (1126 lines changed) but there are just minor changes/fixes. Thanks Miran From pavel.rappo at oracle.com Thu Nov 27 11:58:25 2014 From: pavel.rappo at oracle.com (Pavel Rappo) Date: Thu, 27 Nov 2014 11:58:25 +0000 Subject: Library enhancement proposal for copying data from/to Reader/Writer InputStream/OutputStream In-Reply-To: References: <566A428C-1A13-4DEE-940B-2EBE375F2A7A@reini.net> <51329A99-55E5-4A42-9180-D872366B4DEE@oracle.com> <546EABED.9030301@oracle.com> <546EF98C.40704@oracle.com> <1D392C1F-E6CC-494D-97F8-C08C5BA36392@oracle.com> Message-ID: <6C673FD8-6733-4F1B-9EB5-955EB031AA7D@oracle.com> I've created an initial webrev for this change: http://cr.openjdk.java.net/~prappo/ioutil/webrev.00/ Welcome to discussion. P.S. The only thing I changed was the license header. -Pavel > On 27 Nov 2014, at 09:48, Patrick Reinhart wrote: > > Hi Paul, > > Seems not to work. See inline patch? From patrick at reini.net Thu Nov 27 12:12:24 2014 From: patrick at reini.net (Patrick Reinhart) Date: Thu, 27 Nov 2014 13:12:24 +0100 Subject: Library enhancement proposal for copying data from/to Reader/Writer InputStream/OutputStream In-Reply-To: <6C673FD8-6733-4F1B-9EB5-955EB031AA7D@oracle.com> References: <566A428C-1A13-4DEE-940B-2EBE375F2A7A@reini.net> <51329A99-55E5-4A42-9180-D872366B4DEE@oracle.com> <546EABED.9030301@oracle.com> <546EF98C.40704@oracle.com> <1D392C1F-E6CC-494D-97F8-C08C5BA36392@oracle.com> <6C673FD8-6733-4F1B-9EB5-955EB031AA7D@oracle.com> Message-ID: Hi Pavel, Thanks very much for your help. I?ve try my best to get author state ;-) Patrick > Am 27.11.2014 um 12:58 schrieb Pavel Rappo : > > I've created an initial webrev for this change: http://cr.openjdk.java.net/~prappo/ioutil/webrev.00/ > Welcome to discussion. > > P.S. The only thing I changed was the license header. > > -Pavel > >> On 27 Nov 2014, at 09:48, Patrick Reinhart wrote: >> >> Hi Paul, >> >> Seems not to work. See inline patch? > From dl at cs.oswego.edu Thu Nov 27 12:24:21 2014 From: dl at cs.oswego.edu (Doug Lea) Date: Thu, 27 Nov 2014 07:24:21 -0500 Subject: [concurrency-interest] RFR: 8065804: JEP 171:Clarifications/corrections for fence intrinsics In-Reply-To: References: Message-ID: <547717F5.4020907@cs.oswego.edu> On 11/26/2014 09:56 PM, David Holmes wrote: > Martin Buchholz writes: >> On Wed, Nov 26, 2014 at 5:08 PM, David Holmes >> wrote: >>> Please explain why you have changed the defined semantics for >> storeFence. >>> You have completely reversed the direction of the barrier. >> >> Yes. I believe the current spec of storeFence was a copy-paste typo, >> and it seems others feel likewise. > > Can whomever wrote that original spec please confirm that. > The translations of loadFence == [LoadLoad|LoadStore] and storeFence == [StoreStore|LoadStore] into prose got mangled at some point. (Probably by me; sorry if so!) -Doug From pavel.rappo at oracle.com Thu Nov 27 12:26:54 2014 From: pavel.rappo at oracle.com (Pavel Rappo) Date: Thu, 27 Nov 2014 12:26:54 +0000 Subject: Library enhancement proposal for copying data from/to Reader/Writer InputStream/OutputStream In-Reply-To: References: <566A428C-1A13-4DEE-940B-2EBE375F2A7A@reini.net> <51329A99-55E5-4A42-9180-D872366B4DEE@oracle.com> <546EABED.9030301@oracle.com> <546EF98C.40704@oracle.com> <1D392C1F-E6CC-494D-97F8-C08C5BA36392@oracle.com> <6C673FD8-6733-4F1B-9EB5-955EB031AA7D@oracle.com> Message-ID: <460E5A3C-BBA0-4D34-95DB-7F1EB37D93F1@oracle.com> At the bare minimum I would use runtime null/range checks and give a little bit more freedom for those who want to control buffer (or probably reuse already existing one): http://cr.openjdk.java.net/~prappo/ioutil/webrev.01/ I would also add some clarifications in the javadoc. -Pavel From patrick at reini.net Thu Nov 27 13:01:31 2014 From: patrick at reini.net (Patrick Reinhart) Date: Thu, 27 Nov 2014 14:01:31 +0100 Subject: Library enhancement proposal for copying data from/to Reader/Writer InputStream/OutputStream In-Reply-To: <460E5A3C-BBA0-4D34-95DB-7F1EB37D93F1@oracle.com> References: <566A428C-1A13-4DEE-940B-2EBE375F2A7A@reini.net> <51329A99-55E5-4A42-9180-D872366B4DEE@oracle.com> <546EABED.9030301@oracle.com> <546EF98C.40704@oracle.com> <1D392C1F-E6CC-494D-97F8-C08C5BA36392@oracle.com> <6C673FD8-6733-4F1B-9EB5-955EB031AA7D@oracle.com> <460E5A3C-BBA0-4D34-95DB-7F1EB37D93F1@oracle.com> Message-ID: <598AF5CE-BB27-4197-B6D5-BB1503B1E59C@reini.net> Added null checks also to Readable/Appendable copy methods Patrick > Am 27.11.2014 um 13:26 schrieb Pavel Rappo : > > At the bare minimum I would use runtime null/range checks and give a little bit more freedom for those who want to control buffer (or probably reuse already existing one): http://cr.openjdk.java.net/~prappo/ioutil/webrev.01/ > > I would also add some clarifications in the javadoc. > > -Pavel > diff -r 194458fe7339 src/java.base/share/classes/java/io/IOUtil.java --- a/src/java.base/share/classes/java/io/IOUtil.java Thu Nov 27 13:50:36 2014 +0100 +++ b/src/java.base/share/classes/java/io/IOUtil.java Thu Nov 27 14:00:07 2014 +0100 @@ -71,7 +71,6 @@ */ public static long copy(InputStream source, OutputStream target, byte[] buffer) throws IOException { - Objects.requireNonNull(source, "source"); Objects.requireNonNull(target, "target"); Objects.requireNonNull(buffer, "buffer"); @@ -100,15 +99,15 @@ * @throws IOException if an I/O error occurs when reading or writing */ public static long copy(Readable source, Appendable target) throws IOException { - long totalRead = 0L; + Objects.requireNonNull(source, "source"); + Objects.requireNonNull(target, "target"); + long copied = 0L; CharBuffer buffer = CharBuffer.allocate(BUFFER_SIZE); - int read; - while ((read = source.read(buffer)) > -1) { + for (int read; (read = source.read(buffer)) > -1; copied += read) { buffer.flip(); target.append(buffer, 0, read); - totalRead += read; } - return totalRead; + return copied; } } From Alan.Bateman at oracle.com Thu Nov 27 13:51:57 2014 From: Alan.Bateman at oracle.com (Alan Bateman) Date: Thu, 27 Nov 2014 13:51:57 +0000 Subject: Library enhancement proposal for copying data from/to Reader/Writer InputStream/OutputStream In-Reply-To: References: <566A428C-1A13-4DEE-940B-2EBE375F2A7A@reini.net> <51329A99-55E5-4A42-9180-D872366B4DEE@oracle.com> <546EABED.9030301@oracle.com> <546EF98C.40704@oracle.com> <1D392C1F-E6CC-494D-97F8-C08C5BA36392@oracle.com> Message-ID: <54772C7D.7080209@oracle.com> On 27/11/2014 09:48, Patrick Reinhart wrote: > Hi Paul, > > Seems not to work. See inline patch? > > Patrick > In terms of signatures then the methods look like a reasonable start. For the javadoc/spec then there are several issues that will need consideration and you might get some ideas from the javadoc that we have in the existing Files.copy methods - for example, it will need some wording about errors that happen after some (but not all) bytes/chars have been copied, some words to make it clear that the async close or either stream or thread interrupt during the copy is specified to the stream/readers/appendable involved and cannot be specified. It might also be useful to make it clear that IOE is thrown when called when the source or target is closed. -Alan. From patrick at reini.net Thu Nov 27 15:11:19 2014 From: patrick at reini.net (Patrick Reinhart) Date: Thu, 27 Nov 2014 16:11:19 +0100 Subject: Library enhancement proposal for copying data from/to Reader/Writer InputStream/OutputStream In-Reply-To: <54772C7D.7080209@oracle.com> References: <566A428C-1A13-4DEE-940B-2EBE375F2A7A@reini.net> <51329A99-55E5-4A42-9180-D872366B4DEE@oracle.com> <546EABED.9030301@oracle.com> <546EF98C.40704@oracle.com> <1D392C1F-E6CC-494D-97F8-C08C5BA36392@oracle.com> <54772C7D.7080209@oracle.com> Message-ID: <769E83F8-FD6D-4F8B-B415-DD044C5D08BA@reini.net> Inlined patch containing also additional java doc changes based on the version http://cr.openjdk.java.net/~prappo/ioutil/webrev.01/ It may contain some non proper english ;-) -Patrick > Am 27.11.2014 um 14:51 schrieb Alan Bateman : > > On 27/11/2014 09:48, Patrick Reinhart wrote: >> Hi Paul, >> >> Seems not to work. See inline patch? >> >> Patrick >> > In terms of signatures then the methods look like a reasonable start. For the javadoc/spec then there are several issues that will need consideration and you might get some ideas from the javadoc that we have in the existing Files.copy methods - for example, it will need some wording about errors that happen after some (but not all) bytes/chars have been copied, some words to make it clear that the async close or either stream or thread interrupt during the copy is specified to the stream/readers/appendable involved and cannot be specified. It might also be useful to make it clear that IOE is thrown when called when the source or target is closed. > > -Alan. diff -r 194458fe7339 src/java.base/share/classes/java/io/IOUtil.java --- a/src/java.base/share/classes/java/io/IOUtil.java Thu Nov 27 13:50:36 2014 +0100 +++ b/src/java.base/share/classes/java/io/IOUtil.java Thu Nov 27 16:08:09 2014 +0100 @@ -47,10 +47,28 @@ /** * Reads all bytes from an input stream and writes them to an output stream. * - * @param source the input stream to read from - * @param target the output stream to write to - * @return the number of bytes successfully copied - * @throws IOException if an I/O error occurs when reading or writing + *

If an I/O error occurs reading from the input stream or writing to + * the output stream, then it may do so after the some bytes have been read + * or written. Consequently the input stream may not be at end of stream and + * may be in an inconsistent state. It is strongly recommended that the input + * and output stream be promptly closed if an I/O error occurs. + * + *

This method may block indefinitely reading from the input stream (or + * writing to the output stream). The behavior for the case that the input + * and output stream is asynchronously closed or the thread interrupted + * during the copy is highly input and output stream system provider specific + * and therefore not specified. + * + * @param source + * the input stream to read from + * @param target + * the output stream to write to + * + * @return the number of bytes successfully copied + * @throws IOException + * if an I/O error occurs when reading or writing. In particular, + * an IOException may be thrown if the output stream + * has been closed. */ public static long copy(InputStream source, OutputStream target) throws IOException { @@ -62,16 +80,34 @@ /** * Reads all bytes from an input stream and writes them to an output stream. * - * @param source the input stream to read from (not {@code null}) - * @param target the output stream to write to (not {@code null}) - * @param buffer the intermediate buffer to use (not {@code null}, - * and {@code buffer.length >= 1}) - * @return the number of bytes successfully copied - * @throws IOException if an I/O error occurs when reading or writing + *

If an I/O error occurs reading from the input stream or writing to + * the output stream, then it may do so after the some bytes have been read + * or written. Consequently the input stream may not be at end of stream and + * may be in an inconsistent state. It is strongly recommended that the input + * and output stream be promptly closed if an I/O error occurs. + * + *

This method may block indefinitely reading from the input stream (or + * writing to the output stream). The behavior for the case that the input + * and output stream is asynchronously closed or the thread interrupted + * during the copy is highly input and output stream system provider specific + * and therefore not specified. + * + * @param source + * the input stream to read from (not {@code null}) + * @param target + * the output stream to write to (not {@code null}) + * @param buffer + * the intermediate buffer to use (not {@code null}, + * and {@code buffer.length >= 1}) + * + * @return the number of bytes successfully copied + * @throws IOException + * if an I/O error occurs when reading or writing. In particular, + * an IOException may be thrown if the output stream + * has been closed. */ public static long copy(InputStream source, OutputStream target, byte[] buffer) throws IOException { - Objects.requireNonNull(source, "source"); Objects.requireNonNull(target, "target"); Objects.requireNonNull(buffer, "buffer"); @@ -92,23 +128,39 @@ /** * Reads all characters from an readable and writes them to an appendable. * - * @param source the readable to read from - * @param target the appendable to write to + *

If an I/O error occurs reading from the readable or writing to the + * appendable, then it may do so after the some characters have been read + * or written. Consequently the readable may not be at end of data and may + * be in an inconsistent state. It is strongly recommended that the readable + * and writable be promptly closed (in case of closables) if an I/O error + * occurs. * - * @return the number of characters successfully read and written + *

This method may block indefinitely reading from the readable (or + * writing to the appendable). The behavior for the case that the input and + * appendable is asynchronously closed or the thread interrupted + * during the copy is highly readable and appendable system provider specific + * and therefore not specified. * - * @throws IOException if an I/O error occurs when reading or writing + * @param source + * the readable to read from + * @param target + * the appendable to write to + * + * @return the number of characters successfully copied + * + * @throws IOException + * if an I/O error occurs when reading or writing. */ public static long copy(Readable source, Appendable target) throws IOException { - long totalRead = 0L; + Objects.requireNonNull(source, "source"); + Objects.requireNonNull(target, "target"); + long copied = 0L; CharBuffer buffer = CharBuffer.allocate(BUFFER_SIZE); - int read; - while ((read = source.read(buffer)) > -1) { + for (int read; (read = source.read(buffer)) > -1; copied += read) { buffer.flip(); target.append(buffer, 0, read); - totalRead += read; } - return totalRead; + return copied; } } From pavel.rappo at oracle.com Thu Nov 27 15:56:59 2014 From: pavel.rappo at oracle.com (Pavel Rappo) Date: Thu, 27 Nov 2014 15:56:59 +0000 Subject: Library enhancement proposal for copying data from/to Reader/Writer InputStream/OutputStream In-Reply-To: <769E83F8-FD6D-4F8B-B415-DD044C5D08BA@reini.net> References: <566A428C-1A13-4DEE-940B-2EBE375F2A7A@reini.net> <51329A99-55E5-4A42-9180-D872366B4DEE@oracle.com> <546EABED.9030301@oracle.com> <546EF98C.40704@oracle.com> <1D392C1F-E6CC-494D-97F8-C08C5BA36392@oracle.com> <54772C7D.7080209@oracle.com> <769E83F8-FD6D-4F8B-B415-DD044C5D08BA@reini.net> Message-ID: Patrick, as soon as I get feedback from Alan and Paul I'll merge these changes into the current version. 1. I wonder if it makes sense to provide an additional method where client specifies CharBuffer to be used? Moreover if the first argument is CharSequence then we don't even need a buffer: /** * Reads all characters from a readable and appends them to an appendable. * * @param source the readable to read from ({@code source != null}) * @param target the appendable to append to ({@code target != null}) * * @return the number of characters copied * * @throws IOException if an I/O error occurs when reading or appending */ public static long copy(Readable source, Appendable target) throws IOException { Objects.requireNonNull(source, "source"); Objects.requireNonNull(target, "target"); if (source instanceof CharSequence) return copy1((CharSequence) source, target); return copy0(source, target, CharBuffer.allocate(BUFFER_SIZE)); } /** * Reads all characters from a readable and appends them to an appendable. * * @param source the readable to read from ({@code source != null}) * @param target the appendable to append to ({@code target != null}) * @param buffer the intermediate buffer to use * ({@code buffer != null && buffer.hasRemaining() * && !buffer.isReadOnly()}) * @return the number of characters copied * @throws IOException if an I/O error occurs when reading or appending */ public static long copy(Readable source, Appendable target, CharBuffer buffer) throws IOException { Objects.requireNonNull(source, "source"); Objects.requireNonNull(target, "target"); Objects.requireNonNull(buffer, "buffer"); if (!buffer.hasRemaining()) throw new IllegalArgumentException("Insufficient buffer size"); if (buffer.isReadOnly()) throw new IllegalArgumentException("buffer is read only"); return copy0(source, target, buffer); } private static long copy1(CharSequence source, Appendable target) throws IOException { long copied = source.length(); target.append(source); return copied; } private static long copy0(Readable source, Appendable target, CharBuffer buffer) throws IOException { 2. I think guts of the copy method could be changed to something like this: long copied = 0L; + int oldLim = buffer.limit(); for (int read; (read = source.read(buffer)) > -1; copied += read) { buffer.flip(); target.append(buffer, 0, read); + buffer.limit(oldLim); } return copied; Otherwise on each iteration you're potentially shrinking available buffer storage based on amount of data read into it (if I'm not mistaken). So each time you should restore the limit. 3. We should think of revisiting and including the only method from sun.misc.IOUtils. Removing the class and reparenting its dependencies. -Pavel > diff -r 194458fe7339 src/java.base/share/classes/java/io/IOUtil.java > --- a/src/java.base/share/classes/java/io/IOUtil.java Thu Nov 27 13:50:36 2014 +0100 > +++ b/src/java.base/share/classes/java/io/IOUtil.java Thu Nov 27 16:08:09 2014 +0100 > @@ -47,10 +47,28 @@ > /** > * Reads all bytes from an input stream and writes them to an output stream. > * > - * @param source the input stream to read from > - * @param target the output stream to write to > - * @return the number of bytes successfully copied > - * @throws IOException if an I/O error occurs when reading or writing > + *

If an I/O error occurs reading from the input stream or writing to > + * the output stream, then it may do so after the some bytes have been read > + * or written. Consequently the input stream may not be at end of stream and > + * may be in an inconsistent state. It is strongly recommended that the input > + * and output stream be promptly closed if an I/O error occurs. > + * > + *

This method may block indefinitely reading from the input stream (or > + * writing to the output stream). The behavior for the case that the input > + * and output stream is asynchronously closed or the thread interrupted > + * during the copy is highly input and output stream system provider specific > + * and therefore not specified. > + * > + * @param source > + * the input stream to read from > + * @param target > + * the output stream to write to > + * > + * @return the number of bytes successfully copied > + * @throws IOException > + * if an I/O error occurs when reading or writing. In particular, > + * an IOException may be thrown if the output stream > + * has been closed. > */ > public static long copy(InputStream source, OutputStream target) > throws IOException { > @@ -62,16 +80,34 @@ > /** > * Reads all bytes from an input stream and writes them to an output stream. > * > - * @param source the input stream to read from (not {@code null}) > - * @param target the output stream to write to (not {@code null}) > - * @param buffer the intermediate buffer to use (not {@code null}, > - * and {@code buffer.length >= 1}) > - * @return the number of bytes successfully copied > - * @throws IOException if an I/O error occurs when reading or writing > + *

If an I/O error occurs reading from the input stream or writing to > + * the output stream, then it may do so after the some bytes have been read > + * or written. Consequently the input stream may not be at end of stream and > + * may be in an inconsistent state. It is strongly recommended that the input > + * and output stream be promptly closed if an I/O error occurs. > + * > + *

This method may block indefinitely reading from the input stream (or > + * writing to the output stream). The behavior for the case that the input > + * and output stream is asynchronously closed or the thread interrupted > + * during the copy is highly input and output stream system provider specific > + * and therefore not specified. > + * > + * @param source > + * the input stream to read from (not {@code null}) > + * @param target > + * the output stream to write to (not {@code null}) > + * @param buffer > + * the intermediate buffer to use (not {@code null}, > + * and {@code buffer.length >= 1}) > + * > + * @return the number of bytes successfully copied > + * @throws IOException > + * if an I/O error occurs when reading or writing. In particular, > + * an IOException may be thrown if the output stream > + * has been closed. > */ > public static long copy(InputStream source, OutputStream target, > byte[] buffer) throws IOException { > - > Objects.requireNonNull(source, "source"); > Objects.requireNonNull(target, "target"); > Objects.requireNonNull(buffer, "buffer"); > @@ -92,23 +128,39 @@ > /** > * Reads all characters from an readable and writes them to an appendable. > * > - * @param source the readable to read from > - * @param target the appendable to write to > + *

If an I/O error occurs reading from the readable or writing to the > + * appendable, then it may do so after the some characters have been read > + * or written. Consequently the readable may not be at end of data and may > + * be in an inconsistent state. It is strongly recommended that the readable > + * and writable be promptly closed (in case of closables) if an I/O error > + * occurs. > * > - * @return the number of characters successfully read and written > + *

This method may block indefinitely reading from the readable (or > + * writing to the appendable). The behavior for the case that the input and > + * appendable is asynchronously closed or the thread interrupted > + * during the copy is highly readable and appendable system provider specific > + * and therefore not specified. > * > - * @throws IOException if an I/O error occurs when reading or writing > + * @param source > + * the readable to read from > + * @param target > + * the appendable to write to > + * > + * @return the number of characters successfully copied > + * > + * @throws IOException > + * if an I/O error occurs when reading or writing. > */ > public static long copy(Readable source, Appendable target) throws IOException { > - long totalRead = 0L; > + Objects.requireNonNull(source, "source"); > + Objects.requireNonNull(target, "target"); > + long copied = 0L; > CharBuffer buffer = CharBuffer.allocate(BUFFER_SIZE); > - int read; > - while ((read = source.read(buffer)) > -1) { > + for (int read; (read = source.read(buffer)) > -1; copied += read) { > buffer.flip(); > target.append(buffer, 0, read); > - totalRead += read; > } > - return totalRead; > + return copied; > } > } > > From patrick at reini.net Thu Nov 27 16:10:06 2014 From: patrick at reini.net (Patrick Reinhart) Date: Thu, 27 Nov 2014 17:10:06 +0100 Subject: Library enhancement proposal for copying data from/to Reader/Writer InputStream/OutputStream In-Reply-To: References: <566A428C-1A13-4DEE-940B-2EBE375F2A7A@reini.net> <51329A99-55E5-4A42-9180-D872366B4DEE@oracle.com> <546EABED.9030301@oracle.com> <546EF98C.40704@oracle.com> <1D392C1F-E6CC-494D-97F8-C08C5BA36392@oracle.com> <54772C7D.7080209@oracle.com> <769E83F8-FD6D-4F8B-B415-DD044C5D08BA@reini.net> Message-ID: <0196C3F9-D680-4630-9C5C-87AAF3E492B3@reini.net> > Patrick, as soon as I get feedback from Alan and Paul I'll merge these changes into the current version. > > 1. I wonder if it makes sense to provide an additional method where client specifies CharBuffer to be used? > Moreover if the first argument is CharSequence then we don't even need a buffer: > > /** > * Reads all characters from a readable and appends them to an appendable. > * > * @param source the readable to read from ({@code source != null}) > * @param target the appendable to append to ({@code target != null}) > * > * @return the number of characters copied > * > * @throws IOException if an I/O error occurs when reading or appending > */ > public static long copy(Readable source, Appendable target) > throws IOException { > Objects.requireNonNull(source, "source"); > Objects.requireNonNull(target, "target"); > > if (source instanceof CharSequence) > return copy1((CharSequence) source, target); > > return copy0(source, target, CharBuffer.allocate(BUFFER_SIZE)); > } > > /** > * Reads all characters from a readable and appends them to an appendable. > * > * @param source the readable to read from ({@code source != null}) > * @param target the appendable to append to ({@code target != null}) > * @param buffer the intermediate buffer to use > * ({@code buffer != null && buffer.hasRemaining() > * && !buffer.isReadOnly()}) > * @return the number of characters copied > * @throws IOException if an I/O error occurs when reading or appending > */ > public static long copy(Readable source, Appendable target, > CharBuffer buffer) throws IOException { > Objects.requireNonNull(source, "source"); > Objects.requireNonNull(target, "target"); > Objects.requireNonNull(buffer, "buffer"); > if (!buffer.hasRemaining()) > throw new IllegalArgumentException("Insufficient buffer size"); > if (buffer.isReadOnly()) > throw new IllegalArgumentException("buffer is read only"); > return copy0(source, target, buffer); > } > > private static long copy1(CharSequence source, Appendable target) > throws IOException { > long copied = source.length(); > target.append(source); > return copied; > } > > private static long copy0(Readable source, Appendable target, > CharBuffer buffer) throws IOException { > Seems reasonable to me. Will you add those changes additionally to mine then? > 2. I think guts of the copy method could be changed to something like this: > > long copied = 0L; > + int oldLim = buffer.limit(); > > for (int read; (read = source.read(buffer)) > -1; copied += read) { > buffer.flip(); > target.append(buffer, 0, read); > + buffer.limit(oldLim); > } > return copied; > > Otherwise on each iteration you're potentially shrinking available buffer storage based on > amount of data read into it (if I'm not mistaken). So each time you should restore the limit. There I?m not that experienced either. I?m shamed about it, but never used CharBuffer by myself so far? (my implementation could therefore be not completely correct) > 3. We should think of revisiting and including the only method from sun.misc.IOUtils. Removing > the class and reparenting its dependencies. That would be a followup task - on the way to authorship ;-) -Patrick From jonathan.gibbons at oracle.com Thu Nov 27 16:32:48 2014 From: jonathan.gibbons at oracle.com (Jonathan Gibbons) Date: Thu, 27 Nov 2014 08:32:48 -0800 Subject: NIO question: Path.toFile() In-Reply-To: <5476DB61.8070307@oracle.com> References: <547655AF.6020805@oracle.com> <5476DB61.8070307@oracle.com> Message-ID: <54775230.3080609@oracle.com> On 11/27/2014 12:05 AM, Alan Bateman wrote: > On 26/11/2014 22:35, Jonathan Gibbons wrote: >> NIO folk, >> >> java.nio.file.Path.toFile() is specified to throw >> UnsupportedOperationException if this Path is not associated with the >> default provider. >> >> UOE seems a somewhat strange choice here. IllegalStateException or a >> custom exception might be seen as more obvious alternatives. (It's >> also somewhat weird that UOE is explicitly documented as being a >> member of the Java Collections Framework.) >> >> So, if there any background rationale of why UOE was chosen here? I'm >> not looking to change the spec for Path.toFile() in any way, but I am >> looking to design a higher level API that may encounter the same >> underlying problem, that a Path may not be associated with the >> default provider. At my higher level, I need to decide whether to >> go with UOE, just because that is what Path.toFile() does, or whether >> to convert it into some other exception. > > I think UOE is right because it must fail for every instance of an > implementation type that is not associated with the default provider. > I don't think IllegalStateException would have worked there aren't any > states where this method could succeed. > > A blog that I've re-read a few things is Kevin Bourrillion's take on > unchecked exceptions: > http://smallwig.blogspot.co.uk/2008/06/common-java-unchecked-exception-types.html > > > It's probably an area where more guidelines need to be written down, > also probably a topic where it's impossible to get broad agreement. > > -Alan. > > Alan, Thanks for the perspective on why it is UOE. Kevin's blog is interesting, but as I read it, it argues against UOE :-) Given a collection of Path objects, generated from various file systems, some may return a value from Path.toFile(), and some may throw an exception, depending on whether something like the following condition is true or not path.getFileSystem() == FileSystems.getDefault() (Of course, that expression itself is scary, since the javadoc for FileSystems.getDefault() says it may throw "an unspecified error".) I think this means that from the point of view of an API using java.nio.file.Path internally, UOE should not be propagated to clients of that API. Instead, it should be caught and rethrown as something more specific to that level of API. -- Jon From pavel.rappo at oracle.com Thu Nov 27 16:46:45 2014 From: pavel.rappo at oracle.com (Pavel Rappo) Date: Thu, 27 Nov 2014 16:46:45 +0000 Subject: Library enhancement proposal for copying data from/to Reader/Writer InputStream/OutputStream In-Reply-To: <0196C3F9-D680-4630-9C5C-87AAF3E492B3@reini.net> References: <566A428C-1A13-4DEE-940B-2EBE375F2A7A@reini.net> <51329A99-55E5-4A42-9180-D872366B4DEE@oracle.com> <546EABED.9030301@oracle.com> <546EF98C.40704@oracle.com> <1D392C1F-E6CC-494D-97F8-C08C5BA36392@oracle.com> <54772C7D.7080209@oracle.com> <769E83F8-FD6D-4F8B-B415-DD044C5D08BA@reini.net> <0196C3F9-D680-4630-9C5C-87AAF3E492B3@reini.net> Message-ID: Yes I will. If I don't see any objections. -Pavel > On 27 Nov 2014, at 16:10, Patrick Reinhart wrote: > > Will you add those changes additionally to mine then? From Alan.Bateman at oracle.com Thu Nov 27 16:47:33 2014 From: Alan.Bateman at oracle.com (Alan Bateman) Date: Thu, 27 Nov 2014 16:47:33 +0000 Subject: NIO question: Path.toFile() In-Reply-To: <54775230.3080609@oracle.com> References: <547655AF.6020805@oracle.com> <5476DB61.8070307@oracle.com> <54775230.3080609@oracle.com> Message-ID: <547755A5.30406@oracle.com> On 27/11/2014 16:32, Jonathan Gibbons wrote: > > Alan, > > Thanks for the perspective on why it is UOE. Kevin's blog is > interesting, but as I read it, it argues against UOE :-) I think there's a discussion point there as Path is an interface and there are different concrete types. In any case, we can't change this of course. > : > > I think this means that from the point of view of an API using > java.nio.file.Path internally, UOE should not be propagated to clients > of that API. Instead, it should be caught and rethrown as something > more specific to that level of API. The toFile() method is meant for interoperability with the legacy API and so some care is required to avoid using it on Paths to objects in arbitrary file system that could not be represented by java.io.File. I think I'd need to see more context to know if you should be propagating anything or whether the issue is just an incorrect assumption somewhere. -Alan From Alan.Bateman at oracle.com Thu Nov 27 16:57:18 2014 From: Alan.Bateman at oracle.com (Alan Bateman) Date: Thu, 27 Nov 2014 16:57:18 +0000 Subject: Library enhancement proposal for copying data from/to Reader/Writer InputStream/OutputStream In-Reply-To: References: <566A428C-1A13-4DEE-940B-2EBE375F2A7A@reini.net> <51329A99-55E5-4A42-9180-D872366B4DEE@oracle.com> <546EABED.9030301@oracle.com> <546EF98C.40704@oracle.com> <1D392C1F-E6CC-494D-97F8-C08C5BA36392@oracle.com> <54772C7D.7080209@oracle.com> <769E83F8-FD6D-4F8B-B415-DD044C5D08BA@reini.net> Message-ID: <547757EE.4090205@oracle.com> On 27/11/2014 15:56, Pavel Rappo wrote: > Patrick, as soon as I get feedback from Alan and Paul I'll merge these changes into the current version. > > 1. I wonder if it makes sense to provide an additional method where client specifies CharBuffer to be used? > Moreover if the first argument is CharSequence then we don't even need a buffer: > I would suggest starting out with the simple copy(source, target) and only add additional variants if there is strong need. In this case, is a variant that allows the intermediate buffer to be specified be something that would be widely used? -Alan From jonathan.gibbons at oracle.com Thu Nov 27 17:13:13 2014 From: jonathan.gibbons at oracle.com (Jonathan Gibbons) Date: Thu, 27 Nov 2014 09:13:13 -0800 Subject: NIO question: Path.toFile() In-Reply-To: <547755A5.30406@oracle.com> References: <547655AF.6020805@oracle.com> <5476DB61.8070307@oracle.com> <54775230.3080609@oracle.com> <547755A5.30406@oracle.com> Message-ID: <54775BA9.4010106@oracle.com> On 11/27/2014 08:47 AM, Alan Bateman wrote: > On 27/11/2014 16:32, Jonathan Gibbons wrote: >> >> Alan, >> >> Thanks for the perspective on why it is UOE. Kevin's blog is >> interesting, but as I read it, it argues against UOE :-) > I think there's a discussion point there as Path is an interface and > there are different concrete types. In any case, we can't change this > of course. > > >> : >> >> I think this means that from the point of view of an API using >> java.nio.file.Path internally, UOE should not be propagated to >> clients of that API. Instead, it should be caught and rethrown as >> something more specific to that level of API. > > The toFile() method is meant for interoperability with the legacy API > and so some care is required to avoid using it on Paths to objects in > arbitrary file system that could not be represented by java.io.File. I > think I'd need to see more context to know if you should be > propagating anything or whether the issue is just an incorrect > assumption somewhere. > > -Alan The context is that we are considering extending javax.tools.StandardJavaFileManager to include support for java.nio.file.Path as well as java.io.File. This file manager provides implementations for JavaFileObject that can wrap java.io.File today, and java.nio.file.Path (proposed.) An existing client will only use the File-centric methods, and a well-written updated client should probably only use the Path-centric methods, but in terms of complete API design, we are left with the need to specify what happens if someone starts off by using a method like void setLocation(Location, Iterable) and then later on, for some reason, tries to use Iterable getLocation(Location). The "obvious" implementation would seem to be to try and use .toFile() on each element of the sequence given to setLocation. Which gets to the question of what to do with the UOE that might occur. Yes, the potential client of this proposed updated form of StandardJavaFileManager may have made a mistake trying to mix and match File-centric methods and Path-centric methods, but that doesn't absolve us of the responsibility to try and "do something sensible" in this case. My inclination is to try and use IllegalStateException, or maybe a custom subtype thereof. -- Jon From patrick at reini.net Thu Nov 27 17:40:12 2014 From: patrick at reini.net (Patrick Reinhart) Date: Thu, 27 Nov 2014 18:40:12 +0100 Subject: Library enhancement proposal for copying data from/to Reader/Writer InputStream/OutputStream In-Reply-To: <547757EE.4090205@oracle.com> References: <566A428C-1A13-4DEE-940B-2EBE375F2A7A@reini.net> <51329A99-55E5-4A42-9180-D872366B4DEE@oracle.com> <546EABED.9030301@oracle.com> <546EF98C.40704@oracle.com> <1D392C1F-E6CC-494D-97F8-C08C5BA36392@oracle.com> <54772C7D.7080209@oracle.com> <769E83F8-FD6D-4F8B-B415-DD044C5D08BA@reini.net> <547757EE.4090205@oracle.com> Message-ID: > Am 27.11.2014 um 17:57 schrieb Alan Bateman : > > On 27/11/2014 15:56, Pavel Rappo wrote: >> Patrick, as soon as I get feedback from Alan and Paul I'll merge these changes into the current version. >> >> 1. I wonder if it makes sense to provide an additional method where client specifies CharBuffer to be used? >> Moreover if the first argument is CharSequence then we don't even need a buffer: >> > I would suggest starting out with the simple copy(source, target) and only add additional variants if there is strong need. In this case, is a variant that allows the intermediate buffer to be specified be something that would be widely used? > > -Alan In our codebase here we have variants of different buffer sizes, mostly there are some with smaller sizes than the default I took from the Files class. To have the copy function within the JDK instead using a separate library or have this code written over and over again is much more important to me. -Patrick From Alan.Bateman at oracle.com Thu Nov 27 18:01:54 2014 From: Alan.Bateman at oracle.com (Alan Bateman) Date: Thu, 27 Nov 2014 18:01:54 +0000 Subject: Library enhancement proposal for copying data from/to Reader/Writer InputStream/OutputStream In-Reply-To: References: <566A428C-1A13-4DEE-940B-2EBE375F2A7A@reini.net> <51329A99-55E5-4A42-9180-D872366B4DEE@oracle.com> <546EABED.9030301@oracle.com> <546EF98C.40704@oracle.com> <1D392C1F-E6CC-494D-97F8-C08C5BA36392@oracle.com> <54772C7D.7080209@oracle.com> <769E83F8-FD6D-4F8B-B415-DD044C5D08BA@reini.net> <547757EE.4090205@oracle.com> Message-ID: <54776712.2050404@oracle.com> On 27/11/2014 17:40, Patrick Reinhart wrote: > : > In our codebase here we have variants of different buffer sizes, mostly there are some with smaller sizes than the default I took from the Files class. To have the copy function within the JDK instead using a separate library or have this code written over and over again is much more important to me. > I think a variant that has a size as an argument is worth considering, we've all done that. That said, Pavel's suggestion seem to be a intermediate buffer the hint as to the chunk size. -Alan From pavel.rappo at oracle.com Thu Nov 27 18:05:01 2014 From: pavel.rappo at oracle.com (Pavel Rappo) Date: Thu, 27 Nov 2014 18:05:01 +0000 Subject: Library enhancement proposal for copying data from/to Reader/Writer InputStream/OutputStream In-Reply-To: References: <566A428C-1A13-4DEE-940B-2EBE375F2A7A@reini.net> <51329A99-55E5-4A42-9180-D872366B4DEE@oracle.com> <546EABED.9030301@oracle.com> <546EF98C.40704@oracle.com> <1D392C1F-E6CC-494D-97F8-C08C5BA36392@oracle.com> <54772C7D.7080209@oracle.com> <769E83F8-FD6D-4F8B-B415-DD044C5D08BA@reini.net> <547757EE.4090205@oracle.com> Message-ID: Patrick, Alan is right in the sense that we have to estimate usefulness of each method wisely. Unfortunately your codebase (or anyone else's) might have some very specific biases and thus might not be a good sample to base conclusions upon. We need some more data to justify this proposal. The good start could be the jdk itself -- to see if those copy-with-my-buffer methods could be useful here. -Pavel > On 27 Nov 2014, at 17:40, Patrick Reinhart wrote: > > >> Am 27.11.2014 um 17:57 schrieb Alan Bateman : >> >> On 27/11/2014 15:56, Pavel Rappo wrote: >>> Patrick, as soon as I get feedback from Alan and Paul I'll merge these changes into the current version. >>> >>> 1. I wonder if it makes sense to provide an additional method where client specifies CharBuffer to be used? >>> Moreover if the first argument is CharSequence then we don't even need a buffer: >>> >> I would suggest starting out with the simple copy(source, target) and only add additional variants if there is strong need. In this case, is a variant that allows the intermediate buffer to be specified be something that would be widely used? >> >> -Alan > > In our codebase here we have variants of different buffer sizes, mostly there are some with smaller sizes than the default I took from the Files class. To have the copy function within the JDK instead using a separate library or have this code written over and over again is much more important to me. > > -Patrick From martinrb at google.com Thu Nov 27 19:21:18 2014 From: martinrb at google.com (Martin Buchholz) Date: Thu, 27 Nov 2014 11:21:18 -0800 Subject: RFR: 8062771: Core reflection should use final fields whenever possible In-Reply-To: <5476CBEA.3060503@oracle.com> References: <843282E1-DC2B-48B0-B0E4-A60B5078D75A@oracle.com> <45A50B54-43A3-4134-9EB5-D1AEDA649587@oracle.com> <545E9293.5010203@oracle.com> <20141110090015.GA22573@oracle.com> <5476CBEA.3060503@oracle.com> Message-ID: Approved! Making fields final is super-safe. On Wed, Nov 26, 2014 at 10:59 PM, Ivan Gerasimov wrote: > Hi! > > I can take care of jdk8 backport too, if you want. > I've checked it, and the port seemed quite straight-forward (modulo > unshuffling). > > Only changes to a few files [1] had to be skipped, as the changes appear to > already be there in jdk8u. > > Here's the webrev of jdk8u backport for JDK-8062771: > > WEBREV: http://cr.openjdk.java.net/~igerasim/8062771/0/webrev/ > > > [1] And here's the list of the files that needed to be excluded from the > changeset: > src/share/classes/sun/reflect/generics/factory/CoreReflectionFactory.java > src/share/classes/sun/reflect/generics/reflectiveObjects/LazyReflectiveObjectGenerator.java > src/share/classes/sun/reflect/generics/repository/AbstractRepository.java > src/share/classes/sun/reflect/generics/scope/AbstractScope.java > src/share/classes/sun/reflect/generics/tree/ClassSignature.java > > Sincerely yours, > Ivan > > > > On 10.11.2014 12:00, Joel Borggr?n-Franck wrote: >> >> That would be great >> >> cheers >> /Joel >> >> On 2014-11-09, Ivan Gerasimov wrote: >>> >>> Hi! >>> >>> I think I can handle the backport to jdk7, if you still need a volunteer. >>> >>> Sincerely yours, >>> Ivan >>> >>> On 08.11.2014 1:48, Martin Buchholz wrote: >>>> >>>> Hi Joel, >>>> >>>> Thanks for volunteering. I foisted all I have in >>>> >>>> https://bugs.openjdk.java.net/browse/JDK-8064391 >>>> >>>> I volunteer to be your reviewer for the backports. >>>> >>>> On Fri, Nov 7, 2014 at 1:36 PM, Joel Borggr?n-Franck >>>> wrote: >>>>> >>>>> Hi Martin, >>>>> >>>>> Thanks for the clarification. >>>>> >>>>> On 6 nov 2014, at 20:51, Martin Buchholz wrote: >>>>> >>>>>> Hi Joel, >>>>>> >>>>>> On Thu, Nov 6, 2014 at 2:48 AM, Joel Borggr?n-Franck >>>>>> wrote: >>>>>>> >>>>>>> Hi, >>>>>>> >>>>>>> I?m having a hard time following this thread, which webrev has been >>>>>>> updated, for which release, fixing which issue? >>>>>> >>>>>> I am submitting the changeset for JDK-8062771, adding the finals and >>>>>> the (failed reproduction) test. >>>>>> Hopefully there will be followon changes to add even more thread >>>>>> safety. >>>>>> >>>>> Great. >>>>> >>>>> Are we sure the accidental fix of making ClassRepository volatile is >>>>> good on all platforms? I can?t establish the happens before relations that >>>>> would prove the fix but then again I?m not an expert on concurrency. >>>>> >>>>>>> Martin, as far as I can see you are the only one of us who has >>>>>>> replied to this thread who is a jdk7u committer (or reviewer). >>>>>> >>>>>> I am entirely in favor of having more reviewers. >>>>>> >>>>>> We are not planning to push these into jdk7u or jdk8u ourselves, but >>>>>> we are willing to help anyone who wants to take on that task. Joel, >>>>>> are you volunteering? >>>>>> We believe all versions of stock openjdk still have (rarely seen) >>>>>> thread safety issues in core reflection. We have applied changes >>>>>> locally at Google to fix those. >>>>>> >>>>> I?ll take care of 8u, and I can hunt down someone to fix this in 7u if >>>>> we can find a suitable reviewer (sigh). >>>>> >>>>> cheers >>>>> /Joel >>>>> >>>>> >> > From patrick at reini.net Thu Nov 27 22:59:04 2014 From: patrick at reini.net (Patrick Reinhart) Date: Thu, 27 Nov 2014 23:59:04 +0100 Subject: Library enhancement proposal for copying data from/to Reader/Writer InputStream/OutputStream In-Reply-To: References: <566A428C-1A13-4DEE-940B-2EBE375F2A7A@reini.net> <51329A99-55E5-4A42-9180-D872366B4DEE@oracle.com> <546EABED.9030301@oracle.com> <546EF98C.40704@oracle.com> <1D392C1F-E6CC-494D-97F8-C08C5BA36392@oracle.com> <54772C7D.7080209@oracle.com> <769E83F8-FD6D-4F8B-B415-DD044C5D08BA@reini.net> <547757EE.4090205@oracle.com> Message-ID: <15B58029-95FA-4D9E-AA28-7A6C72D6CB3D@reini.net> > Patrick, Alan is right in the sense that we have to estimate usefulness of each method wisely. Unfortunately your codebase (or anyone else?s) might have some very specific biases and thus might not be a good sample to base conclusions upon. Aggreed. > We need some more data to justify this proposal. The good start could be the jdk itself -- to see if those copy-with-my-buffer methods could be useful here. > > -Pavel I have to say, that I normally use my IDE to search for references. What technique do you use to search for such patterns? -Patrick From joel.franck at oracle.com Fri Nov 28 09:02:57 2014 From: joel.franck at oracle.com (=?utf-8?Q?Joel_Borggr=C3=A9n-Franck?=) Date: Fri, 28 Nov 2014 10:02:57 +0100 Subject: RFR: 8062771: Core reflection should use final fields whenever possible In-Reply-To: References: <843282E1-DC2B-48B0-B0E4-A60B5078D75A@oracle.com> <45A50B54-43A3-4134-9EB5-D1AEDA649587@oracle.com> <545E9293.5010203@oracle.com> <20141110090015.GA22573@oracle.com> <5476CBEA.3060503@oracle.com> Message-ID: <02EA0B10-3836-470E-AD49-0F59C0E29F49@oracle.com> Hi, > On 27 nov 2014, at 20:21, Martin Buchholz wrote: > > Approved! Making fields final is super-safe. > I agree. All fields are private (or package private for Label) so this is safe. +1 cheers /Joel > On Wed, Nov 26, 2014 at 10:59 PM, Ivan Gerasimov > wrote: >> Hi! >> >> I can take care of jdk8 backport too, if you want. >> I've checked it, and the port seemed quite straight-forward (modulo >> unshuffling). >> >> Only changes to a few files [1] had to be skipped, as the changes appear to >> already be there in jdk8u. >> >> Here's the webrev of jdk8u backport for JDK-8062771: >> >> WEBREV: http://cr.openjdk.java.net/~igerasim/8062771/0/webrev/ >> >> >> [1] And here's the list of the files that needed to be excluded from the >> changeset: >> src/share/classes/sun/reflect/generics/factory/CoreReflectionFactory.java >> src/share/classes/sun/reflect/generics/reflectiveObjects/LazyReflectiveObjectGenerator.java >> src/share/classes/sun/reflect/generics/repository/AbstractRepository.java >> src/share/classes/sun/reflect/generics/scope/AbstractScope.java >> src/share/classes/sun/reflect/generics/tree/ClassSignature.java >> >> Sincerely yours, >> Ivan >> >> >> >> On 10.11.2014 12:00, Joel Borggr?n-Franck wrote: >>> >>> That would be great >>> >>> cheers >>> /Joel >>> >>> On 2014-11-09, Ivan Gerasimov wrote: >>>> >>>> Hi! >>>> >>>> I think I can handle the backport to jdk7, if you still need a volunteer. >>>> >>>> Sincerely yours, >>>> Ivan >>>> >>>> On 08.11.2014 1:48, Martin Buchholz wrote: >>>>> >>>>> Hi Joel, >>>>> >>>>> Thanks for volunteering. I foisted all I have in >>>>> >>>>> https://bugs.openjdk.java.net/browse/JDK-8064391 >>>>> >>>>> I volunteer to be your reviewer for the backports. >>>>> >>>>> On Fri, Nov 7, 2014 at 1:36 PM, Joel Borggr?n-Franck >>>>> wrote: >>>>>> >>>>>> Hi Martin, >>>>>> >>>>>> Thanks for the clarification. >>>>>> >>>>>> On 6 nov 2014, at 20:51, Martin Buchholz wrote: >>>>>> >>>>>>> Hi Joel, >>>>>>> >>>>>>> On Thu, Nov 6, 2014 at 2:48 AM, Joel Borggr?n-Franck >>>>>>> wrote: >>>>>>>> >>>>>>>> Hi, >>>>>>>> >>>>>>>> I?m having a hard time following this thread, which webrev has been >>>>>>>> updated, for which release, fixing which issue? >>>>>>> >>>>>>> I am submitting the changeset for JDK-8062771, adding the finals and >>>>>>> the (failed reproduction) test. >>>>>>> Hopefully there will be followon changes to add even more thread >>>>>>> safety. >>>>>>> >>>>>> Great. >>>>>> >>>>>> Are we sure the accidental fix of making ClassRepository volatile is >>>>>> good on all platforms? I can?t establish the happens before relations that >>>>>> would prove the fix but then again I?m not an expert on concurrency. >>>>>> >>>>>>>> Martin, as far as I can see you are the only one of us who has >>>>>>>> replied to this thread who is a jdk7u committer (or reviewer). >>>>>>> >>>>>>> I am entirely in favor of having more reviewers. >>>>>>> >>>>>>> We are not planning to push these into jdk7u or jdk8u ourselves, but >>>>>>> we are willing to help anyone who wants to take on that task. Joel, >>>>>>> are you volunteering? >>>>>>> We believe all versions of stock openjdk still have (rarely seen) >>>>>>> thread safety issues in core reflection. We have applied changes >>>>>>> locally at Google to fix those. >>>>>>> >>>>>> I?ll take care of 8u, and I can hunt down someone to fix this in 7u if >>>>>> we can find a suitable reviewer (sigh). >>>>>> >>>>>> cheers >>>>>> /Joel >>>>>> >>>>>> >>> >> From amy.lu at oracle.com Fri Nov 28 10:35:15 2014 From: amy.lu at oracle.com (Amy Lu) Date: Fri, 28 Nov 2014 18:35:15 +0800 Subject: RFR: Update java/nio/charset/Charset/NIOCharsetAvailabilityTest.java to eliminate dependency on sun.misc.Launcher Message-ID: <54784FE3.2040508@oracle.com> jdk/test/java/nio/charset/Charset/NIOCharsetAvailabilityTest.java has dependency on sun.misc.Launcher This fix is to remove above internal API dependency from the test. bug: https://bugs.openjdk.java.net/browse/JDK-8066131 webrev: http://cr.openjdk.java.net/~weijun/8066131/webrev.00/ Thanks, Amy From pavel.rappo at oracle.com Fri Nov 28 12:09:11 2014 From: pavel.rappo at oracle.com (Pavel Rappo) Date: Fri, 28 Nov 2014 12:09:11 +0000 Subject: Library enhancement proposal for copying data from/to Reader/Writer InputStream/OutputStream In-Reply-To: <15B58029-95FA-4D9E-AA28-7A6C72D6CB3D@reini.net> References: <566A428C-1A13-4DEE-940B-2EBE375F2A7A@reini.net> <51329A99-55E5-4A42-9180-D872366B4DEE@oracle.com> <546EABED.9030301@oracle.com> <546EF98C.40704@oracle.com> <1D392C1F-E6CC-494D-97F8-C08C5BA36392@oracle.com> <54772C7D.7080209@oracle.com> <769E83F8-FD6D-4F8B-B415-DD044C5D08BA@reini.net> <547757EE.4090205@oracle.com> <15B58029-95FA-4D9E-AA28-7A6C72D6CB3D@reini.net> Message-ID: <13156FAC-6EDC-462B-9BE7-2DFAFB9124FD@oracle.com> I'm happy with any tool which is capable of "usages search". Configured IDE is a good choice. -Pavel > On 27 Nov 2014, at 22:59, Patrick Reinhart wrote: > > I have to say, that I normally use my IDE to search for references. > > What technique do you use to search for such patterns? From hendrikdev22 at gmail.com Fri Nov 28 12:40:58 2014 From: hendrikdev22 at gmail.com (Hendrik Dev) Date: Fri, 28 Nov 2014 13:40:58 +0100 Subject: JEP 198 JDK-8046390 involvement Message-ID: I'd like to get involved in this JEP. When active development will be kicked off? Is it enough to read this mailing list or are other sources more recommended and is http://hg.openjdk.java.net/jdk9/dev/jdk/ the correct repo? Thanks Hendrik -- Hendrik Saly (salyh, hendrikdev22) @hendrikdev22 PGP: 0x22D7F6EC From Alan.Bateman at oracle.com Fri Nov 28 13:25:17 2014 From: Alan.Bateman at oracle.com (Alan Bateman) Date: Fri, 28 Nov 2014 13:25:17 +0000 Subject: RFR: Update java/nio/charset/Charset/NIOCharsetAvailabilityTest.java to eliminate dependency on sun.misc.Launcher In-Reply-To: <54784FE3.2040508@oracle.com> References: <54784FE3.2040508@oracle.com> Message-ID: <547877BD.3070400@oracle.com> On 28/11/2014 10:35, Amy Lu wrote: > jdk/test/java/nio/charset/Charset/NIOCharsetAvailabilityTest.java has > dependency on sun.misc.Launcher > This fix is to remove above internal API dependency from the test. > > bug: https://bugs.openjdk.java.net/browse/JDK-8066131 > webrev: http://cr.openjdk.java.net/~weijun/8066131/webrev.00/ The change looks okay but I think the test needs a bigger overhaul. The main thing is that the extensions directory goes away via JEP 220 so I think some of this test can go away too. The other thing is that getClassList appears to treat URL paths as file paths so it might not never have worked as expected on all platforms. -Alan. From ivan.gerasimov at oracle.com Fri Nov 28 13:37:59 2014 From: ivan.gerasimov at oracle.com (Ivan Gerasimov) Date: Fri, 28 Nov 2014 16:37:59 +0300 Subject: [7u-dev] Request for Review + Request for Approval to Backport: 8064391: More thread safety problems in core reflection Message-ID: <54787AB7.2070604@oracle.com> Hello! Would you please approve the backport from jdk 8u to 7u? The patch didn't apply cleanly due to generics in these two files: src/share/classes/sun/reflect/BootstrapConstructorAccessorImpl.java src/share/classes/sun/reflect/NativeConstructorAccessorImpl.java All the other changes are identical. Bug: https://bugs.openjdk.java.net/browse/JDK-8062771 Jdk9 changeset: http://hg.openjdk.java.net/jdk9/dev/jdk/rev/1e5c2bd424d5 Jdk 8u changeset: http://hg.openjdk.java.net/jdk8u/jdk8u-dev/jdk/rev/fc4f55464170 Webrev: http://cr.openjdk.java.net/~igerasim/8062771/1/webrev/ Jdk 9 Review: http://mail.openjdk.java.net/pipermail/core-libs-dev/2014-November/029485.html Jdk 8u Review: http://mail.openjdk.java.net/pipermail/core-libs-dev/2014-November/029901.html Sincerely yours, Ivan From sean.coffey at oracle.com Fri Nov 28 14:10:15 2014 From: sean.coffey at oracle.com (=?UTF-8?B?U2XDoW4gQ29mZmV5?=) Date: Fri, 28 Nov 2014 14:10:15 +0000 Subject: [7u-dev] Request for Review + Request for Approval to Backport: 8064391: More thread safety problems in core reflection In-Reply-To: <54787AB7.2070604@oracle.com> References: <54787AB7.2070604@oracle.com> Message-ID: <54788247.1090507@oracle.com> Looks fine to me Ivan. Please get jdk7u reviewer to review. Approved. regards, Sean. On 28/11/2014 13:37, Ivan Gerasimov wrote: > Hello! > > Would you please approve the backport from jdk 8u to 7u? > > The patch didn't apply cleanly due to generics in these two files: > src/share/classes/sun/reflect/BootstrapConstructorAccessorImpl.java > src/share/classes/sun/reflect/NativeConstructorAccessorImpl.java > > All the other changes are identical. > > Bug: https://bugs.openjdk.java.net/browse/JDK-8062771 > Jdk9 changeset: http://hg.openjdk.java.net/jdk9/dev/jdk/rev/1e5c2bd424d5 > Jdk 8u changeset: > http://hg.openjdk.java.net/jdk8u/jdk8u-dev/jdk/rev/fc4f55464170 > Webrev: http://cr.openjdk.java.net/~igerasim/8062771/1/webrev/ > Jdk 9 Review: > http://mail.openjdk.java.net/pipermail/core-libs-dev/2014-November/029485.html > Jdk 8u Review: > http://mail.openjdk.java.net/pipermail/core-libs-dev/2014-November/029901.html > > Sincerely yours, > Ivan > > > From lance.andersen at oracle.com Fri Nov 28 18:01:33 2014 From: lance.andersen at oracle.com (Lance Andersen) Date: Fri, 28 Nov 2014 13:01:33 -0500 Subject: RFR 8066188: BaseRowSet default value for escape processing is not correct Message-ID: Hi all, Attached is a trivial change to address the default value for escape processing for BaseRowSet. This aligns with the javadocs now for BaseRowSet as well as the default for escape process per the JDBC spec and other RowSets. This also uncovered a JCK bug which I have made the JCK team aware of. The change is trivial and I have included the diff: hg diff diff -r 62fe5a5e2bae src/java.sql.rowset/share/classes/javax/sql/rowset/BaseRowSet.java --- a/src/java.sql.rowset/share/classes/javax/sql/rowset/BaseRowSet.java Tue Nov 11 20:11:34 2014 +0000 +++ b/src/java.sql.rowset/share/classes/javax/sql/rowset/BaseRowSet.java Fri Nov 28 12:54:05 2014 -0500 @@ -462,7 +462,7 @@ * false that it is not. The default is true. * @serial */ - private boolean escapeProcessing; + private boolean escapeProcessing = true; /** * A constant indicating the isolation level of the connection diff -r 62fe5a5e2bae test/javax/sql/testng/test/rowset/BaseRowSetTests.java --- a/test/javax/sql/testng/test/rowset/BaseRowSetTests.java Tue Nov 11 20:11:34 2014 +0000 +++ b/test/javax/sql/testng/test/rowset/BaseRowSetTests.java Fri Nov 28 12:54:05 2014 -0500 @@ -186,11 +186,11 @@ } /* - * Validate that getEscapeProcessing() returns false by default + * Validate that getEscapeProcessing() returns true by default */ @Test public void test08() throws Exception { - assertFalse(brs.getEscapeProcessing()); + assertTrue(brs.getEscapeProcessing()); } Best, Lance Lance Andersen| Principal Member of Technical Staff | +1.781.442.2037 Oracle Java Engineering 1 Network Drive Burlington, MA 01803 Lance.Andersen at oracle.com From martinrb at google.com Fri Nov 28 18:08:23 2014 From: martinrb at google.com (Martin Buchholz) Date: Fri, 28 Nov 2014 10:08:23 -0800 Subject: [7u-dev] Request for Review + Request for Approval to Backport: 8064391: More thread safety problems in core reflection In-Reply-To: <54787AB7.2070604@oracle.com> References: <54787AB7.2070604@oracle.com> Message-ID: Approved! On Fri, Nov 28, 2014 at 5:37 AM, Ivan Gerasimov wrote: > Hello! > > Would you please approve the backport from jdk 8u to 7u? > > The patch didn't apply cleanly due to generics in these two files: > src/share/classes/sun/reflect/BootstrapConstructorAccessorImpl.java > src/share/classes/sun/reflect/NativeConstructorAccessorImpl.java > > All the other changes are identical. > > Bug: https://bugs.openjdk.java.net/browse/JDK-8062771 > Jdk9 changeset: http://hg.openjdk.java.net/jdk9/dev/jdk/rev/1e5c2bd424d5 > Jdk 8u changeset: > http://hg.openjdk.java.net/jdk8u/jdk8u-dev/jdk/rev/fc4f55464170 > Webrev: http://cr.openjdk.java.net/~igerasim/8062771/1/webrev/ > Jdk 9 Review: > http://mail.openjdk.java.net/pipermail/core-libs-dev/2014-November/029485.html > Jdk 8u Review: > http://mail.openjdk.java.net/pipermail/core-libs-dev/2014-November/029901.html > > Sincerely yours, > Ivan > > > From ecki at zusammenkunft.net Fri Nov 28 19:15:46 2014 From: ecki at zusammenkunft.net (Bernd Eckenfels) Date: Fri, 28 Nov 2014 20:15:46 +0100 Subject: Library enhancement proposal for copying data from/to Reader/Writer InputStream/OutputStream In-Reply-To: <13156FAC-6EDC-462B-9BE7-2DFAFB9124FD@oracle.com> References: <566A428C-1A13-4DEE-940B-2EBE375F2A7A@reini.net> <51329A99-55E5-4A42-9180-D872366B4DEE@oracle.com> <546EABED.9030301@oracle.com> <546EF98C.40704@oracle.com> <1D392C1F-E6CC-494D-97F8-C08C5BA36392@oracle.com> <54772C7D.7080209@oracle.com> <769E83F8-FD6D-4F8B-B415-DD044C5D08BA@reini.net> <547757EE.4090205@oracle.com> <15B58029-95FA-4D9E-AA28-7A6C72D6CB3D@reini.net> <13156FAC-6EDC-462B-9BE7-2DFAFB9124FD@oracle.com> Message-ID: <20141128201546.000067ce.ecki@zusammenkunft.net> Am Fri, 28 Nov 2014 12:09:11 +0000 schrieb Pavel Rappo : > I'm happy with any tool which is capable of "usages search". > Configured IDE is a good choice. What usage would you actually search in the OpenJDK code base? I think it is not very representative to search for usage in the JDK code base even if it would be possible. However you can use for commons IO or Guava. Both have those helpers. (Which initself might be justification enough - the question is only pro or con including it in JCL :) Gruss Bernd From patrick at reini.net Fri Nov 28 20:13:10 2014 From: patrick at reini.net (Patrick Reinhart) Date: Fri, 28 Nov 2014 21:13:10 +0100 Subject: Library enhancement proposal for copying data from/to Reader/Writer InputStream/OutputStream In-Reply-To: <20141128201546.000067ce.ecki@zusammenkunft.net> References: <566A428C-1A13-4DEE-940B-2EBE375F2A7A@reini.net> <51329A99-55E5-4A42-9180-D872366B4DEE@oracle.com> <546EABED.9030301@oracle.com> <546EF98C.40704@oracle.com> <1D392C1F-E6CC-494D-97F8-C08C5BA36392@oracle.com> <54772C7D.7080209@oracle.com> <769E83F8-FD6D-4F8B-B415-DD044C5D08BA@reini.net> <547757EE.4090205@oracle.com> <15B58029-95FA-4D9E-AA28-7A6C72D6CB3D@reini.net> <13156FAC-6EDC-462B-9BE7-2DFAFB9124FD@oracle.com> <20141128201546.000067ce.ecki@zusammenkunft.net> Message-ID: > Am 28.11.2014 um 20:15 schrieb Bernd Eckenfels : > > Am Fri, 28 Nov 2014 12:09:11 +0000 > schrieb Pavel Rappo : > >> I'm happy with any tool which is capable of "usages search". >> Configured IDE is a good choice. > > What usage would you actually search in the OpenJDK code base? > > I think it is not very representative to search for usage in the JDK > code base even if it would be possible. However you can use for commons > IO or Guava. Both have those helpers. (Which initself might be > justification enough - the question is only pro or con including it in > JCL :) > > Gruss > Bernd When looking at the guava library there seems to be no need for copy method that is able to pass in a separate byte array or CharBuffer. It would make more sense to have also a copy channels method on the existing java.nio.channels.Channels class instead. In all the projects I have worked on, there was always the point of having to either implement my own copy methods or use a certain library (for mostly only using the copy methods). Having said that, it seems to be a natural fit. Also when looking on the java.nio.files.Files class for copying with source or target Path?s it makes even more sense. -Patrick From pavel.rappo at oracle.com Fri Nov 28 20:35:16 2014 From: pavel.rappo at oracle.com (Pavel Rappo) Date: Fri, 28 Nov 2014 20:35:16 +0000 Subject: Library enhancement proposal for copying data from/to Reader/Writer InputStream/OutputStream In-Reply-To: <20141128201546.000067ce.ecki@zusammenkunft.net> References: <566A428C-1A13-4DEE-940B-2EBE375F2A7A@reini.net> <51329A99-55E5-4A42-9180-D872366B4DEE@oracle.com> <546EABED.9030301@oracle.com> <546EF98C.40704@oracle.com> <1D392C1F-E6CC-494D-97F8-C08C5BA36392@oracle.com> <54772C7D.7080209@oracle.com> <769E83F8-FD6D-4F8B-B415-DD044C5D08BA@reini.net> <547757EE.4090205@oracle.com> <15B58029-95FA-4D9E-AA28-7A6C72D6CB3D@reini.net> <13156FAC-6EDC-462B-9BE7-2DFAFB9124FD@oracle.com> <20141128201546.000067ce.ecki@zusammenkunft.net> Message-ID: <216A14EF-B62C-4971-8194-EFFBD30933B0@oracle.com> > What usage would you actually search in the OpenJDK code base? We're talking about copying with provided buffer, right? long copy(InputStream source, OutputStream target, byte[] buffer) ^^^^^^^^^^^^^ It would be very helpful to verify that this method is needed extremely rare, if at all. In our case it would be an array of very specific (other than 1024, 2048, 4096 or 8192) size, i.e. unusually large or small or heavily reused one. To copy from an input stream to an output stream you need to try to read a byte from the input stream, at least once. Therefore search for usages of these methods will be a good start: java.io.InputStream#read(byte[]) java.io.InputStream#read(byte[], int, int) -Pavel From Alan.Bateman at oracle.com Sat Nov 29 10:12:59 2014 From: Alan.Bateman at oracle.com (Alan Bateman) Date: Sat, 29 Nov 2014 10:12:59 +0000 Subject: RFR 8066188: BaseRowSet default value for escape processing is not correct In-Reply-To: References: Message-ID: <54799C2B.2090401@oracle.com> On 28/11/2014 18:01, Lance Andersen wrote: > Hi all, > > Attached is a trivial change to address the default value for escape processing for BaseRowSet. This aligns with the javadocs now for BaseRowSet as well as the default for escape process per the JDBC spec and other RowSets. > > The javadoc/spec is clear that escape processing is enabled by default so I think this change is okay (I'm ignoring the issue that it's not volatile and there's no synchronization, the assumption being that it is not documented as being thread safe). -Alan. From lance.andersen at oracle.com Sat Nov 29 12:34:09 2014 From: lance.andersen at oracle.com (Lance @ Oracle) Date: Sat, 29 Nov 2014 07:34:09 -0500 Subject: RFR 8066188: BaseRowSet default value for escape processing is not correct In-Reply-To: <54799C2B.2090401@oracle.com> References: <54799C2B.2090401@oracle.com> Message-ID: <9ABD878C-1499-45EF-B93D-396E56EFBAB0@oracle.com> Thank you Alan. Yes, the spec is silent on thread safety. Once I finish with adding tests, I will look to update the javadocs to clarify this. Best, Lance Lance Andersen| Principal Member of Technical Staff | +1.781.442.2037 Oracle Java Engineering 1 Network Drive Burlington, MA 01803 Lance.Andersen at oracle.com Sent from my iPad > On Nov 29, 2014, at 5:12 AM, Alan Bateman wrote: > >> On 28/11/2014 18:01, Lance Andersen wrote: >> Hi all, >> >> Attached is a trivial change to address the default value for escape processing for BaseRowSet. This aligns with the javadocs now for BaseRowSet as well as the default for escape process per the JDBC spec and other RowSets. >> >> > The javadoc/spec is clear that escape processing is enabled by default so I think this change is okay (I'm ignoring the issue that it's not volatile and there's no synchronization, the assumption being that it is not documented as being thread safe). > > -Alan. From igor.ignatyev at oracle.com Sat Nov 29 16:36:29 2014 From: igor.ignatyev at oracle.com (Igor Ignatyev) Date: Sat, 29 Nov 2014 19:36:29 +0300 Subject: RFR(S) : 8039953 : [TESTBUG] Timeout java/lang/invoke/MethodHandles/CatchExceptionTest.java Message-ID: <5479F60D.70403@oracle.com> http://cr.openjdk.java.net/~iignatyev/8039953/webrev.00 98 lines changed: 93 ins; 3 del; 2 mod; Hi all, Please review the patch: Problem: on some configurations, java/lang/invoke/MethodHandles/CatchExceptionTest.java can timeout before all test cases are run Fix: interrupt test execution if it's not enough time to continue bug : https://bugs.openjdk.java.net/browse/JDK-8039953 changes in testlibrary : https://bugs.openjdk.java.net/browse/JDK-8066191 testing: locally -- Igor From peter.levart at gmail.com Sun Nov 30 21:48:16 2014 From: peter.levart at gmail.com (Peter Levart) Date: Sun, 30 Nov 2014 22:48:16 +0100 Subject: RFR: 8061950: Class.getMethods() exhibits quadratic time complexity In-Reply-To: References: <900D2AE3-B3A6-4B1B-B25A-C564B10334AA@oracle.com> <54492255.3040108@gmail.com> <544D58D4.7020909@gmail.com> <544D7B74.5060607@gmail.com> <5450CEE9.7060707@gmail.com> <545120D2.5010301@gmail.com> <54526D10.5060101@gmail.com> <54551D2F.407@gmail.com> <545A572D.3050608@gmail.com> Message-ID: <547B90A0.1080508@gmail.com> Hi Joel, I managed to find some time to create some tests for this patch: http://cr.openjdk.java.net/~plevart/jdk9-dev/Class.getMethods/webrev.07/ Both MethodTable and HashArray unit tests are provided. I had to create a special TestProxy to access package-private classes from the tests. There are no changes to j.l.Class or j.l.r.Method from webrev.06 (I just re-based them to current tip). I also included the patch to StarInheritance test that I forgot to include in webrev.06. Comments inline... On 11/13/2014 10:39 AM, Joel Borggr?n-Franck wrote: > Hi Peter, > > As always, thanks for taking a look at this, > > This is quite big so in order to make this more approachable perhaps you can split the patch up into a series? If you start with creating the MethodTable interface, adding tests for how the interface should behave and refactored the current MethodArray into implementing that interface while also changing the lookup logic that would be easier to review. Well, there's not much to refactor in MethodArray when implementing MethodTable. They are two entirely different APIs with entirely different implementations. > Then you could add different implementations of MethodTable (with additional unit tests) as follow up patches. You can view the MethodTable.SimpleArrayImpl as the basic implementation of the MethodTable API and a replacement for MethodArray. MethodTable.HashArrayImpl is the alternative implementation for bigger sizes. The same unit tests are executed against both implementations. > I am a bit concerned about the size and scope of the implementations. In general I would prefer if you targeted these to the precise need of core reflection today. If you want to expand these to general purpose data structures (even internal ones) I think that is a larger effort. I stripped HashArray and only left those methods that are needed to implement MethodTable API and execute the tests. > In general I think the changes to Class are sound, but there is a slight change in the default method pruning. The call to removeLessSpecifics was deliberately placed at the end, so that all default methods would be present (the algorithm is sensitive to the order of pair vise comparisons). Since we add methods in a deterministic order, I think consolidate() as you go should result in the same set of methods, but I haven?t 100% convinced myself of this just yet. I think it results in the same methods. I haven't yet found an example where it would result in different set of methods. All JDK classes return same methods with current implementation as with patched one. > Have you double checked that all methods returning root Method/Ctors are private? I checked all usages of private methods that I have changed and are now returning root objects and made sure those are copied before being exposed to the outside or being modified. Regards, Peter > On 5 nov 2014, at 17:58, Peter Levart wrote: > >> Here's new webrev: >> >> http://cr.openjdk.java.net/~plevart/jdk9-dev/Class.getMethods/webrev.06/ >> >> >> The optimizations made from webrev.05 are: >> >> - getMethod() skips construction of MethodTable if there are no (super)interfaces. >> - getMethods() returns just declared public methods if there are no superclass and no (super)interfaces. >> - comparing method parameter types is optimized by adding two methods to Method/LangReflectAccess/ReflectionFactory. >> >> New MethodTable implementation based on a linear-probe hash table is a space/garbage improvement. I took IdentityHashMap, removed unneeded stuff and modified it's API. The result is a HashArray. It's API is similar in function and form to java.util.Map, but doesn't use separate keys and values. An element of HashArray is a key and a value at the same time. Elements are always non-null, so the method return values are unambiguous. As HashArray is a linear-probe hash table and there are no Map.Entry objects involved, the underlying data structure is very simple and memory efficient. It is just a sparse array of elements with length that is always a power of two and larger than 3 * size / 2. It also features overriddable element equals/hashCode methods. I made it a separate generic class because I think it can find it's usage elsewhere (for example as a cannonicalizing cache). >> >> Since HashArray based MethodTable is more space-efficient I moved the line between simple array based and HashArray based MethodTable down to 20 elements to minimize the worst-case scenario effect. Calling getMethods() on all rt.jar classes now constructs about 3/4 simple array based and 1/4 HashArray based MethodTables. >> > HashArray.java: > > I was hoping for a decent set of unit tests for the new HashArray data structure. I think it is reasonable to test the corner cases/non-trivial areas of the table (closeDeletion(), rezise() etc). Perhaps also run these over the simple implementation. Also, please document thread safety (there is none IFAICT it should just be noted). > > Instead of using inheritance to change the behavior of equals() and hash() you could give it two lambdas at table creation time, a ToIntFunction for hash() and a BiPredicate for equals(). Might not give you the performance we need though. > > Note that the file doesn?t actually compile in jdk9/dev, you have two unchecked casts and we build with -Werror. > > MethodTable.java > > HashMapImpl is missing serialVersionUID, but it looks like this class won?t be needed at all. > > >> Here's also Martin's ManyMethodsBenchmark: >> >> Original: >> >> Base class load time: 129.95 ms >> getDeclaredMethods: 65521 methods, 36.58 ms total time, 0.0006 ms per method >> getMethods : 65530 methods, 47.43 ms total time, 0.0007 ms per method >> Derived class load time: 32216.09 ms >> getDeclaredMethods: 65521 methods, 35.05 ms total time, 0.0005 ms per method >> getMethods : 65530 methods, 8068.66 ms total time, 0.1231 ms per method >> >> >> Patched (using HashArray based MethodTable): >> >> Base class load time: 126.00 ms >> getDeclaredMethods: 65521 methods, 36.83 ms total time, 0.0006 ms per method >> getMethods : 65530 methods, 45.08 ms total time, 0.0007 ms per method >> Derived class load time: 31865.27 ms >> getDeclaredMethods: 65521 methods, 35.01 ms total time, 0.0005 ms per method >> getMethods : 65530 methods, 78.05 ms total time, 0.0012 ms per method >> >> >> All 86 jtreg test in java.lang/Class/ and java/lang/reflect/ still pass. >> > I have seen discussion about allocation, should we measure and compare? You can probably use the Hotspot impl of ThreadMXBean to get the allocation in the tread. > > Also, it might be time to fix the boolean parameters: > > 2741 Method[] declaredMethods = privateGetDeclaredMethods(true); > 2742 Class superclass = getSuperclass(); > 2743 Class[] interfaces = getInterfaces(false); > > Perhaps just add boolean constants somewhere so that it is easier to decode. > > 2741 Method[] declaredMethods = privateGetDeclaredMethods(PUBLIC_METHOD_ONLY); > 2742 Class superclass = getSuperclass(); > 2743 Class[] interfaces = getInterfaces(NO_COPY_RESULT); > > or so. > > HashArray.java: > > 155 if (lookupObj == null) throw new NullPointerException(); > > use Objects.requreNonNull() ? > > cheers > /Joel > From alexyursha at gmail.com Sat Nov 29 16:13:19 2014 From: alexyursha at gmail.com (Alex Yursha) Date: Sat, 29 Nov 2014 19:13:19 +0300 Subject: Packing 2 data points into 1 field in ThreadPoolExecutor In-Reply-To: References: Message-ID: <0F081861-ACBA-484E-9FA7-784D2EE4A023@icloud.com> Hi all, According to javadoc current implementation of ThreadPoolExecutor packs two conceptual fields ?workerCount? and ?runState? into one actual field ?ctl? of type AtomicInteger. Could you please explain are there any performance or other benefits for this? It seems to complicate the class design and I can?t find the positive side of this. Thanks, Alex From dt at flyingtroika.com Tue Nov 25 18:16:22 2014 From: dt at flyingtroika.com (DT) Date: Tue, 25 Nov 2014 10:16:22 -0800 Subject: [concurrency-interest] RFR: 8065804: JEP 171: Clarifications/corrections for fence intrinsics In-Reply-To: References: <5473A344.8090302@oracle.com> Message-ID: <21DC5F06-9597-4CCB-8A25-5CC354A0BCE5@flyingtroika.com> I see time to time comments in the jvm sources referencing membars and fences. Would you say that they are used interchangeably ? Having the same meaning but for different CPU arch. Sent from my iPhone > On Nov 25, 2014, at 6:04 AM, Paul Sandoz wrote: > > Hi Martin, > > Thanks for looking into this. > > 1141 * Currently hotspot's implementation of a Java language-level volatile > 1142 * store has the same effect as a storeFence followed by a relaxed store, > 1143 * although that may be a little stronger than needed. > > IIUC to emulate hotpot's volatile store you will need to say that a fullFence immediately follows the relaxed store. > > The bit that always confuses me about release and acquire is ordering is restricted to one direction, as talked about in orderAccess.hpp [1]. So for a release, accesses prior to the release cannot move below it, but accesses succeeding the release can move above it. And that seems to apply to Unsafe.storeFence [2] (acting like a monitor exit). Is that contrary to C++ release fences where ordering is restricted both to prior and succeeding accesses? [3] > > So what about the following? > > a = r1; // Cannot move below the fence > Unsafe.storeFence(); > b = r2; // Can move above the fence? > > Paul. > > [1] In orderAccess.hpp > // Execution by a processor of release makes the effect of all memory > // accesses issued by it previous to the release visible to all > // processors *before* the release completes. The effect of subsequent > // memory accesses issued by it *may* be made visible *before* the > // release. I.e., subsequent memory accesses may float above the > // release, but prior ones may not float below it. > > [2] In memnode.hpp > // "Release" - no earlier ref can move after (but later refs can move > // up, like a speculative pipelined cache-hitting Load). Requires > // multi-cpu visibility. Inserted independent of any store, as required > // for intrinsic sun.misc.Unsafe.storeFence(). > class StoreFenceNode: public MemBarNode { > public: > StoreFenceNode(Compile* C, int alias_idx, Node* precedent) > : MemBarNode(C, alias_idx, precedent) {} > virtual int Opcode() const; > }; > > [3] http://preshing.com/20131125/acquire-and-release-fences-dont-work-the-way-youd-expect/ > >> On Nov 25, 2014, at 1:47 AM, Martin Buchholz wrote: >> >> OK, I worked in some wording for comparison with volatiles. >> I believe you when you say that the semantics of the corresponding C++ >> fences are slightly different, but it's rather subtle - can we say >> anything more than "closely related to"? >> >> On Mon, Nov 24, 2014 at 1:29 PM, Aleksey Shipilev >> wrote: >>> Hi Martin, >>> >>>> On 11/24/2014 11:56 PM, Martin Buchholz wrote: >>>> Review carefully - I am trying to learn about fences by explaining them! >>>> I have borrowed some wording from my reviewers! >>>> >>>> https://bugs.openjdk.java.net/browse/JDK-8065804 >>>> http://cr.openjdk.java.net/~martin/webrevs/openjdk9/fence-intrinsics/ >>> >>> I think "implies the effect of C++11" is too strong wording. "related" >>> might be more appropriate. >>> >>> See also comments here for connection with "volatiles": >>> https://bugs.openjdk.java.net/browse/JDK-8038978 >>> >>> Take note the Hans' correction that fences generally imply more than >>> volatile load/store, but since you are listing the related things in the >>> docs, I think the "native" Java example is good to have. >>> >>> -Aleksey. >> _______________________________________________ >> Concurrency-interest mailing list >> Concurrency-interest at cs.oswego.edu >> http://cs.oswego.edu/mailman/listinfo/concurrency-interest > > _______________________________________________ > Concurrency-interest mailing list > Concurrency-interest at cs.oswego.edu > http://cs.oswego.edu/mailman/listinfo/concurrency-interest From boehm at acm.org Tue Nov 25 19:15:36 2014 From: boehm at acm.org (Hans Boehm) Date: Tue, 25 Nov 2014 11:15:36 -0800 Subject: [concurrency-interest] RFR: 8065804: JEP 171: Clarifications/corrections for fence intrinsics In-Reply-To: <21DC5F06-9597-4CCB-8A25-5CC354A0BCE5@flyingtroika.com> References: <5473A344.8090302@oracle.com> <21DC5F06-9597-4CCB-8A25-5CC354A0BCE5@flyingtroika.com> Message-ID: It seems to me that a (dubiuously named) loadFence is intended to have essentially the same semantics as the (perhaps slightly less dubiously named) C++ atomic_thread_fence(memory_order_acquire), and a storeFence matches atomic_thread_fence(memory_order_release). The C++ standard and, even more so, Mark Batty's work have a precise definition of what those mean in terms of implied "synchronizes with" relationships. It looks to me like this whole implementation model for volatiles in terms of fences is fundamentally doomed, and it probably makes more sense to get rid of it rather than spending time on renaming it (though we just did the latter in Android to avoid similar confusion about semantics). It's fundamentally incompatible with the way volatiles/atomics are intended to be implemented on ARMv8 (and Itanium). Which I think fundamentally get this much closer to right than traditional fence-based ISAs. I'm no hardware architect, but fundamentally it seems to me that load x acquire_fence imposes a much more stringent constraint than load_acquire x Consider the case in which the load from x is an L1 hit, but a preceding load (from say y) is a long-latency miss. If we enforce ordering by just waiting for completion of prior operation, the former has to wait for the load from y to complete; while the latter doesn't. I find it hard to believe that this doesn't leave an appreciable amount of performance on the table, at least for some interesting microarchitectures. Along similar lines, it seems to me that Doug's JSR cookbook was a great contribution originally, in that it finally pinned down understandable rules for implementors, while providing a lot of useful intuition. At this point, it is still quite useful for intuition, but http://www.cl.cam.ac.uk/~pes20/cpp/cpp0xmappings.html (remembering that Java volatile = C++ SC atomic) is a much better guide for implementors, especially on Power. The SPARC-like fence primitives used in the cookbook are no longer reflective of the most widely used architectures. And they do not reflect the fence types actually needed by Java. In addition, for better or worse, fencing requirements on at least Power are actually driven as much by store atomicity issues, as by the ordering issues discussed in the cookbook. This was not understood in 2005, and unfortunately doesn't seem to be amenable to the kind of straightforward explanation as in Doug's cookbook. Hans On Tue, Nov 25, 2014 at 10:16 AM, DT

wrote: > I see time to time comments in the jvm sources referencing membars and > fences. Would you say that they are used interchangeably ? Having the same > meaning but for different CPU arch. > > Sent from my iPhone > > > On Nov 25, 2014, at 6:04 AM, Paul Sandoz wrote: > > > > Hi Martin, > > > > Thanks for looking into this. > > > > 1141 * Currently hotspot's implementation of a Java language-level > volatile > > 1142 * store has the same effect as a storeFence followed by a > relaxed store, > > 1143 * although that may be a little stronger than needed. > > > > IIUC to emulate hotpot's volatile store you will need to say that a > fullFence immediately follows the relaxed store. > > > > The bit that always confuses me about release and acquire is ordering is > restricted to one direction, as talked about in orderAccess.hpp [1]. So for > a release, accesses prior to the release cannot move below it, but accesses > succeeding the release can move above it. And that seems to apply to > Unsafe.storeFence [2] (acting like a monitor exit). Is that contrary to C++ > release fences where ordering is restricted both to prior and succeeding > accesses? [3] > > > > So what about the following? > > > > a = r1; // Cannot move below the fence > > Unsafe.storeFence(); > > b = r2; // Can move above the fence? > > > > Paul. > > > > [1] In orderAccess.hpp > > // Execution by a processor of release makes the effect of all memory > > // accesses issued by it previous to the release visible to all > > // processors *before* the release completes. The effect of subsequent > > // memory accesses issued by it *may* be made visible *before* the > > // release. I.e., subsequent memory accesses may float above the > > // release, but prior ones may not float below it. > > > > [2] In memnode.hpp > > // "Release" - no earlier ref can move after (but later refs can move > > // up, like a speculative pipelined cache-hitting Load). Requires > > // multi-cpu visibility. Inserted independent of any store, as required > > // for intrinsic sun.misc.Unsafe.storeFence(). > > class StoreFenceNode: public MemBarNode { > > public: > > StoreFenceNode(Compile* C, int alias_idx, Node* precedent) > > : MemBarNode(C, alias_idx, precedent) {} > > virtual int Opcode() const; > > }; > > > > [3] > http://preshing.com/20131125/acquire-and-release-fences-dont-work-the-way-youd-expect/ > > > >> On Nov 25, 2014, at 1:47 AM, Martin Buchholz > wrote: > >> > >> OK, I worked in some wording for comparison with volatiles. > >> I believe you when you say that the semantics of the corresponding C++ > >> fences are slightly different, but it's rather subtle - can we say > >> anything more than "closely related to"? > >> > >> On Mon, Nov 24, 2014 at 1:29 PM, Aleksey Shipilev > >> wrote: > >>> Hi Martin, > >>> > >>>> On 11/24/2014 11:56 PM, Martin Buchholz wrote: > >>>> Review carefully - I am trying to learn about fences by explaining > them! > >>>> I have borrowed some wording from my reviewers! > >>>> > >>>> https://bugs.openjdk.java.net/browse/JDK-8065804 > >>>> http://cr.openjdk.java.net/~martin/webrevs/openjdk9/fence-intrinsics/ > >>> > >>> I think "implies the effect of C++11" is too strong wording. "related" > >>> might be more appropriate. > >>> > >>> See also comments here for connection with "volatiles": > >>> https://bugs.openjdk.java.net/browse/JDK-8038978 > >>> > >>> Take note the Hans' correction that fences generally imply more than > >>> volatile load/store, but since you are listing the related things in > the > >>> docs, I think the "native" Java example is good to have. > >>> > >>> -Aleksey. > >> _______________________________________________ > >> Concurrency-interest mailing list > >> Concurrency-interest at cs.oswego.edu > >> http://cs.oswego.edu/mailman/listinfo/concurrency-interest > > > > _______________________________________________ > > Concurrency-interest mailing list > > Concurrency-interest at cs.oswego.edu > > http://cs.oswego.edu/mailman/listinfo/concurrency-interest > > _______________________________________________ > Concurrency-interest mailing list > Concurrency-interest at cs.oswego.edu > http://cs.oswego.edu/mailman/listinfo/concurrency-interest > From stephan.diestelhorst at gmail.com Tue Nov 25 23:24:59 2014 From: stephan.diestelhorst at gmail.com (Stephan Diestelhorst) Date: Tue, 25 Nov 2014 23:24:59 +0000 Subject: [concurrency-interest] RFR: 8065804: JEP 171: Clarifications/corrections for fence intrinsics In-Reply-To: References: <21DC5F06-9597-4CCB-8A25-5CC354A0BCE5@flyingtroika.com> Message-ID: <2936699.i7HuM0iO2v@d-allen> Am Dienstag, 25. November 2014, 11:15:36 schrieb Hans Boehm: > I'm no hardware architect, but fundamentally it seems to me that > > load x > acquire_fence > > imposes a much more stringent constraint than > > load_acquire x > > Consider the case in which the load from x is an L1 hit, but a preceding > load (from say y) is a long-latency miss. If we enforce ordering by just > waiting for completion of prior operation, the former has to wait for the > load from y to complete; while the latter doesn't. I find it hard to > believe that this doesn't leave an appreciable amount of performance on the > table, at least for some interesting microarchitectures. I agree, Hans, that this is a reasonable assumption. Load_acquire x does allow roach motel, whereas the acquire fence does not. > In addition, for better or worse, fencing requirements on at least > Power are actually driven as much by store atomicity issues, as by > the ordering issues discussed in the cookbook. This was not > understood in 2005, and unfortunately doesn't seem to be amenable to > the kind of straightforward explanation as in Doug's cookbook. Coming from a strongly ordered architecture to a weakly ordered one myself, I also needed some mental adjustment about store (multi-copy) atomicity. I can imagine others will be unaware of this difference, too, even in 2014. Stephan From stephan.diestelhorst at gmail.com Tue Nov 25 23:52:42 2014 From: stephan.diestelhorst at gmail.com (Stephan Diestelhorst) Date: Tue, 25 Nov 2014 23:52:42 +0000 Subject: [concurrency-interest] RFR: 8065804: JEP 171:Clarifications/corrections for fence intrinsics In-Reply-To: References: Message-ID: <2458774.qkf0xEMZGz@d-allen> David Holmes wrote: > Stephan Diestelhorst writes: > > Am Dienstag, 25. November 2014, 11:15:36 schrieb Hans Boehm: > > > I'm no hardware architect, but fundamentally it seems to me that > > > > > > load x > > > acquire_fence > > > > > > imposes a much more stringent constraint than > > > > > > load_acquire x > > > > > > Consider the case in which the load from x is an L1 hit, but a > > > preceding load (from say y) is a long-latency miss. If we enforce > > > ordering by just waiting for completion of prior operation, the > > > former has to wait for the load from y to complete; while the > > > latter doesn't. I find it hard to believe that this doesn't leave > > > an appreciable amount of performance on the table, at least for > > > some interesting microarchitectures. > > > > I agree, Hans, that this is a reasonable assumption. Load_acquire x > > does allow roach motel, whereas the acquire fence does not. > > > > > In addition, for better or worse, fencing requirements on at least > > > Power are actually driven as much by store atomicity issues, as by > > > the ordering issues discussed in the cookbook. This was not > > > understood in 2005, and unfortunately doesn't seem to be amenable to > > > the kind of straightforward explanation as in Doug's cookbook. > > > > Coming from a strongly ordered architecture to a weakly ordered one > > myself, I also needed some mental adjustment about store (multi-copy) > > atomicity. I can imagine others will be unaware of this difference, > > too, even in 2014. > > Sorry I'm missing the connection between fences and multi-copy atomicity. One example is the classic IRIW. With non-multi copy atomic stores, but ordered (say through a dependency) loads in the following example: Memory: foo = bar = 0 _T1_ _T2_ _T3_ _T4_ st (foo),1 st (bar),1 ld r1, (bar) ld r3,(foo) ld r2, (foo) ld r4, (bar) You may observe r1 = 1, r2 = 0, r3 = 1, r4 = 0 on non-multi-copy atomic machines. On TSO boxes, this is not possible. That means that the memory fence that will prevent such a behaviour (DMB on ARM) needs to carry some additional oomph in ensuring multi-copy atomicity, or rather prevent you from seeing it (which is the same thing). Stephan From boehm at acm.org Wed Nov 26 02:04:12 2014 From: boehm at acm.org (Hans Boehm) Date: Tue, 25 Nov 2014 18:04:12 -0800 Subject: [concurrency-interest] RFR: 8065804: JEP171:Clarifications/corrections for fence intrinsics In-Reply-To: References: <2458774.qkf0xEMZGz@d-allen> Message-ID: To be concrete here, on Power, loads can normally be ordered by an address dependency or light-weight fence (lwsync). However, neither is enough to prevent the questionable outcome for IRIW, since it doesn't ensure that the stores in T1 and T2 will be made visible to other threads in a consistent order. That outcome can be prevented by using heavyweight fences (sync) instructions between the loads instead. Peter Sewell's group concluded that to enforce correct volatile behavior on Power, you essentially need a a heavyweight fence between every pair of volatile operations on Power. That cannot be understood based on simple ordering constraints. As Stephan pointed out, there are similar issues on ARM, but they're less commonly encountered in a Java implementation. If you're lucky, you can get to the right implementation recipe by looking at only reordering, I think. On Tue, Nov 25, 2014 at 4:36 PM, David Holmes wrote: > Stephan Diestelhorst writes: > > > > David Holmes wrote: > > > Stephan Diestelhorst writes: > > > > Am Dienstag, 25. November 2014, 11:15:36 schrieb Hans Boehm: > > > > > I'm no hardware architect, but fundamentally it seems to me that > > > > > > > > > > load x > > > > > acquire_fence > > > > > > > > > > imposes a much more stringent constraint than > > > > > > > > > > load_acquire x > > > > > > > > > > Consider the case in which the load from x is an L1 hit, but a > > > > > preceding load (from say y) is a long-latency miss. If we enforce > > > > > ordering by just waiting for completion of prior operation, the > > > > > former has to wait for the load from y to complete; while the > > > > > latter doesn't. I find it hard to believe that this doesn't leave > > > > > an appreciable amount of performance on the table, at least for > > > > > some interesting microarchitectures. > > > > > > > > I agree, Hans, that this is a reasonable assumption. Load_acquire x > > > > does allow roach motel, whereas the acquire fence does not. > > > > > > > > > In addition, for better or worse, fencing requirements on at least > > > > > Power are actually driven as much by store atomicity issues, as by > > > > > the ordering issues discussed in the cookbook. This was not > > > > > understood in 2005, and unfortunately doesn't seem to be > > amenable to > > > > > the kind of straightforward explanation as in Doug's cookbook. > > > > > > > > Coming from a strongly ordered architecture to a weakly ordered one > > > > myself, I also needed some mental adjustment about store (multi-copy) > > > > atomicity. I can imagine others will be unaware of this difference, > > > > too, even in 2014. > > > > > > Sorry I'm missing the connection between fences and multi-copy > > atomicity. > > > > One example is the classic IRIW. With non-multi copy atomic stores, but > > ordered (say through a dependency) loads in the following example: > > > > Memory: foo = bar = 0 > > _T1_ _T2_ _T3_ _T4_ > > st (foo),1 st (bar),1 ld r1, (bar) ld r3,(foo) > > > > ld r2, (foo) ld r4, (bar) > > > > You may observe r1 = 1, r2 = 0, r3 = 1, r4 = 0 on non-multi-copy atomic > > machines. On TSO boxes, this is not possible. That means that the > > memory fence that will prevent such a behaviour (DMB on ARM) needs to > > carry some additional oomph in ensuring multi-copy atomicity, or rather > > prevent you from seeing it (which is the same thing). > > I take it as given that any code for which you may have ordering > constraints, must first have basic atomicity properties for loads and > stores. I would not expect any kind of fence to add multi-copy-atomicity > where there was none. > > David > > > Stephan > > > > _______________________________________________ > > Concurrency-interest mailing list > > Concurrency-interest at cs.oswego.edu > > http://cs.oswego.edu/mailman/listinfo/concurrency-interest > > _______________________________________________ > Concurrency-interest mailing list > Concurrency-interest at cs.oswego.edu > http://cs.oswego.edu/mailman/listinfo/concurrency-interest > From boehm at acm.org Wed Nov 26 18:48:33 2014 From: boehm at acm.org (Hans Boehm) Date: Wed, 26 Nov 2014 10:48:33 -0800 Subject: [concurrency-interest] RFR: 8065804: JEP171:Clarifications/corrections for fence intrinsics In-Reply-To: References: Message-ID: Definitions here seem to be less clear than I would like. What I meant by "store atomicity", which I think is more or less synonymous with "multi-copy atomicity" is that a store becomes visible to all observers at the same time, or equivalently stores become visible to all observers in a consistent order. In my view, IRIW is the canonical test for that. I agree with Roman that IRIW requirements for Java volatiles are here to stay. Many of us thought about ways to relax the requirement about 8 or 9 years ago. In my view: - Sequential consistency for data-race-free code is the only model that we can possibly explain to the majority of programmers. (Even stronger models like some notions of region serializability may also make sense, but they'll cost you.) This requires IRIW. This model is also by far the easiest to reason about formally. - The next weaker model that seems to be somewhat explainable, but really only to experts, is something along the lines of the C++ acquire/release model. This doesn't require IRIW. It's clearly too weak to replace Java volatile behavior, since it also fails to work for Dekkers-like settings, which are fairly common. (Nonexperts perhaps shouldn't write lock-free Dekkers-like code, but it's hard to explain precisely what they shouldn't be doing.) - A large amount of effort to generate models between those two failed to generate anything viable. The general experience was that once you no longer require IRIW, you also end up failing various other, potentially more important, litmus tests in ways that are really difficult to explain. And those models generally looked too complex to me to form a viable basis for real programs I think many people, even those who would rather not enforce IRIW, generally agree with this characterization. Hans On Tue, Nov 25, 2014 at 6:10 PM, David Holmes wrote: > Hi Hans, > > Given IRIW is a thorn in everyone's side and has no known useful benefit, > and can hopefully be killed off in the future, lets not get bogged down in > IRIW. But none of what you say below relates to multi-copy-atomicity. > > Cheers, > David > > -----Original Message----- > *From:* hjkhboehm at gmail.com [mailto:hjkhboehm at gmail.com]*On Behalf Of *Hans > Boehm > *Sent:* Wednesday, 26 November 2014 12:04 PM > *To:* dholmes at ieee.org > *Cc:* Stephan Diestelhorst; concurrency-interest at cs.oswego.edu; > core-libs-dev > *Subject:* Re: [concurrency-interest] RFR: 8065804: > JEP171:Clarifications/corrections for fence intrinsics > > To be concrete here, on Power, loads can normally be ordered by an address > dependency or light-weight fence (lwsync). However, neither is enough to > prevent the questionable outcome for IRIW, since it doesn't ensure that the > stores in T1 and T2 will be made visible to other threads in a consistent > order. That outcome can be prevented by using heavyweight fences (sync) > instructions between the loads instead. Peter Sewell's group concluded > that to enforce correct volatile behavior on Power, you essentially need a > a heavyweight fence between every pair of volatile operations on Power. > That cannot be understood based on simple ordering constraints. > > As Stephan pointed out, there are similar issues on ARM, but they're less > commonly encountered in a Java implementation. If you're lucky, you can > get to the right implementation recipe by looking at only reordering, I > think. > > > On Tue, Nov 25, 2014 at 4:36 PM, David Holmes > wrote: > >> Stephan Diestelhorst writes: >> > >> > David Holmes wrote: >> > > Stephan Diestelhorst writes: >> > > > Am Dienstag, 25. November 2014, 11:15:36 schrieb Hans Boehm: >> > > > > I'm no hardware architect, but fundamentally it seems to me that >> > > > > >> > > > > load x >> > > > > acquire_fence >> > > > > >> > > > > imposes a much more stringent constraint than >> > > > > >> > > > > load_acquire x >> > > > > >> > > > > Consider the case in which the load from x is an L1 hit, but a >> > > > > preceding load (from say y) is a long-latency miss. If we enforce >> > > > > ordering by just waiting for completion of prior operation, the >> > > > > former has to wait for the load from y to complete; while the >> > > > > latter doesn't. I find it hard to believe that this doesn't leave >> > > > > an appreciable amount of performance on the table, at least for >> > > > > some interesting microarchitectures. >> > > > >> > > > I agree, Hans, that this is a reasonable assumption. Load_acquire x >> > > > does allow roach motel, whereas the acquire fence does not. >> > > > >> > > > > In addition, for better or worse, fencing requirements on at >> least >> > > > > Power are actually driven as much by store atomicity issues, as >> by >> > > > > the ordering issues discussed in the cookbook. This was not >> > > > > understood in 2005, and unfortunately doesn't seem to be >> > amenable to >> > > > > the kind of straightforward explanation as in Doug's cookbook. >> > > > >> > > > Coming from a strongly ordered architecture to a weakly ordered one >> > > > myself, I also needed some mental adjustment about store >> (multi-copy) >> > > > atomicity. I can imagine others will be unaware of this difference, >> > > > too, even in 2014. >> > > >> > > Sorry I'm missing the connection between fences and multi-copy >> > atomicity. >> > >> > One example is the classic IRIW. With non-multi copy atomic stores, but >> > ordered (say through a dependency) loads in the following example: >> > >> > Memory: foo = bar = 0 >> > _T1_ _T2_ _T3_ _T4_ >> > st (foo),1 st (bar),1 ld r1, (bar) ld r3,(foo) >> > >> > ld r2, (foo) ld r4, (bar) >> > >> > You may observe r1 = 1, r2 = 0, r3 = 1, r4 = 0 on non-multi-copy atomic >> > machines. On TSO boxes, this is not possible. That means that the >> > memory fence that will prevent such a behaviour (DMB on ARM) needs to >> > carry some additional oomph in ensuring multi-copy atomicity, or rather >> > prevent you from seeing it (which is the same thing). >> >> I take it as given that any code for which you may have ordering >> constraints, must first have basic atomicity properties for loads and >> stores. I would not expect any kind of fence to add multi-copy-atomicity >> where there was none. >> >> David >> >> > Stephan >> > >> > _______________________________________________ >> > Concurrency-interest mailing list >> > Concurrency-interest at cs.oswego.edu >> > http://cs.oswego.edu/mailman/listinfo/concurrency-interest >> >> _______________________________________________ >> Concurrency-interest mailing list >> Concurrency-interest at cs.oswego.edu >> http://cs.oswego.edu/mailman/listinfo/concurrency-interest >> > > From boehm at acm.org Sat Nov 29 04:56:04 2014 From: boehm at acm.org (Hans Boehm) Date: Fri, 28 Nov 2014 20:56:04 -0800 Subject: [concurrency-interest] RFR: 8065804: JEP 171: Clarifications/corrections for fence intrinsics In-Reply-To: <5476D751.5040407@gmail.com> References: <5476D751.5040407@gmail.com> Message-ID: I basically agree with David's observation. However the C++ atomic_thread_fence(memory_order_acquire) actually has somewhat different semantics from load(memory_order_acquire). It basically ensures that prior atomic loads L are not reordered with later (i.e. following the fence in program order) loads and stores, making it something like a LoadLoad|LoadStore fence. Thus the fence orders two sets of operations where the acquire load orders a single operation with respect to a set. This makes the fence versions of memory_order_acquire and memory_order_release meaningful, but somewhat different from the non-fence versions. The terminology is probably not great, but that seems to be the most common usage now. On Wed, Nov 26, 2014 at 11:48 PM, Peter Levart wrote: > On 11/27/2014 04:00 AM, David Holmes wrote: > >> Can I make an observation about acquire() and release() - to me they are >> meaningless when considered in isolation. Given their definitions they >> allow >> anything to move into a region bounded by acquire() and release(), then >> you >> can effectively move the whole program into the region and thus the >> acquire() and release() do not constrain any reorderings. >> acquire() and >> release() only make sense when their own movement is constrained with >> respect to something else - such as lock acquisition/release, or when >> combined with specific load/store actions. >> > > ...or another acquire/release region? > > Regards, Peter > > > >> David >> >> Martin Buchholz writes: >> >>> On Tue, Nov 25, 2014 at 6:04 AM, Paul Sandoz >>> wrote: >>> >>>> Hi Martin, >>>> >>>> Thanks for looking into this. >>>> >>>> 1141 * Currently hotspot's implementation of a Java >>>> >>> language-level volatile >>> >>>> 1142 * store has the same effect as a storeFence followed >>>> >>> by a relaxed store, >>> >>>> 1143 * although that may be a little stronger than needed. >>>> >>>> IIUC to emulate hotpot's volatile store you will need to say >>>> >>> that a fullFence immediately follows the relaxed store. >>> >>> Right - I've been groking that. >>> >>> The bit that always confuses me about release and acquire is >>>> >>> ordering is restricted to one direction, as talked about in >>> orderAccess.hpp [1]. So for a release, accesses prior to the >>> release cannot move below it, but accesses succeeding the release >>> can move above it. And that seems to apply to Unsafe.storeFence >>> [2] (acting like a monitor exit). Is that contrary to C++ release >>> fences where ordering is restricted both to prior and succeeding >>> accesses? [3] >>> >>>> So what about the following? >>>> >>>> a = r1; // Cannot move below the fence >>>> Unsafe.storeFence(); >>>> b = r2; // Can move above the fence? >>>> >>> I think the hotspot docs need to be more precise about when they're >>> talking about movement of stores and when about loads. >>> >>> // release. I.e., subsequent memory accesses may float above the >>>> // release, but prior ones may not float below it. >>>> >>> As I've said elsewhere, the above makes no sense without restricting >>> the type of access. >>> >>> _______________________________________________ >>> Concurrency-interest mailing list >>> Concurrency-interest at cs.oswego.edu >>> http://cs.oswego.edu/mailman/listinfo/concurrency-interest >>> >> > _______________________________________________ > Concurrency-interest mailing list > Concurrency-interest at cs.oswego.edu > http://cs.oswego.edu/mailman/listinfo/concurrency-interest > From TEREKHOV at de.ibm.com Sat Nov 29 14:16:00 2014 From: TEREKHOV at de.ibm.com (Alexander Terekhov) Date: Sat, 29 Nov 2014 15:16:00 +0100 Subject: [concurrency-interest] RFR: 8065804: JEP 171: Clarifications/corrections for fence intrinsics In-Reply-To: Message-ID: > atomic_thread_fence(memory_order_acquire) int r1 = x; // ordinary load int r2 = y.load(memory_order_relaxed); // atomic relaxed load if (z.load(memory_order_relaxed) == SOMETHING_I_AM_WAITING_FOR) { atomic_thread_fence(memory_order_acquire); . . . } IIUC r1 and r2 (and etc.) loads prior to z can be reordered with respect to the fence... and hence the term 'acquire' fits quite well. regards, alexander. Hans Boehm @cs.oswego.edu on 29.11.2014 05:56:04 Sent by: concurrency-interest-bounces at cs.oswego.edu To: Peter Levart cc: Vladimir Kozlov , concurrency-interest , Martin Buchholz , core-libs-dev , dholmes at ieee.org Subject: Re: [concurrency-interest] RFR: 8065804: JEP 171: Clarifications/corrections for fence intrinsics I basically agree with David's observation.? However the C++ atomic_thread_fence(memory_order_acquire) actually has somewhat different semantics from load(memory_order_acquire). It basically ensures that prior atomic loads L are not reordered with later (i.e. following the fence in program order) loads and stores, making it something like a LoadLoad|LoadStore fence.? Thus the fence orders two sets of operations where the acquire load orders a single operation with respect to a set.? This makes the fence versions of memory_order_acquire and memory_order_release meaningful, but somewhat different from the non-fence versions.? The terminology is probably not great, but that seems to be the most common usage now. On Wed, Nov 26, 2014 at 11:48 PM, Peter Levart wrote: On 11/27/2014 04:00 AM, David Holmes wrote: Can I make an observation about acquire() and release() - to me they are meaningless when considered in isolation. Given their definitions they allow anything to move into a region bounded by acquire() and release (), then you can effectively move the whole program into the region and thus the acquire() and release() do not constrain any reorderings. ? acquire() and release() only make sense when their own movement is constrained with respect to something else - such as lock acquisition/release, or when combined with specific load/store actions. ...or another acquire/release region? Regards, Peter David Martin Buchholz writes: On Tue, Nov 25, 2014 at 6:04 AM, Paul Sandoz wrote: Hi Martin, Thanks for looking into this. 1141? ? ? * Currently hotspot's implementation of a Java language-level volatile 1142? ? ? * store has the same effect as a storeFence followed by a relaxed store, 1143? ? ? * although that may be a little stronger than needed. IIUC to emulate hotpot's volatile store you will need to say that a fullFence immediately follows the relaxed store. Right - I've been groking that. The bit that always confuses me about release and acquire is ordering is restricted to one direction, as talked about in orderAccess.hpp [1]. So for a release, accesses prior to the release cannot move below it, but accesses succeeding the release can move above it. And that seems to apply to Unsafe.storeFence [2] (acting like a monitor exit). Is that contrary to C++ release fences where ordering is restricted both to prior and succeeding accesses? [3] So what about the following? ? ?a = r1; // Cannot move below the fence ? ?Unsafe.storeFence(); ? ?b = r2; // Can move above the fence? I think the hotspot docs need to be more precise about when they're talking about movement of stores and when about loads. // release.? I.e., subsequent memory accesses may float above the // release, but prior ones may not float below it. As I've said elsewhere, the above makes no sense without restricting the type of access. _______________________________________________ Concurrency-interest mailing list Concurrency-interest at cs.oswego.edu http://cs.oswego.edu/mailman/listinfo/concurrency-interest _______________________________________________ Concurrency-interest mailing list Concurrency-interest at cs.oswego.edu http://cs.oswego.edu/mailman/listinfo/concurrency-interest _______________________________________________ Concurrency-interest mailing list Concurrency-interest at cs.oswego.edu http://cs.oswego.edu/mailman/listinfo/concurrency-interest From TEREKHOV at de.ibm.com Sat Nov 29 15:12:38 2014 From: TEREKHOV at de.ibm.com (Alexander Terekhov) Date: Sat, 29 Nov 2014 16:12:38 +0100 Subject: [concurrency-interest] RFR: 8065804: JEP 171: Clarifications/corrections for fence intrinsics In-Reply-To: Message-ID: > memory_order_release meaningful, but somewhat different from the non-fence it would be nice to have release fence with an artificial dependency to define a set of actually release stores and not constraining other subsequent stores (and the order of release stores with respect to each other), e.g.: // set multiple flags each indicating 'release' without imposing // ordering on 'release' stores respect to each other and not // constraining other subsequent stores . . . if (atomic_thread_fence(memory_order_release)) { flag1.store(READY, memory_order_relaxed); flag2.store(READY, memory_order_relaxed); } regards, alexander. Hans Boehm @cs.oswego.edu on 29.11.2014 05:56:04 Sent by: concurrency-interest-bounces at cs.oswego.edu To: Peter Levart cc: Vladimir Kozlov , concurrency-interest , Martin Buchholz , core-libs-dev , dholmes at ieee.org Subject: Re: [concurrency-interest] RFR: 8065804: JEP 171: Clarifications/corrections for fence intrinsics I basically agree with David's observation.? However the C++ atomic_thread_fence(memory_order_acquire) actually has somewhat different semantics from load(memory_order_acquire). It basically ensures that prior atomic loads L are not reordered with later (i.e. following the fence in program order) loads and stores, making it something like a LoadLoad|LoadStore fence.? Thus the fence orders two sets of operations where the acquire load orders a single operation with respect to a set.? This makes the fence versions of memory_order_acquire and memory_order_release meaningful, but somewhat different from the non-fence versions.? The terminology is probably not great, but that seems to be the most common usage now. On Wed, Nov 26, 2014 at 11:48 PM, Peter Levart wrote: On 11/27/2014 04:00 AM, David Holmes wrote: Can I make an observation about acquire() and release() - to me they are meaningless when considered in isolation. Given their definitions they allow anything to move into a region bounded by acquire() and release (), then you can effectively move the whole program into the region and thus the acquire() and release() do not constrain any reorderings. ? acquire() and release() only make sense when their own movement is constrained with respect to something else - such as lock acquisition/release, or when combined with specific load/store actions. ...or another acquire/release region? Regards, Peter David Martin Buchholz writes: On Tue, Nov 25, 2014 at 6:04 AM, Paul Sandoz wrote: Hi Martin, Thanks for looking into this. 1141? ? ? * Currently hotspot's implementation of a Java language-level volatile 1142? ? ? * store has the same effect as a storeFence followed by a relaxed store, 1143? ? ? * although that may be a little stronger than needed. IIUC to emulate hotpot's volatile store you will need to say that a fullFence immediately follows the relaxed store. Right - I've been groking that. The bit that always confuses me about release and acquire is ordering is restricted to one direction, as talked about in orderAccess.hpp [1]. So for a release, accesses prior to the release cannot move below it, but accesses succeeding the release can move above it. And that seems to apply to Unsafe.storeFence [2] (acting like a monitor exit). Is that contrary to C++ release fences where ordering is restricted both to prior and succeeding accesses? [3] So what about the following? ? ?a = r1; // Cannot move below the fence ? ?Unsafe.storeFence(); ? ?b = r2; // Can move above the fence? I think the hotspot docs need to be more precise about when they're talking about movement of stores and when about loads. // release.? I.e., subsequent memory accesses may float above the // release, but prior ones may not float below it. As I've said elsewhere, the above makes no sense without restricting the type of access. _______________________________________________ Concurrency-interest mailing list Concurrency-interest at cs.oswego.edu http://cs.oswego.edu/mailman/listinfo/concurrency-interest _______________________________________________ Concurrency-interest mailing list Concurrency-interest at cs.oswego.edu http://cs.oswego.edu/mailman/listinfo/concurrency-interest _______________________________________________ Concurrency-interest mailing list Concurrency-interest at cs.oswego.edu http://cs.oswego.edu/mailman/listinfo/concurrency-interest