From Dalibor.Topic at Sun.COM Mon Jun 1 10:46:20 2009 From: Dalibor.Topic at Sun.COM (Dalibor Topic) Date: Mon, 01 Jun 2009 03:46:20 -0700 Subject: OpenJDK Forum: Core Libraries Round Table In-Reply-To: <4A201662.1090002@sun.com> References: <4A1D759A.5020504@sun.com> <4A201662.1090002@sun.com> Message-ID: <4A23B17C.9030601@sun.com> Dalibor Topic wrote: > Dalibor Topic wrote: >> Hi core libraries developers, >> >> I met Alan Bateman last evening, and we thought that we >> should have another OpenJDK Core Libraries Forum this week, >> so (drumroll) it's time again on Friday, for the >> >> OpenJDK Forum >> >> Date/Time: Friday May 29th, 8 AM Pacific, 1600 GMT, 5 PM Germany >> >> Subject: Core libraries round table > > Hi everyone, > > a recording of the call in the Ogg Vorbis format is available at > http://mediacast.sun.com/users/robilad/media/openjdk-forum-3.ogg/details > For the Ogg-player-less would-be-listeners, there is an mp3 version generated from the ogg file at http://mediacast.sun.com/users/robilad/media/openjdk-forum-3.mp3/details cheers, dalibor topic -- ******************************************************************* Dalibor Topic Tel: (+49 40) 23 646 738 Java F/OSS Ambassador AIM: robiladonaim Sun Microsystems GmbH Mobile: (+49 177) 2664 192 Nagelsweg 55 http://openjdk.java.net D-20097 Hamburg mailto:Dalibor.Topic at sun.com Sitz der Gesellschaft: Sonnenallee 1, D-85551 Kirchheim-Heimstetten Amtsgericht M?nchen: HRB 161028 Gesch?ftsf?hrer: Thomas Schr?der, Wolfgang Engels, Wolf Frenkel Vorsitzender des Aufsichtsrates: Martin H?ring From martinrb at google.com Tue Jun 2 00:27:54 2009 From: martinrb at google.com (Martin Buchholz) Date: Mon, 1 Jun 2009 17:27:54 -0700 Subject: Review request for 5049299 In-Reply-To: <1ccfd1c10905301051j527d2e91m217020d037ac1732@mail.gmail.com> References: <4A1678DB.9010209@sun.com> <4A172398.9000100@sun.com> <1ccfd1c10905221757t30a7f610jd343084e50ec88bb@mail.gmail.com> <4A17B3BB.4090706@redhat.com> <1ccfd1c10905231112k61d8e13h31f693951ed91b76@mail.gmail.com> <4A1849CD.3060306@redhat.com> <1ccfd1c10905231810k17a99cd3vfa68d6108e28b46@mail.gmail.com> <4A191063.5060401@redhat.com> <1ccfd1c10905241809m10ded53bpb4c9360f76b36888@mail.gmail.com> <1ccfd1c10905301051j527d2e91m217020d037ac1732@mail.gmail.com> Message-ID: <1ccfd1c10906011727g794e9ca0m89c350fa8dae7c54@mail.gmail.com> I've been working on clone+exec. I now have a perhaps-ready-for-prime-time implementation that appears to fix the problem on Linux and also includes some new tests for script invocation http://cr.openjdk.java.net/~martin/clone-exec This change conflicts with Michael's changes for Solaris. We'll have to work something out. My changes are designed to have no effect on Solaris, but I have done zero testing there. Martin On Sat, May 30, 2009 at 10:51, Martin Buchholz wrote: > I have a prototype implementation that works only for Linux, > only when the environment is inherited, > but seems to show that the clone/fork approach works. > > The main idea is that passing these flags: > CLONE_VM | SIGCHLD > to clone() appears to work. > Passing CLONE_VFORK in addition also seems to work, > and may be more reliable. > > Michael, we will need to coordinate to come up with > something that will work for all our Unices. > > Martin > -------------- next part -------------- An HTML attachment was scrubbed... URL: From Michael.McMahon at Sun.COM Tue Jun 2 14:29:17 2009 From: Michael.McMahon at Sun.COM (Michael McMahon) Date: Tue, 02 Jun 2009 15:29:17 +0100 Subject: Review request for 5049299 In-Reply-To: <1ccfd1c10906011727g794e9ca0m89c350fa8dae7c54@mail.gmail.com> References: <4A1678DB.9010209@sun.com> <4A172398.9000100@sun.com> <1ccfd1c10905221757t30a7f610jd343084e50ec88bb@mail.gmail.com> <4A17B3BB.4090706@redhat.com> <1ccfd1c10905231112k61d8e13h31f693951ed91b76@mail.gmail.com> <4A1849CD.3060306@redhat.com> <1ccfd1c10905231810k17a99cd3vfa68d6108e28b46@mail.gmail.com> <4A191063.5060401@redhat.com> <1ccfd1c10905241809m10ded53bpb4c9360f76b36888@mail.gmail.com> <1ccfd1c10905301051j527d2e91m217020d037ac1732@mail.gmail.com> <1ccfd1c10906011727g794e9ca0m89c350fa8dae7c54@mail.gmail.com> Message-ID: <4A25373D.9090906@sun.com> Martin, I had done something similar with clone & exec for Linux, but hadn't got round to testing it. So, it seems reasonable to take yours. Do you want to send me your updated versions of process_md.c and the test? I can take care of the merge with the Solaris code. Thanks, Michael. Martin Buchholz wrote: > I've been working on clone+exec. > I now have a perhaps-ready-for-prime-time implementation > that appears to fix the problem on Linux > and also includes some new tests for script invocation > > http://cr.openjdk.java.net/~martin/clone-exec > > > This change conflicts with Michael's changes for Solaris. > We'll have to work something out. > My changes are designed to have no effect on Solaris, > but I have done zero testing there. > > Martin > > On Sat, May 30, 2009 at 10:51, Martin Buchholz > wrote: > > I have a prototype implementation that works only for Linux, > only when the environment is inherited, > but seems to show that the clone/fork approach works. > > The main idea is that passing these flags: > CLONE_VM | SIGCHLD > to clone() appears to work. > Passing CLONE_VFORK in addition also seems to work, > and may be more reliable. > > Michael, we will need to coordinate to come up with > something that will work for all our Unices. > > Martin > > From aph at redhat.com Tue Jun 2 14:43:39 2009 From: aph at redhat.com (Andrew Haley) Date: Tue, 02 Jun 2009 15:43:39 +0100 Subject: hgext.fetch Message-ID: <4A253A9B.2090801@redhat.com> I can't get the fetch extension to work properly. I was told that I need to add [extensions] hgext.fetch = -m Merge but that doesn't work: *** failed to import extension hgext.fetch from -m Merge: [Errno 2] No such file or directory fetch extension - no help text available It seems that the RHS of the = is supposed to be the path of the extension, not the options, and I can't find any information in the Mercurial manuals. Thanks, Andrew. From Anthony.Petrov at Sun.COM Tue Jun 2 14:46:01 2009 From: Anthony.Petrov at Sun.COM (Anthony Petrov) Date: Tue, 02 Jun 2009 18:46:01 +0400 Subject: hgext.fetch In-Reply-To: <4A253A9B.2090801@redhat.com> References: <4A253A9B.2090801@redhat.com> Message-ID: <4A253B29.9010804@sun.com> Do the following instead: [extensions] hgext.fetch = [defaults] fetch = -m Merge -- best regards, Anthony On 06/02/2009 06:43 PM, Andrew Haley wrote: > I can't get the fetch extension to work properly. > > I was told that I need to add > > [extensions] > hgext.fetch = -m Merge > > but that doesn't work: > > *** failed to import extension hgext.fetch from -m Merge: [Errno 2] No such file or directory > fetch extension - no help text available > > It seems that the RHS of the = is supposed to be the path of the extension, > not the options, and I can't find any information in the Mercurial manuals. > > Thanks, > Andrew. From aph at redhat.com Tue Jun 2 14:50:07 2009 From: aph at redhat.com (Andrew Haley) Date: Tue, 02 Jun 2009 15:50:07 +0100 Subject: hgext.fetch In-Reply-To: <4A253B29.9010804@sun.com> References: <4A253A9B.2090801@redhat.com> <4A253B29.9010804@sun.com> Message-ID: <4A253C1F.9060202@redhat.com> Anthony Petrov wrote: > Do the following instead: > > [extensions] > hgext.fetch = > > [defaults] > fetch = -m Merge OK, ta. Andrew. From Anthony.Petrov at Sun.COM Tue Jun 2 17:22:24 2009 From: Anthony.Petrov at Sun.COM (Anthony Petrov) Date: Tue, 02 Jun 2009 21:22:24 +0400 Subject: hgext.fetch In-Reply-To: <4A254537.7050105@sun.com> References: <4A253A9B.2090801@redhat.com> <4A253B29.9010804@sun.com> <4A253C1F.9060202@redhat.com> <4A254537.7050105@sun.com> Message-ID: <4A255FD0.1070703@sun.com> On 6/2/2009 7:28 PM Kelly O'Hair wrote: > And also add ffetch variations too. ;^) In the [defaults] section only. The ffetch command itself is available anyway if you have both fetch and forest extensions enabled. -- best regards, Anthony > If you want to use ffetch... > > -kto > > Andrew Haley wrote: >> Anthony Petrov wrote: >>> Do the following instead: >>> >>> [extensions] >>> hgext.fetch = >>> >>> [defaults] >>> fetch = -m Merge >> >> OK, ta. >> >> Andrew. From aph at redhat.com Tue Jun 2 17:30:34 2009 From: aph at redhat.com (Andrew Haley) Date: Tue, 02 Jun 2009 18:30:34 +0100 Subject: hgext.fetch In-Reply-To: <4A255FD0.1070703@sun.com> References: <4A253A9B.2090801@redhat.com> <4A253B29.9010804@sun.com> <4A253C1F.9060202@redhat.com> <4A254537.7050105@sun.com> <4A255FD0.1070703@sun.com> Message-ID: <4A2561BA.1030506@redhat.com> Anthony Petrov wrote: > On 6/2/2009 7:28 PM Kelly O'Hair wrote: >> And also add ffetch variations too. ;^) > In the [defaults] section only. The ffetch command itself is available > anyway if you have both fetch and forest extensions enabled. > Like this, I presume. [defaults] fetch = -m Merge ffetch = -m Merge From Kelly.Ohair at Sun.COM Tue Jun 2 15:28:55 2009 From: Kelly.Ohair at Sun.COM (Kelly O'Hair) Date: Tue, 02 Jun 2009 08:28:55 -0700 Subject: hgext.fetch In-Reply-To: <4A253C1F.9060202@redhat.com> References: <4A253A9B.2090801@redhat.com> <4A253B29.9010804@sun.com> <4A253C1F.9060202@redhat.com> Message-ID: <4A254537.7050105@sun.com> And also add ffetch variations too. ;^) If you want to use ffetch... -kto Andrew Haley wrote: > Anthony Petrov wrote: >> Do the following instead: >> >> [extensions] >> hgext.fetch = >> >> [defaults] >> fetch = -m Merge > > OK, ta. > > Andrew. From Kelly.Ohair at Sun.COM Tue Jun 2 23:39:44 2009 From: Kelly.Ohair at Sun.COM (Kelly O'Hair) Date: Tue, 02 Jun 2009 16:39:44 -0700 Subject: hgext.fetch In-Reply-To: <4A2561BA.1030506@redhat.com> References: <4A253A9B.2090801@redhat.com> <4A253B29.9010804@sun.com> <4A253C1F.9060202@redhat.com> <4A254537.7050105@sun.com> <4A255FD0.1070703@sun.com> <4A2561BA.1030506@redhat.com> Message-ID: <4A25B840.9000608@sun.com> Yup. -kto Andrew Haley wrote: > Anthony Petrov wrote: >> On 6/2/2009 7:28 PM Kelly O'Hair wrote: >>> And also add ffetch variations too. ;^) >> In the [defaults] section only. The ffetch command itself is available >> anyway if you have both fetch and forest extensions enabled. >> > > Like this, I presume. > > > [defaults] > fetch = -m Merge > ffetch = -m Merge > From dmytro_sheyko at hotmail.com Wed Jun 3 14:20:27 2009 From: dmytro_sheyko at hotmail.com (Dmytro Sheyko) Date: Wed, 3 Jun 2009 21:20:27 +0700 Subject: Cant detect deadlock when thread reenters synchronization block in Object.wait() Message-ID: Hi, Could you have a look at following patch regarding deadlock detection: Cant detect deadlock when thread reenters synchronization block in Object.wait() https://bugs.openjdk.java.net/show_bug.cgi?id=100058 http://bugs.sun.com/bugdatabase/view_bug.do?bug_id=6841139 Thank you, Dmytro Sheyko _________________________________________________________________ Show them the way! Add maps and directions to your party invites. http://www.microsoft.com/windows/windowslive/products/events.aspx -------------- next part -------------- An HTML attachment was scrubbed... URL: From David.Holmes at Sun.COM Wed Jun 3 15:04:23 2009 From: David.Holmes at Sun.COM (David Holmes) Date: Thu, 04 Jun 2009 01:04:23 +1000 Subject: Cant detect deadlock when thread reenters synchronization block in Object.wait() In-Reply-To: References: Message-ID: <4A2690F7.5020507@sun.com> Hi Dmytro, Can you split this into two separate bug reports please. The changes to the VM are quite separate from the changes to the java.util.concurrent classes. The j.u.c changes should then be discussed on the concurrent-interest mailing list: http://cs.oswego.edu/mailman/listinfo/concurrency-interest This will let all the JSR-166 EG members, and the genercal c-i community evaluate what you have done. At this stage I have not evaluated the actual proposals. Thanks, David Holmes (j.u.c bug evaluator) Dmytro Sheyko wrote: > Hi, > > Could you have a look at following patch regarding deadlock detection: > > Cant detect deadlock when thread reenters synchronization block in > Object.wait() > https://bugs.openjdk.java.net/show_bug.cgi?id=100058 > http://bugs.sun.com/bugdatabase/view_bug.do?bug_id=6841139 > > Thank you, > Dmytro Sheyko > > ------------------------------------------------------------------------ > See all the ways you can stay connected to friends and family > From alex14n at gmail.com Wed Jun 3 23:21:51 2009 From: alex14n at gmail.com (Alex Yakovlev) Date: Thu, 4 Jun 2009 03:21:51 +0400 Subject: Faster HashMap implementation Message-ID: Hello, While playing with scala programming language I've come to a HashMap implementation that appears to be faster than current java.util.HashMap, also with a lower memory footprint. Alex Miller advised me to post about it in this maillist. I've put current implementation (both scala and java) and some benchmark results at github: http://github.com/alex14n/CompactHashMap Current java version implements most of the external HashMap behaviour inluding serialization, but does not throw ConcurrentModificationException and does not implement internal methods that can be requeired for some subclasses. Main trick is in storing all data in 2 arrays without any HashEntry objects. One is array of Objects with keys and values, another in a complex array of indices. Index array has 2 parts. First part contains hashcode-to-index mappings, second part is index-to-index mapping for elements with the same hashcode. The rest bits are used for: 1. 'Deleted' flag to mark empty spots after key removal (other bits part of such values is used as deleted-linked-list indices) 2. 'End-of-list' flag allowing not to access next index if this element is last. 3. Some bits of hashcode used to compare with hashcode of the key we are looking for. Finding an existing value for a key has approximately the same speed as the currrent HashMap implementation. It is: 1. Computing element hashcode, 2. Accessing array to get index/address for this hashcode 3a. Current HashMap access HashEntry object with key/hashcode/value/next properties 4a. In my implementation hashcode and next are packed in index array bits, key and value are stored in adjacent cells of the second array. When looking for missing key my implementation can be twice faster in some situations, since it's only one random memory access (element index array). If it's empty, or its hashcode bits are not equal to hashcode bits of the key being searched, and it's `end-of-list` bit is set - it is enough to stop searching. Current HashMap needs 2 random memory reads: (1) array cell with address and (2) HashEntry at that address. Insertion of a new key does not require any memory allocation (new HashEntry object) since all arrays are pre-allocated except for resize operation. Resize consists of Arrays.copyOf to a twice large array and almost sequental rehashing - reading one array and writing it to either the same index or with `1` in highest bit. When adding several key/values they are sequentally written to key/values array, and sequental memory read is faster and more cache-friendly. Having 2 arrays instead of a lot of HashEntry objects is more garbage collector friendly and has significantly lower memory footprint. Iteration over elements is a sequental array read which is faster. To clone such HashMap we only need to clone 2 arrays with is also much faster. Elements order is preserved even after resize, only when some key is removed it leaves an empty 'hole' that will be filled after next key insertion. Thus iteration order is more predictable and I think it's possible not to throw ConcurrentModificationException in most situations. Iterator needs only 1 parameter - array index - and even resize does not break iteration order. But there can be problems. For example if we iterated over some key that was deleted after that and then inserted again into higher array index, it can eventually came up second time into iterator. But if we save highest non-empty index that was at iteration start, and monitor insert/delete operations (possible holes) in indices we've not iterated over yet, we will not need to throw ConcurrentModificationException in most cases. I've done some benchmarks that proves my assumptions: * Reading existing key-value mapping has the same speed, * Adding new mapping and looking for missing key is significantly faster. But there can be some problems in my testing strategy or benchmark can be somehow biased, so it's better to retest it with some independent benchmarks. Hope it will help to make java collections faster since such approach (preallocated arrays instead of ...Entry objects) can be used to speedup other collections. In my scala version both HashMap and HashSet are implemented this way. It also stores primitive types like int in primitive arrays which saves a lot of memory. But on one hand scala has built-in boxed arrays, and on the other hand such array boxing is slow. It also has to store keys and values in different arrays, which gives extra random memory read and thus slower - when keys and values are in adjacent cells of the same array value will usually be pre-fetched when key is read. Best, Alex Yakovlev -------------- next part -------------- An HTML attachment was scrubbed... URL: From xuelei.fan at sun.com Thu Jun 4 03:47:49 2009 From: xuelei.fan at sun.com (xuelei.fan at sun.com) Date: Thu, 04 Jun 2009 03:47:49 +0000 Subject: hg: jdk7/tl/jdk: 6847459: Allow trust anchor self-issued intermediate version 1 and version 2 certificate Message-ID: <20090604034818.93379EF4E@hg.openjdk.java.net> Changeset: 045743e0eb2d Author: xuelei Date: 2009-06-04 11:28 +0800 URL: http://hg.openjdk.java.net/jdk7/tl/jdk/rev/045743e0eb2d 6847459: Allow trust anchor self-issued intermediate version 1 and version 2 certificate Reviewed-by: weijun ! src/share/classes/sun/security/provider/certpath/ConstraintsChecker.java From Ulf.Zibis at gmx.de Thu Jun 4 10:50:47 2009 From: Ulf.Zibis at gmx.de (Ulf Zibis) Date: Thu, 04 Jun 2009 12:50:47 +0200 Subject: Faster HashMap implementation In-Reply-To: References: Message-ID: <4A27A707.7050605@gmx.de> See also: http://bugs.sun.com/bugdatabase/view_bug.do?bug_id=6812862 -Ulf Am 04.06.2009 01:21, Alex Yakovlev schrieb: > Hello, > > While playing with scala programming language I've come to a HashMap > implementation > that appears to be faster than current java.util.HashMap, also with a > lower memory footprint. > > Alex Miller advised me to post about it in this maillist. > > I've put current implementation (both scala and java) and some > benchmark results at github: > http://github.com/alex14n/CompactHashMap > > Current java version implements most of the external HashMap behaviour > inluding serialization, but does not throw ConcurrentModificationException > and does not implement internal methods that can be requeired for some > subclasses. > > Main trick is in storing all data in 2 arrays without any HashEntry > objects. > One is array of Objects with keys and values, another in a complex > array of indices. > > Index array has 2 parts. First part contains hashcode-to-index mappings, > second part is index-to-index mapping for elements with the same hashcode. > The rest bits are used for: > 1. 'Deleted' flag to mark empty spots after key removal > (other bits part of such values is used as deleted-linked-list indices) > 2. 'End-of-list' flag allowing not to access next index if this > element is last. > 3. Some bits of hashcode used to compare with hashcode of the key we > are looking for. > > Finding an existing value for a key has approximately the same speed > as the currrent > HashMap implementation. It is: > 1. Computing element hashcode, > 2. Accessing array to get index/address for this hashcode > 3a. Current HashMap access HashEntry object with > key/hashcode/value/next properties > 4a. In my implementation hashcode and next are packed in index array bits, > key and value are stored in adjacent cells of the second array. > > When looking for missing key my implementation can be twice faster in > some situations, > since it's only one random memory access (element index array). If > it's empty, > or its hashcode bits are not equal to hashcode bits of the key being > searched, > and it's `end-of-list` bit is set - it is enough to stop searching. > Current HashMap > needs 2 random memory reads: (1) array cell with address and (2) > HashEntry at that address. > > Insertion of a new key does not require any memory allocation (new > HashEntry object) > since all arrays are pre-allocated except for resize operation. Resize > consists of > Arrays.copyOf to a twice large array and almost sequental rehashing - > reading one array > and writing it to either the same index or with `1` in highest bit. > > When adding several key/values they are sequentally written to > key/values array, > and sequental memory read is faster and more cache-friendly. > > Having 2 arrays instead of a lot of HashEntry objects is more garbage > collector > friendly and has significantly lower memory footprint. > > Iteration over elements is a sequental array read which is faster. > To clone such HashMap we only need to clone 2 arrays with is also much > faster. > > Elements order is preserved even after resize, only when some key is > removed > it leaves an empty 'hole' that will be filled after next key > insertion. Thus iteration > order is more predictable and I think it's possible not to throw > ConcurrentModificationException > in most situations. Iterator needs only 1 parameter - array index - > and even resize > does not break iteration order. > > But there can be problems. For example if we iterated over some key > that was deleted after that and then inserted again into higher array > index, > it can eventually came up second time into iterator. But if we save > highest > non-empty index that was at iteration start, and monitor insert/delete > operations (possible holes) in indices we've not iterated over yet, > we will not need to throw ConcurrentModificationException in most cases. > > I've done some benchmarks that proves my assumptions: > * Reading existing key-value mapping has the same speed, > * Adding new mapping and looking for missing key is significantly faster. > > But there can be some problems in my testing strategy > or benchmark can be somehow biased, so it's better to retest it > with some independent benchmarks. > > Hope it will help to make java collections faster since such approach > (preallocated arrays instead of ...Entry objects) can be used to speedup > other collections. In my scala version both HashMap and HashSet > are implemented this way. It also stores primitive types like int > in primitive arrays which saves a lot of memory. But on one hand > scala has built-in boxed arrays, and on the other hand such array boxing > is slow. It also has to store keys and values in different arrays, > which gives extra random memory read and thus slower - > when keys and values are in adjacent cells of the same array > value will usually be pre-fetched when key is read. > > Best, > Alex Yakovlev From eliasen at mindspring.com Thu Jun 4 12:02:50 2009 From: eliasen at mindspring.com (Alan Eliasen) Date: Thu, 04 Jun 2009 06:02:50 -0600 Subject: Further BigInteger performance improvements In-Reply-To: <47E89FE3.6080203@mindspring.com> References: <47A14D21.8020807@mindspring.com> <47BB539B.8090901@mindspring.com> <87skzo6l5r.fsf@mid.deneb.enyo.de> <47BCAD9C.1030002@mindspring.com> <47E89FE3.6080203@mindspring.com> Message-ID: <4A27B7EA.8040809@mindspring.com> 16 months ago, I posted the first of several patches to vastly improve the performance of the BigInteger class. These included implementation of Karatsuba and 3-way Toom-Cook multiplication and Karatsuba and Toom-Cook squaring. The particular algorithm used for multiplication or squaring is chosen according to the size of the numbers, and are these algorithms are asymptotically faster than the O(n^2) algorithms in previous versions of Java. (Karatsuba is O(n^1.585), 3-way Toom Cook is O(n^1.465)). Unfortunately, none of these patches has yet been applied or reviewed in any way by Sun. (Although they have been reviewed by some of the top researchers in the field of high-performance multiplication.) Nor have I received answers to my questions about how long Sun wants regression tests to run, size of output, etc., nor have I been provided with promised existing regression tests for BigInteger to see if they need improvement. (If I can find these somewhere, please let me know. They're not apparently in my checkout.) I'm still hoping to get these changes into Java 1.7, because the performance increase is so significant, and because this is necessary to make Java a feasible platform for number theory and numerics. Recently, Xiaobin Lu submitted patches to improve the performance of BigDecimal, and also improved BigInteger's performance in the process. Great job! Luckily, the overlap between our patches was negligible. Xiaobin may now be the person most suited to look over my patches, as he has worked intimately in the class recently. My fixes are further improved by his fixes to give some truly spectacular improvements in performance for big numbers, which should also help large BigDecimals. I was disappointed, though, that Sun knew about my larger performance improvements for a long time and didn't use this opportunity to evaluate and apply them, nor communicate with me, despite my repeated attempts and my frankly hard work on this. Below, interspersed among my previous statements, are some updated performance numbers with my patches and Xiaobin's. For multiplying the numbers 3^1000000 * 3^1000001, (with my fixes to do the exponentiation hundreds of thousands of times faster factored out; without these, JDK 1.6 would be thousands of times slower,) the times for 20 iterations are: JDK 1.6 292987 ms OpenJDK1.7 + my Karatsuba 28650 ms OpenJDK1.7 + my Toom/Cook 18054 ms Plus Xiaobin's improvements 12880 ms *** Performance improvement over Java 1.6: 22.7x For multiplying numbers 3^14000000 * 3^14000001, the time for 1 iteration is: JDK 1.6 3499115 ms OpenJDK1.7 + my Karatsuba 89505 ms OpenJDK1.7 + my Toom/Cook 43636 ms Plus Xiaobin's improvements 29813 ms *** Performance improvement over Java 1.6: 117.3x You can see that operations that weren't even feasible to consider in JDK 1.6 are now possible in Java. Operations that used to take almost an hour can be done in less than 30 seconds. This encompasses a lot of different bug reports: 4228681: Some BigInteger operations are slow with very large numbers http://bugs.sun.com/bugdatabase/view_bug.do?bug_id=4228681 (This was closed but never fixed.) 4837946: Implement Karatsuba multiplication algorithm in BigInteger http://bugs.sun.com/bugdatabase/view_bug.do?bug_id=4837946 This is done, along with Toom-Cook multiplication. My implementation is intended to be easy to read, understand, and check. It significantly improves multiplication performance for large numbers. 4646474: BigInteger.pow() algorithm slow in 1.4.0 http://bugs.sun.com/bugdatabase/view_bug.do?bug_id=4646474 This is improved in many ways: * Rewrote pow() to use Karatsuba and Toom-Cook multiplication * Implemented Karatsuba squaring * Immplemented 3-way Toom-Cook squaring * Found good threshholds for Karatsuba and Toom-Cook squaring * Added an optimization to use left-shifting for multiples of 2 in the base. This improved speed by thousands of times for things like Mersenne numbers, and may be one of the most significant improvements for practical prgrams. 4641897: BigInteger.toString() algorithm slow for large numbers http://bugs.sun.com/bugdatabase/view_bug.do?bug_id=4641897 This algorithm uses a very inefficient algorithm for large numbers. I plan to replace it with a recursive divide-and-conquer algorithm devised by Schoenhage and Strassen. I have developed and tested this in my own software. This operates hundreds or thousands of times faster than the current version for large numbers. What takes a day in Java 1.6, I can do in 5 minutes. It will also benefit from faster multiplication and exponentiation. This is likely to be the most controversial algorithm because it has potential threading and synchronization and caching and memory-vs-speed tradeoffs, so I'm going to submit it separately, if I can get Sun to review the multiply and pow patches first. My patches are designed to be as readable and simple as possible. They all build on existing functions, and eschew lots of low-level bit-fiddling, as those types of changes are harder to understand and debug. I think it's best to get working algorithms with better asymptotic efficiency, as those will vastly improve performance for large numbers, and tune them by doing more low-level bit fiddling later if necessary. Even without being tuned to the nth degree, the new algorithms are vastly faster for large numbers, and identical for small numbers. I've been asked by Sun to submit my patches in small chunks, so I plan to submit just the multiplication and squaring patch, and leave the patches for pow() for later unless I hear otherwise. I'd rather they had it *all* and got it tested and integrated as soon as possible, after all this waiting, and the work it takes to separate out working code to make a smaller patch. I will be submitting a patch in the next day or two. I'm running my *huge* regression tests which produce about 250 GB of output and take at least a day to run. (And you have to run it twice against a known good platform to have something to compare to... luckily I've done that.) For those who'd just like to replace their file with my improved version (includes Xiaobin's patches), it's available at: http://futureboy.us/temp/BigInteger.java From the queries I get, this is important to a lot of people. The performance of BigInteger can be improved by tens or hundreds or thousands of times (or even more in the case of certain arguments of pow()), and should be done to make Java a more viable platform for numerics. This work has been in Sun's hands for a long time, and really needs to get into 1.7. -- Alan Eliasen eliasen at mindspring.com http://futureboy.us/ From aph at redhat.com Thu Jun 4 12:30:19 2009 From: aph at redhat.com (Andrew Haley) Date: Thu, 04 Jun 2009 13:30:19 +0100 Subject: Further BigInteger performance improvements In-Reply-To: <4A27B7EA.8040809@mindspring.com> References: <47A14D21.8020807@mindspring.com> <47BB539B.8090901@mindspring.com> <87skzo6l5r.fsf@mid.deneb.enyo.de> <47BCAD9C.1030002@mindspring.com> <47E89FE3.6080203@mindspring.com> <4A27B7EA.8040809@mindspring.com> Message-ID: <4A27BE5B.8070302@redhat.com> Alan Eliasen wrote: > From the queries I get, this is important to a lot of people. The > performance of BigInteger can be improved by tens or hundreds or > thousands of times (or even more in the case of certain arguments of > pow()), and should be done to make Java a more viable platform for numerics. > > This work has been in Sun's hands for a long time, and really needs > to get into 1.7. You give examples of the speedup for very large bignums, but you don't say the size of numbers at which your approach becomes faster than the current code. Of course any asymptotic improvement helps with numbers that are half a million decimal digits long, but where's the crossover? Andrew. From dl at cs.oswego.edu Thu Jun 4 12:32:10 2009 From: dl at cs.oswego.edu (Doug Lea) Date: Thu, 04 Jun 2009 08:32:10 -0400 Subject: Faster HashMap implementation In-Reply-To: References: Message-ID: <4A27BECA.6090106@cs.oswego.edu> Alex Yakovlev wrote: > Hello, > > While playing with scala programming language I've come to a HashMap > implementation > that appears to be faster than current java.util.HashMap, also with a > lower memory footprint. > This is a promising approach. The indirection using the index array makes for a better time/space tradeoff than current HashMap for many usages. There are however a couple of constraints on HashMap that have made it difficult to replace it with overhauled internals, despite a few explorations in doing so. The main one is that LinkedHashMap is declared as a subclass of HashMap. There's not an obvious way to add insertion- or access- ordered links to your version. This still might not be a huge obstacle, since with some care, and some wastage, LinkedHashMap could ignore its superclass implementation and re-establish current approach. Also, the meaning/impact of load-factor parameters differs in your version. It seems possible to reinterpret these internally to provide approximately equivalent statistical properties. (For example a loadFactor argument of 2.0 might translate into internal one of around 0.9.) Also, the time/space impact of per-node Entry allocation in entrySet iterators will be noticeable to some applications. This is likely not a big concern though. Across such issues, it would take some further work to make a strong case for actually replacing HashMap. We once faced similar issues for IdentityHashMap, which doesn't bear any class relationship to HashMap, doesn't expose load factors, and is even more compact than yours since it doesn't cache hashCodes. Not caching hashCodes, and so using IdentityHashMap-style table would also be a good idea for many common keys like Strings, numerical keys, and key classes that don't override hashCode, but users have come to expect that HashMap will not recompute hashCodes after insertion. If we had reified generics, we might be able to do something about this but otherwise there seems to be no good way to adapt internal algorithms to key types. Similarly for values. -Doug From i30817 at gmail.com Thu Jun 4 18:10:16 2009 From: i30817 at gmail.com (Paulo Levi) Date: Thu, 4 Jun 2009 19:10:16 +0100 Subject: core-libs-dev Digest, Vol 26, Issue 4 In-Reply-To: References: Message-ID: <212322090906041110n5dd58fa9qf6961b08395c3f40@mail.gmail.com> I hope you get this in. -------------- next part -------------- An HTML attachment was scrubbed... URL: From eliasen at mindspring.com Thu Jun 4 22:58:48 2009 From: eliasen at mindspring.com (Alan Eliasen) Date: Thu, 04 Jun 2009 16:58:48 -0600 Subject: Further BigInteger performance improvements In-Reply-To: <4A27BE5B.8070302@redhat.com> References: <47A14D21.8020807@mindspring.com> <47BB539B.8090901@mindspring.com> <87skzo6l5r.fsf@mid.deneb.enyo.de> <47BCAD9C.1030002@mindspring.com> <47E89FE3.6080203@mindspring.com> <4A27B7EA.8040809@mindspring.com> <4A27BE5B.8070302@redhat.com> Message-ID: <4A2851A8.4080208@mindspring.com> Andrew Haley wrote: > You give examples of the speedup for very large bignums, but you don't say > the size of numbers at which your approach becomes faster than the current > code. Of course any asymptotic improvement helps with numbers that are > half a million decimal digits long, but where's the crossover? Andrew, Good question. Like most Karatsuba and Toom-Cook implementations, the optimal crossover point is vague, as it can depend on the size of both operands, and the curves don't separate fast at that point. If you look at the updated file (again, at http://futureboy.us/temp/BigInteger.java ) you'll see the crossover points: /** * The threshold value for using Karatsuba multiplication. If the number * of ints in both mag arrays are greater than this number, then * Karatsuba multiplication will be used. This value is found * experimentally to work well. */ private static final int KARATSUBA_THRESHOLD = 50; /** * The threshold value for using 3-way Toom-Cook multiplication. * If the number of ints in both mag arrays are greater than this number, * then Toom-Cook multiplication will be used. This value is found * experimentally to work well. */ private static final int TOOM_COOK_THRESHOLD = 75; /** * The threshold value for using Karatsuba squaring. If the number * of ints in the number are larger than this value, * Karatsuba squaring will be used. This value is found * experimentally to work well. */ private static final int KARATSUBA_SQUARE_THRESHOLD = 90; /** * The threshold value for using Toom-Cook squaring. If the number * of ints in the number are larger than this value, * Karatsuba squaring will be used. This value is found * experimentally to work well. */ private static final int TOOM_COOK_SQUARE_THRESHOLD = 140; Thus, the first crossover for Karatsuba multiplication is at 50*32 bits (1600 bits), or about 482 decimal digits. Toom-Cook at 2400 bits. (Hint: divide the number of bits by 3.32 to get the approximate number of decimal digits.) These crossovers are determined experimentally, of course, and constants are chosen that work well for a variety of number forms and operand combinations. These numbers were found to work well after a lot of timing runs with a lot of different number sizes and forms. Of course, it's possible that different thresholds can work better on different architectures and VM settings. I'm not proposing we tune for each architecture, but rather choose conservative crossover thresholds which work well, which I believe these are. It generally isn't damaging if you're just "in the ballpark" when setting your thresholds; the algorithms will still work fine, and performance will still be very similar for most operands. The curves don't diverge too quickly around the crossover point, and there's a lot of random variation for different size operands, and all of your threshold points interact with each other, so it's a multi-dimensional optimization problem. In fact, the effects of choosing different thresholds is hard to predict. There may be little local minima and maxima in performance whether you choose, say, 45 or 48 or 50 or 52 for the crossover, and operands of different lengths may behave quite differently with different crossovers. You won't determine a good crossover point by asymptotic analysis. You also have to see if a certain combination of crossovers exhibits anomalously bad behavior for certain combinations of operand sizes and avoid those, which is why it's as much black art as science. The crossover point also depends on how efficient your "grade school" multiplication vs. your Karatsuba multiplication vs. your Toom-Cook implementation is (and there are *lots* of different ways to split and combine numbers in the Toom-Cook method.) My method for Toom-Cook is the "optimal" algorithm found by Marco Bodrato, (see citations in source code) who is perhaps the leading researcher into Toom-Cook multiplication. He also analyzed this patch, corrected my original Toom-Cook to a slightly different, more optimal variation, (and this changed the Toom-Cook threshold downwards somewhat, as it was a bit faster.) There are also possible optimizations in the future for "unbalanced" operand sizes, where one operand is significantly larger than the other. There are unbalanced Toom-Cook multiplies that can achieve better performance than the straight 3-way Toom-Cook. And of course there are higher-order Toom-Cook multiplications possible, for even larger numbers, but the marginal benefits of higher orders diminishes, and the code is more complex. Again, these crossovers were found experimentally to work well. They still work well with tests of Xiaobin's improvements to multiplication, but I may run yet another set of exhaustive tuning tests to see if the thresholds can be again improved after my massive regression test finishes, hopefully in the next 24 hours. Note that my optimizations for the pow() function give vastly better performance at even small bit sizes for many operands, as they factor out powers of 2 in the exponent and perform these very rapidly as bit-shifts. -- Alan Eliasen eliasen at mindspring.com http://futureboy.us/ From eliasen at mindspring.com Thu Jun 4 23:15:40 2009 From: eliasen at mindspring.com (Alan Eliasen) Date: Thu, 04 Jun 2009 17:15:40 -0600 Subject: Further BigInteger performance improvements In-Reply-To: <4A2851A8.4080208@mindspring.com> References: <47A14D21.8020807@mindspring.com> <47BB539B.8090901@mindspring.com> <87skzo6l5r.fsf@mid.deneb.enyo.de> <47BCAD9C.1030002@mindspring.com> <47E89FE3.6080203@mindspring.com> <4A27B7EA.8040809@mindspring.com> <4A27BE5B.8070302@redhat.com> <4A2851A8.4080208@mindspring.com> Message-ID: <4A28559C.4010802@mindspring.com> Alan Eliasen wrote: > Note that my optimizations for the pow() function give vastly better > performance at even small bit sizes for many operands, as they factor > out powers of 2 in the exponent and perform these very rapidly as > bit-shifts. Oops. I mean powers of 2 in the *base*, of course. The exponentiation of these can be done extremely rapidly as left-shifts, and the remaining portion of the number can be exponentiated more rapidly because it's smaller. Otherwise, exponentiation is done by repeated squaring as before, but with much better algorithms that take advantage of new implementations of Karatsuba and Toom-Cook squaring for larger operands. -- Alan Eliasen eliasen at mindspring.com http://futureboy.us/ From alex14n at gmail.com Fri Jun 5 05:05:56 2009 From: alex14n at gmail.com (Alex Yakovlev) Date: Fri, 5 Jun 2009 09:05:56 +0400 Subject: Faster HashMap implementation In-Reply-To: <4A27BECA.6090106@cs.oswego.edu> References: <4A27BECA.6090106@cs.oswego.edu> Message-ID: Doug, On Thu, Jun 4, 2009 at 4:32 PM, Doug Lea
wrote: > > The main one is that LinkedHashMap is declared as a > subclass of HashMap. There's not > an obvious way to add insertion- or access- ordered links > to your version. Well, it can be done by adding another index array, I've commited it to http://github.com/alex14n/CompactHashMap have a look. It is not tested, sorry, so there might be some bugs, code is not optimized and even basic HashMap performance can degrade because of couple of extra virtual calls. Also, now it has to create a new Entry on each removeEldestEntry check - maybe it could be cached, but I am not sure if my approach would be better than just using original version. Also, the meaning/impact of load-factor parameters differs > in your version. It seems possible to reinterpret these internally > to provide approximately equivalent statistical properties. > (For example a loadFactor argument of 2.0 might translate into > internal one of around 0.9.) Now I just force load factor to 1 if argument is greater than 1. I've also added some modCount/ConcurrentModificationException checks for better compatibility with original version, again I am not sure if it is enough. Also, the time/space impact of per-node Entry allocation in > entrySet iterators will be noticeable to some applications. > This is likely not a big concern though. Hope so. Across such issues, it would take some further work to > make a strong case for actually replacing HashMap. I hope my recent modifications will help. If we had reified generics, we might be able to do something > about this but otherwise there seems to be no good > way to adapt internal algorithms to key types. Similarly > for values. > Scala language announce type specializations that will help this greatly: http://lamp.epfl.ch/~dragos/files/scala-spec.pdf For java there already are good implementations like fastutil: http://fastutil.dsi.unimi.it/ Alex -------------- next part -------------- An HTML attachment was scrubbed... URL: From aph at redhat.com Fri Jun 5 09:21:27 2009 From: aph at redhat.com (Andrew Haley) Date: Fri, 05 Jun 2009 10:21:27 +0100 Subject: Further BigInteger performance improvements In-Reply-To: <4A2851A8.4080208@mindspring.com> References: <47A14D21.8020807@mindspring.com> <47BB539B.8090901@mindspring.com> <87skzo6l5r.fsf@mid.deneb.enyo.de> <47BCAD9C.1030002@mindspring.com> <47E89FE3.6080203@mindspring.com> <4A27B7EA.8040809@mindspring.com> <4A27BE5B.8070302@redhat.com> <4A2851A8.4080208@mindspring.com> Message-ID: <4A28E397.1040209@redhat.com> Alan Eliasen wrote: > Andrew Haley wrote: >> You give examples of the speedup for very large bignums, but you >> don't say the size of numbers at which your approach becomes faster >> than the current code. Of course any asymptotic improvement helps >> with numbers that are half a million decimal digits long, but >> where's the crossover? > > Good question. Like most Karatsuba and Toom-Cook > implementations, the optimal crossover point is vague, as it can > depend on the size of both operands, and the curves don't separate > fast at that point. If you look at the updated file (again, at > http://futureboy.us/temp/BigInteger.java ) you'll see the crossover > points: > > Thus, the first crossover for Karatsuba multiplication is at 50*32 > bits (1600 bits), or about 482 decimal digits. Toom-Cook at 2400 bits. OK, that's a little more encouraging. The measures you posted before were for nubers about half a million decimal digits long, which is well into FFT multiplication territory. That we can see some improvement for ~ 500-digit numbers is good to see. > These crossovers are determined experimentally, of course, and > constants are chosen that work well for a variety of number forms and > operand combinations. These numbers were found to work well after a lot > of timing runs with a lot of different number sizes and forms. Of > course, it's possible that different thresholds can work better on > different architectures and VM settings. I'm not proposing we tune for > each architecture, but rather choose conservative crossover thresholds > which work well, which I believe these are. > > It generally isn't damaging if you're just "in the ballpark" when > setting your thresholds; the algorithms will still work fine, and > performance will still be very similar for most operands. Sure, that makes sense. Andrew. From martinrb at google.com Sat Jun 6 02:13:07 2009 From: martinrb at google.com (Martin Buchholz) Date: Fri, 5 Jun 2009 19:13:07 -0700 Subject: Review request for 5049299 In-Reply-To: <4A25373D.9090906@sun.com> References: <4A1678DB.9010209@sun.com> <4A17B3BB.4090706@redhat.com> <1ccfd1c10905231112k61d8e13h31f693951ed91b76@mail.gmail.com> <4A1849CD.3060306@redhat.com> <1ccfd1c10905231810k17a99cd3vfa68d6108e28b46@mail.gmail.com> <4A191063.5060401@redhat.com> <1ccfd1c10905241809m10ded53bpb4c9360f76b36888@mail.gmail.com> <1ccfd1c10905301051j527d2e91m217020d037ac1732@mail.gmail.com> <1ccfd1c10906011727g794e9ca0m89c350fa8dae7c54@mail.gmail.com> <4A25373D.9090906@sun.com> Message-ID: <1ccfd1c10906051913o1850b256pca173d2568b25736@mail.gmail.com> Michael, I think the best way to handle the coordination is in two steps. I'd like to get my Linux-clone changes in first (you should review, I will commit) and then we switch hats and I will review your Solaris changes. It seems best to do this in two steps: to better place blame when it breaks (this is very tricky stuff to get right). If you agree, please review my posted changes. Aside: Instead of griping about the missing execvpe, I filed a bug against glibc, and was surprised to find that Ulrich Drepper had implemented it a couple of days later. It will probably be in glibc-2.11. Perhaps in 5 years we can use it ourselves...). Thanks, Uli! Martin On Tue, Jun 2, 2009 at 07:29, Michael McMahon wrote: > Martin, > > I had done something similar with clone & exec for Linux, but hadn't got > round to testing it. > So, it seems reasonable to take yours. Do you want to send me your updated > versions of > process_md.c and the test? I can take care of the merge with the Solaris > code. -------------- next part -------------- An HTML attachment was scrubbed... URL: From fw at deneb.enyo.de Sat Jun 6 06:44:33 2009 From: fw at deneb.enyo.de (Florian Weimer) Date: Sat, 06 Jun 2009 08:44:33 +0200 Subject: Further BigInteger performance improvements In-Reply-To: <4A28559C.4010802@mindspring.com> (Alan Eliasen's message of "Thu, 04 Jun 2009 17:15:40 -0600") References: <47A14D21.8020807@mindspring.com> <47BB539B.8090901@mindspring.com> <87skzo6l5r.fsf@mid.deneb.enyo.de> <47BCAD9C.1030002@mindspring.com> <47E89FE3.6080203@mindspring.com> <4A27B7EA.8040809@mindspring.com> <4A27BE5B.8070302@redhat.com> <4A2851A8.4080208@mindspring.com> <4A28559C.4010802@mindspring.com> Message-ID: <87ab4l275a.fsf@mid.deneb.enyo.de> * Alan Eliasen: > Alan Eliasen wrote: >> Note that my optimizations for the pow() function give vastly better >> performance at even small bit sizes for many operands, as they factor >> out powers of 2 in the exponent and perform these very rapidly as >> bit-shifts. > > Oops. I mean powers of 2 in the *base*, of course. The > exponentiation of these can be done extremely rapidly as left-shifts, > and the remaining portion of the number can be exponentiated more > rapidly because it's smaller. To provide some more background, most of us probably worry about BigInteger performance in the 512 to 2048 bit range because that's the range used for RSA cryptography (assuming that Java uses the Chinese Reminder Theorem optimization for private key operations). From alex14n at gmail.com Sat Jun 6 17:20:33 2009 From: alex14n at gmail.com (Alex Yakovlev) Date: Sat, 6 Jun 2009 21:20:33 +0400 Subject: Faster HashMap implementation In-Reply-To: <4A27BECA.6090106@cs.oswego.edu> References: <4A27BECA.6090106@cs.oswego.edu> Message-ID: Doug, I've finished HashMap, HashSet, LinkedHashMap, and LinkedHashSet porting to my approach. I run several tests I've found in jdk/test/java/util/* that are directly related to these classes, not sure I've found all of them. I have not compiled the whole jdk classes - only these 4 (removed 'Fast' prefix from names and put them into java.util package) and replaced those in my rt.jar of binary jdk7b59. Applications like Eclipse and Tomcat are running fine. Are there any thorough regression/compliance tests for java collections? On Thu, Jun 4, 2009 at 4:32 PM, Doug Lea
wrote: > Alex Yakovlev wrote: > >> While playing with scala programming language I've come toa HashMap >> implementation >> that appears to be faster than current java.util.HashMap, also with a >> lower memory footprint. >> > > This is a promising approach. The indirection using the > index array makes for a better time/space tradeoff than > current HashMap for many usages. > > There are however a couple of constraints on HashMap > that have made it difficult to replace it with > overhauled internals, despite a few explorations in > doing so. > > The main one is that LinkedHashMap is declared as a > subclass of HashMap. There's not > an obvious way to add insertion- or access- ordered links > to your version. This still might not be a huge obstacle, > since with some care, and some wastage, LinkedHashMap > could ignore its superclass implementation and re-establish > current approach. > > Also, the meaning/impact of load-factor parameters differs > in your version. It seems possible to reinterpret these internally > to provide approximately equivalent statistical properties. > (For example a loadFactor argument of 2.0 might translate into > internal one of around 0.9.) > > Across such issues, it would take some further work to > make a strong case for actually replacing HashMap. > -------------- next part -------------- An HTML attachment was scrubbed... URL: From Joe.Kearney at morganstanley.com Sat Jun 6 17:47:16 2009 From: Joe.Kearney at morganstanley.com (Joe Kearney) Date: Sat, 6 Jun 2009 18:47:16 +0100 Subject: Faster HashMap implementation In-Reply-To: References: <4A27BECA.6090106@cs.oswego.edu> Message-ID: Alex, Might I suggest you look at the test framework in Google Collections? Extremely good coverage and only a couple of lines to set up. Tests for maps in java.util: http://www.google.com/codesearch/p?hl=en#YXcrkXezIpQ/trunk/testfw/com/google/common/collect/testing/TestsForMapsInJavaUtil.java See for example line 118. Thanks, Joe 2009/6/6 Alex Yakovlev > Doug, > > I've finished HashMap, HashSet, LinkedHashMap, and LinkedHashSet > porting to my approach. I run several tests I've found in > jdk/test/java/util/* > that are directly related to these classes, not sure I've found all of > them. > > I have not compiled the whole jdk classes - only these 4 (removed 'Fast' > prefix from names and put them into java.util package) and replaced > those in my rt.jar of binary jdk7b59. Applications like Eclipse and Tomcat > are running fine. > > Are there any thorough regression/compliance tests for java collections? > > On Thu, Jun 4, 2009 at 4:32 PM, Doug Lea
wrote: > >> Alex Yakovlev wrote: >> >>> While playing with scala programming language I've come toa HashMap >>> implementation >>> that appears to be faster than current java.util.HashMap, also with a >>> lower memory footprint. >>> >> >> This is a promising approach. The indirection using the >> index array makes for a better time/space tradeoff than >> current HashMap for many usages. >> >> There are however a couple of constraints on HashMap >> that have made it difficult to replace it with >> overhauled internals, despite a few explorations in >> doing so. >> >> The main one is that LinkedHashMap is declared as a >> subclass of HashMap. There's not >> an obvious way to add insertion- or access- ordered links >> to your version. This still might not be a huge obstacle, >> since with some care, and some wastage, LinkedHashMap >> could ignore its superclass implementation and re-establish >> current approach. >> >> Also, the meaning/impact of load-factor parameters differs >> in your version. It seems possible to reinterpret these internally >> to provide approximately equivalent statistical properties. >> (For example a loadFactor argument of 2.0 might translate into >> internal one of around 0.9.) >> >> Across such issues, it would take some further work to >> make a strong case for actually replacing HashMap. >> > > -------------- next part -------------- An HTML attachment was scrubbed... URL: From alex14n at gmail.com Sat Jun 6 19:17:55 2009 From: alex14n at gmail.com (Alex Yakovlev) Date: Sat, 6 Jun 2009 23:17:55 +0400 Subject: Faster HashMap implementation In-Reply-To: References: <4A27BECA.6090106@cs.oswego.edu> Message-ID: Joe, Thank you for the link. What I found strange is that non-Linked versions (simple HashSet and HashMap) passes all tests with CollectionFeature.KNOWN_ORDER TestSuite source I ran (scala): http://github.com/alex14n/CompactHashMap/blob/e2b81fb9ccf5446eff961d86eb3a0dc084a8a88c/test/Tests.scala Alex On Sat, Jun 6, 2009 at 9:47 PM, Joe Kearney wrote: > Alex, > Might I suggest you look at the test framework in Google Collections? > Extremely good coverage and only a couple of lines to set up. > Tests for maps in java.util: > http://www.google.com/codesearch/p?hl=en#YXcrkXezIpQ/trunk/testfw/com/google/common/collect/testing/TestsForMapsInJavaUtil.java > See > for example line 118. > > Thanks, > Joe > -------------- next part -------------- An HTML attachment was scrubbed... URL: From gkorland at gmail.com Sun Jun 7 14:01:59 2009 From: gkorland at gmail.com (Guy Korland) Date: Sun, 7 Jun 2009 17:01:59 +0300 Subject: RMI stuck on network disconnection Message-ID: <79be5fa30906070701o40e0cc27k48730dc927a79aaa@mail.gmail.com> Hi, I found out that an RMI call might never return on network disconnection and will not throw RemoteException as expected. The scenario is very simple: 1. Find a remote stub (the stub should be behind a router). 2. Physically disconnect the server from the LAN. 3. Invoke a method from the client, the client can stuck without an exception up to 30 min. Any ideas? -- Thanks, Guy -------------- next part -------------- An HTML attachment was scrubbed... URL: From fw at deneb.enyo.de Sun Jun 7 17:46:21 2009 From: fw at deneb.enyo.de (Florian Weimer) Date: Sun, 07 Jun 2009 19:46:21 +0200 Subject: Further BigInteger performance improvements In-Reply-To: <4A2AFEE8.1020304@mindspring.com> (Alan Eliasen's message of "Sat, 06 Jun 2009 17:42:32 -0600") References: <47A14D21.8020807@mindspring.com> <47BB539B.8090901@mindspring.com> <87skzo6l5r.fsf@mid.deneb.enyo.de> <47BCAD9C.1030002@mindspring.com> <47E89FE3.6080203@mindspring.com> <4A27B7EA.8040809@mindspring.com> <4A27BE5B.8070302@redhat.com> <4A2851A8.4080208@mindspring.com> <4A28559C.4010802@mindspring.com> <87ab4l275a.fsf@mid.deneb.enyo.de> <4A2AFEE8.1020304@mindspring.com> Message-ID: <87ws7oq6mq.fsf@mid.deneb.enyo.de> * Alan Eliasen: > Florian Weimer wrote: >> To provide some more background, most of us probably worry about >> BigInteger performance in the 512 to 2048 bit range because that's the >> range used for RSA cryptography (assuming that Java uses the Chinese >> Reminder Theorem optimization for private key operations). > > I understand the importance of this range in cryptography, and I > especially understand the greater importance of the even smaller range > that actually makes up the bulk of CPU time spent in cryptography, down > in the 128-256 bit sizes. Most practical crypto schemes tend to use > those public-key algorithms with numbers of 512 to 4096 bits for the > initial key exchange, but then use something like 256-bit AES for > encrypting the actual data stream. The symmetric algorithms relevant in practice (such as AES) don't do arithmetic on big integers. Your To: line was wildly wrong, I tried to correct that. From fw at deneb.enyo.de Mon Jun 8 13:08:45 2009 From: fw at deneb.enyo.de (Florian Weimer) Date: Mon, 08 Jun 2009 15:08:45 +0200 Subject: Fix for Sun Alert 246387 included in OpenJDk 6 b11? Message-ID: <87ab4ic1pe.fsf@mid.deneb.enyo.de> Was the fix for Sun Alert 246387 (aka CVE-2008-5345) included in OpenJDK 6b11? From mark at klomp.org Mon Jun 8 13:32:42 2009 From: mark at klomp.org (Mark Wielaard) Date: Mon, 08 Jun 2009 15:32:42 +0200 Subject: Fix for Sun Alert 246387 included in OpenJDk 6 b11? In-Reply-To: <87ab4ic1pe.fsf@mid.deneb.enyo.de> References: <87ab4ic1pe.fsf@mid.deneb.enyo.de> Message-ID: <1244467962.3661.150.camel@hermans.wildebeest.org> Hi Florian, On Mon, 2009-06-08 at 15:08 +0200, Florian Weimer wrote: > Was the fix for Sun Alert 246387 (aka CVE-2008-5345) included in > OpenJDK 6b11? I believe CVE-2008-5345 is a catch all for a bundle of security update patches: http://mail.openjdk.java.net/pipermail/distro-pkg-dev/2009-March/005209.html which were later folded into OpenJDK6 b16: http://mail.openjdk.java.net/pipermail/jdk6-dev/2009-April/thread.html#436 Cheers, Mark From fw at deneb.enyo.de Mon Jun 8 13:37:23 2009 From: fw at deneb.enyo.de (Florian Weimer) Date: Mon, 08 Jun 2009 15:37:23 +0200 Subject: Fix for Sun Alert 246387 included in OpenJDk 6 b11? In-Reply-To: <1244467962.3661.150.camel@hermans.wildebeest.org> (Mark Wielaard's message of "Mon, 08 Jun 2009 15:32:42 +0200") References: <87ab4ic1pe.fsf@mid.deneb.enyo.de> <1244467962.3661.150.camel@hermans.wildebeest.org> Message-ID: <87d49ealt8.fsf@mid.deneb.enyo.de> * Mark Wielaard: > Hi Florian, > > On Mon, 2009-06-08 at 15:08 +0200, Florian Weimer wrote: >> Was the fix for Sun Alert 246387 (aka CVE-2008-5345) included in >> OpenJDK 6b11? > > I believe CVE-2008-5345 is a catch all for a bundle of security update > patches: > http://mail.openjdk.java.net/pipermail/distro-pkg-dev/2009-March/005209.html > which were later folded into OpenJDK6 b16: > http://mail.openjdk.java.net/pipermail/jdk6-dev/2009-April/thread.html#436 The dates don't match. Sun Alert 246387 was published in December 2008. Lillian's commit were prompted by a later round of fixes, I think. From mark at klomp.org Mon Jun 8 13:38:58 2009 From: mark at klomp.org (Mark Wielaard) Date: Mon, 08 Jun 2009 15:38:58 +0200 Subject: Fix for Sun Alert 246387 included in OpenJDk 6 b11? In-Reply-To: <1244467962.3661.150.camel@hermans.wildebeest.org> References: <87ab4ic1pe.fsf@mid.deneb.enyo.de> <1244467962.3661.150.camel@hermans.wildebeest.org> Message-ID: <1244468338.3661.155.camel@hermans.wildebeest.org> On Mon, 2009-06-08 at 15:32 +0200, Mark Wielaard wrote: > Hi Florian, > > On Mon, 2009-06-08 at 15:08 +0200, Florian Weimer wrote: > > Was the fix for Sun Alert 246387 (aka CVE-2008-5345) included in > > OpenJDK 6b11? > > I believe CVE-2008-5345 is a catch all for a bundle of security update > patches: > http://mail.openjdk.java.net/pipermail/distro-pkg-dev/2009-March/005209.html > which were later folded into OpenJDK6 b16: > http://mail.openjdk.java.net/pipermail/jdk6-dev/2009-April/thread.html#436 Better URLs of the release notes, including CVS numbers: http://langel.wordpress.com/2009/02/02/icedtea6-14-released/ and other bug numbers: http://blogs.sun.com/darcy/entry/openjdk_6_sources_for_b14 (Note b14, I said b16 before, but that contained other security fixes) Cheers, Mark From fw at deneb.enyo.de Mon Jun 8 13:44:59 2009 From: fw at deneb.enyo.de (Florian Weimer) Date: Mon, 08 Jun 2009 15:44:59 +0200 Subject: Fix for Sun Alert 246387 included in OpenJDk 6 b11? In-Reply-To: <1244468338.3661.155.camel@hermans.wildebeest.org> (Mark Wielaard's message of "Mon, 08 Jun 2009 15:38:58 +0200") References: <87ab4ic1pe.fsf@mid.deneb.enyo.de> <1244467962.3661.150.camel@hermans.wildebeest.org> <1244468338.3661.155.camel@hermans.wildebeest.org> Message-ID: <8763f6algk.fsf@mid.deneb.enyo.de> * Mark Wielaard: > On Mon, 2009-06-08 at 15:32 +0200, Mark Wielaard wrote: >> Hi Florian, >> >> On Mon, 2009-06-08 at 15:08 +0200, Florian Weimer wrote: >> > Was the fix for Sun Alert 246387 (aka CVE-2008-5345) included in >> > OpenJDK 6b11? >> >> I believe CVE-2008-5345 is a catch all for a bundle of security update >> patches: >> http://mail.openjdk.java.net/pipermail/distro-pkg-dev/2009-March/005209.html >> which were later folded into OpenJDK6 b16: >> http://mail.openjdk.java.net/pipermail/jdk6-dev/2009-April/thread.html#436 > > Better URLs of the release notes, including CVS numbers: > http://langel.wordpress.com/2009/02/02/icedtea6-14-released/ > and other bug numbers: > http://blogs.sun.com/darcy/entry/openjdk_6_sources_for_b14 > (Note b14, I said b16 before, but that contained other security fixes) Sorry, but this is way too late to be relevant to my question (which is about b11, not b14): The CVE-2008-5345 fix was not listed explicitly in the b14 round of fixes, otherwise I'd have an isolated patch I could examine. From mark at klomp.org Mon Jun 8 14:04:00 2009 From: mark at klomp.org (Mark Wielaard) Date: Mon, 08 Jun 2009 16:04:00 +0200 Subject: Fix for Sun Alert 246387 included in OpenJDk 6 b11? In-Reply-To: <8763f6algk.fsf@mid.deneb.enyo.de> References: <87ab4ic1pe.fsf@mid.deneb.enyo.de> <1244467962.3661.150.camel@hermans.wildebeest.org> <1244468338.3661.155.camel@hermans.wildebeest.org> <8763f6algk.fsf@mid.deneb.enyo.de> Message-ID: <1244469840.3661.156.camel@hermans.wildebeest.org> Hi Florian, On Mon, 2009-06-08 at 15:44 +0200, Florian Weimer wrote: > Sorry, but this is way too late to be relevant to my question (which > is about b11, not b14): The CVE-2008-5345 fix was not listed > explicitly in the b14 round of fixes, otherwise I'd have an isolated > patch I could examine. OK. Can you give some more information about what CVE-2008-5345 precisely covers. I couldn't find any details. Thanks, Mark From Michael.McMahon at Sun.COM Mon Jun 8 14:08:37 2009 From: Michael.McMahon at Sun.COM (Michael McMahon) Date: Mon, 08 Jun 2009 15:08:37 +0100 Subject: Review request for 5049299 In-Reply-To: <1ccfd1c10906051913o1850b256pca173d2568b25736@mail.gmail.com> References: <4A1678DB.9010209@sun.com> <4A17B3BB.4090706@redhat.com> <1ccfd1c10905231112k61d8e13h31f693951ed91b76@mail.gmail.com> <4A1849CD.3060306@redhat.com> <1ccfd1c10905231810k17a99cd3vfa68d6108e28b46@mail.gmail.com> <4A191063.5060401@redhat.com> <1ccfd1c10905241809m10ded53bpb4c9360f76b36888@mail.gmail.com> <1ccfd1c10905301051j527d2e91m217020d037ac1732@mail.gmail.com> <1ccfd1c10906011727g794e9ca0m89c350fa8dae7c54@mail.gmail.com> <4A25373D.9090906@sun.com> <1ccfd1c10906051913o1850b256pca173d2568b25736@mail.gmail.com> Message-ID: <4A2D1B65.5020002@sun.com> That's fine Martin. We can do it that way. Do you really need to #include ? As far as I can see clone() only requires When you allocate the clone stack for the child the memory is byte aligned. Is this ok for Linux or should stacks be aligned on larger boundaries? Also, I don't follow why we need the execve_as_traditional_shell_script() function. Can you explain the reason for that? Thanks, Michael. Martin Buchholz wrote: > Michael, > > I think the best way to handle the coordination is in two steps. > I'd like to get my Linux-clone changes in first (you should review, > I will commit) > and then we switch hats and I will review your Solaris changes. > It seems best to do this in two steps: to better place blame when > it breaks (this is very tricky stuff to get right). > If you agree, please review my posted changes. > > Aside: Instead of griping about the missing execvpe, > I filed a bug against glibc, and was surprised to find > that Ulrich Drepper had implemented it a couple of days later. > It will probably be in glibc-2.11. Perhaps in 5 years we can > use it ourselves...). Thanks, Uli! > > Martin > > On Tue, Jun 2, 2009 at 07:29, Michael McMahon > wrote: > > Martin, > > I had done something similar with clone & exec for Linux, but > hadn't got round to testing it. > So, it seems reasonable to take yours. Do you want to send me your > updated versions of > process_md.c and the test? I can take care of the merge with the > Solaris code. > > From fw at deneb.enyo.de Mon Jun 8 14:08:58 2009 From: fw at deneb.enyo.de (Florian Weimer) Date: Mon, 08 Jun 2009 16:08:58 +0200 Subject: Fix for Sun Alert 246387 included in OpenJDk 6 b11? In-Reply-To: <1244469840.3661.156.camel@hermans.wildebeest.org> (Mark Wielaard's message of "Mon, 08 Jun 2009 16:04:00 +0200") References: <87ab4ic1pe.fsf@mid.deneb.enyo.de> <1244467962.3661.150.camel@hermans.wildebeest.org> <1244468338.3661.155.camel@hermans.wildebeest.org> <8763f6algk.fsf@mid.deneb.enyo.de> <1244469840.3661.156.camel@hermans.wildebeest.org> Message-ID: <87vdn695s5.fsf@mid.deneb.enyo.de> * Mark Wielaard: > On Mon, 2009-06-08 at 15:44 +0200, Florian Weimer wrote: >> Sorry, but this is way too late to be relevant to my question (which >> is about b11, not b14): The CVE-2008-5345 fix was not listed >> explicitly in the b14 round of fixes, otherwise I'd have an isolated >> patch I could examine. > > OK. Can you give some more information about what CVE-2008-5345 > precisely covers. I couldn't find any details. This is precisely my problem. I want to make sure that we have the fix in Debian stable. If it was included in the b11 source drop, we should be fine. I thought Sun might be willing to share this piece of information, even if they don't want to disclose the precise nature of the bug. From Alan.Bateman at Sun.COM Mon Jun 8 14:17:19 2009 From: Alan.Bateman at Sun.COM (Alan Bateman) Date: Mon, 08 Jun 2009 15:17:19 +0100 Subject: Fix for Sun Alert 246387 included in OpenJDk 6 b11? In-Reply-To: <87ab4ic1pe.fsf@mid.deneb.enyo.de> References: <87ab4ic1pe.fsf@mid.deneb.enyo.de> Message-ID: <4A2D1D6F.4050402@sun.com> Florian Weimer wrote: > Was the fix for Sun Alert 246387 (aka CVE-2008-5345) included in > OpenJDK 6b11? > Is this the alert that you are looking at: http://sunsolve.sun.com/search/document.do?assetkey=1-66-246387-1 I checked the Sun bug that is referenced (6704154) and its in the plugin area and so not applicable to OpenJDK. -Alan. From fw at deneb.enyo.de Mon Jun 8 14:19:05 2009 From: fw at deneb.enyo.de (Florian Weimer) Date: Mon, 08 Jun 2009 16:19:05 +0200 Subject: Fix for Sun Alert 246387 included in OpenJDk 6 b11? In-Reply-To: <4A2D1D6F.4050402@sun.com> (Alan Bateman's message of "Mon, 08 Jun 2009 15:17:19 +0100") References: <87ab4ic1pe.fsf@mid.deneb.enyo.de> <4A2D1D6F.4050402@sun.com> Message-ID: <87k53m95ba.fsf@mid.deneb.enyo.de> * Alan Bateman: > Florian Weimer wrote: >> Was the fix for Sun Alert 246387 (aka CVE-2008-5345) included in >> OpenJDK 6b11? >> > Is this the alert that you are looking at: > http://sunsolve.sun.com/search/document.do?assetkey=1-66-246387-1 Yes. > I checked the Sun bug that is referenced (6704154) and its in the > plugin area and so not applicable to OpenJDK. Great, thanks for confirming this! From dl at cs.oswego.edu Mon Jun 8 14:24:57 2009 From: dl at cs.oswego.edu (Doug Lea) Date: Mon, 08 Jun 2009 10:24:57 -0400 Subject: Faster HashMap implementation In-Reply-To: References: <4A27BECA.6090106@cs.oswego.edu> Message-ID: <4A2D1F39.90601@cs.oswego.edu> Alex Yakovlev wrote: > Doug, > > I've finished HashMap, HashSet, LinkedHashMap, and LinkedHashSet > porting to my approach. I run several tests I've found in > jdk/test/java/util/* > that are directly related to these classes, not sure I've found all of them. > There are a lot of tests around for HashMaps, strewn all over. The ones in you found are the "jtreg" regression tests that correspond to bug reports. We JSR166 folks have a bunch of miscellaneous functionality and performance tests for maps. Some of them are geared to concurrent maps, but if you grab the tarball at http://gee.cs.oswego.edu/cgi-bin/viewcvs.cgi/jsr166/src/test/loops/ you can browse through *Map*.java to find ones that apply, such as MapCheck, MapWordLoops, DenseMapMicroBenchmark, plus a few like IteratorLoops that can be used on maps. Sorry that there's no documentation for running them. (I just noticed a few were stale in CVS so updated.) While running some of them I noticed that your simplification of our bit-spreading function leads to too many conflicts on some data sets. You probably want to revert to... h ^= (h >>> 20) ^ (h >>> 12); return h ^ (h >>> 7) ^ (h >>> 4); I'll send some other notes on your implementation hopefully soon. HashMaps are also prominent in some SPEC tests, that don't have public sources. One of the main tests (jbb) includes some truly idiotic usages, like using a HashMap instead of a dense array. (The DenseMapMicrobenchmark independently tests some similar usages). Because SPEC scores are overly important to vendors, some of them do weird things with HashMap (like attempt to use alternative implementations that are rarely otherwise better) under switches like -XX:+ +AggressiveOpts. Beware. -Doug From Alan.Bateman at Sun.COM Mon Jun 8 14:27:03 2009 From: Alan.Bateman at Sun.COM (Alan Bateman) Date: Mon, 08 Jun 2009 15:27:03 +0100 Subject: RMI stuck on network disconnection In-Reply-To: <79be5fa30906070701o40e0cc27k48730dc927a79aaa@mail.gmail.com> References: <79be5fa30906070701o40e0cc27k48730dc927a79aaa@mail.gmail.com> Message-ID: <4A2D1FB7.9020206@sun.com> Guy Korland wrote: > Hi, > > I found out that an RMI call might never return on network > disconnection and will not throw RemoteException as expected. > The scenario is very simple: > > 1. Find a remote stub (the stub should be behind a router). > 2. Physically disconnect the server from the LAN. > 3. Invoke a method from the client, the client can stuck without an > exception up to 30 min. > > Any ideas? > > -- > Thanks, > Guy I would suggest searching the archives of the rmi-users list [1]. Also check the properties that control timeouts [2]. -Alan. [1] http://archives.java.sun.com/archives/rmi-users.html [2] http://java.sun.com/javase/6/docs/technotes/guides/rmi/sunrmiproperties.html From dl at cs.oswego.edu Mon Jun 8 16:07:05 2009 From: dl at cs.oswego.edu (Doug Lea) Date: Mon, 08 Jun 2009 12:07:05 -0400 Subject: Faster HashMap implementation In-Reply-To: References: Message-ID: <4A2D3729.3040904@cs.oswego.edu> A few notes on your implementation... Alex Yakovlev wrote: > > Main trick is in storing all data in 2 arrays without any HashEntry objects. > One is array of Objects with keys and values, another in a complex array > of indices. > Notice that footprint comparisons with current HashMap are load-dependent. Let C be capacity and N be # items. Assume 32bit pointers. Yours requires about 4*C storage versus 1*C + 5N for HashMap. 4*C = key + value + main part of index + overflow part of index, although a little less with loadFactor < 1 (see below) 5*N = 4 fields per entry plus 1 word Object header. There is a crossover point where yours uses less memory, which will often be the case when grown using the usual resizing thresholds. Plus GC should be faster in yours. However, statistically, most HashMaps have very few elements. I once instrumented some of the "reference workload" usages and found that over half of the HashMaps/Hashtables had a max 3 or fewer elements during their lifetimes. So on average using your version will likely increase application footprints. It seems possible to deal with this though. For example, you might consider separately allocating the back-half of index array only when needed. The unloaded overhead is worse for your Linked version, although that is probably less of a concern. It is very difficult to empirically compare footprints in the most interesting cases, of using a bunch of HashMaps of various sizes, in part because GC may or may not actually collect all collectable garbage. > Now I just force load factor to 1 if argument is greater than 1. The effects of loadFactors are pretty much incommensurate across versions. Because you trade element-space for index-space, varying loadFactor have little bearing on total space. (But still a little, since an index costs half of a key+value). While it remains approximately true in yours that loadFactors < 1 are the average number of traversals to find existing key near the resize threshold point, probably some more work/thought is needed to make a better story about interpreting them internally. > > When looking for missing key my implementation can be twice faster in > some situations, Right. In the tests I ran, yours is generally faster for failed lookups. It is also sometimes faster for successful lookups for some key types, distributions, and/or machines I test on. (For some reason, more often better on AMDs than Intels or Sparcs). It strikes me that there might be a bigger win available here by using some hybrid strategy based on IdentityHashMap-like single-indirect (i.e., usually at most a single cache miss) for collision-free accesses but using a variant of your clever packed-indices under collisions. I might try this out someday. > > Iteration over elements is a sequental array read which is faster. > To clone such HashMap we only need to clone 2 arrays with is also much > faster. > I don't understand why you don't simply traverse the element array for HashMap iterators? You cannot do this for your linked version, but it would not hurt to make completely different iterator types. > Thus iteration > order is more predictable and I think it's possible not to throw > ConcurrentModificationException > in most situations. For non-concurrent collections, the goal of ConcurrentModificationExceptions is to help people find/fix incorrect code; not necessarily to only blow up if proceeding is impossible. I hope this helps! Despite all the concerns above, this is the first alternative version of HashMap that I can recall seeing that seems promising enough to continue exploring as a possible replacement. -Doug From fweimer at bfk.de Mon Jun 8 16:13:43 2009 From: fweimer at bfk.de (Florian Weimer) Date: Mon, 08 Jun 2009 18:13:43 +0200 Subject: Faster HashMap implementation In-Reply-To: <4A2D3729.3040904@cs.oswego.edu> (Doug Lea's message of "Mon\, 08 Jun 2009 12\:07\:05 -0400") References: <4A2D3729.3040904@cs.oswego.edu> Message-ID: <82ocsyitzc.fsf@mid.bfk.de> * Doug Lea: > I once instrumented some of the "reference workload" usages > and found that over half of the HashMaps/Hashtables had > a max 3 or fewer elements during their lifetimes. > So on average using your version will likely increase > application footprints. It seems possible to deal with > this though. For example, you might consider separately > allocating the back-half of index array only when needed. Or don't use the hash structure at all and just do a sequential search. Then the index array isn't needed at all. -- Florian Weimer BFK edv-consulting GmbH http://www.bfk.de/ Kriegsstra?e 100 tel: +49-721-96201-1 D-76133 Karlsruhe fax: +49-721-96201-99 From dl at cs.oswego.edu Mon Jun 8 16:34:22 2009 From: dl at cs.oswego.edu (Doug Lea) Date: Mon, 08 Jun 2009 12:34:22 -0400 Subject: Faster HashMap implementation In-Reply-To: <82ocsyitzc.fsf@mid.bfk.de> References: <4A2D3729.3040904@cs.oswego.edu> <82ocsyitzc.fsf@mid.bfk.de> Message-ID: <4A2D3D8E.60300@cs.oswego.edu> Florian Weimer wrote: > Or don't use the hash structure at all and just do a sequential > search. Then the index array isn't needed at all. It is possible to use a look-aside strategy for tiny HashMaps. I think one of the Apache commons maps does this. But the idea of using open-address tables to cover this case more nicely doesn't work out very well. While open-addressing is used in IdentityHashMap (and in a very specialized form in ThreadLocal), you cannot live with linear-probed versions otherwise: Many user-defined hashCodes (and some JDK-defined ones too!) are not very good. While we can use bit-spreading corrections to help with this, we cannot do anything about the fact that duplicated hash codes (that is, two different objects with the same hash code) are vastly more common than you'd hope. This causes big pile-ups under linear probing but is only a localized problem with binned maps like current HashMap. Which is one reason to consider the hybrid scheme I mentioned. -Doug From alex14n at gmail.com Mon Jun 8 16:59:11 2009 From: alex14n at gmail.com (Alex Yakovlev) Date: Mon, 8 Jun 2009 20:59:11 +0400 Subject: Faster HashMap implementation In-Reply-To: <4A2D3729.3040904@cs.oswego.edu> References: <4A2D3729.3040904@cs.oswego.edu> Message-ID: Doug, On your previous message: 1) I reverted hash function, 2) did some tuniing for MapCheck benchmark - equals, hashCode, toString, and putAll (if argument have the same type) now does not use entrySet() iterator which is very expensive. Very big speedup would be to reuse Entry object, since in most cases previous one is dropped when next() is called, but not always - entries can be stores somewhere, and I do not know how to tell is it possible to reuse Entry or not. Maybe JVM EscapeAnalysis can somehow help, if we'll declare all iterator-related methods as final? Or use some switch like +XX:+ReuseHashMapIteratorEntry :) Also, in the latest version I made Entry as close to current HashMap as I could - its setValue method not updates the map, getValue re-reads the value from array, etc. Not doing that can improve performance. 3) as for DenseMapMicroBenchmark test - I think that CompactHashMap.scala will perform well in such tests since it stores primitive types in primitive arrays, and it is true my Benchmark.scala on Integers (sample result is included in ReadMe file). On Mon, Jun 8, 2009 at 8:07 PM, Doug Lea
wrote: > Notice that footprint comparisons with current HashMap > are load-dependent. ... > There is a crossover point where yours uses > less memory, which will often be the case when grown using > the usual resizing thresholds. Plus GC should be faster in yours. That is true. > However, statistically, most HashMaps have very few elements. > I once instrumented some of the "reference workload" usages > and found that over half of the HashMaps/Hashtables had > a max 3 or fewer elements during their lifetimes. > So on average using your version will likely increase > application footprints. It seems possible to deal with > this though. For example, you might consider separately > allocating the back-half of index array only when needed. Actually, scala version is optimized for memory footprint: 1) its initial capacity is 4 elements, not 16 2) default load factor is 1, not .75 3) on tiny maps (<= 128 elements) is uses byte index array, and on medium sized (<32K) arrays of shorts. I have an idea how not to use 'deleted' bit and put 256 element into byte-index-map. But thus there is no place to store all of hashcode bits, and there are very slow resizes when switching from byte to short and from short to int arrays. Also, in MapWordLoops it seems that original HashMap is faster on 50-elements test. Maybe there is a sence to optimize 1-2-3-element cases by directly storing the values without hashing, don't know. While it remains approximately true in yours that loadFactors < 1 > are the average number of traversals to find existing key > near the resize threshold point, probably some more work/thought > is needed to make a better story about interpreting them internally. Well... It is also possible to reorganize index bit structure to allow for example up to 2.0 load factor but I am not sure if it is really needed. It strikes me that there might be a bigger win available > here by using some hybrid strategy based on > IdentityHashMap-like single-indirect (i.e., usually > at most a single cache miss) for collision-free accesses > but using a variant of your clever packed-indices > under collisions. I might try this out someday. Interesting, I'm looking forward to see it. > Iteration over elements is a sequental array read which is faster. >> To clone such HashMap we only need to clone 2 arrays with is also much >> faster. >> > > I don't understand why you don't simply traverse the element > array for HashMap iterators? You cannot do this for your linked > version, but it would not hurt to make completely different > iterator types. I don't get your question. In most cases I really do this like: for (int i = 0; i < firstEmptyIndex; i++) if (!isEmpty(i)) { .... or you mean iterateFirst/iterateNext methods? Their only purpose is to simplify LinkedMap and reuse code, but maybe having different iterators will give some speedup, but it's just one virtual call - you think it's really that bad? But hence it could not be inlined by HotSpot... You can look up git history - versions prior to LinkedMap had 'direct' iterators, you can benchmark it if you want. For non-concurrent collections, the goal of ConcurrentModificationExceptions > is to help people find/fix incorrect code; not necessarily to only > blow up if proceeding is impossible. I was thinking if it would be possible to make concurrent implementation based on AtomicIntegerArray with CAS/retries, but there are some problems I cannot solve. Not sure for concurrent resize - I have some ideas how it could be done, but management of deleted indices (holes) that can be reused seems impossible without either a full quasi-synchronization with read locks that can also be stored as bits in index... I hope this helps! Despite all the concerns above, this is the > first alternative version of HashMap that I can recall seeing > that seems promising enough to continue exploring as a > possible replacement. > Thanks! Alex -------------- next part -------------- An HTML attachment was scrubbed... URL: From scolebourne at joda.org Mon Jun 8 17:31:27 2009 From: scolebourne at joda.org (Stephen Colebourne) Date: Mon, 8 Jun 2009 18:31:27 +0100 Subject: Faster HashMap implementation In-Reply-To: <4A2D3D8E.60300@cs.oswego.edu> References: <4A2D3729.3040904@cs.oswego.edu> <82ocsyitzc.fsf@mid.bfk.de> <4A2D3D8E.60300@cs.oswego.edu> Message-ID: <4b4f45e00906081031g766830b2yff12f213a7b69570@mail.gmail.com> 2009/6/8 Doug Lea
: > It is possible to use a look-aside strategy for tiny > HashMaps. I think one of the Apache commons maps > does this. FYI - Commons Collections Flat3Map: http://commons.apache.org/collections/api-3.2/org/apache/commons/collections/map/Flat3Map.html Stephen From eliasen at mindspring.com Mon Jun 8 20:44:13 2009 From: eliasen at mindspring.com (Alan Eliasen) Date: Mon, 08 Jun 2009 14:44:13 -0600 Subject: [UPDATED PATCH] 4837946: Implement Karatsuba multiplication algorithm in BigInteger Message-ID: <4A2D781D.7070003@mindspring.com> Attached is an UPDATED patch for bug 4837946, (and others) for implementing asymptotically faster algorithms for multiplication of large numbers in the BigInteger class: http://bugs.sun.com/bugdatabase/view_bug.do?bug_id=4837946 This also affects other bugs: 4228681: Some BigInteger operations are slow with very large numbers http://bugs.sun.com/bugdatabase/view_bug.do?bug_id=4228681 (This was closed but never fixed.) 4837946: Implement Karatsuba multiplication algorithm in BigInteger http://bugs.sun.com/bugdatabase/view_bug.do?bug_id=4837946 This is done, along with Toom-Cook multiplication. My implementation is intended to be easy to read, understand, and check. It significantly improves multiplication performance for large numbers. This bug will be able to be closed with this patch. This patch now also implements 3-way Toom-Cook multiplication and 3-way Toom-Cook squaring in addition to the Karatsuba squaring. 3-way Toom-Cook multiplication has an asymptotic efficiency of O(n^1.465), compared to O(n^1.585) for Karatsuba and O(n^2) for the "grade-school" algorithm, making it significantly faster for very large numbers. This patch is almost identical to the ones posted earlier, but I've merged Xiaobin Lu's recent changes to BigInteger with my code. The other change was removing an unnecessary carry check from the exactDivideBy3 routine. I have again run tuning tests with Xiaobin's changes (no changes were made to the thresholds as a result; the previous combinations worked well.) This has been extensively tested. My regression tests are probably significant overkill for Sun's purposes. They take about 30 hours to run and produce about 232 gigabytes of output. (Multiply each of those numbers by 2 because you have to run it twice to compare the outputs of different VMs, and then compare output.) This patch contains only multiplication- and squaring-related patches. I will be submitting another patch of my changes to make the pow() function very much faster, but that will be a separate patch. Sun has requested patches as small as possible. I will also have patches for pow() (see http://bugs.sun.com/bugdatabase/view_bug.do?bug_id=4646474 ) and for toString ( http://bugs.sun.com/bugdatabase/view_bug.do?bug_id=4641897 ) once these are approved and I'm informed that I'm working in the right direction. This patch now also implements 3-way Toom-Cook multiplication and 3-way Toom-Cook squaring in addition to the Karatsuba squaring. 3-way Toom-Cook multiplication has an asymptotic efficiency of O(n^1.465), compared to O(n^1.585) for Karatsuba and O(n^2) for the "grade-school" algorithm, making it significantly faster for very large numbers. For those who'd rather just replace their BigInteger with my much faster version, that also contains my patches for pow(), it's available at: http://futureboy.us/temp/BigInteger.java These patches are designed to be as easy to read as possible, and are implemented in terms of already-existing methods in BigInteger. Some more performance might be squeezed out of them by doing more low-level bit-fiddling, but I wanted to get a working version in and tested. -- Alan Eliasen | "Furious activity is no substitute eliasen at mindspring.com | for understanding." http://futureboy.us/ | --H.H. Williams -------------- next part -------------- An embedded and charset-unspecified text was scrubbed... Name: BigInteger.patch URL: From martinrb at google.com Mon Jun 8 20:52:47 2009 From: martinrb at google.com (Martin Buchholz) Date: Mon, 8 Jun 2009 13:52:47 -0700 Subject: Review request for 5049299 In-Reply-To: <4A2D1B65.5020002@sun.com> References: <4A1678DB.9010209@sun.com> <4A1849CD.3060306@redhat.com> <1ccfd1c10905231810k17a99cd3vfa68d6108e28b46@mail.gmail.com> <4A191063.5060401@redhat.com> <1ccfd1c10905241809m10ded53bpb4c9360f76b36888@mail.gmail.com> <1ccfd1c10905301051j527d2e91m217020d037ac1732@mail.gmail.com> <1ccfd1c10906011727g794e9ca0m89c350fa8dae7c54@mail.gmail.com> <4A25373D.9090906@sun.com> <1ccfd1c10906051913o1850b256pca173d2568b25736@mail.gmail.com> <4A2D1B65.5020002@sun.com> Message-ID: <1ccfd1c10906081352i12feb1a5j54133adb4f3f5c25@mail.gmail.com> On Mon, Jun 8, 2009 at 07:08, Michael McMahon wrote: > That's fine Martin. We can do it that way. > > Do you really need to #include ? > As far as I can see clone() only requires > You're right. I removed #include of syscall.h. I think I picked it up from some man page somewhere. > > When you allocate the clone stack for the child > the memory is byte aligned. Is this ok for Linux or should stacks > be aligned on larger boundaries? > Good question. The man page is silent on the matter. I ensure that the stack pointer is as aligned as the return from malloc(), which is supposed to be suitable for any C object. I suspect that if there are more stringent requirements, then the kernel itself will adjust the alignment. Let's keep the current code unless we know of a reason not to. > > Also, I don't follow why we need the execve_as_traditional_shell_script() > function. Can you explain the reason for that? > I think my comment for that function explains it fairly well. /** * Exec FILE as a traditional Bourne shell script (i.e. one without #!). * If we could do it over again, we would probably not support such an ancient * misfeature, but compatibility wins over sanity. The original support for * this was imported accidentally from execvp(). */ The tests I added also pass on the older implementation, so execve_as_traditional_shell_script() prevents a regression. We always supported "traditional shell scripts" - we just didn't know it. --- I updated the public version of the patch at: http://cr.openjdk.java.net/~martin/clone-exec Martin > > Thanks, > Michael. > > Martin Buchholz wrote: > >> Michael, >> >> I think the best way to handle the coordination is in two steps. >> I'd like to get my Linux-clone changes in first (you should review, >> I will commit) >> and then we switch hats and I will review your Solaris changes. >> It seems best to do this in two steps: to better place blame when >> it breaks (this is very tricky stuff to get right). >> If you agree, please review my posted changes. >> >> Aside: Instead of griping about the missing execvpe, >> I filed a bug against glibc, and was surprised to find >> that Ulrich Drepper had implemented it a couple of days later. >> It will probably be in glibc-2.11. Perhaps in 5 years we can >> use it ourselves...). Thanks, Uli! >> >> Martin >> >> On Tue, Jun 2, 2009 at 07:29, Michael McMahon > Michael.McMahon at sun.com>> wrote: >> >> Martin, >> >> I had done something similar with clone & exec for Linux, but >> hadn't got round to testing it. >> So, it seems reasonable to take yours. Do you want to send me your >> updated versions of >> process_md.c and the test? I can take care of the merge with the >> Solaris code. >> >> >> > -------------- next part -------------- An HTML attachment was scrubbed... URL: From weijun.wang at sun.com Tue Jun 9 06:22:06 2009 From: weijun.wang at sun.com (weijun.wang at sun.com) Date: Tue, 09 Jun 2009 06:22:06 +0000 Subject: hg: jdk7/tl/jdk: 6578647: Undefined requesting URL in java.net.Authenticator.getPasswordAuthentication() Message-ID: <20090609062242.79975E192@hg.openjdk.java.net> Changeset: 8f405b65ddac Author: weijun Date: 2009-06-09 14:17 +0800 URL: http://hg.openjdk.java.net/jdk7/tl/jdk/rev/8f405b65ddac 6578647: Undefined requesting URL in java.net.Authenticator.getPasswordAuthentication() Reviewed-by: chegar, valeriep ! src/share/classes/sun/net/www/protocol/http/AuthenticationHeader.java + src/share/classes/sun/net/www/protocol/http/HttpCallerInfo.java ! src/share/classes/sun/net/www/protocol/http/HttpURLConnection.java ! src/share/classes/sun/net/www/protocol/http/NegotiateAuthentication.java ! src/share/classes/sun/net/www/protocol/http/NegotiateCallbackHandler.java ! src/share/classes/sun/net/www/protocol/http/NegotiatorImpl.java + src/share/classes/sun/security/jgss/GSSCaller.java ! src/share/classes/sun/security/jgss/GSSManagerImpl.java ! src/share/classes/sun/security/jgss/GSSUtil.java + src/share/classes/sun/security/jgss/HttpCaller.java ! src/share/classes/sun/security/jgss/LoginConfigImpl.java ! src/share/classes/sun/security/jgss/ProviderList.java ! src/share/classes/sun/security/jgss/krb5/InitialToken.java ! src/share/classes/sun/security/jgss/krb5/Krb5AcceptCredential.java ! src/share/classes/sun/security/jgss/krb5/Krb5Context.java ! src/share/classes/sun/security/jgss/krb5/Krb5InitCredential.java ! src/share/classes/sun/security/jgss/krb5/Krb5MechFactory.java ! src/share/classes/sun/security/jgss/krb5/Krb5Util.java ! src/share/classes/sun/security/jgss/spnego/SpNegoMechFactory.java ! src/share/classes/sun/security/jgss/wrapper/NativeGSSFactory.java ! src/share/classes/sun/security/ssl/ClientHandshaker.java ! src/share/classes/sun/security/ssl/KerberosClientKeyExchange.java ! src/share/classes/sun/security/ssl/ServerHandshaker.java ! test/sun/security/jgss/DefaultGssConfig.java ! test/sun/security/jgss/GssNPE.java + test/sun/security/krb5/auto/HttpNegotiateServer.java ! test/sun/security/krb5/auto/KDC.java + test/sun/security/krb5/auto/META-INF/services/sun.net.spi.nameservice.NameServiceDescriptor From eliasen at mindspring.com Tue Jun 9 09:10:47 2009 From: eliasen at mindspring.com (Alan Eliasen) Date: Tue, 09 Jun 2009 03:10:47 -0600 Subject: [UPDATED PATCH] 4837946: Implement Karatsuba multiplication algorithm in BigInteger In-Reply-To: <4A2D781D.7070003@mindspring.com> References: <4A2D781D.7070003@mindspring.com> Message-ID: <4A2E2717.6090903@mindspring.com> Attached is an UPDATED patch for bug 4837946, (and others) for implementing asymptotically faster algorithms for multiplication of large numbers in the BigInteger class (which also improves the performance of large numbers BigDecimal, etc.) This patch slightly modifies the patch I sent before, primarily by removing an unused method that I let slip through when editing the patch file. It also adds comments and links to further information about the algorithms, and in one place renames a variable in one of my functions for pedagogical correctness. It also slightly changes a signum test to follow the new convention set elsewhere by Xiaobin Lu, which will be slightly more efficient. This patch supersedes any others. As always, if you just want to drop in my whole BigInteger class with other proposed fixes for pow(), it's available at: http://futureboy.us/temp/BigInteger.java I'd also like to put out a call for volunteers to help Sun review this patch and get it into JDK 1.7. The Karatsuba and Toom-Cook multiplication algorithms are well-understood and straightforward, and have been written in as simple and direct way as possible, building on existing methods in Java. If you are able to review this patch, please contact me. Thanks! From all the e-mails I get, I know this is very important to a lot of people. Alan Eliasen wrote: > Attached is an UPDATED patch for bug 4837946, (and others) for > implementing asymptotically faster algorithms for multiplication of > large numbers in the BigInteger class: > > http://bugs.sun.com/bugdatabase/view_bug.do?bug_id=4837946 > > This also affects other bugs: > > 4228681: Some BigInteger operations are slow with very large numbers > http://bugs.sun.com/bugdatabase/view_bug.do?bug_id=4228681 > > (This was closed but never fixed.) > > > 4837946: Implement Karatsuba multiplication algorithm in BigInteger > http://bugs.sun.com/bugdatabase/view_bug.do?bug_id=4837946 > > This is done, along with Toom-Cook multiplication. My > implementation is intended to be easy to read, understand, and check. > It significantly improves multiplication performance for large numbers. > This bug will be able to be closed with this patch. > > This patch now also implements 3-way Toom-Cook multiplication and > 3-way Toom-Cook squaring in addition to the Karatsuba squaring. 3-way > Toom-Cook multiplication has an asymptotic efficiency of O(n^1.465), > compared to O(n^1.585) for Karatsuba and O(n^2) for the "grade-school" > algorithm, making it significantly faster for very large numbers. > > > This patch is almost identical to the ones posted earlier, but I've > merged Xiaobin Lu's recent changes to BigInteger with my code. The > other change was removing an unnecessary carry check from the > exactDivideBy3 routine. > > I have again run tuning tests with Xiaobin's changes (no changes were > made to the thresholds as a result; the previous combinations worked > well.) > > This has been extensively tested. My regression tests are probably > significant overkill for Sun's purposes. They take about 30 hours to > run and produce about 232 gigabytes of output. (Multiply each of those > numbers by 2 because you have to run it twice to compare the outputs of > different VMs, and then compare output.) > > This patch contains only multiplication- and squaring-related > patches. I will be submitting another patch of my changes to make the > pow() function very much faster, but that will be a separate patch. Sun > has requested patches as small as possible. > > I will also have patches for pow() (see > http://bugs.sun.com/bugdatabase/view_bug.do?bug_id=4646474 ) and for > toString ( http://bugs.sun.com/bugdatabase/view_bug.do?bug_id=4641897 ) > once these are approved and I'm informed that I'm working in the right > direction. > > This patch now also implements 3-way Toom-Cook multiplication and > 3-way Toom-Cook squaring in addition to the Karatsuba squaring. 3-way > Toom-Cook multiplication has an asymptotic efficiency of O(n^1.465), > compared to O(n^1.585) for Karatsuba and O(n^2) for the "grade-school" > algorithm, making it significantly faster for very large numbers. > > > For those who'd rather just replace their BigInteger with my much > faster version, that also contains my patches for pow(), it's available at: > > http://futureboy.us/temp/BigInteger.java > > These patches are designed to be as easy to read as possible, and are > implemented in terms of already-existing methods in BigInteger. Some > more performance might be squeezed out of them by doing more low-level > bit-fiddling, but I wanted to get a working version in and tested. > > -- Alan Eliasen eliasen at mindspring.com http://futureboy.us/ -------------- next part -------------- An embedded and charset-unspecified text was scrubbed... Name: BigInteger.patch URL: From dl at cs.oswego.edu Tue Jun 9 12:09:45 2009 From: dl at cs.oswego.edu (Doug Lea) Date: Tue, 09 Jun 2009 08:09:45 -0400 Subject: Faster HashMap implementation In-Reply-To: References: <4A2D3729.3040904@cs.oswego.edu> Message-ID: <4A2E5109.4050804@cs.oswego.edu> Alex Yakovlev wrote: > entrySet() iterator which is very expensive. > > Very big speedup would be to reuse Entry object, We had originally done this in a few classes. The Iterator spec even allows it. However, most usages ignore that part of the spec, and it led to a bunch of bug reports, so now all Entry iterators for classes using non-Entry-based representations create new ones. When running on the -server compiler this is not a huge loss, but it hurts noticeably using -client. BTW, when checking performance, do always run both, and also run across multiple machines. Especially for hash tables, there are usually interactions with processor-level properties. > > 3) as for DenseMapMicroBenchmark test - I think that > CompactHashMap.scala will perform well in such tests > since it stores primitive types in primitive arrays, Right. The upcoming Scala specialization support is great for collections especially. Too bad there doesn't seem to be a plausible path to this in Java. (Paul Tyma once created a Java "specifics compiler" to do it manually, but I don't see this around any more -- http://www.classhat.com seems to have gone away.) > > Actually, scala version is optimized for memory footprint: > ... > and there are very slow resizes when switching > from byte to short and from short to int arrays. More generally, the cost of switching representations and the overhead for choosing and switching are the main reasons there are so few JDK classes that use multiple internal data structures. But still there are probably a few cases where it is worthwhile. > > Well... It is also possible to reorganize index bit structure > to allow for example up to 2.0 load factor but I am not sure > if it is really needed. The main issue is that people (should) use large load factors only when they want to save pre-allocated array (table) overhead, which yours doesn't do. Using non-default load factors is uncommon though. Scanning http://www.google.com/codesearch for lang:java new\sHashMap \(\S+,\s \d+.\d+f\) and further variants (which aren't completely accurate but close enough) finds only a few > 1, and none I saw > 2.0. Still, there probably needs to be a better effort to approximately match current space consumption in these cases. > > It strikes me that there might be a bigger win available > here by using some hybrid strategy based on > IdentityHashMap-like single-indirect (i.e., usually > at most a single cache miss) for collision-free accesses > but using a variant of your clever packed-indices > under collisions. I might try this out someday. > > > Interesting, I'm looking forward to see it. On a little bit of preliminary exploration, it might be better yet to use something like ternary tries for collision/overflow. It might be a while before I get to this though. > or you mean iterateFirst/iterateNext methods? > Their only purpose is to simplify LinkedMap and reuse code, > but maybe having different iterators will give some speedup, > but it's just one virtual call - you think it's really that bad? > But hence it could not be inlined by HotSpot... Right. Overridden virtual calls inside iterators usually hurt enough to avoid, but it is worth empirically checking. > > I was thinking if it would be possible to make concurrent > implementation based on AtomicIntegerArray with CAS/retries, > but there are some problems I cannot solve. Cliff Click's hash table (http://sourceforge.net/projects/high-scale-lib/) does something along these lines. The tradeoffs involved here only sometimes seem to win out over current ConcurentHashMap. -Doug From Michael.McMahon at Sun.COM Tue Jun 9 13:56:15 2009 From: Michael.McMahon at Sun.COM (Michael McMahon) Date: Tue, 09 Jun 2009 14:56:15 +0100 Subject: Review request for 5049299 In-Reply-To: <1ccfd1c10906081352i12feb1a5j54133adb4f3f5c25@mail.gmail.com> References: <4A1678DB.9010209@sun.com> <4A1849CD.3060306@redhat.com> <1ccfd1c10905231810k17a99cd3vfa68d6108e28b46@mail.gmail.com> <4A191063.5060401@redhat.com> <1ccfd1c10905241809m10ded53bpb4c9360f76b36888@mail.gmail.com> <1ccfd1c10905301051j527d2e91m217020d037ac1732@mail.gmail.com> <1ccfd1c10906011727g794e9ca0m89c350fa8dae7c54@mail.gmail.com> <4A25373D.9090906@sun.com> <1ccfd1c10906051913o1850b256pca173d2568b25736@mail.gmail.com> <4A2D1B65.5020002@sun.com> <1ccfd1c10906081352i12feb1a5j54133adb4f3f5c25@mail.gmail.com> Message-ID: <4A2E69FF.6080908@sun.com> Martin Buchholz wrote: > > > > Also, I don't follow why we need the > execve_as_traditional_shell_script() > function. Can you explain the reason for that? > > > I think my comment for that function explains it fairly well. > > /** > * Exec FILE as a traditional Bourne shell script (i.e. one without #!). > * If we could do it over again, we would probably not support such an > ancient > * misfeature, but compatibility wins over sanity. The original > support for > * this was imported accidentally from execvp(). > */ > Actually, I was really wondering why is this code needed now? What has it to do with the specifics of converting fork()+exec() to clone()+exec() Thanks, Michael. > The tests I added also pass on the older implementation, > so execve_as_traditional_shell_script() prevents a regression. > We always supported "traditional shell scripts" - we just didn't know it. > > --- > > I updated the public version of the patch at: > http://cr.openjdk.java.net/~martin/clone-exec > > > Martin > > > > > Thanks, > Michael. > > Martin Buchholz wrote: > > Michael, > > I think the best way to handle the coordination is in two steps. > I'd like to get my Linux-clone changes in first (you should > review, > I will commit) > and then we switch hats and I will review your Solaris changes. > It seems best to do this in two steps: to better place blame when > it breaks (this is very tricky stuff to get right). > If you agree, please review my posted changes. > > Aside: Instead of griping about the missing execvpe, > I filed a bug against glibc, and was surprised to find > that Ulrich Drepper had implemented it a couple of days later. > It will probably be in glibc-2.11. Perhaps in 5 years we can > use it ourselves...). Thanks, Uli! > > Martin > > On Tue, Jun 2, 2009 at 07:29, Michael McMahon > > >> wrote: > > Martin, > > I had done something similar with clone & exec for Linux, but > hadn't got round to testing it. > So, it seems reasonable to take yours. Do you want to send > me your > updated versions of > process_md.c and the test? I can take care of the merge > with the > Solaris code. > > > > From alex14n at gmail.com Tue Jun 9 14:43:26 2009 From: alex14n at gmail.com (Alex Yakovlev) Date: Tue, 9 Jun 2009 18:43:26 +0400 Subject: Faster HashMap implementation In-Reply-To: <4A2E5109.4050804@cs.oswego.edu> References: <4A2D3729.3040904@cs.oswego.edu> <4A2E5109.4050804@cs.oswego.edu> Message-ID: Doug, On Tue, Jun 9, 2009 at 4:09 PM, Doug Lea
wrote: > Alex Yakovlev wrote: > >> entrySet() iterator which is very expensive. >> >> Very big speedup would be to reuse Entry object, >> > > We had originally done this in a few classes. > The Iterator spec even allows it. However, most > usages ignore that part of the spec, and it led > to a bunch of bug reports, so now all Entry iterators > for classes using non-Entry-based representations > create new ones. When running on the -server compiler > this is not a huge loss, but it hurts noticeably > using -client. BTW, when checking performance, > do always run both, and also run across multiple > machines. Especially for hash tables, there are > usually interactions with processor-level properties. FYI: I've run MapCheck with -XX:+DoEscapeAnalysis "Iter Entry" got very significant boost, almost 10x but "Iter EntrySet contains" did not (in source code entry object is passed into contains method thus cannot be stack-allocated). I'll note testing ClientVM too, thanx mentioning it. Still, there probably needs to be a better effort to > approximately match current space consumption in these cases. I currently have no idea how to do that, anyway this is not a major issue. BTW, on your last message on memory consumption, object header is not one word, I don't remember exaclty but in memory analyzer I saw ~12 bytes header on 32-bit JVM and ~20 bytes on 64-bit. Google search says about 8 bytes on 32bit and 16 on 64: http://kohlerm.blogspot.com/2008/12/how-much-memory-is-used-by-my-java.html maybe my data was greater because of alignment. Currently I've changed default capacity from 16 to 4, with 0.75 load factor it's exactly 3 elements you wrote about and total memory is about ~60 bytes, approx. same as current HashMap. It strikes me that there might be a bigger win available >> here by using some hybrid strategy based on >> IdentityHashMap-like single-indirect (i.e., usually >> at most a single cache miss) for collision-free accesses >> but using a variant of your clever packed-indices >> under collisions. I might try this out someday. >> >> Interesting, I'm looking forward to see it. >> > > On a little bit of preliminary exploration, it might > be better yet to use something like ternary tries for > collision/overflow. It might be a while before I get to > this though. Well. I was curious enough myself to write some proof-of-concept code to test this approach: http://github.com/alex14n/CompactHashMap/blob/8f935e1b5fc7c673e04c93eca2402aada5137b39/java/HybridHashMap.java There are a lot of to do: Map interface is not implemented, no iterators, no key removal, resize is very slow, management of overflow data can be done several ways, etc. But current version shows ~10% performance improvement in reading existing mappings over both HashMap and my map. Misses are slower but is approximately the same as HashMap. > or you mean iterateFirst/iterateNext methods? >> Their only purpose is to simplify LinkedMap and reuse code, >> but maybe having different iterators will give some speedup, >> but it's just one virtual call - you think it's really that bad? >> But hence it could not be inlined by HotSpot... >> > > Right. Overridden virtual calls inside iterators usually hurt > enough to avoid, but it is worth empirically checking. I did a bit of testing and there were no significant speedup, but I was testing ServerVM. But I might do it someday anyway. > I was thinking if it would be possible to make concurrent >> implementation based on AtomicIntegerArray with CAS/retries, >> but there are some problems I cannot solve. >> > > Cliff Click's hash table (http://sourceforge.net/projects/high-scale-lib/) > does something along these lines. The tradeoffs involved here > only sometimes seem to win out over current ConcurentHashMap. > Honestly I don't think that array-based approach can be bettter than current concurrent implementation since there'll be more retries because of more competition over the same array index compared to current possibility just to call new to create new object, so I see no reason even to try. Maybe I'll change my mind over time. Alex -------------- next part -------------- An HTML attachment was scrubbed... URL: From martinrb at google.com Tue Jun 9 15:51:04 2009 From: martinrb at google.com (Martin Buchholz) Date: Tue, 9 Jun 2009 08:51:04 -0700 Subject: Review request for 5049299 In-Reply-To: <4A2E69FF.6080908@sun.com> References: <4A1678DB.9010209@sun.com> <4A191063.5060401@redhat.com> <1ccfd1c10905241809m10ded53bpb4c9360f76b36888@mail.gmail.com> <1ccfd1c10905301051j527d2e91m217020d037ac1732@mail.gmail.com> <1ccfd1c10906011727g794e9ca0m89c350fa8dae7c54@mail.gmail.com> <4A25373D.9090906@sun.com> <1ccfd1c10906051913o1850b256pca173d2568b25736@mail.gmail.com> <4A2D1B65.5020002@sun.com> <1ccfd1c10906081352i12feb1a5j54133adb4f3f5c25@mail.gmail.com> <4A2E69FF.6080908@sun.com> Message-ID: <1ccfd1c10906090851g6498733dw78743283cdfe405f@mail.gmail.com> On Tue, Jun 9, 2009 at 06:56, Michael McMahon wrote: > Martin Buchholz wrote: > >> >> >> Also, I don't follow why we need the >> execve_as_traditional_shell_script() >> function. Can you explain the reason for that? >> >> >> I think my comment for that function explains it fairly well. >> >> /** >> * Exec FILE as a traditional Bourne shell script (i.e. one without #!). >> * If we could do it over again, we would probably not support such an >> ancient >> * misfeature, but compatibility wins over sanity. The original support >> for >> * this was imported accidentally from execvp(). >> */ >> >> Actually, I was really wondering why is this code needed now? > What has it to do with the specifics of converting fork()+exec() > to clone()+exec() > In the old implementation, we used the strategy of fork+mutate environ+execvp and we got the traditional shell script semantics from our use of execvp. Since environ is now a global shared-with-parent variable, we can't mutate it any more, therefore can't use execvp, and must implement the missing execvpe ourselves. Now, strictly speaking, execvp's traditional shell script semantics is an unspecified implementation detail of execvp; on other Unix platforms we might not need this. We can leave this to, e.g. the BSD porters reading this, to add some #ifdefs. (It would also be good if you ran the updated tests on Solaris with the old implementation to confirm my understanding) The other non-portable implementation detail we need to deal with is the default value of PATH if undefined in the environment, but we had already been doing that. Martin > > Thanks, > Michael. > >> The tests I added also pass on the older implementation, >> so execve_as_traditional_shell_script() prevents a regression. >> We always supported "traditional shell scripts" - we just didn't know it. >> >> --- >> >> I updated the public version of the patch at: >> http://cr.openjdk.java.net/~martin/clone-exec< >> http://cr.openjdk.java.net/%7Emartin/clone-exec> >> >> >> Martin >> >> >> >> Thanks, >> Michael. >> >> Martin Buchholz wrote: >> >> Michael, >> >> I think the best way to handle the coordination is in two steps. >> I'd like to get my Linux-clone changes in first (you should >> review, >> I will commit) >> and then we switch hats and I will review your Solaris changes. >> It seems best to do this in two steps: to better place blame when >> it breaks (this is very tricky stuff to get right). >> If you agree, please review my posted changes. >> >> Aside: Instead of griping about the missing execvpe, >> I filed a bug against glibc, and was surprised to find >> that Ulrich Drepper had implemented it a couple of days later. >> It will probably be in glibc-2.11. Perhaps in 5 years we can >> use it ourselves...). Thanks, Uli! >> >> Martin >> >> On Tue, Jun 2, 2009 at 07:29, Michael McMahon >> >> > >> wrote: >> >> Martin, >> >> I had done something similar with clone & exec for Linux, but >> hadn't got round to testing it. >> So, it seems reasonable to take yours. Do you want to send >> me your >> updated versions of >> process_md.c and the test? I can take care of the merge >> with the >> Solaris code. >> >> >> >> >> > -------------- next part -------------- An HTML attachment was scrubbed... URL: From Michael.McMahon at Sun.COM Tue Jun 9 16:28:17 2009 From: Michael.McMahon at Sun.COM (Michael McMahon) Date: Tue, 09 Jun 2009 17:28:17 +0100 Subject: Review request for 5049299 In-Reply-To: <1ccfd1c10906090851g6498733dw78743283cdfe405f@mail.gmail.com> References: <4A1678DB.9010209@sun.com> <4A191063.5060401@redhat.com> <1ccfd1c10905241809m10ded53bpb4c9360f76b36888@mail.gmail.com> <1ccfd1c10905301051j527d2e91m217020d037ac1732@mail.gmail.com> <1ccfd1c10906011727g794e9ca0m89c350fa8dae7c54@mail.gmail.com> <4A25373D.9090906@sun.com> <1ccfd1c10906051913o1850b256pca173d2568b25736@mail.gmail.com> <4A2D1B65.5020002@sun.com> <1ccfd1c10906081352i12feb1a5j54133adb4f3f5c25@mail.gmail.com> <4A2E69FF.6080908@sun.com> <1ccfd1c10906090851g6498733dw78743283cdfe405f@mail.gmail.com> Message-ID: <4A2E8DA1.3050004@sun.com> Martin Buchholz wrote: > In the old implementation, we used the strategy of > fork+mutate environ+execvp > and we got the traditional shell script semantics > from our use of execvp. > Since environ is now a global shared-with-parent variable, > we can't mutate it any more, therefore can't use execvp, > and must implement the missing execvpe ourselves. > > Now, strictly speaking, execvp's traditional shell script semantics > is an unspecified implementation detail of execvp; > on other Unix platforms we might not need this. > We can leave this to, e.g. the BSD porters reading this, > to add some #ifdefs. > Ok, got you. > (It would also be good if you ran the updated tests on > Solaris with the old implementation to confirm my understanding) I'm just building it now, and will run the test then. Thanks, Michael. From martinrb at google.com Tue Jun 9 18:42:04 2009 From: martinrb at google.com (Martin Buchholz) Date: Tue, 9 Jun 2009 11:42:04 -0700 Subject: Review request for 5049299 In-Reply-To: <4A2E8DA1.3050004@sun.com> References: <4A1678DB.9010209@sun.com> <1ccfd1c10905301051j527d2e91m217020d037ac1732@mail.gmail.com> <1ccfd1c10906011727g794e9ca0m89c350fa8dae7c54@mail.gmail.com> <4A25373D.9090906@sun.com> <1ccfd1c10906051913o1850b256pca173d2568b25736@mail.gmail.com> <4A2D1B65.5020002@sun.com> <1ccfd1c10906081352i12feb1a5j54133adb4f3f5c25@mail.gmail.com> <4A2E69FF.6080908@sun.com> <1ccfd1c10906090851g6498733dw78743283cdfe405f@mail.gmail.com> <4A2E8DA1.3050004@sun.com> Message-ID: <1ccfd1c10906091142h465bf9adoaa1b5c87bcf6ef9f@mail.gmail.com> On Tue, Jun 9, 2009 at 09:28, Michael McMahon wrote: > Martin Buchholz wrote: > >> In the old implementation, we used the strategy of >> fork+mutate environ+execvp >> and we got the traditional shell script semantics >> from our use of execvp. >> Since environ is now a global shared-with-parent variable, >> we can't mutate it any more, therefore can't use execvp, >> and must implement the missing execvpe ourselves. >> > One more note: we could try to have two implementations of execvpe, one that mutated environ and one that didn't, since each has its own advantages, but I think that would be even harder to understand and maintain. Hmmmmmmmmmmmmmmmmmm............................. Looking at the code again, it seems like it would not be too much work. Here's an untested example of how we could do both. static void execve_with_shell_fallback(const char *file, const char *argv[], const char *const envp[]) { #if USE_CLONE execve(file, (char **) argv, (char **) envp); if (errno == ENOEXEC) execve_as_traditional_shell_script(file, argv, envp); #else extern char **environ; environ = (char **) envp; execvp(file, (char **) argv); #endif } What do you think? Martin > >> Now, strictly speaking, execvp's traditional shell script semantics >> is an unspecified implementation detail of execvp; >> on other Unix platforms we might not need this. We can leave this to, e.g. >> the BSD porters reading this, >> to add some #ifdefs. >> >> Ok, got you. > >> (It would also be good if you ran the updated tests on >> Solaris with the old implementation to confirm my understanding) >> > > I'm just building it now, and will run the test then. > > Thanks, > Michael. > -------------- next part -------------- An HTML attachment was scrubbed... URL: From martinrb at google.com Tue Jun 9 19:24:43 2009 From: martinrb at google.com (Martin Buchholz) Date: Tue, 9 Jun 2009 12:24:43 -0700 Subject: Review request for 5049299 In-Reply-To: <1ccfd1c10906091142h465bf9adoaa1b5c87bcf6ef9f@mail.gmail.com> References: <4A1678DB.9010209@sun.com> <1ccfd1c10906011727g794e9ca0m89c350fa8dae7c54@mail.gmail.com> <4A25373D.9090906@sun.com> <1ccfd1c10906051913o1850b256pca173d2568b25736@mail.gmail.com> <4A2D1B65.5020002@sun.com> <1ccfd1c10906081352i12feb1a5j54133adb4f3f5c25@mail.gmail.com> <4A2E69FF.6080908@sun.com> <1ccfd1c10906090851g6498733dw78743283cdfe405f@mail.gmail.com> <4A2E8DA1.3050004@sun.com> <1ccfd1c10906091142h465bf9adoaa1b5c87bcf6ef9f@mail.gmail.com> Message-ID: <1ccfd1c10906091224p6cc0ccefpfa49453466e99640@mail.gmail.com> Following up some more... I did some cleanup of the USE_CLONE vs. non-USE_CLONE code and tested both of them (on Linux at least). I took the liberty of removing the references to fork1, since OpenJDK7 only supports Solaris10 going forward. The main fork logic now looks like this: { #if USE_CLONE /* See clone(2). * Instead of worrying about which direction the stack grows, just * allocate twice as much and start the stack in the middle. */ const int stack_size = 64 * 1024; if ((clone_stack = NEW(char, 2 * stack_size)) == NULL) goto Catch; resultPid = clone(childProcess, clone_stack + stack_size, /* CLONE_VFORK | // works, but unnecessary */ CLONE_VM | SIGCHLD, c); #else /* From fork(2): In Solaris 10, a call to fork() is identical * to a call to fork1(); only the calling thread is replicated * in the child process. This is the POSIX-specified behavior * for fork(). */ resultPid = fork(); if (resultPid == 0) { childProcess(c); assert(0); /* childProcess must not return */ } #endif } Patch updated on cr.openjdk.java.net. Martin On Tue, Jun 9, 2009 at 11:42, Martin Buchholz wrote: > > > On Tue, Jun 9, 2009 at 09:28, Michael McMahon wrote: > >> Martin Buchholz wrote: >> >>> In the old implementation, we used the strategy of >>> fork+mutate environ+execvp >>> and we got the traditional shell script semantics >>> from our use of execvp. >>> Since environ is now a global shared-with-parent variable, >>> we can't mutate it any more, therefore can't use execvp, >>> and must implement the missing execvpe ourselves. >>> >> > One more note: we could try to have two implementations of > execvpe, one that mutated environ and one that didn't, > since each has its own advantages, > but I think that would be even harder to understand and maintain. > > Hmmmmmmmmmmmmmmmmmm............................. > Looking at the code again, it seems like it would not be too much work. > > Here's an untested example of how we could do both. > > static void > execve_with_shell_fallback(const char *file, > const char *argv[], > const char *const envp[]) > { > #if USE_CLONE > execve(file, (char **) argv, (char **) envp); > if (errno == ENOEXEC) > execve_as_traditional_shell_script(file, argv, envp); > #else > extern char **environ; > environ = (char **) envp; > execvp(file, (char **) argv); > #endif > } > > What do you think? > > Martin > > >> >>> Now, strictly speaking, execvp's traditional shell script semantics >>> is an unspecified implementation detail of execvp; >>> on other Unix platforms we might not need this. We can leave this to, >>> e.g. the BSD porters reading this, >>> to add some #ifdefs. >>> >>> Ok, got you. >> >>> (It would also be good if you ran the updated tests on >>> Solaris with the old implementation to confirm my understanding) >>> >> >> I'm just building it now, and will run the test then. >> >> Thanks, >> Michael. >> > > -------------- next part -------------- An HTML attachment was scrubbed... URL: From martinrb at google.com Tue Jun 9 19:44:02 2009 From: martinrb at google.com (Martin Buchholz) Date: Tue, 9 Jun 2009 12:44:02 -0700 Subject: Review request for 5049299 In-Reply-To: <1ccfd1c10906091224p6cc0ccefpfa49453466e99640@mail.gmail.com> References: <4A1678DB.9010209@sun.com> <4A25373D.9090906@sun.com> <1ccfd1c10906051913o1850b256pca173d2568b25736@mail.gmail.com> <4A2D1B65.5020002@sun.com> <1ccfd1c10906081352i12feb1a5j54133adb4f3f5c25@mail.gmail.com> <4A2E69FF.6080908@sun.com> <1ccfd1c10906090851g6498733dw78743283cdfe405f@mail.gmail.com> <4A2E8DA1.3050004@sun.com> <1ccfd1c10906091142h465bf9adoaa1b5c87bcf6ef9f@mail.gmail.com> <1ccfd1c10906091224p6cc0ccefpfa49453466e99640@mail.gmail.com> Message-ID: <1ccfd1c10906091244u14607dcesc40fc6017adc7808@mail.gmail.com> I broke down and finally created a "proper" webrev, just like the good old days. http://cr.openjdk.java.net/~martin/clone-exec/ Martin On Tue, Jun 9, 2009 at 12:24, Martin Buchholz wrote: > > Patch updated on cr.openjdk.java.net. > > Martin -------------- next part -------------- An HTML attachment was scrubbed... URL: From alex14n at gmail.com Tue Jun 9 21:02:23 2009 From: alex14n at gmail.com (Alex Yakovlev) Date: Wed, 10 Jun 2009 01:02:23 +0400 Subject: HashMap - Hybrid approach Message-ID: Doug, To have single-indirect access we need to read key/value objects with the first array read. This is because we cannot store int and object in one array - there's no structures in java. But this approach have a negative side: we don't have stored hashcode to compare, hence we need to directly call equals method (or have reference equality on keys). And in the same array we can store other pointers - it will be prefetched into CPU cache. Thus we can store 3 pointers in each hashtable entry: our key, value, and HashEntry structure with collisions. Thus in many situations we'll have "at most a single cache miss". I've tested this approach on Objects - and it is 20-30% faster in all tests compared to HashMap, but my map still has faster puts. But when equals is expensive this approach is slower in some tests. And if we read stored hashcode before comparing real keys - it's 2 cache misses on existing mapping read, same as in HashMap (pointer to Entry and Entry contents) and my map (int index, object key). So this approach is questionable, but can give significant performance boots in some situations (fast equals and reference equality of looked up and stored keys). And we cannot store collisons in pre-allocated arrays - to read it from the same array with keys/values it needs to be an object. Thus writing speed on my original map will always be faster. On Mon, Jun 8, 2009 at 8:07 PM, Doug Lea
wrote: > It strikes me that there might be a bigger win available > here by using some hybrid strategy based on > IdentityHashMap-like single-indirect (i.e., usually > at most a single cache miss) for collision-free accesses > but using a variant of your clever packed-indices > under collisions. I might try this out someday. From pcj at roundroom.net Wed Jun 10 03:51:51 2009 From: pcj at roundroom.net (Peter Jones) Date: Tue, 9 Jun 2009 23:51:51 -0400 Subject: RMI stuck on network disconnection In-Reply-To: <4A2D1FB7.9020206@sun.com> References: <79be5fa30906070701o40e0cc27k48730dc927a79aaa@mail.gmail.com> <4A2D1FB7.9020206@sun.com> Message-ID: On Jun 8, 2009, at 10:27 AM, Alan Bateman wrote: > Guy Korland wrote: >> Hi, >> >> I found out that an RMI call might never return on network >> disconnection and will not throw RemoteException as expected. >> The scenario is very simple: >> >> 1. Find a remote stub (the stub should be behind a router). >> 2. Physically disconnect the server from the LAN. >> 3. Invoke a method from the client, the client can stuck without an >> exception up to 30 min. >> >> Any ideas? >> >> -- >> Thanks, >> Guy > I would suggest searching the archives of the rmi-users list [1]. > Also check the properties that control timeouts [2]. > > -Alan. > > [1] http://archives.java.sun.com/archives/rmi-users.html > [2] http://java.sun.com/javase/6/docs/technotes/guides/rmi/sunrmiproperties.html Hmm, it had been my understanding that the LISTSERV at java.sun.com would cease operation at the beginning of this month, but at least the archives still seem available for now (which is great). Here is a post from some years ago about the behavior of an RMI/JRMP client in the face of network disconnection at various stages of a connection's life cycle, and some ways of controlling it: http://archives.java.sun.com/cgi-bin/wa?A2=ind0212&L=rmi- users&P=731 Are you saying that the invocation eventually fails with an exception, after 30 minutes? In the situation you describe, with the connection attempt not starting until after the network disconnection has occurred, I would expect a RemoteException when the TCP retransmission for connection establishment expires-- the default timeout for this varies by OS but is typically from one to several minutes. -- Peter From dl at cs.oswego.edu Wed Jun 10 12:59:32 2009 From: dl at cs.oswego.edu (Doug Lea) Date: Wed, 10 Jun 2009 08:59:32 -0400 Subject: HashMap - Hybrid approach In-Reply-To: References: Message-ID: <4A2FAE34.3080902@cs.oswego.edu> Alex Yakovlev wrote: > To have single-indirect access we need to read > key/value objects with the first array read. Backing up a little, there are five main forces in hash table design. These are fresh in my mind because I've been trying to put together CustomConcurrentHashMap, the last planned JSR166y JDK7 component, that handles weak/soft refs, custom equality comparisons, eviction, etc. (Concurrency adds further forces omitted here.) 1. Memory stalls - Cache misses due to randomization of indexing can easily cost hundreds of cycles. This is made worse if there are multiple levels of indirection each with poor locality. (Multiple levels with good locality are much less of a problem, which is why ConcurrentHashMap is pretty fast despite adding one). And this is made even worse when there are too many computations before processor can even do a prefetch, which is one reason why bit-spreading functions require careful speed vs uniformity tradeoff analysis. 2. Method Dispatching - Both Object.equals and Object.hashCode are often "megamorphic", i.e., cannot be inlined because they are overridden in too many classes. In some usages, this accounts for the main performance bottleneck. So caching hashcodes, which usually also allows skipping calls to equals for different objects, has a noticeable effect. Also, it is sometimes possible to bias the structure of code to encourage compilers to do more per-call inlining, which helps a lot. Note that these effects can be difficult to test. Test programs that include multiple key types (like our DenseMapMicrobenchmark) are a little more accurate indicators of megamorphism effects, but even here, these kinds of microbenchmarks will often cause compilers to do deeper inlining than they would in most actual usages. 3. Collision handling - Collisions lead to some sort of slower scanning. Especially given the highly variable quality of user-supplied hashCodes, you'd like to localize the impact of collisions to avoid per-map slowdowns. Linear-probes and related techniques don't fare very well on these grounds. 4. Biasing for expected operation mixes - I once checked that collection (and in particular HashMap/Hashtable) usage approximately obeys the usual 4:1 read vs write ratio seen in just about every aspect of computing. In fact it seems a bit more extreme -- something like 84% read (get() or iterate) 15% add (put() and variants) and 1% delete (remove, clear). So making reads fast is clearly the highest priority. 5. Footprint - Many Java frameworks (especially EE) can create hundreds of thousands of collections, many of them very small, but also some huge ones. This leads to second-order bloat effects, like high TLB miss rates, and much slower GC. For some details see papers by Gary Sevitsky and colleagues at http://domino.research.ibm.com/comm/research_people.nsf/pages/sevitsky.pubs.html This is a difficult issue with hash tables since the whole idea is to preallocate space to allow random indexing. And because resizing is not cheap, you don't want to be overly incremental about it. > > And we cannot store collisons in pre-allocated > arrays - to read it from the same array with keys/values > it needs to be an object. Thus writing speed on > my original map will always be faster. > Right. My thought here is to use an alternative structure for collisions. So, given capacity C, you'd have: elements[] -- 2*C of key/value hashes[] -- 1*C of hash + bookkeeping bits collisions -- a dynamic bitwise trie or somesuch Compared to current HashMap... + It allows parallel prefetches on get; of both elements[2*h] and hashes[h], so should reduce memory stalls. - It triples the unloaded footprint. You can get most of this back in default-constructor usages by not allocating arrays, but instead using only collisions tree until size reaches say 4, and then using a larger (say 64) initial capacity once you know you aren't a tiny map. = Collisions may or may not be a little more expensive to resolve - Iterators are likely slower and definitely a lot messier - The trie nodes needed here would require 2 or 3 more pointer fields than do list-based nodes, so the incremental footprint grows faster. - Resizing is slower and more complicated since the collisions may or may not be moved to main table. Without fleshing this out, I'm not sure whether this hits the right tradeoffs. -Doug From sean.mullan at sun.com Wed Jun 10 16:29:39 2009 From: sean.mullan at sun.com (sean.mullan at sun.com) Date: Wed, 10 Jun 2009 16:29:39 +0000 Subject: hg: jdk7/tl/jdk: 6845161: Bottleneck in Configuration.getConfiguration synchronized call Message-ID: <20090610163002.1692EE2EB@hg.openjdk.java.net> Changeset: 4da7b972b391 Author: mullan Date: 2009-06-10 09:12 -0700 URL: http://hg.openjdk.java.net/jdk7/tl/jdk/rev/4da7b972b391 6845161: Bottleneck in Configuration.getConfiguration synchronized call Summary: Reduce scope of synchronized block Reviewed-by: weijun ! src/share/classes/javax/security/auth/login/Configuration.java From alex14n at gmail.com Thu Jun 11 08:49:51 2009 From: alex14n at gmail.com (Alex Yakovlev) Date: Thu, 11 Jun 2009 12:49:51 +0400 Subject: HashMap - Hybrid approach In-Reply-To: <4A2FAE34.3080902@cs.oswego.edu> References: <4A2FAE34.3080902@cs.oswego.edu> Message-ID: Doug, I thought about tree-like structures for collisions, but what came to my mind is that the problem is not in search depth that can be reduced by non-linear structures, but in memory fragmentation. But in my array-based approach it is possible to defragment it and (almost) always store all collisions in sequental array indices. Here is the code for such approach: http://github.com/alex14n/CompactHashMap/tree/defrag Now it do defragmentation only on put operations. Maybe there is a sence to do some reorder in resize too, when splitting big baskets. That leaves about ~1% fragmentation but data is still very close Maybe there is also some sence in some power of 2 index aligning to better fit/prefetch in CPU caches. But tests shows that this does not give much performance. In my tests I see ~5% increase on misses, and no significant changes on successfull reads. And writing speed decrease is significant. It can be partially solved be defragmenting only large baskets, leaving small (<= 2 or even 3 elements) as is. Now defragmentation happens in ~40% of puts. But it was done to rest read speed so it's better to have data not fragmented at all. I gathered some statisics, and in successful reads when map is almost full with .75 load factor just before resize, 70% of gets happens on first try, 23% on second, 5% on third lookup, 1% on fourth. Misses are resolved on first try (empty basket) in 47% cases, in 35% cases hashcode/key are compared, in 13% of cases a second lookup happens, third in 3% cases. This distribution is almost the same on all data I tested. So defragmentation helps only in ~7% of successful reads and ~18% of misses. Data structures that are fragmented will probably perform worse, here goes any objects with links to each other. Maybe there is a sence in some kind of binary search in the same array. When we look at first key and its hashcode we can set bit flags if there are keys with hashcode greater than that or less than that. Or even have 2 next indices one will branch to greater hashcodes, another to less or equals. Still with defragmentation for better localization in memory. Not sure about balansing such trees. And actually not sure if all of this is needed since sequental memory reading of collisions is not expensive at all. Alex On Wed, Jun 10, 2009 at 4:59 PM, Doug Lea
wrote: > 1. Memory stalls > ? - Cache misses due to randomization of indexing can easily > ? ? cost hundreds of cycles. This is made worse if there are > ? ? multiple levels of indirection each with poor locality. > ? ? (Multiple levels with good locality are much less of a > ? ? problem, which is why ConcurrentHashMap is pretty fast > ? ? despite adding one). And this is made even worse when > ? ? there are too many computations before processor can even > ? ? do a prefetch, which is one reason why bit-spreading functions > ? ? require careful speed vs uniformity tradeoff analysis. > 3. Collision handling > ? - Collisions lead to some sort of slower scanning. Especially > ? ? given the highly variable quality of user-supplied > ? ? hashCodes, you'd like to localize the impact of collisions > ? ? to avoid per-map slowdowns. Linear-probes and related techniques > ? ? don't fare very well on these grounds. > 4. Biasing for expected operation mixes > ? - I once checked that collection (and in particular HashMap/Hashtable) > ? ? usage approximately obeys the usual 4:1 read vs write ratio > ? ? seen in just about every aspect of computing. In fact it seems > ? ? a bit more extreme -- something like 84% read (get() or iterate) > ? ? 15% add (put() and variants) and 1% delete (remove, clear). So > ? ? making reads fast is clearly the highest priority. >> And we cannot store collisons in pre-allocated >> arrays - to read it from the same array with keys/values >> it needs to be an object. Thus writing speed on >> my original map will always be faster. > Right. My thought here is to use an alternative structure > for collisions. > ?+ It allows parallel prefetches on get; of both > ? ?elements[2*h] and hashes[h], so should reduce memory stalls. > ?= Collisions may or may not be a little more expensive to resolve > ?- Iterators are likely slower and definitely a lot messier > ?- The trie nodes needed here would require 2 or 3 more > ? ?pointer fields than do list-based nodes, so the incremental > ? ?footprint grows faster. From Michael.McMahon at Sun.COM Thu Jun 11 14:22:47 2009 From: Michael.McMahon at Sun.COM (Michael McMahon) Date: Thu, 11 Jun 2009 15:22:47 +0100 Subject: Review request for 5049299 In-Reply-To: <1ccfd1c10906091244u14607dcesc40fc6017adc7808@mail.gmail.com> References: <4A1678DB.9010209@sun.com> <4A25373D.9090906@sun.com> <1ccfd1c10906051913o1850b256pca173d2568b25736@mail.gmail.com> <4A2D1B65.5020002@sun.com> <1ccfd1c10906081352i12feb1a5j54133adb4f3f5c25@mail.gmail.com> <4A2E69FF.6080908@sun.com> <1ccfd1c10906090851g6498733dw78743283cdfe405f@mail.gmail.com> <4A2E8DA1.3050004@sun.com> <1ccfd1c10906091142h465bf9adoaa1b5c87bcf6ef9f@mail.gmail.com> <1ccfd1c10906091224p6cc0ccefpfa49453466e99640@mail.gmail.com> <1ccfd1c10906091244u14607dcesc40fc6017adc7808@mail.gmail.com> Message-ID: <4A311337.9070706@sun.com> Martin Buchholz wrote: > I broke down and finally created a "proper" webrev, > just like the good old days. > > http://cr.openjdk.java.net/~martin/clone-exec/ > > I've run the regression tests on Solaris and Linux and they seem fine. There is a compile warning on solaris at line 654: no return value from function. Aside from that, I'm happy with the change now - Michael. From Ulf.Zibis at gmx.de Thu Jun 11 12:49:32 2009 From: Ulf.Zibis at gmx.de (Ulf Zibis) Date: Thu, 11 Jun 2009 14:49:32 +0200 Subject: Fastpath for new String(bytes..) and String#getBytes(..) for ASCII + ISO-8859-1 Message-ID: <4A30FD5C.3040902@gmx.de> Hi Sherman, may be you are interested in this bug: http://bugs.sun.com/bugdatabase/view_bug.do?bug_id=6826329 -Ulf From dl at cs.oswego.edu Thu Jun 11 14:53:08 2009 From: dl at cs.oswego.edu (Doug Lea) Date: Thu, 11 Jun 2009 10:53:08 -0400 Subject: HashMap - Hybrid approach In-Reply-To: References: <4A2FAE34.3080902@cs.oswego.edu> Message-ID: <4A311A54.1070602@cs.oswego.edu> Alex Yakovlev wrote: > I thought about tree-like structures for collisions, but what came to my mind > is that the problem is not in search depth that can be reduced by non-linear > structures, but in memory fragmentation. > There were three reasons I suggested tries for collisions (1) to remove a dependent load, allowing prefetch into elements array (2) to decrease footprint, (3) to conform to current threshold parameters, while still avoiding long sequential pile-ups that you can get using using open-addressing-style collision handling, especially if the number of duplicate hash codes is higher than statistically expected. It may be possible to still get most of the effects without them though. Here are some notes on some possible ways to get there: First a digression: One source of hash duplicates is that System.identityHashCode (which you also get as Object.hashCode for keys that don't override it) has had from 21 to 25 significant bits in various releases of hotspot (and fewer in some other JVMs), which means you start seeing more and more of them in very large tables. It is a known problem that IdentityHashMaps may have crummy performance for tables past a few million elements. On the other hand, because identityHashCodes are nicely randomized by the JVM, and because IdentityHashMaps avoid an indirection compared to other maps, performance on less-than-huge tables is by far faster than for all others. About footprint (again; refining/correcting my previous mail): In 32bit mode, under default 3/4 load factor, the current Hashmap allocates C+6N words (6 = 4 fields plus 2 words object header). versus 13C/4 for yours. In steady state, C = 4N/3 just before a resize, and 8N/3 just after, with average at 6N/3 = 2N. So the steady state footprints below look good, but the initial sizes under default 16 capacity don't. (While I'm at it, for further comparison, the footprint for IdentityHashMap is also below. It hardwires 2/3 not 3/4 threshold.) steady state initial min ave max (default cap) yours: 4.33N 6.50N 8.67N 52 current: 7.33N 8.00N 8.67N 16 (Identity: 3.00N 4.50N 6.00N 32) To cope with known bloat issues, any possible HashMap replacement should have a *smaller* footprint for unused and tiny maps. Using a dynamic collision tree is one way to reduce initial footprint. Another way is to just to be more cautious on startup: * Lazily create all arrays * Separate the front and back of index array. (Note that you don't have any locality here to lose by doing so, since you jump to index h+hashlen.) * Use default initial capacity of 4, but jump to 32 or 64 on first collision. This would have overhead 0 (plus fixed fields) for unused maps and 12 for most tiny maps. Also, currently you initially allocate 64 words for a 1.0 loadfactor, which is surprisingly greater than 52 for 0.75 default. Under above scheme initial values would be closer. Further note that the steady state space differences between 0.75 and 1.0 are very small (6.25%): steady state initial min ave max (default cap) lf=1.0: 4.00N 6.00N 8.00N 64 And further, the performance differences are very small across these load factors in a few tests I ran. This suggests a different sort of tactic for helping with indirections and prefetches. Suppose that index[h] was usually just h; that is, with the key in elements[2*h]. (This only works in current design if loadFactor==1, but can work with smaller values if elements array is sized at hashlen.) Under this scheme, you can optimistically read this element at the same time as the index. To make this work, you'd need a bitmap of available slots rather than a freelist, adding C/32 space overhead, plus some heuristic juggling around inside put and resize. It might be worth an experiment or two. Doing this requires yet further work/thought about how sizing varies with loadFactor arguments. > But in my array-based approach it is possible to defragment it and (almost) > always store all collisions in sequental array indices. In current HashMap, the nodes chained inside bins tend to have good locality after a GC because GC will usually relocate them to be together. Analogously, it would seem to make sense to defrag only on resize, perhaps accompanied by heuristic one-place swaps during put. > I gathered some statisics, and in successful reads when map is almost full > with .75 load factor just before resize, 70% of gets happens on first try, > 23% on second, 5% on third lookup, 1% on fourth. The exact values depend of course on hash quality, insertion order, etc. But this also reflects fact that the collisions array (or back half of current index array) is typically less than 30% occupied. This suggests shrinking its default size and letting it independently resize, using the same sort of approach as with main hash: start out with say 25% size, by letting sets of 4 indices share an overflow slot, and expand as needed. Or maybe start with 50%, which will still be enough footprint savings to bother. -Doug From alex14n at gmail.com Thu Jun 11 17:05:06 2009 From: alex14n at gmail.com (Alex Yakovlev) Date: Thu, 11 Jun 2009 21:05:06 +0400 Subject: HashMap - Hybrid approach In-Reply-To: <4A311A54.1070602@cs.oswego.edu> References: <4A2FAE34.3080902@cs.oswego.edu> <4A311A54.1070602@cs.oswego.edu> Message-ID: Doug On Thu, Jun 11, 2009 at 6:53 PM, Doug Lea
wrote: > ?* Lazily create all arrays Done. (in scala version it was done from the very beginning) > ?* Separate the front and back of index array. (Note that > ? ?you don't have any locality here to lose by doing so, > ? ?since you jump to index h+hashlen.) Initially it were 2 different arrays, 'firstIndex' and 'nextIndex', but extra array is one extra pointer in HashMap object and extra array object header, so overall 2 arrays = more memory And second array contents now is tied to elements, it's used to store/check deleted list, etc. so it's not trivial to make lazy allocation. > ?* Use default initial capacity of 4, Done already > And further, the performance differences are very small across > these load factors in a few tests I ran. ?This suggests a > different sort of tactic for helping with indirections and > prefetches. ?Suppose that index[h] was usually just h; that is, > with the key in elements[2*h]. (This only works in current > design if loadFactor==1, but can work with smaller values if > elements array is sized at hashlen.) ?Under this scheme, you can > optimistically read this element at the same time as the I did a little test of this and saw no performance gain, I even posted a link to sources right after changing subject from 'fast' to 'hybrid'. Do you have any idea why second array prefetching may not happen? > index. To make this work, you'd need a bitmap of available slots > rather than a freelist, adding C/32 space overhead, plus some > heuristic juggling around inside put and resize. ?It might be > worth an experiment or two. ?Doing this requires yet further > work/thought about how sizing varies with loadFactor > arguments. There is no need for extra bitmap - it can be stored along hashcodes. > The exact values depend of course on hash quality, insertion > order, etc. But this also reflects fact that the collisions array > (or back half of current index array) is typically less than 30% > occupied. This suggests shrinking its default size and letting > it independently resize, using the same sort of approach as > with main hash: start out with say 25% size, by letting sets of > 4 indices share an overflow slot, and expand as needed. Or > maybe start with 50%, which will still be enough footprint savings > to bother. As we already discussed, this approach has greater footprint since we always need to alllocate 3*(hashtable size) words PLUS overflow. There would be sence to do that only if prefetching trick with the same array index for hash and key/value would work. But it did not in my experiment, don't know why. Maybe I'll test it on different computers lately, or you'll help with some advice. Alex From martinrb at google.com Thu Jun 11 21:16:37 2009 From: martinrb at google.com (Martin Buchholz) Date: Thu, 11 Jun 2009 14:16:37 -0700 Subject: Review request for 5049299 In-Reply-To: <4A311337.9070706@sun.com> References: <4A1678DB.9010209@sun.com> <4A2D1B65.5020002@sun.com> <1ccfd1c10906081352i12feb1a5j54133adb4f3f5c25@mail.gmail.com> <4A2E69FF.6080908@sun.com> <1ccfd1c10906090851g6498733dw78743283cdfe405f@mail.gmail.com> <4A2E8DA1.3050004@sun.com> <1ccfd1c10906091142h465bf9adoaa1b5c87bcf6ef9f@mail.gmail.com> <1ccfd1c10906091224p6cc0ccefpfa49453466e99640@mail.gmail.com> <1ccfd1c10906091244u14607dcesc40fc6017adc7808@mail.gmail.com> <4A311337.9070706@sun.com> Message-ID: <1ccfd1c10906111416x1b0fveb52eb9d3db56ded@mail.gmail.com> Thanks, Michael I'm hoping the following will placate sun studio cc: diff --git a/src/solaris/native/java/lang/UNIXProcess_md.c b/src/solaris/native/java/lang/UNIXProcess_md.c --- a/src/solaris/native/java/lang/UNIXProcess_md.c +++ b/src/solaris/native/java/lang/UNIXProcess_md.c @@ -651,6 +651,7 @@ } close(FAIL_FILENO); _exit(-1); + return 0; /* Suppress warning "no return value from function" */ } I'm also adding my manual test case BigFork.java. It may be helpful while implementing the Solaris version of this feature. webrev updated. I need a Sun bug to commit these changes for Linux. Please create one. Synopsis: * (process) Use clone(CLONE_VM), not fork, on Linux to avoid swap exhaustion * Description: On Linux it is possible to use clone with CLONE_VM, but not CLONE_THREAD, which is like fork() but much cheaper and avoids swap exhaustion due to momentary overcommit of swap space. One has to be very careful in this case to not mutate global variables such as environ, but it's worth it. Evaluation: Make it so. See also: 5049299 Once that is done, I will commit my changes. Thanks, Martin On Thu, Jun 11, 2009 at 07:22, Michael McMahon wrote: > Martin Buchholz wrote: > >> I broke down and finally created a "proper" webrev, >> just like the good old days. >> >> http://cr.openjdk.java.net/~martin/clone-exec/< >> http://cr.openjdk.java.net/%7Emartin/clone-exec/> >> >> I've run the regression tests on Solaris and Linux and they seem fine. > There is a compile warning on solaris at line 654: no return value from > function. > Aside from that, I'm happy with the change now > > - Michael. > -------------- next part -------------- An HTML attachment was scrubbed... URL: From xuelei.fan at sun.com Fri Jun 12 03:16:49 2009 From: xuelei.fan at sun.com (xuelei.fan at sun.com) Date: Fri, 12 Jun 2009 03:16:49 +0000 Subject: hg: jdk7/tl/jdk: 6570344: Invalid RSA OID in sun.security.x509.AlgorithmId Message-ID: <20090612031707.D2B3FE3ED@hg.openjdk.java.net> Changeset: ffbcf1d1103c Author: xuelei Date: 2009-06-12 09:00 +0800 URL: http://hg.openjdk.java.net/jdk7/tl/jdk/rev/ffbcf1d1103c 6570344: Invalid RSA OID in sun.security.x509.AlgorithmId Summary: change RSA OID to "2.5.8.1.1" Reviewed-by: mullan ! src/share/classes/sun/security/x509/AlgorithmId.java From tim.bell at sun.com Fri Jun 12 04:56:33 2009 From: tim.bell at sun.com (tim.bell at sun.com) Date: Fri, 12 Jun 2009 04:56:33 +0000 Subject: hg: jdk7/tl: 7 new changesets Message-ID: <20090612045633.48AD4E42A@hg.openjdk.java.net> Changeset: a942ea653d97 Author: aph Date: 2009-04-17 15:37 +0100 URL: http://hg.openjdk.java.net/jdk7/tl/rev/a942ea653d97 6829575: 100028: Debug information is incomplete or missing Summary: Enable debugging in many places Reviewed-by: ohair Contributed-by: Andrew Haley ! make/sanity-rules.gmk Changeset: f5ab6d6ae4b1 Author: xdono Date: 2009-05-07 10:30 -0700 URL: http://hg.openjdk.java.net/jdk7/tl/rev/f5ab6d6ae4b1 Merge - make/jprt.config Changeset: 37fad8722d92 Author: ohair Date: 2009-03-26 16:46 -0700 URL: http://hg.openjdk.java.net/jdk7/tl/rev/37fad8722d92 6822913: Consolidate make/jprt.config files, let JPRT manage this file make it optional in repos Reviewed-by: tbell - make/jprt.config Changeset: b284e021293c Author: ohair Date: 2009-05-08 16:40 -0700 URL: http://hg.openjdk.java.net/jdk7/tl/rev/b284e021293c Merge Changeset: 39565502682c Author: ohair Date: 2009-05-15 13:27 -0700 URL: http://hg.openjdk.java.net/jdk7/tl/rev/39565502682c Merge Changeset: 472c21584cfd Author: xdono Date: 2009-06-11 10:54 -0700 URL: http://hg.openjdk.java.net/jdk7/tl/rev/472c21584cfd Added tag jdk7-b60 for changeset 39565502682c ! .hgtags Changeset: 2734c0deab69 Author: tbell Date: 2009-06-11 21:09 -0700 URL: http://hg.openjdk.java.net/jdk7/tl/rev/2734c0deab69 Merge From tim.bell at sun.com Fri Jun 12 05:03:50 2009 From: tim.bell at sun.com (tim.bell at sun.com) Date: Fri, 12 Jun 2009 05:03:50 +0000 Subject: hg: jdk7/tl/corba: 7 new changesets Message-ID: <20090612050356.EF54AE435@hg.openjdk.java.net> Changeset: 7b47536c234e Author: ohair Date: 2009-03-26 16:47 -0700 URL: http://hg.openjdk.java.net/jdk7/tl/corba/rev/7b47536c234e 6822374: Windows: detect X64 when PROCESSOR_IDENTIFIER contains EM64T or Intel64 6822913: Consolidate make/jprt.config files, let JPRT manage this file make it optional in repos Reviewed-by: tbell ! make/common/shared/Platform.gmk - make/jprt.config Changeset: 39aa6ae82075 Author: ohair Date: 2009-05-08 16:42 -0700 URL: http://hg.openjdk.java.net/jdk7/tl/corba/rev/39aa6ae82075 Merge Changeset: da27d54e06bd Author: ohair Date: 2009-05-15 13:18 -0700 URL: http://hg.openjdk.java.net/jdk7/tl/corba/rev/da27d54e06bd Merge Changeset: 5dcbe748e580 Author: ohair Date: 2009-05-19 17:38 -0700 URL: http://hg.openjdk.java.net/jdk7/tl/corba/rev/5dcbe748e580 6843041: Remove duplicate README files in repositories (make/README) Reviewed-by: robilad - make/README Changeset: f1e1cccbd13a Author: ohair Date: 2009-05-19 18:09 -0700 URL: http://hg.openjdk.java.net/jdk7/tl/corba/rev/f1e1cccbd13a 6733313: corba build warnings: /bin/sh: gcc: not found Reviewed-by: tbell ! make/common/shared/Compiler-gcc.gmk ! make/common/shared/Compiler-sun.gmk Changeset: e906b16a12a9 Author: xdono Date: 2009-06-11 10:54 -0700 URL: http://hg.openjdk.java.net/jdk7/tl/corba/rev/e906b16a12a9 Added tag jdk7-b60 for changeset f1e1cccbd13a ! .hgtags Changeset: 23f2c435056b Author: tbell Date: 2009-06-11 21:11 -0700 URL: http://hg.openjdk.java.net/jdk7/tl/corba/rev/23f2c435056b Merge From tim.bell at sun.com Fri Jun 12 05:14:42 2009 From: tim.bell at sun.com (tim.bell at sun.com) Date: Fri, 12 Jun 2009 05:14:42 +0000 Subject: hg: jdk7/tl/hotspot: 6 new changesets Message-ID: <20090612051454.48030E446@hg.openjdk.java.net> Changeset: 5d4dd2f5f6a1 Author: aph Date: 2009-04-17 15:50 +0100 URL: http://hg.openjdk.java.net/jdk7/tl/hotspot/rev/5d4dd2f5f6a1 6829575: 100028: Debug information is incomplete or missing Summary: Enable debugging in many places Reviewed-by: ohair Contributed-by: Andrew Haley ! make/linux/makefiles/gcc.make ! make/linux/makefiles/jsig.make ! make/linux/makefiles/saproc.make Changeset: 7a485bc4da16 Author: xdono Date: 2009-05-07 10:30 -0700 URL: http://hg.openjdk.java.net/jdk7/tl/hotspot/rev/7a485bc4da16 Merge Changeset: 116b019a3961 Author: ohair Date: 2009-05-08 14:33 -0700 URL: http://hg.openjdk.java.net/jdk7/tl/hotspot/rev/116b019a3961 6839126: Type error found by newer windows compiler Reviewed-by: never, kvn ! src/share/vm/adlc/filebuff.hpp Changeset: f5ee65f94d9a Author: ohair Date: 2009-05-15 13:41 -0700 URL: http://hg.openjdk.java.net/jdk7/tl/hotspot/rev/f5ee65f94d9a Merge - make/jprt.config ! make/linux/makefiles/gcc.make ! make/linux/makefiles/jsig.make ! make/linux/makefiles/saproc.make Changeset: a77eddcd510c Author: ohair Date: 2009-05-19 17:40 -0700 URL: http://hg.openjdk.java.net/jdk7/tl/hotspot/rev/a77eddcd510c 6843041: Remove duplicate README files in repositories (make/README) Reviewed-by: robilad - make/README Changeset: 86092459c54d Author: xdono Date: 2009-06-11 10:54 -0700 URL: http://hg.openjdk.java.net/jdk7/tl/hotspot/rev/86092459c54d Added tag jdk7-b60 for changeset a77eddcd510c ! .hgtags From tim.bell at sun.com Fri Jun 12 05:29:35 2009 From: tim.bell at sun.com (tim.bell at sun.com) Date: Fri, 12 Jun 2009 05:29:35 +0000 Subject: hg: jdk7/tl/jaxp: 9 new changesets Message-ID: <20090612052948.B4CBEE44E@hg.openjdk.java.net> Changeset: 19c316392d9e Author: aph Date: 2009-04-17 15:55 +0100 URL: http://hg.openjdk.java.net/jdk7/tl/jaxp/rev/19c316392d9e 6829575: 100028: Debug information is incomplete or missing Summary: Enable debugging in many places Reviewed-by: ohair Contributed-by: Andrew Haley ! make/Makefile ! make/build.xml Changeset: 7967d26b229c Author: aph Date: 2009-04-20 19:00 +0100 URL: http://hg.openjdk.java.net/jdk7/tl/jaxp/rev/7967d26b229c 6832141: Bug 100045 - Fix for 100028 breaks debug info for class files Summary: Correct fallout from 100028 patch Reviewed-by: ohair Contributed-by: Andrew Haley ! make/Makefile Changeset: 04af70c1189c Author: ohair Date: 2009-05-06 11:27 -0700 URL: http://hg.openjdk.java.net/jdk7/tl/jaxp/rev/04af70c1189c 6837665: Deal with windows ant problem where commas in -D options do not work Reviewed-by: xdono ! make/Makefile ! make/build.properties Changeset: 44e5ca2a846c Author: xdono Date: 2009-05-07 10:30 -0700 URL: http://hg.openjdk.java.net/jdk7/tl/jaxp/rev/44e5ca2a846c Merge Changeset: 0ea9bb9c6ddc Author: xdono Date: 2009-05-07 12:26 -0700 URL: http://hg.openjdk.java.net/jdk7/tl/jaxp/rev/0ea9bb9c6ddc Merge - src/share/classes/com/sun/org/apache/xalan/internal/client/XSLTProcessorApplet.java - src/share/classes/com/sun/org/apache/xalan/internal/client/package.html Changeset: cdc8761f136a Author: ohair Date: 2009-05-15 13:24 -0700 URL: http://hg.openjdk.java.net/jdk7/tl/jaxp/rev/cdc8761f136a Merge Changeset: 259aef5045a1 Author: ohair Date: 2009-05-19 17:38 -0700 URL: http://hg.openjdk.java.net/jdk7/tl/jaxp/rev/259aef5045a1 6843041: Remove duplicate README files in repositories (make/README) Reviewed-by: robilad - make/README Changeset: f1ac756616ea Author: xdono Date: 2009-06-11 10:54 -0700 URL: http://hg.openjdk.java.net/jdk7/tl/jaxp/rev/f1ac756616ea Added tag jdk7-b60 for changeset 259aef5045a1 ! .hgtags Changeset: 97344798aaf7 Author: tbell Date: 2009-06-11 21:26 -0700 URL: http://hg.openjdk.java.net/jdk7/tl/jaxp/rev/97344798aaf7 Merge ! make/Makefile ! make/build.properties ! make/build.xml From tim.bell at sun.com Fri Jun 12 05:36:13 2009 From: tim.bell at sun.com (tim.bell at sun.com) Date: Fri, 12 Jun 2009 05:36:13 +0000 Subject: hg: jdk7/tl/jaxws: 9 new changesets Message-ID: <20090612053625.53640E457@hg.openjdk.java.net> Changeset: a92183572d99 Author: aph Date: 2009-04-17 15:56 +0100 URL: http://hg.openjdk.java.net/jdk7/tl/jaxws/rev/a92183572d99 6829575: 100028: Debug information is incomplete or missing Summary: Enable debugging in many places Reviewed-by: ohair Contributed-by: Andrew Haley ! make/Makefile ! make/build.xml Changeset: ab30d5761947 Author: aph Date: 2009-04-20 19:01 +0100 URL: http://hg.openjdk.java.net/jdk7/tl/jaxws/rev/ab30d5761947 6832141: Bug 100045 - Fix for 100028 breaks debug info for class files Summary: Correct fallout from 100028 patch Reviewed-by: ohair Contributed-by: Andrew Haley ! make/Makefile Changeset: 35c29ff8d904 Author: ohair Date: 2009-05-06 11:29 -0700 URL: http://hg.openjdk.java.net/jdk7/tl/jaxws/rev/35c29ff8d904 6837665: Deal with windows ant problem where commas in -D options do not work Reviewed-by: xdono ! make/Makefile ! make/build.properties Changeset: d95fec0fa489 Author: xdono Date: 2009-05-07 10:30 -0700 URL: http://hg.openjdk.java.net/jdk7/tl/jaxws/rev/d95fec0fa489 Merge ! make/Makefile ! make/build.xml - src/share/classes/com/sun/tools/internal/txw2/AntErrorListener.java - src/share/classes/com/sun/tools/internal/txw2/ConsoleErrorReporter.java - src/share/classes/com/sun/tools/internal/txw2/ErrorListener.java - src/share/classes/com/sun/tools/internal/txw2/Main.java - src/share/classes/com/sun/tools/internal/txw2/NameUtil.java - src/share/classes/com/sun/tools/internal/txw2/RELAXNGLoader.java - src/share/classes/com/sun/tools/internal/txw2/SchemaBuilder.java - src/share/classes/com/sun/tools/internal/txw2/TxwOptions.java - src/share/classes/com/sun/tools/internal/txw2/TxwTask.java - src/share/classes/com/sun/tools/internal/txw2/XmlSchemaLoader.java - src/share/classes/com/sun/tools/internal/txw2/builder/relaxng/AnnotationsImpl.java - src/share/classes/com/sun/tools/internal/txw2/builder/relaxng/CommentListImpl.java - src/share/classes/com/sun/tools/internal/txw2/builder/relaxng/DataPatternBuilderImpl.java - src/share/classes/com/sun/tools/internal/txw2/builder/relaxng/DatatypeFactory.java - src/share/classes/com/sun/tools/internal/txw2/builder/relaxng/DivImpl.java - src/share/classes/com/sun/tools/internal/txw2/builder/relaxng/ElementAnnotationBuilderImpl.java - src/share/classes/com/sun/tools/internal/txw2/builder/relaxng/GrammarImpl.java - src/share/classes/com/sun/tools/internal/txw2/builder/relaxng/GrammarSectionImpl.java - src/share/classes/com/sun/tools/internal/txw2/builder/relaxng/SchemaBuilderImpl.java - src/share/classes/com/sun/tools/internal/txw2/builder/relaxng/package.html - src/share/classes/com/sun/tools/internal/txw2/builder/xsd/XmlSchemaBuilder.java - src/share/classes/com/sun/tools/internal/txw2/builder/xsd/package.html - src/share/classes/com/sun/tools/internal/txw2/model/Attribute.java - src/share/classes/com/sun/tools/internal/txw2/model/CycleIterator.java - src/share/classes/com/sun/tools/internal/txw2/model/Data.java - src/share/classes/com/sun/tools/internal/txw2/model/Define.java - src/share/classes/com/sun/tools/internal/txw2/model/Element.java - src/share/classes/com/sun/tools/internal/txw2/model/Empty.java - src/share/classes/com/sun/tools/internal/txw2/model/Grammar.java - src/share/classes/com/sun/tools/internal/txw2/model/Leaf.java - src/share/classes/com/sun/tools/internal/txw2/model/List.java - src/share/classes/com/sun/tools/internal/txw2/model/Node.java - src/share/classes/com/sun/tools/internal/txw2/model/NodeSet.java - src/share/classes/com/sun/tools/internal/txw2/model/Ref.java - src/share/classes/com/sun/tools/internal/txw2/model/Text.java - src/share/classes/com/sun/tools/internal/txw2/model/Value.java - src/share/classes/com/sun/tools/internal/txw2/model/WriterNode.java - src/share/classes/com/sun/tools/internal/txw2/model/XmlNode.java - src/share/classes/com/sun/tools/internal/txw2/model/prop/AttributeProp.java - src/share/classes/com/sun/tools/internal/txw2/model/prop/ElementProp.java - src/share/classes/com/sun/tools/internal/txw2/model/prop/LeafElementProp.java - src/share/classes/com/sun/tools/internal/txw2/model/prop/Prop.java - src/share/classes/com/sun/tools/internal/txw2/model/prop/ValueProp.java - src/share/classes/com/sun/tools/internal/txw2/model/prop/XmlItemProp.java - src/share/classes/com/sun/tools/internal/ws/processor/Processor.java - src/share/classes/com/sun/tools/internal/ws/processor/ProcessorAction.java - src/share/classes/com/sun/tools/internal/ws/processor/ProcessorActionVersion.java - src/share/classes/com/sun/tools/internal/ws/processor/ProcessorConstants.java - src/share/classes/com/sun/tools/internal/ws/processor/ProcessorNotificationListener.java - src/share/classes/com/sun/tools/internal/ws/processor/ProcessorOptions.java - src/share/classes/com/sun/tools/internal/ws/processor/config/ClassModelInfo.java - src/share/classes/com/sun/tools/internal/ws/processor/config/Configuration.java - src/share/classes/com/sun/tools/internal/ws/processor/config/ConfigurationException.java - src/share/classes/com/sun/tools/internal/ws/processor/config/HandlerChainInfo.java - src/share/classes/com/sun/tools/internal/ws/processor/config/HandlerInfo.java - src/share/classes/com/sun/tools/internal/ws/processor/config/ModelInfo.java - src/share/classes/com/sun/tools/internal/ws/processor/config/WSDLModelInfo.java - src/share/classes/com/sun/tools/internal/ws/processor/config/parser/ClassModelParser.java - src/share/classes/com/sun/tools/internal/ws/processor/config/parser/CustomizationParser.java - src/share/classes/com/sun/tools/internal/ws/processor/config/parser/InputParser.java - src/share/classes/com/sun/tools/internal/ws/processor/config/parser/JAXWSBindingInfoParser.java - src/share/classes/com/sun/tools/internal/ws/processor/config/parser/ParserUtil.java - src/share/classes/com/sun/tools/internal/ws/processor/config/parser/Reader.java - src/share/classes/com/sun/tools/internal/ws/processor/generator/JAXBTypeGenerator.java - src/share/classes/com/sun/tools/internal/ws/processor/generator/SimpleToBoxedUtil.java - src/share/classes/com/sun/tools/internal/ws/processor/modeler/ModelerUtils.java - src/share/classes/com/sun/tools/internal/ws/processor/modeler/annotation/WebServiceReferenceCollector.java - src/share/classes/com/sun/tools/internal/ws/processor/util/ClientProcessorEnvironment.java - src/share/classes/com/sun/tools/internal/ws/processor/util/GeneratedFileInfo.java - src/share/classes/com/sun/tools/internal/ws/processor/util/ProcessorEnvironment.java - src/share/classes/com/sun/tools/internal/ws/processor/util/ProcessorEnvironmentBase.java - src/share/classes/com/sun/tools/internal/ws/util/JAXWSClassFactory.java - src/share/classes/com/sun/tools/internal/ws/util/JavaCompilerHelper.java - src/share/classes/com/sun/tools/internal/ws/util/MapBase.java - src/share/classes/com/sun/tools/internal/ws/util/ToolBase.java - src/share/classes/com/sun/tools/internal/ws/util/xml/NodeListIterator.java - src/share/classes/com/sun/tools/internal/ws/util/xml/NullEntityResolver.java - src/share/classes/com/sun/tools/internal/ws/util/xml/PrettyPrintingXmlWriter.java - src/share/classes/com/sun/tools/internal/ws/util/xml/XmlWriter.java - src/share/classes/com/sun/tools/internal/ws/wscompile/ActionConstants.java - src/share/classes/com/sun/tools/internal/ws/wscompile/CompileTool.java - src/share/classes/com/sun/tools/internal/ws/wsdl/document/schema/BuiltInTypes.java - src/share/classes/com/sun/tools/internal/ws/wsdl/document/schema/Schema.java - src/share/classes/com/sun/tools/internal/ws/wsdl/document/schema/SchemaAttribute.java - src/share/classes/com/sun/tools/internal/ws/wsdl/document/schema/SchemaDocument.java - src/share/classes/com/sun/tools/internal/ws/wsdl/document/schema/SchemaElement.java - src/share/classes/com/sun/tools/internal/ws/wsdl/document/schema/SchemaEntity.java - src/share/classes/com/sun/tools/internal/ws/wsdl/framework/Extensible.java - src/share/classes/com/sun/tools/internal/ws/wsdl/framework/Extension.java - src/share/classes/com/sun/tools/internal/ws/wsdl/framework/ParserContext.java - src/share/classes/com/sun/tools/internal/ws/wsdl/framework/WriterContext.java - src/share/classes/com/sun/tools/internal/ws/wsdl/parser/ExtensionHandler.java - src/share/classes/com/sun/tools/internal/ws/wsdl/parser/ExtensionHandlerBase.java - src/share/classes/com/sun/tools/internal/ws/wsdl/parser/SchemaExtensionHandler.java - src/share/classes/com/sun/tools/internal/ws/wsdl/parser/SchemaParser.java - src/share/classes/com/sun/tools/internal/ws/wsdl/parser/SchemaWriter.java - src/share/classes/com/sun/tools/internal/ws/wsdl/parser/WSDLWriter.java - src/share/classes/com/sun/tools/internal/xjc/reader/dtd/bindinfo/DOM4JLocator.java - src/share/classes/com/sun/tools/internal/xjc/util/XMLStreamReaderToContentHandler.java - src/share/classes/com/sun/xml/internal/bind/v2/doc-files/packages.png - src/share/classes/com/sun/xml/internal/bind/v2/doc-files/packages.vsd - src/share/classes/com/sun/xml/internal/bind/v2/doc-files/readme.txt - src/share/classes/com/sun/xml/internal/ws/binding/http/HTTPBindingImpl.java - src/share/classes/com/sun/xml/internal/ws/binding/soap/SOAPBindingImpl.java - src/share/classes/com/sun/xml/internal/ws/client/AsyncHandlerService.java - src/share/classes/com/sun/xml/internal/ws/client/ClientConfigurationException.java - src/share/classes/com/sun/xml/internal/ws/client/ContactInfoBase.java - src/share/classes/com/sun/xml/internal/ws/client/ContactInfoListImpl.java - src/share/classes/com/sun/xml/internal/ws/client/ContactInfoListIteratorBase.java - src/share/classes/com/sun/xml/internal/ws/client/ContextMap.java - src/share/classes/com/sun/xml/internal/ws/client/EndpointIFBase.java - src/share/classes/com/sun/xml/internal/ws/client/EndpointIFContext.java - src/share/classes/com/sun/xml/internal/ws/client/EndpointIFInvocationHandler.java - src/share/classes/com/sun/xml/internal/ws/client/InternalBindingProvider.java - src/share/classes/com/sun/xml/internal/ws/client/PortInfoBase.java - src/share/classes/com/sun/xml/internal/ws/client/ServiceContext.java - src/share/classes/com/sun/xml/internal/ws/client/ServiceContextBuilder.java - src/share/classes/com/sun/xml/internal/ws/client/WSFuture.java - src/share/classes/com/sun/xml/internal/ws/client/dispatch/DispatchBase.java - src/share/classes/com/sun/xml/internal/ws/client/dispatch/DispatchContext.java - src/share/classes/com/sun/xml/internal/ws/client/dispatch/ResponseImpl.java - src/share/classes/com/sun/xml/internal/ws/client/dispatch/impl/DispatchContactInfoList.java - src/share/classes/com/sun/xml/internal/ws/client/dispatch/impl/DispatchDelegate.java - src/share/classes/com/sun/xml/internal/ws/client/dispatch/impl/encoding/DispatchSerializer.java - src/share/classes/com/sun/xml/internal/ws/client/dispatch/impl/encoding/DispatchUtil.java - src/share/classes/com/sun/xml/internal/ws/client/dispatch/impl/protocol/MessageDispatcherHelper.java - src/share/classes/com/sun/xml/internal/ws/encoding/EncoderDecoderBase.java - src/share/classes/com/sun/xml/internal/ws/encoding/JAXWSAttachmentMarshaller.java - src/share/classes/com/sun/xml/internal/ws/encoding/JAXWSAttachmentUnmarshaller.java - src/share/classes/com/sun/xml/internal/ws/encoding/internal/InternalEncoder.java - src/share/classes/com/sun/xml/internal/ws/encoding/jaxb/JAXBBeanInfo.java - src/share/classes/com/sun/xml/internal/ws/encoding/jaxb/JAXBBridgeInfo.java - src/share/classes/com/sun/xml/internal/ws/encoding/jaxb/JAXBTypeSerializer.java - src/share/classes/com/sun/xml/internal/ws/encoding/jaxb/RpcLitPayload.java - src/share/classes/com/sun/xml/internal/ws/encoding/jaxb/RpcLitPayloadSerializer.java - src/share/classes/com/sun/xml/internal/ws/encoding/simpletype/EncoderUtils.java - src/share/classes/com/sun/xml/internal/ws/encoding/simpletype/SimpleTypeConstants.java - src/share/classes/com/sun/xml/internal/ws/encoding/soap/ClientEncoderDecoder.java - src/share/classes/com/sun/xml/internal/ws/encoding/soap/EncoderDecoder.java - src/share/classes/com/sun/xml/internal/ws/encoding/soap/SOAPDecoder.java - src/share/classes/com/sun/xml/internal/ws/encoding/soap/SOAPEPTFactory.java - src/share/classes/com/sun/xml/internal/ws/encoding/soap/SOAPEncoder.java - src/share/classes/com/sun/xml/internal/ws/encoding/soap/SOAPVersion.java - src/share/classes/com/sun/xml/internal/ws/encoding/soap/ServerEncoderDecoder.java - src/share/classes/com/sun/xml/internal/ws/encoding/soap/client/SOAP12XMLDecoder.java - src/share/classes/com/sun/xml/internal/ws/encoding/soap/client/SOAP12XMLEncoder.java - src/share/classes/com/sun/xml/internal/ws/encoding/soap/client/SOAPXMLDecoder.java - src/share/classes/com/sun/xml/internal/ws/encoding/soap/client/SOAPXMLEncoder.java - src/share/classes/com/sun/xml/internal/ws/encoding/soap/internal/AttachmentBlock.java - src/share/classes/com/sun/xml/internal/ws/encoding/soap/internal/BodyBlock.java - src/share/classes/com/sun/xml/internal/ws/encoding/soap/internal/DelegateBase.java - src/share/classes/com/sun/xml/internal/ws/encoding/soap/internal/HeaderBlock.java - src/share/classes/com/sun/xml/internal/ws/encoding/soap/internal/InternalMessage.java - src/share/classes/com/sun/xml/internal/ws/encoding/soap/internal/MessageBlock.java - src/share/classes/com/sun/xml/internal/ws/encoding/soap/internal/MessageInfoBase.java - src/share/classes/com/sun/xml/internal/ws/encoding/soap/internal/SOAP12NotUnderstoodHeaderBlock.java - src/share/classes/com/sun/xml/internal/ws/encoding/soap/message/FaultCode.java - src/share/classes/com/sun/xml/internal/ws/encoding/soap/message/FaultCodeEnum.java - src/share/classes/com/sun/xml/internal/ws/encoding/soap/message/FaultReason.java - src/share/classes/com/sun/xml/internal/ws/encoding/soap/message/FaultReasonText.java - src/share/classes/com/sun/xml/internal/ws/encoding/soap/message/FaultSubcode.java - src/share/classes/com/sun/xml/internal/ws/encoding/soap/message/SOAP12FaultInfo.java - src/share/classes/com/sun/xml/internal/ws/encoding/soap/message/SOAPFaultInfo.java - src/share/classes/com/sun/xml/internal/ws/encoding/soap/message/SOAPMsgCreateException.java - src/share/classes/com/sun/xml/internal/ws/encoding/soap/message/SOAPMsgFactoryCreateException.java - src/share/classes/com/sun/xml/internal/ws/encoding/soap/server/ProviderSED.java - src/share/classes/com/sun/xml/internal/ws/encoding/soap/server/SOAP12XMLDecoder.java - src/share/classes/com/sun/xml/internal/ws/encoding/soap/server/SOAP12XMLEncoder.java - src/share/classes/com/sun/xml/internal/ws/encoding/soap/server/SOAPXMLDecoder.java - src/share/classes/com/sun/xml/internal/ws/encoding/soap/server/SOAPXMLEncoder.java - src/share/classes/com/sun/xml/internal/ws/encoding/xml/XMLDecoder.java - src/share/classes/com/sun/xml/internal/ws/encoding/xml/XMLEPTFactory.java - src/share/classes/com/sun/xml/internal/ws/encoding/xml/XMLEncoder.java - src/share/classes/com/sun/xml/internal/ws/handler/HandlerChainCaller.java - src/share/classes/com/sun/xml/internal/ws/handler/HandlerContext.java - src/share/classes/com/sun/xml/internal/ws/handler/HandlerResolverImpl.java - src/share/classes/com/sun/xml/internal/ws/handler/MessageContextUtil.java - src/share/classes/com/sun/xml/internal/ws/handler/SHDSOAPMessageContext.java - src/share/classes/com/sun/xml/internal/ws/handler/SOAPHandlerContext.java - src/share/classes/com/sun/xml/internal/ws/handler/XMLHandlerContext.java - src/share/classes/com/sun/xml/internal/ws/handler/XMLLogicalMessageContextImpl.java - src/share/classes/com/sun/xml/internal/ws/handler/XMLLogicalMessageImpl.java - src/share/classes/com/sun/xml/internal/ws/handler/package-info.java - src/share/classes/com/sun/xml/internal/ws/model/CheckedException.java - src/share/classes/com/sun/xml/internal/ws/model/ExceptionType.java - src/share/classes/com/sun/xml/internal/ws/model/JavaMethod.java - src/share/classes/com/sun/xml/internal/ws/model/Mode.java - src/share/classes/com/sun/xml/internal/ws/model/Parameter.java - src/share/classes/com/sun/xml/internal/ws/model/ParameterBinding.java - src/share/classes/com/sun/xml/internal/ws/model/RuntimeModel.java - src/share/classes/com/sun/xml/internal/ws/model/soap/SOAPBinding.java - src/share/classes/com/sun/xml/internal/ws/model/soap/SOAPRuntimeModel.java - src/share/classes/com/sun/xml/internal/ws/model/soap/Style.java - src/share/classes/com/sun/xml/internal/ws/model/soap/Use.java - src/share/classes/com/sun/xml/internal/ws/modeler/RuntimeModeler.java - src/share/classes/com/sun/xml/internal/ws/modeler/RuntimeModelerException.java - src/share/classes/com/sun/xml/internal/ws/pept/Delegate.java - src/share/classes/com/sun/xml/internal/ws/pept/encoding/Decoder.java - src/share/classes/com/sun/xml/internal/ws/pept/encoding/Encoder.java - src/share/classes/com/sun/xml/internal/ws/pept/ept/Acceptor.java - src/share/classes/com/sun/xml/internal/ws/pept/ept/ContactInfo.java - src/share/classes/com/sun/xml/internal/ws/pept/ept/ContactInfoList.java - src/share/classes/com/sun/xml/internal/ws/pept/ept/ContactInfoListIterator.java - src/share/classes/com/sun/xml/internal/ws/pept/ept/EPTFactory.java - src/share/classes/com/sun/xml/internal/ws/pept/ept/MessageInfo.java - src/share/classes/com/sun/xml/internal/ws/pept/presentation/MessageStruct.java - src/share/classes/com/sun/xml/internal/ws/pept/presentation/Stub.java - src/share/classes/com/sun/xml/internal/ws/pept/presentation/TargetFinder.java - src/share/classes/com/sun/xml/internal/ws/pept/presentation/Tie.java - src/share/classes/com/sun/xml/internal/ws/pept/protocol/Interceptors.java - src/share/classes/com/sun/xml/internal/ws/pept/protocol/MessageDispatcher.java - src/share/classes/com/sun/xml/internal/ws/protocol/soap/client/SOAPMessageDispatcher.java - src/share/classes/com/sun/xml/internal/ws/protocol/soap/server/ProviderSOAPMD.java - src/share/classes/com/sun/xml/internal/ws/protocol/soap/server/SOAPMessageDispatcher.java - src/share/classes/com/sun/xml/internal/ws/protocol/xml/client/XMLMessageDispatcher.java - src/share/classes/com/sun/xml/internal/ws/protocol/xml/server/ProviderXMLMD.java - src/share/classes/com/sun/xml/internal/ws/protocol/xml/server/XMLMessageDispatcher.java - src/share/classes/com/sun/xml/internal/ws/server/AppMsgContextImpl.java - src/share/classes/com/sun/xml/internal/ws/server/DocInfo.java - src/share/classes/com/sun/xml/internal/ws/server/EPTFactoryBase.java - src/share/classes/com/sun/xml/internal/ws/server/EPTFactoryFactoryBase.java - src/share/classes/com/sun/xml/internal/ws/server/PeptTie.java - src/share/classes/com/sun/xml/internal/ws/server/RuntimeContext.java - src/share/classes/com/sun/xml/internal/ws/server/RuntimeEndpointInfo.java - src/share/classes/com/sun/xml/internal/ws/server/TargetFinderImpl.java - src/share/classes/com/sun/xml/internal/ws/server/Tie.java - src/share/classes/com/sun/xml/internal/ws/server/XMLEPTFactoryImpl.java - src/share/classes/com/sun/xml/internal/ws/server/provider/ProviderModel.java - src/share/classes/com/sun/xml/internal/ws/server/provider/ProviderPeptTie.java - src/share/classes/com/sun/xml/internal/ws/spi/runtime/Binding.java - src/share/classes/com/sun/xml/internal/ws/spi/runtime/ClientTransportFactory.java - src/share/classes/com/sun/xml/internal/ws/spi/runtime/ClientTransportFactoryTypes.java - src/share/classes/com/sun/xml/internal/ws/spi/runtime/InternalSoapEncoder.java - src/share/classes/com/sun/xml/internal/ws/spi/runtime/Invoker.java - src/share/classes/com/sun/xml/internal/ws/spi/runtime/MessageContext.java - src/share/classes/com/sun/xml/internal/ws/spi/runtime/MtomCallback.java - src/share/classes/com/sun/xml/internal/ws/spi/runtime/RuntimeEndpointInfo.java - src/share/classes/com/sun/xml/internal/ws/spi/runtime/SOAPMessageContext.java - src/share/classes/com/sun/xml/internal/ws/spi/runtime/StubBase.java - src/share/classes/com/sun/xml/internal/ws/spi/runtime/SystemHandlerDelegate.java - src/share/classes/com/sun/xml/internal/ws/spi/runtime/SystemHandlerDelegateFactory.java - src/share/classes/com/sun/xml/internal/ws/spi/runtime/Tie.java - src/share/classes/com/sun/xml/internal/ws/spi/runtime/WSConnection.java - src/share/classes/com/sun/xml/internal/ws/spi/runtime/WebServiceContext.java - src/share/classes/com/sun/xml/internal/ws/spi/runtime/package-info.java - src/share/classes/com/sun/xml/internal/ws/streaming/XMLStreamReaderFactory.java - src/share/classes/com/sun/xml/internal/ws/streaming/XMLStreamWriterFactory.java - src/share/classes/com/sun/xml/internal/ws/transport/WSConnectionImpl.java - src/share/classes/com/sun/xml/internal/ws/transport/http/client/HttpClientTransportFactory.java - src/share/classes/com/sun/xml/internal/ws/transport/http/server/EndpointDocInfo.java - src/share/classes/com/sun/xml/internal/ws/transport/http/server/EndpointEntityResolver.java - src/share/classes/com/sun/xml/internal/ws/transport/http/server/WebServiceContextImpl.java - src/share/classes/com/sun/xml/internal/ws/transport/local/LocalMessage.java - src/share/classes/com/sun/xml/internal/ws/transport/local/client/LocalClientTransport.java - src/share/classes/com/sun/xml/internal/ws/transport/local/client/LocalClientTransportFactory.java - src/share/classes/com/sun/xml/internal/ws/transport/local/server/LocalConnectionImpl.java - src/share/classes/com/sun/xml/internal/ws/transport/local/server/LocalWSContextImpl.java - src/share/classes/com/sun/xml/internal/ws/util/Base64Util.java - src/share/classes/com/sun/xml/internal/ws/util/MessageInfoUtil.java - src/share/classes/com/sun/xml/internal/ws/util/NullIterator.java - src/share/classes/com/sun/xml/internal/ws/util/SOAPConnectionUtil.java - src/share/classes/com/sun/xml/internal/ws/util/SOAPUtil.java - src/share/classes/com/sun/xml/internal/ws/util/SunStAXReflection.java - src/share/classes/com/sun/xml/internal/ws/util/XMLConnectionUtil.java - src/share/classes/com/sun/xml/internal/ws/util/xml/XMLStreamReaderToContentHandler.java - src/share/classes/com/sun/xml/internal/ws/wsdl/WSDLContext.java - src/share/classes/com/sun/xml/internal/ws/wsdl/parser/Binding.java - src/share/classes/com/sun/xml/internal/ws/wsdl/parser/BindingOperation.java - src/share/classes/com/sun/xml/internal/ws/wsdl/parser/Message.java - src/share/classes/com/sun/xml/internal/ws/wsdl/parser/Part.java - src/share/classes/com/sun/xml/internal/ws/wsdl/parser/Port.java - src/share/classes/com/sun/xml/internal/ws/wsdl/parser/PortType.java - src/share/classes/com/sun/xml/internal/ws/wsdl/parser/PortTypeOperation.java - src/share/classes/com/sun/xml/internal/ws/wsdl/parser/Service.java - src/share/classes/com/sun/xml/internal/ws/wsdl/parser/WSDLDocument.java - src/share/classes/com/sun/xml/internal/ws/wsdl/writer/WSDLOutputResolver.java - src/share/classes/com/sun/xml/internal/xsom/impl/util/ConcatIterator.java - src/share/classes/com/sun/xml/internal/xsom/impl/util/FilterIterator.java Changeset: 1626ba49a98e Author: xdono Date: 2009-05-07 12:26 -0700 URL: http://hg.openjdk.java.net/jdk7/tl/jaxws/rev/1626ba49a98e Merge Changeset: af4d62e31af8 Author: ohair Date: 2009-05-15 13:25 -0700 URL: http://hg.openjdk.java.net/jdk7/tl/jaxws/rev/af4d62e31af8 Merge Changeset: 3b054db3e277 Author: ohair Date: 2009-05-19 17:39 -0700 URL: http://hg.openjdk.java.net/jdk7/tl/jaxws/rev/3b054db3e277 6843041: Remove duplicate README files in repositories (make/README) Reviewed-by: robilad - make/README Changeset: aeabf802f2a1 Author: xdono Date: 2009-06-11 10:54 -0700 URL: http://hg.openjdk.java.net/jdk7/tl/jaxws/rev/aeabf802f2a1 Added tag jdk7-b60 for changeset 3b054db3e277 ! .hgtags Changeset: 2ec98e99e4ea Author: tbell Date: 2009-06-11 21:30 -0700 URL: http://hg.openjdk.java.net/jdk7/tl/jaxws/rev/2ec98e99e4ea Merge ! make/Makefile ! make/build.properties ! make/build.xml From tim.bell at sun.com Fri Jun 12 05:45:21 2009 From: tim.bell at sun.com (tim.bell at sun.com) Date: Fri, 12 Jun 2009 05:45:21 +0000 Subject: hg: jdk7/tl/jdk: 32 new changesets Message-ID: <20090612055149.29B5FE461@hg.openjdk.java.net> Changeset: 9ad7e6462145 Author: aph Date: 2009-04-17 15:56 +0100 URL: http://hg.openjdk.java.net/jdk7/tl/jdk/rev/9ad7e6462145 6829575: 100028: Debug information is incomplete or missing Summary: Enable debugging in many places Reviewed-by: ohair Contributed-by: Andrew Haley ! make/common/Defs-linux.gmk ! make/sun/awt/mawt.gmk Changeset: 5ceb9eb621d1 Author: chegar Date: 2009-05-07 17:02 +0100 URL: http://hg.openjdk.java.net/jdk7/tl/jdk/rev/5ceb9eb621d1 6837982: SCTP API docs not being generated. Summary: Update docs makefile to build javadoc for the com.sun.nio.sctp package. Reviewed-by: jccollet, alanb, weijun ! make/docs/Makefile Changeset: 86d2541a9ba2 Author: xdono Date: 2009-05-07 10:31 -0700 URL: http://hg.openjdk.java.net/jdk7/tl/jdk/rev/86d2541a9ba2 Merge - src/share/native/java/util/zip/ZipEntry.c - src/share/native/sun/java2d/pipe/RenderBuffer.c - test/com/sun/awt/Translucency/TranslucentJAppletTest/TranslucentJAppletTest.java - test/com/sun/awt/Translucency/TranslucentShapedFrameTest/TSFrame.java - test/com/sun/awt/Translucency/TranslucentShapedFrameTest/TranslucentShapedFrameTest.form - test/com/sun/awt/Translucency/TranslucentShapedFrameTest/TranslucentShapedFrameTest.java Changeset: 39d93fb6926c Author: xdono Date: 2009-05-07 12:26 -0700 URL: http://hg.openjdk.java.net/jdk7/tl/jdk/rev/39d93fb6926c Merge Changeset: 6ca1c622dd6e Author: ohair Date: 2009-05-07 18:19 -0700 URL: http://hg.openjdk.java.net/jdk7/tl/jdk/rev/6ca1c622dd6e 6835803: Type error in src/windows/native/sun/windows/awt_Window.cpp Reviewed-by: prr ! src/windows/native/sun/windows/awt_Window.cpp Changeset: 7ec6857812d2 Author: ohair Date: 2009-05-08 11:24 -0700 URL: http://hg.openjdk.java.net/jdk7/tl/jdk/rev/7ec6857812d2 Merge ! src/windows/native/sun/windows/awt_Window.cpp Changeset: 9eeeeee69368 Author: ohair Date: 2009-05-15 13:14 -0700 URL: http://hg.openjdk.java.net/jdk7/tl/jdk/rev/9eeeeee69368 6841873: Fix windows redist default location for msvc runtime dlls Reviewed-by: tbell ! make/common/shared/Defs-windows.gmk Changeset: 97064d73976f Author: ohair Date: 2009-05-15 13:21 -0700 URL: http://hg.openjdk.java.net/jdk7/tl/jdk/rev/97064d73976f Merge Changeset: fdbc48164a8b Author: ohair Date: 2009-05-18 10:36 -0700 URL: http://hg.openjdk.java.net/jdk7/tl/jdk/rev/fdbc48164a8b 6842023: Improve test reliability, Increase timeout factor on jtreg tests, etc. Reviewed-by: tbell ! make/jprt.properties ! test/Makefile ! test/java/lang/ThreadGroup/NullThreadName.java ! test/java/util/ResourceBundle/RestrictedBundleTest.java ! test/java/util/WeakHashMap/GCDuringIteration.java Changeset: c06d30bd8c69 Author: andrew Date: 2009-05-21 16:29 +0100 URL: http://hg.openjdk.java.net/jdk7/tl/jdk/rev/c06d30bd8c69 6841728: Make building the Nimbus L 'n' F optional (100054) Summary: Add DISABLE_NIMBUS variable to prevent Nimbus subdirs being built Reviewed-by: mr, ohair ! make/common/shared/Sanity.gmk ! make/javax/swing/plaf/Makefile ! make/tools/Makefile Changeset: 238591c80bc5 Author: aph Date: 2009-05-21 18:41 +0100 URL: http://hg.openjdk.java.net/jdk7/tl/jdk/rev/238591c80bc5 6839133: lcms 1.18 update breaks ICC_ProfileRGB Tests Reviewed-by: avu, prr ! src/share/native/sun/java2d/cmm/lcms/LCMS.c Changeset: f62f7fcc9965 Author: art Date: 2009-05-15 15:40 +0400 URL: http://hg.openjdk.java.net/jdk7/tl/jdk/rev/f62f7fcc9965 6678385: Random java.lang.StackOverflowError from various JDKs Reviewed-by: stayer ! make/sun/xawt/mapfile-vers ! src/solaris/classes/sun/awt/X11/MotifDnDConstants.java ! src/solaris/classes/sun/awt/X11/MotifDnDDragSourceProtocol.java ! src/solaris/classes/sun/awt/X11/MotifDnDDropTargetProtocol.java ! src/solaris/classes/sun/awt/X11/WindowPropertyGetter.java ! src/solaris/classes/sun/awt/X11/XAWTXSettings.java ! src/solaris/classes/sun/awt/X11/XDecoratedPeer.java ! src/solaris/classes/sun/awt/X11/XDnDDragSourceProtocol.java ! src/solaris/classes/sun/awt/X11/XDnDDropTargetProtocol.java ! src/solaris/classes/sun/awt/X11/XDragSourceProtocol.java ! src/solaris/classes/sun/awt/X11/XDropTargetRegistry.java ! src/solaris/classes/sun/awt/X11/XEmbedCanvasPeer.java + src/solaris/classes/sun/awt/X11/XErrorHandler.java ! src/solaris/classes/sun/awt/X11/XProtocol.java ! src/solaris/classes/sun/awt/X11/XQueryTree.java ! src/solaris/classes/sun/awt/X11/XToolkit.java ! src/solaris/classes/sun/awt/X11/XTranslateCoordinates.java ! src/solaris/classes/sun/awt/X11/XWM.java ! src/solaris/classes/sun/awt/X11/XlibUtil.java ! src/solaris/classes/sun/awt/X11/XlibWrapper.java ! src/solaris/native/sun/awt/awt_GraphicsEnv.c ! src/solaris/native/sun/awt/awt_InputMethod.c ! src/solaris/native/sun/awt/awt_MToolkit.c ! src/solaris/native/sun/xawt/XToolkit.c ! src/solaris/native/sun/xawt/XlibWrapper.c Changeset: 019fd945ebc5 Author: yan Date: 2009-05-18 12:39 +0400 URL: http://hg.openjdk.java.net/jdk7/tl/jdk/rev/019fd945ebc5 6834525: PIT: RowToleranceTransitivityTest test fail with crash on rhel4 x86 and rhel 5x86 Summary: do not try to use released XKB resources Reviewed-by: art ! src/solaris/classes/sun/awt/X11/XKeysym.java ! src/solaris/classes/sun/awt/X11/XToolkit.java ! src/solaris/classes/sun/awt/X11/keysym2ucs.h Changeset: 875524a2b311 Author: anthony Date: 2009-05-19 12:15 +0400 URL: http://hg.openjdk.java.net/jdk7/tl/jdk/rev/875524a2b311 6811219: Deadlock java AWT in XWarningWindow Summary: The locking scheme has been re-architected, the code slightly refactored. Reviewed-by: art, dcherepanov ! src/solaris/classes/sun/awt/X11/XWarningWindow.java ! src/solaris/classes/sun/awt/X11/XWindowPeer.java Changeset: 5eaa495dc929 Author: anthony Date: 2009-05-19 14:14 +0400 URL: http://hg.openjdk.java.net/jdk7/tl/jdk/rev/5eaa495dc929 6812298: Dynamic GraphicsConfig changes don't work on X11 platforms Summary: The peer gets recreated if the visual of the new GC differs from the previous one Reviewed-by: art, dcherepanov ! src/share/classes/java/awt/Component.java ! src/share/classes/java/awt/Container.java ! src/share/classes/java/awt/peer/ComponentPeer.java ! src/share/classes/sun/awt/NullComponentPeer.java ! src/solaris/classes/sun/awt/X11/XComponentPeer.java ! src/solaris/classes/sun/awt/X11/XEmbedChildProxyPeer.java ! src/solaris/classes/sun/awt/X11/XWindow.java ! src/windows/classes/sun/awt/windows/WComponentPeer.java Changeset: ac08fa3d6c98 Author: anthony Date: 2009-05-19 14:43 +0400 URL: http://hg.openjdk.java.net/jdk7/tl/jdk/rev/ac08fa3d6c98 6833444: _BOOTDIR1/_BOOTDIR2 on MS Windows should be consistent with other platforms Summary: Added optional _BOOTDIR3 that provides the J: path for the BOOTDIR on Windows Reviewed-by: ohair, xdono ! make/common/Sanity.gmk ! make/common/shared/Defs-windows.gmk ! make/common/shared/Defs.gmk ! make/common/shared/Sanity.gmk Changeset: 315f315b8d3c Author: anthony Date: 2009-05-19 17:03 +0400 URL: http://hg.openjdk.java.net/jdk7/tl/jdk/rev/315f315b8d3c 6839999: Cumulative fix for 6762511 and 6838003 Summary: Adds support for ARGB and ABGR X11 surfaces. Reviewed-by: art, yan ! src/solaris/classes/sun/awt/X11/generator/sizes.64-solaris-i386 ! src/solaris/classes/sun/awt/X11/generator/xlibtypes.txt ! src/solaris/classes/sun/awt/X11GraphicsConfig.java ! src/solaris/classes/sun/java2d/x11/X11PMBlitBgLoops.java ! src/solaris/classes/sun/java2d/x11/X11PMBlitLoops.java ! src/solaris/classes/sun/java2d/x11/X11SurfaceData.java ! src/solaris/native/sun/awt/X11Color.c ! src/solaris/native/sun/awt/awt_GraphicsEnv.c ! src/solaris/native/sun/awt/awt_p.h Changeset: b33466bb2fed Author: art Date: 2009-05-21 12:29 +0400 URL: http://hg.openjdk.java.net/jdk7/tl/jdk/rev/b33466bb2fed 6794764: Translucent windows are completely repainted on every paint event, on Windows 6719382: Printing of AWT components on windows is not working 6726866: Repainting artifacts when resizing or dragging JInternalFrames in non-opaque toplevel 6683775: Painting artifacts is seen when panel is made setOpaque(false) for a translucent window Reviewed-by: anthony, tdv, alexp ! src/share/classes/java/awt/GraphicsConfiguration.java ! src/share/classes/java/awt/GraphicsDevice.java ! src/share/classes/java/awt/Window.java ! src/share/classes/java/awt/peer/WindowPeer.java ! src/share/classes/javax/swing/DefaultDesktopManager.java ! src/share/classes/javax/swing/JComponent.java ! src/share/classes/javax/swing/RepaintManager.java ! src/share/classes/sun/awt/AWTAccessor.java ! src/share/classes/sun/awt/EmbeddedFrame.java ! src/solaris/classes/sun/awt/X11/XWindowPeer.java ! src/windows/classes/sun/awt/windows/TranslucentWindowPainter.java ! src/windows/classes/sun/awt/windows/WCanvasPeer.java ! src/windows/classes/sun/awt/windows/WComponentPeer.java ! src/windows/classes/sun/awt/windows/WObjectPeer.java ! src/windows/classes/sun/awt/windows/WWindowPeer.java ! src/windows/native/sun/windows/awt_Component.cpp ! src/windows/native/sun/windows/awt_Component.h ! src/windows/native/sun/windows/awt_Window.cpp ! src/windows/native/sun/windows/awt_Window.h + test/javax/swing/JComponent/6683775/bug6683775.java + test/javax/swing/JInternalFrame/6726866/bug6726866.html + test/javax/swing/JInternalFrame/6726866/bug6726866.java Changeset: 97ece6b3d84f Author: ant Date: 2009-05-21 15:04 +0400 URL: http://hg.openjdk.java.net/jdk7/tl/jdk/rev/97ece6b3d84f 6833019: KeyboardFocusManager.getCurrentKeyboardFocusManager() throws unspecified HeadlessException Reviewed-by: art ! src/share/classes/sun/awt/HeadlessToolkit.java Changeset: cfe73335a065 Author: dav Date: 2009-05-22 16:09 +0400 URL: http://hg.openjdk.java.net/jdk7/tl/jdk/rev/cfe73335a065 6799099: All automatic regression tests that create Robot fail on X11 Reviewed-by: art, ant ! make/sun/xawt/mapfile-vers ! src/share/classes/java/awt/Robot.java ! src/share/classes/java/awt/event/InputEvent.java ! src/share/classes/java/awt/event/MouseEvent.java ! src/share/classes/java/awt/peer/RobotPeer.java ! src/share/classes/sun/awt/SunToolkit.java ! src/solaris/classes/sun/awt/X11/XBaseWindow.java ! src/solaris/classes/sun/awt/X11/XDragSourceContextPeer.java ! src/solaris/classes/sun/awt/X11/XRobotPeer.java ! src/solaris/classes/sun/awt/X11/XToolkit.java ! src/solaris/classes/sun/awt/X11/XWindow.java ! src/solaris/classes/sun/awt/X11/XWindowPeer.java ! src/solaris/classes/sun/awt/motif/MToolkit.java ! src/solaris/native/sun/awt/awt_MToolkit.c ! src/solaris/native/sun/awt/awt_Robot.c ! src/solaris/native/sun/xawt/XToolkit.c ! src/windows/classes/sun/awt/windows/WRobotPeer.java ! src/windows/classes/sun/awt/windows/WToolkit.java ! src/windows/native/sun/windows/awt_Robot.cpp ! src/windows/native/sun/windows/awt_Toolkit.cpp Changeset: 52493efeb137 Author: dav Date: 2009-05-25 18:22 +0400 URL: http://hg.openjdk.java.net/jdk7/tl/jdk/rev/52493efeb137 6844750: Solaris build failed after 6799099 Reviewed-by: yan ! src/solaris/native/sun/xawt/XToolkit.c Changeset: 7da360c3baf6 Author: yan Date: 2009-06-01 01:05 -0700 URL: http://hg.openjdk.java.net/jdk7/tl/jdk/rev/7da360c3baf6 Merge Changeset: f29cd35647b1 Author: peytoia Date: 2009-05-12 15:21 +0900 URL: http://hg.openjdk.java.net/jdk7/tl/jdk/rev/f29cd35647b1 6834474: (tz) Support tzdata2009g Reviewed-by: okutsu ! make/sun/javazic/tzdata/VERSION ! make/sun/javazic/tzdata/africa ! make/sun/javazic/tzdata/asia ! make/sun/javazic/tzdata/leapseconds ! make/sun/javazic/tzdata/northamerica ! make/sun/javazic/tzdata/southamerica ! src/share/classes/sun/util/resources/TimeZoneNames.java ! src/share/classes/sun/util/resources/TimeZoneNames_de.java ! src/share/classes/sun/util/resources/TimeZoneNames_es.java ! src/share/classes/sun/util/resources/TimeZoneNames_fr.java ! src/share/classes/sun/util/resources/TimeZoneNames_it.java ! src/share/classes/sun/util/resources/TimeZoneNames_ja.java ! src/share/classes/sun/util/resources/TimeZoneNames_ko.java ! src/share/classes/sun/util/resources/TimeZoneNames_sv.java ! src/share/classes/sun/util/resources/TimeZoneNames_zh_CN.java ! src/share/classes/sun/util/resources/TimeZoneNames_zh_TW.java Changeset: 62bfe2674e48 Author: yan Date: 2009-05-14 00:17 -0700 URL: http://hg.openjdk.java.net/jdk7/tl/jdk/rev/62bfe2674e48 Merge - src/share/native/sun/java2d/pipe/RenderBuffer.c Changeset: 455b357442c7 Author: peterz Date: 2009-05-14 18:12 +0400 URL: http://hg.openjdk.java.net/jdk7/tl/jdk/rev/455b357442c7 6741426: ClassCastException from ComboBoxEditableState (Nimbus LaF) in JDK 1.6.0_10 RC Reviewed-by: rupashka ! src/share/classes/javax/swing/plaf/nimbus/skin.laf + test/javax/swing/plaf/nimbus/Test6741426.java Changeset: af491a9b7c1d Author: peterz Date: 2009-05-15 12:06 +0400 URL: http://hg.openjdk.java.net/jdk7/tl/jdk/rev/af491a9b7c1d 6827581: Contextkey does not work in Nimbus Reviewed-by: rupashka ! src/share/classes/sun/swing/plaf/GTKKeybindings.java ! src/share/classes/sun/swing/plaf/WindowsKeybindings.java Changeset: 993a5f0fe2e0 Author: rupashka Date: 2009-05-15 17:26 +0400 URL: http://hg.openjdk.java.net/jdk7/tl/jdk/rev/993a5f0fe2e0 6713352: Deadlock in JFileChooser with synchronized custom FileSystemView Reviewed-by: malenkov, peterz ! src/share/classes/javax/swing/plaf/basic/BasicDirectoryModel.java ! src/share/classes/sun/awt/shell/ShellFolder.java ! src/windows/classes/sun/awt/shell/Win32ShellFolder2.java ! src/windows/classes/sun/awt/shell/Win32ShellFolderManager2.java + test/javax/swing/JFileChooser/6713352/bug6713352.java Changeset: 019908df0313 Author: rupashka Date: 2009-05-28 18:11 +0400 URL: http://hg.openjdk.java.net/jdk7/tl/jdk/rev/019908df0313 6845805: Test for CR 6713352 is failed under Linux Reviewed-by: malenkov ! test/javax/swing/JFileChooser/6713352/bug6713352.java Changeset: 951ecbad4068 Author: yan Date: 2009-06-01 01:06 -0700 URL: http://hg.openjdk.java.net/jdk7/tl/jdk/rev/951ecbad4068 Merge Changeset: 0c3ef2d612a4 Author: yan Date: 2009-06-09 23:47 -0700 URL: http://hg.openjdk.java.net/jdk7/tl/jdk/rev/0c3ef2d612a4 Merge ! make/common/shared/Defs-windows.gmk ! make/common/shared/Sanity.gmk ! src/windows/native/sun/windows/awt_Window.cpp Changeset: f72c0dc047b9 Author: xdono Date: 2009-06-11 10:54 -0700 URL: http://hg.openjdk.java.net/jdk7/tl/jdk/rev/f72c0dc047b9 Added tag jdk7-b60 for changeset 0c3ef2d612a4 ! .hgtags Changeset: 328148f45b31 Author: tbell Date: 2009-06-11 21:32 -0700 URL: http://hg.openjdk.java.net/jdk7/tl/jdk/rev/328148f45b31 Merge ! make/docs/Makefile - src/share/classes/sun/nio/cs/ext/DBCSDecoderMapping.java - src/share/classes/sun/nio/cs/ext/DBCS_IBM_ASCII_Decoder.java - src/share/classes/sun/nio/cs/ext/DBCS_IBM_ASCII_Encoder.java - src/share/classes/sun/nio/cs/ext/DBCS_IBM_EBCDIC_Decoder.java - src/share/classes/sun/nio/cs/ext/DBCS_IBM_EBCDIC_Encoder.java - src/share/classes/sun/nio/cs/ext/DBCS_ONLY_IBM_EBCDIC_Decoder.java - src/share/classes/sun/nio/cs/ext/IBM1381.java - src/share/classes/sun/nio/cs/ext/IBM1383.java - src/share/classes/sun/nio/cs/ext/IBM930.java - src/share/classes/sun/nio/cs/ext/IBM933.java - src/share/classes/sun/nio/cs/ext/IBM935.java - src/share/classes/sun/nio/cs/ext/IBM937.java - src/share/classes/sun/nio/cs/ext/IBM939.java - src/share/classes/sun/nio/cs/ext/IBM942.java - src/share/classes/sun/nio/cs/ext/IBM943.java - src/share/classes/sun/nio/cs/ext/IBM948.java - src/share/classes/sun/nio/cs/ext/IBM949.java - src/share/classes/sun/nio/cs/ext/IBM950.java - src/share/classes/sun/nio/cs/ext/IBM970.java - src/share/classes/sun/nio/cs/ext/SimpleEUCDecoder.java ! test/Makefile From tim.bell at sun.com Fri Jun 12 06:05:18 2009 From: tim.bell at sun.com (tim.bell at sun.com) Date: Fri, 12 Jun 2009 06:05:18 +0000 Subject: hg: jdk7/tl/langtools: 9 new changesets Message-ID: <20090612060533.AC5F4E46C@hg.openjdk.java.net> Changeset: 4b72c2556789 Author: aph Date: 2009-04-17 15:56 +0100 URL: http://hg.openjdk.java.net/jdk7/tl/langtools/rev/4b72c2556789 6829575: 100028: Debug information is incomplete or missing Summary: Enable debugging in many places Reviewed-by: ohair Contributed-by: Andrew Haley ! make/Makefile Changeset: 321854d9ab19 Author: aph Date: 2009-04-20 19:01 +0100 URL: http://hg.openjdk.java.net/jdk7/tl/langtools/rev/321854d9ab19 6832141: Bug 100045 - Fix for 100028 breaks debug info for class files Summary: Correct fallout from 100028 patch Reviewed-by: ohair Contributed-by: Andrew Haley ! make/Makefile Changeset: f3d27f02683c Author: aph Date: 2009-05-06 18:04 +0100 URL: http://hg.openjdk.java.net/jdk7/tl/langtools/rev/f3d27f02683c 6837665: Deal with windows ant problem where commas in -D options do not work Summary: Rewrite to avoid commas in -D options Reviewed-by: ohair ! make/Makefile ! make/build.xml Changeset: 43a781cc6473 Author: xdono Date: 2009-05-07 10:32 -0700 URL: http://hg.openjdk.java.net/jdk7/tl/langtools/rev/43a781cc6473 Merge Changeset: 846944dd48a4 Author: xdono Date: 2009-05-07 12:26 -0700 URL: http://hg.openjdk.java.net/jdk7/tl/langtools/rev/846944dd48a4 Merge Changeset: 65f2ee956fb9 Author: ohair Date: 2009-05-15 13:30 -0700 URL: http://hg.openjdk.java.net/jdk7/tl/langtools/rev/65f2ee956fb9 Merge Changeset: 5cdce469ea2a Author: ohair Date: 2009-05-19 17:39 -0700 URL: http://hg.openjdk.java.net/jdk7/tl/langtools/rev/5cdce469ea2a 6843041: Remove duplicate README files in repositories (make/README) Reviewed-by: robilad ! README - make/README Changeset: 522520757dd3 Author: xdono Date: 2009-06-11 10:54 -0700 URL: http://hg.openjdk.java.net/jdk7/tl/langtools/rev/522520757dd3 Added tag jdk7-b60 for changeset 5cdce469ea2a ! .hgtags Changeset: 163f5d75f77a Author: tbell Date: 2009-06-11 21:35 -0700 URL: http://hg.openjdk.java.net/jdk7/tl/langtools/rev/163f5d75f77a Merge ! make/Makefile ! make/build.xml - src/share/classes/sun/tools/javap/AttrData.java - src/share/classes/sun/tools/javap/CPX.java - src/share/classes/sun/tools/javap/CPX2.java - src/share/classes/sun/tools/javap/ClassData.java - src/share/classes/sun/tools/javap/Constants.java - src/share/classes/sun/tools/javap/FieldData.java - src/share/classes/sun/tools/javap/InnerClassData.java - src/share/classes/sun/tools/javap/JavapEnvironment.java - src/share/classes/sun/tools/javap/JavapPrinter.java - src/share/classes/sun/tools/javap/LineNumData.java - src/share/classes/sun/tools/javap/LocVarData.java - src/share/classes/sun/tools/javap/Main.java - src/share/classes/sun/tools/javap/MethodData.java - src/share/classes/sun/tools/javap/RuntimeConstants.java - src/share/classes/sun/tools/javap/StackMapData.java - src/share/classes/sun/tools/javap/StackMapTableData.java - src/share/classes/sun/tools/javap/Tables.java - src/share/classes/sun/tools/javap/TrapData.java - src/share/classes/sun/tools/javap/TypeSignature.java - test/tools/javac/code/ArrayClone.sh - test/tools/javap/ListTest.java - test/tools/javap/OptionTest.java From dl at cs.oswego.edu Fri Jun 12 14:23:58 2009 From: dl at cs.oswego.edu (Doug Lea) Date: Fri, 12 Jun 2009 10:23:58 -0400 Subject: HashMap - Hybrid approach In-Reply-To: References: <4A2FAE34.3080902@cs.oswego.edu> <4A311A54.1070602@cs.oswego.edu> Message-ID: <4A3264FE.9020604@cs.oswego.edu> Alex Yakovlev wrote: >> This suggests a >> different sort of tactic for helping with indirections and >> prefetches. Suppose that index[h] was usually just h; that is, >> with the key in elements[2*h]. > > I did a little test of this and saw no performance gain, > I even posted a link to sources right after changing subject > from 'fast' to 'hybrid'. > > Do you have any idea why second array prefetching may not happen? This was fun to figure out. Performance on some of our tests involving get() is sensitive to the serial correlation of slot indices for the keys used across successive calls. The current bit-spreading function is biased to improve locality without hurting collisions too much for numerically nearby hashCodes. So on tests where this holds, prefetching buys very little since cache misses are already relatively low. I'll check in some tests I threw together that help diagnose this after some cleanup. (One reason for having several tests that have this bias is that they are good indicators of performance on other artificial benchmarks like specjbb.) As another quick experiment, I just tried a hacked version of IdentityHashMap carrying hashes array, and did see around a 20% improvement in some tests for get() versus version with index-dependence. So some variant of direct access still seems likely to be worth pursuing. I'll try to put together a more serious version along the lines of my suggestions, although maybe not for a few days. -Doug From mlists at juma.me.uk Sat Jun 13 11:00:39 2009 From: mlists at juma.me.uk (Ismael Juma) Date: Sat, 13 Jun 2009 11:00:39 +0000 (UTC) Subject: Faster HashMap implementation References: <4A2D3729.3040904@cs.oswego.edu> <82ocsyitzc.fsf@mid.bfk.de> <4A2D3D8E.60300@cs.oswego.edu> Message-ID: Hi Doug, Doug Lea
writes: > While open-addressing is used in > IdentityHashMap (and in a very specialized form in > ThreadLocal), you cannot live with linear-probed > versions otherwise: Many user-defined hashCodes > (and some JDK-defined ones too!) are not very good. Out of curiosity, do you know if any tests have been done with an open addressing scheme similar to Python dictionaries[1] in Java? Near the top, there's a comment explaining how it works and the motivation. Best, Ismael [1] http://svn.python.org/view/python/trunk/Objects/dictobject.c?revision=73196&view=markup From dl at cs.oswego.edu Sat Jun 13 12:30:55 2009 From: dl at cs.oswego.edu (Doug Lea) Date: Sat, 13 Jun 2009 08:30:55 -0400 Subject: Faster HashMap implementation In-Reply-To: References: <4A2D3729.3040904@cs.oswego.edu> <82ocsyitzc.fsf@mid.bfk.de> <4A2D3D8E.60300@cs.oswego.edu> Message-ID: <4A339BFF.6010809@cs.oswego.edu> Ismael Juma wrote: > Out of curiosity, do you know if any tests have been done with an open > addressing scheme similar to Python dictionaries[1] in Java? Near the top, > there's a comment explaining how it works and the motivation. > More or less. This is roughly similar to the schemes I mentioned a few days ago, of first direct-addressing. The python approach (as noted in its comments) has most trouble with cases that occur too often in Java to ignore: Poor entropy in low bits (the hashCodes for commonly used Floats and Doubles are especially problematic (Aside: you wouldn't think these would ever be used keys, but they are)) and much-greater-than-expected duplicate codes (mainly the particular value zero). However, these do seem amenable to one of the hybrid approaches I mentioned; I'll try to get a full version together soon. -Doug From rdifalco at tripwire.com Sat Jun 13 16:45:55 2009 From: rdifalco at tripwire.com (Robert DiFalco) Date: Sat, 13 Jun 2009 09:45:55 -0700 Subject: Faster HashMap implementation In-Reply-To: <4A339BFF.6010809@cs.oswego.edu> References: <4A2D3729.3040904@cs.oswego.edu> <82ocsyitzc.fsf@mid.bfk.de><4A2D3D8E.60300@cs.oswego.edu> <4A339BFF.6010809@cs.oswego.edu> Message-ID: <1C21C32F5F592D48B16F28C7877D9BC10370E23A@SVMAIL01.tripwire.com> Sorry for inserting myself into this thread but this has been an interesting conversation filled with a lot of trade-offs and optimization techniques for edge cases. It occurred to me that one of the problems is that the traditional HashMap implementation relies on inheritance rather than composition. For example, if the hash map representation where abstracted out from the interface then you could by default construct a HashMap using the existing approach but alternatively construct it with another approach. If you further abstracted the hashing strategy into its own class then you would have even more flexibility composing a HashMap. Since maps are so often resized you could even have a dynamic map that starts out with open addressing but keeps count of collisions and on resize changes to a more optimal scheme for the current data. This allows you to have a decent default HashMap but allows clients to have an optimal hash map if they know they will allows have Long or String or whatever keys versus doubles, floats, etc. With the hashing strategy, an identity map becomes as easy as mixing in an identity hashing strategy during composition. Anyway, just a thought, I'm sure it's been considered before but I just thought I'd throw it out there. -----Original Message----- From: core-libs-dev-bounces at openjdk.java.net [mailto:core-libs-dev-bounces at openjdk.java.net] On Behalf Of Doug Lea Sent: Saturday, June 13, 2009 5:31 AM To: Ismael Juma Cc: core-libs-dev at openjdk.java.net Subject: Re: Faster HashMap implementation Ismael Juma wrote: > Out of curiosity, do you know if any tests have been done with an open > addressing scheme similar to Python dictionaries[1] in Java? Near the top, > there's a comment explaining how it works and the motivation. > More or less. This is roughly similar to the schemes I mentioned a few days ago, of first direct-addressing. The python approach (as noted in its comments) has most trouble with cases that occur too often in Java to ignore: Poor entropy in low bits (the hashCodes for commonly used Floats and Doubles are especially problematic (Aside: you wouldn't think these would ever be used keys, but they are)) and much-greater-than-expected duplicate codes (mainly the particular value zero). However, these do seem amenable to one of the hybrid approaches I mentioned; I'll try to get a full version together soon. -Doug From alex14n at gmail.com Sat Jun 13 23:47:52 2009 From: alex14n at gmail.com (Alex Yakovlev) Date: Sun, 14 Jun 2009 03:47:52 +0400 Subject: HashMap - Hybrid approach In-Reply-To: <4A3264FE.9020604@cs.oswego.edu> References: <4A2FAE34.3080902@cs.oswego.edu> <4A311A54.1070602@cs.oswego.edu> <4A3264FE.9020604@cs.oswego.edu> Message-ID: Doug, On Fri, Jun 12, 2009 at 6:23 PM, Doug Lea
wrote: > This was fun to figure out. Performance on some of our > tests involving get() is sensitive to the serial > correlation of slot indices for the keys used across > successive calls. Are there any tests that have get() order the same as put() order? Data in my map is stored almost in insertion order :) > As another quick experiment, I just tried a hacked > version of IdentityHashMap carrying hashes > array, and did see around a 20% improvement in some > tests for get() versus version with index-dependence. > So some variant of direct access still seems likely to > be worth pursuing. Sure. But checking stored hashcode always have limited time, and user's equals method is unpredictable and can be slow. We could accumulate nanoseconds statistics for equals calls and choose either to check hashcode first or try direct key.equals, don't know. In my observations it depends on locality of user's data, if equals operates only local properties - direct equals is faster, and even String with it's remove char[] is already slower than checking saved hashcode first. Another way is to check couple of keys contents with reflection, if all contents are just primitive fields - it's save to call direct equals, if not - check hashcode first. But user can access some static classes, have some heave computations, etc. > I'll try to put together a more serious version along > the lines of my suggestions, although maybe not for > a few days. Waiting with a lot of interest. Meanwhile I've tried an open map approach with my smart bits, like marking 'end-of-list', putting first hash bin element always on it's place, etc. In some tests it gives significant performance improvement due to sequental access in all arrays - max 2 cache misses in get. It certainly needs another bit distribution function and load factor management. But on some data it gets piled up very heavilly with awful performance. Maybe this can solved with some bit flags, not only 'end of list' but also choosing between +1 step and some large step - like those suggested by Ismael (in python). Or even a bit flag to look into some external overflow storage. Now I've implemented a bit of this approach in my map - when next element in hash table is empty it is used by its neighbours for faster access. It gives some improvement on data with bad hashcodes distribution (when there are many second lookups). But source code becomes more and more complex and I'm afraid too much conditions and branches could harm performance. Alex From martinrb at google.com Sun Jun 14 17:08:48 2009 From: martinrb at google.com (Martin Buchholz) Date: Sun, 14 Jun 2009 10:08:48 -0700 Subject: Make every java process 20 bytes smaller Message-ID: <1ccfd1c10906141008h7f3bf585y91c615dbf2d67c8e@mail.gmail.com> Hello Kumar, I'd like you to do a code review and file a bug. Synopsis: Fix small memory leak in launcher Description: This little change to the launcher - fixes a leak of 20 bytes in *every* java process - fixes the leak of classpath wildcard expansion when that feature is used. diff --git a/src/share/bin/java.c b/src/share/bin/java.c --- a/src/share/bin/java.c +++ b/src/share/bin/java.c @@ -683,10 +683,14 @@ SetClassPath(const char *s) { char *def; + const char *orig = s; + static const char format[] = "-Djava.class.path=%s"; s = JLI_WildcardExpandClasspath(s); - def = JLI_MemAlloc(JLI_StrLen(s) + 40); - sprintf(def, "-Djava.class.path=%s", s); + def = JLI_MemAlloc(sizeof(format) - 2 + JLI_StrLen(s)); + sprintf(def, format, s); AddOption(def, NULL); + if (s != orig) + JLI_MemFree((char *) s); } /* This code is particularly prone to off-by-one bugs, but I think we can get it right. Webrev at: http://cr.openjdk.java.net/~martin/launcher-leak/ Is Kowalski still around? He may have been the last person to touch this. Martin -------------- next part -------------- An HTML attachment was scrubbed... URL: From martinrb at google.com Sun Jun 14 21:47:44 2009 From: martinrb at google.com (martinrb at google.com) Date: Sun, 14 Jun 2009 21:47:44 +0000 Subject: hg: jdk7/tl/jdk: 2 new changesets Message-ID: <20090614214812.84C18E5F4@hg.openjdk.java.net> Changeset: 74aefd0ab26d Author: martin Date: 2009-06-14 14:23 -0700 URL: http://hg.openjdk.java.net/jdk7/tl/jdk/rev/74aefd0ab26d 6850720: (process) Use clone(CLONE_VM), not fork, on Linux to avoid swap exhaustion Summary: Use clone(CLONE_VM) on Linux; Reluctantly implement execvpe. Reviewed-by: michaelm ! src/solaris/native/java/lang/UNIXProcess_md.c ! test/java/lang/ProcessBuilder/Basic.java + test/java/lang/ProcessBuilder/BigFork.java Changeset: d0de3e41426b Author: martin Date: 2009-06-14 14:33 -0700 URL: http://hg.openjdk.java.net/jdk7/tl/jdk/rev/d0de3e41426b 6511515: poor performance of LogRecord.inferCaller depending on java.lang.Throwable.getStackTraceElement Summary: Allow random access to stack trace elements; retrieve only needed ones Reviewed-by: swamyv Contributed-by: jeremymanson at google.com ! src/share/classes/java/lang/System.java ! src/share/classes/java/lang/Throwable.java ! src/share/classes/java/util/logging/LogRecord.java ! src/share/classes/sun/misc/JavaLangAccess.java From maurizio.cimadamore at sun.com Tue Jun 16 09:58:19 2009 From: maurizio.cimadamore at sun.com (maurizio.cimadamore at sun.com) Date: Tue, 16 Jun 2009 09:58:19 +0000 Subject: hg: jdk7/tl/langtools: 4 new changesets Message-ID: <20090616095826.219E3E747@hg.openjdk.java.net> Changeset: a9c04a57a39f Author: mcimadamore Date: 2009-06-16 10:45 +0100 URL: http://hg.openjdk.java.net/jdk7/tl/langtools/rev/a9c04a57a39f 6845686: basic and raw formatters do not display captured var id properly when javac runs in -XDoldDiags mode Summary: Basic and raw formatters do not override Printer methods properly Reviewed-by: jjg ! src/share/classes/com/sun/tools/javac/util/AbstractDiagnosticFormatter.java ! test/tools/javac/Diagnostics/6799605/T6799605.java ! test/tools/javac/Diagnostics/6799605/T6799605.out Changeset: 3d539f4123b8 Author: mcimadamore Date: 2009-06-16 10:45 +0100 URL: http://hg.openjdk.java.net/jdk7/tl/langtools/rev/3d539f4123b8 6835430: javac does not generate signature attributes for classes extending parameterized inner classes Summary: ClassWriter does not consider outer params of an inner class when emitting signature attributes Reviewed-by: jjg ! src/share/classes/com/sun/tools/javac/jvm/ClassWriter.java + test/tools/javac/6835430/A.java + test/tools/javac/6835430/T6835430.java Changeset: 3ac205ad1f05 Author: mcimadamore Date: 2009-06-16 10:46 +0100 URL: http://hg.openjdk.java.net/jdk7/tl/langtools/rev/3ac205ad1f05 6835428: regression: return-type inference rejects valid code Summary: Redundant subtyping test during type-inference ends up in rejecting legal code Reviewed-by: jjg ! src/share/classes/com/sun/tools/javac/comp/Infer.java + test/tools/javac/generics/inference/T6835428.java Changeset: 22872b24d38c Author: mcimadamore Date: 2009-06-16 10:46 +0100 URL: http://hg.openjdk.java.net/jdk7/tl/langtools/rev/22872b24d38c 6638712: Inference with wildcard types causes selection of inapplicable method Summary: Added global sanity check in order to make sure that return type inference does not violate bounds constraints Reviewed-by: jjg ! src/share/classes/com/sun/tools/javac/code/Type.java ! src/share/classes/com/sun/tools/javac/code/Types.java ! src/share/classes/com/sun/tools/javac/comp/Check.java ! src/share/classes/com/sun/tools/javac/comp/Infer.java ! src/share/classes/com/sun/tools/javac/comp/Resolve.java ! src/share/classes/com/sun/tools/javac/resources/compiler.properties ! test/tools/javac/generics/inference/6302954/T6476073.java + test/tools/javac/generics/inference/6638712/T6638712a.java + test/tools/javac/generics/inference/6638712/T6638712a.out + test/tools/javac/generics/inference/6638712/T6638712b.java + test/tools/javac/generics/inference/6638712/T6638712b.out + test/tools/javac/generics/inference/6638712/T6638712c.java + test/tools/javac/generics/inference/6638712/T6638712c.out + test/tools/javac/generics/inference/6638712/T6638712d.java + test/tools/javac/generics/inference/6638712/T6638712d.out + test/tools/javac/generics/inference/6638712/T6638712e.java + test/tools/javac/generics/inference/6638712/T6638712e.out From xuelei.fan at sun.com Tue Jun 16 12:57:53 2009 From: xuelei.fan at sun.com (xuelei.fan at sun.com) Date: Tue, 16 Jun 2009 12:57:53 +0000 Subject: hg: jdk7/tl/jdk: 6850783: InvalidityDateExtension returns reference to internal mutable state Message-ID: <20090616125831.8D141E764@hg.openjdk.java.net> Changeset: 1299804aa6e7 Author: xuelei Date: 2009-06-16 20:46 +0800 URL: http://hg.openjdk.java.net/jdk7/tl/jdk/rev/1299804aa6e7 6850783: InvalidityDateExtension returns reference to internal mutable state Summary: return cloned instead of referenced object Reviewed-by: weijun ! src/share/classes/sun/security/x509/CertificateVersion.java ! src/share/classes/sun/security/x509/InvalidityDateExtension.java From langel at redhat.com Tue Jun 16 15:58:08 2009 From: langel at redhat.com (Lillian Angel) Date: Tue, 16 Jun 2009 11:58:08 -0400 Subject: MessageUtils JVM crash Message-ID: <4A37C110.3060909@redhat.com> Hi, I opened a bug report about a JVM crash. Test case and patch are attached. https://bugs.openjdk.java.net/show_bug.cgi?id=100074 Cheers, Lillian From Alan.Bateman at Sun.COM Tue Jun 16 16:09:23 2009 From: Alan.Bateman at Sun.COM (Alan Bateman) Date: Tue, 16 Jun 2009 17:09:23 +0100 Subject: MessageUtils JVM crash In-Reply-To: <4A37C110.3060909@redhat.com> References: <4A37C110.3060909@redhat.com> Message-ID: <4A37C3B3.2030902@sun.com> Lillian Angel wrote: > Hi, > > I opened a bug report about a JVM crash. Test case and patch are > attached. > > https://bugs.openjdk.java.net/show_bug.cgi?id=100074 > > > Cheers, > Lillian Out of curiosity, how did you run into this? Just wondering if there is somewhere in the JDK that does call it with null (I see the test case is calling sun.misc.MessageUtil directly, somewhere that applications should never do). -Alan. From langel at redhat.com Tue Jun 16 16:12:00 2009 From: langel at redhat.com (Lillian Angel) Date: Tue, 16 Jun 2009 12:12:00 -0400 Subject: MessageUtils JVM crash In-Reply-To: <4A37C3B3.2030902@sun.com> References: <4A37C110.3060909@redhat.com> <4A37C3B3.2030902@sun.com> Message-ID: <4A37C450.7010700@redhat.com> Alan Bateman wrote: > Lillian Angel wrote: >> Hi, >> >> I opened a bug report about a JVM crash. Test case and patch are >> attached. >> >> https://bugs.openjdk.java.net/show_bug.cgi?id=100074 >> >> >> Cheers, >> Lillian > Out of curiosity, how did you run into this? Just wondering if there > is somewhere in the JDK that does call it with null (I see the test > case is calling sun.misc.MessageUtil directly, somewhere that > applications should never do). I have CC'ed Marc Schoenfeld, he initially ran into this problem. Lillian From mschoene at redhat.com Tue Jun 16 16:46:33 2009 From: mschoene at redhat.com (Marc Schoenefeld) Date: Tue, 16 Jun 2009 18:46:33 +0200 Subject: MessageUtils JVM crash In-Reply-To: <4A37C450.7010700@redhat.com> References: <4A37C110.3060909@redhat.com> <4A37C3B3.2030902@sun.com> <4A37C450.7010700@redhat.com> Message-ID: <4A37CC69.3080509@redhat.com> Hi, originally I wrote a fuzzing tool to test all native functions in jdk131 , then gave a list of the results to the Sun representatives at RSA conference 2003. Unfortunately I never received any reaction to this bug report, nor were the bugs fixed. So I put the bugs in a drawer, but used the chance to write a fix for OpenJDK. Setting the parameter to null could allow an attacker to conduct denial of service attacks: - http://www.blackhat.com/presentations/win-usa-03/bh-win-03-schoenfeld.pdf or - http://seclists.org/bugtraq/2003/Sep/0270.html Cheers Marc Lillian Angel wrote: > Alan Bateman wrote: >> Lillian Angel wrote: >>> Hi, >>> >>> I opened a bug report about a JVM crash. Test case and patch are >>> attached. >>> >>> https://bugs.openjdk.java.net/show_bug.cgi?id=100074 >>> >>> >>> Cheers, >>> Lillian >> Out of curiosity, how did you run into this? Just wondering if there >> is somewhere in the JDK that does call it with null (I see the test >> case is calling sun.misc.MessageUtil directly, somewhere that >> applications should never do). > > > I have CC'ed Marc Schoenfeld, he initially ran into this problem. > > > Lillian -- Marc Schoenefeld / Red Hat Security Response Team From Xueming.Shen at Sun.COM Wed Jun 17 02:50:37 2009 From: Xueming.Shen at Sun.COM (Xueming Shen) Date: Tue, 16 Jun 2009 19:50:37 -0700 Subject: Fastpath for new String(bytes..) and String#getBytes(..) for ASCII + ISO-8859-1 In-Reply-To: <4A30FD5C.3040902@gmx.de> References: <4A30FD5C.3040902@gmx.de> Message-ID: <4A3859FD.9020300@sun.com> I would be very hesitated to add methods with specific encoding names, even these encodings are VERY important, such as ASCII and ISO8859. Hack a fast path in the implementation to boost the performance for some important encodings is something we want to do, but add specific methods into the API is totally different thing. sherman Ulf Zibis wrote: > Hi Sherman, may be you are interested in this bug: > > http://bugs.sun.com/bugdatabase/view_bug.do?bug_id=6826329 > > -Ulf > > From weijun.wang at sun.com Wed Jun 17 07:37:54 2009 From: weijun.wang at sun.com (weijun.wang at sun.com) Date: Wed, 17 Jun 2009 07:37:54 +0000 Subject: hg: jdk7/tl/jdk: 6849275: enhance krb5 reg tests Message-ID: <20090617073821.8C691E823@hg.openjdk.java.net> Changeset: bc2c9dbdcc70 Author: weijun Date: 2009-06-17 15:26 +0800 URL: http://hg.openjdk.java.net/jdk7/tl/jdk/rev/bc2c9dbdcc70 6849275: enhance krb5 reg tests Reviewed-by: xuelei ! test/sun/security/krb5/auto/CrossRealm.java ! test/sun/security/krb5/auto/HttpNegotiateServer.java ! test/sun/security/krb5/auto/KDC.java ! test/sun/security/krb5/auto/META-INF/services/sun.net.spi.nameservice.NameServiceDescriptor ! test/sun/security/krb5/auto/OneKDC.java ! test/sun/security/krb5/auto/basic.sh From Alan.Bateman at Sun.COM Wed Jun 17 08:11:28 2009 From: Alan.Bateman at Sun.COM (Alan Bateman) Date: Wed, 17 Jun 2009 09:11:28 +0100 Subject: MessageUtils JVM crash In-Reply-To: <4A37CC69.3080509@redhat.com> References: <4A37C110.3060909@redhat.com> <4A37C3B3.2030902@sun.com> <4A37C450.7010700@redhat.com> <4A37CC69.3080509@redhat.com> Message-ID: <4A38A530.6060607@sun.com> Marc Schoenefeld wrote: > Hi, > > originally I wrote a fuzzing tool to test all native functions in jdk131 > , then gave a list of the results to the Sun representatives at RSA > conference 2003. > Unfortunately I never received any reaction to this bug report, nor were > the bugs fixed. So I put the bugs in a drawer, but used the chance to > write a fix > for OpenJDK. > > Setting the parameter to null could allow an attacker to conduct denial > of service attacks: > - > http://www.blackhat.com/presentations/win-usa-03/bh-win-03-schoenfeld.pdf > or > - http://seclists.org/bugtraq/2003/Sep/0270.html > I wasn't at the RSA conference in 2003 so it wasn't me :-) It may be that the attacks involved calling sun.* APIs directly, something that you can't do if there is a security manager. The XSLT issue is more significant and I'm pretty sure that specific issue was fixed a few years ago. As regards sun.misc.MessageUtils, I don't see any problem fixing this. I notice the return from NewStringUTF isn't checked. Unfortunately this (very old) code is also missing checks for the calls to GetStringChars and malloc. Also, I assume that the additional \0 isn't needed. -Alan. From Ulf.Zibis at gmx.de Wed Jun 17 12:28:56 2009 From: Ulf.Zibis at gmx.de (Ulf Zibis) Date: Wed, 17 Jun 2009 14:28:56 +0200 Subject: Fastpath for new String(bytes..) and String#getBytes(..) for ASCII + ISO-8859-1 In-Reply-To: <4A3859FD.9020300@sun.com> References: <4A30FD5C.3040902@gmx.de> <4A3859FD.9020300@sun.com> Message-ID: <4A38E188.4000103@gmx.de> Sherman, thanks for your answer. Do you have overseen, that String#getASCIIBytes(..) and String#getISO8859_1Bytes(..) was only mentioned as *alternative*? As I agree, adding methods with specific encoding names to API is bad design, I would prefer method signatures like getBytes(byte[] buf, byte mask). This additionally allows 6-bit ASCII de/encoding. We could additionally provide static constants: MASK_ISO_8859_1, MASK_ASCII, MASK_ASCII_6BIT. Imagine, you have very short Strings. Loading/instantiating charset + de/encoder instances (by name) takes more time than the de/encoding itself in current implementation. So there should be additional big performance gain e.g. for CORBA applications. -Ulf Am 17.06.2009 04:50, Xueming Shen schrieb: > I would be very hesitated to add methods with specific encoding names, > even these encodings are > VERY important, such as ASCII and ISO8859. Hack a fast path in the > implementation to boost the > performance for some important encodings is something we want to do, > but add specific methods > into the API is totally different thing. > > sherman > > Ulf Zibis wrote: >> Hi Sherman, may be you are interested in this bug: >> >> http://bugs.sun.com/bugdatabase/view_bug.do?bug_id=6826329 >> >> -Ulf >> >> > > From Thomas.Hawtin at Sun.COM Wed Jun 17 13:06:47 2009 From: Thomas.Hawtin at Sun.COM (Tom Hawtin) Date: Wed, 17 Jun 2009 14:06:47 +0100 Subject: Fastpath for new String(bytes..) and String#getBytes(..) for ASCII + ISO-8859-1 In-Reply-To: <4A3859FD.9020300@sun.com> References: <4A30FD5C.3040902@gmx.de> <4A3859FD.9020300@sun.com> Message-ID: <4A38EA67.2000608@sun.com> Xueming Shen wrote: > I would be very hesitated to add methods with specific encoding names, > even these encodings are > VERY important, such as ASCII and ISO8859. Hack a fast path in the > implementation to boost the > performance for some important encodings is something we want to do, but > add specific methods > into the API is totally different thing. In my opinion, it makes more sense to add methods to Charset than to String - the latter is easily the more fundamental type. Indeed there are already convenience methods encode/decode for ByteBuffer. Trusted Charset implementations could override and use SharedSecrets or rely upon inlining for fast access to String character data. Tom From mschoene at redhat.com Wed Jun 17 08:40:09 2009 From: mschoene at redhat.com (Marc Schoenefeld) Date: Wed, 17 Jun 2009 10:40:09 +0200 Subject: MessageUtils JVM crash In-Reply-To: <4A38A530.6060607@sun.com> References: <4A37C110.3060909@redhat.com> <4A37C3B3.2030902@sun.com> <4A37C450.7010700@redhat.com> <4A37CC69.3080509@redhat.com> <4A38A530.6060607@sun.com> Message-ID: <4A38ABE9.7090907@redhat.com> Hi Alan, Alan Bateman wrote: > I wasn't at the RSA conference in 2003 so it wasn't me :-) It may > be that the attacks involved calling sun.* APIs directly, something > that you can't do if there is a security manager. The XSLT issue is > more significant and I'm pretty sure that specific issue was fixed a > few years ago. Even if there is a security manager, you need still to make sure that no privileged code (having access rights to sun.*) forwards tainted data to the vulnerable sun.* functions. Until 2007 you could use the sun.misc.MessageUtils.toStderr bug to reliably crash OpenOffice in the OObase startup database/script by calling sun.* via HSQLDB (CVE-2007-4575) . SET DATABASE COLLATION "Latin1_General" [...] SELECT * FROM "FirstTable" WHERE ID="sun.misc.MessageUtils.toStderr"(NULL); To my knowledge Java in Openoffice still does not use a security manager in all places yet, so this problem was fixed by blocking arbitrary class access in HSQLDB. So the intention is to finally fix the root cause, instead of furthermore allowing this to cause trouble in unexpected places :) > As regards sun.misc.MessageUtils, I don't see any problem fixing this. > I notice the return from NewStringUTF isn't checked. Unfortunately > this (very old) code is also missing checks for the calls to > GetStringChars and malloc. Also, I assume that the additional \0 isn't > needed. unfortunately most of my JNI knowledge was acquired during the Java 1.1/1.2 era, so it may not be state-of-the-art anymore. We also had an internal discussion about the \0 and decided to keep it in the code. So I leave that up to you :) > -Alan. Best Regards Marc -- Marc Schoenefeld / Red Hat Security Response Team From weijun.wang at sun.com Thu Jun 18 03:17:27 2009 From: weijun.wang at sun.com (weijun.wang at sun.com) Date: Thu, 18 Jun 2009 03:17:27 +0000 Subject: hg: jdk7/tl/jdk: 6712755: jarsigner fails to sign itextasian.jar since 1.5.0_b14, it works with 1.5.0_13 Message-ID: <20090618031740.1CBE1E9A0@hg.openjdk.java.net> Changeset: 863351d5d244 Author: weijun Date: 2009-06-18 11:12 +0800 URL: http://hg.openjdk.java.net/jdk7/tl/jdk/rev/863351d5d244 6712755: jarsigner fails to sign itextasian.jar since 1.5.0_b14, it works with 1.5.0_13 Reviewed-by: mullan ! src/share/classes/sun/security/tools/JarSigner.java + test/sun/security/tools/jarsigner/emptymanifest.sh From Alan.Bateman at Sun.COM Thu Jun 18 08:52:17 2009 From: Alan.Bateman at Sun.COM (Alan Bateman) Date: Thu, 18 Jun 2009 09:52:17 +0100 Subject: MessageUtils JVM crash In-Reply-To: <4A38ABE9.7090907@redhat.com> References: <4A37C110.3060909@redhat.com> <4A37C3B3.2030902@sun.com> <4A37C450.7010700@redhat.com> <4A37CC69.3080509@redhat.com> <4A38A530.6060607@sun.com> <4A38ABE9.7090907@redhat.com> Message-ID: <4A3A0041.4030003@sun.com> Marc Schoenefeld wrote: > : > Even if there is a security manager, you need still to make sure that no > privileged code (having access rights to sun.*) forwards tainted data > to the > vulnerable sun.* functions. > > Until 2007 you could use the sun.misc.MessageUtils.toStderr bug to > reliably crash OpenOffice in the OObase startup database/script > by calling sun.* via HSQLDB (CVE-2007-4575) . > > SET DATABASE COLLATION "Latin1_General" > [...] > SELECT * FROM "FirstTable" > WHERE ID="sun.misc.MessageUtils.toStderr"(NULL); > > To my knowledge Java in Openoffice still does not use a security manager > in all places yet, so this problem was fixed by blocking arbitrary > class access in HSQLDB. > > So the intention is to finally fix the root cause, instead of > furthermore allowing this to cause trouble in unexpected places :) > If there isn't a security manager then there aren't any checks and so code can do any manner of nasty thing. We can fix sun.misc.MessageUtils and that solves that specific issue but it doesn't stop code calling System.exit to terminate the VM or using public APIs to delete your files. I'm not familiar with how OO uses the runtime but preventing it from running arbitrary code sounds right. Lillian - are you taking this one with a view to getting it into jdk7/tl/jdk? (I wasn't sure if you were looking for someone to take it). -Alan. From langel at redhat.com Thu Jun 18 14:07:24 2009 From: langel at redhat.com (Lillian Angel) Date: Thu, 18 Jun 2009 10:07:24 -0400 Subject: MessageUtils JVM crash In-Reply-To: <4A3A0041.4030003@sun.com> References: <4A37C110.3060909@redhat.com> <4A37C3B3.2030902@sun.com> <4A37C450.7010700@redhat.com> <4A37CC69.3080509@redhat.com> <4A38A530.6060607@sun.com> <4A38ABE9.7090907@redhat.com> <4A3A0041.4030003@sun.com> Message-ID: <4A3A4A1C.6030007@redhat.com> Alan Bateman wrote: > Marc Schoenefeld wrote: >> : >> Even if there is a security manager, you need still to make sure that no >> privileged code (having access rights to sun.*) forwards tainted data >> to the >> vulnerable sun.* functions. >> Until 2007 you could use the sun.misc.MessageUtils.toStderr bug to >> reliably crash OpenOffice in the OObase startup database/script >> by calling sun.* via HSQLDB (CVE-2007-4575) . >> >> SET DATABASE COLLATION "Latin1_General" >> [...] >> SELECT * FROM "FirstTable" >> WHERE ID="sun.misc.MessageUtils.toStderr"(NULL); >> >> To my knowledge Java in Openoffice still does not use a security manager >> in all places yet, so this problem was fixed by blocking arbitrary >> class access in HSQLDB. >> >> So the intention is to finally fix the root cause, instead of >> furthermore allowing this to cause trouble in unexpected places :) >> > If there isn't a security manager then there aren't any checks and so > code can do any manner of nasty thing. We can fix > sun.misc.MessageUtils and that solves that specific issue but it > doesn't stop code calling System.exit to terminate the VM or using > public APIs to delete your files. I'm not familiar with how OO uses > the runtime but preventing it from running arbitrary code sounds right. > > Lillian - are you taking this one with a view to getting it into > jdk7/tl/jdk? (I wasn't sure if you were looking for someone to take it). Yep, I can get it committed. Thanks! Lillian From sean.mullan at sun.com Thu Jun 18 14:43:20 2009 From: sean.mullan at sun.com (sean.mullan at sun.com) Date: Thu, 18 Jun 2009 14:43:20 +0000 Subject: hg: jdk7/tl/jdk: 2 new changesets Message-ID: <20090618144344.0D08CE9EE@hg.openjdk.java.net> Changeset: e387bb1367a7 Author: mullan Date: 2009-06-18 09:04 -0400 URL: http://hg.openjdk.java.net/jdk7/tl/jdk/rev/e387bb1367a7 6833839: RFE: improve ManifestDigester by instantiating StringBuilder constructor w/ initial value Reviewed-by: weijun ! src/share/classes/sun/security/util/ManifestDigester.java Changeset: 81c176909720 Author: mullan Date: 2009-06-18 10:38 -0400 URL: http://hg.openjdk.java.net/jdk7/tl/jdk/rev/81c176909720 Merge From weijun.wang at sun.com Fri Jun 19 10:11:53 2009 From: weijun.wang at sun.com (weijun.wang at sun.com) Date: Fri, 19 Jun 2009 10:11:53 +0000 Subject: hg: jdk7/tl/jdk: 6851973: ignore incoming channel binding if acceptor does not set one Message-ID: <20090619101208.6C8BCEAFC@hg.openjdk.java.net> Changeset: 37ed72fe7561 Author: weijun Date: 2009-06-19 18:03 +0800 URL: http://hg.openjdk.java.net/jdk7/tl/jdk/rev/37ed72fe7561 6851973: ignore incoming channel binding if acceptor does not set one Reviewed-by: valeriep ! src/share/classes/sun/security/jgss/krb5/InitialToken.java + test/sun/security/krb5/auto/IgnoreChannelBinding.java From jean-christophe.collet at sun.com Fri Jun 19 12:23:09 2009 From: jean-christophe.collet at sun.com (jean-christophe.collet at sun.com) Date: Fri, 19 Jun 2009 12:23:09 +0000 Subject: hg: jdk7/tl/jdk: 6852108: Remove Preferences dependance from SocksSocketImpl Message-ID: <20090619122324.952ADEB12@hg.openjdk.java.net> Changeset: ed38f9e6ad9a Author: jccollet Date: 2009-06-19 14:12 +0200 URL: http://hg.openjdk.java.net/jdk7/tl/jdk/rev/ed38f9e6ad9a 6852108: Remove Preferences dependance from SocksSocketImpl Summary: Removed Preferences API use and fixed a few findbugs gotchas Reviewed-by: alanb ! src/share/classes/java/net/SocksSocketImpl.java From ebourg at apache.org Fri Jun 19 17:00:21 2009 From: ebourg at apache.org (Emmanuel Bourg) Date: Fri, 19 Jun 2009 19:00:21 +0200 Subject: java.util.logging improvements Message-ID: <4A3BC425.40904@apache.org> Hi, There are some modest enhancement requests filed in the bug database at Sun to improve the java.util.logging API. For example the addition of simpler methods accepting a message and a Throwable: log.warn("Unable to do this", e); or the addition of parametrized messages: log.fine("%s logged in", user.getName()); What is required to get this added to the JDK? Does it require a formal JSR? Could this be contributed directly as patches to OpenJDK7? Is there someone in charge that could be contacted to discuss the proposal? Thank you, Emmanuel Bourg From jonathan.gibbons at sun.com Fri Jun 19 18:49:29 2009 From: jonathan.gibbons at sun.com (jonathan.gibbons at sun.com) Date: Fri, 19 Jun 2009 18:49:29 +0000 Subject: hg: jdk7/tl/langtools: 6852856: javap changes to facilitate subclassing javap for variants Message-ID: <20090619184931.616BFED02@hg.openjdk.java.net> Changeset: ed989c347b3c Author: jjg Date: 2009-06-19 11:40 -0700 URL: http://hg.openjdk.java.net/jdk7/tl/langtools/rev/ed989c347b3c 6852856: javap changes to facilitate subclassing javap for variants Reviewed-by: mcimadamore ! src/share/classes/com/sun/tools/classfile/AccessFlags.java ! src/share/classes/com/sun/tools/classfile/ConstantPool.java ! src/share/classes/com/sun/tools/javap/AttributeWriter.java ! src/share/classes/com/sun/tools/javap/ClassWriter.java ! src/share/classes/com/sun/tools/javap/ConstantWriter.java ! src/share/classes/com/sun/tools/javap/JavapTask.java ! src/share/classes/com/sun/tools/javap/SourceWriter.java From xueming.shen at sun.com Fri Jun 19 22:12:02 2009 From: xueming.shen at sun.com (xueming.shen at sun.com) Date: Fri, 19 Jun 2009 22:12:02 +0000 Subject: hg: jdk7/tl/jdk: 6299219: euro sign failed to be printed in Console on Localized Windows platform with GBK encoding; ... Message-ID: <20090619221221.B056DEDCF@hg.openjdk.java.net> Changeset: 77367060d119 Author: sherman Date: 2009-06-19 14:39 -0700 URL: http://hg.openjdk.java.net/jdk7/tl/jdk/rev/77367060d119 6299219: euro sign failed to be printed in Console on Localized Windows platform with GBK encoding 4891024: EUC-KR and JOHAB converters need to be updated to include two new characters 4287467: Character converter generator tool Summary: Migrated some of the doublebyte charsets to the new implementation. Reviewed-by: okutsu ! make/sun/nio/FILES_java.gmk ! make/sun/nio/Makefile + make/tools/CharsetMapping/EUC_CN.map + make/tools/CharsetMapping/EUC_KR.map + make/tools/CharsetMapping/GBK.map + make/tools/CharsetMapping/Johab.map + make/tools/CharsetMapping/MS932.c2b + make/tools/CharsetMapping/MS932.map + make/tools/CharsetMapping/MS932.nr + make/tools/CharsetMapping/MS936.map + make/tools/CharsetMapping/MS949.map + make/tools/CharsetMapping/MS950.map + make/tools/CharsetMapping/MS950.nr ! make/tools/CharsetMapping/dbcs ! make/tools/src/build/tools/charsetmapping/GenerateDBCS.java ! src/share/classes/sun/io/ByteToCharEUC_CN.java ! src/share/classes/sun/io/ByteToCharEUC_KR.java ! src/share/classes/sun/io/ByteToCharGBK.java ! src/share/classes/sun/io/ByteToCharJohab.java ! src/share/classes/sun/io/ByteToCharMS932.java - src/share/classes/sun/io/ByteToCharMS932DB.java ! src/share/classes/sun/io/ByteToCharMS936.java ! src/share/classes/sun/io/ByteToCharMS949.java ! src/share/classes/sun/io/ByteToCharMS950.java ! src/share/classes/sun/io/ByteToCharMS950_HKSCS.java ! src/share/classes/sun/io/CharToByteEUC_CN.java ! src/share/classes/sun/io/CharToByteEUC_KR.java ! src/share/classes/sun/io/CharToByteGBK.java ! src/share/classes/sun/io/CharToByteJohab.java ! src/share/classes/sun/io/CharToByteMS932.java - src/share/classes/sun/io/CharToByteMS932DB.java ! src/share/classes/sun/io/CharToByteMS936.java ! src/share/classes/sun/io/CharToByteMS949.java ! src/share/classes/sun/io/CharToByteMS950.java ! src/share/classes/sun/io/CharToByteMS950_HKSCS.java ! src/share/classes/sun/nio/cs/ext/DoubleByte.java - src/share/classes/sun/nio/cs/ext/EUC_CN.java - src/share/classes/sun/nio/cs/ext/EUC_KR.java - src/share/classes/sun/nio/cs/ext/GBK.java ! src/share/classes/sun/nio/cs/ext/ISO2022_CN.java - src/share/classes/sun/nio/cs/ext/Johab.java - src/share/classes/sun/nio/cs/ext/MS932.java - src/share/classes/sun/nio/cs/ext/MS932DB.java ! src/share/classes/sun/nio/cs/ext/MS932_0213.java - src/share/classes/sun/nio/cs/ext/MS936.java - src/share/classes/sun/nio/cs/ext/MS949.java - src/share/classes/sun/nio/cs/ext/MS950.java ! src/share/classes/sun/nio/cs/ext/MS950_HKSCS.java ! src/solaris/classes/sun/awt/motif/X11GB2312.java ! src/solaris/classes/sun/awt/motif/X11GBK.java ! src/solaris/classes/sun/awt/motif/X11KSC5601.java + test/sun/nio/cs/OLD/DoubleByteDecoder.java + test/sun/nio/cs/OLD/DoubleByteEncoder.java + test/sun/nio/cs/OLD/EUC_CN_OLD.java + test/sun/nio/cs/OLD/EUC_KR_OLD.java + test/sun/nio/cs/OLD/GBK_OLD.java + test/sun/nio/cs/OLD/Johab_OLD.java + test/sun/nio/cs/OLD/MS932DB.java + test/sun/nio/cs/OLD/MS932_OLD.java + test/sun/nio/cs/OLD/MS936_OLD.java + test/sun/nio/cs/OLD/MS949_OLD.java + test/sun/nio/cs/OLD/MS950_OLD.java ! test/sun/nio/cs/OLD/TestIBMDB.java + test/sun/nio/cs/OLD/TestX11CS.java + test/sun/nio/cs/OLD/X11GB2312_OLD.java + test/sun/nio/cs/OLD/X11GBK_OLD.java + test/sun/nio/cs/OLD/X11KSC5601_OLD.java From xiaobin.lu at sun.com Sun Jun 21 05:48:29 2009 From: xiaobin.lu at sun.com (xiaobin.lu at sun.com) Date: Sun, 21 Jun 2009 05:48:29 +0000 Subject: hg: jdk7/tl/jdk: 6850606: Regression from JDK 1.6.0_12 Message-ID: <20090621054845.7CB0DEDFA@hg.openjdk.java.net> Changeset: 28d4c9f5c9e9 Author: xlu Date: 2009-06-20 13:34 -0700 URL: http://hg.openjdk.java.net/jdk7/tl/jdk/rev/28d4c9f5c9e9 6850606: Regression from JDK 1.6.0_12 Summary: The returned result from multiply should be constructed by using valueOf to take care of the INFLATED case. Reviewed-by: darcy ! src/share/classes/java/math/BigDecimal.java ! test/java/math/BigDecimal/MultiplyTests.java From martinrb at google.com Mon Jun 22 23:46:37 2009 From: martinrb at google.com (martinrb at google.com) Date: Mon, 22 Jun 2009 23:46:37 +0000 Subject: hg: jdk7/tl/jdk: 6851653: (launcher) Make every java process 20 bytes smaller Message-ID: <20090622234651.0BBDEEEB6@hg.openjdk.java.net> Changeset: b0b249933c37 Author: martin Date: 2009-06-22 16:41 -0700 URL: http://hg.openjdk.java.net/jdk7/tl/jdk/rev/b0b249933c37 6851653: (launcher) Make every java process 20 bytes smaller Summary: Carefully keep track of every byte Reviewed-by: ksrini, xlu ! src/share/bin/java.c From martinrb at google.com Tue Jun 23 00:45:59 2009 From: martinrb at google.com (Martin Buchholz) Date: Mon, 22 Jun 2009 17:45:59 -0700 Subject: Review request for 5049299 In-Reply-To: <1ccfd1c10906111416x1b0fveb52eb9d3db56ded@mail.gmail.com> References: <4A1678DB.9010209@sun.com> <1ccfd1c10906081352i12feb1a5j54133adb4f3f5c25@mail.gmail.com> <4A2E69FF.6080908@sun.com> <1ccfd1c10906090851g6498733dw78743283cdfe405f@mail.gmail.com> <4A2E8DA1.3050004@sun.com> <1ccfd1c10906091142h465bf9adoaa1b5c87bcf6ef9f@mail.gmail.com> <1ccfd1c10906091224p6cc0ccefpfa49453466e99640@mail.gmail.com> <1ccfd1c10906091244u14607dcesc40fc6017adc7808@mail.gmail.com> <4A311337.9070706@sun.com> <1ccfd1c10906111416x1b0fveb52eb9d3db56ded@mail.gmail.com> Message-ID: <1ccfd1c10906221745u87082b0m80dd3b79b4646bdf@mail.gmail.com> clone-exec update: I submitted the changes for this, but jtreg tests failed on 32-bit Linux (I had only tested on 64-bit Linux) We disabled (but did not roll back) the use of clone to allow the TL integration to proceed. (As I promised elsewhere...) I just filed a bug against upstream glibc demonstrating the problem with clone(CLONE_VM). You can see the small C program in my bug report below. Probably any discussion related just to the glibc bug can occur on the public glibc bugzilla at http://sources.redhat.com/bugzilla/show_bug.cgi?id=10311 glibc maintainer Uli Drepper has already responded saying "If you use clone() you're on your own." so if we are going to fix it, we'll have to do it ourselves. Help from threading/kernel hackers appreciated. Thanks much, Martin ---------- Forwarded message ---------- From: martinrb at google dot com Date: Mon, Jun 22, 2009 at 12:23 Subject: [Bug nptl/10311] New: clone(CLONE_VM) fails with pthread_getattr_np on i386 To: martinrb at google.com I'm using clone() with flags CLONE_VM, but not CLONE_THREAD. (background: I'm trying to solve the ancient overcommit failure when spawning a small Unix process from a big process). The act of calling clone appears to mess up the pthread library, but only on i386, not on x86_64, using glibc version 2.7 (The bugzilla Version drop-down does not allow one to specify 2.7; y'all should fix that) Here's a shell transcript containing a program that demonstrates the problem, and shows that the problem does not occur when running in 64-bit mode on 64-bit Linux. (The problem also occurs when running in 32-bit mode on 32-bit Linux). A program like this would be a fine addition to the glibc test suite. $ set -x; for flag in -m32 -m64; do gcc $flag -lpthread ./clone_bug.c && ./a.out; done; cat clone_bug.c; uname -a; getconf GNU_LIBPTHREAD_VERSION; getconf GNU_LIBC_VERSION +zsh:1464> set -x +zsh:1464> flag=-m32 +zsh:1464> gcc -m32 -lpthread ./clone_bug.c +zsh:1464> ./a.out count=2, pthread_getattr_np failed with errno = "No such process" +zsh:1464> flag=-m64 +zsh:1464> gcc -m64 -lpthread ./clone_bug.c +zsh:1464> ./a.out +zsh:1464> cat clone_bug.c #include #include #include #include #include #include #include #include #include #include #include static void debugPrint(char *format, ...) { FILE *tty = fopen("/dev/tty", "w"); va_list ap; va_start(ap, format); vfprintf(tty, format, ap); va_end(ap); fclose(tty); } static void debugPids(void) { // debugPrint("getpid()=%d gettid()=%d, syscall(getpid)=%d pthread_self=%d\n", // getpid(), syscall(SYS_gettid), syscall(SYS_getpid), pthread_self()); static int count = 0; pthread_attr_t attr; int result; ++count; if ((result = pthread_getattr_np(pthread_self(), &attr)) != 0) debugPrint("count=%d, pthread_getattr_np failed with errno = \"%s\"\n", count, strerror(result)); } static int childProcess(void *ignored) { _exit(0); // debugPrint("child\n"); // execve("/bin/true", NULL, NULL); // perror("execve"); } // I'm sure there's a better way to do this, // but pthread_join ain't it - we can't trust it. volatile int done = 0; void* run(void *x) { const int stack_size = 1024 * 1024; void *clone_stack = malloc(2 * stack_size); int status; debugPids(); int pid = clone(childProcess, clone_stack + stack_size, CLONE_VM | SIGCHLD, NULL); waitpid(pid, &status, 0); debugPids(); done = 1; pthread_exit(0); return NULL; } int main(int argc, char *argv[]) { pthread_attr_t attr; pthread_t tid; pthread_attr_init(&attr); pthread_create(&tid, &attr, (void* (*)(void*)) run, NULL); // pthread_join(tid, NULL); while (! done) ; } +zsh:1464> uname -a Linux spraggett.mtv.corp.google.com 2.6.24-gg23-generic #1 SMP Fri Jan 30 14:07:49 PST 2009 x86_64 GNU/Linux +zsh:1464> getconf GNU_LIBPTHREAD_VERSION NPTL 2.7 +zsh:1464> getconf GNU_LIBC_VERSION glibc 2.7 -- Summary: clone(CLONE_VM) fails with pthread_getattr_np on i386 Product: glibc Version: 2.8 Status: NEW Severity: normal Priority: P2 Component: nptl AssignedTo: drepper at redhat dot com ReportedBy: martinrb at google dot com CC: glibc-bugs at sources dot redhat dot com GCC host triplet: x86_64-unknown-linux-gnu http://sourceware.org/bugzilla/show_bug.cgi?id=10311 Martin On Thu, Jun 11, 2009 at 14:16, Martin Buchholz wrote: > Thanks, Michael > > I'm hoping the following will placate sun studio cc: > > diff --git a/src/solaris/native/java/lang/UNIXProcess_md.c > b/src/solaris/native/java/lang/UNIXProcess_md.c > --- a/src/solaris/native/java/lang/UNIXProcess_md.c > +++ b/src/solaris/native/java/lang/UNIXProcess_md.c > @@ -651,6 +651,7 @@ > } > close(FAIL_FILENO); > _exit(-1); > + return 0; /* Suppress warning "no return value from function" */ > } > > I'm also adding my manual test case BigFork.java. > It may be helpful while implementing the Solaris version of this feature. > > webrev updated. > > I need a Sun bug to commit these changes for Linux. Please create one. > > Synopsis: * (process) Use clone(CLONE_VM), not fork, on Linux to avoid > swap exhaustion * > > Description: > On Linux it is possible to use clone with CLONE_VM, but not CLONE_THREAD, > which is like fork() but much cheaper and avoids swap exhaustion due to > momentary > overcommit of swap space. One has to be very careful in this case to not > mutate global > variables such as environ, but it's worth it. > > Evaluation: > Make it so. > > See also: 5049299 > > Once that is done, I will commit my changes. > > Thanks, > > Martin > > > On Thu, Jun 11, 2009 at 07:22, Michael McMahon wrote: > >> Martin Buchholz wrote: >> >>> I broke down and finally created a "proper" webrev, >>> just like the good old days. >>> >>> http://cr.openjdk.java.net/~martin/clone-exec/< >>> http://cr.openjdk.java.net/%7Emartin/clone-exec/> >>> >>> I've run the regression tests on Solaris and Linux and they seem fine. >> There is a compile warning on solaris at line 654: no return value from >> function. >> Aside from that, I'm happy with the change now >> >> - Michael. >> > > -------------- next part -------------- An HTML attachment was scrubbed... URL: From xueming.shen at sun.com Tue Jun 23 02:33:26 2009 From: xueming.shen at sun.com (xueming.shen at sun.com) Date: Tue, 23 Jun 2009 02:33:26 +0000 Subject: hg: jdk7/tl/jdk: 6847092: (cs) CharsetEncoder.isLegalReplacement of US_ASCII behaves differently since Message-ID: <20090623023338.78F8CEEBF@hg.openjdk.java.net> Changeset: 7704895771b5 Author: sherman Date: 2009-06-22 19:22 -0700 URL: http://hg.openjdk.java.net/jdk7/tl/jdk/rev/7704895771b5 6847092: (cs) CharsetEncoder.isLegalReplacement of US_ASCII behaves differently since Summary: Updated the US_ASCII and ISO-8859-1 to fix the failure. Reviewed-by: alanb, martin ! src/share/classes/sun/nio/cs/ISO_8859_1.java ! src/share/classes/sun/nio/cs/US_ASCII.java + test/sun/nio/cs/FindASCIIReplBugs.java From martinrb at google.com Tue Jun 23 04:14:26 2009 From: martinrb at google.com (martinrb at google.com) Date: Tue, 23 Jun 2009 04:14:26 +0000 Subject: hg: jdk7/tl/jdk: 2 new changesets Message-ID: <20090623041450.C04F8EECC@hg.openjdk.java.net> Changeset: ce55eb6668d9 Author: martin Date: 2009-06-22 20:47 -0700 URL: http://hg.openjdk.java.net/jdk7/tl/jdk/rev/ce55eb6668d9 6834805: Improve jar -C performance Summary: Store "-C" directories in a HashSet, not List, to remove duplicates Reviewed-by: sherman Contributed-by: jeremymanson at google.com ! src/share/classes/sun/tools/jar/Main.java Changeset: ff32c270102a Author: martin Date: 2009-06-22 21:07 -0700 URL: http://hg.openjdk.java.net/jdk7/tl/jdk/rev/ff32c270102a 6853806: Prefer (cd $dir && jar) to jar -C for performance reasons Summary: Eliminate (most) uses of jar -C Reviewed-by: ohair ! make/common/Release.gmk From aph at redhat.com Tue Jun 23 14:33:36 2009 From: aph at redhat.com (Andrew Haley) Date: Tue, 23 Jun 2009 15:33:36 +0100 Subject: Review request for 5049299 In-Reply-To: <1ccfd1c10906221745u87082b0m80dd3b79b4646bdf@mail.gmail.com> References: <4A1678DB.9010209@sun.com> <1ccfd1c10906081352i12feb1a5j54133adb4f3f5c25@mail.gmail.com> <4A2E69FF.6080908@sun.com> <1ccfd1c10906090851g6498733dw78743283cdfe405f@mail.gmail.com> <4A2E8DA1.3050004@sun.com> <1ccfd1c10906091142h465bf9adoaa1b5c87bcf6ef9f@mail.gmail.com> <1ccfd1c10906091224p6cc0ccefpfa49453466e99640@mail.gmail.com> <1ccfd1c10906091244u14607dcesc40fc6017adc7808@mail.gmail.com> <4A311337.9070706@sun.com> <1ccfd1c10906111416x1b0fveb52eb9d3db56ded@mail.gmail.com> <1ccfd1c10906221745u87082b0m80dd3b79b4646bdf@mail.gmail.com> Message-ID: <4A40E7C0.5080308@redhat.com> Martin Buchholz wrote: > clone-exec update: > > I submitted the changes for this, but jtreg tests failed on 32-bit Linux > (I had only tested on 64-bit Linux) > > We disabled (but did not roll back) the use of clone to allow the > TL integration to proceed. > > (As I promised elsewhere...) > I just filed a bug against upstream glibc demonstrating the problem > with clone(CLONE_VM). You can see the small C program > in my bug report below. > Probably any discussion related just to the glibc bug > can occur on the public glibc bugzilla at > http://sources.redhat.com/bugzilla/show_bug.cgi?id=10311 > > glibc maintainer Uli Drepper has already responded > saying > > "If you use clone() you're on your own." > > so if we are going to fix it, we'll have to do it ourselves. > Help from threading/kernel hackers appreciated. I can debug this. Please try first syscall(SYS_clone ...) to bypass the libc gubbins. That might be all you need. If that doesn't help I'll have a look. Isn't there some point at which you have to say to a Linux user "Your system is simply misconfigured. Fix the overcommit parameter and this problem will go away" ? Andrew. From christos at zoulas.com Tue Jun 23 14:51:26 2009 From: christos at zoulas.com (Christos Zoulas) Date: Tue, 23 Jun 2009 10:51:26 -0400 Subject: Review request for 5049299 In-Reply-To: <4A40E7C0.5080308@redhat.com> from Andrew Haley (Jun 23, 3:33pm) Message-ID: <20090623145126.6236C5654F@rebar.astron.com> On Jun 23, 3:33pm, aph at redhat.com (Andrew Haley) wrote: -- Subject: Re: Review request for 5049299 | I can debug this. | | Please try first syscall(SYS_clone ...) to bypass the libc gubbins. | That might be all you need. If that doesn't help I'll have a look. | | Isn't there some point at which you have to say to a Linux user "Your | system is simply misconfigured. Fix the overcommit parameter and this | problem will go away" ? Another thing to try is to add CLONE_VFORK to suspend the parent. christos From Michael.McMahon at Sun.COM Tue Jun 23 14:55:27 2009 From: Michael.McMahon at Sun.COM (Michael McMahon) Date: Tue, 23 Jun 2009 15:55:27 +0100 Subject: Review request for 5049299 In-Reply-To: <20090623145126.6236C5654F@rebar.astron.com> References: <20090623145126.6236C5654F@rebar.astron.com> Message-ID: <4A40ECDF.1060108@sun.com> Christos Zoulas wrote: > On Jun 23, 3:33pm, aph at redhat.com (Andrew Haley) wrote: > -- Subject: Re: Review request for 5049299 > > | I can debug this. > | > | Please try first syscall(SYS_clone ...) to bypass the libc gubbins. > | That might be all you need. If that doesn't help I'll have a look. > | > | Isn't there some point at which you have to say to a Linux user "Your > | system is simply misconfigured. Fix the overcommit parameter and this > | problem will go away" ? > > Another thing to try is to add CLONE_VFORK to suspend the parent. > That doesn't help, I've tried it. It seems to be a CLONE_VM problem. Michael. From dl at cs.oswego.edu Tue Jun 23 16:04:38 2009 From: dl at cs.oswego.edu (Doug Lea) Date: Tue, 23 Jun 2009 12:04:38 -0400 Subject: Faster HashMap implementation In-Reply-To: References: <4A2D3729.3040904@cs.oswego.edu> <4A2E5109.4050804@cs.oswego.edu> Message-ID: <4A40FD16.6040503@cs.oswego.edu> Sorry for the delay; I've been side-tracked exploring concurrent trie/hash algorithms But I did want to point out a problem shared with all purely table-based approaches, including yours. Assuming power of 2 capacities, you can only hold up to 1 << 29 entries, because max power of 2 array length is 1 << 30. This holds even in -d64 mode. Current HashMap works up through size == Integer.MAX_VALUE. (And even "works" if size overflows so long as you don't call size().) I know that there are some extremely large hash maps out there. Maybe some would newly encounters failures. There are ways out of this, using some sort of dynamic structures for overflow. You might think that it is unfair that IdentityHashMap has always had a similar limitation and gotten away with it, but the ground rules here are that you cannot cause programs that used to work, to fail in JDK updates. And worse, we must conservatively assume that any legitimate usage that is possible actually exists. -Doug From maurizio.cimadamore at sun.com Wed Jun 24 10:01:41 2009 From: maurizio.cimadamore at sun.com (maurizio.cimadamore at sun.com) Date: Wed, 24 Jun 2009 10:01:41 +0000 Subject: hg: jdk7/tl/langtools: 3 new changesets Message-ID: <20090624100145.CF369EF40@hg.openjdk.java.net> Changeset: 18e0269f25e3 Author: mcimadamore Date: 2009-06-24 10:50 +0100 URL: http://hg.openjdk.java.net/jdk7/tl/langtools/rev/18e0269f25e3 6822637: ResolveError hierarchy needs to be refactored Summary: Break ResolveError class into a hierarchy representing different kinds of resolution errors Reviewed-by: jjg ! src/share/classes/com/sun/tools/javac/code/Kinds.java ! src/share/classes/com/sun/tools/javac/comp/Resolve.java ! src/share/classes/com/sun/tools/javac/util/JCDiagnostic.java Changeset: 8ec37cf2b37e Author: mcimadamore Date: 2009-06-24 10:50 +0100 URL: http://hg.openjdk.java.net/jdk7/tl/langtools/rev/8ec37cf2b37e 6852595: Accessing scope using JSR199 API on erroneous tree causes Illegal Argument Exception Summary: Fixed problem with empty DiagnosticSource objects causing IAE in the JCDiagnostic constructor Reviewed-by: jjg ! src/share/classes/com/sun/tools/javac/comp/Attr.java ! src/share/classes/com/sun/tools/javac/util/AbstractLog.java ! src/share/classes/com/sun/tools/javac/util/DiagnosticSource.java + test/tools/javac/api/6852595/T6852595.java Changeset: 1d9e61e0a075 Author: mcimadamore Date: 2009-06-24 10:51 +0100 URL: http://hg.openjdk.java.net/jdk7/tl/langtools/rev/1d9e61e0a075 6852649: The Rich formatter printer should be an explicit class to facilitate overriding Summary: Improve reusabiliy of the rich formatter by removing anonymous inner classes/changing visibility of fields Reviewed-by: jjg ! src/share/classes/com/sun/tools/javac/util/AbstractDiagnosticFormatter.java ! src/share/classes/com/sun/tools/javac/util/RichDiagnosticFormatter.java From i30817 at gmail.com Wed Jun 24 17:35:37 2009 From: i30817 at gmail.com (Paulo Levi) Date: Wed, 24 Jun 2009 18:35:37 +0100 Subject: ServiceLoader loads services as singletons? Message-ID: <212322090906241035i1ea607b5r720b1111989a5a89@mail.gmail.com> I've noticed something disturbing about service loader... I thought its iterator returned a new object by default, so i made a protected method in my abstract class to serve as a "constructor" and i thought i was done. A simple system.out.println reveals the horrible truth. util.io.compressed.RarExtractor at 1ca3f82 util.io.compressed.ZipExtractor at 1e6ac83 util.io.compressed.RarExtractor at 1ca3f82 util.io.compressed.RarExtractor at 1ca3f82 util.io.compressed.RarExtractor at 1ca3f82 util.io.compressed.RarExtractor at 1ca3f82 util.io.compressed.RarExtractor at 1ca3f82 util.io.compressed.RarExtractor at 1ca3f82 util.io.compressed.RarExtractor at 1ca3f82 util.io.compressed.RarExtractor at 1ca3f82 util.io.compressed.ZipExtractor at 1e6ac83 util.io.compressed.RarExtractor at 1ca3f82 util.io.compressed.RarExtractor at 1ca3f82 util.io.compressed.RarExtractor at 1ca3f82 util.io.compressed.RarExtractor at 1ca3f82 util.io.compressed.ZipExtractor at 1e6ac83 util.io.compressed.RarExtractor at 1ca3f82 util.io.compressed.RarExtractor at 1ca3f82 util.io.compressed.RarExtractor at 1ca3f82 So no way to specify that object returned by the iterator should always be new? This should at least be specified in large and bold letters. Preferably a setting with default create a new object would be better. -------------- next part -------------- An HTML attachment was scrubbed... URL: From tim.bell at sun.com Thu Jun 25 00:57:06 2009 From: tim.bell at sun.com (tim.bell at sun.com) Date: Thu, 25 Jun 2009 00:57:06 +0000 Subject: hg: jdk7/tl: 2 new changesets Message-ID: <20090625005706.EBB47E013@hg.openjdk.java.net> Changeset: 68836ec8bcc7 Author: xdono Date: 2009-06-18 13:05 -0700 URL: http://hg.openjdk.java.net/jdk7/tl/rev/68836ec8bcc7 Added tag jdk7-b61 for changeset 472c21584cfd ! .hgtags Changeset: e84b527d8be8 Author: tbell Date: 2009-06-21 23:49 -0700 URL: http://hg.openjdk.java.net/jdk7/tl/rev/e84b527d8be8 Merge From tim.bell at sun.com Thu Jun 25 01:01:53 2009 From: tim.bell at sun.com (tim.bell at sun.com) Date: Thu, 25 Jun 2009 01:01:53 +0000 Subject: hg: jdk7/tl/corba: 2 new changesets Message-ID: <20090625010154.B3944E018@hg.openjdk.java.net> Changeset: c73934e09f00 Author: xdono Date: 2009-06-18 13:05 -0700 URL: http://hg.openjdk.java.net/jdk7/tl/corba/rev/c73934e09f00 Added tag jdk7-b61 for changeset e906b16a12a9 ! .hgtags Changeset: 65b66117dbd7 Author: tbell Date: 2009-06-21 23:50 -0700 URL: http://hg.openjdk.java.net/jdk7/tl/corba/rev/65b66117dbd7 Merge From tim.bell at sun.com Thu Jun 25 01:09:07 2009 From: tim.bell at sun.com (tim.bell at sun.com) Date: Thu, 25 Jun 2009 01:09:07 +0000 Subject: hg: jdk7/tl/hotspot: 32 new changesets Message-ID: <20090625011006.03A0CE041@hg.openjdk.java.net> Changeset: 47ffceb239d0 Author: thurka Date: 2009-05-20 09:36 +0200 URL: http://hg.openjdk.java.net/jdk7/tl/hotspot/rev/47ffceb239d0 6839599: JVM crash while profiling Tomcat and Liferay Summary: constantPoolOopDesc::copy_cpool_bytes() - do the brute-force search search through 'tbl' when SymbolTable::lookup_only() returns NULL Reviewed-by: kamg ! src/share/vm/oops/constantPoolOop.cpp Changeset: f1f3a2719a55 Author: xlu Date: 2009-05-22 16:40 -0700 URL: http://hg.openjdk.java.net/jdk7/tl/hotspot/rev/f1f3a2719a55 6843580: JavaThread.getStackBase throws sun.jvm.hotspot.WrongTypeException invoked by jstack Reviewed-by: phh, dice, never, swamyv ! agent/src/share/classes/sun/jvm/hotspot/runtime/JavaThread.java Changeset: 93c14e5562c4 Author: twisti Date: 2009-05-06 00:27 -0700 URL: http://hg.openjdk.java.net/jdk7/tl/hotspot/rev/93c14e5562c4 6823354: Add intrinsics for {Integer,Long}.{numberOfLeadingZeros,numberOfTrailingZeros}() Summary: These methods can be instrinsified by using bit scan, bit test, and population count instructions. Reviewed-by: kvn, never ! src/cpu/sparc/vm/sparc.ad ! src/cpu/x86/vm/assembler_x86.cpp ! src/cpu/x86/vm/assembler_x86.hpp ! src/cpu/x86/vm/vm_version_x86.cpp ! src/cpu/x86/vm/vm_version_x86.hpp ! src/cpu/x86/vm/x86_32.ad ! src/cpu/x86/vm/x86_64.ad ! src/share/vm/classfile/vmSymbols.hpp ! src/share/vm/opto/classes.hpp ! src/share/vm/opto/connode.cpp ! src/share/vm/opto/connode.hpp ! src/share/vm/opto/library_call.cpp ! src/share/vm/opto/matcher.hpp ! src/share/vm/runtime/globals.hpp + test/compiler/6823354/Test6823354.java Changeset: e85af0c0c94b Author: twisti Date: 2009-05-07 00:28 -0700 URL: http://hg.openjdk.java.net/jdk7/tl/hotspot/rev/e85af0c0c94b Merge Changeset: f53b154d959d Author: twisti Date: 2009-05-06 08:57 -0700 URL: http://hg.openjdk.java.net/jdk7/tl/hotspot/rev/f53b154d959d 6837906: compiler tests of 6636138 fail with IllegalAccessException Summary: The compiler tests of 6636138 fail with an IllegalAccessException. Reviewed-by: kvn ! test/compiler/6636138/Test1.java ! test/compiler/6636138/Test2.java Changeset: f2954231d190 Author: twisti Date: 2009-05-07 04:16 -0700 URL: http://hg.openjdk.java.net/jdk7/tl/hotspot/rev/f2954231d190 Merge Changeset: d0e0d6d824d8 Author: kvn Date: 2009-05-08 10:34 -0700 URL: http://hg.openjdk.java.net/jdk7/tl/hotspot/rev/d0e0d6d824d8 Merge ! src/share/vm/runtime/globals.hpp Changeset: c96bf21b756f Author: kvn Date: 2009-05-08 10:44 -0700 URL: http://hg.openjdk.java.net/jdk7/tl/hotspot/rev/c96bf21b756f 6788527: Server vm intermittently fails with assertion "live value must not be garbage" with fastdebug bits Summary: Cache Jvmti and DTrace flags used by Compiler. Reviewed-by: never ! src/cpu/sparc/vm/c1_LIRAssembler_sparc.cpp ! src/cpu/sparc/vm/c1_MacroAssembler_sparc.cpp ! src/cpu/x86/vm/c1_LIRAssembler_x86.cpp ! src/cpu/x86/vm/c1_MacroAssembler_x86.cpp ! src/share/vm/c1/c1_Compilation.cpp ! src/share/vm/c1/c1_GraphBuilder.cpp ! src/share/vm/c1/c1_LIRGenerator.cpp ! src/share/vm/ci/ciEnv.cpp ! src/share/vm/ci/ciEnv.hpp ! src/share/vm/ci/ciMethod.cpp ! src/share/vm/compiler/compileBroker.cpp ! src/share/vm/opto/doCall.cpp ! src/share/vm/opto/graphKit.cpp ! src/share/vm/opto/macro.cpp ! src/share/vm/opto/parse1.cpp ! src/share/vm/opto/parse2.cpp Changeset: 44ccd7a9065c Author: ohair Date: 2009-05-08 15:16 -0700 URL: http://hg.openjdk.java.net/jdk7/tl/hotspot/rev/44ccd7a9065c 6839151: Add a JPRT default test of -Xshare:dump when new hotspot is built Reviewed-by: never, kvn ! make/jprt.properties ! test/Makefile Changeset: 900e4df4b0c3 Author: ohair Date: 2009-05-08 23:00 -0700 URL: http://hg.openjdk.java.net/jdk7/tl/hotspot/rev/900e4df4b0c3 Merge Changeset: a9e116455022 Author: kvn Date: 2009-05-11 17:59 -0700 URL: http://hg.openjdk.java.net/jdk7/tl/hotspot/rev/a9e116455022 6832293: JIT compiler got wrong result in type checking with -server Summary: Check for an object array of interface in CmpPNode::sub(). Reviewed-by: never ! src/share/vm/opto/subnode.cpp + test/compiler/6832293/Test.java Changeset: b2934faac289 Author: kvn Date: 2009-05-11 18:30 -0700 URL: http://hg.openjdk.java.net/jdk7/tl/hotspot/rev/b2934faac289 6836054: java/util/Arrays/CopyMethods.java fails on solaris-sparc with IllegalArgumentException Summary: Do not mark an allocation as scalar replaceable if its actual type in unknown statically. Reviewed-by: never ! src/share/vm/opto/escape.cpp Changeset: 2056494941db Author: twisti Date: 2009-05-13 00:45 -0700 URL: http://hg.openjdk.java.net/jdk7/tl/hotspot/rev/2056494941db 6814842: Load shortening optimizations Summary: 6797305 handles load widening but no shortening which should be covered here. Reviewed-by: never, kvn ! src/cpu/sparc/vm/sparc.ad ! src/cpu/x86/vm/x86_32.ad ! src/cpu/x86/vm/x86_64.ad ! src/share/vm/adlc/output_c.cpp + test/compiler/6814842/Test6814842.java Changeset: 27d660246893 Author: ohair Date: 2009-05-15 18:14 -0700 URL: http://hg.openjdk.java.net/jdk7/tl/hotspot/rev/27d660246893 Merge ! make/linux/makefiles/gcc.make ! make/linux/makefiles/jsig.make ! make/linux/makefiles/saproc.make Changeset: aabd393cf1ee Author: kvn Date: 2009-05-21 10:05 -0700 URL: http://hg.openjdk.java.net/jdk7/tl/hotspot/rev/aabd393cf1ee 6772683: Thread.isInterrupted() fails to return true on multiprocessor PC Summary: Set the control edge for the field _interrupted load in inline_native_isInterrupted(). Reviewed-by: never ! src/share/vm/opto/library_call.cpp + test/compiler/6772683/InterruptedTest.java Changeset: 1851e1fb420e Author: kvn Date: 2009-05-27 12:35 -0700 URL: http://hg.openjdk.java.net/jdk7/tl/hotspot/rev/1851e1fb420e 6843752: missing code for an anti-dependent Phi in GCM Summary: Don't place a load below anti-dependent PHI. Reviewed-by: never, twisti ! src/share/vm/opto/gcm.cpp + test/compiler/6843752/Test.java Changeset: 273b2358ef1a Author: cfang Date: 2009-05-28 09:37 -0700 URL: http://hg.openjdk.java.net/jdk7/tl/hotspot/rev/273b2358ef1a 6837146: Should perform unswitch before maximally unroll in loop transformation Summary: Move loop unswitch before maximally unroll Reviewed-by: never ! src/share/vm/opto/loopTransform.cpp Changeset: 435f0808b826 Author: never Date: 2009-06-03 15:02 -0700 URL: http://hg.openjdk.java.net/jdk7/tl/hotspot/rev/435f0808b826 6847305: solaris reorder mapfiles generate too many warnings Reviewed-by: kvn ! make/solaris/makefiles/reorder_COMPILER1_i486 ! make/solaris/makefiles/reorder_COMPILER1_sparc ! make/solaris/makefiles/reorder_COMPILER2_amd64 ! make/solaris/makefiles/reorder_COMPILER2_sparcv9 ! make/solaris/makefiles/reorder_TIERED_i486 ! make/solaris/makefiles/reorder_TIERED_sparc Changeset: 8b0b8998e1c3 Author: never Date: 2009-06-03 15:16 -0700 URL: http://hg.openjdk.java.net/jdk7/tl/hotspot/rev/8b0b8998e1c3 Merge Changeset: 085dd9ee61aa Author: never Date: 2009-06-03 18:15 -0700 URL: http://hg.openjdk.java.net/jdk7/tl/hotspot/rev/085dd9ee61aa Merge Changeset: eacd97c88873 Author: cfang Date: 2009-06-05 10:25 -0700 URL: http://hg.openjdk.java.net/jdk7/tl/hotspot/rev/eacd97c88873 6848466: frame::frame_size() assertion failure with -XX:+DebugDeoptimization Summary: add a RegisterMap* argument to frame::frame_size() to correctly compute the sender frame Reviewed-by: never ! src/cpu/sparc/vm/frame_sparc.inline.hpp ! src/cpu/x86/vm/frame_x86.cpp ! src/share/vm/runtime/frame.hpp ! src/share/vm/runtime/vframe.cpp Changeset: 315a5d70b295 Author: iveresov Date: 2009-05-11 16:30 -0700 URL: http://hg.openjdk.java.net/jdk7/tl/hotspot/rev/315a5d70b295 6484957: G1: parallel concurrent refinement 6826318: G1: remove traversal-based refinement code Summary: Removed traversal-based refinement code as it's no longer used. Made the concurrent refinement (queue-based) parallel. Reviewed-by: tonyp ! src/cpu/sparc/vm/assembler_sparc.cpp ! src/share/vm/gc_implementation/g1/concurrentG1Refine.cpp ! src/share/vm/gc_implementation/g1/concurrentG1Refine.hpp ! src/share/vm/gc_implementation/g1/concurrentG1RefineThread.cpp ! src/share/vm/gc_implementation/g1/concurrentG1RefineThread.hpp ! src/share/vm/gc_implementation/g1/concurrentMarkThread.hpp ! src/share/vm/gc_implementation/g1/concurrentZFThread.hpp ! src/share/vm/gc_implementation/g1/dirtyCardQueue.cpp ! src/share/vm/gc_implementation/g1/g1CollectedHeap.cpp ! src/share/vm/gc_implementation/g1/g1CollectorPolicy.cpp ! src/share/vm/gc_implementation/g1/g1CollectorPolicy.hpp ! src/share/vm/gc_implementation/g1/g1RemSet.cpp ! src/share/vm/gc_implementation/g1/g1RemSet.hpp ! src/share/vm/gc_implementation/g1/g1_globals.hpp ! src/share/vm/gc_implementation/g1/heapRegionRemSet.cpp ! src/share/vm/gc_implementation/g1/heapRegionRemSet.hpp ! src/share/vm/gc_implementation/g1/ptrQueue.cpp ! src/share/vm/gc_implementation/includeDB_gc_g1 ! src/share/vm/gc_implementation/shared/concurrentGCThread.cpp ! src/share/vm/gc_implementation/shared/concurrentGCThread.hpp ! src/share/vm/memory/cardTableRS.cpp ! src/share/vm/runtime/mutexLocker.cpp ! src/share/vm/runtime/mutexLocker.hpp Changeset: 215f81b4d9b3 Author: iveresov Date: 2009-05-18 11:52 -0700 URL: http://hg.openjdk.java.net/jdk7/tl/hotspot/rev/215f81b4d9b3 6841831: G1: assert(contains_reference(from),"We just added it!") fires Summary: During parallel rset updating we have to make sure that the worker ids of the refinement threads do not intersect with the worker ids that can be claimed by the mutator threads. Reviewed-by: tonyp ! src/share/vm/gc_implementation/g1/concurrentG1Refine.cpp ! src/share/vm/gc_implementation/g1/concurrentG1Refine.hpp ! src/share/vm/gc_implementation/g1/concurrentG1RefineThread.cpp ! src/share/vm/gc_implementation/g1/concurrentG1RefineThread.hpp ! src/share/vm/gc_implementation/g1/dirtyCardQueue.cpp ! src/share/vm/gc_implementation/g1/heapRegionRemSet.cpp ! src/share/vm/gc_implementation/includeDB_gc_g1 Changeset: 29e7d79232b9 Author: apetrusenko Date: 2009-05-19 04:05 -0700 URL: http://hg.openjdk.java.net/jdk7/tl/hotspot/rev/29e7d79232b9 6819065: G1: eliminate high serial card table clearing time Reviewed-by: iveresov, tonyp ! src/share/vm/gc_implementation/g1/g1CollectedHeap.cpp ! src/share/vm/gc_implementation/g1/g1CollectedHeap.hpp ! src/share/vm/gc_implementation/g1/g1RemSet.cpp ! src/share/vm/gc_implementation/g1/heapRegion.cpp ! src/share/vm/gc_implementation/g1/heapRegion.hpp Changeset: 7fd05714f579 Author: jcoomes Date: 2009-05-26 16:43 -0700 URL: http://hg.openjdk.java.net/jdk7/tl/hotspot/rev/7fd05714f579 Merge Changeset: fe1574da39fc Author: ysr Date: 2009-06-07 00:27 -0700 URL: http://hg.openjdk.java.net/jdk7/tl/hotspot/rev/fe1574da39fc 6848641: CMSCollector::_roots_scanning_options should be initialized Summary: The field is now initialized in the constructor. Reviewed-by: iveresov, jmasa, johnc ! src/share/vm/gc_implementation/concurrentMarkSweep/concurrentMarkSweepGeneration.cpp ! src/share/vm/gc_implementation/concurrentMarkSweep/concurrentMarkSweepGeneration.hpp Changeset: f89cf529c3c7 Author: iveresov Date: 2009-06-08 16:14 -0700 URL: http://hg.openjdk.java.net/jdk7/tl/hotspot/rev/f89cf529c3c7 6849122: G1: Typo introduced during implementation of the parallel refinement Summary: Typo fix Reviewed-by: jcoomes ! src/share/vm/gc_implementation/g1/concurrentG1Refine.cpp Changeset: 7295839252de Author: jmasa Date: 2009-06-10 14:57 -0700 URL: http://hg.openjdk.java.net/jdk7/tl/hotspot/rev/7295839252de Merge Changeset: cf4f487696ba Author: trims Date: 2009-06-11 17:46 -0700 URL: http://hg.openjdk.java.net/jdk7/tl/hotspot/rev/cf4f487696ba Merge Changeset: 08f86fa55a31 Author: trims Date: 2009-06-11 17:56 -0700 URL: http://hg.openjdk.java.net/jdk7/tl/hotspot/rev/08f86fa55a31 6850551: Bump the HS16 build number to 04 Summary: Update the HS16 build number to 04 Reviewed-by: jcoomes ! make/hotspot_version Changeset: 27b728fd1281 Author: trims Date: 2009-06-11 21:01 -0700 URL: http://hg.openjdk.java.net/jdk7/tl/hotspot/rev/27b728fd1281 Merge Changeset: a88386380bda Author: xdono Date: 2009-06-18 13:05 -0700 URL: http://hg.openjdk.java.net/jdk7/tl/hotspot/rev/a88386380bda Added tag jdk7-b61 for changeset 27b728fd1281 ! .hgtags From tim.bell at sun.com Thu Jun 25 01:21:22 2009 From: tim.bell at sun.com (tim.bell at sun.com) Date: Thu, 25 Jun 2009 01:21:22 +0000 Subject: hg: jdk7/tl/jaxp: 2 new changesets Message-ID: <20090625012125.83E6CE046@hg.openjdk.java.net> Changeset: db1d07f881a1 Author: xdono Date: 2009-06-18 13:05 -0700 URL: http://hg.openjdk.java.net/jdk7/tl/jaxp/rev/db1d07f881a1 Added tag jdk7-b61 for changeset f1ac756616ea ! .hgtags Changeset: a97dd57a6260 Author: tbell Date: 2009-06-21 23:51 -0700 URL: http://hg.openjdk.java.net/jdk7/tl/jaxp/rev/a97dd57a6260 Merge From tim.bell at sun.com Thu Jun 25 01:26:03 2009 From: tim.bell at sun.com (tim.bell at sun.com) Date: Thu, 25 Jun 2009 01:26:03 +0000 Subject: hg: jdk7/tl/jaxws: 2 new changesets Message-ID: <20090625012605.DD9C7E04B@hg.openjdk.java.net> Changeset: 55681156ec1a Author: xdono Date: 2009-06-18 13:05 -0700 URL: http://hg.openjdk.java.net/jdk7/tl/jaxws/rev/55681156ec1a Added tag jdk7-b61 for changeset aeabf802f2a1 ! .hgtags Changeset: 75c801c13ea1 Author: tbell Date: 2009-06-21 23:52 -0700 URL: http://hg.openjdk.java.net/jdk7/tl/jaxws/rev/75c801c13ea1 Merge From tim.bell at sun.com Thu Jun 25 01:30:53 2009 From: tim.bell at sun.com (tim.bell at sun.com) Date: Thu, 25 Jun 2009 01:30:53 +0000 Subject: hg: jdk7/tl/jdk: 4 new changesets Message-ID: <20090625013142.9A0C5E050@hg.openjdk.java.net> Changeset: 5a5b56904855 Author: tbell Date: 2009-06-21 12:02 -0700 URL: http://hg.openjdk.java.net/jdk7/tl/jdk/rev/5a5b56904855 6853336: (process) disable or remove clone-exec feature (6850720) Summary: clone-exec feature (6850720) needs more work on 32-bit Linux Reviewed-by: alanb, michaelm Contributed-by: Martin Buchholz ! src/solaris/native/java/lang/UNIXProcess_md.c Changeset: 03f2ac812821 Author: xdono Date: 2009-06-18 13:05 -0700 URL: http://hg.openjdk.java.net/jdk7/tl/jdk/rev/03f2ac812821 Added tag jdk7-b61 for changeset f72c0dc047b9 ! .hgtags Changeset: 55a584478eac Author: tbell Date: 2009-06-21 23:52 -0700 URL: http://hg.openjdk.java.net/jdk7/tl/jdk/rev/55a584478eac Merge Changeset: b3444a42fd40 Author: tbell Date: 2009-06-23 22:07 -0700 URL: http://hg.openjdk.java.net/jdk7/tl/jdk/rev/b3444a42fd40 Merge - src/share/classes/sun/io/ByteToCharMS932DB.java - src/share/classes/sun/io/CharToByteMS932DB.java - src/share/classes/sun/nio/cs/ext/EUC_CN.java - src/share/classes/sun/nio/cs/ext/EUC_KR.java - src/share/classes/sun/nio/cs/ext/GBK.java - src/share/classes/sun/nio/cs/ext/Johab.java - src/share/classes/sun/nio/cs/ext/MS932.java - src/share/classes/sun/nio/cs/ext/MS932DB.java - src/share/classes/sun/nio/cs/ext/MS936.java - src/share/classes/sun/nio/cs/ext/MS949.java - src/share/classes/sun/nio/cs/ext/MS950.java From tim.bell at sun.com Thu Jun 25 01:43:35 2009 From: tim.bell at sun.com (tim.bell at sun.com) Date: Thu, 25 Jun 2009 01:43:35 +0000 Subject: hg: jdk7/tl/langtools: 4 new changesets Message-ID: <20090625014344.30DA3E055@hg.openjdk.java.net> Changeset: 950d50e13a9e Author: xdono Date: 2009-06-18 13:05 -0700 URL: http://hg.openjdk.java.net/jdk7/tl/langtools/rev/950d50e13a9e Added tag jdk7-b61 for changeset 522520757dd3 ! .hgtags Changeset: 6855e5aa3348 Author: tbell Date: 2009-06-21 23:55 -0700 URL: http://hg.openjdk.java.net/jdk7/tl/langtools/rev/6855e5aa3348 Merge Changeset: fe077c71cd47 Author: tbell Date: 2009-06-23 22:09 -0700 URL: http://hg.openjdk.java.net/jdk7/tl/langtools/rev/fe077c71cd47 Merge Changeset: 812d5486a023 Author: tbell Date: 2009-06-24 17:34 -0700 URL: http://hg.openjdk.java.net/jdk7/tl/langtools/rev/812d5486a023 Merge From martinrb at google.com Thu Jun 25 02:34:50 2009 From: martinrb at google.com (Martin Buchholz) Date: Wed, 24 Jun 2009 19:34:50 -0700 Subject: Miscellaneous improvements to "jar". Message-ID: <1ccfd1c10906241934i21247553w685cf0b0b4cbd7b2@mail.gmail.com> Hi jar team, I have a bunch of minor improvements to src/share/classes/sun/tools/jar/Main.java Toby and Xueming, please review. Warning: the index code has not been maintained for many years. Xueming, please file a bug. Synopsis: Miscellaneous improvements to "jar". Description: - Use standard jdk coding style for javadoc - Don't create a temp file for jar index in STORED mode. - Don't use synchronized collections. - Fix javac warnings. - Don't define new names for things like INDEX_NAME; use static import instead. - more efficiently compare special file names in update mode. Update mode should be measurably faster. - make CRC32OutputStream a nested class. refactor crc32.reset and updating entry into CRC32OutputStream. - Fix apparently benign bug updating n in CRC32OutputStream.write(byte[], int, int) Evaluation: Yep. http://cr.openjdk.java.net/~martin/jar-misc/ Thanks, Martin -------------- next part -------------- An HTML attachment was scrubbed... URL: From Xueming.Shen at Sun.COM Thu Jun 25 03:17:51 2009 From: Xueming.Shen at Sun.COM (Xueming Shen) Date: Wed, 24 Jun 2009 20:17:51 -0700 Subject: Miscellaneous improvements to "jar". In-Reply-To: <1ccfd1c10906241934i21247553w685cf0b0b4cbd7b2@mail.gmail.com> References: <1ccfd1c10906241934i21247553w685cf0b0b4cbd7b2@mail.gmail.com> Message-ID: <4A42EC5F.4080808@sun.com> 6854795 Martin Buchholz wrote: > Hi jar team, > > I have a bunch of minor improvements to > src/share/classes/sun/tools/jar/Main.java > > Toby and Xueming, please review. > > Warning: the index code has not been maintained for many years. > > Xueming, please file a bug. > > Synopsis: Miscellaneous improvements to "jar". > Description: > - Use standard jdk coding style for javadoc > - Don't create a temp file for jar index in STORED mode. > - Don't use synchronized collections. > - Fix javac warnings. > - Don't define new names for things like INDEX_NAME; > use static import instead. > - more efficiently compare special file names in update mode. > Update mode should be measurably faster. > - make CRC32OutputStream a nested class. > refactor crc32.reset and updating entry into CRC32OutputStream. > - Fix apparently benign bug updating n in > CRC32OutputStream.write(byte[], int, int) > > Evaluation: Yep. > > http://cr.openjdk.java.net/~martin/jar-misc/ > > > Thanks, > > Martin From Ulf.Zibis at gmx.de Thu Jun 25 13:20:05 2009 From: Ulf.Zibis at gmx.de (Ulf Zibis) Date: Thu, 25 Jun 2009 15:20:05 +0200 Subject: Miscellaneous improvements to "jar". In-Reply-To: <1ccfd1c10906241934i21247553w685cf0b0b4cbd7b2@mail.gmail.com> References: <1ccfd1c10906241934i21247553w685cf0b0b4cbd7b2@mail.gmail.com> Message-ID: <4A437985.5000001@gmx.de> Martin, 1. typo in line 1188 2. Is it convention for javadoc to have 2 blanks after period? 3. I guess your change is targeted on performance. IMHO jar read/write itself should be improved: http://bugs.sun.com/bugdatabase/view_bug.do?bug_id=6818737 Ulf Am 25.06.2009 04:34, Martin Buchholz schrieb: > Hi jar team, > > I have a bunch of minor improvements to > src/share/classes/sun/tools/jar/Main.java > > Toby and Xueming, please review. > > Warning: the index code has not been maintained for many years. > > Xueming, please file a bug. > > Synopsis: Miscellaneous improvements to "jar". > Description: > - Use standard jdk coding style for javadoc > - Don't create a temp file for jar index in STORED mode. > - Don't use synchronized collections. > - Fix javac warnings. > - Don't define new names for things like INDEX_NAME; > use static import instead. > - more efficiently compare special file names in update mode. > Update mode should be measurably faster. > - make CRC32OutputStream a nested class. > refactor crc32.reset and updating entry into CRC32OutputStream. > - Fix apparently benign bug updating n in > CRC32OutputStream.write(byte[], int, int) > > Evaluation: Yep. > > http://cr.openjdk.java.net/~martin/jar-misc/ > > > Thanks, > > Martin From Ulf.Zibis at gmx.de Thu Jun 25 13:37:44 2009 From: Ulf.Zibis at gmx.de (Ulf Zibis) Date: Thu, 25 Jun 2009 15:37:44 +0200 Subject: Miscellaneous improvements to "jar". /typo correction In-Reply-To: <4A437985.5000001@gmx.de> References: <1ccfd1c10906241934i21247553w685cf0b0b4cbd7b2@mail.gmail.com> <4A437985.5000001@gmx.de> Message-ID: <4A437DA8.9040307@gmx.de> Oops, typo in typo complaint: Typo is in line 1088. -Ulf Am 25.06.2009 15:20, Ulf Zibis schrieb: > Martin, > > 1. typo in line 1188 > 2. Is it convention for javadoc to have 2 blanks after period? > 3. I guess your change is targeted on performance. IMHO jar read/write > itself should be improved: > http://bugs.sun.com/bugdatabase/view_bug.do?bug_id=6818737 > > Ulf > > > Am 25.06.2009 04:34, Martin Buchholz schrieb: >> Hi jar team, >> >> I have a bunch of minor improvements to >> src/share/classes/sun/tools/jar/Main.java >> >> Toby and Xueming, please review. >> >> Warning: the index code has not been maintained for many years. >> >> Xueming, please file a bug. >> >> Synopsis: Miscellaneous improvements to "jar". >> Description: >> - Use standard jdk coding style for javadoc >> - Don't create a temp file for jar index in STORED mode. >> - Don't use synchronized collections. >> - Fix javac warnings. >> - Don't define new names for things like INDEX_NAME; >> use static import instead. >> - more efficiently compare special file names in update mode. >> Update mode should be measurably faster. >> - make CRC32OutputStream a nested class. >> refactor crc32.reset and updating entry into CRC32OutputStream. >> - Fix apparently benign bug updating n in >> CRC32OutputStream.write(byte[], int, int) >> >> Evaluation: Yep. >> >> http://cr.openjdk.java.net/~martin/jar-misc/ >> >> >> Thanks, >> >> Martin > > From gustav.trede at gmail.com Thu Jun 25 13:45:46 2009 From: gustav.trede at gmail.com (gustav trede) Date: Thu, 25 Jun 2009 15:45:46 +0200 Subject: toggle for intrisinics Message-ID: <311e0eaf0906250645i764f9b3cid882882deb00938d@mail.gmail.com> Hello, Is it from a technical standpoint possible to: Add an -XX parameter that toggles direct intrinsics to the cpu native version of some Math methods like cos , sin without any extra (parameter range check with its fast and slow code paths). If so i would be happy to implement it myself , with some minor initial guideline to the codebase please. Despite breaking spec, the non exact results are good enough for many practical uses, like 3D engines etc where competitive performance with other languages is of importance. It would be nice if such a patch could be integrated into the jdk itself so others can use it, else i will have to bundle a private compiled version with my application. -- regards gustav trede -------------- next part -------------- An HTML attachment was scrubbed... URL: From Alan.Bateman at Sun.COM Thu Jun 25 14:11:49 2009 From: Alan.Bateman at Sun.COM (Alan Bateman) Date: Thu, 25 Jun 2009 15:11:49 +0100 Subject: toggle for intrisinics In-Reply-To: <311e0eaf0906250645i764f9b3cid882882deb00938d@mail.gmail.com> References: <311e0eaf0906250645i764f9b3cid882882deb00938d@mail.gmail.com> Message-ID: <4A4385A5.7090800@sun.com> gustav trede wrote: > Hello, > > Is it from a technical standpoint possible to: > > Add an -XX parameter that toggles direct intrinsics to the cpu native > version of some Math methods like cos , sin without any extra > (parameter range check with its fast and slow code paths). > : > There are options, such as InlineMathNatives, that might point you to the right code but it would be better to mail hotspot-compile-dev for more authoritative pointers. -Alan. From martinrb at google.com Thu Jun 25 15:12:43 2009 From: martinrb at google.com (Martin Buchholz) Date: Thu, 25 Jun 2009 08:12:43 -0700 Subject: Miscellaneous improvements to "jar". In-Reply-To: <4A437985.5000001@gmx.de> References: <1ccfd1c10906241934i21247553w685cf0b0b4cbd7b2@mail.gmail.com> <4A437985.5000001@gmx.de> Message-ID: <1ccfd1c10906250812i239a4cc7q7b5e11753542c33f@mail.gmail.com> Ulf, Thanks for the review. On Thu, Jun 25, 2009 at 06:20, Ulf Zibis wrote: > Martin, > > 1. typo in line 1188 Done. > > 2. Is it convention for javadoc to have 2 blanks after period? This is not consistent in the JDK. I do it myself, partially because I've been trained by emacs. > > 3. I guess your change is targeted on performance. IMHO jar read/write > itself should be improved: > http://bugs.sun.com/bugdatabase/view_bug.do?bug_id=6818737 > Many things to improve... I'm not planning to work on that one myself. Martin > > Ulf > > > Am 25.06.2009 04:34, Martin Buchholz schrieb: > >> Hi jar team, >> >> I have a bunch of minor improvements to >> src/share/classes/sun/tools/jar/Main.java >> >> Toby and Xueming, please review. >> >> Warning: the index code has not been maintained for many years. >> >> Xueming, please file a bug. >> >> Synopsis: Miscellaneous improvements to "jar". >> Description: >> - Use standard jdk coding style for javadoc >> - Don't create a temp file for jar index in STORED mode. >> - Don't use synchronized collections. >> - Fix javac warnings. >> - Don't define new names for things like INDEX_NAME; >> use static import instead. >> - more efficiently compare special file names in update mode. >> Update mode should be measurably faster. >> - make CRC32OutputStream a nested class. >> refactor crc32.reset and updating entry into CRC32OutputStream. >> - Fix apparently benign bug updating n in CRC32OutputStream.write(byte[], >> int, int) >> >> Evaluation: Yep. >> >> http://cr.openjdk.java.net/~martin/jar-misc/< >> http://cr.openjdk.java.net/%7Emartin/jar-misc/> >> >> Thanks, >> >> Martin >> > > -------------- next part -------------- An HTML attachment was scrubbed... URL: From Xueming.Shen at Sun.COM Thu Jun 25 16:12:42 2009 From: Xueming.Shen at Sun.COM (Xueming Shen) Date: Thu, 25 Jun 2009 09:12:42 -0700 Subject: Miscellaneous improvements to "jar". In-Reply-To: <1ccfd1c10906241934i21247553w685cf0b0b4cbd7b2@mail.gmail.com> References: <1ccfd1c10906241934i21247553w685cf0b0b4cbd7b2@mail.gmail.com> Message-ID: <4A43A1FA.4040809@sun.com> do we have a Turkish i "regression" for the index name comparing by assuming ASCII only? yes, the original impl is not consistent already when dealing with manifest and index , one use toUpperCase(Locale.ENGLISH), one does not. Martin Buchholz wrote: > Hi jar team, > > I have a bunch of minor improvements to > src/share/classes/sun/tools/jar/Main.java > > Toby and Xueming, please review. > > Warning: the index code has not been maintained for many years. > > Xueming, please file a bug. > > Synopsis: Miscellaneous improvements to "jar". > Description: > - Use standard jdk coding style for javadoc > - Don't create a temp file for jar index in STORED mode. > - Don't use synchronized collections. > - Fix javac warnings. > - Don't define new names for things like INDEX_NAME; > use static import instead. > - more efficiently compare special file names in update mode. > Update mode should be measurably faster. > - make CRC32OutputStream a nested class. > refactor crc32.reset and updating entry into CRC32OutputStream. > - Fix apparently benign bug updating n in > CRC32OutputStream.write(byte[], int, int) > > Evaluation: Yep. > > http://cr.openjdk.java.net/~martin/jar-misc/ > > > Thanks, > > Martin From jean-christophe.collet at sun.com Thu Jun 25 17:07:50 2009 From: jean-christophe.collet at sun.com (jean-christophe.collet at sun.com) Date: Thu, 25 Jun 2009 17:07:50 +0000 Subject: hg: jdk7/tl/jdk: 6811297: Add more logging to HTTP protocol handler Message-ID: <20090625170859.70BC6E0FA@hg.openjdk.java.net> Changeset: 70c0a927e21a Author: jccollet Date: 2009-06-25 18:56 +0200 URL: http://hg.openjdk.java.net/jdk7/tl/jdk/rev/70c0a927e21a 6811297: Add more logging to HTTP protocol handler Summary: Added extra logging to HttpURLConnection and HttpClient. Added a capture tool. Reviewed-by: chegar ! make/sun/net/FILES_java.gmk + src/share/classes/sun/net/www/http/HttpCapture.java + src/share/classes/sun/net/www/http/HttpCaptureInputStream.java + src/share/classes/sun/net/www/http/HttpCaptureOutputStream.java ! src/share/classes/sun/net/www/http/HttpClient.java + src/share/classes/sun/net/www/protocol/http/HttpLogFormatter.java ! src/share/classes/sun/net/www/protocol/http/HttpURLConnection.java From jim.andreou at gmail.com Thu Jun 25 17:17:56 2009 From: jim.andreou at gmail.com (Jim Andreou) Date: Thu, 25 Jun 2009 20:17:56 +0300 Subject: hg: jdk7/tl/jdk: 6811297: Add more logging to HTTP protocol handler In-Reply-To: <20090625170859.70BC6E0FA@hg.openjdk.java.net> References: <20090625170859.70BC6E0FA@hg.openjdk.java.net> Message-ID: <7d7138c10906251017t4aafc6c5r6939ac979e35cc3b@mail.gmail.com> Hi, A quick note: Wouldn't it be nice if clients could add URL handling interceptors and monitor incoming/outgoing data themselves? For any URL protocol they would be interested into, not http in particular. Recently I looked into the existing URL protocol handling architecture (e.g. [1], which promises a new era for protocol handlers, but it seems that the existing support is cumbersome and restricting - sorry, I can't elaborate right now) but it seems to make any such scenario impossible. Any comments? Thanks, Dimitris Andreou [1] http://java.sun.com/developer/onlineTraining/protocolhandlers/ 2009/6/25 > Changeset: 70c0a927e21a > Author: jccollet > Date: 2009-06-25 18:56 +0200 > URL: http://hg.openjdk.java.net/jdk7/tl/jdk/rev/70c0a927e21a > > 6811297: Add more logging to HTTP protocol handler > Summary: Added extra logging to HttpURLConnection and HttpClient. Added a > capture tool. > Reviewed-by: chegar > > ! make/sun/net/FILES_java.gmk > + src/share/classes/sun/net/www/http/HttpCapture.java > + src/share/classes/sun/net/www/http/HttpCaptureInputStream.java > + src/share/classes/sun/net/www/http/HttpCaptureOutputStream.java > ! src/share/classes/sun/net/www/http/HttpClient.java > + src/share/classes/sun/net/www/protocol/http/HttpLogFormatter.java > ! src/share/classes/sun/net/www/protocol/http/HttpURLConnection.java > > -------------- next part -------------- An HTML attachment was scrubbed... URL: From martinrb at google.com Thu Jun 25 17:50:36 2009 From: martinrb at google.com (Martin Buchholz) Date: Thu, 25 Jun 2009 10:50:36 -0700 Subject: Miscellaneous improvements to "jar". In-Reply-To: <4A43A1FA.4040809@sun.com> References: <1ccfd1c10906241934i21247553w685cf0b0b4cbd7b2@mail.gmail.com> <4A43A1FA.4040809@sun.com> Message-ID: <1ccfd1c10906251050l3597ad9aw1f80e7e0bcc71e1@mail.gmail.com> Xueming, Congratulations for finding the incompatible change! Yes, if you are running in a turkish locale (unlikely) and if your jar file has the index entry in lower case, (unlikely - hopefully INDEX.LIST files are all machine generated). then it will not be found in the old implementation, and will be found in the new one. Can we consider this a simple bug fix? Please? Even if all of the above occurs, the change in behavior may be benign - the resulting jar file will contain both upper and lower case index files. Perhaps I should add a test, but it would be painful to write one. It was never a good idea to make these entry names case-insensitive. Martin On Thu, Jun 25, 2009 at 09:12, Xueming Shen wrote: > > do we have a Turkish i "regression" for the index name comparing by > assuming ASCII only? > yes, the original impl is not consistent already when dealing with manifest > and index , one use > toUpperCase(Locale.ENGLISH), one does not. > > > Martin Buchholz wrote: > >> Hi jar team, >> >> I have a bunch of minor improvements to >> src/share/classes/sun/tools/jar/Main.java >> >> Toby and Xueming, please review. >> >> Warning: the index code has not been maintained for many years. >> >> Xueming, please file a bug. >> >> Synopsis: Miscellaneous improvements to "jar". >> Description: >> - Use standard jdk coding style for javadoc >> - Don't create a temp file for jar index in STORED mode. >> - Don't use synchronized collections. >> - Fix javac warnings. >> - Don't define new names for things like INDEX_NAME; >> use static import instead. >> - more efficiently compare special file names in update mode. >> Update mode should be measurably faster. >> - make CRC32OutputStream a nested class. >> refactor crc32.reset and updating entry into CRC32OutputStream. >> - Fix apparently benign bug updating n in CRC32OutputStream.write(byte[], >> int, int) >> >> Evaluation: Yep. >> >> http://cr.openjdk.java.net/~martin/jar-misc/< >> http://cr.openjdk.java.net/%7Emartin/jar-misc/> >> >> Thanks, >> >> Martin >> > > -------------- next part -------------- An HTML attachment was scrubbed... URL: From Xueming.Shen at Sun.COM Thu Jun 25 18:28:00 2009 From: Xueming.Shen at Sun.COM (Xueming Shen) Date: Thu, 25 Jun 2009 11:28:00 -0700 Subject: Miscellaneous improvements to "jar". In-Reply-To: <1ccfd1c10906251050l3597ad9aw1f80e7e0bcc71e1@mail.gmail.com> References: <1ccfd1c10906241934i21247553w685cf0b0b4cbd7b2@mail.gmail.com> <4A43A1FA.4040809@sun.com> <1ccfd1c10906251050l3597ad9aw1f80e7e0bcc71e1@mail.gmail.com> Message-ID: <4A43C1B0.6000901@sun.com> Martin Buchholz wrote: > Xueming, > > Congratulations for finding the incompatible change! > > Yes, if you are running in a turkish locale (unlikely) > and if your jar file has the index entry in lower case, > (unlikely - hopefully INDEX.LIST files are all machine generated). > then it will not be found in the old implementation, > and will be found in the new one. > Can we consider this a simple bug fix? Please? It's still a "regression" because a "-dotless i-ndex.l-dotless i-st" now is skipped in:-) I agree it's "unlikely". Given we are ignoring the same issue in manifest name in the existing implementation for so many years without a single complain (yes, we have one "i" in META-INF and another one in MANIFEST.MF:-)). I'm OK with the change, it might be better to add one simple comment. Sherman > Even if all of the above occurs, the change in behavior may be benign - > the resulting jar file will contain both upper and lower case index files. > Perhaps I should add a test, but it would be painful to write one. > From martinrb at google.com Thu Jun 25 19:00:32 2009 From: martinrb at google.com (Martin Buchholz) Date: Thu, 25 Jun 2009 12:00:32 -0700 Subject: Miscellaneous improvements to "jar". In-Reply-To: <1ccfd1c10906241934i21247553w685cf0b0b4cbd7b2@mail.gmail.com> References: <1ccfd1c10906241934i21247553w685cf0b0b4cbd7b2@mail.gmail.com> Message-ID: <1ccfd1c10906251200n3bd78beek400abb51adcece40@mail.gmail.com> I have an updated version of this fix, with these changes: - Documented the turkish i problem /** * Compares two strings for equality, ignoring case. The second * argument must contain only upper-case ASCII characters. * We don't want case comparison to be locale-dependent (else we * have the notorious "turkish i bug"). */ private boolean equalsIgnoreCase(String s, String upper) { - Refactored code so that updateEntry now also sets the method to STORED. /** * Updates a ZipEntry which describes the data read by this * output stream, in STORED mode. */ public void updateEntry(ZipEntry e) { e.setMethod(ZipEntry.STORED); e.setSize(n); e.setCrc(crc.getValue()); } - addIndex was never updating the size in the ZipEntry (as required), which was not previously noticed because closeEntry was never called. private void addIndex(JarIndex index, ZipOutputStream zos) throws IOException { ZipEntry e = new ZipEntry(INDEX_NAME); e.setTime(System.currentTimeMillis()); if (flag0) { CRC32OutputStream os = new CRC32OutputStream(crc32); index.write(os); os.updateEntry(e); } zos.putNextEntry(e); index.write(zos); zos.closeEntry(); } http://cr.openjdk.java.net/~martin/jar-misc/ Previous webrev: http://cr.openjdk.java.net/~martin/jar-misc.0/ Martin On Wed, Jun 24, 2009 at 19:34, Martin Buchholz wrote: > Hi jar team, > > I have a bunch of minor improvements to > src/share/classes/sun/tools/jar/Main.java > > Toby and Xueming, please review. > > Warning: the index code has not been maintained for many years. > > Xueming, please file a bug. > > Synopsis: Miscellaneous improvements to "jar". > Description: > - Use standard jdk coding style for javadoc > - Don't create a temp file for jar index in STORED mode. > - Don't use synchronized collections. > - Fix javac warnings. > - Don't define new names for things like INDEX_NAME; > use static import instead. > - more efficiently compare special file names in update mode. > Update mode should be measurably faster. > - make CRC32OutputStream a nested class. > refactor crc32.reset and updating entry into CRC32OutputStream. > - Fix apparently benign bug updating n in CRC32OutputStream.write(byte[], > int, int) > > Evaluation: Yep. > > http://cr.openjdk.java.net/~martin/jar-misc/ > > Thanks, > > Martin > -------------- next part -------------- An HTML attachment was scrubbed... URL: From Xueming.Shen at Sun.COM Thu Jun 25 20:04:25 2009 From: Xueming.Shen at Sun.COM (Xueming Shen) Date: Thu, 25 Jun 2009 13:04:25 -0700 Subject: Miscellaneous improvements to "jar". In-Reply-To: <1ccfd1c10906251200n3bd78beek400abb51adcece40@mail.gmail.com> References: <1ccfd1c10906241934i21247553w685cf0b0b4cbd7b2@mail.gmail.com> <1ccfd1c10906251200n3bd78beek400abb51adcece40@mail.gmail.com> Message-ID: <4A43D849.9030202@sun.com> Martin Buchholz wrote: > I have an updated version of this fix, with these changes: > > - Documented the turkish i problem > > /** > * Compares two strings for equality, ignoring case. The second > * argument must contain only upper-case ASCII characters. > * We don't want case comparison to be locale-dependent (else we > * have the notorious "turkish i bug"). > */ > private boolean equalsIgnoreCase(String s, String upper) { > > - Refactored code so that updateEntry now also sets the method to STORED. > > /** > * Updates a ZipEntry which describes the data read by this > * output stream, in STORED mode. > */ > public void updateEntry(ZipEntry e) { > e.setMethod(ZipEntry.STORED); > e.setSize(n); > e.setCrc(crc.getValue()); > } > > - addIndex was never updating the size in the ZipEntry (as required), > which was not previously noticed because closeEntry was never called. > > private void addIndex(JarIndex index, ZipOutputStream zos) > throws IOException > { > ZipEntry e = new ZipEntry(INDEX_NAME); > e.setTime(System.currentTimeMillis()); > if (flag0) { > CRC32OutputStream os = new CRC32OutputStream(crc32); > index.write(os); > os.updateEntry(e); > } > zos.putNextEntry(e); > index.write(zos); > zos.closeEntry(); > } > > http://cr.openjdk.java.net/~martin/jar-misc/ > > Previous webrev: > http://cr.openjdk.java.net/~martin/jar-misc.0/ > > > Martin > yes, it now solved the puzzle:-) (1)closeEntry() is not strictly necessary, either putNextEntry or close() will close the previous entry (2)you might want to go a little further to "encapsulate" the updateEntry into crc32os, pass the entry into the constructor and setup the value with a crc32os.close()...bad idea? btw, sherman From langel at redhat.com Thu Jun 25 21:09:48 2009 From: langel at redhat.com (langel at redhat.com) Date: Thu, 25 Jun 2009 21:09:48 +0000 Subject: hg: jdk7/tl/jdk: 6852607: MessageUtils JVM crash Message-ID: <20090625211117.7B6F8E15A@hg.openjdk.java.net> Changeset: 5a3a5388756c Author: langel Date: 2009-06-25 17:01 -0400 URL: http://hg.openjdk.java.net/jdk7/tl/jdk/rev/5a3a5388756c 6852607: MessageUtils JVM crash Summary: Fixes crash by checking null field Reviewed-by: alanb ! src/share/native/sun/misc/MessageUtils.c From martinrb at google.com Fri Jun 26 00:33:03 2009 From: martinrb at google.com (Martin Buchholz) Date: Thu, 25 Jun 2009 17:33:03 -0700 Subject: Miscellaneous improvements to "jar". In-Reply-To: <4A43D849.9030202@sun.com> References: <1ccfd1c10906241934i21247553w685cf0b0b4cbd7b2@mail.gmail.com> <1ccfd1c10906251200n3bd78beek400abb51adcece40@mail.gmail.com> <4A43D849.9030202@sun.com> Message-ID: <1ccfd1c10906251733g5de1812ckd03e3b62763fa7a4@mail.gmail.com> On Thu, Jun 25, 2009 at 13:04, Xueming Shen wrote: > >> yes, it now solved the puzzle:-) > > (1)closeEntry() is not strictly necessary, either putNextEntry or close() > will close the previous entry > (2)you might want to go a little further to "encapsulate" the updateEntry > into crc32os, pass the entry > into the constructor and setup the value with a crc32os.close()...bad idea? > It's somewhat magical to have close to this, and close is not currently called automatically anywhere. explicit updateEntry is at least easy to understand. Martin -------------- next part -------------- An HTML attachment was scrubbed... URL: From martinrb at google.com Fri Jun 26 00:57:34 2009 From: martinrb at google.com (Martin Buchholz) Date: Thu, 25 Jun 2009 17:57:34 -0700 Subject: Miscellaneous improvements to "jar". In-Reply-To: <1ccfd1c10906251200n3bd78beek400abb51adcece40@mail.gmail.com> References: <1ccfd1c10906241934i21247553w685cf0b0b4cbd7b2@mail.gmail.com> <1ccfd1c10906251200n3bd78beek400abb51adcece40@mail.gmail.com> Message-ID: <1ccfd1c10906251757y72ccaca1l289dfc79f18b002c@mail.gmail.com> I did some benchmarking, and found that my changes "only" make jar 10-20% faster. Disappointing - we expect an order of magnitude for every commit! While benchmarking, I discovered to my horror that the simple jar cf /tmp/t1 ... jar i /tmp/t1 fails, because it tries to create the replacement jar in "." and then rename() it into place. Oops... Better refactor out the code that puts the replacement temp file in the same directory. Better write some tests for that, too. /** * A variant of File.getParentFile that always returns a valid * directory (i.e. returns new File(".") where File.getParentFile * would return null). */ private static File directoryOf(File file) { File dir = file.getParentFile(); return (dir != null) ? dir : new File("."); } /** * Creates a new empty temporary file in the same directory as the * specified file. A variant of File.createTempFile. */ private static File createTempFileInSameDirectoryAs(File file) throws IOException { return File.createTempFile("jartmp", null, directoryOf(file)); } --- While we're here, let's remove the double buffering on file copy operations. And the repeated allocations of big buffers. /** * A buffer for use only by copy(InputStream, OutputStream). * Not as clean as allocating a new buffer as needed by copy, * but significantly more efficient. */ private byte[] copyBuf = new byte[8192]; /** * Copies all bytes from the input stream to the output stream. * Does not close or flush either stream. * * @param from the input stream to read from * @param to the output stream to write to * @throws IOException if an I/O error occurs */ private void copy(InputStream from, OutputStream to) throws IOException { int n; while ((n = from.read(copyBuf)) != -1) to.write(copyBuf, 0, n); } /** * Copies all bytes from the input file to the output stream. * Does not close or flush the output stream. * * @param from the input file to read from * @param to the output stream to write to * @throws IOException if an I/O error occurs */ private void copy(File from, OutputStream to) throws IOException { InputStream in = new FileInputStream(from); try { copy(in, to); } finally { in.close(); } } /** * Copies all bytes from the input stream to the output file. * Does not close the input stream. * * @param from the input stream to read from * @param to the output file to write to * @throws IOException if an I/O error occurs */ private void copy(InputStream from, File to) throws IOException { OutputStream out = new FileOutputStream(to); try { copy(from, out); } finally { out.close(); } } ---- On the other hand, allocating a CRC32 is very cheap, so let's make that private to CRC32OutputStream. + private static class CRC32OutputStream extends java.io.OutputStream { + final CRC32 crc = new CRC32(); ---- We should add a try { finally } to delete the tempfile for jar i + try { boolean updateOk = update(new FileInputStream(jarFile), - new FileOutputStream(scratchFile), + new FileOutputStream(tmpFile), null, index); + if (updateOk) { jarFile.delete(); - if (!scratchFile.renameTo(jarFile)) { - scratchFile.delete(); + if (!tmpFile.renameTo(jarFile)) { throw new IOException(getMsg("error.write.file")); } - scratchFile.delete(); + } + } finally { + tmpFile.delete(); + } } ---- Webrev updated. http://cr.openjdk.java.net/~martin/jar-misc/ There are now many small changes combined in this fix. Sorry about that. I'd be more inclined to separate them if creating new bugs was easier. I'm not planning on making any more changes. Really. Martin On Thu, Jun 25, 2009 at 12:00, Martin Buchholz wrote: > I have an updated version of this fix, with these changes: > > - Documented the turkish i problem > > /** > * Compares two strings for equality, ignoring case. The second > * argument must contain only upper-case ASCII characters. > * We don't want case comparison to be locale-dependent (else we > * have the notorious "turkish i bug"). > */ > private boolean equalsIgnoreCase(String s, String upper) { > > - Refactored code so that updateEntry now also sets the method to STORED. > > /** > * Updates a ZipEntry which describes the data read by this > * output stream, in STORED mode. > */ > public void updateEntry(ZipEntry e) { > e.setMethod(ZipEntry.STORED); > e.setSize(n); > e.setCrc(crc.getValue()); > } > > - addIndex was never updating the size in the ZipEntry (as required), > which was not previously noticed because closeEntry was never called. > > private void addIndex(JarIndex index, ZipOutputStream zos) > throws IOException > { > ZipEntry e = new ZipEntry(INDEX_NAME); > e.setTime(System.currentTimeMillis()); > if (flag0) { > CRC32OutputStream os = new CRC32OutputStream(crc32); > index.write(os); > os.updateEntry(e); > } > zos.putNextEntry(e); > index.write(zos); > zos.closeEntry(); > } > > http://cr.openjdk.java.net/~martin/jar-misc/ > Previous webrev: > http://cr.openjdk.java.net/~martin/jar-misc.0/ > > Martin > > > > On Wed, Jun 24, 2009 at 19:34, Martin Buchholz wrote: > >> Hi jar team, >> >> I have a bunch of minor improvements to >> src/share/classes/sun/tools/jar/Main.java >> >> Toby and Xueming, please review. >> >> Warning: the index code has not been maintained for many years. >> >> Xueming, please file a bug. >> >> Synopsis: Miscellaneous improvements to "jar". >> Description: >> - Use standard jdk coding style for javadoc >> - Don't create a temp file for jar index in STORED mode. >> - Don't use synchronized collections. >> - Fix javac warnings. >> - Don't define new names for things like INDEX_NAME; >> use static import instead. >> - more efficiently compare special file names in update mode. >> Update mode should be measurably faster. >> - make CRC32OutputStream a nested class. >> refactor crc32.reset and updating entry into CRC32OutputStream. >> - Fix apparently benign bug updating n in CRC32OutputStream.write(byte[], >> int, int) >> >> Evaluation: Yep. >> >> http://cr.openjdk.java.net/~martin/jar-misc/ >> >> Thanks, >> >> Martin >> > > -------------- next part -------------- An HTML attachment was scrubbed... URL: From martinrb at google.com Fri Jun 26 01:14:44 2009 From: martinrb at google.com (Martin Buchholz) Date: Thu, 25 Jun 2009 18:14:44 -0700 Subject: Miscellaneous improvements to "jar". In-Reply-To: <1ccfd1c10906251757y72ccaca1l289dfc79f18b002c@mail.gmail.com> References: <1ccfd1c10906241934i21247553w685cf0b0b4cbd7b2@mail.gmail.com> <1ccfd1c10906251200n3bd78beek400abb51adcece40@mail.gmail.com> <1ccfd1c10906251757y72ccaca1l289dfc79f18b002c@mail.gmail.com> Message-ID: <1ccfd1c10906251814k45e7e3a9k17f86719677fb9a1@mail.gmail.com> I forgot to mention that I upgraded the "n" field in CRC32OutputStream from int to long. That might be a bug fix for when the input file is longer than Integer.MAX_VALUE, which is possible even without ZIP64 support. - int n = 0; + long n = 0; Martin -------------- next part -------------- An HTML attachment was scrubbed... URL: From Ulf.Zibis at gmx.de Fri Jun 26 08:37:39 2009 From: Ulf.Zibis at gmx.de (Ulf Zibis) Date: Fri, 26 Jun 2009 10:37:39 +0200 Subject: Miscellaneous improvements to "jar". In-Reply-To: <1ccfd1c10906251757y72ccaca1l289dfc79f18b002c@mail.gmail.com> References: <1ccfd1c10906241934i21247553w685cf0b0b4cbd7b2@mail.gmail.com> <1ccfd1c10906251200n3bd78beek400abb51adcece40@mail.gmail.com> <1ccfd1c10906251757y72ccaca1l289dfc79f18b002c@mail.gmail.com> Message-ID: <4A4488D3.9030407@gmx.de> Am 26.06.2009 02:57, Martin Buchholz schrieb: > I did some benchmarking, > and found that my changes "only" make jar 10-20% faster. > Disappointing - we expect an order of magnitude for every commit! 1. Hopefully some volunteer would be found to fix http://bugs.sun.com/bugdatabase/view_bug.do?bug_id=6818737 before JDK7 API-freeze. Especially, if jar is not compressed, as in case of normal JDK installation, reading entries from jar should be much faster through java.nio.channels, than via BuffererdInputStream. > > While benchmarking, I discovered to my horror that the simple > > jar cf /tmp/t1 ... > jar i /tmp/t1 > > fails, because it tries to create the replacement jar in "." and then > rename() it into place. Oops... Better refactor out the code that > puts the replacement temp file in the same directory. > Better write some tests for that, too. 2. I don't like to refactor out the code in case of only once used, and only to better "comment" what the 2 lines are doing. It blows up the code, and following the code demands annoying scrolling. Better add additional comment to original code. 3. What happens, if original file is exactly named "jartmp" I think you would better add ".tmp" at the end of the filename, and remove it later. Does your new code work with? : jar cf /jartmp/t1 ... jar i /jartmp/t1 -Ulf From Alan.Bateman at Sun.COM Fri Jun 26 12:23:54 2009 From: Alan.Bateman at Sun.COM (Alan Bateman) Date: Fri, 26 Jun 2009 13:23:54 +0100 Subject: jdk7/tl breakage Message-ID: <4A44BDDA.2020602@sun.com> Just a heads up that jdk7/tl/jdk isn't currently buildable on Windows. One of the recent changeset seems to have accidentally deleted the hooks in the http protocol handler that are needed for NTLM authentication. Jessie is working on a fix now so if you had planned to sync up your forest today, you might want to hold off until this is fixed. -Alan. From martinrb at google.com Fri Jun 26 13:53:09 2009 From: martinrb at google.com (Martin Buchholz) Date: Fri, 26 Jun 2009 06:53:09 -0700 Subject: Miscellaneous improvements to "jar". In-Reply-To: <4A4488D3.9030407@gmx.de> References: <1ccfd1c10906241934i21247553w685cf0b0b4cbd7b2@mail.gmail.com> <1ccfd1c10906251200n3bd78beek400abb51adcece40@mail.gmail.com> <1ccfd1c10906251757y72ccaca1l289dfc79f18b002c@mail.gmail.com> <4A4488D3.9030407@gmx.de> Message-ID: <1ccfd1c10906260653j2f91a7ffx4a3056d67129f6a0@mail.gmail.com> On Fri, Jun 26, 2009 at 01:37, Ulf Zibis wrote: > Am 26.06.2009 02:57, Martin Buchholz schrieb: > >> I did some benchmarking, >> and found that my changes "only" make jar 10-20% faster. >> Disappointing - we expect an order of magnitude for every commit! >> > > 1. Hopefully some volunteer would be found to fix > http://bugs.sun.com/bugdatabase/view_bug.do?bug_id=6818737 > before JDK7 API-freeze. > Especially, if jar is not compressed, as in case of normal JDK > installation, reading entries from jar should be much faster through > java.nio.channels, than via BuffererdInputStream. > The way to motivate us around here is to provide the prototype implementation that demonstrates the speedup. > > > >> While benchmarking, I discovered to my horror that the simple >> >> jar cf /tmp/t1 ... >> jar i /tmp/t1 >> >> fails, because it tries to create the replacement jar in "." and then >> rename() it into place. Oops... Better refactor out the code that >> puts the replacement temp file in the same directory. >> Better write some tests for that, too. >> > > 2. I don't like to refactor out the code in case of only once used, and > only to better "comment" what the 2 lines are doing. > It blows up the code, and following the code demands annoying scrolling. > Better add additional comment to original code. > The original code created temp files in *two* places, and did it differently. I think the name createTempFileInSameDirectoryAs makes the current code much clearer. Also, JITs tend to be very good at inlining. > > 3. What happens, if original file is exactly named "jartmp" > I think you would better add ".tmp" at the end of the filename, and remove > it later. > Does your new code work with? : > jar cf /jartmp/t1 ... > jar i /jartmp/t1 > File.createTempFile doesn't literally create a file named jartmp. That's only the prefix. And it promises to return a freshly created empty file. Martin > -Ulf > > > > -------------- next part -------------- An HTML attachment was scrubbed... URL: From jean-christophe.collet at sun.com Fri Jun 26 15:11:55 2009 From: jean-christophe.collet at sun.com (jean-christophe.collet at sun.com) Date: Fri, 26 Jun 2009 15:11:55 +0000 Subject: hg: jdk7/tl/jdk: 6855297: Windows build breaks after 6811297 Message-ID: <20090626151221.6C456E336@hg.openjdk.java.net> Changeset: 0b6571d4b4b5 Author: jccollet Date: 2009-06-26 16:50 +0200 URL: http://hg.openjdk.java.net/jdk7/tl/jdk/rev/0b6571d4b4b5 6855297: Windows build breaks after 6811297 Summary: re-introduced the mistakenly taken out authObj member Reviewed-by: chegar ! src/share/classes/sun/net/www/protocol/http/HttpURLConnection.java From Xueming.Shen at Sun.COM Fri Jun 26 15:47:49 2009 From: Xueming.Shen at Sun.COM (Xueming Shen) Date: Fri, 26 Jun 2009 08:47:49 -0700 Subject: Miscellaneous improvements to "jar". In-Reply-To: <1ccfd1c10906260653j2f91a7ffx4a3056d67129f6a0@mail.gmail.com> References: <1ccfd1c10906241934i21247553w685cf0b0b4cbd7b2@mail.gmail.com> <1ccfd1c10906251200n3bd78beek400abb51adcece40@mail.gmail.com> <1ccfd1c10906251757y72ccaca1l289dfc79f18b002c@mail.gmail.com> <4A4488D3.9030407@gmx.de> <1ccfd1c10906260653j2f91a7ffx4a3056d67129f6a0@mail.gmail.com> Message-ID: <4A44EDA5.304@sun.com> Martin Buchholz wrote: > > > On Fri, Jun 26, 2009 at 01:37, Ulf Zibis > wrote: > > Am 26.06.2009 02:57, Martin Buchholz schrieb: > > I did some benchmarking, > and found that my changes "only" make jar 10-20% faster. > Disappointing - we expect an order of magnitude for every commit! > > > 1. Hopefully some volunteer would be found to fix > http://bugs.sun.com/bugdatabase/view_bug.do?bug_id=6818737 > before JDK7 API-freeze. > Especially, if jar is not compressed, as in case of normal JDK > installation, reading entries from jar should be much faster > through java.nio.channels, than via BuffererdInputStream. > > > The way to motivate us around here > is to provide the prototype implementation that > demonstrates the speedup. Ulf, I do have a "prototype implementation" that uses buffer based read/write on Jar/ZipFile, it is not that "much" faster as you would have expected (basically the gain of using direct buffer comes from saving one or two memory copy of the content, which is very faster, compared to the "real hard" work of moving bits from harddisk to memory). While it's still something worth doing, but definitely not a high priority for now, yes, it's on the list. Sherman From martinrb at google.com Fri Jun 26 16:04:18 2009 From: martinrb at google.com (Martin Buchholz) Date: Fri, 26 Jun 2009 09:04:18 -0700 Subject: Miscellaneous improvements to "jar". In-Reply-To: <4A44EDA5.304@sun.com> References: <1ccfd1c10906241934i21247553w685cf0b0b4cbd7b2@mail.gmail.com> <1ccfd1c10906251200n3bd78beek400abb51adcece40@mail.gmail.com> <1ccfd1c10906251757y72ccaca1l289dfc79f18b002c@mail.gmail.com> <4A4488D3.9030407@gmx.de> <1ccfd1c10906260653j2f91a7ffx4a3056d67129f6a0@mail.gmail.com> <4A44EDA5.304@sun.com> Message-ID: <1ccfd1c10906260904r5e758feydfc3ca29e8616710@mail.gmail.com> Removing one layer of BufferedInputStream in my change saves one bulk copy per file. And reusing the same buffer saves on cache misses and GC. But bulk copy is actually very fast, (especially as memory is becoming more like disk), so the win is relatively small. I would be surprised if you could get more than the 10-20% that I've gotten with this change, by using direct buffers. Martin On Fri, Jun 26, 2009 at 08:47, Xueming Shen wrote: > > I do have a "prototype implementation" that uses buffer based read/write on > Jar/ZipFile, it > is not that "much" faster as you would have expected (basically the gain of > using direct buffer > comes from saving one or two memory copy of the content, which is very > faster, compared to > the "real hard" work of moving bits from harddisk to memory). While it's > still something > worth doing, but definitely not a high priority for now, yes, it's on the > list. > > Sherman > -------------- next part -------------- An HTML attachment was scrubbed... URL: From Ulf.Zibis at gmx.de Fri Jun 26 16:31:33 2009 From: Ulf.Zibis at gmx.de (Ulf Zibis) Date: Fri, 26 Jun 2009 18:31:33 +0200 Subject: Miscellaneous improvements to "jar". In-Reply-To: <1ccfd1c10906260653j2f91a7ffx4a3056d67129f6a0@mail.gmail.com> References: <1ccfd1c10906241934i21247553w685cf0b0b4cbd7b2@mail.gmail.com> <1ccfd1c10906251200n3bd78beek400abb51adcece40@mail.gmail.com> <1ccfd1c10906251757y72ccaca1l289dfc79f18b002c@mail.gmail.com> <4A4488D3.9030407@gmx.de> <1ccfd1c10906260653j2f91a7ffx4a3056d67129f6a0@mail.gmail.com> Message-ID: <4A44F7E5.2080103@gmx.de> Martin, thanks for taking the time. Am 26.06.2009 15:53, Martin Buchholz schrieb: > > > On Fri, Jun 26, 2009 at 01:37, Ulf Zibis > wrote: > > 1. Hopefully some volunteer would be found to fix > http://bugs.sun.com/bugdatabase/view_bug.do?bug_id=6818737 > before JDK7 API-freeze. > Especially, if jar is not compressed, as in case of normal JDK > installation, reading entries from jar should be much faster > through java.nio.channels, than via BuffererdInputStream. > > > The way to motivate us around here > is to provide the prototype implementation that > demonstrates the speedup. Sorry, I'm not the specialist on how to provide NIO buffers from native memory, and first, I will finish my work on charsets. Motivation: Xueming states: *"dat" based uses less disk space, but it has larger startup time, reading an additional "big" dat file during class loading/initializing actually takes much longer time than I expected (I hit the extreme when I worked on the EUC_TW, which I make the size only 30% of the existing one, but startup is a disaster regression, ... * If loading x bytes from dat file via getResourceAsStream() takes much longer time than loading x+30% bytes from class file, processing the UTF-8 conversion, instantiating and initializing additional Class objects, I imperatively presume, that there must be a big chance for significantly improving read speed from uncompressed jar file (here charsets.jar), by using direct channels or how ever. I presume, enhancing reading from jar files would be a big fish in performance gain for the whole JDK, as it is very common task in JVM's daily work. > > > > > > While benchmarking, I discovered to my horror that the simple > > jar cf /tmp/t1 ... > jar i /tmp/t1 > > fails, because it tries to create the replacement jar in "." > and then > rename() it into place. Oops... Better refactor out the code that > puts the replacement temp file in the same directory. > Better write some tests for that, too. > > > 2. I don't like to refactor out the code in case of only once > used, and only to better "comment" what the 2 lines are doing. > It blows up the code, and following the code demands annoying > scrolling. > Better add additional comment to original code. > > > The original code created temp files in *two* places, > and did it differently. Oops, at my first search on your code I only found *one* usage of createTempFileInSameDirectoryAs(). Did you add the 2nd later? But there is only one usage of directoryOf(). Shouldn't you inline this? > I think the name > createTempFileInSameDirectoryAs > makes the current code much clearer. Yes, this is pretty clear, but the cost is 19 lines against 2+2 plus demanding the reader for annoying scrolling. Thinking about directoryOf() I guess, following this strategy you would find ten's of locations in Main.java where you could refactor out code into small well self-explaining methods, but wouldn't this end up in a mess of unreadable blown-up code? > > Also, JITs tend to be very good at inlining. (... after some loops), yes, I know > > > > 3. What happens, if original file is exactly named "jartmp" > I think you would better add ".tmp" at the end of the filename, > and remove it later. > Does your new code work with? : > jar cf /jartmp/t1 ... > jar i /jartmp/t1 > > > File.createTempFile doesn't literally create a file named jartmp. > That's only the prefix. And it promises to return > a freshly created empty file. Now I understand deeper. I just wondered why in fact just renaming "tmp" to "jartmp" would resolve your bug. I didn't recognize the 2nd location, where wrong "." was used for dir name. -Ulf -------------- next part -------------- An HTML attachment was scrubbed... URL: From Ulf.Zibis at gmx.de Fri Jun 26 16:35:27 2009 From: Ulf.Zibis at gmx.de (Ulf Zibis) Date: Fri, 26 Jun 2009 18:35:27 +0200 Subject: Miscellaneous improvements to "jar". In-Reply-To: <1ccfd1c10906260904r5e758feydfc3ca29e8616710@mail.gmail.com> References: <1ccfd1c10906241934i21247553w685cf0b0b4cbd7b2@mail.gmail.com> <1ccfd1c10906251200n3bd78beek400abb51adcece40@mail.gmail.com> <1ccfd1c10906251757y72ccaca1l289dfc79f18b002c@mail.gmail.com> <4A4488D3.9030407@gmx.de> <1ccfd1c10906260653j2f91a7ffx4a3056d67129f6a0@mail.gmail.com> <4A44EDA5.304@sun.com> <1ccfd1c10906260904r5e758feydfc3ca29e8616710@mail.gmail.com> Message-ID: <4A44F8CF.8050406@gmx.de> Oops, I'm not as fast as you two, writing in english. :-( -Ulf Am 26.06.2009 18:04, Martin Buchholz schrieb: > Removing one layer of BufferedInputStream > in my change saves one bulk copy per file. > And reusing the same buffer saves on cache misses > and GC. But bulk copy is actually very fast, > (especially as memory is becoming more like disk), > so the win is relatively small. > > I would be surprised if you could get more than > the 10-20% that I've gotten with this change, > by using direct buffers. > > Martin > > On Fri, Jun 26, 2009 at 08:47, Xueming Shen > wrote: > > > I do have a "prototype implementation" that uses buffer based > read/write on Jar/ZipFile, it > is not that "much" faster as you would have expected (basically > the gain of using direct buffer > comes from saving one or two memory copy of the content, which is > very faster, compared to > the "real hard" work of moving bits from harddisk to memory). > While it's still something > worth doing, but definitely not a high priority for now, yes, > it's on the list. > > Sherman > > From Xueming.Shen at Sun.COM Fri Jun 26 16:45:34 2009 From: Xueming.Shen at Sun.COM (Xueming Shen) Date: Fri, 26 Jun 2009 09:45:34 -0700 Subject: Miscellaneous improvements to "jar". In-Reply-To: <4A44F7E5.2080103@gmx.de> References: <1ccfd1c10906241934i21247553w685cf0b0b4cbd7b2@mail.gmail.com> <1ccfd1c10906251200n3bd78beek400abb51adcece40@mail.gmail.com> <1ccfd1c10906251757y72ccaca1l289dfc79f18b002c@mail.gmail.com> <4A4488D3.9030407@gmx.de> <1ccfd1c10906260653j2f91a7ffx4a3056d67129f6a0@mail.gmail.com> <4A44F7E5.2080103@gmx.de> Message-ID: <4A44FB2E.2040707@sun.com> Ulf Zibis wrote: > > Motivation: > Xueming states: > *"dat" based uses less disk space, but it has larger startup time, > reading an additional "big" dat file during class loading/initializing > actually takes much longer time than I expected (I hit the extreme > when I worked on the EUC_TW, which I make the size only 30% of the > existing one, but startup is a disaster regression, ... > * > If loading x bytes from dat file via getResourceAsStream() takes much > longer time than loading x+30% bytes from class file, processing the > UTF-8 conversion, instantiating and initializing additional Class > objects, I imperatively presume, that there must be a big chance for > significantly improving read speed from uncompressed jar file (here > charsets.jar), by using direct channels or how ever. I presume, > enhancing reading from jar files would be a big fish in performance > gain for the whole JDK, as it is very common task in JVM's daily work. > > Ulf, the "jar reading" part of the "class loading/initializing" of charsets.jar (and most of the "core lib" jars) is not via java.util.jar/zip interface, there is a native level interface for the vm to access those jar files, so adding a nio buffer interface is not going to help startup of jdk/jre itself. Martin's estimate of 10%-20% gain indeed is very close, I got less than 5%-8% (for jaring) gain from my prototype implementation. As I said it's still something worthing doing, especially it makes the life easier when work together with other nio/channel APIs. Sherman From martinrb at google.com Fri Jun 26 17:11:28 2009 From: martinrb at google.com (Martin Buchholz) Date: Fri, 26 Jun 2009 10:11:28 -0700 Subject: Miscellaneous improvements to "jar". In-Reply-To: <4A44F7E5.2080103@gmx.de> References: <1ccfd1c10906241934i21247553w685cf0b0b4cbd7b2@mail.gmail.com> <1ccfd1c10906251200n3bd78beek400abb51adcece40@mail.gmail.com> <1ccfd1c10906251757y72ccaca1l289dfc79f18b002c@mail.gmail.com> <4A4488D3.9030407@gmx.de> <1ccfd1c10906260653j2f91a7ffx4a3056d67129f6a0@mail.gmail.com> <4A44F7E5.2080103@gmx.de> Message-ID: <1ccfd1c10906261011pe8dc194l7d547b4039b6f8a0@mail.gmail.com> On Fri, Jun 26, 2009 at 09:31, Ulf Zibis wrote: > Martin, thanks for taking the time. > > Am 26.06.2009 15:53, Martin Buchholz schrieb: > > > > On Fri, Jun 26, 2009 at 01:37, Ulf Zibis wrote: > >> 1. Hopefully some volunteer would be found to fix >> http://bugs.sun.com/bugdatabase/view_bug.do?bug_id=6818737 >> before JDK7 API-freeze. >> Especially, if jar is not compressed, as in case of normal JDK >> installation, reading entries from jar should be much faster through >> java.nio.channels, than via BuffererdInputStream. > > > The way to motivate us around here > is to provide the prototype implementation that > demonstrates the speedup. > > > Sorry, I'm not the specialist on how to provide NIO buffers from native > memory, and first, I will finish my work on charsets. > > Motivation: > Xueming states: > *"dat" based uses less disk space, but it has larger startup time, reading > an additional "big" dat file during class loading/initializing actually > takes much longer time than I expected (I hit the extreme when I worked on > the EUC_TW, which I make the size only 30% of the existing one, but startup > is a disaster regression, ... > * > I'm surprised. I would expect startup to actually be faster. I assume we're only reading the bytes that are necessary > > If loading x bytes from dat file via getResourceAsStream() takes much > longer time than loading x+30% bytes from class file, processing the UTF-8 > conversion, instantiating and initializing additional Class objects, I > imperatively presume, that there must be a big chance for significantly > improving read speed from uncompressed jar file (here charsets.jar), by > using direct channels or how ever. I presume, enhancing reading from jar > files would be a big fish in performance gain for the whole JDK, as it is > very common task in JVM's daily work. > > > > >> >> >> >>> While benchmarking, I discovered to my horror that the simple >>> >>> jar cf /tmp/t1 ... >>> jar i /tmp/t1 >>> >>> fails, because it tries to create the replacement jar in "." and then >>> rename() it into place. Oops... Better refactor out the code that >>> puts the replacement temp file in the same directory. >>> Better write some tests for that, too. >>> >> >> 2. I don't like to refactor out the code in case of only once used, and >> only to better "comment" what the 2 lines are doing. >> It blows up the code, and following the code demands annoying scrolling. >> Better add additional comment to original code. >> > > The original code created temp files in *two* places, > and did it differently. > > > Oops, at my first search on your code I only found *one* usage of > createTempFileInSameDirectoryAs(). Did you add the 2nd later? > But there is only one usage of directoryOf(). Shouldn't you inline this? > This is modern software engineering. We are all encouraged to write many small methods. > > I think the name > createTempFileInSameDirectoryAs > makes the current code much clearer. > > > Yes, this is pretty clear, but the cost is 19 lines against 2+2 plus > demanding the reader for annoying scrolling. > Thinking about directoryOf() I guess, following this strategy you would > find ten's of locations in Main.java where you could refactor out code into > small well self-explaining methods, but wouldn't this end up in a mess of > unreadable blown-up code? > Find suitable abstractions and refactor them into a separate piece of code. The win is a lot bigger if you make the new abstractions public supported parts of the API, but that is harder with the JDK. > Also, JITs tend to be very good at inlining. > > > (... after some loops), yes, I know > > > >> >> 3. What happens, if original file is exactly named "jartmp" >> I think you would better add ".tmp" at the end of the filename, and remove >> it later. >> Does your new code work with? : >> jar cf /jartmp/t1 ... >> jar i /jartmp/t1 >> > > File.createTempFile doesn't literally create a file named jartmp. > That's only the prefix. And it promises to return > a freshly created empty file. > > > Now I understand deeper. I just wondered why in fact just renaming "tmp" to > "jartmp" would resolve your bug. I didn't recognize the 2nd location, where > wrong "." was used for dir name. > The renaming jar -> jartmp is not significant. Martin > > -Ulf > > -------------- next part -------------- An HTML attachment was scrubbed... URL: From Xueming.Shen at Sun.COM Fri Jun 26 17:33:10 2009 From: Xueming.Shen at Sun.COM (Xueming Shen) Date: Fri, 26 Jun 2009 10:33:10 -0700 Subject: Miscellaneous improvements to "jar". In-Reply-To: <1ccfd1c10906251757y72ccaca1l289dfc79f18b002c@mail.gmail.com> References: <1ccfd1c10906241934i21247553w685cf0b0b4cbd7b2@mail.gmail.com> <1ccfd1c10906251200n3bd78beek400abb51adcece40@mail.gmail.com> <1ccfd1c10906251757y72ccaca1l289dfc79f18b002c@mail.gmail.com> Message-ID: <4A450656.7070008@sun.com> The latest version looks good. 2 nit comments (1) it's reasonable to have createTempFileInSamDirectoryAs separate out, but I would keep the directoryOf within it. Yes, it's "clearer":-) (2)the "updateOK" in dumpIndex really serves nobody but "clearer":-) sherman btw, while you are here:-) do you have time to make the update() NOT to put jarIndex the first one when there is a manifest present? The JarInputStream assumes the manifest is always the first entry (or the second if the manifest dir is the first), which causes problem...I'm not saying to mix with this one:-) just in case you have time and interested while you are here:-) Martin Buchholz wrote: > I did some benchmarking, > and found that my changes "only" make jar 10-20% faster. > Disappointing - we expect an order of magnitude for every commit! > > While benchmarking, I discovered to my horror that the simple > > jar cf /tmp/t1 ... > jar i /tmp/t1 > > fails, because it tries to create the replacement jar in "." and then > rename() it into place. Oops... Better refactor out the code that > puts the replacement temp file in the same directory. > Better write some tests for that, too. > > /** > * A variant of File.getParentFile that always returns a valid > * directory (i.e. returns new File(".") where File.getParentFile > * would return null). > */ > private static File directoryOf(File file) { > File dir = file.getParentFile(); > return (dir != null) ? dir : new File("."); > } > > /** > * Creates a new empty temporary file in the same directory as the > * specified file. A variant of File.createTempFile. > */ > private static File createTempFileInSameDirectoryAs(File file) > throws IOException { > return File.createTempFile("jartmp", null, directoryOf(file)); > } > > --- > > While we're here, let's remove the double buffering on file copy > operations. > And the repeated allocations of big buffers. > > /** > * A buffer for use only by copy(InputStream, OutputStream). > * Not as clean as allocating a new buffer as needed by copy, > * but significantly more efficient. > */ > private byte[] copyBuf = new byte[8192]; > > /** > * Copies all bytes from the input stream to the output stream. > * Does not close or flush either stream. > * > * @param from the input stream to read from > * @param to the output stream to write to > * @throws IOException if an I/O error occurs > */ > private void copy(InputStream from, OutputStream to) throws > IOException { > int n; > while ((n = from.read(copyBuf)) != -1) > to.write(copyBuf, 0, n); > } > > /** > * Copies all bytes from the input file to the output stream. > * Does not close or flush the output stream. > * > * @param from the input file to read from > * @param to the output stream to write to > * @throws IOException if an I/O error occurs > */ > private void copy(File from, OutputStream to) throws IOException { > InputStream in = new FileInputStream(from); > try { > copy(in, to); > } finally { > in.close(); > } > } > > /** > * Copies all bytes from the input stream to the output file. > * Does not close the input stream. > * > * @param from the input stream to read from > * @param to the output file to write to > * @throws IOException if an I/O error occurs > */ > private void copy(InputStream from, File to) throws IOException { > OutputStream out = new FileOutputStream(to); > try { > copy(from, out); > } finally { > out.close(); > } > } > > ---- > > On the other hand, allocating a CRC32 is very cheap, so let's make that > private to CRC32OutputStream. > + private static class CRC32OutputStream extends java.io.OutputStream { > + final CRC32 crc = new CRC32(); > > ---- > > We should add a try { finally } to delete the tempfile for jar i > + try { > boolean updateOk = update(new FileInputStream(jarFile), > - new FileOutputStream(scratchFile), > > + new FileOutputStream(tmpFile), > null, index); > + if (updateOk) { > jarFile.delete(); > > - if (!scratchFile.renameTo(jarFile)) { > - scratchFile.delete(); > + if (!tmpFile.renameTo(jarFile)) { > > throw new IOException(getMsg("error.write.file")); > } > - scratchFile.delete(); > + } > + } finally { > > + tmpFile.delete(); > + } > } > > > ---- > > Webrev updated. > http://cr.openjdk.java.net/~martin/jar-misc/ > > > There are now many small changes combined in this fix. Sorry about that. > I'd be more inclined to separate them if creating new bugs was easier. > > I'm not planning on making any more changes. Really. > > Martin > > On Thu, Jun 25, 2009 at 12:00, Martin Buchholz > wrote: > > I have an updated version of this fix, with these changes: > > - Documented the turkish i problem > > /** > * Compares two strings for equality, ignoring case. The second > * argument must contain only upper-case ASCII characters. > * We don't want case comparison to be locale-dependent (else we > * have the notorious "turkish i bug"). > */ > private boolean equalsIgnoreCase(String s, String upper) { > > - Refactored code so that updateEntry now also sets the method to > STORED. > > /** > * Updates a ZipEntry which describes the data read by this > * output stream, in STORED mode. > */ > public void updateEntry(ZipEntry e) { > e.setMethod(ZipEntry.STORED); > e.setSize(n); > e.setCrc(crc.getValue()); > } > > - addIndex was never updating the size in the ZipEntry (as required), > which was not previously noticed because closeEntry was never > called. > > private void addIndex(JarIndex index, ZipOutputStream zos) > throws IOException > { > ZipEntry e = new ZipEntry(INDEX_NAME); > e.setTime(System.currentTimeMillis()); > if (flag0) { > CRC32OutputStream os = new CRC32OutputStream(crc32); > index.write(os); > os.updateEntry(e); > } > zos.putNextEntry(e); > index.write(zos); > zos.closeEntry(); > > } > > http://cr.openjdk.java.net/~martin/jar-misc/ > > Previous webrev: > http://cr.openjdk.java.net/~martin/jar-misc.0/ > > > Martin > > > > On Wed, Jun 24, 2009 at 19:34, Martin Buchholz > > wrote: > > Hi jar team, > > I have a bunch of minor improvements to > src/share/classes/sun/tools/jar/Main.java > > Toby and Xueming, please review. > > Warning: the index code has not been maintained for many years. > > Xueming, please file a bug. > > Synopsis: Miscellaneous improvements to "jar". > Description: > - Use standard jdk coding style for javadoc > - Don't create a temp file for jar index in STORED mode. > - Don't use synchronized collections. > - Fix javac warnings. > - Don't define new names for things like INDEX_NAME; > use static import instead. > - more efficiently compare special file names in update mode. > Update mode should be measurably faster. > - make CRC32OutputStream a nested class. > refactor crc32.reset and updating entry into CRC32OutputStream. > - Fix apparently benign bug updating n in > CRC32OutputStream.write(byte[], int, int) > > Evaluation: Yep. > > http://cr.openjdk.java.net/~martin/jar-misc/ > > > Thanks, > > Martin > > > From Ulf.Zibis at gmx.de Fri Jun 26 18:28:28 2009 From: Ulf.Zibis at gmx.de (Ulf Zibis) Date: Fri, 26 Jun 2009 20:28:28 +0200 Subject: Miscellaneous improvements to "jar". In-Reply-To: <4A44FB2E.2040707@sun.com> References: <1ccfd1c10906241934i21247553w685cf0b0b4cbd7b2@mail.gmail.com> <1ccfd1c10906251200n3bd78beek400abb51adcece40@mail.gmail.com> <1ccfd1c10906251757y72ccaca1l289dfc79f18b002c@mail.gmail.com> <4A4488D3.9030407@gmx.de> <1ccfd1c10906260653j2f91a7ffx4a3056d67129f6a0@mail.gmail.com> <4A44F7E5.2080103@gmx.de> <4A44FB2E.2040707@sun.com> Message-ID: <4A45134C.2080105@gmx.de> Am 26.06.2009 18:45, Xueming Shen schrieb: > Ulf Zibis wrote: >> >> Motivation: >> Xueming states: >> *"dat" based uses less disk space, but it has larger startup time, >> reading an additional "big" dat file during class >> loading/initializing actually takes much longer time than I expected >> (I hit the extreme when I worked on the EUC_TW, which I make the size >> only 30% of the existing one, but startup is a disaster regression, ... >> * >> If loading x bytes from dat file via getResourceAsStream() takes much >> longer time than loading x+30% bytes from class file, processing the >> UTF-8 conversion, instantiating and initializing additional Class >> objects, I imperatively presume, that there must be a big chance for >> significantly improving read speed from uncompressed jar file (here >> charsets.jar), by using direct channels or how ever. I presume, >> enhancing reading from jar files would be a big fish in performance >> gain for the whole JDK, as it is very common task in JVM's daily work. >> >> > Ulf, the "jar reading" part of the "class loading/initializing" of > charsets.jar (and most of the "core lib" jars) is not via > java.util.jar/zip interface, Oops, sorry for misunderstanding ... > there is a native level interface for the vm to access those jar > files, so adding a nio buffer interface is not going to help startup > of jdk/jre itself. ... I was only thinking about the numerous "resources" loaded on startup of jdk/jre AND later from user application, and, of course, for more motivation to avoid heavy static data in class files. Additionally, some time ago on a debug session through getResourceAsStream(), I noticed, that the construction of the resources file's URL seams to be very wasteful (looping StringBuffer#apend() char by char (or even strings of length 1), even on 50..100 chars long Strings). Any chance about outsourcing some code from API's jar code to native code, so API customers could profit from the speed of the native level interface of the vm to access jar files? > Martin's estimate of 10%-20% gain indeed is very close, I got less > than 5%-8% (for jaring) gain from my prototype implementation. As I > said it's still something worthing doing, especially it makes the life > easier when work together with other nio/channel APIs. I agree! -Ulf From Ulf.Zibis at gmx.de Fri Jun 26 18:34:24 2009 From: Ulf.Zibis at gmx.de (Ulf Zibis) Date: Fri, 26 Jun 2009 20:34:24 +0200 Subject: Miscellaneous improvements to "jar". In-Reply-To: <4A44EDA5.304@sun.com> References: <1ccfd1c10906241934i21247553w685cf0b0b4cbd7b2@mail.gmail.com> <1ccfd1c10906251200n3bd78beek400abb51adcece40@mail.gmail.com> <1ccfd1c10906251757y72ccaca1l289dfc79f18b002c@mail.gmail.com> <4A4488D3.9030407@gmx.de> <1ccfd1c10906260653j2f91a7ffx4a3056d67129f6a0@mail.gmail.com> <4A44EDA5.304@sun.com> Message-ID: <4A4514B0.7010200@gmx.de> Am 26.06.2009 17:47, Xueming Shen schrieb: > Ulf, > > I do have a "prototype implementation" that uses buffer based > read/write on Jar/ZipFile, it > is not that "much" faster as you would have expected (basically the > gain of using direct buffer > comes from saving one or two memory copy of the content, which is > very faster, compared to > the "real hard" work of moving bits from harddisk to memory). Often those bits are just cached in memory from OS side, maybe also big chunks of rt.jar and charsets.jar after VM is launched. Another 2 cents! -Ulf From Ulf.Zibis at gmx.de Fri Jun 26 18:42:35 2009 From: Ulf.Zibis at gmx.de (Ulf Zibis) Date: Fri, 26 Jun 2009 20:42:35 +0200 Subject: Miscellaneous improvements to "jar". In-Reply-To: <1ccfd1c10906261011pe8dc194l7d547b4039b6f8a0@mail.gmail.com> References: <1ccfd1c10906241934i21247553w685cf0b0b4cbd7b2@mail.gmail.com> <1ccfd1c10906251200n3bd78beek400abb51adcece40@mail.gmail.com> <1ccfd1c10906251757y72ccaca1l289dfc79f18b002c@mail.gmail.com> <4A4488D3.9030407@gmx.de> <1ccfd1c10906260653j2f91a7ffx4a3056d67129f6a0@mail.gmail.com> <4A44F7E5.2080103@gmx.de> <1ccfd1c10906261011pe8dc194l7d547b4039b6f8a0@mail.gmail.com> Message-ID: <4A45169B.4000009@gmx.de> > Find suitable abstractions and refactor them into a separate piece of > code. > > The win is a lot bigger if you make the new abstractions > public supported parts of the API, > but that is harder with the JDK. Yep, I agree. Often that's my motivation to report RFE's e.g.: http://bugs.sun.com/bugdatabase/view_bug.do?bug_id=6798511 http://bugs.sun.com/bugdatabase/view_bug.do?bug_id=6695386 http://bugs.sun.com/bugdatabase/view_bug.do?bug_id=6595143 http://bugs.sun.com/bugdatabase/view_bug.do?bug_id=6812862 http://bugs.sun.com/bugdatabase/view_bug.do?bug_id=6826329 -Ulf From Xueming.Shen at Sun.COM Fri Jun 26 18:43:01 2009 From: Xueming.Shen at Sun.COM (Xueming Shen) Date: Fri, 26 Jun 2009 11:43:01 -0700 Subject: Miscellaneous improvements to "jar". In-Reply-To: <4A4514B0.7010200@gmx.de> References: <1ccfd1c10906241934i21247553w685cf0b0b4cbd7b2@mail.gmail.com> <1ccfd1c10906251200n3bd78beek400abb51adcece40@mail.gmail.com> <1ccfd1c10906251757y72ccaca1l289dfc79f18b002c@mail.gmail.com> <4A4488D3.9030407@gmx.de> <1ccfd1c10906260653j2f91a7ffx4a3056d67129f6a0@mail.gmail.com> <4A44EDA5.304@sun.com> <4A4514B0.7010200@gmx.de> Message-ID: <4A4516B5.5050302@sun.com> Ulf Zibis wrote: > Am 26.06.2009 17:47, Xueming Shen schrieb: >> Ulf, >> >> I do have a "prototype implementation" that uses buffer based >> read/write on Jar/ZipFile, it >> is not that "much" faster as you would have expected (basically the >> gain of using direct buffer >> comes from saving one or two memory copy of the content, which is >> very faster, compared to >> the "real hard" work of moving bits from harddisk to memory). > > Often those bits are just cached in memory from OS side, maybe also > big chunks of rt.jar and charsets.jar after VM is launched. > Another 2 cents! > > -Ulf > > > Our performance team works on both warm start and cold start, the terms we use for the them:-) From joe.darcy at sun.com Fri Jun 26 19:31:02 2009 From: joe.darcy at sun.com (joe.darcy at sun.com) Date: Fri, 26 Jun 2009 19:31:02 +0000 Subject: hg: jdk7/tl/langtools: 6593082: MirroredTypeException constructor does not throw NPE when type is null Message-ID: <20090626193106.D88FEE393@hg.openjdk.java.net> Changeset: ca063536e4a6 Author: darcy Date: 2009-06-26 12:22 -0700 URL: http://hg.openjdk.java.net/jdk7/tl/langtools/rev/ca063536e4a6 6593082: MirroredTypeException constructor does not throw NPE when type is null Reviewed-by: jjg ! src/share/classes/javax/lang/model/type/MirroredTypeException.java + test/tools/javac/processing/model/type/MirroredTypeEx/NpeTest.java From Alan.Bateman at Sun.COM Fri Jun 26 20:22:30 2009 From: Alan.Bateman at Sun.COM (Alan Bateman) Date: Fri, 26 Jun 2009 21:22:30 +0100 Subject: Review request for 6844054: (bf) Eliminate dependency on javax.management.ObjectName Message-ID: <4A452E06.3060102@sun.com> Mandy - you do mind reviewing this change to eliminate the dependency in java.nio on javax.management.ObjectName that we've chatted about. The changes move the creation of the management interfaces to ManagementFactoryHelper. The webrev is here: http://cr.openjdk.java.net/~alanb/6844054/webrev.00/ Thanks, Alan. From Mandy.Chung at Sun.COM Fri Jun 26 21:48:01 2009 From: Mandy.Chung at Sun.COM (Mandy Chung) Date: Fri, 26 Jun 2009 14:48:01 -0700 Subject: Review request for 6844054: (bf) Eliminate dependency on javax.management.ObjectName In-Reply-To: <4A452E06.3060102@sun.com> References: <4A452E06.3060102@sun.com> Message-ID: <4A454211.6020007@sun.com> Alan, Looks good. Thanks for making the change. I'll fix java.util.logging to eliminate such dependency. Mandy Alan Bateman wrote: > Mandy - you do mind reviewing this change to eliminate the dependency > in java.nio on javax.management.ObjectName that we've chatted about. > The changes move the creation of the management interfaces to > ManagementFactoryHelper. The webrev is here: > http://cr.openjdk.java.net/~alanb/6844054/webrev.00/ > > Thanks, > > Alan. From jonathan.gibbons at sun.com Sat Jun 27 01:45:47 2009 From: jonathan.gibbons at sun.com (jonathan.gibbons at sun.com) Date: Sat, 27 Jun 2009 01:45:47 +0000 Subject: hg: jdk7/tl/jdk: 6843077: JSR 308: Annotations on types Message-ID: <20090627014629.5E760E3CA@hg.openjdk.java.net> Changeset: a5f7d97c3f82 Author: jjg Date: 2009-06-26 18:39 -0700 URL: http://hg.openjdk.java.net/jdk7/tl/jdk/rev/a5f7d97c3f82 6843077: JSR 308: Annotations on types Reviewed-by: jjg, mcimadamore, darcy Contributed-by: mernst at cs.washington.edu, mali at csail.mit.edu, mpapi at csail.mit.edu ! src/share/classes/java/lang/annotation/ElementType.java From jonathan.gibbons at sun.com Sat Jun 27 01:57:36 2009 From: jonathan.gibbons at sun.com (jonathan.gibbons at sun.com) Date: Sat, 27 Jun 2009 01:57:36 +0000 Subject: hg: jdk7/tl/langtools: 6843077: JSR 308: Annotations on types Message-ID: <20090627015741.AE1CFE3D1@hg.openjdk.java.net> Changeset: 03944ee4fac4 Author: jjg Date: 2009-06-26 18:51 -0700 URL: http://hg.openjdk.java.net/jdk7/tl/langtools/rev/03944ee4fac4 6843077: JSR 308: Annotations on types Reviewed-by: jjg, mcimadamore, darcy Contributed-by: mernst at cs.washington.edu, mali at csail.mit.edu, mpapi at csail.mit.edu ! src/share/bin/launcher.sh-template ! src/share/classes/com/sun/source/tree/MethodTree.java ! src/share/classes/com/sun/source/tree/Tree.java ! src/share/classes/com/sun/source/tree/TreeVisitor.java ! src/share/classes/com/sun/source/tree/TypeParameterTree.java ! src/share/classes/com/sun/source/util/SimpleTreeVisitor.java ! src/share/classes/com/sun/source/util/TreePath.java ! src/share/classes/com/sun/source/util/TreeScanner.java ! src/share/classes/com/sun/source/util/Trees.java ! src/share/classes/com/sun/tools/classfile/Attribute.java ! src/share/classes/com/sun/tools/classfile/ClassWriter.java ! src/share/classes/com/sun/tools/javac/api/JavacTaskImpl.java ! src/share/classes/com/sun/tools/javac/api/JavacTrees.java ! src/share/classes/com/sun/tools/javac/code/Attribute.java ! src/share/classes/com/sun/tools/javac/code/Source.java ! src/share/classes/com/sun/tools/javac/code/Symbol.java ! src/share/classes/com/sun/tools/javac/comp/Attr.java ! src/share/classes/com/sun/tools/javac/comp/Check.java ! src/share/classes/com/sun/tools/javac/comp/Flow.java ! src/share/classes/com/sun/tools/javac/comp/Lower.java ! src/share/classes/com/sun/tools/javac/comp/MemberEnter.java ! src/share/classes/com/sun/tools/javac/comp/TransTypes.java ! src/share/classes/com/sun/tools/javac/jvm/ClassReader.java ! src/share/classes/com/sun/tools/javac/jvm/ClassWriter.java ! src/share/classes/com/sun/tools/javac/jvm/Code.java ! src/share/classes/com/sun/tools/javac/jvm/Gen.java ! src/share/classes/com/sun/tools/javac/parser/JavacParser.java ! src/share/classes/com/sun/tools/javac/processing/JavacProcessingEnvironment.java ! src/share/classes/com/sun/tools/javac/processing/JavacRoundEnvironment.java ! src/share/classes/com/sun/tools/javac/resources/compiler.properties ! src/share/classes/com/sun/tools/javac/tree/JCTree.java ! src/share/classes/com/sun/tools/javac/tree/Pretty.java ! src/share/classes/com/sun/tools/javac/tree/TreeCopier.java ! src/share/classes/com/sun/tools/javac/tree/TreeInfo.java ! src/share/classes/com/sun/tools/javac/tree/TreeMaker.java ! src/share/classes/com/sun/tools/javac/tree/TreeScanner.java ! src/share/classes/com/sun/tools/javac/tree/TreeTranslator.java ! src/share/classes/com/sun/tools/javac/util/Names.java ! src/share/classes/com/sun/tools/javap/AnnotationWriter.java ! src/share/classes/com/sun/tools/javap/AttributeWriter.java ! test/tools/javac/6341866/T6341866.java ! test/tools/javac/processing/6348499/T6348499.java ! test/tools/javac/processing/6414633/T6414633.java ! test/tools/javac/processing/6430209/T6430209.java ! test/tools/javac/processing/T6439826.java From jonathan.gibbons at sun.com Sat Jun 27 02:19:35 2009 From: jonathan.gibbons at sun.com (jonathan.gibbons at sun.com) Date: Sat, 27 Jun 2009 02:19:35 +0000 Subject: hg: jdk7/tl/langtools: 6855544: add missing files Message-ID: <20090627021940.09F45E3D6@hg.openjdk.java.net> Changeset: 664edca41e34 Author: jjg Date: 2009-06-26 19:12 -0700 URL: http://hg.openjdk.java.net/jdk7/tl/langtools/rev/664edca41e34 6855544: add missing files Reviewed-by: jjg, mcimadamore, darcy Contributed-by: mernst at cs.washington.edu, mali at csail.mit.edu, mpapi at csail.mit.edu + src/share/classes/com/sun/source/tree/AnnotatedTypeTree.java + src/share/classes/com/sun/source/util/AbstractTypeProcessor.java + src/share/classes/com/sun/tools/classfile/ExtendedAnnotation.java + src/share/classes/com/sun/tools/classfile/RuntimeInvisibleTypeAnnotations_attribute.java + src/share/classes/com/sun/tools/classfile/RuntimeTypeAnnotations_attribute.java + src/share/classes/com/sun/tools/classfile/RuntimeVisibleTypeAnnotations_attribute.java + src/share/classes/com/sun/tools/javac/code/TargetType.java + src/share/classes/com/sun/tools/javac/code/TypeAnnotationPosition.java + test/tools/javac/api/TestTreePath.java + test/tools/javac/meth/InvokeMH_BAD68.java + test/tools/javac/meth/InvokeMH_BAD72.java + test/tools/javac/quid/QuotedIdent_BAD61.java + test/tools/javac/quid/QuotedIdent_BAD62.java + test/tools/javac/quid/QuotedIdent_BAD63.java + test/tools/javac/typeAnnotations/InnerClass.java + test/tools/javac/typeAnnotations/MultipleTargets.java + test/tools/javac/typeAnnotations/TypeParameterTarget.java + test/tools/javac/typeAnnotations/TypeUseTarget.java + test/tools/javac/typeAnnotations/attribution/Scopes.java + test/tools/javac/typeAnnotations/failures/AnnotationVersion.java + test/tools/javac/typeAnnotations/failures/AnnotationVersion.out + test/tools/javac/typeAnnotations/failures/IncompleteArray.java + test/tools/javac/typeAnnotations/failures/IncompleteArray.out + test/tools/javac/typeAnnotations/failures/IncompleteVararg.java + test/tools/javac/typeAnnotations/failures/IncompleteVararg.out + test/tools/javac/typeAnnotations/failures/IndexArray.java + test/tools/javac/typeAnnotations/failures/IndexArray.out + test/tools/javac/typeAnnotations/failures/LintCast.java + test/tools/javac/typeAnnotations/failures/LintCast.out + test/tools/javac/typeAnnotations/failures/OldArray.java + test/tools/javac/typeAnnotations/failures/OldArray.out + test/tools/javac/typeAnnotations/failures/Scopes.java + test/tools/javac/typeAnnotations/failures/Scopes.out + test/tools/javac/typeAnnotations/failures/StaticFields.java + test/tools/javac/typeAnnotations/failures/StaticFields.out + test/tools/javac/typeAnnotations/failures/StaticMethods.java + test/tools/javac/typeAnnotations/failures/StaticMethods.out + test/tools/javac/typeAnnotations/failures/VoidGenericMethod.java + test/tools/javac/typeAnnotations/failures/common/arrayclass/DuplicateAnnotationValue.java + test/tools/javac/typeAnnotations/failures/common/arrayclass/DuplicateAnnotationValue.out + test/tools/javac/typeAnnotations/failures/common/arrayclass/DuplicateTypeAnnotation.java + test/tools/javac/typeAnnotations/failures/common/arrayclass/DuplicateTypeAnnotation.out + test/tools/javac/typeAnnotations/failures/common/arrayclass/InvalidLocation.java + test/tools/javac/typeAnnotations/failures/common/arrayclass/InvalidLocation.out + test/tools/javac/typeAnnotations/failures/common/arrayclass/MissingAnnotationValue.java + test/tools/javac/typeAnnotations/failures/common/arrayclass/MissingAnnotationValue.out + test/tools/javac/typeAnnotations/failures/common/arrays/DuplicateAnnotationValue.java + test/tools/javac/typeAnnotations/failures/common/arrays/DuplicateAnnotationValue.out + test/tools/javac/typeAnnotations/failures/common/arrays/DuplicateTypeAnnotation.java + test/tools/javac/typeAnnotations/failures/common/arrays/DuplicateTypeAnnotation.out + test/tools/javac/typeAnnotations/failures/common/arrays/InvalidLocation.java + test/tools/javac/typeAnnotations/failures/common/arrays/InvalidLocation.out + test/tools/javac/typeAnnotations/failures/common/arrays/MissingAnnotationValue.java + test/tools/javac/typeAnnotations/failures/common/arrays/MissingAnnotationValue.out + test/tools/javac/typeAnnotations/failures/common/innertypeparams/DuplicateAnnotationValue.java + test/tools/javac/typeAnnotations/failures/common/innertypeparams/DuplicateAnnotationValue.out + test/tools/javac/typeAnnotations/failures/common/innertypeparams/DuplicateTypeAnnotation.java + test/tools/javac/typeAnnotations/failures/common/innertypeparams/DuplicateTypeAnnotation.out + test/tools/javac/typeAnnotations/failures/common/innertypeparams/InvalidLocation.java + test/tools/javac/typeAnnotations/failures/common/innertypeparams/InvalidLocation.out + test/tools/javac/typeAnnotations/failures/common/innertypeparams/MissingAnnotationValue.java + test/tools/javac/typeAnnotations/failures/common/innertypeparams/MissingAnnotationValue.out + test/tools/javac/typeAnnotations/failures/common/newarray/DuplicateAnnotationValue.java + test/tools/javac/typeAnnotations/failures/common/newarray/DuplicateAnnotationValue.out + test/tools/javac/typeAnnotations/failures/common/newarray/DuplicateTypeAnnotation.java + test/tools/javac/typeAnnotations/failures/common/newarray/DuplicateTypeAnnotation.out + test/tools/javac/typeAnnotations/failures/common/newarray/InvalidLocation.java + test/tools/javac/typeAnnotations/failures/common/newarray/InvalidLocation.out + test/tools/javac/typeAnnotations/failures/common/newarray/MissingAnnotationValue.java + test/tools/javac/typeAnnotations/failures/common/newarray/MissingAnnotationValue.out + test/tools/javac/typeAnnotations/failures/common/parambounds/DuplicateAnnotationValue.java + test/tools/javac/typeAnnotations/failures/common/parambounds/DuplicateAnnotationValue.out + test/tools/javac/typeAnnotations/failures/common/parambounds/DuplicateTypeAnnotation.java + test/tools/javac/typeAnnotations/failures/common/parambounds/DuplicateTypeAnnotation.out + test/tools/javac/typeAnnotations/failures/common/parambounds/InvalidLocation.java + test/tools/javac/typeAnnotations/failures/common/parambounds/InvalidLocation.out + test/tools/javac/typeAnnotations/failures/common/parambounds/MissingAnnotationValue.java + test/tools/javac/typeAnnotations/failures/common/parambounds/MissingAnnotationValue.out + test/tools/javac/typeAnnotations/failures/common/receiver/DuplicateAnnotationValue.java + test/tools/javac/typeAnnotations/failures/common/receiver/DuplicateAnnotationValue.out + test/tools/javac/typeAnnotations/failures/common/receiver/DuplicateTypeAnnotation.java + test/tools/javac/typeAnnotations/failures/common/receiver/DuplicateTypeAnnotation.out + test/tools/javac/typeAnnotations/failures/common/receiver/InvalidLocation.java + test/tools/javac/typeAnnotations/failures/common/receiver/InvalidLocation.out + test/tools/javac/typeAnnotations/failures/common/receiver/MissingAnnotationValue.java + test/tools/javac/typeAnnotations/failures/common/receiver/MissingAnnotationValue.out + test/tools/javac/typeAnnotations/failures/common/rest/DuplicateAnnotationValue.java + test/tools/javac/typeAnnotations/failures/common/rest/DuplicateAnnotationValue.out + test/tools/javac/typeAnnotations/failures/common/rest/DuplicateTypeAnnotation.java + test/tools/javac/typeAnnotations/failures/common/rest/DuplicateTypeAnnotation.out + test/tools/javac/typeAnnotations/failures/common/rest/InvalidLocation.java + test/tools/javac/typeAnnotations/failures/common/rest/InvalidLocation.out + test/tools/javac/typeAnnotations/failures/common/rest/MissingAnnotationValue.java + test/tools/javac/typeAnnotations/failures/common/rest/MissingAnnotationValue.out + test/tools/javac/typeAnnotations/failures/common/typeArgs/DuplicateAnnotationValue.java + test/tools/javac/typeAnnotations/failures/common/typeArgs/DuplicateAnnotationValue.out + test/tools/javac/typeAnnotations/failures/common/typeArgs/DuplicateTypeAnnotation.java + test/tools/javac/typeAnnotations/failures/common/typeArgs/DuplicateTypeAnnotation.out + test/tools/javac/typeAnnotations/failures/common/typeArgs/InvalidLocation.java + test/tools/javac/typeAnnotations/failures/common/typeArgs/InvalidLocation.out + test/tools/javac/typeAnnotations/failures/common/typeArgs/MissingAnnotationValue.java + test/tools/javac/typeAnnotations/failures/common/typeArgs/MissingAnnotationValue.out + test/tools/javac/typeAnnotations/failures/common/typeparams/DuplicateAnnotationValue.java + test/tools/javac/typeAnnotations/failures/common/typeparams/DuplicateAnnotationValue.out + test/tools/javac/typeAnnotations/failures/common/typeparams/DuplicateTypeAnnotation.java + test/tools/javac/typeAnnotations/failures/common/typeparams/DuplicateTypeAnnotation.out + test/tools/javac/typeAnnotations/failures/common/typeparams/InvalidLocation.java + test/tools/javac/typeAnnotations/failures/common/typeparams/InvalidLocation.out + test/tools/javac/typeAnnotations/failures/common/typeparams/MissingAnnotationValue.java + test/tools/javac/typeAnnotations/failures/common/typeparams/MissingAnnotationValue.out + test/tools/javac/typeAnnotations/failures/common/wildcards/DuplicateAnnotationValue.java + test/tools/javac/typeAnnotations/failures/common/wildcards/DuplicateAnnotationValue.out + test/tools/javac/typeAnnotations/failures/common/wildcards/DuplicateTypeAnnotation.java + test/tools/javac/typeAnnotations/failures/common/wildcards/DuplicateTypeAnnotation.out + test/tools/javac/typeAnnotations/failures/common/wildcards/InvalidLocation.java + test/tools/javac/typeAnnotations/failures/common/wildcards/InvalidLocation.out + test/tools/javac/typeAnnotations/failures/common/wildcards/MissingAnnotationValue.java + test/tools/javac/typeAnnotations/failures/common/wildcards/MissingAnnotationValue.out + test/tools/javac/typeAnnotations/failures/target/Constructor.java + test/tools/javac/typeAnnotations/failures/target/Constructor.out + test/tools/javac/typeAnnotations/failures/target/IncompleteArray.java + test/tools/javac/typeAnnotations/failures/target/IncompleteArray.out + test/tools/javac/typeAnnotations/failures/target/NotTypeParameter.java + test/tools/javac/typeAnnotations/failures/target/NotTypeParameter.out + test/tools/javac/typeAnnotations/failures/target/NotTypeUse.java + test/tools/javac/typeAnnotations/failures/target/NotTypeUse.out + test/tools/javac/typeAnnotations/failures/target/VoidMethod.java + test/tools/javac/typeAnnotations/failures/target/VoidMethod.out + test/tools/javac/typeAnnotations/newlocations/BasicTest.java + test/tools/javac/typeAnnotations/newlocations/ClassExtends.java + test/tools/javac/typeAnnotations/newlocations/ClassLiterals.java + test/tools/javac/typeAnnotations/newlocations/ClassParameters.java + test/tools/javac/typeAnnotations/newlocations/ConstructorTypeArgs.java + test/tools/javac/typeAnnotations/newlocations/Expressions.java + test/tools/javac/typeAnnotations/newlocations/Fields.java + test/tools/javac/typeAnnotations/newlocations/LocalVariables.java + test/tools/javac/typeAnnotations/newlocations/MethodReturnType.java + test/tools/javac/typeAnnotations/newlocations/MethodTypeArgs.java + test/tools/javac/typeAnnotations/newlocations/MethodTypeParameters.java + test/tools/javac/typeAnnotations/newlocations/Parameters.java + test/tools/javac/typeAnnotations/newlocations/Receivers.java + test/tools/javac/typeAnnotations/newlocations/Throws.java + test/tools/javac/typeAnnotations/newlocations/TypeCasts.java + test/tools/javac/typeAnnotations/newlocations/TypeParameters.java + test/tools/javac/typeAnnotations/newlocations/Wildcards.java + test/tools/javap/typeAnnotations/ClassLiterals.java + test/tools/javap/typeAnnotations/JSR175Annotations.java + test/tools/javap/typeAnnotations/NewArray.java + test/tools/javap/typeAnnotations/Presence.java + test/tools/javap/typeAnnotations/PresenceInner.java + test/tools/javap/typeAnnotations/Visibility.java From jonathan.gibbons at sun.com Sat Jun 27 02:53:36 2009 From: jonathan.gibbons at sun.com (jonathan.gibbons at sun.com) Date: Sat, 27 Jun 2009 02:53:36 +0000 Subject: hg: jdk7/tl/langtools: 6854796: update JSR308 impl with latest code from type-annotations repo Message-ID: <20090627025340.B9A90E3DB@hg.openjdk.java.net> Changeset: 7c154fdc3547 Author: jjg Date: 2009-06-26 19:47 -0700 URL: http://hg.openjdk.java.net/jdk7/tl/langtools/rev/7c154fdc3547 6854796: update JSR308 impl with latest code from type-annotations repo Reviewed-by: jjg, mcimadamore, darcy Contributed-by: mernst at cs.washington.edu, mali at csail.mit.edu, mpapi at csail.mit.edu ! src/share/classes/com/sun/tools/javac/code/TargetType.java ! src/share/classes/com/sun/tools/javac/code/TypeAnnotationPosition.java ! src/share/classes/com/sun/tools/javac/comp/Lower.java ! src/share/classes/com/sun/tools/javac/jvm/ClassWriter.java ! src/share/classes/com/sun/tools/javac/jvm/Code.java ! src/share/classes/com/sun/tools/javac/jvm/Gen.java ! src/share/classes/com/sun/tools/javac/main/JavaCompiler.java ! src/share/classes/com/sun/tools/javac/parser/JavacParser.java ! src/share/classes/com/sun/tools/javac/processing/JavacProcessingEnvironment.java ! src/share/classes/com/sun/tools/javac/processing/JavacRoundEnvironment.java From alex14n at gmail.com Sat Jun 27 03:54:12 2009 From: alex14n at gmail.com (Alex Yakovlev) Date: Sat, 27 Jun 2009 07:54:12 +0400 Subject: Faster HashMap implementation In-Reply-To: <4A40FD16.6040503@cs.oswego.edu> References: <4A2D3729.3040904@cs.oswego.edu> <4A2E5109.4050804@cs.oswego.edu> <4A40FD16.6040503@cs.oswego.edu> Message-ID: Also it should be mentioned that memory-wise my implementation uses more memory in resize: it need to copy 2 arrays of ~2*C size (C = capacity) and current HashMap needs only one array of C size, data in Entry structures are not copied. This can be addressed if it is important by (1) splitting integer arrays into main and overflow parts, (2) some splitting of key/value array... As for huge maps - I was thinking how to solve this issue better. Using additional arrays does not looks like a solution mainly because now 2 bit are reserved for control flags which leaves only up to 30 bits for real index value. Rather simple way is to have a secondary map where to put elements when main map is full. But performance will be poor - there'll be two (or number of such maps) lookups for each get/put. We could use javolution-like approach with 'map of maps', i.e using some lower bits to choose submaps. But it is an extra indirection step, thus worse performance. Another way is not to use integer array at all when capacity is at maximum, and use array of objects as in current HashMap to store HashEntries. Maybe this is a good way, but code complexity will increase - basically there will be two implementations, array-bases and entry-bases, just mixed in once class to reduce virtual calls. (Or just to proxy all calls to another class when capacity is at maximum). Or to somehow mix entry and array based storage, like adding another bit flag to look in dynamic storage, and in key/value array set key to some reserved object that will signal that value is not a real value but an Entry-like dynamic structure. But on huge maps this would be a waste of memory - it would use 3 words (integer index, reserved key value and real pointer) just to address an Entry object. Do you see better way to do that? OTOH, such a huge maps are probably used on 64-bit JVM, where array-based approach uses a lot less memory so finding a way to use pure arrays would be a big win (keeping resize problem in mind - which is also solved by javolution-like approach). On Tue, Jun 23, 2009 at 8:04 PM, Doug Lea
wrote: > > But I did want to point out a problem shared with > all purely table-based approaches, including yours. > Assuming power of 2 capacities, you can only hold up to 1 << 29 > entries, because max power of 2 array length is 1 << 30. > This holds even in -d64 mode. > Current HashMap works up through size == Integer.MAX_VALUE. > (And even "works" if size overflows so long as you don't > call size().) I know that there are some extremely > large hash maps out there. Maybe some would newly > encounters failures. > > There are ways out of this, using some sort > of dynamic structures for overflow. From martinrb at google.com Sat Jun 27 16:56:35 2009 From: martinrb at google.com (Martin Buchholz) Date: Sat, 27 Jun 2009 09:56:35 -0700 Subject: Miscellaneous improvements to "jar". In-Reply-To: <4A450656.7070008@sun.com> References: <1ccfd1c10906241934i21247553w685cf0b0b4cbd7b2@mail.gmail.com> <1ccfd1c10906251200n3bd78beek400abb51adcece40@mail.gmail.com> <1ccfd1c10906251757y72ccaca1l289dfc79f18b002c@mail.gmail.com> <4A450656.7070008@sun.com> Message-ID: <1ccfd1c10906270956g6c2f6208ne31f511ebcfaa41a@mail.gmail.com> On Fri, Jun 26, 2009 at 10:33, Xueming Shen wrote: > The latest version looks good. 2 nit comments > > (1) it's reasonable to have createTempFileInSamDirectoryAs separate out, > but I would > keep the directoryOf within it. Yes, it's "clearer":-) > Done. } /** - * A variant of File.getParentFile that always returns a valid - * directory (i.e. returns new File(".") where File.getParentFile - * would return null). - */ - private static File directoryOf(File file) { - File dir = file.getParentFile(); - return (dir != null) ? dir : new File("."); - } - - /** * Creates a new empty temporary file in the same directory as the * specified file. A variant of File.createTempFile. */ private static File createTempFileInSameDirectoryAs(File file) throws IOException { - return File.createTempFile("jartmp", null, directoryOf(file)); + File dir = file.getParentFile(); + if (dir == null) + dir = new File("."); + return File.createTempFile("jartmp", null, dir); } private boolean ok; > > (2)the "updateOK" in dumpIndex really serves nobody but "clearer":-) > I tried to improve dumpIndex by - removing updateOk variable. - using Path.moveTo is more likely to be atomic, so that a concurrent process is less likely to find the jar being updated missing. (Alan: is there a better way to do the common task of replacing a file with its transformed output?) */ void dumpIndex(String rootjar, JarIndex index) throws IOException { File jarFile = new File(rootjar); - File tmpFile = createTempFileInSameDirectoryAs(jarFile); + Path jarPath = jarFile.toPath(); + Path tmpPath = createTempFileInSameDirectoryAs(jarFile).toPath(); try { - boolean updateOk = update(new FileInputStream(jarFile), - new FileOutputStream(tmpFile), - null, index); - if (updateOk) { - jarFile.delete(); - if (!tmpFile.renameTo(jarFile)) { - throw new IOException(getMsg("error.write.file")); + if (update(jarPath.newInputStream(), + tmpPath.newOutputStream(), + null, index)) { + try { + tmpPath.moveTo(jarPath, REPLACE_EXISTING); + } catch (IOException e) { + throw new IOException(getMsg("error.write.file"), e); } } } finally { - tmpFile.delete(); + tmpPath.delete(false); } } webrev updated. Martin > > sherman > > btw, while you are here:-) do you have time to make the update() NOT to put > jarIndex the > first one when there is a manifest present? The JarInputStream assumes the > manifest is always > the first entry (or the second if the manifest dir is the first), which > causes problem...I'm not saying > to mix with this one:-) just in case you have time and interested while you > are here:-) > > > Martin Buchholz wrote: > >> I did some benchmarking, >> and found that my changes "only" make jar 10-20% faster. >> Disappointing - we expect an order of magnitude for every commit! >> >> While benchmarking, I discovered to my horror that the simple >> >> jar cf /tmp/t1 ... >> jar i /tmp/t1 >> >> fails, because it tries to create the replacement jar in "." and then >> rename() it into place. Oops... Better refactor out the code that >> puts the replacement temp file in the same directory. >> Better write some tests for that, too. >> >> /** >> * A variant of File.getParentFile that always returns a valid >> * directory (i.e. returns new File(".") where File.getParentFile >> * would return null). >> */ >> private static File directoryOf(File file) { >> File dir = file.getParentFile(); >> return (dir != null) ? dir : new File("."); >> } >> >> /** >> * Creates a new empty temporary file in the same directory as the >> * specified file. A variant of File.createTempFile. >> */ >> private static File createTempFileInSameDirectoryAs(File file) throws >> IOException { >> return File.createTempFile("jartmp", null, directoryOf(file)); >> } >> >> --- >> >> While we're here, let's remove the double buffering on file copy >> operations. >> And the repeated allocations of big buffers. >> >> /** >> * A buffer for use only by copy(InputStream, OutputStream). >> * Not as clean as allocating a new buffer as needed by copy, >> * but significantly more efficient. >> */ >> private byte[] copyBuf = new byte[8192]; >> >> /** >> * Copies all bytes from the input stream to the output stream. >> * Does not close or flush either stream. >> * >> * @param from the input stream to read from >> * @param to the output stream to write to >> * @throws IOException if an I/O error occurs >> */ >> private void copy(InputStream from, OutputStream to) throws IOException >> { >> int n; >> while ((n = from.read(copyBuf)) != -1) >> to.write(copyBuf, 0, n); >> } >> >> /** >> * Copies all bytes from the input file to the output stream. >> * Does not close or flush the output stream. >> * >> * @param from the input file to read from >> * @param to the output stream to write to >> * @throws IOException if an I/O error occurs >> */ >> private void copy(File from, OutputStream to) throws IOException { >> InputStream in = new FileInputStream(from); >> try { >> copy(in, to); >> } finally { >> in.close(); >> } >> } >> >> /** >> * Copies all bytes from the input stream to the output file. >> * Does not close the input stream. >> * >> * @param from the input stream to read from >> * @param to the output file to write to >> * @throws IOException if an I/O error occurs >> */ >> private void copy(InputStream from, File to) throws IOException { >> OutputStream out = new FileOutputStream(to); >> try { >> copy(from, out); >> } finally { >> out.close(); >> } >> } >> >> ---- >> >> On the other hand, allocating a CRC32 is very cheap, so let's make that >> private to CRC32OutputStream. >> + private static class CRC32OutputStream extends java.io.OutputStream { >> + final CRC32 crc = new CRC32(); >> >> ---- >> >> We should add a try { finally } to delete the tempfile for jar i >> + try { >> boolean updateOk = update(new FileInputStream(jarFile), >> - new FileOutputStream(scratchFile), >> >> + new FileOutputStream(tmpFile), >> null, index); >> + if (updateOk) { >> jarFile.delete(); >> >> - if (!scratchFile.renameTo(jarFile)) { >> - scratchFile.delete(); >> + if (!tmpFile.renameTo(jarFile)) { >> >> throw new IOException(getMsg("error.write.file")); >> } >> - scratchFile.delete(); >> + } >> + } finally { >> >> + tmpFile.delete(); >> + } >> } >> >> ---- >> >> Webrev updated. >> http://cr.openjdk.java.net/~martin/jar-misc/< >> http://cr.openjdk.java.net/%7Emartin/jar-misc/> >> >> There are now many small changes combined in this fix. Sorry about that. >> I'd be more inclined to separate them if creating new bugs was easier. >> >> I'm not planning on making any more changes. Really. >> >> Martin >> >> On Thu, Jun 25, 2009 at 12:00, Martin Buchholz > martinrb at google.com>> wrote: >> >> I have an updated version of this fix, with these changes: >> >> - Documented the turkish i problem >> >> /** >> * Compares two strings for equality, ignoring case. The second >> * argument must contain only upper-case ASCII characters. >> * We don't want case comparison to be locale-dependent (else we >> * have the notorious "turkish i bug"). >> */ >> private boolean equalsIgnoreCase(String s, String upper) { >> >> - Refactored code so that updateEntry now also sets the method to >> STORED. >> >> /** >> * Updates a ZipEntry which describes the data read by this >> * output stream, in STORED mode. >> */ >> public void updateEntry(ZipEntry e) { >> e.setMethod(ZipEntry.STORED); >> e.setSize(n); >> e.setCrc(crc.getValue()); >> } >> >> - addIndex was never updating the size in the ZipEntry (as required), >> which was not previously noticed because closeEntry was never >> called. >> >> private void addIndex(JarIndex index, ZipOutputStream zos) >> throws IOException >> { >> ZipEntry e = new ZipEntry(INDEX_NAME); >> e.setTime(System.currentTimeMillis()); >> if (flag0) { >> CRC32OutputStream os = new CRC32OutputStream(crc32); >> index.write(os); >> os.updateEntry(e); >> } >> zos.putNextEntry(e); >> index.write(zos); >> zos.closeEntry(); >> >> } >> >> http://cr.openjdk.java.net/~martin/jar-misc/ >> >> Previous webrev: >> http://cr.openjdk.java.net/~martin/jar-misc.0/ >> >> >> Martin >> >> >> >> On Wed, Jun 24, 2009 at 19:34, Martin Buchholz >> > wrote: >> >> Hi jar team, >> >> I have a bunch of minor improvements to >> src/share/classes/sun/tools/jar/Main.java >> >> Toby and Xueming, please review. >> >> Warning: the index code has not been maintained for many years. >> >> Xueming, please file a bug. >> >> Synopsis: Miscellaneous improvements to "jar". >> Description: >> - Use standard jdk coding style for javadoc >> - Don't create a temp file for jar index in STORED mode. >> - Don't use synchronized collections. >> - Fix javac warnings. >> - Don't define new names for things like INDEX_NAME; >> use static import instead. >> - more efficiently compare special file names in update mode. >> Update mode should be measurably faster. >> - make CRC32OutputStream a nested class. >> refactor crc32.reset and updating entry into CRC32OutputStream. >> - Fix apparently benign bug updating n in >> CRC32OutputStream.write(byte[], int, int) >> >> Evaluation: Yep. >> >> http://cr.openjdk.java.net/~martin/jar-misc/ >> >> >> Thanks, >> >> Martin >> >> >> >> > -------------- next part -------------- An HTML attachment was scrubbed... URL: From Ulf.Zibis at gmx.de Sat Jun 27 17:17:28 2009 From: Ulf.Zibis at gmx.de (Ulf Zibis) Date: Sat, 27 Jun 2009 19:17:28 +0200 Subject: Miscellaneous improvements to "jar". In-Reply-To: <1ccfd1c10906270956g6c2f6208ne31f511ebcfaa41a@mail.gmail.com> References: <1ccfd1c10906241934i21247553w685cf0b0b4cbd7b2@mail.gmail.com> <1ccfd1c10906251200n3bd78beek400abb51adcece40@mail.gmail.com> <1ccfd1c10906251757y72ccaca1l289dfc79f18b002c@mail.gmail.com> <4A450656.7070008@sun.com> <1ccfd1c10906270956g6c2f6208ne31f511ebcfaa41a@mail.gmail.com> Message-ID: <4A465428.7080405@gmx.de> Am 27.06.2009 18:56, Martin Buchholz schrieb: > > > On Fri, Jun 26, 2009 at 10:33, Xueming Shen > wrote: > > The latest version looks good. 2 nit comments > > (1) it's reasonable to have createTempFileInSamDirectoryAs > separate out, but I would > keep the directoryOf within it. Yes, it's "clearer":-) > > > Done. > > } > > /** > - * A variant of File.getParentFile that always returns a valid > - * directory (i.e. returns new File(".") where File.getParentFile > - * would return null). > - */ > - private static File directoryOf(File file) { > - File dir = file.getParentFile(); > - return (dir != null) ? dir : new File("."); > - } > - > - /** > * Creates a new empty temporary file in the same directory as the > * specified file. A variant of File.createTempFile. > */ > private static File createTempFileInSameDirectoryAs(File file) > throws IOException { > - return File.createTempFile("jartmp", null, directoryOf(file)); > + File dir = file.getParentFile(); > + if (dir == null) > + dir = new File("."); > + return File.createTempFile("jartmp", null, dir); > } > Additional 2 cents, I more would like: + return File.createTempFile("jartmp", null, (dir != null ? dir : new File("."))); Short and clear (at least for me ;-) ) -Ulf From Ulf.Zibis at gmx.de Sat Jun 27 17:20:08 2009 From: Ulf.Zibis at gmx.de (Ulf Zibis) Date: Sat, 27 Jun 2009 19:20:08 +0200 Subject: Miscellaneous improvements to "jar". In-Reply-To: <4A465428.7080405@gmx.de> References: <1ccfd1c10906241934i21247553w685cf0b0b4cbd7b2@mail.gmail.com> <1ccfd1c10906251200n3bd78beek400abb51adcece40@mail.gmail.com> <1ccfd1c10906251757y72ccaca1l289dfc79f18b002c@mail.gmail.com> <4A450656.7070008@sun.com> <1ccfd1c10906270956g6c2f6208ne31f511ebcfaa41a@mail.gmail.com> <4A465428.7080405@gmx.de> Message-ID: <4A4654C8.30609@gmx.de> Additional 2 cents, I more would like: > > + return File.createTempFile("jartmp", null, (dir != null ? dir > : new File("."))); or: + return File.createTempFile("jartmp", null, + dir != null ? dir : new File(".")); -Ulf From martinrb at google.com Sat Jun 27 17:21:58 2009 From: martinrb at google.com (Martin Buchholz) Date: Sat, 27 Jun 2009 10:21:58 -0700 Subject: Review request for 5049299 In-Reply-To: <20090623145126.6236C5654F@rebar.astron.com> References: <4A40E7C0.5080308@redhat.com> <20090623145126.6236C5654F@rebar.astron.com> Message-ID: <1ccfd1c10906271021ia3b2af8r969a2024ff967617@mail.gmail.com> Although clone(CLONE_VFORK...) didn't work out, using glibc's vfork instead did. The glibc code to handle vfork is quite different from the code for clone(CLONE_VM | CLONE_VFORK), especially for saving/restoring pids. This time, I tested on 32-bit and 64-bit Linux. Michael, please review. http://cr.openjdk.java.net/~martin/vfork-exec/ As always, we'll need a bug filed. Synopsis: (process) Use vfork, not fork, on Linux to avoid swap exhaustion And again, my changes are conflicting with Michael's changes for Solaris. I will negotiate with Michael for who gets to commit first. We will likely end up with 4 different strategies for "forking": fork, clone, vfork, and helper process. Note to integrators: the process changes continue to be high-risk. Extra JPRT runs might be a good idea. Martin On Tue, Jun 23, 2009 at 07:51, Christos Zoulas wrote: > On Jun 23, 3:33pm, aph at redhat.com (Andrew Haley) wrote: > -- Subject: Re: Review request for 5049299 > > | I can debug this. > | > | Please try first syscall(SYS_clone ...) to bypass the libc gubbins. > | That might be all you need. If that doesn't help I'll have a look. > | > | Isn't there some point at which you have to say to a Linux user "Your > | system is simply misconfigured. Fix the overcommit parameter and this > | problem will go away" ? > > Another thing to try is to add CLONE_VFORK to suspend the parent. > > christos > -------------- next part -------------- An HTML attachment was scrubbed... URL: From martinrb at google.com Sat Jun 27 17:35:33 2009 From: martinrb at google.com (Martin Buchholz) Date: Sat, 27 Jun 2009 10:35:33 -0700 Subject: Review request for 5049299 In-Reply-To: <4A40E7C0.5080308@redhat.com> References: <4A1678DB.9010209@sun.com> <1ccfd1c10906090851g6498733dw78743283cdfe405f@mail.gmail.com> <4A2E8DA1.3050004@sun.com> <1ccfd1c10906091142h465bf9adoaa1b5c87bcf6ef9f@mail.gmail.com> <1ccfd1c10906091224p6cc0ccefpfa49453466e99640@mail.gmail.com> <1ccfd1c10906091244u14607dcesc40fc6017adc7808@mail.gmail.com> <4A311337.9070706@sun.com> <1ccfd1c10906111416x1b0fveb52eb9d3db56ded@mail.gmail.com> <1ccfd1c10906221745u87082b0m80dd3b79b4646bdf@mail.gmail.com> <4A40E7C0.5080308@redhat.com> Message-ID: <1ccfd1c10906271035g660800a5n7de04efc6f1ce504@mail.gmail.com> [+ roland, jakub] Hi Andrew, I never tried using the clone() system call directly. I was scared off by all the assembly code in glibc surrounding the system call. I went in the alternative direction of using vfork, which I have been avoiding for the past decade, but seems to work. I continue to believe my glibc bug report is valid and contains an excellent test case for glibc. Perhaps other past maintainers of glibc's process code could be persuaded to try to persuade Uli? This is mostly for the good of glibc, not for java. Even if glibc fixes their bug, we won't be able to make use of it for years. OK, I'll try adding Roland and Jakub to this conversation. Hope they're still at redhat. http://sources.redhat.com/bugzilla/show_bug.cgi?id=10311 http://cr.openjdk.java.net/~martin/vfork-exec/ Martin On Tue, Jun 23, 2009 at 07:33, Andrew Haley wrote: > Martin Buchholz wrote: > > clone-exec update: > > > > I submitted the changes for this, but jtreg tests failed on 32-bit Linux > > (I had only tested on 64-bit Linux) > > > > We disabled (but did not roll back) the use of clone to allow the > > TL integration to proceed. > > > > (As I promised elsewhere...) > > I just filed a bug against upstream glibc demonstrating the problem > > with clone(CLONE_VM). You can see the small C program > > in my bug report below. > > Probably any discussion related just to the glibc bug > > can occur on the public glibc bugzilla at > > http://sources.redhat.com/bugzilla/show_bug.cgi?id=10311 > > > > glibc maintainer Uli Drepper has already responded > > saying > > > > "If you use clone() you're on your own." > > > > so if we are going to fix it, we'll have to do it ourselves. > > Help from threading/kernel hackers appreciated. > > I can debug this. > > Please try first syscall(SYS_clone ...) to bypass the libc gubbins. > That might be all you need. If that doesn't help I'll have a look. > > Isn't there some point at which you have to say to a Linux user "Your > system is simply misconfigured. Fix the overcommit parameter and this > problem will go away" ? > > Andrew. > -------------- next part -------------- An HTML attachment was scrubbed... URL: From martinrb at google.com Sat Jun 27 17:41:54 2009 From: martinrb at google.com (Martin Buchholz) Date: Sat, 27 Jun 2009 10:41:54 -0700 Subject: Review request for 5049299 In-Reply-To: <4A40E7C0.5080308@redhat.com> References: <4A1678DB.9010209@sun.com> <1ccfd1c10906090851g6498733dw78743283cdfe405f@mail.gmail.com> <4A2E8DA1.3050004@sun.com> <1ccfd1c10906091142h465bf9adoaa1b5c87bcf6ef9f@mail.gmail.com> <1ccfd1c10906091224p6cc0ccefpfa49453466e99640@mail.gmail.com> <1ccfd1c10906091244u14607dcesc40fc6017adc7808@mail.gmail.com> <4A311337.9070706@sun.com> <1ccfd1c10906111416x1b0fveb52eb9d3db56ded@mail.gmail.com> <1ccfd1c10906221745u87082b0m80dd3b79b4646bdf@mail.gmail.com> <4A40E7C0.5080308@redhat.com> Message-ID: <1ccfd1c10906271041y60e2083ahffd1ba8a8ff3712f@mail.gmail.com> On Tue, Jun 23, 2009 at 07:33, Andrew Haley wrote: > > Isn't there some point at which you have to say to a Linux user "Your > system is simply misconfigured. Fix the overcommit parameter and this > problem will go away" ? > 99% of users will use the default value of the overcommit parameter and we have to try to make that work well. I have thought about what we could do to fix Linux generally. Perhaps we could have a variant of fork() that promised the kernel that we are about to exec. Then the COW'ed pages after fork wouldn't count towards overcommit. If memory was *very* tight, one could suspend all the threads in the parent process until the child exec'ed, to minimize the number of pages that were written to in the parent. Martin > > Andrew. > -------------- next part -------------- An HTML attachment was scrubbed... URL: From jonathan.gibbons at sun.com Sat Jun 27 19:11:18 2009 From: jonathan.gibbons at sun.com (jonathan.gibbons at sun.com) Date: Sat, 27 Jun 2009 19:11:18 +0000 Subject: hg: jdk7/tl/langtools: 6855563: test broken after merge with latest parser Message-ID: <20090627191122.A106FE407@hg.openjdk.java.net> Changeset: 464d58654324 Author: jjg Date: 2009-06-27 12:04 -0700 URL: http://hg.openjdk.java.net/jdk7/tl/langtools/rev/464d58654324 6855563: test broken after merge with latest parser Reviewed-by: jjg Contributed-by: mali at csail.mit.edu ! test/tools/javac/typeAnnotations/failures/OldArray.java - test/tools/javac/typeAnnotations/failures/OldArray.out From martinrb at google.com Sat Jun 27 19:33:23 2009 From: martinrb at google.com (Martin Buchholz) Date: Sat, 27 Jun 2009 12:33:23 -0700 Subject: Review request: Use RESTARTABLE in UNIXProcess_md.c Message-ID: <1ccfd1c10906271233k4639bf5bt69aca2463cf695eb@mail.gmail.com> Michael, sorry again for the conflict. Alan, can we find a slightly more public place to put RESTARTABLE so that make/java/java/Makefile makes it accessible by default? Perhaps into java/io/io_util.h? Please file a bug and review. Synopsis: Use RESTARTABLE in UNIXProcess_md.c Description: Many system calls in UNIXProcess_md.c should handle EINTR by using the handy RESTARTABLE macro in nio_util.h http://cr.openjdk.java.net/~martin/RESTARTABLE/ Martin -------------- next part -------------- An HTML attachment was scrubbed... URL: From Alan.Bateman at Sun.COM Sat Jun 27 20:27:52 2009 From: Alan.Bateman at Sun.COM (Alan Bateman) Date: Sat, 27 Jun 2009 21:27:52 +0100 Subject: Review request: Use RESTARTABLE in UNIXProcess_md.c In-Reply-To: <1ccfd1c10906271233k4639bf5bt69aca2463cf695eb@mail.gmail.com> References: <1ccfd1c10906271233k4639bf5bt69aca2463cf695eb@mail.gmail.com> Message-ID: <4A4680C8.4030704@sun.com> Martin Buchholz wrote: > : > > Alan, can we find a slightly more public place to put RESTARTABLE > so that make/java/java/Makefile makes it accessible by default? > Perhaps into java/io/io_util.h? Good idea, as we need this everywhere we do system calls. A common header file in src/solaris/native/common is probably right (rather than io_util.h, which is for java.io). -Alan. From martinrb at google.com Sat Jun 27 21:27:18 2009 From: martinrb at google.com (Martin Buchholz) Date: Sat, 27 Jun 2009 14:27:18 -0700 Subject: Miscellaneous improvements to "jar". In-Reply-To: <4A4654C8.30609@gmx.de> References: <1ccfd1c10906241934i21247553w685cf0b0b4cbd7b2@mail.gmail.com> <1ccfd1c10906251200n3bd78beek400abb51adcece40@mail.gmail.com> <1ccfd1c10906251757y72ccaca1l289dfc79f18b002c@mail.gmail.com> <4A450656.7070008@sun.com> <1ccfd1c10906270956g6c2f6208ne31f511ebcfaa41a@mail.gmail.com> <4A465428.7080405@gmx.de> <4A4654C8.30609@gmx.de> Message-ID: <1ccfd1c10906271427h290601ebs51466aa75add1ba6@mail.gmail.com> Let's take the latest code style as a compromise we can all live with. Martin On Sat, Jun 27, 2009 at 10:20, Ulf Zibis wrote: > Additional 2 cents, I more would like: > >> >> + return File.createTempFile("jartmp", null, (dir != null ? dir : >> new File("."))); >> > > or: > > + return File.createTempFile("jartmp", null, > + dir != null ? dir : new File(".")); > > -Ulf > > > -------------- next part -------------- An HTML attachment was scrubbed... URL: From martinrb at google.com Sun Jun 28 00:21:35 2009 From: martinrb at google.com (Martin Buchholz) Date: Sat, 27 Jun 2009 17:21:35 -0700 Subject: Review request: Use RESTARTABLE in UNIXProcess_md.c In-Reply-To: <4A4680C8.4030704@sun.com> References: <1ccfd1c10906271233k4639bf5bt69aca2463cf695eb@mail.gmail.com> <4A4680C8.4030704@sun.com> Message-ID: <1ccfd1c10906271721u329970cem38eee691d1b21df3@mail.gmail.com> Although I am a strong advocate of creating common C-level infrastructure in the JDK, that's a sufficiently scary project that I'm going to leave it for now and do what everyone else has done, namely to make yet another copy of RESTARTABLE in UNIXProcess_md.c. http://cr.openjdk.java.net/~martin/RESTARTABLE/ Martin On Sat, Jun 27, 2009 at 13:27, Alan Bateman wrote: > Martin Buchholz wrote: > >> : >> >> Alan, can we find a slightly more public place to put RESTARTABLE >> so that make/java/java/Makefile makes it accessible by default? >> Perhaps into java/io/io_util.h? >> > Good idea, as we need this everywhere we do system calls. A common header > file in src/solaris/native/common is probably right (rather than io_util.h, > which is for java.io). > > -Alan. > > > > -------------- next part -------------- An HTML attachment was scrubbed... URL: From roland at redhat.com Sun Jun 28 02:33:45 2009 From: roland at redhat.com (Roland McGrath) Date: Sat, 27 Jun 2009 19:33:45 -0700 (PDT) Subject: Review request for 5049299 In-Reply-To: Martin Buchholz's message of Saturday, 27 June 2009 10:35:33 -0700 <1ccfd1c10906271035g660800a5n7de04efc6f1ce504@mail.gmail.com> References: <4A1678DB.9010209@sun.com> <1ccfd1c10906090851g6498733dw78743283cdfe405f@mail.gmail.com> <4A2E8DA1.3050004@sun.com> <1ccfd1c10906091142h465bf9adoaa1b5c87bcf6ef9f@mail.gmail.com> <1ccfd1c10906091224p6cc0ccefpfa49453466e99640@mail.gmail.com> <1ccfd1c10906091244u14607dcesc40fc6017adc7808@mail.gmail.com> <4A311337.9070706@sun.com> <1ccfd1c10906111416x1b0fveb52eb9d3db56ded@mail.gmail.com> <1ccfd1c10906221745u87082b0m80dd3b79b4646bdf@mail.gmail.com> <4A40E7C0.5080308@redhat.com> <1ccfd1c10906271035g660800a5n7de04efc6f1ce504@mail.gmail.com> Message-ID: <20090628023345.C970A420F6@magilla.sf.frob.com> I am a bit surprised that you see that failure mode, and it's possible it indicates an actual bug in pthread_getattr_np that you could find some other (kosher) way to provoke. But it is generally true that if you use -lpthread then attempting using clone() with CLONE_VM on your own at all is not a reasonable thing to expect to work. I get the impression that all you are trying to do is spawn an exec'd process in some fashion where you need to do a little bit more work on the child side than just exec alone, but not much. This is exactly the case that vfork exists to optimize, and I am rather shocked and amazed that you should ever have considered anything else before just using vfork. Aside from the portability issues, that is just the simplest and easiest option by far. I am bemused that you cite "dire warnings" about use of vfork, but are less dissuaded from something so deep in the bowels of Linux implementation, so under-specified, and so hard to use as clone. Perhaps you didn't notice any comparable dire warnings about clone because it never occurred to anyone that someone who needed to be warned would ever stumble into a thicket normally so hidden from view, and so obviously fraught with peril, as clone. The vfork caveats are suitably dire indeed--strongly indicating that the only proper use of vfork is precisely the use you need it for--but these same warnings have been posted and remained consistent since the invention of vfork in 4.2BSD something like 25 years ago. Thanks, Roland From Ulf.Zibis at gmx.de Sun Jun 28 13:10:29 2009 From: Ulf.Zibis at gmx.de (Ulf Zibis) Date: Sun, 28 Jun 2009 15:10:29 +0200 Subject: Miscellaneous improvements to "jar". In-Reply-To: <1ccfd1c10906271427h290601ebs51466aa75add1ba6@mail.gmail.com> References: <1ccfd1c10906241934i21247553w685cf0b0b4cbd7b2@mail.gmail.com> <1ccfd1c10906251200n3bd78beek400abb51adcece40@mail.gmail.com> <1ccfd1c10906251757y72ccaca1l289dfc79f18b002c@mail.gmail.com> <4A450656.7070008@sun.com> <1ccfd1c10906270956g6c2f6208ne31f511ebcfaa41a@mail.gmail.com> <4A465428.7080405@gmx.de> <4A4654C8.30609@gmx.de> <1ccfd1c10906271427h290601ebs51466aa75add1ba6@mail.gmail.com> Message-ID: <4A476BC5.8020807@gmx.de> Yes, thanks for the copious discussion. -Ulf Am 27.06.2009 23:27, Martin Buchholz schrieb: > Let's take the latest code style as a compromise we can all live with. > > Martin > > On Sat, Jun 27, 2009 at 10:20, Ulf Zibis > wrote: > > Additional 2 cents, I more would like: > > > + return File.createTempFile("jartmp", null, (dir != > null ? dir : new File("."))); > > > or: > > > + return File.createTempFile("jartmp", null, > + dir != null ? dir : new File(".")); > > -Ulf > > > From Alan.Bateman at Sun.COM Sun Jun 28 16:08:18 2009 From: Alan.Bateman at Sun.COM (Alan Bateman) Date: Sun, 28 Jun 2009 17:08:18 +0100 Subject: Miscellaneous improvements to "jar". In-Reply-To: <1ccfd1c10906270956g6c2f6208ne31f511ebcfaa41a@mail.gmail.com> References: <1ccfd1c10906241934i21247553w685cf0b0b4cbd7b2@mail.gmail.com> <1ccfd1c10906251200n3bd78beek400abb51adcece40@mail.gmail.com> <1ccfd1c10906251757y72ccaca1l289dfc79f18b002c@mail.gmail.com> <4A450656.7070008@sun.com> <1ccfd1c10906270956g6c2f6208ne31f511ebcfaa41a@mail.gmail.com> Message-ID: <4A479572.3080704@sun.com> Martin Buchholz wrote: > : > > - using Path.moveTo is more likely to be atomic, so that > a concurrent process is less likely to find the jar being updated > missing. > (Alan: is there a better way to do the common task of replacing a file > with its transformed output?) The moveTo method does have the ATOMIC_MOVE option for cases where you require it to be atomic but I don't think it is needed here. The transform into a second file and replacing the original seems fine, and probably the safest in that the original isn't trashed if something goes wrong. -Alan. From alan.bateman at sun.com Sun Jun 28 16:16:34 2009 From: alan.bateman at sun.com (alan.bateman at sun.com) Date: Sun, 28 Jun 2009 16:16:34 +0000 Subject: hg: jdk7/tl/jdk: 2 new changesets Message-ID: <20090628161726.C7BBDE416@hg.openjdk.java.net> Changeset: 5208d0c90d73 Author: alanb Date: 2009-06-27 21:46 +0100 URL: http://hg.openjdk.java.net/jdk7/tl/jdk/rev/5208d0c90d73 6838333: New I/O: Update file system API to jsr203/nio2-b101 6844313: New I/O: File timestamps should be represented by a FileTime rather than a long+TimeUnit Reviewed-by: sherman ! make/java/java/FILES_java.gmk ! make/java/nio/FILES_java.gmk ! make/java/nio/mapfile-linux ! make/java/nio/mapfile-solaris ! src/share/classes/java/io/File.java + src/share/classes/java/io/TempFileHelper.java ! src/share/classes/java/nio/channels/SeekableByteChannel.java ! src/share/classes/java/nio/file/AccessMode.java ! src/share/classes/java/nio/file/DirectoryStream.java - src/share/classes/java/nio/file/DirectoryStreamFilters.java - src/share/classes/java/nio/file/FileAction.java ! src/share/classes/java/nio/file/FileRef.java ! src/share/classes/java/nio/file/FileStore.java ! src/share/classes/java/nio/file/FileTreeWalker.java ! src/share/classes/java/nio/file/FileVisitor.java ! src/share/classes/java/nio/file/Files.java ! src/share/classes/java/nio/file/LinkPermission.java ! src/share/classes/java/nio/file/OpenOption.java ! src/share/classes/java/nio/file/Path.java ! src/share/classes/java/nio/file/Paths.java ! src/share/classes/java/nio/file/SecureDirectoryStream.java ! src/share/classes/java/nio/file/SimpleFileVisitor.java ! src/share/classes/java/nio/file/StandardWatchEventKind.java ! src/share/classes/java/nio/file/WatchKey.java ! src/share/classes/java/nio/file/attribute/AclFileAttributeView.java ! src/share/classes/java/nio/file/attribute/AttributeView.java ! src/share/classes/java/nio/file/attribute/Attributes.java ! src/share/classes/java/nio/file/attribute/BasicFileAttributeView.java ! src/share/classes/java/nio/file/attribute/BasicFileAttributes.java ! src/share/classes/java/nio/file/attribute/DosFileAttributeView.java ! src/share/classes/java/nio/file/attribute/FileAttributeView.java ! src/share/classes/java/nio/file/attribute/FileOwnerAttributeView.java ! src/share/classes/java/nio/file/attribute/FileStoreSpaceAttributeView.java + src/share/classes/java/nio/file/attribute/FileTime.java ! src/share/classes/java/nio/file/attribute/PosixFileAttributeView.java ! src/share/classes/java/nio/file/attribute/PosixFilePermissions.java ! src/share/classes/java/nio/file/attribute/UserDefinedFileAttributeView.java ! src/share/classes/java/nio/file/attribute/UserPrincipalLookupService.java - src/share/classes/java/nio/file/spi/AbstractPath.java ! src/share/classes/java/nio/file/spi/FileSystemProvider.java ! src/share/classes/java/util/Scanner.java ! src/share/classes/sun/nio/fs/AbstractAclFileAttributeView.java ! src/share/classes/sun/nio/fs/AbstractBasicFileAttributeView.java - src/share/classes/sun/nio/fs/AbstractFileStoreSpaceAttributeView.java ! src/share/classes/sun/nio/fs/AbstractFileTypeDetector.java + src/share/classes/sun/nio/fs/AbstractPath.java ! src/share/classes/sun/nio/fs/AbstractUserDefinedFileAttributeView.java + src/share/classes/sun/nio/fs/DynamicFileAttributeView.java ! src/share/classes/sun/nio/fs/FileOwnerAttributeViewImpl.java - src/share/classes/sun/nio/fs/MimeType.java ! src/share/classes/sun/nio/fs/PollingWatchService.java + src/share/classes/sun/nio/fs/Util.java ! src/share/sample/nio/file/Copy.java ! src/share/sample/nio/file/Xdd.java ! src/solaris/classes/sun/nio/fs/LinuxDosFileAttributeView.java ! src/solaris/classes/sun/nio/fs/LinuxFileStore.java ! src/solaris/classes/sun/nio/fs/LinuxFileSystem.java ! src/solaris/classes/sun/nio/fs/SolarisAclFileAttributeView.java ! src/solaris/classes/sun/nio/fs/SolarisFileStore.java ! src/solaris/classes/sun/nio/fs/SolarisFileSystem.java ! src/solaris/classes/sun/nio/fs/UnixChannelFactory.java ! src/solaris/classes/sun/nio/fs/UnixCopyFile.java ! src/solaris/classes/sun/nio/fs/UnixDirectoryStream.java ! src/solaris/classes/sun/nio/fs/UnixFileAttributeViews.java ! src/solaris/classes/sun/nio/fs/UnixFileAttributes.java ! src/solaris/classes/sun/nio/fs/UnixFileKey.java ! src/solaris/classes/sun/nio/fs/UnixFileModeAttribute.java ! src/solaris/classes/sun/nio/fs/UnixFileStore.java ! src/solaris/classes/sun/nio/fs/UnixFileSystem.java ! src/solaris/classes/sun/nio/fs/UnixFileSystemProvider.java ! src/solaris/classes/sun/nio/fs/UnixMountEntry.java ! src/solaris/classes/sun/nio/fs/UnixNativeDispatcher.java ! src/solaris/classes/sun/nio/fs/UnixPath.java ! src/solaris/classes/sun/nio/fs/UnixSecureDirectoryStream.java ! src/solaris/classes/sun/nio/fs/UnixUserPrincipals.java ! src/solaris/native/sun/nio/fs/UnixNativeDispatcher.c ! src/solaris/native/sun/nio/fs/genUnixConstants.c ! src/windows/classes/sun/nio/fs/WindowsConstants.java ! src/windows/classes/sun/nio/fs/WindowsDirectoryStream.java ! src/windows/classes/sun/nio/fs/WindowsFileAttributeViews.java ! src/windows/classes/sun/nio/fs/WindowsFileAttributes.java ! src/windows/classes/sun/nio/fs/WindowsFileStore.java ! src/windows/classes/sun/nio/fs/WindowsFileSystem.java ! src/windows/classes/sun/nio/fs/WindowsLinkSupport.java ! src/windows/classes/sun/nio/fs/WindowsNativeDispatcher.java ! src/windows/classes/sun/nio/fs/WindowsPath.java ! src/windows/native/sun/nio/fs/WindowsNativeDispatcher.c ! test/java/nio/file/DirectoryStream/Basic.java - test/java/nio/file/DirectoryStream/Filters.java ! test/java/nio/file/DirectoryStream/SecureDS.java ! test/java/nio/file/FileSystem/Basic.java ! test/java/nio/file/Files/ContentType.java ! test/java/nio/file/Files/Misc.java - test/java/nio/file/Files/content_type.sh ! test/java/nio/file/Path/CopyAndMove.java + test/java/nio/file/Path/FileAttributes.java ! test/java/nio/file/Path/Links.java ! test/java/nio/file/Path/Misc.java ! test/java/nio/file/Path/PathOps.java ! test/java/nio/file/Path/TemporaryFiles.java - test/java/nio/file/Path/temporary_files.sh ! test/java/nio/file/TestUtil.java ! test/java/nio/file/WatchService/Basic.java ! test/java/nio/file/WatchService/FileTreeModifier.java ! test/java/nio/file/attribute/AclFileAttributeView/Basic.java - test/java/nio/file/attribute/Attributes/Basic.java ! test/java/nio/file/attribute/BasicFileAttributeView/Basic.java ! test/java/nio/file/attribute/DosFileAttributeView/Basic.java ! test/java/nio/file/attribute/FileStoreAttributeView/Basic.java + test/java/nio/file/attribute/FileTime/Basic.java ! test/java/nio/file/attribute/PosixFileAttributeView/Basic.java ! test/java/nio/file/attribute/UserDefinedFileAttributeView/Basic.java Changeset: 77dd50ba670b Author: alanb Date: 2009-06-27 21:49 +0100 URL: http://hg.openjdk.java.net/jdk7/tl/jdk/rev/77dd50ba670b 6844054: (bf) Eliminate dependency on javax.management.ObjectName Reviewed-by: mchung ! src/share/classes/java/lang/management/PlatformComponent.java ! src/share/classes/java/nio/Bits.java ! src/share/classes/java/nio/Direct-X-Buffer.java ! src/share/classes/sun/management/ManagementFactoryHelper.java ! src/share/classes/sun/misc/JavaNioAccess.java ! src/share/classes/sun/nio/ch/FileChannelImpl.java From kelly.ohair at sun.com Sun Jun 28 19:04:14 2009 From: kelly.ohair at sun.com (kelly.ohair at sun.com) Date: Sun, 28 Jun 2009 19:04:14 +0000 Subject: hg: jdk7/tl/jdk: 2 new changesets Message-ID: <20090628190501.C7C38E41D@hg.openjdk.java.net> Changeset: dd20c662d463 Author: ohair Date: 2009-06-26 21:52 -0700 URL: http://hg.openjdk.java.net/jdk7/tl/jdk/rev/dd20c662d463 6855180: Fix classfile version check in java_crw_demo Reviewed-by: jjg ! src/share/demo/jvmti/java_crw_demo/java_crw_demo.c ! src/share/javavm/export/classfile_constants.h Changeset: cbb5964d97ef Author: ohair Date: 2009-06-28 11:47 -0700 URL: http://hg.openjdk.java.net/jdk7/tl/jdk/rev/cbb5964d97ef Merge - src/share/classes/java/nio/file/DirectoryStreamFilters.java - src/share/classes/java/nio/file/FileAction.java - src/share/classes/java/nio/file/spi/AbstractPath.java - src/share/classes/sun/nio/fs/AbstractFileStoreSpaceAttributeView.java - src/share/classes/sun/nio/fs/MimeType.java - test/java/nio/file/DirectoryStream/Filters.java - test/java/nio/file/Files/content_type.sh - test/java/nio/file/Path/temporary_files.sh - test/java/nio/file/attribute/Attributes/Basic.java From Xueming.Shen at Sun.COM Mon Jun 29 05:47:10 2009 From: Xueming.Shen at Sun.COM (Xueming Shen) Date: Sun, 28 Jun 2009 22:47:10 -0700 Subject: Miscellaneous improvements to "jar". In-Reply-To: <1ccfd1c10906270956g6c2f6208ne31f511ebcfaa41a@mail.gmail.com> References: <1ccfd1c10906241934i21247553w685cf0b0b4cbd7b2@mail.gmail.com> <1ccfd1c10906251200n3bd78beek400abb51adcece40@mail.gmail.com> <1ccfd1c10906251757y72ccaca1l289dfc79f18b002c@mail.gmail.com> <4A450656.7070008@sun.com> <1ccfd1c10906270956g6c2f6208ne31f511ebcfaa41a@mail.gmail.com> Message-ID: <4A48555E.10900@sun.com> Looks fine. (1) "import java.nio.file.Paths" is not used by anyone. (2)nio2 APIs are good, but now I can't not just copy/paste the jar Main to 6.x:-) sherman Martin Buchholz wrote: > > > On Fri, Jun 26, 2009 at 10:33, Xueming Shen > wrote: > > The latest version looks good. 2 nit comments > > (1) it's reasonable to have createTempFileInSamDirectoryAs > separate out, but I would > keep the directoryOf within it. Yes, it's "clearer":-) > > > Done. > > } > > /** > - * A variant of File.getParentFile that always returns a valid > - * directory (i.e. returns new File(".") where File.getParentFile > - * would return null). > - */ > - private static File directoryOf(File file) { > - File dir = file.getParentFile(); > - return (dir != null) ? dir : new File("."); > - } > - > - /** > * Creates a new empty temporary file in the same directory as the > * specified file. A variant of File.createTempFile. > */ > private static File createTempFileInSameDirectoryAs(File file) > throws IOException { > - return File.createTempFile("jartmp", null, directoryOf(file)); > + File dir = file.getParentFile(); > + if (dir == null) > + dir = new File("."); > + return File.createTempFile("jartmp", null, dir); > } > > private boolean ok; > > > > > (2)the "updateOK" in dumpIndex really serves nobody but "clearer":-) > > > I tried to improve dumpIndex by > - removing updateOk variable. > - using Path.moveTo is more likely to be atomic, so that > a concurrent process is less likely to find the jar being updated > missing. > (Alan: is there a better way to do the common task of replacing a file > with its transformed output?) > > */ > void dumpIndex(String rootjar, JarIndex index) throws IOException { > File jarFile = new File(rootjar); > - File tmpFile = createTempFileInSameDirectoryAs(jarFile); > + Path jarPath = jarFile.toPath(); > + Path tmpPath = createTempFileInSameDirectoryAs(jarFile).toPath(); > try { > - boolean updateOk = update(new FileInputStream(jarFile), > - new FileOutputStream(tmpFile), > - null, index); > - if (updateOk) { > - jarFile.delete(); > - if (!tmpFile.renameTo(jarFile)) { > - throw new IOException(getMsg("error.write.file")); > + if (update(jarPath.newInputStream(), > + tmpPath.newOutputStream(), > + null, index)) { > + try { > + tmpPath.moveTo(jarPath, REPLACE_EXISTING); > + } catch (IOException e) { > + throw new IOException(getMsg("error.write.file"), e); > } > } > } finally { > - tmpFile.delete(); > + tmpPath.delete(false); > } > } > > webrev updated. > > Martin > > > > > > sherman > > btw, while you are here:-) do you have time to make the update() > NOT to put jarIndex the > first one when there is a manifest present? The JarInputStream > assumes the manifest is always > the first entry (or the second if the manifest dir is the first), > which causes problem...I'm not saying > to mix with this one:-) just in case you have time and interested > while you are here:-) > > > Martin Buchholz wrote: > > I did some benchmarking, > and found that my changes "only" make jar 10-20% faster. > Disappointing - we expect an order of magnitude for every commit! > > While benchmarking, I discovered to my horror that the simple > > jar cf /tmp/t1 ... > jar i /tmp/t1 > > fails, because it tries to create the replacement jar in "." > and then > rename() it into place. Oops... Better refactor out the code that > puts the replacement temp file in the same directory. > Better write some tests for that, too. > > /** > * A variant of File.getParentFile that always returns a valid > * directory (i.e. returns new File(".") where > File.getParentFile > * would return null). > */ > private static File directoryOf(File file) { > File dir = file.getParentFile(); > return (dir != null) ? dir : new File("."); > } > > /** > * Creates a new empty temporary file in the same directory > as the > * specified file. A variant of File.createTempFile. > */ > private static File createTempFileInSameDirectoryAs(File > file) throws IOException { > return File.createTempFile("jartmp", null, > directoryOf(file)); > } > > --- > > While we're here, let's remove the double buffering on file > copy operations. > And the repeated allocations of big buffers. > > /** > * A buffer for use only by copy(InputStream, OutputStream). > * Not as clean as allocating a new buffer as needed by copy, > * but significantly more efficient. > */ > private byte[] copyBuf = new byte[8192]; > > /** > * Copies all bytes from the input stream to the output stream. > * Does not close or flush either stream. > * > * @param from the input stream to read from > * @param to the output stream to write to > * @throws IOException if an I/O error occurs > */ > private void copy(InputStream from, OutputStream to) throws > IOException { > int n; > while ((n = from.read(copyBuf)) != -1) > to.write(copyBuf, 0, n); > } > > /** > * Copies all bytes from the input file to the output stream. > * Does not close or flush the output stream. > * > * @param from the input file to read from > * @param to the output stream to write to > * @throws IOException if an I/O error occurs > */ > private void copy(File from, OutputStream to) throws > IOException { > InputStream in = new FileInputStream(from); > try { > copy(in, to); > } finally { > in.close(); > } > } > > /** > * Copies all bytes from the input stream to the output file. > * Does not close the input stream. > * > * @param from the input stream to read from > * @param to the output file to write to > * @throws IOException if an I/O error occurs > */ > private void copy(InputStream from, File to) throws > IOException { > OutputStream out = new FileOutputStream(to); > try { > copy(from, out); > } finally { > out.close(); > } > } > > ---- > > On the other hand, allocating a CRC32 is very cheap, so let's > make that > private to CRC32OutputStream. > + private static class CRC32OutputStream extends > java.io.OutputStream { > + final CRC32 crc = new CRC32(); > > ---- > > We should add a try { finally } to delete the tempfile for jar i > + try { > boolean updateOk = update(new FileInputStream(jarFile), > - new > FileOutputStream(scratchFile), > > + new > FileOutputStream(tmpFile), > null, index); > + if (updateOk) { > jarFile.delete(); > > - if (!scratchFile.renameTo(jarFile)) { > - scratchFile.delete(); > + if (!tmpFile.renameTo(jarFile)) { > > throw new IOException(getMsg("error.write.file")); > } > - scratchFile.delete(); > + } > + } finally { > > + tmpFile.delete(); > + } > } > > ---- > > Webrev updated. > http://cr.openjdk.java.net/~martin/jar-misc/ > > > > > There are now many small changes combined in this fix. Sorry > about that. > I'd be more inclined to separate them if creating new bugs was > easier. > > I'm not planning on making any more changes. Really. > > Martin > > On Thu, Jun 25, 2009 at 12:00, Martin Buchholz > > >> wrote: > > I have an updated version of this fix, with these changes: > > - Documented the turkish i problem > > /** > * Compares two strings for equality, ignoring case. > The second > * argument must contain only upper-case ASCII characters. > * We don't want case comparison to be locale-dependent > (else we > * have the notorious "turkish i bug"). > */ > private boolean equalsIgnoreCase(String s, String upper) { > > - Refactored code so that updateEntry now also sets the > method to > STORED. > > /** > * Updates a ZipEntry which describes the data read > by this > * output stream, in STORED mode. > */ > public void updateEntry(ZipEntry e) { > e.setMethod(ZipEntry.STORED); > e.setSize(n); > e.setCrc(crc.getValue()); > } > > - addIndex was never updating the size in the ZipEntry (as > required), > which was not previously noticed because closeEntry was never > called. > > private void addIndex(JarIndex index, ZipOutputStream zos) > throws IOException > { > ZipEntry e = new ZipEntry(INDEX_NAME); > e.setTime(System.currentTimeMillis()); > if (flag0) { > CRC32OutputStream os = new > CRC32OutputStream(crc32); > index.write(os); > os.updateEntry(e); > } > zos.putNextEntry(e); > index.write(zos); > zos.closeEntry(); > > } > > http://cr.openjdk.java.net/~martin/jar-misc/ > > > > Previous webrev: > http://cr.openjdk.java.net/~martin/jar-misc.0/ > > > > > Martin > > > > On Wed, Jun 24, 2009 at 19:34, Martin Buchholz > > >> wrote: > > Hi jar team, > > I have a bunch of minor improvements to > src/share/classes/sun/tools/jar/Main.java > > Toby and Xueming, please review. > > Warning: the index code has not been maintained for > many years. > > Xueming, please file a bug. > > Synopsis: Miscellaneous improvements to "jar". > Description: > - Use standard jdk coding style for javadoc > - Don't create a temp file for jar index in STORED mode. > - Don't use synchronized collections. > - Fix javac warnings. > - Don't define new names for things like INDEX_NAME; > use static import instead. > - more efficiently compare special file names in update > mode. > Update mode should be measurably faster. > - make CRC32OutputStream a nested class. > refactor crc32.reset and updating entry into > CRC32OutputStream. > - Fix apparently benign bug updating n in > CRC32OutputStream.write(byte[], int, int) > > Evaluation: Yep. > > http://cr.openjdk.java.net/~martin/jar-misc/ > > > > Thanks, > > Martin > > > > > From michael.mcmahon at sun.com Mon Jun 29 12:35:11 2009 From: michael.mcmahon at sun.com (michael.mcmahon at sun.com) Date: Mon, 29 Jun 2009 12:35:11 +0000 Subject: hg: jdk7/tl/jdk: 2 new changesets Message-ID: <20090629123550.923FEE540@hg.openjdk.java.net> Changeset: 806c5e4d1265 Author: michaelm Date: 2009-06-29 13:10 +0100 URL: http://hg.openjdk.java.net/jdk7/tl/jdk/rev/806c5e4d1265 6513803: httpserver regression test Test13 failing and causing NullPointerException Summary: check for NPEs Reviewed-by: chegar ! test/com/sun/net/httpserver/Test1.java ! test/com/sun/net/httpserver/Test12.java ! test/com/sun/net/httpserver/Test13.java ! test/com/sun/net/httpserver/Test3.java ! test/com/sun/net/httpserver/Test4.java ! test/com/sun/net/httpserver/Test5.java ! test/com/sun/net/httpserver/Test9.java ! test/com/sun/net/httpserver/Test9a.java ! test/com/sun/net/httpserver/TestLogging.java Changeset: b6721df9ae0a Author: michaelm Date: 2009-06-29 13:29 +0100 URL: http://hg.openjdk.java.net/jdk7/tl/jdk/rev/b6721df9ae0a Merge From christopher.hegarty at sun.com Mon Jun 29 13:59:17 2009 From: christopher.hegarty at sun.com (christopher.hegarty at sun.com) Date: Mon, 29 Jun 2009 13:59:17 +0000 Subject: hg: jdk7/tl/jdk: 6855335: Several changes in the SCTP implementation. Message-ID: <20090629135957.E49B4E547@hg.openjdk.java.net> Changeset: a44009aba19f Author: chegar Date: 2009-06-29 14:53 +0100 URL: http://hg.openjdk.java.net/jdk7/tl/jdk/rev/a44009aba19f 6855335: Several changes in the SCTP implementation. Reviewed-by: michaelm ! make/com/sun/nio/sctp/mapfile-vers ! src/solaris/classes/sun/nio/ch/SctpChannelImpl.java ! src/solaris/classes/sun/nio/ch/SctpMultiChannelImpl.java ! src/solaris/classes/sun/nio/ch/SctpNet.java ! src/solaris/classes/sun/nio/ch/SctpResultContainer.java ! src/solaris/classes/sun/nio/ch/SctpServerChannelImpl.java ! src/solaris/native/sun/nio/ch/Sctp.h ! src/solaris/native/sun/nio/ch/SctpChannelImpl.c ! src/solaris/native/sun/nio/ch/SctpNet.c ! test/com/sun/nio/sctp/SctpChannel/Connect.java ! test/com/sun/nio/sctp/SctpChannel/Shutdown.java ! test/com/sun/nio/sctp/SctpChannel/SocketOptionTests.java + test/com/sun/nio/sctp/SctpMultiChannel/Branch.java + test/com/sun/nio/sctp/SctpMultiChannel/SocketOptionTests.java From michael.mcmahon at sun.com Mon Jun 29 14:15:32 2009 From: michael.mcmahon at sun.com (michael.mcmahon at sun.com) Date: Mon, 29 Jun 2009 14:15:32 +0000 Subject: hg: jdk7/tl/jdk: 2 new changesets Message-ID: <20090629141600.3E5FCE54C@hg.openjdk.java.net> Changeset: 89b14d3740dc Author: michaelm Date: 2009-06-29 15:05 +0100 URL: http://hg.openjdk.java.net/jdk7/tl/jdk/rev/89b14d3740dc 6827999: 6827999: URLClassLoader.addURL(URL) adds URLs to closed class loader Reviewed-by: chegar ! src/share/classes/sun/misc/URLClassPath.java + test/java/net/URLClassLoader/B6827999.java Changeset: 424420bf5917 Author: michaelm Date: 2009-06-29 15:08 +0100 URL: http://hg.openjdk.java.net/jdk7/tl/jdk/rev/424420bf5917 Merge - src/share/classes/java/nio/file/DirectoryStreamFilters.java - src/share/classes/java/nio/file/FileAction.java - src/share/classes/java/nio/file/spi/AbstractPath.java - src/share/classes/sun/nio/fs/AbstractFileStoreSpaceAttributeView.java - src/share/classes/sun/nio/fs/MimeType.java - test/java/nio/file/DirectoryStream/Filters.java - test/java/nio/file/Files/content_type.sh - test/java/nio/file/Path/temporary_files.sh - test/java/nio/file/attribute/Attributes/Basic.java From Alan.Bateman at Sun.COM Mon Jun 29 17:09:01 2009 From: Alan.Bateman at Sun.COM (Alan Bateman) Date: Mon, 29 Jun 2009 18:09:01 +0100 Subject: Review request for 6843003: Windows Server 2008 R2 system recognition Message-ID: <4A48F52D.7040604@sun.com> I need a reviewer for a small update so that the os.name property is set correctly on Windows Server 2008 R2. The webrev is here: http://cr.openjdk.java.net/~alanb/6843003/webrev.00/ Thanks, Alan. From Kelly.Ohair at Sun.COM Mon Jun 29 17:19:32 2009 From: Kelly.Ohair at Sun.COM (Kelly O'Hair) Date: Mon, 29 Jun 2009 10:19:32 -0700 Subject: Review request for 6843003: Windows Server 2008 R2 system recognition In-Reply-To: <4A48F52D.7040604@sun.com> References: <4A48F52D.7040604@sun.com> Message-ID: <4A48F7A4.8050406@sun.com> Looks fine with me. -kto Alan Bateman wrote: > I need a reviewer for a small update so that the os.name property is set > correctly on Windows Server 2008 R2. The webrev is here: > http://cr.openjdk.java.net/~alanb/6843003/webrev.00/ > > Thanks, > Alan. From Xueming.Shen at Sun.COM Mon Jun 29 17:20:03 2009 From: Xueming.Shen at Sun.COM (Xueming Shen) Date: Mon, 29 Jun 2009 10:20:03 -0700 Subject: Review request for 6843003: Windows Server 2008 R2 system recognition In-Reply-To: <4A48F52D.7040604@sun.com> References: <4A48F52D.7040604@sun.com> Message-ID: <4A48F7C3.9050004@sun.com> Alan Bateman wrote: > I need a reviewer for a small update so that the os.name property is > set correctly on Windows Server 2008 R2. The webrev is here: > http://cr.openjdk.java.net/~alanb/6843003/webrev.00/ > > Thanks, > Alan. The fix itself looks OK. 2 random thoughts (1)I see lots of sprops.os_name = "Windows NT (unknown)"; It might be better to simply initialize the sprops.os_name to "Windows NT (unknown)" at the very beginning of the nt block and then depends on the version number to see if we have a better guess. (2)We have been doing the same exercise every time MS released a new os, it might be the time to extract this out to a "configurable" file and simple do a lookup based on the platform, verMajor, verMinor. Sherman From jjb at google.com Mon Jun 29 17:52:55 2009 From: jjb at google.com (Joshua Bloch) Date: Mon, 29 Jun 2009 10:52:55 -0700 Subject: OpenJDK Forum: Core Libraries Round Table In-Reply-To: <49E4852A.2020306@sun.com> References: <49DCCF4B.20506@sun.com> <49DE293B.50401@sun.com> <49DE3449.5090002@sun.com> <49DE7141.7040100@sun.com> <49DE836A.3020204@sun.com> <108fcdeb0904101124r37d49da8jf36b6b4cfe56aa8d@mail.gmail.com> <49E4770C.8020209@sun.com> <49E482C9.3050108@gmx.de> <49E4852A.2020306@sun.com> Message-ID: <17b2302a0906291052x3fbb9b47t4ea9c65bd499ca32@mail.gmail.com> Folks, I ran my TimSort benchmark on JDK6u14, server and client compilers. Here are the results: http://spreadsheets.google.com/pub?key=rn84RNB-6DNTCgzXUL9I34g&output=html http://spreadsheets.google.com/pub?key=rMiauaBPz4mNCtDN0zXhNeg&output=html . These results are on my Windows XP SP 3 laptop (Intel T2600 @ 2.16GHz with 1 GB of RAM; gotta put some more RAM in that laptop!). I'm really happy with the server compiler results, and reasonably happy with the client compiler results. I've attached the benchmark in case anyone wants to play with it. It isn't terribly complete, but easy to extend. Josh -------------- next part -------------- An HTML attachment was scrubbed... URL: -------------- next part -------------- A non-text attachment was scrubbed... Name: SortPerf.java Type: application/octet-stream Size: 1636 bytes Desc: not available URL: -------------- next part -------------- A non-text attachment was scrubbed... Name: ComparableTimSort.java Type: application/octet-stream Size: 34143 bytes Desc: not available URL: -------------- next part -------------- A non-text attachment was scrubbed... Name: Sorter.java Type: application/octet-stream Size: 836 bytes Desc: not available URL: -------------- next part -------------- A non-text attachment was scrubbed... Name: ArrayBuilder.java Type: application/octet-stream Size: 3365 bytes Desc: not available URL: From Alan.Bateman at Sun.COM Mon Jun 29 20:39:58 2009 From: Alan.Bateman at Sun.COM (Alan Bateman) Date: Mon, 29 Jun 2009 21:39:58 +0100 Subject: Review request for 6843003: Windows Server 2008 R2 system recognition In-Reply-To: <4A48F7C3.9050004@sun.com> References: <4A48F52D.7040604@sun.com> <4A48F7C3.9050004@sun.com> Message-ID: <4A49269E.5030207@sun.com> Xueming Shen wrote: > : > > The fix itself looks OK. > > 2 random thoughts > > (1)I see lots of > sprops.os_name = "Windows NT (unknown)"; > > It might be better to simply initialize the sprops.os_name to > "Windows NT (unknown)" at > the very beginning of the nt block and then depends on the version > number to see if we have > a better guess. > > (2)We have been doing the same exercise every time MS released a new > os, it might be the > time to extract this out to a "configurable" file and simple do a > lookup based on the platform, > verMajor, verMinor. It is true that we need to touch this code whenever a new edition of Windows comes along. As it runs at startup I would be hesitant to introduce a lookup file. I agree that the paths in the decision tree leading to unknown aren't prettry and this could be better. Unfortunately I don't have cycles for it just now (changing this code is trivial, but verifying it on each edition is tedious). So are you okay if this is pushed as is? -Alan. From Xueming.Shen at Sun.COM Mon Jun 29 21:03:26 2009 From: Xueming.Shen at Sun.COM (Xueming Shen) Date: Mon, 29 Jun 2009 14:03:26 -0700 Subject: Review request for 6843003: Windows Server 2008 R2 system recognition In-Reply-To: <4A49269E.5030207@sun.com> References: <4A48F52D.7040604@sun.com> <4A48F7C3.9050004@sun.com> <4A49269E.5030207@sun.com> Message-ID: <4A492C1E.3000405@sun.com> Yes, I'm OK to push asis. Alan Bateman wrote: > Xueming Shen wrote: >> : >> >> The fix itself looks OK. >> >> 2 random thoughts >> >> (1)I see lots of >> sprops.os_name = "Windows NT (unknown)"; >> >> It might be better to simply initialize the sprops.os_name to >> "Windows NT (unknown)" at >> the very beginning of the nt block and then depends on the version >> number to see if we have >> a better guess. >> >> (2)We have been doing the same exercise every time MS released a new >> os, it might be the >> time to extract this out to a "configurable" file and simple do a >> lookup based on the platform, >> verMajor, verMinor. > It is true that we need to touch this code whenever a new edition of > Windows comes along. As it runs at startup I would be hesitant to > introduce a lookup file. I agree that the paths in the decision tree > leading to unknown aren't prettry and this could be better. > Unfortunately I don't have cycles for it just now (changing this code > is trivial, but verifying it on each edition is tedious). So are you > okay if this is pushed as is? > > -Alan. From Ulf.Zibis at gmx.de Mon Jun 29 21:38:55 2009 From: Ulf.Zibis at gmx.de (Ulf Zibis) Date: Mon, 29 Jun 2009 23:38:55 +0200 Subject: Miscellaneous improvements to "jar". In-Reply-To: <4A45134C.2080105@gmx.de> References: <1ccfd1c10906241934i21247553w685cf0b0b4cbd7b2@mail.gmail.com> <1ccfd1c10906251200n3bd78beek400abb51adcece40@mail.gmail.com> <1ccfd1c10906251757y72ccaca1l289dfc79f18b002c@mail.gmail.com> <4A4488D3.9030407@gmx.de> <1ccfd1c10906260653j2f91a7ffx4a3056d67129f6a0@mail.gmail.com> <4A44F7E5.2080103@gmx.de> <4A44FB2E.2040707@sun.com> <4A45134C.2080105@gmx.de> Message-ID: <4A49346F.2080006@gmx.de> Am 26.06.2009 20:28, Ulf Zibis schrieb: > Am 26.06.2009 18:45, Xueming Shen schrieb: >> Ulf Zibis wrote: >>> >>> Motivation: >>> Xueming states: >>> *"dat" based uses less disk space, but it has larger startup time, >>> reading an additional "big" dat file during class >>> loading/initializing actually takes much longer time than I expected >>> (I hit the extreme when I worked on the EUC_TW, which I make the >>> size only 30% of the existing one, but startup is a disaster >>> regression, ... >>> * >>> If loading x bytes from dat file via getResourceAsStream() takes >>> much longer time than loading x+30% bytes from class file, >>> processing the UTF-8 conversion, instantiating and initializing >>> additional Class objects, I imperatively presume, that there must be >>> a big chance for significantly improving read speed from >>> uncompressed jar file (here charsets.jar), by using direct channels >>> or how ever. I presume, enhancing reading from jar files would be a >>> big fish in performance gain for the whole JDK, as it is very common >>> task in JVM's daily work. >>> >>> >> Ulf, the "jar reading" part of the "class loading/initializing" of >> charsets.jar (and most of the "core lib" jars) is not via >> java.util.jar/zip interface, > > Oops, sorry for misunderstanding ... > >> there is a native level interface for the vm to access those jar >> files, so adding a nio buffer interface is not going to help startup >> of jdk/jre itself. > > ... I was only thinking about the numerous "resources" loaded on > startup of jdk/jre AND later from user application, and, of course, > for more motivation to avoid heavy static data in class files. To give additional motivation on jar-lib performance, I have attached result from NetBeans slowness detector while project scan. As you can see, reading from zip files should be a big fish for performance improvement. > Additionally, some time ago on a debug session through > getResourceAsStream(), I noticed, that the construction of the > resources file's URL seams to be very wasteful (looping > StringBuffer#apend() char by char (or even strings of length 1), even > on 50..100 chars long Strings). Additionally ...findClassPathImpl() slowness is *maybe* caused by slow construction of the resources file's URL Regards, Ulf > > Any chance about outsourcing some code from API's jar code to native > code, so API customers could profit from the speed of the native level > interface of the vm to access jar files? > >> Martin's estimate of 10%-20% gain indeed is very close, I got less >> than 5%-8% (for jaring) gain from my prototype implementation. As I >> said it's still something worthing doing, especially it makes the >> life easier when work together with other nio/channel APIs. > > I agree! > > -Ulf > > > -------------- next part -------------- A non-text attachment was scrubbed... Name: NetBeans Slowness detector result.png Type: image/png Size: 120191 bytes Desc: not available URL: From martinrb at google.com Mon Jun 29 23:21:57 2009 From: martinrb at google.com (Martin Buchholz) Date: Mon, 29 Jun 2009 16:21:57 -0700 Subject: timsort Message-ID: <1ccfd1c10906291621v3a50fd2fid98c3aa14dddc8af@mail.gmail.com> Hi sort team! Google would like to contribute a new implementation for sorting of Object arrays, which has much better performance for input that is already partially sorted, based on Tim Peter's sort used in Python. This sort is already being used in the java.util. that comes with Android. Written by Josh Bloch. http://cr.openjdk.java.net/~martin/timsort/ Strictly speaking, no further review may be necessary, since it has already seen much review by Google engineers, (including some who are OpenJDK committers), and it has seen real-world usage. Nevertheless, interested parties are invited to further review it. The proposed webrev includes some very minor change to the javadoc for Arrays.sort, that we would like to include, but are also content leaving out, or to have a Sun engineer shepherd through CCC (perhaps Chris or Alan?). Martin -------------- next part -------------- An HTML attachment was scrubbed... URL: From jonathan.gibbons at sun.com Mon Jun 29 23:36:45 2009 From: jonathan.gibbons at sun.com (jonathan.gibbons at sun.com) Date: Mon, 29 Jun 2009 23:36:45 +0000 Subject: hg: jdk7/tl/jdk: 6855069: rmic should support v51 class files. Message-ID: <20090629233742.896B4E5AF@hg.openjdk.java.net> Changeset: d926534a1c17 Author: jjg Date: 2009-06-29 16:28 -0700 URL: http://hg.openjdk.java.net/jdk7/tl/jdk/rev/d926534a1c17 6855069: rmic should support v51 class files. Reviewed-by: jrose ! src/share/classes/sun/tools/java/RuntimeConstants.java From martinrb at google.com Mon Jun 29 23:52:24 2009 From: martinrb at google.com (Martin Buchholz) Date: Mon, 29 Jun 2009 16:52:24 -0700 Subject: Miscellaneous improvements to "jar". In-Reply-To: <4A48555E.10900@sun.com> References: <1ccfd1c10906241934i21247553w685cf0b0b4cbd7b2@mail.gmail.com> <1ccfd1c10906251200n3bd78beek400abb51adcece40@mail.gmail.com> <1ccfd1c10906251757y72ccaca1l289dfc79f18b002c@mail.gmail.com> <4A450656.7070008@sun.com> <1ccfd1c10906270956g6c2f6208ne31f511ebcfaa41a@mail.gmail.com> <4A48555E.10900@sun.com> Message-ID: <1ccfd1c10906291652k20d9cc60x2054f4b66c9eec0a@mail.gmail.com> On Sun, Jun 28, 2009 at 22:47, Xueming Shen wrote: > > Looks fine. > > (1) "import java.nio.file.Paths" is not used by anyone. Let's make sure to start using nio2 then... > > (2)nio2 APIs are good, but now I can't not just copy/paste the jar Main to > 6.x:-) > That *is* a problem. Fortunately, it's a small change. Is there anything else stopping the jdk6 version from being identical to jdk7? Martin > > sherman > > > Martin Buchholz wrote: > >> >> >> On Fri, Jun 26, 2009 at 10:33, Xueming Shen > Xueming.Shen at sun.com>> wrote: >> >> The latest version looks good. 2 nit comments >> >> (1) it's reasonable to have createTempFileInSamDirectoryAs >> separate out, but I would >> keep the directoryOf within it. Yes, it's "clearer":-) >> >> >> Done. >> >> } >> /** >> - * A variant of File.getParentFile that always returns a valid >> - * directory (i.e. returns new File(".") where File.getParentFile >> - * would return null). >> - */ >> - private static File directoryOf(File file) { >> - File dir = file.getParentFile(); >> - return (dir != null) ? dir : new File("."); >> - } >> - >> - /** >> * Creates a new empty temporary file in the same directory as the >> * specified file. A variant of File.createTempFile. >> */ >> private static File createTempFileInSameDirectoryAs(File file) >> throws IOException { >> - return File.createTempFile("jartmp", null, directoryOf(file)); >> + File dir = file.getParentFile(); >> + if (dir == null) >> + dir = new File("."); >> + return File.createTempFile("jartmp", null, dir); >> } >> private boolean ok; >> >> >> >> (2)the "updateOK" in dumpIndex really serves nobody but "clearer":-) >> >> >> I tried to improve dumpIndex by >> - removing updateOk variable. >> - using Path.moveTo is more likely to be atomic, so that >> a concurrent process is less likely to find the jar being updated >> missing. >> (Alan: is there a better way to do the common task of replacing a file >> with its transformed output?) >> >> */ >> void dumpIndex(String rootjar, JarIndex index) throws IOException { >> File jarFile = new File(rootjar); >> - File tmpFile = createTempFileInSameDirectoryAs(jarFile); >> + Path jarPath = jarFile.toPath(); >> + Path tmpPath = createTempFileInSameDirectoryAs(jarFile).toPath(); >> try { >> - boolean updateOk = update(new FileInputStream(jarFile), >> - new FileOutputStream(tmpFile), >> - null, index); >> - if (updateOk) { >> - jarFile.delete(); >> - if (!tmpFile.renameTo(jarFile)) { >> - throw new IOException(getMsg("error.write.file")); >> + if (update(jarPath.newInputStream(), >> + tmpPath.newOutputStream(), >> + null, index)) { >> + try { >> + tmpPath.moveTo(jarPath, REPLACE_EXISTING); >> + } catch (IOException e) { >> + throw new IOException(getMsg("error.write.file"), e); >> } >> } >> } finally { >> - tmpFile.delete(); >> + tmpPath.delete(false); >> } >> } >> >> webrev updated. >> Martin >> >> >> >> >> sherman >> >> btw, while you are here:-) do you have time to make the update() >> NOT to put jarIndex the >> first one when there is a manifest present? The JarInputStream >> assumes the manifest is always >> the first entry (or the second if the manifest dir is the first), >> which causes problem...I'm not saying >> to mix with this one:-) just in case you have time and interested >> while you are here:-) >> >> >> Martin Buchholz wrote: >> >> I did some benchmarking, >> and found that my changes "only" make jar 10-20% faster. >> Disappointing - we expect an order of magnitude for every commit! >> >> While benchmarking, I discovered to my horror that the simple >> >> jar cf /tmp/t1 ... >> jar i /tmp/t1 >> >> fails, because it tries to create the replacement jar in "." >> and then >> rename() it into place. Oops... Better refactor out the code that >> puts the replacement temp file in the same directory. >> Better write some tests for that, too. >> >> /** >> * A variant of File.getParentFile that always returns a valid >> * directory (i.e. returns new File(".") where >> File.getParentFile >> * would return null). >> */ >> private static File directoryOf(File file) { >> File dir = file.getParentFile(); >> return (dir != null) ? dir : new File("."); >> } >> >> /** >> * Creates a new empty temporary file in the same directory >> as the >> * specified file. A variant of File.createTempFile. >> */ >> private static File createTempFileInSameDirectoryAs(File >> file) throws IOException { >> return File.createTempFile("jartmp", null, >> directoryOf(file)); >> } >> >> --- >> >> While we're here, let's remove the double buffering on file >> copy operations. >> And the repeated allocations of big buffers. >> >> /** >> * A buffer for use only by copy(InputStream, OutputStream). >> * Not as clean as allocating a new buffer as needed by copy, >> * but significantly more efficient. >> */ >> private byte[] copyBuf = new byte[8192]; >> >> /** >> * Copies all bytes from the input stream to the output stream. >> * Does not close or flush either stream. >> * >> * @param from the input stream to read from >> * @param to the output stream to write to >> * @throws IOException if an I/O error occurs >> */ >> private void copy(InputStream from, OutputStream to) throws >> IOException { >> int n; >> while ((n = from.read(copyBuf)) != -1) >> to.write(copyBuf, 0, n); >> } >> >> /** >> * Copies all bytes from the input file to the output stream. >> * Does not close or flush the output stream. >> * >> * @param from the input file to read from >> * @param to the output stream to write to >> * @throws IOException if an I/O error occurs >> */ >> private void copy(File from, OutputStream to) throws >> IOException { >> InputStream in = new FileInputStream(from); >> try { >> copy(in, to); >> } finally { >> in.close(); >> } >> } >> >> /** >> * Copies all bytes from the input stream to the output file. >> * Does not close the input stream. >> * >> * @param from the input stream to read from >> * @param to the output file to write to >> * @throws IOException if an I/O error occurs >> */ >> private void copy(InputStream from, File to) throws >> IOException { >> OutputStream out = new FileOutputStream(to); >> try { >> copy(from, out); >> } finally { >> out.close(); >> } >> } >> >> ---- >> >> On the other hand, allocating a CRC32 is very cheap, so let's >> make that >> private to CRC32OutputStream. >> + private static class CRC32OutputStream extends >> java.io.OutputStream { >> + final CRC32 crc = new CRC32(); >> >> ---- >> >> We should add a try { finally } to delete the tempfile for jar i >> + try { >> boolean updateOk = update(new FileInputStream(jarFile), >> - new >> FileOutputStream(scratchFile), >> >> + new >> FileOutputStream(tmpFile), >> null, index); >> + if (updateOk) { >> jarFile.delete(); >> >> - if (!scratchFile.renameTo(jarFile)) { >> - scratchFile.delete(); >> + if (!tmpFile.renameTo(jarFile)) { >> >> throw new IOException(getMsg("error.write.file")); >> } >> - scratchFile.delete(); >> + } >> + } finally { >> >> + tmpFile.delete(); >> + } >> } >> ---- >> >> Webrev updated. >> http://cr.openjdk.java.net/~martin/jar-misc/ >> >> >> >> >> There are now many small changes combined in this fix. Sorry >> about that. >> I'd be more inclined to separate them if creating new bugs was >> easier. >> >> I'm not planning on making any more changes. Really. >> >> Martin >> >> On Thu, Jun 25, 2009 at 12:00, Martin Buchholz >> >> >> wrote: >> >> I have an updated version of this fix, with these changes: >> >> - Documented the turkish i problem >> >> /** >> * Compares two strings for equality, ignoring case. >> The second >> * argument must contain only upper-case ASCII characters. >> * We don't want case comparison to be locale-dependent >> (else we >> * have the notorious "turkish i bug"). >> */ >> private boolean equalsIgnoreCase(String s, String upper) { >> >> - Refactored code so that updateEntry now also sets the >> method to >> STORED. >> >> /** >> * Updates a ZipEntry which describes the data read >> by this >> * output stream, in STORED mode. >> */ >> public void updateEntry(ZipEntry e) { >> e.setMethod(ZipEntry.STORED); >> e.setSize(n); >> e.setCrc(crc.getValue()); >> } >> >> - addIndex was never updating the size in the ZipEntry (as >> required), >> which was not previously noticed because closeEntry was never >> called. >> >> private void addIndex(JarIndex index, ZipOutputStream zos) >> throws IOException >> { >> ZipEntry e = new ZipEntry(INDEX_NAME); >> e.setTime(System.currentTimeMillis()); >> if (flag0) { >> CRC32OutputStream os = new >> CRC32OutputStream(crc32); >> index.write(os); >> os.updateEntry(e); >> } >> zos.putNextEntry(e); >> index.write(zos); >> zos.closeEntry(); >> >> } >> >> http://cr.openjdk.java.net/~martin/jar-misc/ >> >> >> >> Previous webrev: >> http://cr.openjdk.java.net/~martin/jar-misc.0/ >> >> >> >> >> Martin >> >> >> >> On Wed, Jun 24, 2009 at 19:34, Martin Buchholz >> >> >> wrote: >> >> Hi jar team, >> >> I have a bunch of minor improvements to >> src/share/classes/sun/tools/jar/Main.java >> >> Toby and Xueming, please review. >> >> Warning: the index code has not been maintained for >> many years. >> >> Xueming, please file a bug. >> >> Synopsis: Miscellaneous improvements to "jar". >> Description: >> - Use standard jdk coding style for javadoc >> - Don't create a temp file for jar index in STORED mode. >> - Don't use synchronized collections. >> - Fix javac warnings. >> - Don't define new names for things like INDEX_NAME; >> use static import instead. >> - more efficiently compare special file names in update >> mode. >> Update mode should be measurably faster. >> - make CRC32OutputStream a nested class. >> refactor crc32.reset and updating entry into >> CRC32OutputStream. >> - Fix apparently benign bug updating n in >> CRC32OutputStream.write(byte[], int, int) >> >> Evaluation: Yep. >> >> http://cr.openjdk.java.net/~martin/jar-misc/ >> >> >> >> Thanks, >> >> Martin >> >> >> >> >> >> > -------------- next part -------------- An HTML attachment was scrubbed... URL: From Xueming.Shen at Sun.COM Mon Jun 29 23:58:48 2009 From: Xueming.Shen at Sun.COM (Xueming Shen) Date: Mon, 29 Jun 2009 16:58:48 -0700 Subject: Miscellaneous improvements to "jar". In-Reply-To: <1ccfd1c10906291652k20d9cc60x2054f4b66c9eec0a@mail.gmail.com> References: <1ccfd1c10906241934i21247553w685cf0b0b4cbd7b2@mail.gmail.com> <1ccfd1c10906251200n3bd78beek400abb51adcece40@mail.gmail.com> <1ccfd1c10906251757y72ccaca1l289dfc79f18b002c@mail.gmail.com> <4A450656.7070008@sun.com> <1ccfd1c10906270956g6c2f6208ne31f511ebcfaa41a@mail.gmail.com> <4A48555E.10900@sun.com> <1ccfd1c10906291652k20d9cc60x2054f4b66c9eec0a@mail.gmail.com> Message-ID: <4A495538.5040605@sun.com> Martin Buchholz wrote: > > > On Sun, Jun 28, 2009 at 22:47, Xueming Shen > wrote: > > > Looks fine. > > (1) "import java.nio.file.Paths" is not used by anyone. > > > Let's make sure to start using nio2 then... I meant this "import" might not be necessary? > > > > (2)nio2 APIs are good, but now I can't not just copy/paste the jar > Main to 6.x:-) > > > That *is* a problem. Fortunately, it's a small change. > > Is there anything else stopping the jdk6 version from being identical > to jdk7? > Probably this nio2 one is the only one now. From mr at sun.com Tue Jun 30 00:08:37 2009 From: mr at sun.com (Mark Reinhold) Date: Mon, 29 Jun 2009 17:08:37 -0700 Subject: Review request for 6843003: Windows Server 2008 R2 system recognition In-Reply-To: alan.bateman@sun.com; Mon, 29 Jun 2009 21:39:58 BST; <4A49269E.5030207@sun.com> Message-ID: <20090630000837.DAE0494A8@callebaut.niobe.net> > Date: Mon, 29 Jun 2009 21:39:58 +0100 > From: alan.bateman at sun.com > Xueming Shen wrote: >> ... >> >> It might be better to simply initialize the sprops.os_name to >> "Windows NT (unknown)" at >> the very beginning of the nt block and then depends on the version >> number to see if we have >> a better guess. >> >> (2)We have been doing the same exercise every time MS released a new >> os, it might be the >> time to extract this out to a "configurable" file and simple do a >> lookup based on the platform, >> verMajor, verMinor. > > It is true that we need to touch this code whenever a new edition of > Windows comes along. As it runs at startup I would be hesitant to > introduce a lookup file. Indeed. We should be reducing the number of files we open at startup, not adding to them. - Mark From martinrb at google.com Tue Jun 30 00:43:01 2009 From: martinrb at google.com (Martin Buchholz) Date: Mon, 29 Jun 2009 17:43:01 -0700 Subject: Miscellaneous improvements to "jar". In-Reply-To: <4A495538.5040605@sun.com> References: <1ccfd1c10906241934i21247553w685cf0b0b4cbd7b2@mail.gmail.com> <1ccfd1c10906251200n3bd78beek400abb51adcece40@mail.gmail.com> <1ccfd1c10906251757y72ccaca1l289dfc79f18b002c@mail.gmail.com> <4A450656.7070008@sun.com> <1ccfd1c10906270956g6c2f6208ne31f511ebcfaa41a@mail.gmail.com> <4A48555E.10900@sun.com> <1ccfd1c10906291652k20d9cc60x2054f4b66c9eec0a@mail.gmail.com> <4A495538.5040605@sun.com> Message-ID: <1ccfd1c10906291743w247cf00fv8fcd1e77fea7225d@mail.gmail.com> On Mon, Jun 29, 2009 at 16:58, Xueming Shen wrote: > Martin Buchholz wrote: > >> >> >> On Sun, Jun 28, 2009 at 22:47, Xueming Shen > Xueming.Shen at sun.com>> wrote: >> >> >> Looks fine. >> >> (1) "import java.nio.file.Paths" is not used by anyone. >> >> >> Let's make sure to start using nio2 then... >> > I meant this "import" might not be necessary? Oops. Fixed. I also sync'ed and updated to the very latest Path.deleteIfExists(). http://cr.openjdk.java.net/~martin/jar-misc/ Martin -------------- next part -------------- An HTML attachment was scrubbed... URL: From jonathan.gibbons at sun.com Tue Jun 30 00:52:13 2009 From: jonathan.gibbons at sun.com (jonathan.gibbons at sun.com) Date: Tue, 30 Jun 2009 00:52:13 +0000 Subject: hg: jdk7/tl/langtools: 6855993: fix comments in langtools launcher script Message-ID: <20090630005218.BCA08E5C9@hg.openjdk.java.net> Changeset: 7913e72a24b0 Author: jjg Date: 2009-06-29 17:45 -0700 URL: http://hg.openjdk.java.net/jdk7/tl/langtools/rev/7913e72a24b0 6855993: fix comments in langtools launcher script Reviewed-by: ohair ! src/share/bin/launcher.sh-template From martinrb at google.com Tue Jun 30 01:23:40 2009 From: martinrb at google.com (Martin Buchholz) Date: Mon, 29 Jun 2009 18:23:40 -0700 Subject: Review request for 5049299 In-Reply-To: <20090628023345.C970A420F6@magilla.sf.frob.com> References: <4A1678DB.9010209@sun.com> <1ccfd1c10906091142h465bf9adoaa1b5c87bcf6ef9f@mail.gmail.com> <1ccfd1c10906091224p6cc0ccefpfa49453466e99640@mail.gmail.com> <1ccfd1c10906091244u14607dcesc40fc6017adc7808@mail.gmail.com> <4A311337.9070706@sun.com> <1ccfd1c10906111416x1b0fveb52eb9d3db56ded@mail.gmail.com> <1ccfd1c10906221745u87082b0m80dd3b79b4646bdf@mail.gmail.com> <4A40E7C0.5080308@redhat.com> <1ccfd1c10906271035g660800a5n7de04efc6f1ce504@mail.gmail.com> <20090628023345.C970A420F6@magilla.sf.frob.com> Message-ID: <1ccfd1c10906291823p366255e5t678bcade6c5aa08@mail.gmail.com> My thinking on vfork/clone was affected by: --- the solaris man page says http://docs.sun.com/app/docs/doc/816-5167/vfork-2?a=view The use of *vfork()* in multithreaded applications, however, is unsafe due to race conditons that can cause the child process to become deadlocked and consequently block both the child and parent process from execution indefinitely. (In fact, I'm still afraid that one of those race conditions will kill the vfork option even on Linux) That man page also says, The *vfork()* function is deprecated. Its sole legitimate use as a prelude to an immediate call to a function from the exec family can be achieved safely by posix_spawn(3C)or posix_spawnp(3C) . But...posix_spawn doesn't give you any way to delete *all* file descriptors and if you try to collect them before spawning, there is a race in a multithreaded program. (Aside: I also wonder why glibc's implementation of posix_spawn avoids using vfork if there are file actions specified) --- Linux clone has avoided vfork's bad press, and has occasionally been described as "elegant". For a while I believed that clone() was the only system call that created new processes, and that vfork() was just an inflexible special case of clone(), and on ia64 that appears to be true, but on x86 clone(), fork() and vfork() are separate system calls, and glibc has different gobs of assembly code around each. In particular, glibc's clone fiddles with TIDs even when CLONE_THREAD is not specified, while vfork never does. That feels like a bug. ~/src/glibc-2.10.1 $ for x in vfork clone; do echo --- $x --- ; find -name $x.S | xargs grep -l -i tid; done --- vfork --- --- clone --- ./sysdeps/unix/sysv/linux/s390/s390-64/clone.S ./sysdeps/unix/sysv/linux/s390/s390-32/clone.S ./sysdeps/unix/sysv/linux/x86_64/clone.S ./sysdeps/unix/sysv/linux/powerpc/powerpc64/clone.S ./sysdeps/unix/sysv/linux/powerpc/powerpc32/clone.S ./sysdeps/unix/sysv/linux/i386/clone.S ./sysdeps/unix/sysv/linux/sh/clone.S ./sysdeps/unix/sysv/linux/sparc/sparc32/clone.S ./sysdeps/unix/sysv/linux/sparc/sparc64/clone.S I'm still hoping we can fix something in glibc. Martin On Sat, Jun 27, 2009 at 19:33, Roland McGrath wrote: > I am a bit surprised that you see that failure mode, and it's possible it > indicates an actual bug in pthread_getattr_np that you could find some > other (kosher) way to provoke. But it is generally true that if you use > -lpthread then attempting using clone() with CLONE_VM on your own at all is > not a reasonable thing to expect to work. > > I get the impression that all you are trying to do is spawn an exec'd > process in some fashion where you need to do a little bit more work on the > child side than just exec alone, but not much. This is exactly the case > that vfork exists to optimize, and I am rather shocked and amazed that you > should ever have considered anything else before just using vfork. Aside > from the portability issues, that is just the simplest and easiest option > by far. > > I am bemused that you cite "dire warnings" about use of vfork, but are less > dissuaded from something so deep in the bowels of Linux implementation, so > under-specified, and so hard to use as clone. Perhaps you didn't notice > any comparable dire warnings about clone because it never occurred to > anyone that someone who needed to be warned would ever stumble into a > thicket normally so hidden from view, and so obviously fraught with peril, > as clone. The vfork caveats are suitably dire indeed--strongly indicating > that the only proper use of vfork is precisely the use you need it for--but > these same warnings have been posted and remained consistent since the > invention of vfork in 4.2BSD something like 25 years ago. > > > Thanks, > Roland > -------------- next part -------------- An HTML attachment was scrubbed... URL: From roland at redhat.com Tue Jun 30 02:28:22 2009 From: roland at redhat.com (Roland McGrath) Date: Mon, 29 Jun 2009 19:28:22 -0700 (PDT) Subject: Review request for 5049299 In-Reply-To: Martin Buchholz's message of Monday, 29 June 2009 18:23:40 -0700 <1ccfd1c10906291823p366255e5t678bcade6c5aa08@mail.gmail.com> References: <4A1678DB.9010209@sun.com> <1ccfd1c10906091142h465bf9adoaa1b5c87bcf6ef9f@mail.gmail.com> <1ccfd1c10906091224p6cc0ccefpfa49453466e99640@mail.gmail.com> <1ccfd1c10906091244u14607dcesc40fc6017adc7808@mail.gmail.com> <4A311337.9070706@sun.com> <1ccfd1c10906111416x1b0fveb52eb9d3db56ded@mail.gmail.com> <1ccfd1c10906221745u87082b0m80dd3b79b4646bdf@mail.gmail.com> <4A40E7C0.5080308@redhat.com> <1ccfd1c10906271035g660800a5n7de04efc6f1ce504@mail.gmail.com> <20090628023345.C970A420F6@magilla.sf.frob.com> <1ccfd1c10906291823p366255e5t678bcade6c5aa08@mail.gmail.com> Message-ID: <20090630022822.2F69F420F6@magilla.sf.frob.com> > But...posix_spawn doesn't give you any way to delete *all* file descriptors > and if you try to collect them before spawning, there is a race in a > multithreaded program. This is something you should never want to do. There is notoriously no good way to do it on some systems, where getdtablesize() can return RLIM_INFINITY and is not proper to use in a loop. Use FD_CLOEXEC when you open the descriptor in the first place. > (Aside: I also wonder why glibc's implementation of posix_spawn avoids > using vfork if there are file actions specified) Hmm, I'm not sure about that. I also have no idea why you aren't asking these questions on the libc-alpha mailing list. > Linux clone has avoided vfork's bad press, and has occasionally been > described as "elegant". For a while I believed that clone() was the only > system call that created new processes, and that vfork() was just an > inflexible special case of clone(), and on ia64 that appears to be true, > but on x86 clone(), fork() and vfork() are separate system calls, > and glibc has different gobs of assembly code around each. Internally it's true. On some machines there are separate vfork and/or fork calls, some not. If you are worried about what the semantics mean, it is immaterial which system call number you used and how many arguments you passed. It's all the same "clone" code in the kernel. Thanks, Roland From weijun.wang at sun.com Tue Jun 30 04:04:42 2009 From: weijun.wang at sun.com (weijun.wang at sun.com) Date: Tue, 30 Jun 2009 04:04:42 +0000 Subject: hg: jdk7/tl/jdk: 6855671: DerOutputStream encodes negative integer incorrectly Message-ID: <20090630040542.95433E5D3@hg.openjdk.java.net> Changeset: 605d3fa6764e Author: weijun Date: 2009-06-30 11:55 +0800 URL: http://hg.openjdk.java.net/jdk7/tl/jdk/rev/605d3fa6764e 6855671: DerOutputStream encodes negative integer incorrectly Reviewed-by: xuelei ! src/share/classes/sun/security/util/DerOutputStream.java + test/sun/security/util/DerValue/NegInt.java From xueming.shen at sun.com Tue Jun 30 04:26:20 2009 From: xueming.shen at sun.com (xueming.shen at sun.com) Date: Tue, 30 Jun 2009 04:26:20 +0000 Subject: hg: jdk7/tl/jdk: 2 new changesets Message-ID: <20090630042720.8104EE5D8@hg.openjdk.java.net> Changeset: 1cc9eb0c952e Author: sherman Date: 2009-06-29 19:57 -0700 URL: http://hg.openjdk.java.net/jdk7/tl/jdk/rev/1cc9eb0c952e 6707281: Adler32.update() JavaDoc is wrong 6553961: java.util.zip.{CRC32,Adler32}.update(int) doc errors 6646605: Missing method ZipFile.getComment() 6841232: ZipFile should implement Closeable 4985614: Failure on calls to ZipFile constructor 5032358: "java.util.zip.ZipException: The system cannot find the file specified" 6846616: java/util/zip/ZipFile/ReadAfterClose.java failed after fix for 6735255 Summary: some misc bug/rfe fixes for zipfile Reviewed-by: alanb ! make/java/java/mapfile-vers ! make/java/zip/mapfile-vers ! src/share/classes/java/util/zip/Adler32.java ! src/share/classes/java/util/zip/CRC32.java ! src/share/classes/java/util/zip/ZipFile.java ! src/share/native/java/util/zip/ZipFile.c ! src/share/native/java/util/zip/zip_util.c ! src/share/native/java/util/zip/zip_util.h ! test/java/util/zip/ZipFile/ReadAfterClose.java ! test/java/util/zip/ZipFile/ReadZip.java Changeset: b144685f6694 Author: sherman Date: 2009-06-29 21:16 -0700 URL: http://hg.openjdk.java.net/jdk7/tl/jdk/rev/b144685f6694 Merge From Michael.McMahon at Sun.COM Tue Jun 30 09:48:05 2009 From: Michael.McMahon at Sun.COM (Michael McMahon) Date: Tue, 30 Jun 2009 10:48:05 +0100 Subject: Review request for 5049299 In-Reply-To: <20090630022822.2F69F420F6@magilla.sf.frob.com> References: <4A1678DB.9010209@sun.com> <1ccfd1c10906091142h465bf9adoaa1b5c87bcf6ef9f@mail.gmail.com> <1ccfd1c10906091224p6cc0ccefpfa49453466e99640@mail.gmail.com> <1ccfd1c10906091244u14607dcesc40fc6017adc7808@mail.gmail.com> <4A311337.9070706@sun.com> <1ccfd1c10906111416x1b0fveb52eb9d3db56ded@mail.gmail.com> <1ccfd1c10906221745u87082b0m80dd3b79b4646bdf@mail.gmail.com> <4A40E7C0.5080308@redhat.com> <1ccfd1c10906271035g660800a5n7de04efc6f1ce504@mail.gmail.com> <20090628023345.C970A420F6@magilla.sf.frob.com> <1ccfd1c10906291823p366255e5t678bcade6c5aa08@mail.gmail.com> <20090630022822.2F69F420F6@magilla.sf.frob.com> Message-ID: <4A49DF55.5090807@sun.com> Roland McGrath wrote: >> But...posix_spawn doesn't give you any way to delete *all* file descriptors >> and if you try to collect them before spawning, there is a race in a >> multithreaded program. >> > > This is something you should never want to do. There is notoriously no > good way to do it on some systems, where getdtablesize() can return > RLIM_INFINITY and is not proper to use in a loop. > > Use FD_CLOEXEC when you open the descriptor in the first place. > > We also need to chdir() before calling exec. And while some descriptors already have FD_CLOEXEC set, it's not easy to guarantee this for all of them. - Michael. From Alan.Bateman at Sun.COM Tue Jun 30 10:04:56 2009 From: Alan.Bateman at Sun.COM (Alan Bateman) Date: Tue, 30 Jun 2009 11:04:56 +0100 Subject: timsort In-Reply-To: <1ccfd1c10906291621v3a50fd2fid98c3aa14dddc8af@mail.gmail.com> References: <1ccfd1c10906291621v3a50fd2fid98c3aa14dddc8af@mail.gmail.com> Message-ID: <4A49E348.5090902@sun.com> Martin Buchholz wrote: > Hi sort team! > > Google would like to contribute a new implementation for sorting of > Object arrays, > which has much better performance for input that is already partially > sorted, > based on Tim Peter's sort used in Python. > > This sort is already being used in the java.util. that comes with Android. > > Written by Josh Bloch. > > http://cr.openjdk.java.net/~martin/timsort/ > > > Strictly speaking, no further review may be necessary, > since it has already seen much review by Google engineers, > (including some who are OpenJDK committers), > and it has seen real-world usage. > > Nevertheless, interested parties are invited to further review it. > > The proposed webrev includes some very minor change to > the javadoc for Arrays.sort, that we would like to include, > but are also content leaving out, or to have a Sun engineer > shepherd through CCC (perhaps Chris or Alan?). The results look very good. I assume the only contentious issue is going to be the IAE when a broken implementation of Comparable is detected. When you say "also content leaving out", do you mean just the spec update, or do you mean that the exception won't be thrown if detected? I wonder if this will need a compatibility switch so as not to change the failure mode of existing (broken) applications. Also, I assume failure atomicity is not possible here (right?); just wondering if this needs to be stated in the javadoc for this exception. I haven't studied the code but from a brief glance, I think the "Classpath" exception might be missing from the new files in src/share/classes/java/util. Also, should the mergeSort implementation be removed or moved out of Arrays? -Alan. From alan.bateman at sun.com Tue Jun 30 13:11:02 2009 From: alan.bateman at sun.com (alan.bateman at sun.com) Date: Tue, 30 Jun 2009 13:11:02 +0000 Subject: hg: jdk7/tl/jdk: 2 new changesets Message-ID: <20090630131150.5BD11E6A4@hg.openjdk.java.net> Changeset: b22f9e823be7 Author: alanb Date: 2009-06-30 11:11 +0100 URL: http://hg.openjdk.java.net/jdk7/tl/jdk/rev/b22f9e823be7 6843003: Windows Server 2008 R2 system recognition Reviewed-by: ohair, sherman ! src/windows/native/java/lang/java_props_md.c Changeset: d57c10cf07c5 Author: alanb Date: 2009-06-30 11:13 +0100 URL: http://hg.openjdk.java.net/jdk7/tl/jdk/rev/d57c10cf07c5 Merge From martinrb at google.com Tue Jun 30 16:12:27 2009 From: martinrb at google.com (Martin Buchholz) Date: Tue, 30 Jun 2009 09:12:27 -0700 Subject: timsort In-Reply-To: <4A49E348.5090902@sun.com> References: <1ccfd1c10906291621v3a50fd2fid98c3aa14dddc8af@mail.gmail.com> <4A49E348.5090902@sun.com> Message-ID: <1ccfd1c10906300912u2250d0c5l819535e1ea4ffec4@mail.gmail.com> Thanks, Alan. You're a good reviewer. As you suggested, I added the Classpath exception wording to TimSort.java and ComparableTimSort.java. I excised the old mergesort code from Arrays.java. webrev regenerated. Adding all-or-nothing wording would be good to add, perhaps to the class-level javadoc. But it doesn't have to be part of this change. The JDK project has unusually high compatibility concerns. I and Josh believe the introduction of a possible IAE if the comparator doesn't satisfy its contract is the right thing, but we'd also be willing to change the code to swallow the IAE or to do so conditionally based on the value of a system property. Keep in mind that a call to foreign code can throw any throwable at all, so a rogue comparator can cause arbitrary behavior even today. It's up to Sun/CCC. (There is the deeper governance issue of who gets to make such decisions. I would like most of such decisions to be made by the "group" of engineers who do the work. For collections/concurrency, such a group has worked informally for many years.) Martin On Tue, Jun 30, 2009 at 03:04, Alan Bateman wrote: > Martin Buchholz wrote: > >> Hi sort team! >> >> Google would like to contribute a new implementation for sorting of Object >> arrays, >> which has much better performance for input that is already partially >> sorted, >> based on Tim Peter's sort used in Python. >> >> This sort is already being used in the java.util. that comes with Android. >> >> Written by Josh Bloch. >> >> http://cr.openjdk.java.net/~martin/timsort/< >> http://cr.openjdk.java.net/%7Emartin/timsort/> >> >> Strictly speaking, no further review may be necessary, >> since it has already seen much review by Google engineers, >> (including some who are OpenJDK committers), >> and it has seen real-world usage. >> >> Nevertheless, interested parties are invited to further review it. >> >> The proposed webrev includes some very minor change to >> the javadoc for Arrays.sort, that we would like to include, >> but are also content leaving out, or to have a Sun engineer >> shepherd through CCC (perhaps Chris or Alan?). >> > The results look very good. > > I assume the only contentious issue is going to be the IAE when a broken > implementation of Comparable is detected. When you say "also content leaving > out", do you mean just the spec update, or do you mean that the exception > won't be thrown if detected? I wonder if this will need a compatibility > switch so as not to change the failure mode of existing (broken) > applications. Also, I assume failure atomicity is not possible here > (right?); just wondering if this needs to be stated in the javadoc for this > exception. > > I haven't studied the code but from a brief glance, I think the "Classpath" > exception might be missing from the new files in > src/share/classes/java/util. Also, should the mergeSort implementation be > removed or moved out of Arrays? > > -Alan. > > -------------- next part -------------- An HTML attachment was scrubbed... URL: From gnu_andrew at member.fsf.org Tue Jun 30 16:24:43 2009 From: gnu_andrew at member.fsf.org (Andrew John Hughes) Date: Tue, 30 Jun 2009 17:24:43 +0100 Subject: timsort In-Reply-To: <1ccfd1c10906300912u2250d0c5l819535e1ea4ffec4@mail.gmail.com> References: <1ccfd1c10906291621v3a50fd2fid98c3aa14dddc8af@mail.gmail.com> <4A49E348.5090902@sun.com> <1ccfd1c10906300912u2250d0c5l819535e1ea4ffec4@mail.gmail.com> Message-ID: <17c6771e0906300924m435356aah37b63f98baf5b2ea@mail.gmail.com> 2009/6/30 Martin Buchholz : > Thanks, Alan.? You're a good reviewer. > > As you suggested, I added the Classpath exception wording > to TimSort.java and ComparableTimSort.java. > > I excised the old mergesort code from Arrays.java. > > webrev regenerated. > > Adding all-or-nothing wording would be good to add, > perhaps to the class-level javadoc.? But it doesn't > have to be part of this change. > > The JDK project has unusually high compatibility concerns. > I and Josh believe the introduction of a possible IAE if the > comparator doesn't satisfy its contract is the right thing, > but we'd also be willing to change the code to swallow the IAE > or to do so conditionally based on the value of a system property. > Keep in mind that a call to foreign code can throw any throwable at all, > so a rogue comparator can cause arbitrary behavior even today. > It's up to Sun/CCC. > Given that this is a new release (JDK7), I'd strongly agree that introducing such a change is both timely and appropriate. Users should expect some minor changes/incompatibilities with a completely new major release, especially when we are talking about something that has been enforced verbally (in the Javadocs) beforehand. The only change here is that this is enforced at a technical level and may actually catch some errors ahead of time that would otherwise result in strange behaviour. I'm sure all of us want to limit the number of hacks to preserve legacy code wherever possible. > (There is the deeper governance issue of who gets to make > such decisions.? I would like most of such decisions to be made > by the "group" of engineers who do the work.? For collections/concurrency, > such a group has worked informally for many years.) > OpenJDK is (or at least should now be) a community-driven open source project. And so, the community as a whole should be making such decisions, not just those who happen to be employed by Sun. > Martin > > On Tue, Jun 30, 2009 at 03:04, Alan Bateman wrote: >> >> Martin Buchholz wrote: >>> >>> Hi sort team! >>> >>> Google would like to contribute a new implementation for sorting of >>> Object arrays, >>> which has much better performance for input that is already partially >>> sorted, >>> based on Tim Peter's sort used in Python. >>> >>> This sort is already being used in the java.util. that comes with >>> Android. >>> >>> Written by Josh Bloch. >>> >>> http://cr.openjdk.java.net/~martin/timsort/ >>> >>> >>> Strictly speaking, no further review may be necessary, >>> since it has already seen much review by Google engineers, >>> (including some who are OpenJDK committers), >>> and it has seen real-world usage. >>> >>> Nevertheless, interested parties are invited to further review it. >>> >>> The proposed webrev includes some very minor change to >>> the javadoc for Arrays.sort, that we would like to include, >>> but are also content leaving out, or to have a Sun engineer >>> shepherd through CCC (perhaps Chris or Alan?). >> >> The results look very good. >> >> I assume the only contentious issue is going to be the IAE when a broken >> implementation of Comparable is detected. When you say "also content leaving >> out", do you mean just the spec update, or do you mean that the exception >> won't be thrown if detected? I wonder if this will need a compatibility >> switch so as not to change the failure mode of existing (broken) >> applications. ?Also, I assume failure atomicity is not possible here >> (right?); just wondering if this needs to be stated in the javadoc for this >> exception. >> >> I haven't studied the code but from a brief glance, I think the >> "Classpath" exception might be missing from the new files in >> src/share/classes/java/util. Also, should the mergeSort implementation be >> removed or moved out of Arrays? >> >> -Alan. >> > > -- Andrew :-) Free Java Software Engineer Red Hat, Inc. (http://www.redhat.com) Support Free Java! Contribute to GNU Classpath and the OpenJDK http://www.gnu.org/software/classpath http://openjdk.java.net PGP Key: 94EFD9D8 (http://subkeys.pgp.net) Fingerprint: F8EF F1EA 401E 2E60 15FA 7927 142C 2591 94EF D9D8 From aph at redhat.com Tue Jun 30 16:25:47 2009 From: aph at redhat.com (Andrew Haley) Date: Tue, 30 Jun 2009 17:25:47 +0100 Subject: timsort In-Reply-To: <1ccfd1c10906300912u2250d0c5l819535e1ea4ffec4@mail.gmail.com> References: <1ccfd1c10906291621v3a50fd2fid98c3aa14dddc8af@mail.gmail.com> <4A49E348.5090902@sun.com> <1ccfd1c10906300912u2250d0c5l819535e1ea4ffec4@mail.gmail.com> Message-ID: <4A4A3C8B.1050409@redhat.com> Martin Buchholz wrote: > Thanks, Alan. You're a good reviewer. > > As you suggested, I added the Classpath exception wording > to TimSort.java and ComparableTimSort.java. > > I excised the old mergesort code from Arrays.java. > > webrev regenerated. > > Adding all-or-nothing wording would be good to add, > perhaps to the class-level javadoc. But it doesn't > have to be part of this change. > > The JDK project has unusually high compatibility concerns. > I and Josh believe the introduction of a possible IAE if the > comparator doesn't satisfy its contract is the right thing, > but we'd also be willing to change the code to swallow the IAE > or to do so conditionally based on the value of a system property. > Keep in mind that a call to foreign code can throw any throwable at all, > so a rogue comparator can cause arbitrary behavior even today. > It's up to Sun/CCC. > > (There is the deeper governance issue of who gets to make > such decisions. I would like most of such decisions to be made > by the "group" of engineers who do the work. For collections/concurrency, > such a group has worked informally for many years.) > > Martin > > On Tue, Jun 30, 2009 at 03:04, Alan Bateman > wrote: > > Martin Buchholz wrote: > > Hi sort team! > > Google would like to contribute a new implementation for sorting > of Object arrays, > which has much better performance for input that is already > partially sorted, > based on Tim Peter's sort used in Python. > > This sort is already being used in the java.util. that comes > with Android. > > Written by Josh Bloch. > > http://cr.openjdk.java.net/~martin/timsort/ > > > > > Strictly speaking, no further review may be necessary, > since it has already seen much review by Google engineers, > (including some who are OpenJDK committers), > and it has seen real-world usage. > > Nevertheless, interested parties are invited to further review it. > > The proposed webrev includes some very minor change to > the javadoc for Arrays.sort, that we would like to include, > but are also content leaving out, or to have a Sun engineer > shepherd through CCC (perhaps Chris or Alan?). > > The results look very good. > > I assume the only contentious issue is going to be the IAE when a > broken implementation of Comparable is detected. We had a similar problem with GNU Classpath. Our binary search compared the elements in a different order from that in Sun's library, and this caused many problems. My attitude was "well, you program is broken", but in the end we had to fix Classpath anyway. Andrew. Index: Arrays.java =================================================================== --- Arrays.java (revision 121090) +++ Arrays.java (revision 121091) @@ -1,5 +1,5 @@ /* Arrays.java -- Utility class with methods to operate on arrays - Copyright (C) 1998, 1999, 2000, 2001, 2002, 2003, 2004, 2005, 2006, + Copyright (C) 1998, 1999, 2000, 2001, 2002, 2003, 2004, 2005, 2006, 2007, Free Software Foundation, Inc. This file is part of GNU Classpath. @@ -370,10 +370,13 @@ while (low <= hi) { mid = (low + hi) >>> 1; - final int d = Collections.compare(key, a[mid], c); + // NOTE: Please keep the order of a[mid] and key. Although + // not required by the specs, the RI has it in this order as + // well, and real programs (erroneously) depend on it. + final int d = Collections.compare(a[mid], key, c); if (d == 0) return mid; - else if (d < 0) + else if (d > 0) hi = mid - 1; else // This gets the insertion point right on the last loop Andrew. From gnu_andrew at member.fsf.org Tue Jun 30 16:29:23 2009 From: gnu_andrew at member.fsf.org (Andrew John Hughes) Date: Tue, 30 Jun 2009 17:29:23 +0100 Subject: timsort In-Reply-To: <4A4A3C8B.1050409@redhat.com> References: <1ccfd1c10906291621v3a50fd2fid98c3aa14dddc8af@mail.gmail.com> <4A49E348.5090902@sun.com> <1ccfd1c10906300912u2250d0c5l819535e1ea4ffec4@mail.gmail.com> <4A4A3C8B.1050409@redhat.com> Message-ID: <17c6771e0906300929r41c6b657ubd19fde4a26ebe7@mail.gmail.com> 2009/6/30 Andrew Haley : > Martin Buchholz wrote: >> Thanks, Alan. ?You're a good reviewer. >> >> As you suggested, I added the Classpath exception wording >> to TimSort.java and ComparableTimSort.java. >> >> I excised the old mergesort code from Arrays.java. >> >> webrev regenerated. >> >> Adding all-or-nothing wording would be good to add, >> perhaps to the class-level javadoc. ?But it doesn't >> have to be part of this change. >> >> The JDK project has unusually high compatibility concerns. >> I and Josh believe the introduction of a possible IAE if the >> comparator doesn't satisfy its contract is the right thing, >> but we'd also be willing to change the code to swallow the IAE >> or to do so conditionally based on the value of a system property. >> Keep in mind that a call to foreign code can throw any throwable at all, >> so a rogue comparator can cause arbitrary behavior even today. >> It's up to Sun/CCC. >> >> (There is the deeper governance issue of who gets to make >> such decisions. ?I would like most of such decisions to be made >> by the "group" of engineers who do the work. ?For collections/concurrency, >> such a group has worked informally for many years.) >> >> Martin >> >> On Tue, Jun 30, 2009 at 03:04, Alan Bateman > > wrote: >> >> ? ? Martin Buchholz wrote: >> >> ? ? ? ? Hi sort team! >> >> ? ? ? ? Google would like to contribute a new implementation for sorting >> ? ? ? ? of Object arrays, >> ? ? ? ? which has much better performance for input that is already >> ? ? ? ? partially sorted, >> ? ? ? ? based on Tim Peter's sort used in Python. >> >> ? ? ? ? This sort is already being used in the java.util. that comes >> ? ? ? ? with Android. >> >> ? ? ? ? Written by Josh Bloch. >> >> ? ? ? ? http://cr.openjdk.java.net/~martin/timsort/ >> ? ? ? ? >> ? ? ? ? >> >> >> ? ? ? ? Strictly speaking, no further review may be necessary, >> ? ? ? ? since it has already seen much review by Google engineers, >> ? ? ? ? (including some who are OpenJDK committers), >> ? ? ? ? and it has seen real-world usage. >> >> ? ? ? ? Nevertheless, interested parties are invited to further review it. >> >> ? ? ? ? The proposed webrev includes some very minor change to >> ? ? ? ? the javadoc for Arrays.sort, that we would like to include, >> ? ? ? ? but are also content leaving out, or to have a Sun engineer >> ? ? ? ? shepherd through CCC (perhaps Chris or Alan?). >> >> ? ? The results look very good. >> >> ? ? I assume the only contentious issue is going to be the IAE when a >> ? ? broken implementation of Comparable is detected. > > We had a similar problem with GNU Classpath. ?Our binary search compared > the elements in a different order from that in Sun's library, and this > caused many problems. ?My attitude was "well, you program is broken", > but in the end we had to fix Classpath anyway. > Sigh... I can remember all too many such cases. At least with OpenJDK we have more leverage now to set the playing field, rather than always being the ones trying to catch up. > Andrew. > > > Index: Arrays.java > =================================================================== > --- Arrays.java (revision 121090) > +++ Arrays.java (revision 121091) > @@ -1,5 +1,5 @@ > ?/* Arrays.java -- Utility class with methods to operate on arrays > - ? Copyright (C) 1998, 1999, 2000, 2001, 2002, 2003, 2004, 2005, 2006, > + ? Copyright (C) 1998, 1999, 2000, 2001, 2002, 2003, 2004, 2005, 2006, 2007, > ? ?Free Software Foundation, Inc. > > ?This file is part of GNU Classpath. > @@ -370,10 +370,13 @@ > ? ? while (low <= hi) > ? ? ? { > ? ? ? ? mid = (low + hi) >>> 1; > - ? ? ? ?final int d = Collections.compare(key, a[mid], c); > + ? ? ? // NOTE: Please keep the order of a[mid] and key. ?Although > + ? ? ? // not required by the specs, the RI has it in this order as > + ? ? ? // well, and real programs (erroneously) depend on it. > + ? ? ? ?final int d = Collections.compare(a[mid], key, c); > ? ? ? ? if (d == 0) > ? ? ? ? ? return mid; > - ? ? ? ?else if (d < 0) > + ? ? ? ?else if (d > 0) > ? ? ? ? ? hi = mid - 1; > ? ? ? ? else > ? ? ? ? ? // This gets the insertion point right on the last loop > > Andrew. > -- Andrew :-) Free Java Software Engineer Red Hat, Inc. (http://www.redhat.com) Support Free Java! Contribute to GNU Classpath and the OpenJDK http://www.gnu.org/software/classpath http://openjdk.java.net PGP Key: 94EFD9D8 (http://subkeys.pgp.net) Fingerprint: F8EF F1EA 401E 2E60 15FA 7927 142C 2591 94EF D9D8 From jason_mehrens at hotmail.com Tue Jun 30 17:03:20 2009 From: jason_mehrens at hotmail.com (Jason Mehrens) Date: Tue, 30 Jun 2009 12:03:20 -0500 Subject: timsort In-Reply-To: <1ccfd1c10906300912u2250d0c5l819535e1ea4ffec4@mail.gmail.com> References: <1ccfd1c10906291621v3a50fd2fid98c3aa14dddc8af@mail.gmail.com> <4A49E348.5090902@sun.com> <1ccfd1c10906300912u2250d0c5l819535e1ea4ffec4@mail.gmail.com> Message-ID: Martin, Regarding this IAE issue, another approach might be given non null Comparator 'c': if(c.getClass().desiredAssertionStatus()) { throw new AssertionError("....."); } Then there is no compatibility problem,no need to add a new switch to the JDK, and the coder gets smacked with an error. Jason Mehrens >The JDK project has unusually high compatibility concerns. >I and Josh believe the introduction of a possible IAE if the >comparator doesn't satisfy its contract is the right thing, >but we'd also be willing to change the code to swallow the IAE >or to do so conditionally based on the value of a system property. >Keep in mind that a call to foreign code can throw any throwable at all, >so a rogue comparator can cause arbitrary behavior even today. I>t's up to Sun/CCC. _________________________________________________________________ Lauren found her dream laptop. Find the PC that?s right for you. http://www.microsoft.com/windows/choosepc/?ocid=ftp_val_wl_290 -------------- next part -------------- An HTML attachment was scrubbed... URL: From martinrb at google.com Tue Jun 30 18:01:56 2009 From: martinrb at google.com (Martin Buchholz) Date: Tue, 30 Jun 2009 11:01:56 -0700 Subject: timsort In-Reply-To: <17c6771e0906300924m435356aah37b63f98baf5b2ea@mail.gmail.com> References: <1ccfd1c10906291621v3a50fd2fid98c3aa14dddc8af@mail.gmail.com> <4A49E348.5090902@sun.com> <1ccfd1c10906300912u2250d0c5l819535e1ea4ffec4@mail.gmail.com> <17c6771e0906300924m435356aah37b63f98baf5b2ea@mail.gmail.com> Message-ID: <1ccfd1c10906301101w43a20d11me6350d7b0706aaca@mail.gmail.com> On Tue, Jun 30, 2009 at 09:24, Andrew John Hughes wrote: > 2009/6/30 Martin Buchholz : > > > (There is the deeper governance issue of who gets to make > > such decisions. I would like most of such decisions to be made > > by the "group" of engineers who do the work. For > collections/concurrency, > > such a group has worked informally for many years.) > > > > OpenJDK is (or at least should now be) a community-driven open source > project. And so, the community as a whole should be making such > decisions, not just those who happen to be employed by Sun. > Right. There is a problem when different sets of contributors have different objectives for things like compatibility, portability, stability, benchmark performance.... It might be that a significant contributor (like Sun or IcedTea) would maintain a separate set of patches essentially forever, since they would not be acceptable to the greater community. Oh, I guess that's already happened, eh? There's also the issue of the size of the project. Is OpenJDK a "project", or is it an aggregator of projects maintained by various third parties, like a Linux distro? Given how the code base has grown, at least a large number of components should be treated in the latter way. It is already the case, I believe, that the JAX* code is essentially imported unchanged from upstream maintainers. Martin -------------- next part -------------- An HTML attachment was scrubbed... URL: From martinrb at google.com Tue Jun 30 18:08:37 2009 From: martinrb at google.com (Martin Buchholz) Date: Tue, 30 Jun 2009 11:08:37 -0700 Subject: timsort In-Reply-To: References: <1ccfd1c10906291621v3a50fd2fid98c3aa14dddc8af@mail.gmail.com> <4A49E348.5090902@sun.com> <1ccfd1c10906300912u2250d0c5l819535e1ea4ffec4@mail.gmail.com> Message-ID: <1ccfd1c10906301108r6b162bd8v97cefa6b96d3dffb@mail.gmail.com> On Tue, Jun 30, 2009 at 10:03, Jason Mehrens wrote: > Martin, > > Regarding this IAE issue, another approach might be given non > null Comparator 'c': > > if(c.getClass().desiredAssertionStatus()) { > throw new AssertionError("....."); > } > > Then there is no compatibility problem,no need to add a new switch to the > JDK, and the coder gets smacked with an error. > > ... if assertions are turned on (which happens far too rarely) But still an interesting idea - I had not seen it before. I wonder what Josh thinks. Martin -------------- next part -------------- An HTML attachment was scrubbed... URL: From Alan.Bateman at Sun.COM Tue Jun 30 18:28:49 2009 From: Alan.Bateman at Sun.COM (Alan Bateman) Date: Tue, 30 Jun 2009 19:28:49 +0100 Subject: timsort In-Reply-To: <1ccfd1c10906300912u2250d0c5l819535e1ea4ffec4@mail.gmail.com> References: <1ccfd1c10906291621v3a50fd2fid98c3aa14dddc8af@mail.gmail.com> <4A49E348.5090902@sun.com> <1ccfd1c10906300912u2250d0c5l819535e1ea4ffec4@mail.gmail.com> Message-ID: <4A4A5961.6070201@sun.com> Martin Buchholz wrote: > : > > As you suggested, I added the Classpath exception wording > to TimSort.java and ComparableTimSort.java. > > I excised the old mergesort code from Arrays.java. > > webrev regenerated. Thanks for doing this. > > Adding all-or-nothing wording would be good to add, > perhaps to the class-level javadoc. But it doesn't > have to be part of this change. I brought it up because it looks like (and you can correct me) that the IAE may be thrown after some reordering of the array has taken place. This might be unexpected. > > The JDK project has unusually high compatibility concerns. > I and Josh believe the introduction of a possible IAE if the > comparator doesn't satisfy its contract is the right thing, > but we'd also be willing to change the code to swallow the IAE > or to do so conditionally based on the value of a system property. > Keep in mind that a call to foreign code can throw any throwable at all, > so a rogue comparator can cause arbitrary behavior even today. I think most people would agree that that catching these otherwise-silent failures is a good thing. The problem is the customer that upgrades the JDK on a production system and gets burnt by some broken third party code that "used" to work. Having some way to restore existing behavior would make this an easier sell. The suggestion from Jason to change it to an assertion might be worth thinking about too but I suspect that few developers test with assertions enabled. -Alan. From aph at redhat.com Tue Jun 30 18:32:09 2009 From: aph at redhat.com (Andrew Haley) Date: Tue, 30 Jun 2009 19:32:09 +0100 Subject: timsort In-Reply-To: <1ccfd1c10906301101w43a20d11me6350d7b0706aaca@mail.gmail.com> References: <1ccfd1c10906291621v3a50fd2fid98c3aa14dddc8af@mail.gmail.com> <4A49E348.5090902@sun.com> <1ccfd1c10906300912u2250d0c5l819535e1ea4ffec4@mail.gmail.com> <17c6771e0906300924m435356aah37b63f98baf5b2ea@mail.gmail.com> <1ccfd1c10906301101w43a20d11me6350d7b0706aaca@mail.gmail.com> Message-ID: <4A4A5A29.6@redhat.com> Martin Buchholz wrote: > > > On Tue, Jun 30, 2009 at 09:24, Andrew John Hughes > > wrote: > > 2009/6/30 Martin Buchholz >: > > > (There is the deeper governance issue of who gets to make > > such decisions. I would like most of such decisions to be made > > by the "group" of engineers who do the work. For > collections/concurrency, > > such a group has worked informally for many years.) > > > > OpenJDK is (or at least should now be) a community-driven open source > project. And so, the community as a whole should be making such > decisions, not just those who happen to be employed by Sun. > Right. There is a problem when different sets of contributors have > different objectives for things like compatibility, portability, > stability, benchmark performance.... > It might be that a significant contributor (like Sun or IcedTea) > would maintain a separate set of patches essentially forever, since > they would not be acceptable to the greater community. Oh, I guess > that's already happened, eh? Ouch, that touched a nerve. I certainly hope not. Goodness knows, we're really trying to make that separate set of patches go away. Andrew. From martinrb at google.com Tue Jun 30 19:10:58 2009 From: martinrb at google.com (Martin Buchholz) Date: Tue, 30 Jun 2009 12:10:58 -0700 Subject: timsort In-Reply-To: <4A4A5A29.6@redhat.com> References: <1ccfd1c10906291621v3a50fd2fid98c3aa14dddc8af@mail.gmail.com> <4A49E348.5090902@sun.com> <1ccfd1c10906300912u2250d0c5l819535e1ea4ffec4@mail.gmail.com> <17c6771e0906300924m435356aah37b63f98baf5b2ea@mail.gmail.com> <1ccfd1c10906301101w43a20d11me6350d7b0706aaca@mail.gmail.com> <4A4A5A29.6@redhat.com> Message-ID: <1ccfd1c10906301210of9b33bbn939d4889fc01688a@mail.gmail.com> On Tue, Jun 30, 2009 at 11:32, Andrew Haley wrote: > Martin Buchholz wrote: > > > Right. There is a problem when different sets of contributors have > > different objectives for things like compatibility, portability, > > stability, benchmark performance.... > > > It might be that a significant contributor (like Sun or IcedTea) > > would maintain a separate set of patches essentially forever, since > > they would not be acceptable to the greater community. Oh, I guess > > that's already happened, eh? > > Ouch, that touched a nerve. > > I certainly hope not. Goodness knows, we're really trying to make that > separate set of patches go away. > I wasn't trying to criticize IcedTea specifically. Both IcedTea and Google are working hard to send changes upstream, but I know that Google has some that will remain private, and I suspect that may be true for all 3 of our organizations going forward, despite our best efforts to achieve a common code base. Martin > > Andrew. > -------------- next part -------------- An HTML attachment was scrubbed... URL: From dl at cs.oswego.edu Tue Jun 30 22:09:04 2009 From: dl at cs.oswego.edu (Doug Lea) Date: Tue, 30 Jun 2009 18:09:04 -0400 Subject: Faster HashMap implementation In-Reply-To: References: <4A2D3729.3040904@cs.oswego.edu> <4A2E5109.4050804@cs.oswego.edu> <4A40FD16.6040503@cs.oswego.edu> Message-ID: <4A4A8D00.8000506@cs.oswego.edu> Inspired by the combination of Alex's efforts and ongoing work on concurrent caches, I put together a version of HashMap (called HashMapV2 for now) with a snapshot at http://gee.cs.oswego.edu/dl/wwwtmp/HashMapV2.java (This retains the openjdk GPL header etc.) There are a few remaining things to consider before taking this too seriously as a replacement. I'll be out for 10 days starting tomorrow so probably won't get a chance to do so soon. It does seem to be the best plan of attack for open-table scheme that people keep wanting (because of fewer cache misses etc). I think it hits all of the other issues and concerns I listed in mail a few weeks ago. It looks very good on various performance tests, including some I need to add/update in our "loops" tests. (I wish the concurrent cache version for j.u.c was in this good a state...) Pasted below is some of the internal documentation: * Overview: * * The main table uses open-addressing, with [key, value] pairs in * alternating elements of "kvTable" array, plus a side-array * "hashes" holding hashCodes of keys (plus status bit). If uses * hash preconditioning for first probe, and pseudo-random probing * from there. This gives a very good approximation of "uniform * hashing" (see for example CLR ch12) across various key types. * * Key removal sometimes requires replacing keys with ABSENT * markers. We minimize need for these by recording (via UNIQUE * bit, see below) whether we can instead safely null out entry. * Markers are cleared out on put when there are too many of them, * as triggered by decrementing "threshold" on adding marker. * * In steady state, total footprint (ignoring non-dynamic fields) * for open tables is less than chained, even with the side array * of hashes. At default load factors, the total numbers words is * about: * steady state initial * min ave max (default cap) * Open: 4.00N 6.00N 8.00N 24 * Chained: 7.33N 8.00N 8.67N 16 * * To conserve space for unused maps, arrays are left unallocated * upon construction and nulled upon clearing, but with the target * allocation size stored in negated form in threshold field, * which conveniently requires a (size > threshold) check on * insertion anyway. Additionally, to ensure that tables do not * fill up with deletion markers, the threshold is decremented * each time an element is replaced with a ABSENT marker. This * forces resize occurring on next insertion to then replace table * with one without markers. * * Maps of size >= CEILING_LENGTH hold overflowing entries in a * ternary bitwise trie (see nested class HashTrie). This provides * bounded handling without running out of array indices, at the * price of about 4X average operation costs and 2.5X per-item * space overhead. The trie is also used for Null keys, which * simplifies and speeds up most other table operations. Because * operations for Maps allowing nulls don't compose in simple ways * (null return values do not reliably mean absent), the HashTrie * class is not itself a Map, and so not useful outside this * class. * From martinrb at google.com Tue Jun 30 23:39:35 2009 From: martinrb at google.com (Martin Buchholz) Date: Tue, 30 Jun 2009 16:39:35 -0700 Subject: Review request for 5049299 In-Reply-To: <4A49DF55.5090807@sun.com> References: <4A1678DB.9010209@sun.com> <4A311337.9070706@sun.com> <1ccfd1c10906111416x1b0fveb52eb9d3db56ded@mail.gmail.com> <1ccfd1c10906221745u87082b0m80dd3b79b4646bdf@mail.gmail.com> <4A40E7C0.5080308@redhat.com> <1ccfd1c10906271035g660800a5n7de04efc6f1ce504@mail.gmail.com> <20090628023345.C970A420F6@magilla.sf.frob.com> <1ccfd1c10906291823p366255e5t678bcade6c5aa08@mail.gmail.com> <20090630022822.2F69F420F6@magilla.sf.frob.com> <4A49DF55.5090807@sun.com> Message-ID: <1ccfd1c10906301639j68a389a8w7cac8c235d72c5c0@mail.gmail.com> [+libc-alpha] I just filed glibc bug http://sources.redhat.com/bugzilla/show_bug.cgi?id=10353 Methods for deleting all file descriptors greater than given integer In the JDK... We close all non-relevant file descriptors in the child instead of relying on having the FD_CLOEXEC bit set properly for every fd in the parent. The technique of choice appears to be to read /proc/self/fd and close all the fds found therein. That's even portable between linux and solaris! Martin On Tue, Jun 30, 2009 at 02:48, Michael McMahon wrote: > Roland McGrath wrote: > >> But...posix_spawn doesn't give you any way to delete *all* file >>> descriptors >>> and if you try to collect them before spawning, there is a race in a >>> multithreaded program. >>> >>> >> >> This is something you should never want to do. There is notoriously no >> good way to do it on some systems, where getdtablesize() can return >> RLIM_INFINITY and is not proper to use in a loop. >> >> Use FD_CLOEXEC when you open the descriptor in the first place. >> >> >> > We also need to chdir() before calling exec. And while some descriptors > already have FD_CLOEXEC set, it's not easy to guarantee this for all of > them. > > - Michael. > -------------- next part -------------- An HTML attachment was scrubbed... URL: From martinrb at google.com Tue Jun 30 23:47:33 2009 From: martinrb at google.com (Martin Buchholz) Date: Tue, 30 Jun 2009 16:47:33 -0700 Subject: posix_spawn should use vfork() in more cases than presently Message-ID: <1ccfd1c10906301647t15f0eab3p1e0771fed6a6bd14@mail.gmail.com> I just filed glibc bug posix_spawn should use vfork() in more cases than presently http://sources.redhat.com/bugzilla/show_bug.cgi?id=10354 glibc posix_spawn uses vfork() in some cases, fork() in others. Currently it is rather conservative in this regard. For example, if there are any file actions, vfork() is avoided. This restriction can be lifted, I think, especially for the common case of closing file descriptors. Martin On Mon, Jun 29, 2009 at 19:28, Roland McGrath wrote: > > > (Aside: I also wonder why glibc's implementation of posix_spawn avoids > > using vfork if there are file actions specified) > > Hmm, I'm not sure about that. I also have no idea why you aren't asking > these questions on the libc-alpha mailing list. > -------------- next part -------------- An HTML attachment was scrubbed... URL: From martinrb at google.com Tue Jun 30 23:58:02 2009 From: martinrb at google.com (Martin Buchholz) Date: Tue, 30 Jun 2009 16:58:02 -0700 Subject: Review request for 5049299 In-Reply-To: <20090630022822.2F69F420F6@magilla.sf.frob.com> References: <4A1678DB.9010209@sun.com> <1ccfd1c10906091244u14607dcesc40fc6017adc7808@mail.gmail.com> <4A311337.9070706@sun.com> <1ccfd1c10906111416x1b0fveb52eb9d3db56ded@mail.gmail.com> <1ccfd1c10906221745u87082b0m80dd3b79b4646bdf@mail.gmail.com> <4A40E7C0.5080308@redhat.com> <1ccfd1c10906271035g660800a5n7de04efc6f1ce504@mail.gmail.com> <20090628023345.C970A420F6@magilla.sf.frob.com> <1ccfd1c10906291823p366255e5t678bcade6c5aa08@mail.gmail.com> <20090630022822.2F69F420F6@magilla.sf.frob.com> Message-ID: <1ccfd1c10906301658l6bc57155t20c3630a80d82263@mail.gmail.com> Roland, You write "It's all the same "clone" code in the kernel." and that may be true, but I'm thinking about the glibc wrappers around them. In particular, it seems to me that vfork and clone(CLONE_VM|CLONE_VFORK) should have similar assembly wrappers around them. But the vfork code uses SAVE_PID/RESTORE_PID which frobs only the pid, not the tid, while clone used RESET_PID which frobs both the pid and tid. It seems to me they should share common infrastructure. Martin On Mon, Jun 29, 2009 at 19:28, Roland McGrath wrote: > > > Linux clone has avoided vfork's bad press, and has occasionally been > > described as "elegant". For a while I believed that clone() was the only > > system call that created new processes, and that vfork() was just an > > inflexible special case of clone(), and on ia64 that appears to be true, > > but on x86 clone(), fork() and vfork() are separate system calls, > > and glibc has different gobs of assembly code around each. > > Internally it's true. On some machines there are separate vfork and/or > fork calls, some not. If you are worried about what the semantics mean, > it is immaterial which system call number you used and how many arguments > you passed. It's all the same "clone" code in the kernel. > > > Thanks, > Roland > -------------- next part -------------- An HTML attachment was scrubbed... URL: