From vitalyd at gmail.com Sat Feb 1 01:53:24 2014 From: vitalyd at gmail.com (Vitaly Davidovich) Date: Fri, 31 Jan 2014 20:53:24 -0500 Subject: ObjectIn/OutputStream improvements In-Reply-To: <52EBF484.9070909@oracle.com> References: <91644C57-6E0D-48ED-8E31-6AC217DECCEF@gmx.de> <1C96B28E-5E20-4D8D-B1FD-8FD5DF193EC8@oracle.com> <52EBB1B8.2030401@oracle.com> <52EBF484.9070909@oracle.com> Message-ID: One would have to measure of course, but intuitively, it seems like a good change to reuse the stringbuilder. There's also the issue that the local stringbuilder before was default-sized, and for large content, it would generate even more garbage as the underlying array is expanded. The "temporary short lived allocations are cheap" is, unfortunately, a semi-myth, or at least somewhat misguided. It's true that individually they may be cheap, but they have macro effects. The higher the allocation rate, the more young GCs we get. Every young GC (on hotspot collectors, at least) is a STW pause. Bringing threads to a safepoint has some cost, especially if there are many of them on large many-core machines. With larger heaps these days, young gens tend to be larger as well. When GC runs, it trashes the cpu caches for the mutators, so when they resume, they may get cache misses. At each young GC, some objects may get promoted prematurely to tenured. So no, I wouldn't say it's quite inexpensive :). When you have no option but to allocate, it's nice to have collectors that can handle those necessary allocations well. Otherwise, if it's a perf sensitive area and avoiding allocations doesn't obfuscate or make the code significantly less maintainable, it's usually better to avoid allocations. Just my $.02 Sent from my phone On Jan 31, 2014 2:06 PM, "Stuart Marks" wrote: > On 1/31/14 10:46 AM, Chris Hegarty wrote: > >> I think your patch can be split into two logical, independent, parts. The >> first is the use of unsafe to access the String UTF length. The seconds is >> to reuse, where possible, the existing StringBuilder instances, skipping >> of >> primitive/object reading/writing where applicable, and general cleanup. >> >> Since this is a very sensitive area I would like to consider these >> separately. To that end, I have taken the changes that are applicable to >> the >> latter, removed any subjective stylistic changes, and made some additional >> cleanup improvements. >> >> http://cr.openjdk.java.net/~chegar/serial_stupp.00/webrev/ >> > > I agree with splitting the Unsafe usages so they can be reviewed > separately. New > and changed usage of Unsafe will require exacting scrutiny. > > In general, the cleanups and refactorings look fine. > > I have a question about the change to reuse StringBuilder instances. This > replaces freshly allocated StringBuilders stored in local variables with > reuse of a StringBuilder stored in a field of BlockDataInputStream, which > in turn is stored in a field of ObjectInputStream. Thus, instead of > creating of lots of temporaries that become gc-eligible very quickly, this > creates a single long-lived object whose size is the high-water mark of the > longest string that's been read. The change does reduce allocation > pressure, but the point of generational GC is to make allocation and > collection of temporaries quite inexpensive. Is this the right tradeoff? > > s'marks > > From roger.riggs at oracle.com Sat Feb 1 18:03:41 2014 From: roger.riggs at oracle.com (roger riggs) Date: Sat, 01 Feb 2014 13:03:41 -0500 Subject: RFR: (8031737) CHECK_NULL and CHECK_EXCEPTION macros cleanup Message-ID: <52ED36FD.5070507@oracle.com> Hi, A few small changes to cleanup the name and use of CHECK macros. The CHECK_NULL_* macros are more generally useful without the JNU_ prefix. The JNU_CHECK_EXCEPTION* macros apply only when there is an jni environment. Webrev: http://cr.openjdk.java.net/~rriggs/webrev-check-cleanup-8031737/ Thanks, Roger [1] https://bugs.openjdk.java.net/browse/JDK-8031737 From joe.darcy at oracle.com Sat Feb 1 18:13:32 2014 From: joe.darcy at oracle.com (Joe Darcy) Date: Sat, 01 Feb 2014 10:13:32 -0800 Subject: JDK 9 RFR of JDK-8033416: Remove sun.misc.FpUtils Message-ID: <52ED394C.4010005@oracle.com> Hello, Back in JDK 5, the sun.misc.FpUtils class was added to provide low-level floating-point manipulations methods needed to write tests of the math library. Over time, those low-level functions have generally been added to the standard Java library and most of the methods in FpUtils have thus been deprecated. Therefore, I think the start of JDK 9 is a good time to actually remove FpUtils from the platform; please review my change which does this: http://cr.openjdk.java.net/~darcy/8033416.0/ The implementation of the ilogb and isUnordered methods was moved into the "Tests" helper class in the math library test directory. (At the recommendation of Alan in the bug report, I verified that the fx jar files do not reference the FpUtils class.) Thanks, -Joe From lance.andersen at oracle.com Sat Feb 1 18:58:54 2014 From: lance.andersen at oracle.com (Lance @ Oracle) Date: Sat, 1 Feb 2014 13:58:54 -0500 Subject: RFR: (8031737) CHECK_NULL and CHECK_EXCEPTION macros cleanup In-Reply-To: <52ED36FD.5070507@oracle.com> References: <52ED36FD.5070507@oracle.com> Message-ID: <1243D3E7-C384-4AA8-80CA-9AA4E053F056@oracle.com> Looks fine Which releases are you think of including this in if any besides 9? Lance Andersen| Principal Member of Technical Staff | +1.781.442.2037 Oracle Java Engineering 1 Network Drive Burlington, MA 01803 Lance.Andersen at oracle.com Sent from my iPad On Feb 1, 2014, at 1:03 PM, roger riggs wrote: > Hi, > > A few small changes to cleanup the name and use of CHECK macros. > The CHECK_NULL_* macros are more generally useful without the JNU_ prefix. > The JNU_CHECK_EXCEPTION* macros apply only when there is an jni environment. > > Webrev: > http://cr.openjdk.java.net/~rriggs/webrev-check-cleanup-8031737/ > > Thanks, Roger > > [1] https://bugs.openjdk.java.net/browse/JDK-8031737 From Alan.Bateman at oracle.com Sat Feb 1 21:11:22 2014 From: Alan.Bateman at oracle.com (Alan Bateman) Date: Sat, 01 Feb 2014 21:11:22 +0000 Subject: RFR: (8031737) CHECK_NULL and CHECK_EXCEPTION macros cleanup In-Reply-To: <52ED36FD.5070507@oracle.com> References: <52ED36FD.5070507@oracle.com> Message-ID: <52ED62FA.30707@oracle.com> On 01/02/2014 18:03, roger riggs wrote: > Hi, > > A few small changes to cleanup the name and use of CHECK macros. > The CHECK_NULL_* macros are more generally useful without the JNU_ > prefix. > The JNU_CHECK_EXCEPTION* macros apply only when there is an jni > environment. > > Webrev: > http://cr.openjdk.java.net/~rriggs/webrev-check-cleanup-8031737/ Keeping CHECK_NULL* for the macros that don't take an env makes sense. The changes look okay to me. -Alan From Alan.Bateman at oracle.com Sat Feb 1 21:23:42 2014 From: Alan.Bateman at oracle.com (Alan Bateman) Date: Sat, 01 Feb 2014 21:23:42 +0000 Subject: JDK 9 RFR of JDK-8033416: Remove sun.misc.FpUtils In-Reply-To: <52ED394C.4010005@oracle.com> References: <52ED394C.4010005@oracle.com> Message-ID: <52ED65DE.2020606@oracle.com> On 01/02/2014 18:13, Joe Darcy wrote: > Hello, > > Back in JDK 5, the sun.misc.FpUtils class was added to provide > low-level floating-point manipulations methods needed to write tests > of the math library. Over time, those low-level functions have > generally been added to the standard Java library and most of the > methods in FpUtils have thus been deprecated. > > Therefore, I think the start of JDK 9 is a good time to actually > remove FpUtils from the platform; please review my change which does > this: > > http://cr.openjdk.java.net/~darcy/8033416.0/ > > The implementation of the ilogb and isUnordered methods was moved into > the "Tests" helper class in the math library test directory. > > (At the recommendation of Alan in the bug report, I verified that the > fx jar files do not reference the FpUtils class.) I may have mixed it up with FloatConsts or DouleConsts but I'm pretty sure there was a dependency at one point. The main thing is that we've checked and all is okay. I skimmed over the webrev and it looks good to me (I didn't check ilogb closely but I assume it has just moved without any changes). One minor comment is that I'm not sure that it's worth add the bug number to IeeeRecommendedTests as the change doesn't introduce any tests or fix issues that would be interested in check back on in the future. -Alan. From paul.sandoz at oracle.com Mon Feb 3 09:16:40 2014 From: paul.sandoz at oracle.com (Paul Sandoz) Date: Mon, 3 Feb 2014 10:16:40 +0100 Subject: JDK 9 RFR of JDK-8033416: Remove sun.misc.FpUtils In-Reply-To: <52ED65DE.2020606@oracle.com> References: <52ED394C.4010005@oracle.com> <52ED65DE.2020606@oracle.com> Message-ID: <1D793FA4-6978-45A3-A1EB-927A3F3831BE@oracle.com> On Feb 1, 2014, at 10:23 PM, Alan Bateman wrote: > On 01/02/2014 18:13, Joe Darcy wrote: >> Hello, >> >> Back in JDK 5, the sun.misc.FpUtils class was added to provide low-level floating-point manipulations methods needed to write tests of the math library. Over time, those low-level functions have generally been added to the standard Java library and most of the methods in FpUtils have thus been deprecated. >> >> Therefore, I think the start of JDK 9 is a good time to actually remove FpUtils from the platform; please review my change which does this: >> >> http://cr.openjdk.java.net/~darcy/8033416.0/ >> >> The implementation of the ilogb and isUnordered methods was moved into the "Tests" helper class in the math library test directory. >> >> (At the recommendation of Alan in the bug report, I verified that the fx jar files do not reference the FpUtils class.) > I may have mixed it up with FloatConsts or DouleConsts but I'm pretty sure there was a dependency at one point. The main thing is that we've checked and all is okay. > grepcode shows no usages, which is a good sign :-) Paul. From chris.hegarty at oracle.com Mon Feb 3 10:39:15 2014 From: chris.hegarty at oracle.com (Chris Hegarty) Date: Mon, 03 Feb 2014 10:39:15 +0000 Subject: RFR: (8031737) CHECK_NULL and CHECK_EXCEPTION macros cleanup In-Reply-To: <1243D3E7-C384-4AA8-80CA-9AA4E053F056@oracle.com> References: <52ED36FD.5070507@oracle.com> <1243D3E7-C384-4AA8-80CA-9AA4E053F056@oracle.com> Message-ID: <52EF71D3.6030606@oracle.com> Looks good to me. Thanks Roger. -Chris. On 01/02/14 18:58, Lance @ Oracle wrote: > Looks fine > > Which releases are you think of including this in if any besides 9? > > > Lance Andersen| Principal Member of Technical Staff | +1.781.442.2037 > Oracle Java Engineering > 1 Network Drive > Burlington, MA 01803 > Lance.Andersen at oracle.com > Sent from my iPad > > On Feb 1, 2014, at 1:03 PM, roger riggs wrote: > >> Hi, >> >> A few small changes to cleanup the name and use of CHECK macros. >> The CHECK_NULL_* macros are more generally useful without the JNU_ prefix. >> The JNU_CHECK_EXCEPTION* macros apply only when there is an jni environment. >> >> Webrev: >> http://cr.openjdk.java.net/~rriggs/webrev-check-cleanup-8031737/ >> >> Thanks, Roger >> >> [1] https://bugs.openjdk.java.net/browse/JDK-8031737 From chris.hegarty at oracle.com Mon Feb 3 13:18:26 2014 From: chris.hegarty at oracle.com (Chris Hegarty) Date: Mon, 03 Feb 2014 13:18:26 +0000 Subject: RFR [9] 8031050: [macosx] Crash while awt starting Message-ID: <52EF9722.8090207@oracle.com> Hi, An old issue has resurfaced because of a change in AWT. AWT, now, on Mac OS X, attaches a system graphics thread to the running VM, using JNI_AttachCurrentThread. This change can result in a NPE, if a security manager is installed, and the security manager tries to print the name of the current thread. In this case, attaching a thread to the VM, the j.l.Thread instance/oop is created by the VM in allocate_threadObj(), and then its constructor is run. At this point almost everything has been created for the thread, jni env, etc, so it can query currentThread() to retrieve itself. (Note: this is unusual. Typically the thread running the j.l.Thread constructor is not itself.) Since the j.l.Thread oop was allocated directly by the VM, its fields, other than those directly set by allocate_threadObj(), are not initialized. The call out from Thread.init to the security manager is problematic as the partially constructed thread can be retrieved through Thread.currentThread. Some of our internal tests run into a problem when the security manager tries to print the name of the thread, i.e. NPE. Clearly there are other concerns of accessing a partially constructed thread, not just the thread name, but it seems reasonable to simply move the setting of the name to the beginning of the init() method. I propose the following change for JDK 9, with the intention of backport exactly to JDK 8. Long term this should be revisited in JDK 9. diff --git a/src/share/classes/java/lang/Thread.java b/src/share/classes/java/lang/Thread.java --- a/src/share/classes/java/lang/Thread.java +++ b/src/share/classes/java/lang/Thread.java @@ -366,6 +366,8 @@ throw new NullPointerException("name cannot be null"); } + this.name = name.toCharArray(); + Thread parent = currentThread(); SecurityManager security = System.getSecurityManager(); if (g == null) { @@ -402,7 +404,6 @@ this.group = g; this.daemon = parent.isDaemon(); this.priority = parent.getPriority(); - this.name = name.toCharArray(); if (security == null || isCCLOverridden(parent.getClass())) this.contextClassLoader = parent.getContextClassLoader(); else -Chris. From Alan.Bateman at oracle.com Mon Feb 3 13:54:38 2014 From: Alan.Bateman at oracle.com (Alan Bateman) Date: Mon, 03 Feb 2014 13:54:38 +0000 Subject: RFR [9] 8031050: [macosx] Crash while awt starting In-Reply-To: <52EF9722.8090207@oracle.com> References: <52EF9722.8090207@oracle.com> Message-ID: <52EF9F9E.7060705@oracle.com> On 03/02/2014 13:18, Chris Hegarty wrote: > Hi, > > An old issue has resurfaced because of a change in AWT. AWT, now, on > Mac OS X, attaches a system graphics thread to the running VM, using > JNI_AttachCurrentThread. This change can result in a NPE, if a > security manager is installed, and the security manager tries to print > the name of the current thread. > > In this case, attaching a thread to the VM, the j.l.Thread > instance/oop is created by the VM in allocate_threadObj(), and then > its constructor is run. At this point almost everything has been > created for the thread, jni env, etc, so it can query currentThread() > to retrieve itself. (Note: this is unusual. Typically the thread > running the j.l.Thread constructor is not itself.) Since the > j.l.Thread oop was allocated directly by the VM, its fields, other > than those directly set by allocate_threadObj(), are not initialized. > > The call out from Thread.init to the security manager is problematic > as the partially constructed thread can be retrieved through > Thread.currentThread. Some of our internal tests run into a problem > when the security manager tries to print the name of the thread, i.e. > NPE. > > Clearly there are other concerns of accessing a partially constructed > thread, not just the thread name, but it seems reasonable to simply > move the setting of the name to the beginning of the init() method. > > I propose the following change for JDK 9, with the intention of backport > exactly to JDK 8. Long term this should be revisited in JDK 9. As a short term fix then this looks okay to me. An alternative might be for getName to return the empty String if called before the name is set. I assume that part of re-visiting later will be to update API docs (maybe JNI spec too) to allow for initialization on the "current thread" when attaching as it can't be guaranteed to be fully initialized at this point. -Alan. PS: We should probably change the bug synopsis to make it clearer what this issue is about now. From brian.burkhalter at oracle.com Mon Feb 3 15:42:30 2014 From: brian.burkhalter at oracle.com (Brian Burkhalter) Date: Mon, 3 Feb 2014 07:42:30 -0800 Subject: JDK 9 RFR of JDK-8033416: Remove sun.misc.FpUtils In-Reply-To: <52ED394C.4010005@oracle.com> References: <52ED394C.4010005@oracle.com> Message-ID: <7A37B492-0D2F-4547-BEDF-0D4487F92FC5@oracle.com> +1 from a (lower case) reviewer. ;-) Brian On Feb 1, 2014, at 10:13 AM, Joe Darcy wrote: > Hello, > > Back in JDK 5, the sun.misc.FpUtils class was added to provide low-level floating-point manipulations methods needed to write tests of the math library. Over time, those low-level functions have generally been added to the standard Java library and most of the methods in FpUtils have thus been deprecated. > > Therefore, I think the start of JDK 9 is a good time to actually remove FpUtils from the platform; please review my change which does this: > > http://cr.openjdk.java.net/~darcy/8033416.0/ > > The implementation of the ilogb and isUnordered methods was moved into the "Tests" helper class in the math library test directory. > > (At the recommendation of Alan in the bug report, I verified that the fx jar files do not reference the FpUtils class.) > > Thanks, > > -Joe From mandy.chung at oracle.com Mon Feb 3 17:45:27 2014 From: mandy.chung at oracle.com (Mandy Chung) Date: Mon, 03 Feb 2014 09:45:27 -0800 Subject: RFR: (8031737) CHECK_NULL and CHECK_EXCEPTION macros cleanup In-Reply-To: <52ED36FD.5070507@oracle.com> References: <52ED36FD.5070507@oracle.com> Message-ID: <52EFD5B7.9050902@oracle.com> Looks good. Mandy On 2/1/2014 10:03 AM, roger riggs wrote: > Hi, > > A few small changes to cleanup the name and use of CHECK macros. > The CHECK_NULL_* macros are more generally useful without the JNU_ > prefix. > The JNU_CHECK_EXCEPTION* macros apply only when there is an jni > environment. > > Webrev: > http://cr.openjdk.java.net/~rriggs/webrev-check-cleanup-8031737/ > > Thanks, Roger > > [1] https://bugs.openjdk.java.net/browse/JDK-8031737 From joe.darcy at oracle.com Mon Feb 3 17:57:23 2014 From: joe.darcy at oracle.com (Joe Darcy) Date: Mon, 03 Feb 2014 09:57:23 -0800 Subject: JDK 9 RFR of JDK-8033416: Remove sun.misc.FpUtils In-Reply-To: <52ED65DE.2020606@oracle.com> References: <52ED394C.4010005@oracle.com> <52ED65DE.2020606@oracle.com> Message-ID: <52EFD883.2010605@oracle.com> Hi Alan, I was on the fence about whether or not to include the bug id in the test; after you comment, I've removed it in the version that was pushed. Thanks for the review, -Joe On 02/01/2014 01:23 PM, Alan Bateman wrote: > On 01/02/2014 18:13, Joe Darcy wrote: >> Hello, >> >> Back in JDK 5, the sun.misc.FpUtils class was added to provide >> low-level floating-point manipulations methods needed to write tests >> of the math library. Over time, those low-level functions have >> generally been added to the standard Java library and most of the >> methods in FpUtils have thus been deprecated. >> >> Therefore, I think the start of JDK 9 is a good time to actually >> remove FpUtils from the platform; please review my change which does >> this: >> >> http://cr.openjdk.java.net/~darcy/8033416.0/ >> >> The implementation of the ilogb and isUnordered methods was moved >> into the "Tests" helper class in the math library test directory. >> >> (At the recommendation of Alan in the bug report, I verified that the >> fx jar files do not reference the FpUtils class.) > I may have mixed it up with FloatConsts or DouleConsts but I'm pretty > sure there was a dependency at one point. The main thing is that we've > checked and all is okay. > > I skimmed over the webrev and it looks good to me (I didn't check > ilogb closely but I assume it has just moved without any changes). One > minor comment is that I'm not sure that it's worth add the bug number > to IeeeRecommendedTests as the change doesn't introduce any tests or > fix issues that would be interested in check back on in the future. > > -Alan. From kumar.x.srinivasan at oracle.com Mon Feb 3 18:08:41 2014 From: kumar.x.srinivasan at oracle.com (Kumar Srinivasan) Date: Mon, 03 Feb 2014 10:08:41 -0800 Subject: RFR: (8031737) CHECK_NULL and CHECK_EXCEPTION macros cleanup In-Reply-To: <52EFD5B7.9050902@oracle.com> References: <52ED36FD.5070507@oracle.com> <52EFD5B7.9050902@oracle.com> Message-ID: <52EFDB29.90408@oracle.com> +1 Kumar On 2/3/2014 9:45 AM, Mandy Chung wrote: > Looks good. > > Mandy > > On 2/1/2014 10:03 AM, roger riggs wrote: >> Hi, >> >> A few small changes to cleanup the name and use of CHECK macros. >> The CHECK_NULL_* macros are more generally useful without the JNU_ >> prefix. >> The JNU_CHECK_EXCEPTION* macros apply only when there is an jni >> environment. >> >> Webrev: >> http://cr.openjdk.java.net/~rriggs/webrev-check-cleanup-8031737/ >> >> Thanks, Roger >> >> [1] https://bugs.openjdk.java.net/browse/JDK-8031737 > From huizhe.wang at oracle.com Mon Feb 3 19:19:48 2014 From: huizhe.wang at oracle.com (huizhe wang) Date: Mon, 03 Feb 2014 11:19:48 -0800 Subject: JAXP JEP: Update Xerces implementation in the JDK Message-ID: <52EFEBD4.2010402@oracle.com> Hi, We'd like to propose a JEP to update the Xerces implementation in the JDK and bring it to up to date to the current Xerces release. Please review the draft. Thanks, Joe -------------- next part -------------- JEP: Title: Updating JAXP to Xerces 2.11.0 Author: Joe Wang Organization: Oracle Created: 2014/01/24 Type: Feature State: Draft Exposure: Open Component: core/libs Scope: SE JSR: TBD RFE: Discussion: core dash libs dash dev at openjdk dot java dot net Start: 2014/Q1 Effort: L Duration: L Template: 1.0 Internal-refs: Oracle:A360: Reviewed-by: Alan Bateman, Roger Riggs, Lance Andersen Endorsed-by: Funded-by: Release: 9 Target: 06/01/15 Summary ------- Upgrade the Xerces implementation in the JDK to Xerces 2.11.0, which synchronizes the JDK with the current code base of Apache Xerces. Goals ----- The JDK contains an older Xerces implementation, version 2.7.1. Although there were updates in JDK 7 to bring in some changes, we did not bring it completely up to date to any later release. The goal of this JEP is to complete the update and bring the JDK to Xerces' latest release, 2.11.0. Non-Goals --------- The Xerces implementation in the JDK contains many changes and improvement that are JDK specific on top of version 2.7.1. It is not a goal to completely synchronize the JDK codebase with that of Apache Xerces. Motivation ---------- Upgrading to the latest Apache release would greatly improve the quality of the JDK implementation as demonstrated by JDK7 during which the most important bug fixes in Xerces and all of changes in Xalan were brought over. The JDK currently has all of the critical fixes up to Xerces 2.10. However,since the update was partial, we were unable to update the implementation version. Completing the update would allow us to do so, eliminating any confusion while keeping JDK up to date. Description ----------- Upgrade JAXP to the following standards that Xerces 2.11.0 support: - W3C XML Schema 1.1 (Working Drafts, December 2009) - W3C XML Schema Definition Language (XSD): Component Designators (Candidate Recommendation, January 2010) - Element Traversal Specification (part of DOM Standards, recommendation in 2008) - OASIS XML Catalogs 1.1 Update JAXP with following changes from Xerces 2.11.0: - Implementation of the above standards; - All of the bug fixes since Xerces 2.7.1. Between Xerces 2.7.1 and 2.10.0, All of the blockers and critical fixes along with some major and minor fixes have been integrated into JDK 7. What is left are 97 major, 90 minor, and 14 trivial bug fixes; - Numerous revisions that were not recorded in the Xerces issue tracker. Testing ------- Relevant tests from Xerces will be brought over. New tests related to the standards described above will need to be added to JCK 9. New tests will need to be developed to ensure sufficient coverage of the changes so that there is good confidence in the compatibility. Significant testing will be required to ensure that the update described above will introduce no regressions in the existing implementation. Risks and Assumptions --------------------- The addition of the standard support, as listed above, was achieved by adding new properties. There were no significant changes to the API. Since the integration of Xerces 2.7.1 in JDK6, there have been many changes, improvement and feature additions made independent of the original Xerces Implementation, including the incorporation of the StAX implementation that shared and modified the scanner implementation from Xerces. Bringing over changes from Xerces therefore may introduce conflicts in the existing implementation. Some Xerces patches therefore, may need to be modified to resolve any conflicts that may arise. The assumption is that we will find relevant tests from the Xerces repository. However, there may not be sufficient tests for some changes, and there were changes that went directly into the repository without a bug id associated with them. There may be significant effort needed to add new tests. The update will be done in batches that will contain a number of bug fixes and revisions. Not every revision may have a corresponding test. Verification will in such cases be done by running all existing tests. Impact ------ - Other JDK components: Components such as JAX-WS may be affected. Intermediate JDK builds should be tested to assess any impact on these components. - Java EE components: Java EE components such as JAX-WS may be affected. Intermediate JDK builds should be made available for EE to run compatibility tests. - Compatibility: New standard support, such as W3C XML Schema 1.1, will added with the update. However, the implementation will remain compatible with older standard such as W3C XML Schema 1.0. Full verification will be done periodically to ensure compatibility. - I18n/L10n: Some new error messages will need to be translated. - JCK: New tests will need to be added. From roger.riggs at oracle.com Mon Feb 3 20:17:55 2014 From: roger.riggs at oracle.com (roger riggs) Date: Mon, 03 Feb 2014 15:17:55 -0500 Subject: RFR: (8031737) CHECK_NULL and CHECK_EXCEPTION macros cleanup (re-review) Message-ID: <52EFF973.6070104@oracle.com> Please re-review. I missed a warning that the CHECK_NULL macros was being redefined. Retaining the previous changes to java/util/jar/pack that removed the redefinition addresses the issue. The webrev now only has a few small changes to cleanup the name and use of CHECK macros. The CHECK_NULL_* macros are more generally useful without the JNU_ prefix. The JNU_CHECK_EXCEPTION* macros apply only when there is an jni environment. Webrev: http://cr.openjdk.java.net/~rriggs/webrev-check-cleanup-8031737/ Thanks, Roger [1] https://bugs.openjdk.java.net/browse/JDK-8031737 From chris.hegarty at oracle.com Mon Feb 3 20:21:59 2014 From: chris.hegarty at oracle.com (Chris Hegarty) Date: Mon, 3 Feb 2014 20:21:59 +0000 Subject: RFR: (8031737) CHECK_NULL and CHECK_EXCEPTION macros cleanup (re-review) In-Reply-To: <52EFF973.6070104@oracle.com> References: <52EFF973.6070104@oracle.com> Message-ID: <759B727A-ED9F-40CC-BD5C-693BBAF41876@oracle.com> Looks good Roger. -Chris. On 3 Feb 2014, at 20:17, roger riggs wrote: > Please re-review. > > I missed a warning that the CHECK_NULL macros was being redefined. > > Retaining the previous changes to java/util/jar/pack that removed the redefinition > addresses the issue. > > The webrev now only has a few small changes to cleanup the name and use of CHECK macros. > The CHECK_NULL_* macros are more generally useful without the JNU_ prefix. > The JNU_CHECK_EXCEPTION* macros apply only when there is an jni environment. > > Webrev: > http://cr.openjdk.java.net/~rriggs/webrev-check-cleanup-8031737/ > > Thanks, Roger > > [1] https://bugs.openjdk.java.net/browse/JDK-8031737 > From mandy.chung at oracle.com Mon Feb 3 20:59:55 2014 From: mandy.chung at oracle.com (Mandy Chung) Date: Mon, 03 Feb 2014 12:59:55 -0800 Subject: RFR: (8031737) CHECK_NULL and CHECK_EXCEPTION macros cleanup (re-review) In-Reply-To: <52EFF973.6070104@oracle.com> References: <52EFF973.6070104@oracle.com> Message-ID: <52F0034B.8020105@oracle.com> On 2/3/2014 12:17 PM, roger riggs wrote: > Please re-review. > > I missed a warning that the CHECK_NULL macros was being redefined. > > Retaining the previous changes to java/util/jar/pack that removed the > redefinition > addresses the issue. > > The webrev now only has a few small changes to cleanup the name and > use of CHECK macros. > The CHECK_NULL_* macros are more generally useful without the JNU_ > prefix. > The JNU_CHECK_EXCEPTION* macros apply only when there is an jni > environment. > > Webrev: > http://cr.openjdk.java.net/~rriggs/webrev-check-cleanup-8031737/ > looks good. Mandy From martijnverburg at gmail.com Mon Feb 3 21:13:25 2014 From: martijnverburg at gmail.com (Martijn Verburg) Date: Mon, 3 Feb 2014 22:13:25 +0100 Subject: JAXP JEP: Update Xerces implementation in the JDK In-Reply-To: <52EFEBD4.2010402@oracle.com> References: <52EFEBD4.2010402@oracle.com> Message-ID: Hi Huizhe, Is there a possibility to look at having a more loosely coupled relationship between Xerces and what is core JDK? I'm thinking about (in combination with) Jigsaw that you could allow the Xerces components to be kept up to date more often (assuming API compatibility etc is retained). Cheers, Martijn On 3 February 2014 20:19, huizhe wang wrote: > Hi, > > We'd like to propose a JEP to update the Xerces implementation in the JDK > and bring it to up to date to the current Xerces release. Please review the > draft. > > Thanks, > Joe > > From neugens.limasoftware at gmail.com Mon Feb 3 21:17:25 2014 From: neugens.limasoftware at gmail.com (Mario Torre) Date: Mon, 3 Feb 2014 22:17:25 +0100 Subject: JAXP JEP: Update Xerces implementation in the JDK In-Reply-To: References: <52EFEBD4.2010402@oracle.com> Message-ID: Yes, And it would be even nicer if we could get some of the patches integrated upstream so that it could be eventually possible to not maintaining this code, but instead use upstream bundles directly. Not sure how this could be done, but would be awesome. Cheers, Mario Il 03/feb/2014 22:14 "Martijn Verburg" ha scritto: > Hi Huizhe, > > Is there a possibility to look at having a more loosely coupled > relationship between Xerces and what is core JDK? I'm thinking about (in > combination with) Jigsaw that you could allow the Xerces components to be > kept up to date more often (assuming API compatibility etc is retained). > > Cheers, > Martijn > > > On 3 February 2014 20:19, huizhe wang wrote: > > > Hi, > > > > We'd like to propose a JEP to update the Xerces implementation in the JDK > > and bring it to up to date to the current Xerces release. Please review > the > > draft. > > > > Thanks, > > Joe > > > > > From kumar.x.srinivasan at oracle.com Mon Feb 3 21:20:02 2014 From: kumar.x.srinivasan at oracle.com (Kumar Srinivasan) Date: Mon, 03 Feb 2014 13:20:02 -0800 Subject: RFR: (8031737) CHECK_NULL and CHECK_EXCEPTION macros cleanup (re-review) In-Reply-To: <52EFF973.6070104@oracle.com> References: <52EFF973.6070104@oracle.com> Message-ID: <52F00802.9050106@oracle.com> +1 Kumar On 2/3/2014 12:17 PM, roger riggs wrote: > Please re-review. > > I missed a warning that the CHECK_NULL macros was being redefined. > > Retaining the previous changes to java/util/jar/pack that removed the > redefinition > addresses the issue. > > The webrev now only has a few small changes to cleanup the name and > use of CHECK macros. > The CHECK_NULL_* macros are more generally useful without the JNU_ > prefix. > The JNU_CHECK_EXCEPTION* macros apply only when there is an jni > environment. > > Webrev: > http://cr.openjdk.java.net/~rriggs/webrev-check-cleanup-8031737/ > > Thanks, Roger > > [1] https://bugs.openjdk.java.net/browse/JDK-8031737 > From roger.riggs at oracle.com Mon Feb 3 21:48:28 2014 From: roger.riggs at oracle.com (roger riggs) Date: Mon, 03 Feb 2014 16:48:28 -0500 Subject: RFR: (8031737) CHECK_NULL and CHECK_EXCEPTION macros cleanup In-Reply-To: <1243D3E7-C384-4AA8-80CA-9AA4E053F056@oracle.com> References: <52ED36FD.5070507@oracle.com> <1243D3E7-C384-4AA8-80CA-9AA4E053F056@oracle.com> Message-ID: <52F00EAC.8000900@oracle.com> Hi Lance, The convenience macros are only intended for JDK 9. Roger On 2/1/2014 1:58 PM, Lance @ Oracle wrote: > Looks fine > > Which releases are you think of including this in if any besides 9? > > > Lance > Andersen| Principal Member of Technical Staff | +1.781.442.2037 > > Oracle Java Engineering > 1 Network Drive > Burlington, MA 01803 > Lance.Andersen at oracle.com > Sent from my iPad > > On Feb 1, 2014, at 1:03 PM, roger riggs > wrote: From Alan.Bateman at oracle.com Mon Feb 3 21:49:24 2014 From: Alan.Bateman at oracle.com (Alan Bateman) Date: Mon, 03 Feb 2014 21:49:24 +0000 Subject: JAXP JEP: Update Xerces implementation in the JDK In-Reply-To: References: <52EFEBD4.2010402@oracle.com> Message-ID: <52F00EE4.4070708@oracle.com> On 03/02/2014 21:13, Martijn Verburg wrote: > Hi Huizhe, > > Is there a possibility to look at having a more loosely coupled > relationship between Xerces and what is core JDK? I'm thinking about (in > combination with) Jigsaw that you could allow the Xerces components to be > kept up to date more often (assuming API compatibility etc is retained). > Just to mention that are already service provider interfaces so you can deploy with other parser libraries (or a more up to date Xerces). One of things that we did as part of preparing for modules (JEP 162, JDK 8) was to rev the JAXP API to clean up these the service provider interfaces (for SAX, DOM, XSLT, XPath, Validation, Streaming, Datatype) and specify the use of the ServiceLoader. So we're in a much better place now. As regards getting the Xerces code in OpenJDK in sync with the upstream project then that clearly would be desirable and probably more of a longer term goal. I'll let Joe speak to how much the code has diverged but one area of difference is that the original Apache code wasn't really intended to ever be on the boot class path or run with a security manager. In any case, I think this JEP is a good step as it brings the implementations closer and also revs the support on a number of standards. -Alan From Alan.Bateman at oracle.com Mon Feb 3 21:50:24 2014 From: Alan.Bateman at oracle.com (Alan Bateman) Date: Mon, 03 Feb 2014 21:50:24 +0000 Subject: RFR: (8031737) CHECK_NULL and CHECK_EXCEPTION macros cleanup (re-review) In-Reply-To: <52EFF973.6070104@oracle.com> References: <52EFF973.6070104@oracle.com> Message-ID: <52F00F20.1030103@oracle.com> On 03/02/2014 20:17, roger riggs wrote: > Please re-review. > > I missed a warning that the CHECK_NULL macros was being redefined. > > Retaining the previous changes to java/util/jar/pack that removed the > redefinition > addresses the issue. > > The webrev now only has a few small changes to cleanup the name and > use of CHECK macros. > The CHECK_NULL_* macros are more generally useful without the JNU_ > prefix. > The JNU_CHECK_EXCEPTION* macros apply only when there is an jni > environment. > > Webrev: > http://cr.openjdk.java.net/~rriggs/webrev-check-cleanup-8031737/ Thumbs up from me too. -Alan From neugens.limasoftware at gmail.com Mon Feb 3 21:54:49 2014 From: neugens.limasoftware at gmail.com (Mario Torre) Date: Mon, 3 Feb 2014 22:54:49 +0100 Subject: JAXP JEP: Update Xerces implementation in the JDK In-Reply-To: <52F00EE4.4070708@oracle.com> References: <52EFEBD4.2010402@oracle.com> <52F00EE4.4070708@oracle.com> Message-ID: Il 03/feb/2014 22:50 "Alan Bateman" ha scritto: > In any case, I think this JEP is a good step as it brings the implementations closer and also revs the support on a number of standards. Indeed! Mario From martijnverburg at gmail.com Mon Feb 3 21:55:59 2014 From: martijnverburg at gmail.com (Martijn Verburg) Date: Mon, 3 Feb 2014 22:55:59 +0100 Subject: JAXP JEP: Update Xerces implementation in the JDK In-Reply-To: <52F00EE4.4070708@oracle.com> References: <52EFEBD4.2010402@oracle.com> <52F00EE4.4070708@oracle.com> Message-ID: Makes sense - thanks for the extra explanation! Cheers, Martijn On 3 February 2014 22:49, Alan Bateman wrote: > On 03/02/2014 21:13, Martijn Verburg wrote: > >> Hi Huizhe, >> >> Is there a possibility to look at having a more loosely coupled >> relationship between Xerces and what is core JDK? I'm thinking about (in >> combination with) Jigsaw that you could allow the Xerces components to be >> kept up to date more often (assuming API compatibility etc is retained). >> >> Just to mention that are already service provider interfaces so you can > deploy with other parser libraries (or a more up to date Xerces). One of > things that we did as part of preparing for modules (JEP 162, JDK 8) was to > rev the JAXP API to clean up these the service provider interfaces (for > SAX, DOM, XSLT, XPath, Validation, Streaming, Datatype) and specify the use > of the ServiceLoader. So we're in a much better place now. > > As regards getting the Xerces code in OpenJDK in sync with the upstream > project then that clearly would be desirable and probably more of a longer > term goal. I'll let Joe speak to how much the code has diverged but one > area of difference is that the original Apache code wasn't really intended > to ever be on the boot class path or run with a security manager. In any > case, I think this JEP is a good step as it brings the implementations > closer and also revs the support on a number of standards. > > -Alan > From stuart.marks at oracle.com Mon Feb 3 22:01:27 2014 From: stuart.marks at oracle.com (Stuart Marks) Date: Mon, 03 Feb 2014 14:01:27 -0800 Subject: ObjectIn/OutputStream improvements In-Reply-To: References: <91644C57-6E0D-48ED-8E31-6AC217DECCEF@gmx.de> <1C96B28E-5E20-4D8D-B1FD-8FD5DF193EC8@oracle.com> <52EBB1B8.2030401@oracle.com> <52EBF484.9070909@oracle.com> Message-ID: <52F011B7.7020204@oracle.com> Nobody took the bait on this yet? :-) Certainly there's a lot of semi-myth on this topic, on both sides. Here's my source of mythology (or urban legend, as one might have it): http://www.ibm.com/developerworks/java/library/j-jtp09275/index.html My concern here is not so much about leaking of the StringBuilder held in a field, as Chris seemed to be responding to. I'd expect the ObjectInputStream to be GC'd at some point along with any StringBuilders it contains references to. I'm more concerned about defeating any optimizations that might occur if local variables are converted to fields. The article referenced above mentions escape analysis and possible stack allocation of locally-created objects. Offhand I don't know if stack allocation occurs for any of the builders in ObjectInputStream, but it certainly cannot occur if references are stored in fields. It would good if there were some evidence we could discuss instead of myths and urban legends. :-) Perhaps the original poster (Robert Stupp) can rerun some benchmarks with and without the conversion from locals to fields. (Earlier in the thread he posted some significant performance improvements, but my suspicion is that most of that improvement came from the conversion to use Unsafe.) I'm mindful that this may require a lot of effort. I think it would take a fair bit of work to come up with a benchmark that shows any difference between the two cases. I'm also mindful that one's intuition is often wrong. s'marks On 1/31/14 5:53 PM, Vitaly Davidovich wrote: > > One would have to measure of course, but intuitively, it seems like a good > change to reuse the stringbuilder. There's also the issue that the local > stringbuilder before was default-sized, and for large content, it would > generate even more garbage as the underlying array is expanded. > > The "temporary short lived allocations are cheap" is, unfortunately, a > semi-myth, or at least somewhat misguided. It's true that individually they > may be cheap, but they have macro effects. The higher the allocation rate, > the more young GCs we get. Every young GC (on hotspot collectors, at least) > is a STW pause. Bringing threads to a safepoint has some cost, especially if > there are many of them on large many-core machines. With larger heaps these > days, young gens tend to be larger as well. When GC runs, it trashes the cpu > caches for the mutators, so when they resume, they may get cache misses. At > each young GC, some objects may get promoted prematurely to tenured. > > So no, I wouldn't say it's quite inexpensive :). When you have no option but > to allocate, it's nice to have collectors that can handle those necessary > allocations well. Otherwise, if it's a perf sensitive area and avoiding > allocations doesn't obfuscate or make the code significantly less > maintainable, it's usually better to avoid allocations. > > Just my $.02 > > Sent from my phone > > On Jan 31, 2014 2:06 PM, "Stuart Marks" > wrote: > > On 1/31/14 10:46 AM, Chris Hegarty wrote: > > I think your patch can be split into two logical, independent, parts. The > first is the use of unsafe to access the String UTF length. The seconds is > to reuse, where possible, the existing StringBuilder instances, > skipping of > primitive/object reading/writing where applicable, and general cleanup. > > Since this is a very sensitive area I would like to consider these > separately. To that end, I have taken the changes that are applicable > to the > latter, removed any subjective stylistic changes, and made some additional > cleanup improvements. > > http://cr.openjdk.java.net/~chegar/serial_stupp.00/webrev/ > > > > I agree with splitting the Unsafe usages so they can be reviewed > separately. New > and changed usage of Unsafe will require exacting scrutiny. > > In general, the cleanups and refactorings look fine. > > I have a question about the change to reuse StringBuilder instances. This > replaces freshly allocated StringBuilders stored in local variables with > reuse of a StringBuilder stored in a field of BlockDataInputStream, which > in turn is stored in a field of ObjectInputStream. Thus, instead of > creating of lots of temporaries that become gc-eligible very quickly, this > creates a single long-lived object whose size is the high-water mark of > the longest string that's been read. The change does reduce allocation > pressure, but the point of generational GC is to make allocation and > collection of temporaries quite inexpensive. Is this the right tradeoff? > > s'marks > From sean.mullan at oracle.com Mon Feb 3 22:08:29 2014 From: sean.mullan at oracle.com (Sean Mullan) Date: Mon, 03 Feb 2014 17:08:29 -0500 Subject: JAXP JEP: Update Xerces implementation in the JDK In-Reply-To: <52EFEBD4.2010402@oracle.com> References: <52EFEBD4.2010402@oracle.com> Message-ID: <52F0135D.1030100@oracle.com> On 02/03/2014 02:19 PM, huizhe wang wrote: > The JDK contains an older Xerces implementation, version 2.7.1. Although there were > updates in JDK 7 to bring in some changes, we did not bring it completely up to date > to any later release. The goal of this JEP is to complete the update and bring the JDK > to Xerces' latest release, 2.11.0. Xerces 2.11.0 supports JDK 1.4.2 (!) and up. I am curious if you intend to avoid using newer language features or APIs in the JDK implementation so that you can stay closely in-sync and integrate future updates more easily? --Sean From mandy.chung at oracle.com Mon Feb 3 22:17:26 2014 From: mandy.chung at oracle.com (Mandy Chung) Date: Mon, 03 Feb 2014 14:17:26 -0800 Subject: RFR [9] 8031050: [macosx] Crash while awt starting In-Reply-To: <52EF9722.8090207@oracle.com> References: <52EF9722.8090207@oracle.com> Message-ID: <52F01576.5010908@oracle.com> On 2/3/2014 5:18 AM, Chris Hegarty wrote: > Hi, > > An old issue has resurfaced because of a change in AWT. AWT, now, on > Mac OS X, attaches a system graphics thread to the running VM, using > JNI_AttachCurrentThread. This change can result in a NPE, if a > security manager is installed, and the security manager tries to print > the name of the current thread. > > In this case, attaching a thread to the VM, the j.l.Thread > instance/oop is created by the VM in allocate_threadObj(), and then > its constructor is run. At this point almost everything has been > created for the thread, jni env, etc, so it can query currentThread() > to retrieve itself. (Note: this is unusual. Typically the thread > running the j.l.Thread constructor is not itself.) Since the > j.l.Thread oop was allocated directly by the VM, its fields, other > than those directly set by allocate_threadObj(), are not initialized. > > The call out from Thread.init to the security manager is problematic > as the partially constructed thread can be retrieved through > Thread.currentThread. Some of our internal tests run into a problem > when the security manager tries to print the name of the thread, i.e. > NPE. > > Clearly there are other concerns of accessing a partially constructed > thread, not just the thread name, but it seems reasonable to simply > move the setting of the name to the beginning of the init() method. > > I propose the following change for JDK 9, with the intention of backport > exactly to JDK 8. Long term this should be revisited in JDK 9. This looks okay as a short-term fix to address this particular issue (accessing the name of a partially constructed Thread object). Mandy From roger.riggs at oracle.com Mon Feb 3 22:43:08 2014 From: roger.riggs at oracle.com (roger riggs) Date: Mon, 03 Feb 2014 17:43:08 -0500 Subject: RFR java.time cleanup of javadoc and messages Message-ID: <52F01B7C.1040209@oracle.com> Please review this group of java.time updates: 8032749 : Typo in java.time.Clock 8032888 : Error message typo in TemporalAccessor 8032558 : Instant spec includes incorrect assertion wrt valid range 8032494 : DateTimeFormatter spec includes irrelevant detail on parsing pattern Webrev: http://cr.openjdk.java.net/~rriggs/webrev-typos-8032749/ Thanks, Roger From joe.darcy at oracle.com Mon Feb 3 22:46:54 2014 From: joe.darcy at oracle.com (Joe Darcy) Date: Mon, 03 Feb 2014 14:46:54 -0800 Subject: RFR java.time cleanup of javadoc and messages In-Reply-To: <52F01B7C.1040209@oracle.com> References: <52F01B7C.1040209@oracle.com> Message-ID: <52F01C5E.6050904@oracle.com> Looks fine Roger; cheers, -Joe On 02/03/2014 02:43 PM, roger riggs wrote: > Please review this group of java.time updates: > > 8032749 : Typo in > java.time.Clock > 8032888 : Error > message typo in TemporalAccessor > 8032558 : Instant > spec includes incorrect assertion wrt valid range > 8032494 : > DateTimeFormatter spec includes irrelevant detail on parsing pattern > > Webrev: > http://cr.openjdk.java.net/~rriggs/webrev-typos-8032749/ > > Thanks, Roger > From lance.andersen at oracle.com Mon Feb 3 22:50:12 2014 From: lance.andersen at oracle.com (Lance @ Oracle) Date: Mon, 3 Feb 2014 17:50:12 -0500 Subject: RFR java.time cleanup of javadoc and messages In-Reply-To: <52F01B7C.1040209@oracle.com> References: <52F01B7C.1040209@oracle.com> Message-ID: +1 Lance Andersen| Principal Member of Technical Staff | +1.781.442.2037 Oracle Java Engineering 1 Network Drive Burlington, MA 01803 Lance.Andersen at oracle.com Sent from my iPad On Feb 3, 2014, at 5:43 PM, roger riggs wrote: > Please review this group of java.time updates: > > 8032749 : Typo in java.time.Clock > 8032888 : Error message typo in TemporalAccessor > 8032558 : Instant spec includes incorrect assertion wrt valid range > 8032494 : DateTimeFormatter spec includes irrelevant detail on parsing pattern > > Webrev: > http://cr.openjdk.java.net/~rriggs/webrev-typos-8032749/ > > Thanks, Roger > From xueming.shen at oracle.com Mon Feb 3 22:55:49 2014 From: xueming.shen at oracle.com (Xueming Shen) Date: Mon, 03 Feb 2014 14:55:49 -0800 Subject: RFR java.time cleanup of javadoc and messages In-Reply-To: <52F01B7C.1040209@oracle.com> References: <52F01B7C.1040209@oracle.com> Message-ID: <52F01E75.5070204@oracle.com> The changeset looks fine. But, do we need a CCC for 8032558 and 8032494 "for the record"? -Sherman On 02/03/2014 02:43 PM, roger riggs wrote: > Please review this group of java.time updates: > > 8032749 : Typo in java.time.Clock > 8032888 : Error message typo in TemporalAccessor > 8032558 : Instant spec includes incorrect assertion wrt valid range > 8032494 : DateTimeFormatter spec includes irrelevant detail on parsing pattern > > Webrev: > http://cr.openjdk.java.net/~rriggs/webrev-typos-8032749/ > > Thanks, Roger > From huizhe.wang at oracle.com Mon Feb 3 23:31:31 2014 From: huizhe.wang at oracle.com (huizhe wang) Date: Mon, 03 Feb 2014 15:31:31 -0800 Subject: JAXP JEP: Update Xerces implementation in the JDK In-Reply-To: <52F0135D.1030100@oracle.com> References: <52EFEBD4.2010402@oracle.com> <52F0135D.1030100@oracle.com> Message-ID: <52F026D3.6020907@oracle.com> Thanks Martijn, Mario and Sean for the comments and Alan for the explanation. The Xerces implementation has forked mainly in the following areas: 1. integration The JDK StAX implementation shared the same scanner with that of SAX/DOM parsers. This part of code has been significantly modified. 2. JDK specific requirement As Alan already commented, the original Xerces implementation was not designed to be on the boot class path. There are therefore changes that are specific to meet the JDK's requirement. We do share the improvement, but the eventual changes in Xerces were made by the Xerces team and may be different. 3. JDK support Yes, Xerces still supports JDK 1.4. JAXP and JAXP in JDK however no longer supported JDK 1.4 since 2009, that allowed us to have introduced changes that used features from JDK 5. JAXP 1.6/JDK8 and later are now at source level JDK 6 and above since the ServiceLoader was introduced in JDK 6. It is desirable to get the sources in sync. As Alan said, this update is a good step towards that goal. JAXP 1.6/JEP 162 and JDK9 provide opportunity for us to look further. We shall keep in this in mind in any future changes. Thanks, Joe On 2/3/2014 2:08 PM, Sean Mullan wrote: > On 02/03/2014 02:19 PM, huizhe wang wrote: >> The JDK contains an older Xerces implementation, version 2.7.1. >> Although there were >> updates in JDK 7 to bring in some changes, we did not bring it >> completely up to date >> to any later release. The goal of this JEP is to complete the update >> and bring the JDK >> to Xerces' latest release, 2.11.0. > > Xerces 2.11.0 supports JDK 1.4.2 (!) and up. I am curious if you > intend to avoid using newer language features or APIs in the JDK > implementation so that you can stay closely in-sync and integrate > future updates more easily? > > --Sean From scolebourne at joda.org Tue Feb 4 00:09:43 2014 From: scolebourne at joda.org (Stephen Colebourne) Date: Tue, 4 Feb 2014 00:09:43 +0000 Subject: RFR java.time cleanup of javadoc and messages In-Reply-To: <52F01B7C.1040209@oracle.com> References: <52F01B7C.1040209@oracle.com> Message-ID: +1 by me Stephen On 3 February 2014 22:43, roger riggs wrote: > Please review this group of java.time updates: > > 8032749 : Typo in > java.time.Clock > 8032888 : Error message > typo in TemporalAccessor > 8032558 : Instant spec > includes incorrect assertion wrt valid range > 8032494 : > DateTimeFormatter spec includes irrelevant detail on parsing pattern > > Webrev: > http://cr.openjdk.java.net/~rriggs/webrev-typos-8032749/ > > Thanks, Roger > From tristan.yan at oracle.com Tue Feb 4 01:58:10 2014 From: tristan.yan at oracle.com (Tristan Yan) Date: Mon, 3 Feb 2014 17:58:10 -0800 (PST) Subject: =?utf-8?B?562U5aSNOiBEZW1vIGZvciBQYXJhbGxlbCBDb3JlIENvbGxlY3Rpb24gQVBJ?= In-Reply-To: <0BB78F00-5446-4071-B797-083F6BFB9932@oracle.com> References: <2CFB5F41-3C49-42A4-8C80-84F622C230E8@oracle.com> <36B4EDF7-15DE-482C-AFB7-511A03BCDC92@oracle.com> <52B2E3A5.3070807@oracle.com> <301A54AB-CD30-4763-8370-540C3B6D1914@oracle.com> <0BB78F00-5446-4071-B797-083F6BFB9932@oracle.com> Message-ID: <7a572fac-92cc-4258-b3f0-76fcf43cc281@default> Hi Paul I know this may be a little bit late. But I am still asking you review this. http://cr.openjdk.java.net/~tyan/JDK-8033358/webrev.01/ This is a whole demo code include the stream demo code you reviewed before also included parallel part. There is one other parallel demo that Paul has already pushed. Could you please review it again? Thank you so much Tristan -----????----- ???: Paul Sandoz ????: Tuesday, January 14, 2014 9:27 PM ???: core-libs-dev at openjdk.java.net ??: Tristan Yan ??: Re: Demo for Parallel Core Collection API Hi Tristan, See below for a patch. The location of the code seems a little out of place with the other code. I would have expected a structure such as: stream/parallel/*.java where the source code is in the default no-package. I am not yet convinced of the value of the RandomPrimeNumber example. In your original code you had a parallel stream invoking another parallel stream in the filter (the simple division to return a prime), this has the effect of actually slowing down the computation due to each calculation fighting for F/J resources and threads. Remove the additional parallelism and it is not clear when eyeballing the execution time that parallel is faster than sequential (it probably is, but it is handy to have obvious examples). A simpler example is to generate a list of probable primes of a certain bit length. IMHO a better example in the category of "slightly more complex" is the rendering of the Mandelbrot set, parallelized along the real or imaginary axis. Once can easily generate some nice ASCII-like art :-) Paul. On Dec 20, 2013, at 6:25 PM, Paul Sandoz wrote: > Thanks, I need to look at this in more detail, but here are some quick comments. > > - recommend you try and avoid using limit with parallel ops, for example the Pi example cam be reformulated as: > > long M = LongStream.range(0, N).parallel().filter(sr -> { > double x = ThreadLocalRandom.current().nextDouble(-1, 1); > double y = ThreadLocalRandom.current().nextDouble(-1, 1); > > return x * x + y * y < R * R; // Don't use need to use sqrt > }).count(); > double pi = (M / N) * 4.0; > > the Primes example could be reformulated as: > > LongStream.range(0, limit).parallel().map(/odd > values/).filter(RandomPrimeNumber::isPrime).findAny(); > > you don't need to declare unordered() since findAny implicitly makes the stream unordered by definition. > > The key message here is range has better decomposition characteristics than generate or iterate. > > More later, probably after the break, > Paul. diff -r d56cd0872ec4 src/share/sample/demo/parallel/MonteCarloPI.java --- a/src/share/sample/demo/parallel/MonteCarloPI.java Tue Jan 14 13:34:22 2014 +0100 +++ b/src/share/sample/demo/parallel/MonteCarloPI.java Tue Jan 14 14:15:22 2014 +0100 @@ -1,5 +1,5 @@ /* - * Copyright (c) 2013 Oracle and/or its affiliates. All rights reserved. + * Copyright (c) 2014 Oracle and/or its affiliates. All rights reserved. * * Redistribution and use in source and binary forms, with or without * modification, are permitted provided that the following conditions @@ -36,16 +36,14 @@ * input validation and proper error handling, might not be present in * this sample code. */ -package demo.parallel; import java.util.concurrent.ThreadLocalRandom; import java.util.stream.LongStream; /** - * This demo shows how to use the parallel mode and the Monte Carlo method to - * calculate the value of PI + * This demo shows how to use parallel streams to approximately + calculate the + * value of ? using the Monte Carlo method. * - * @author tyan */ public class MonteCarloPI { @@ -62,39 +60,32 @@ } /** - * Use the Monte Carlo method to calculate the value of PI. basic algorithm + * Use the Monte Carlo method to calculate the value of ?. basic + algorithm * is: 1. Draw a square on the ground, then inscribe a circle within it. 2. * Scatter some objects of uniform size (grains of rice or sand) over the * square. 3. Count the total number of objects inside the circle and the - * total number of objects overall. 4. The ratio of the two total is an - * estimate of the ratio of the two areas, which is PI/4. Multiply the - * result by 4 to estimate PI. + * total number of objects overall. 4. The ratio of the two totals is an + * estimate of the ratio of the two areas, which is pi/4. Multiply the + * result by 4 to estimate ?. * - * @param x how many times randomly selected a point - * @return value of ? by x times calculation + * @param n how many times to randomly selected a point + * @return the approximate value of ? */ - private static double pi(long x) { - return LongStream.generate(() -> hit()). - // using parallel mode - parallel(). - // select only x elements - limit(x).sum() - // perform division before multiplication to reduce ovefflow - // risk - / (double) x * 4; + private static double pi(long n) { + long m = LongStream.range(0, n).parallel().filter(i -> hit()).count(); + return (m / (double) n) * 4.0; } /** - * Use ThreadLocalRandom to simulate that whether a point is inside the - * circle or outside the circle + * Use ThreadLocalRandom to simulate that whether a point is inside a + * circle, of radius 1, or outside. * - * @return 1 randomly selected point is inside the circle 0 randomly - * selected point is outside the circle + * @return true if within the circle, otherwise false */ - private static long hit() { + private static boolean hit() { ThreadLocalRandom lr = ThreadLocalRandom.current(); - double x = lr.nextDouble(1.0); - double y = lr.nextDouble(1.0); - return Math.sqrt(y * y + x * x) <= 1.0 ? 1 : 0; + double x = lr.nextDouble(-1.0, 1.0); + double y = lr.nextDouble(-1.0, 1.0); + return x * x + y * y <= 1.0; } } diff -r d56cd0872ec4 src/share/sample/demo/parallel/RandomPrimeNumber.java --- a/src/share/sample/demo/parallel/RandomPrimeNumber.java Tue Jan 14 13:34:22 2014 +0100 +++ b/src/share/sample/demo/parallel/RandomPrimeNumber.java Tue Jan 14 14:15:22 2014 +0100 @@ -1,5 +1,5 @@ /* - * Copyright (c) 2013 Oracle and/or its affiliates. All rights reserved. + * Copyright (c) 2014 Oracle and/or its affiliates. All rights reserved. * * Redistribution and use in source and binary forms, with or without * modification, are permitted provided that the following conditions @@ -36,19 +36,19 @@ * input validation and proper error handling, might not be present in * this sample code. */ -package demo.parallel; import java.math.BigInteger; import static java.math.BigInteger.ONE; import java.util.BitSet; import java.util.OptionalLong; import java.util.Random; +import java.util.concurrent.ThreadLocalRandom; import java.util.stream.LongStream; /** - * This demo shows how to use the parallel mode to find an unknown lengths - * prime number. The process includes dividing the computation area to an m * n - * segment and then randomly selecting mth and nth segments to try to find a + * This demo shows how to use parallel streams to find a prime number. + * The process includes dividing the computation area to an m * n + * segments and then randomly selecting mth and nth segments to try to + find a * prime number in the computation area. If there is no prime number in this * segment, then this process is repeated until the entire segment has been * examined. Either there will be no prime number in this area, or the first @@ -91,7 +91,7 @@ + "prime number"); return; } - Random random = new Random(System.currentTimeMillis()); + Random random = ThreadLocalRandom.current(); /* * Prime number can not be a even number; it must be odd number. */ @@ -140,23 +140,15 @@ long start = first_number.longValue() + randomHighSeg * LOW_BIT_SEGMENT * SEGMENT_SIZE + randomLowSeg * SEGMENT_SIZE; - OptionalLong anyPrime = LongStream.iterate(start, l -> l + 2). - // You must first specify the limit to guarantee that - // the length of the number is an acceptable length. - limit(limit). - // From this point, use parallel(). - // You have to find only one prime number, so the - // order is not important. Also, an unordered stream - // is more efficient than an ordered stream. + + OptionalLong anyPrime = LongStream.range(0, limit). parallel(). - // we only need find one prime number, so keeping - // order is not needed. Also unordered stream is - // more efficient than ordered stream. - unordered(). + // Generate a sequence of limit elements as as follows + // start, start + 2, start + 4, start + 6 + map(n -> n * 2 + start). // Filter only the prime number filter(RandomPrimeNumber::isPrime). - //Stop the process as soon as you find a prime - //number. + // Stop the process as soon as you find a prime number. findAny(); if (anyPrime.isPresent()) { System.out.println(String.format( @@ -176,13 +168,12 @@ /** * Decide if a BigInteger is a prime number * - * @param integer a number + * @param number a number * @return true if integer is a prime number false if integer is not a prime * number */ private static boolean isPrime(long number) { - //This is a parall version that checks if a number is a prime number - return !LongStream.range(2L, Math.round(Math.sqrt(number))).parallel(). + return !LongStream.range(2L, Math.round(Math.sqrt(number))). anyMatch(divisor -> number % divisor == 0); } From joe.darcy at oracle.com Tue Feb 4 06:09:44 2014 From: joe.darcy at oracle.com (Joe Darcy) Date: Mon, 03 Feb 2014 22:09:44 -0800 Subject: JDK 9 RFR of JDK-8033527: Fix raw type lint warning in sun.nio.ch Message-ID: <52F08428.80100@oracle.com> Hello, Please review this small fix to address JDK-803352: Fix raw type lint warning in sun.nio.ch --- a/src/share/classes/sun/nio/ch/Reflect.java Mon Feb 03 16:58:02 2014 -0500 +++ b/src/share/classes/sun/nio/ch/Reflect.java Mon Feb 03 22:07:47 2014 -0800 @@ -1,5 +1,5 @@ /* - * Copyright (c) 2000, 2011, Oracle and/or its affiliates. All rights reserved. + * Copyright (c) 2000, 2014, Oracle and/or its affiliates. All rights reserved. * DO NOT ALTER OR REMOVE COPYRIGHT NOTICES OR THIS FILE HEADER. * * This code is free software; you can redistribute it and/or modify it @@ -75,7 +75,7 @@ static Method lookupMethod(String className, String methodName, - Class... paramTypes) + Class... paramTypes) { try { Class cl = Class.forName(className); The build succeeds with this change in place. Thanks, -Joe From david.holmes at oracle.com Tue Feb 4 06:11:25 2014 From: david.holmes at oracle.com (David Holmes) Date: Tue, 04 Feb 2014 16:11:25 +1000 Subject: RFR [9] 8031050: [macosx] Crash while awt starting In-Reply-To: <52EF9F9E.7060705@oracle.com> References: <52EF9722.8090207@oracle.com> <52EF9F9E.7060705@oracle.com> Message-ID: <52F0848D.7090101@oracle.com> On 3/02/2014 11:54 PM, Alan Bateman wrote: > On 03/02/2014 13:18, Chris Hegarty wrote: >> Hi, >> >> An old issue has resurfaced because of a change in AWT. AWT, now, on >> Mac OS X, attaches a system graphics thread to the running VM, using >> JNI_AttachCurrentThread. This change can result in a NPE, if a >> security manager is installed, and the security manager tries to print >> the name of the current thread. >> >> In this case, attaching a thread to the VM, the j.l.Thread >> instance/oop is created by the VM in allocate_threadObj(), and then >> its constructor is run. At this point almost everything has been >> created for the thread, jni env, etc, so it can query currentThread() >> to retrieve itself. (Note: this is unusual. Typically the thread >> running the j.l.Thread constructor is not itself.) Since the >> j.l.Thread oop was allocated directly by the VM, its fields, other >> than those directly set by allocate_threadObj(), are not initialized. >> >> The call out from Thread.init to the security manager is problematic >> as the partially constructed thread can be retrieved through >> Thread.currentThread. Some of our internal tests run into a problem >> when the security manager tries to print the name of the thread, i.e. >> NPE. >> >> Clearly there are other concerns of accessing a partially constructed >> thread, not just the thread name, but it seems reasonable to simply >> move the setting of the name to the beginning of the init() method. >> >> I propose the following change for JDK 9, with the intention of backport >> exactly to JDK 8. Long term this should be revisited in JDK 9. > As a short term fix then this looks okay to me. An alternative might be > for getName to return the empty String if called before the name is set. I think the current fix is preferable as it returns the correct name rather than just side-stepping the NPE. > I assume that part of re-visiting later will be to update API docs > (maybe JNI spec too) to allow for initialization on the "current thread" > when attaching as it can't be guaranteed to be fully initialized at this > point. What API docs do you think need updating? The revisiting would be to try and find a complete solution to this initialization problem. Unfortunately it is a catch-22: only an attached java thread can execute Java code; initialization of the Thread object during the attach has to execute Java code. Some possible solutions are suggested in a linked CR, but none of them are ideal. Cheers, David > > -Alan. > > PS: We should probably change the bug synopsis to make it clearer what > this issue is about now. From henry.jen at oracle.com Tue Feb 4 06:18:18 2014 From: henry.jen at oracle.com (Henry Jen) Date: Mon, 03 Feb 2014 22:18:18 -0800 Subject: JDK 9 RFR of JDK-8033527: Fix raw type lint warning in sun.nio.ch In-Reply-To: <52F08428.80100@oracle.com> References: <52F08428.80100@oracle.com> Message-ID: <52F0862A.9040106@oracle.com> +1. Cheers, Henry On 02/03/2014 10:09 PM, Joe Darcy wrote: > Hello, > > Please review this small fix to address > > JDK-803352: Fix raw type lint warning in sun.nio.ch > > --- a/src/share/classes/sun/nio/ch/Reflect.java Mon Feb 03 16:58:02 > 2014 -0500 > +++ b/src/share/classes/sun/nio/ch/Reflect.java Mon Feb 03 22:07:47 > 2014 -0800 > @@ -1,5 +1,5 @@ > /* > - * Copyright (c) 2000, 2011, Oracle and/or its affiliates. All rights > reserved. > + * Copyright (c) 2000, 2014, Oracle and/or its affiliates. All rights > reserved. > * DO NOT ALTER OR REMOVE COPYRIGHT NOTICES OR THIS FILE HEADER. > * > * This code is free software; you can redistribute it and/or modify it > @@ -75,7 +75,7 @@ > > static Method lookupMethod(String className, > String methodName, > - Class... paramTypes) > + Class... paramTypes) > { > try { > Class cl = Class.forName(className); > > The build succeeds with this change in place. > > Thanks, > > -Joe From mandy.chung at oracle.com Tue Feb 4 06:24:17 2014 From: mandy.chung at oracle.com (Mandy Chung) Date: Mon, 03 Feb 2014 22:24:17 -0800 Subject: JDK 9 RFR of JDK-8033527: Fix raw type lint warning in sun.nio.ch In-Reply-To: <52F08428.80100@oracle.com> References: <52F08428.80100@oracle.com> Message-ID: <52F08791.80906@oracle.com> Thumbs up. Mandy On 2/3/2014 10:09 PM, Joe Darcy wrote: > Hello, > > Please review this small fix to address > > JDK-803352: Fix raw type lint warning in sun.nio.ch > > --- a/src/share/classes/sun/nio/ch/Reflect.java Mon Feb 03 16:58:02 > 2014 -0500 > +++ b/src/share/classes/sun/nio/ch/Reflect.java Mon Feb 03 22:07:47 > 2014 -0800 > @@ -1,5 +1,5 @@ > /* > - * Copyright (c) 2000, 2011, Oracle and/or its affiliates. All rights > reserved. > + * Copyright (c) 2000, 2014, Oracle and/or its affiliates. All rights > reserved. > * DO NOT ALTER OR REMOVE COPYRIGHT NOTICES OR THIS FILE HEADER. > * > * This code is free software; you can redistribute it and/or modify it > @@ -75,7 +75,7 @@ > > static Method lookupMethod(String className, > String methodName, > - Class... paramTypes) > + Class... paramTypes) > { > try { > Class cl = Class.forName(className); > > The build succeeds with this change in place. > > Thanks, > > -Joe From paul.sandoz at oracle.com Tue Feb 4 09:34:28 2014 From: paul.sandoz at oracle.com (Paul Sandoz) Date: Tue, 4 Feb 2014 10:34:28 +0100 Subject: Demo for Parallel Core Collection API In-Reply-To: <7a572fac-92cc-4258-b3f0-76fcf43cc281@default> References: <2CFB5F41-3C49-42A4-8C80-84F622C230E8@oracle.com> <36B4EDF7-15DE-482C-AFB7-511A03BCDC92@oracle.com> <52B2E3A5.3070807@oracle.com> <301A54AB-CD30-4763-8370-540C3B6D1914@oracle.com> <0BB78F00-5446-4071-B797-083F6BFB9932@oracle.com> <7a572fac-92cc-4258-b3f0-76fcf43cc281@default> Message-ID: <4EAEBA7F-EDDB-4EB6-98F3-2A10DDE18F9B@oracle.com> On Feb 4, 2014, at 2:58 AM, Tristan Yan wrote: > Hi Paul > I know this may be a little bit late. Yes, likely too late... (see below). > But I am still asking you review this. > > http://cr.openjdk.java.net/~tyan/JDK-8033358/webrev.01/ > > This is a whole demo code include the stream demo code you reviewed before also included parallel part. There is one other parallel demo that Paul has already pushed. > Could you please review it again? Fibonacci - There is no parallel execution; one can write as follows for better splitting: return IntStream.range(0, n).mapToObj(i -> matrix). parallel(). reduce(Fibonacci::times). get(); - While the approach is interesting (elegant in terms of the reduction) one is not likely to observe any speed up (did you measure?) since the long value will overflow on the 93rd number [1]. You would need to use BigInteger and a larger number of iterations. ImageTransform - In rotate/shift/zoom/invert the inner range (over height) should not be parallel, it's just gonna fight for resources with the outer range (over width). - It's not entirely clear to me if BufferedImage is thread safe to concurrently set the pixels MonteCarloPI - hit() method is no longer used. RandomPrimeNumber - better to use ThreadLocalRandom or SplittableRandom, as both of those produce higher quality sequences of random numbers. - i don't think this example has much value as there are better ways to determine primes. As i have said before, if you want to show a simple example it is better to show the parallel generation of probable primes using BigInteger. -- I am sorry to have to say this, i know if you have worked on this for a while, but i think this is now too late to go through another round of reviews. Paul. [1] Think rabbits! http://www.maths.surrey.ac.uk/hosted-sites/R.Knott/Fibonacci/fibtable.html You can think From Alan.Bateman at oracle.com Tue Feb 4 10:35:14 2014 From: Alan.Bateman at oracle.com (Alan Bateman) Date: Tue, 04 Feb 2014 10:35:14 +0000 Subject: 8033372: Check jdk/src/share/native/java/lang for JNI pending exception issues Message-ID: <52F0C262.7040204@oracle.com> There are a number of places in the native methods where there are sequences of JNI calls and where we don't check for errors or pending exceptions. This patch fixes up a number of these issues in the java.lang native code, and one remaining one in the java.io native code that may have been missed by previous clean-up in this area. The changes are trivial and the webrev is here: http://cr.openjdk.java.net/~alanb/8033372/webrev/ Thanks, Alan. From paul.sandoz at oracle.com Tue Feb 4 11:00:27 2014 From: paul.sandoz at oracle.com (Paul Sandoz) Date: Tue, 4 Feb 2014 12:00:27 +0100 Subject: ObjectIn/OutputStream improvements In-Reply-To: <52F011B7.7020204@oracle.com> References: <91644C57-6E0D-48ED-8E31-6AC217DECCEF@gmx.de> <1C96B28E-5E20-4D8D-B1FD-8FD5DF193EC8@oracle.com> <52EBB1B8.2030401@oracle.com> <52EBF484.9070909@oracle.com> <52F011B7.7020204@oracle.com> Message-ID: On Feb 3, 2014, at 11:01 PM, Stuart Marks wrote: > Nobody took the bait on this yet? :-) > > Certainly there's a lot of semi-myth on this topic, on both sides. Here's my source of mythology (or urban legend, as one might have it): > > http://www.ibm.com/developerworks/java/library/j-jtp09275/index.html > > My concern here is not so much about leaking of the StringBuilder held in a field, as Chris seemed to be responding to. I'd expect the ObjectInputStream to be GC'd at some point along with any StringBuilders it contains references to. > > I'm more concerned about defeating any optimizations that might occur if local variables are converted to fields. The article referenced above mentions escape analysis and possible stack allocation of locally-created objects. Offhand I don't know if stack allocation occurs for any of the builders in ObjectInputStream, but it certainly cannot occur if references are stored in fields. > The caching of the StringBuffer seems reasonable point fix. (If one is worried about field access retain/expand the original method signatures so field access is hoisted outside of the loops.) However, if we want to focus on performance i think it better to change the UTF decoding algorithm to target small'ish ASCII strings (size < CHAR_BUF_SIZE) thereby avoiding the use of StringBuilder for presumed common cases. (IIRC a similar approach in a different area was proposed by Sherman for converting strings to lower/upper cases.) I think that would be a better way to spend our performance investigation budget. Thus, it might be better to remove the StringBuilder field from this patch and submit another one focusing on UTF decoding; the other changes in the patch look good. Paul. > It would good if there were some evidence we could discuss instead of myths and urban legends. :-) Perhaps the original poster (Robert Stupp) can rerun some benchmarks with and without the conversion from locals to fields. (Earlier in the thread he posted some significant performance improvements, but my suspicion is that most of that improvement came from the conversion to use Unsafe.) > > I'm mindful that this may require a lot of effort. I think it would take a fair bit of work to come up with a benchmark that shows any difference between the two cases. I'm also mindful that one's intuition is often wrong. > > s'marks From chris.hegarty at oracle.com Tue Feb 4 11:05:00 2014 From: chris.hegarty at oracle.com (Chris Hegarty) Date: Tue, 04 Feb 2014 11:05:00 +0000 Subject: 8033372: Check jdk/src/share/native/java/lang for JNI pending exception issues In-Reply-To: <52F0C262.7040204@oracle.com> References: <52F0C262.7040204@oracle.com> Message-ID: <52F0C95C.2050506@oracle.com> Nice work. Looks good to me. -Chris. On 02/04/2014 10:35 AM, Alan Bateman wrote: > > There are a number of places in the native methods where there are > sequences of JNI calls and where we don't check for errors or pending > exceptions. This patch fixes up a number of these issues in the > java.lang native code, and one remaining one in the java.io native code > that may have been missed by previous clean-up in this area. > > The changes are trivial and the webrev is here: > > http://cr.openjdk.java.net/~alanb/8033372/webrev/ > > Thanks, > > Alan. From lance.andersen at oracle.com Tue Feb 4 11:39:08 2014 From: lance.andersen at oracle.com (Lance @ Oracle) Date: Tue, 4 Feb 2014 06:39:08 -0500 Subject: JDK 9 RFR of JDK-8033527: Fix raw type lint warning in sun.nio.ch In-Reply-To: <52F08428.80100@oracle.com> References: <52F08428.80100@oracle.com> Message-ID: +1 Lance Andersen| Principal Member of Technical Staff | +1.781.442.2037 Oracle Java Engineering 1 Network Drive Burlington, MA 01803 Lance.Andersen at oracle.com Sent from my iPad On Feb 4, 2014, at 1:09 AM, Joe Darcy wrote: > Hello, > > Please review this small fix to address > > JDK-803352: Fix raw type lint warning in sun.nio.ch > > --- a/src/share/classes/sun/nio/ch/Reflect.java Mon Feb 03 16:58:02 2014 -0500 > +++ b/src/share/classes/sun/nio/ch/Reflect.java Mon Feb 03 22:07:47 2014 -0800 > @@ -1,5 +1,5 @@ > /* > - * Copyright (c) 2000, 2011, Oracle and/or its affiliates. All rights reserved. > + * Copyright (c) 2000, 2014, Oracle and/or its affiliates. All rights reserved. > * DO NOT ALTER OR REMOVE COPYRIGHT NOTICES OR THIS FILE HEADER. > * > * This code is free software; you can redistribute it and/or modify it > @@ -75,7 +75,7 @@ > > static Method lookupMethod(String className, > String methodName, > - Class... paramTypes) > + Class... paramTypes) > { > try { > Class cl = Class.forName(className); > > The build succeeds with this change in place. > > Thanks, > > -Joe From sean.coffey at oracle.com Tue Feb 4 13:31:31 2014 From: sean.coffey at oracle.com (=?ISO-8859-1?Q?Se=E1n_Coffey?=) Date: Tue, 04 Feb 2014 13:31:31 +0000 Subject: RFR : 8029020 : Check src/share/native/java/util/zip code for JNI pending exceptions Message-ID: <52F0EBB3.3090602@oracle.com> Looking to correct some JNI handling code in java/util/zip for jdk9-dev. webrev : http://cr.openjdk.java.net/~coffeys/webrev.8029020/webrev/ regards, Sean. From chris.hegarty at oracle.com Tue Feb 4 13:42:08 2014 From: chris.hegarty at oracle.com (Chris Hegarty) Date: Tue, 04 Feb 2014 13:42:08 +0000 Subject: RFR : 8029020 : Check src/share/native/java/util/zip code for JNI pending exceptions In-Reply-To: <52F0EBB3.3090602@oracle.com> References: <52F0EBB3.3090602@oracle.com> Message-ID: <52F0EE30.4040108@oracle.com> Looks good to me. Thanks Sean. -Chris. On 02/04/2014 01:31 PM, Se?n Coffey wrote: > Looking to correct some JNI handling code in java/util/zip for jdk9-dev. > > webrev : http://cr.openjdk.java.net/~coffeys/webrev.8029020/webrev/ > > > regards, > Sean. > From Alan.Bateman at oracle.com Tue Feb 4 13:51:05 2014 From: Alan.Bateman at oracle.com (Alan Bateman) Date: Tue, 04 Feb 2014 13:51:05 +0000 Subject: RFR : 8029020 : Check src/share/native/java/util/zip code for JNI pending exceptions In-Reply-To: <52F0EBB3.3090602@oracle.com> References: <52F0EBB3.3090602@oracle.com> Message-ID: <52F0F049.4080301@oracle.com> On 04/02/2014 13:31, Se?n Coffey wrote: > Looking to correct some JNI handling code in java/util/zip for jdk9-dev. > > webrev : http://cr.openjdk.java.net/~coffeys/webrev.8029020/webrev/ > Looks good. A minor comment is that the ZipFile.c code could use JNU_ClassString instead of FindClass. -Alan From sean.coffey at oracle.com Tue Feb 4 14:10:00 2014 From: sean.coffey at oracle.com (=?ISO-8859-1?Q?Se=E1n_Coffey?=) Date: Tue, 04 Feb 2014 14:10:00 +0000 Subject: RFR : 8029020 : Check src/share/native/java/util/zip code for JNI pending exceptions In-Reply-To: <52F0F049.4080301@oracle.com> References: <52F0EBB3.3090602@oracle.com> <52F0F049.4080301@oracle.com> Message-ID: <52F0F4B8.7070403@oracle.com> thanks for reviews. webrev updated : http://cr.openjdk.java.net/~coffeys/webrev.8029020.v2/webrev/ I'll push pending positive build & test results. regards, Sean. On 04/02/14 13:51, Alan Bateman wrote: > On 04/02/2014 13:31, Se?n Coffey wrote: >> Looking to correct some JNI handling code in java/util/zip for jdk9-dev. >> >> webrev : http://cr.openjdk.java.net/~coffeys/webrev.8029020/webrev/ >> > Looks good. A minor comment is that the ZipFile.c code could use > JNU_ClassString instead of FindClass. > > -Alan From roger.riggs at oracle.com Tue Feb 4 14:33:57 2014 From: roger.riggs at oracle.com (roger riggs) Date: Tue, 04 Feb 2014 09:33:57 -0500 Subject: RFR: Checking for JNI Exceptions in jni_util.c Message-ID: <52F0FA55.9000301@oracle.com> Please review these cleanups to check for JNI pending exceptions in jni_util.c. Webrev: http://cr.openjdk.java.net/~rriggs/webrev-check_exception-8030993/ Thanks, Roger From chris.hegarty at oracle.com Tue Feb 4 14:41:46 2014 From: chris.hegarty at oracle.com (Chris Hegarty) Date: Tue, 04 Feb 2014 14:41:46 +0000 Subject: RFR: Checking for JNI Exceptions in jni_util.c In-Reply-To: <52F0FA55.9000301@oracle.com> References: <52F0FA55.9000301@oracle.com> Message-ID: <52F0FC2A.8070809@oracle.com> Looks good to me Roger. -Chris. On 02/04/2014 02:33 PM, roger riggs wrote: > Please review these cleanups to check for JNI pending exceptions in > jni_util.c. > > Webrev: > http://cr.openjdk.java.net/~rriggs/webrev-check_exception-8030993/ > > Thanks, Roger From Alan.Bateman at oracle.com Tue Feb 4 15:12:49 2014 From: Alan.Bateman at oracle.com (Alan Bateman) Date: Tue, 04 Feb 2014 15:12:49 +0000 Subject: RFR: Checking for JNI Exceptions in jni_util.c In-Reply-To: <52F0FA55.9000301@oracle.com> References: <52F0FA55.9000301@oracle.com> Message-ID: <52F10371.8010306@oracle.com> On 04/02/2014 14:33, roger riggs wrote: > Please review these cleanups to check for JNI pending exceptions in > jni_util.c. > > Webrev: > http://cr.openjdk.java.net/~rriggs/webrev-check_exception-8030993/ This mostly looks okay, I just wonder about the jclass declarations at L686 and 690 as I assume these are declaration-after-use warnings with some compilers. Also there is JNU_ClassString use at L760 that I assume should be replaced with strClazz. -Alan. From Alan.Bateman at oracle.com Tue Feb 4 15:41:24 2014 From: Alan.Bateman at oracle.com (Alan Bateman) Date: Tue, 04 Feb 2014 15:41:24 +0000 Subject: RFR : 8029020 : Check src/share/native/java/util/zip code for JNI pending exceptions In-Reply-To: <52F0F4B8.7070403@oracle.com> References: <52F0EBB3.3090602@oracle.com> <52F0F049.4080301@oracle.com> <52F0F4B8.7070403@oracle.com> Message-ID: <52F10A24.1030703@oracle.com> On 04/02/2014 14:10, Se?n Coffey wrote: > thanks for reviews. webrev updated : > http://cr.openjdk.java.net/~coffeys/webrev.8029020.v2/webrev/ > Looks good to me. -Alan. From xueming.shen at oracle.com Tue Feb 4 16:01:16 2014 From: xueming.shen at oracle.com (Xueming Shen) Date: Tue, 04 Feb 2014 08:01:16 -0800 Subject: RFR : 8029020 : Check src/share/native/java/util/zip code for JNI pending exceptions In-Reply-To: <52F0EBB3.3090602@oracle.com> References: <52F0EBB3.3090602@oracle.com> Message-ID: <52F10ECC.8080705@oracle.com> On 2/4/14 5:31 AM, Se?n Coffey wrote: > Looking to correct some JNI handling code in java/util/zip for jdk9-dev. > > webrev : http://cr.openjdk.java.net/~coffeys/webrev.8029020/webrev/ > > > regards, > Sean. > looks fine. From belliottsmith at datastax.com Tue Feb 4 17:00:43 2014 From: belliottsmith at datastax.com (Benedict Elliott Smith) Date: Tue, 4 Feb 2014 17:00:43 +0000 Subject: ArrayList.removeAll()/retainAll() Message-ID: Hi, I notice this (or a related issue) has been mentioned beforeon this list, but I'm not convinced the correct resolution was reached. We are seeing this problem thrown by antlr, but rather than a bug in antlr, as surmised on the previous exchange, it looks to me that ArrayList is imposing a new constraint that is neither declared by itself nor Collection, and is unnecessary. ArrayList happily supports null elements, so requiring that the provided collection has no null elements is surely a bug? I've pasted the two declarations below for ease of reference. Neither javadocs describe the constraint that is imposed. ArrayList: /** * Removes from this list all of its elements that are contained in the * specified collection. * * @param c collection containing elements to be removed from this list * @return {@code true} if this list changed as a result of the call * @throws ClassCastException if the class of an element of this list * is incompatible with the specified collection * (optional) * @throws NullPointerException if this list contains a null element and the * specified collection does not permit null elements * (optional), * or if the specified collection is null * @see Collection#contains(Object) */ public boolean removeAll(Collection c) { Objects.requireNonNull(c); return batchRemove(c, false); } Collection: /** * Removes all of this collection's elements that are also contained in the * specified collection (optional operation). After this call returns, * this collection will contain no elements in common with the specified * collection. * * @param c collection containing elements to be removed from this collection * @return true if this collection changed as a result of the * call * @throws UnsupportedOperationException if the removeAll method * is not supported by this collection * @throws ClassCastException if the types of one or more elements * in this collection are incompatible with the specified * collection * (optional) * @throws NullPointerException if this collection contains one or more * null elements and the specified collection does not support * null elements * (optional), * or if the specified collection is null * @see #remove(Object) * @see #contains(Object) */ boolean removeAll(Collection c); From tristan.yan at oracle.com Tue Feb 4 17:20:33 2014 From: tristan.yan at oracle.com (Tristan Yan) Date: Tue, 4 Feb 2014 09:20:33 -0800 (PST) Subject: =?gb2312?B?tPC4tDogRGVtbyBmb3IgUGFyYWxsZWwgQ29yZSBDb2xsZWN0aW9uIEFQSQ==?= In-Reply-To: <4EAEBA7F-EDDB-4EB6-98F3-2A10DDE18F9B@oracle.com> References: <2CFB5F41-3C49-42A4-8C80-84F622C230E8@oracle.com> <36B4EDF7-15DE-482C-AFB7-511A03BCDC92@oracle.com> <52B2E3A5.3070807@oracle.com> <301A54AB-CD30-4763-8370-540C3B6D1914@oracle.com> <0BB78F00-5446-4071-B797-083F6BFB9932@oracle.com> <7a572fac-92cc-4258-b3f0-76fcf43cc281@default> <4EAEBA7F-EDDB-4EB6-98F3-2A10DDE18F9B@oracle.com> Message-ID: <8c7fcde8-46b1-44d4-bf70-23762ae676b7@default> Thank you Paul. I was told maybe we will target this on 8u20 if we can't catch 8. Let's do the open review first. http://cr.openjdk.java.net/~tyan/JDK-8033358/webrev.02/ Fibonacci: Update as you suggested. Test shows parallel version is faster than sequential version when the input is greater than 100,000. ImageTransform Inner parallel was eliminated. Also BufferedImage.set is a thread safe method. MonteCarloPI Unused hit method was removed. RandomPrimeNumber It's changed into ProbablePrimeNumber. Following your suggestion to use BigInteger.probablePrime method. Thank you Tristan -----????----- ???: Paul Sandoz ????: Tuesday, February 04, 2014 5:34 PM ???: Tristan Yan ??: core-libs-dev at openjdk.java.net; Aleksandre Iline; Mikhail Kondratyev ??: Re: Demo for Parallel Core Collection API On Feb 4, 2014, at 2:58 AM, Tristan Yan wrote: > Hi Paul > I know this may be a little bit late. Yes, likely too late... (see below). > But I am still asking you review this. > > http://cr.openjdk.java.net/~tyan/JDK-8033358/webrev.01/ > > This is a whole demo code include the stream demo code you reviewed before also included parallel part. There is one other parallel demo that Paul has already pushed. > Could you please review it again? Fibonacci - There is no parallel execution; one can write as follows for better splitting: return IntStream.range(0, n).mapToObj(i -> matrix). parallel(). reduce(Fibonacci::times). get(); - While the approach is interesting (elegant in terms of the reduction) one is not likely to observe any speed up (did you measure?) since the long value will overflow on the 93rd number [1]. You would need to use BigInteger and a larger number of iterations. ImageTransform - In rotate/shift/zoom/invert the inner range (over height) should not be parallel, it's just gonna fight for resources with the outer range (over width). - It's not entirely clear to me if BufferedImage is thread safe to concurrently set the pixels MonteCarloPI - hit() method is no longer used. RandomPrimeNumber - better to use ThreadLocalRandom or SplittableRandom, as both of those produce higher quality sequences of random numbers. - i don't think this example has much value as there are better ways to determine primes. As i have said before, if you want to show a simple example it is better to show the parallel generation of probable primes using BigInteger. -- I am sorry to have to say this, i know if you have worked on this for a while, but i think this is now too late to go through another round of reviews. Paul. [1] Think rabbits! http://www.maths.surrey.ac.uk/hosted-sites/R.Knott/Fibonacci/fibtable.html You can think From wasserman.louis at gmail.com Tue Feb 4 17:22:33 2014 From: wasserman.louis at gmail.com (Louis Wasserman) Date: Tue, 4 Feb 2014 09:22:33 -0800 Subject: ArrayList.removeAll()/retainAll() In-Reply-To: References: Message-ID: I don't follow. It looks like ArrayList is throwing an exception on removeAll(null) -- perfectly valid, in the spec -- not throwing an exception when the collection contains null. Louis Wasserman wasserman.louis at gmail.com http://profiles.google.com/wasserman.louis On Tue, Feb 4, 2014 at 9:00 AM, Benedict Elliott Smith < belliottsmith at datastax.com> wrote: > Hi, > > I notice this (or a related issue) has been mentioned > before< > http://mail.openjdk.java.net/pipermail/core-libs-dev/2013-June/017663.html > >on > this list, but I'm not convinced the correct resolution was reached. > We > are seeing this problem thrown by antlr, but rather than a bug in antlr, as > surmised on the previous exchange, it looks to me that ArrayList is > imposing a new constraint that is neither declared by itself nor > Collection, and is unnecessary. ArrayList happily supports null elements, > so requiring that the provided collection has no null elements is surely a > bug? > > I've pasted the two declarations below for ease of reference. Neither > javadocs describe the constraint that is imposed. > > ArrayList: > /** > * Removes from this list all of its elements that are contained in the > * specified collection. > * > * @param c collection containing elements to be removed from this list > * @return {@code true} if this list changed as a result of the call > * @throws ClassCastException if the class of an element of this list > * is incompatible with the specified collection > * (optional) > * @throws NullPointerException if this list contains a null element > and the > * specified collection does not permit null elements > * (optional), > * or if the specified collection is null > * @see Collection#contains(Object) > */ > public boolean removeAll(Collection c) { > Objects.requireNonNull(c); > return batchRemove(c, false); > } > > Collection: > /** > * Removes all of this collection's elements that are also contained in > the > * specified collection (optional operation). After this call returns, > * this collection will contain no elements in common with the > specified > * collection. > * > * @param c collection containing elements to be removed from this > collection > * @return true if this collection changed as a result of the > * call > * @throws UnsupportedOperationException if the removeAll > method > * is not supported by this collection > * @throws ClassCastException if the types of one or more elements > * in this collection are incompatible with the specified > * collection > * (optional) > * @throws NullPointerException if this collection contains one or more > * null elements and the specified collection does not support > * null elements > * (optional), > * or if the specified collection is null > * @see #remove(Object) > * @see #contains(Object) > */ > boolean removeAll(Collection c); > From belliottsmith at datastax.com Tue Feb 4 17:28:51 2014 From: belliottsmith at datastax.com (Benedict Elliott Smith) Date: Tue, 4 Feb 2014 17:28:51 +0000 Subject: ArrayList.removeAll()/retainAll() In-Reply-To: References: Message-ID: Ah. I'm a hasty idiot. If you could all pretend I never sent that email, that would be great. On 4 February 2014 17:22, Louis Wasserman wrote: > I don't follow. It looks like ArrayList is throwing an exception on > removeAll(null) -- perfectly valid, in the spec -- not throwing an > exception when the collection contains null. > > Louis Wasserman > wasserman.louis at gmail.com > http://profiles.google.com/wasserman.louis > > > On Tue, Feb 4, 2014 at 9:00 AM, Benedict Elliott Smith < > belliottsmith at datastax.com> wrote: > >> Hi, >> >> I notice this (or a related issue) has been mentioned >> before< >> http://mail.openjdk.java.net/pipermail/core-libs-dev/2013-June/017663.html >> >on >> >> this list, but I'm not convinced the correct resolution was reached. >> We >> are seeing this problem thrown by antlr, but rather than a bug in antlr, >> as >> surmised on the previous exchange, it looks to me that ArrayList is >> imposing a new constraint that is neither declared by itself nor >> Collection, and is unnecessary. ArrayList happily supports null elements, >> so requiring that the provided collection has no null elements is surely a >> bug? >> >> I've pasted the two declarations below for ease of reference. Neither >> javadocs describe the constraint that is imposed. >> >> ArrayList: >> /** >> * Removes from this list all of its elements that are contained in >> the >> * specified collection. >> * >> * @param c collection containing elements to be removed from this >> list >> * @return {@code true} if this list changed as a result of the call >> * @throws ClassCastException if the class of an element of this list >> * is incompatible with the specified collection >> * (optional) >> * @throws NullPointerException if this list contains a null element >> and the >> * specified collection does not permit null elements >> * (optional), >> * or if the specified collection is null >> * @see Collection#contains(Object) >> */ >> public boolean removeAll(Collection c) { >> Objects.requireNonNull(c); >> return batchRemove(c, false); >> } >> >> Collection: >> /** >> * Removes all of this collection's elements that are also contained >> in >> the >> * specified collection (optional operation). After this call >> returns, >> * this collection will contain no elements in common with the >> specified >> * collection. >> * >> * @param c collection containing elements to be removed from this >> collection >> * @return true if this collection changed as a result of the >> * call >> * @throws UnsupportedOperationException if the removeAll >> method >> * is not supported by this collection >> * @throws ClassCastException if the types of one or more elements >> * in this collection are incompatible with the specified >> * collection >> * (optional) >> * @throws NullPointerException if this collection contains one or >> more >> * null elements and the specified collection does not support >> * null elements >> * (optional), >> * or if the specified collection is null >> * @see #remove(Object) >> * @see #contains(Object) >> */ >> boolean removeAll(Collection c); >> > > From mike.duigou at oracle.com Tue Feb 4 17:35:42 2014 From: mike.duigou at oracle.com (Mike Duigou) Date: Tue, 4 Feb 2014 09:35:42 -0800 Subject: ArrayList.removeAll()/retainAll() In-Reply-To: References: Message-ID: <1BF8A83E-7998-457C-A1A2-91EBA769B3AC@oracle.com> The condition that is causing the problem is not a collection containing null, which is allowed, but that the provided collection "c" is itself null. The problem was is a consequence of the following implementation: Iterator iterator = iterator(); while(iterator.hasNext()) { if(c.contains(iterator.next()) { iterator.remove(); } } This implementation had an inconsistent behaviour. If a collection contains elements then the first iteration of the while loops will cause a NullPointerException if "c" is null. If, however, the collection is empty then c is never referenced and no NPE is thrown. The change in java 8 which adds an explicit Objects.requireNonNull(c) ensures the behaviour is consistent whether the collection is empty or not. Passing null as the parameter for removeAll and retainAll has never been valid. What's changed is that it's now consistently checked. It's unfortunate that this particular fix tickled a bug in Antlr 3. It appears that it's simply a bug that Antlr doesn't check the result of it's method which returns null. (https://github.com/antlr/antlr3/blob/master/tool/src/main/java/org/antlr/tool/CompositeGrammar.java#L226) The fix would seem to be relatively straightforward to either have getDirectDelegates() return Collections.emptyList() or add a check for null. My personal preference would be for the former as it allows for uniform handling of the result wherever the method is called without having to remember that the result might be null. Mike On Feb 4 2014, at 09:00 , Benedict Elliott Smith wrote: > Hi, > > I notice this (or a related issue) has been mentioned > beforeon > this list, but I'm not convinced the correct resolution was reached. > We > are seeing this problem thrown by antlr, but rather than a bug in antlr, as > surmised on the previous exchange, it looks to me that ArrayList is > imposing a new constraint that is neither declared by itself nor > Collection, and is unnecessary. ArrayList happily supports null elements, > so requiring that the provided collection has no null elements is surely a > bug? > > I've pasted the two declarations below for ease of reference. Neither > javadocs describe the constraint that is imposed. > > ArrayList: > /** > * Removes from this list all of its elements that are contained in the > * specified collection. > * > * @param c collection containing elements to be removed from this list > * @return {@code true} if this list changed as a result of the call > * @throws ClassCastException if the class of an element of this list > * is incompatible with the specified collection > * (optional) > * @throws NullPointerException if this list contains a null element > and the > * specified collection does not permit null elements > * (optional), > * or if the specified collection is null > * @see Collection#contains(Object) > */ > public boolean removeAll(Collection c) { > Objects.requireNonNull(c); > return batchRemove(c, false); > } > > Collection: > /** > * Removes all of this collection's elements that are also contained in > the > * specified collection (optional operation). After this call returns, > * this collection will contain no elements in common with the specified > * collection. > * > * @param c collection containing elements to be removed from this > collection > * @return true if this collection changed as a result of the > * call > * @throws UnsupportedOperationException if the removeAll > method > * is not supported by this collection > * @throws ClassCastException if the types of one or more elements > * in this collection are incompatible with the specified > * collection > * (optional) > * @throws NullPointerException if this collection contains one or more > * null elements and the specified collection does not support > * null elements > * (optional), > * or if the specified collection is null > * @see #remove(Object) > * @see #contains(Object) > */ > boolean removeAll(Collection c); From belliottsmith at datastax.com Tue Feb 4 17:47:27 2014 From: belliottsmith at datastax.com (Benedict Elliott Smith) Date: Tue, 4 Feb 2014 17:47:27 +0000 Subject: ArrayList.removeAll()/retainAll() In-Reply-To: <1BF8A83E-7998-457C-A1A2-91EBA769B3AC@oracle.com> References: <1BF8A83E-7998-457C-A1A2-91EBA769B3AC@oracle.com> Message-ID: Hi Mike, Thanks very much for the careful explanation, it's appreciated. It does look trivial to fix, so I'll see about raising a ticket over there. On 4 February 2014 17:35, Mike Duigou wrote: > The condition that is causing the problem is not a collection containing > null, which is allowed, but that the provided collection "c" is itself null. > > The problem was is a consequence of the following implementation: > > Iterator iterator = iterator(); > > while(iterator.hasNext()) { > if(c.contains(iterator.next()) { > iterator.remove(); > } > } > > This implementation had an inconsistent behaviour. If a collection > contains elements then the first iteration of the while loops will cause a > NullPointerException if "c" is null. If, however, the collection is empty > then c is never referenced and no NPE is thrown. The change in java 8 which > adds an explicit Objects.requireNonNull(c) ensures the behaviour is > consistent whether the collection is empty or not. Passing null as the > parameter for removeAll and retainAll has never been valid. What's changed > is that it's now consistently checked. > > It's unfortunate that this particular fix tickled a bug in Antlr 3. It > appears that it's simply a bug that Antlr doesn't check the result of it's > method which returns null. ( > https://github.com/antlr/antlr3/blob/master/tool/src/main/java/org/antlr/tool/CompositeGrammar.java#L226) > The fix would seem to be relatively straightforward to either have > getDirectDelegates() return Collections.emptyList() or add a check > for null. My personal preference would be for the former as it allows for > uniform handling of the result wherever the method is called without having > to remember that the result might be null. > > Mike > > On Feb 4 2014, at 09:00 , Benedict Elliott Smith < > belliottsmith at datastax.com> wrote: > > > Hi, > > > > I notice this (or a related issue) has been mentioned > > before< > http://mail.openjdk.java.net/pipermail/core-libs-dev/2013-June/017663.html > >on > > this list, but I'm not convinced the correct resolution was reached. > > We > > are seeing this problem thrown by antlr, but rather than a bug in antlr, > as > > surmised on the previous exchange, it looks to me that ArrayList is > > imposing a new constraint that is neither declared by itself nor > > Collection, and is unnecessary. ArrayList happily supports null elements, > > so requiring that the provided collection has no null elements is surely > a > > bug? > > > > I've pasted the two declarations below for ease of reference. Neither > > javadocs describe the constraint that is imposed. > > > > ArrayList: > > /** > > * Removes from this list all of its elements that are contained in > the > > * specified collection. > > * > > * @param c collection containing elements to be removed from this > list > > * @return {@code true} if this list changed as a result of the call > > * @throws ClassCastException if the class of an element of this list > > * is incompatible with the specified collection > > * (optional) > > * @throws NullPointerException if this list contains a null element > > and the > > * specified collection does not permit null elements > > * (optional), > > * or if the specified collection is null > > * @see Collection#contains(Object) > > */ > > public boolean removeAll(Collection c) { > > Objects.requireNonNull(c); > > return batchRemove(c, false); > > } > > > > Collection: > > /** > > * Removes all of this collection's elements that are also contained > in > > the > > * specified collection (optional operation). After this call > returns, > > * this collection will contain no elements in common with the > specified > > * collection. > > * > > * @param c collection containing elements to be removed from this > > collection > > * @return true if this collection changed as a result of the > > * call > > * @throws UnsupportedOperationException if the removeAll > > method > > * is not supported by this collection > > * @throws ClassCastException if the types of one or more elements > > * in this collection are incompatible with the specified > > * collection > > * (optional) > > * @throws NullPointerException if this collection contains one or > more > > * null elements and the specified collection does not support > > * null elements > > * (optional), > > * or if the specified collection is null > > * @see #remove(Object) > > * @see #contains(Object) > > */ > > boolean removeAll(Collection c); > > From roger.riggs at oracle.com Tue Feb 4 18:11:27 2014 From: roger.riggs at oracle.com (roger riggs) Date: Tue, 04 Feb 2014 13:11:27 -0500 Subject: RFR: Checking for JNI Exceptions in jni_util.c In-Reply-To: <52F10371.8010306@oracle.com> References: <52F0FA55.9000301@oracle.com> <52F10371.8010306@oracle.com> Message-ID: <52F12D4F.6000801@oracle.com> Hi Alan, Thanks for the review and suggestions; the webrev has been updated with the recommendations. Webrev: http://cr.openjdk.java.net/~rriggs/webrev-check_exception-8030993/ Thanks, Roger On 2/4/2014 10:12 AM, Alan Bateman wrote: > On 04/02/2014 14:33, roger riggs wrote: >> Please review these cleanups to check for JNI pending exceptions in >> jni_util.c. >> >> Webrev: >> http://cr.openjdk.java.net/~rriggs/webrev-check_exception-8030993/ > This mostly looks okay, I just wonder about the jclass declarations at > L686 and 690 as I assume these are declaration-after-use warnings with > some compilers. > > Also there is JNU_ClassString use at L760 that I assume should be > replaced with strClazz. > > -Alan. From daniel.fuchs at oracle.com Tue Feb 4 18:18:10 2014 From: daniel.fuchs at oracle.com (Daniel Fuchs) Date: Tue, 04 Feb 2014 19:18:10 +0100 Subject: RFR - 8033542: test/java/util/logging/SimpleLogManager might let Loggers be gc'ed too early. Message-ID: <52F12EE2.5050004@oracle.com> Hi, Please find below a fix [1] for: 8033542: test/java/util/logging/SimpleLogManager might let Loggers be gc'ed too early. [2] [1] http://cr.openjdk.java.net/~dfuchs/webrev_8033542/webrev.00 [2] https://bugs.openjdk.java.net/browse/JDK-8033542 The issue is that some of the loggers created by this test may not be strong referenced, which may cause the test to fail. -- daniel From Alan.Bateman at oracle.com Tue Feb 4 20:36:49 2014 From: Alan.Bateman at oracle.com (Alan Bateman) Date: Tue, 04 Feb 2014 20:36:49 +0000 Subject: RFR: Checking for JNI Exceptions in jni_util.c In-Reply-To: <52F12D4F.6000801@oracle.com> References: <52F0FA55.9000301@oracle.com> <52F10371.8010306@oracle.com> <52F12D4F.6000801@oracle.com> Message-ID: <52F14F61.3040300@oracle.com> On 04/02/2014 18:11, roger riggs wrote: > Hi Alan, > > Thanks for the review and suggestions; the webrev has been updated > with the recommendations. > > Webrev: > http://cr.openjdk.java.net/~rriggs/webrev-check_exception-8030993/ The updated webrev looks okay. I note in the JNU_ClassXXX functions that the return from NewGlobalRef is not checked but I believe it's safe to call DeleteLocalRef even if there is a pending exception. One minor comment is that in initializeEncoding it now assumes that the reference returned by JNU_ClassString is a global ref (because it obtains the reference before expanding the space for local refs). An alternative (which might be clearer for future readers) is to do the EnsureLocalCapacity first. -Alan From roger.riggs at oracle.com Tue Feb 4 21:05:19 2014 From: roger.riggs at oracle.com (roger riggs) Date: Tue, 04 Feb 2014 16:05:19 -0500 Subject: RFR: Checking for JNI Exceptions in jni_util.c In-Reply-To: <52F14F61.3040300@oracle.com> References: <52F0FA55.9000301@oracle.com> <52F10371.8010306@oracle.com> <52F12D4F.6000801@oracle.com> <52F14F61.3040300@oracle.com> Message-ID: <52F1560F.9040003@oracle.com> Hi, Updated the webrev with the recommendations. On 2/4/2014 3:36 PM, Alan Bateman wrote: > On 04/02/2014 18:11, roger riggs wrote: >> Hi Alan, >> >> Thanks for the review and suggestions; the webrev has been updated >> with the recommendations. >> >> Webrev: >> http://cr.openjdk.java.net/~rriggs/webrev-check_exception-8030993/ > The updated webrev looks okay. I note in the JNU_ClassXXX functions > that the return from NewGlobalRef is not checked but I believe it's > safe to call DeleteLocalRef even if there is a pending exception. > > One minor comment is that in initializeEncoding it now assumes that > the reference returned by JNU_ClassString is a global ref (because it > obtains the reference before expanding the space for local refs). An > alternative (which might be clearer for future readers) is to do the > EnsureLocalCapacity first. Corrected; but I note in reading the jni specification for EnsureLocalCapacity that the VM ensures that there is capacity for 16 local references before calling the native method so checking for smaller numbers is a noop. Roger > > -Alan From mandy.chung at oracle.com Wed Feb 5 03:25:46 2014 From: mandy.chung at oracle.com (Mandy Chung) Date: Tue, 04 Feb 2014 19:25:46 -0800 Subject: RFR - 8033542: test/java/util/logging/SimpleLogManager might let Loggers be gc'ed too early. In-Reply-To: <52F12EE2.5050004@oracle.com> References: <52F12EE2.5050004@oracle.com> Message-ID: <52F1AF3A.4010903@oracle.com> Thumbs up. Mandy On 2/4/2014 10:18 AM, Daniel Fuchs wrote: > Hi, > > Please find below a fix [1] for: > > 8033542: test/java/util/logging/SimpleLogManager might let > Loggers be gc'ed too early. [2] > > [1] http://cr.openjdk.java.net/~dfuchs/webrev_8033542/webrev.00 > [2] https://bugs.openjdk.java.net/browse/JDK-8033542 > > The issue is that some of the loggers created by this test > may not be strong referenced, which may cause the test to fail. > > -- daniel From peter.levart at gmail.com Wed Feb 5 08:08:20 2014 From: peter.levart at gmail.com (Peter Levart) Date: Wed, 05 Feb 2014 09:08:20 +0100 Subject: RFR - 6857566 (bf) DirectByteBuffer garbage creation can outpace reclamation Message-ID: <52F1F174.8070105@gmail.com> Hi, I'm proposing a patch for the following issue: https://bugs.openjdk.java.net/browse/JDK-6857566 The patch is basically the same as developed and tested as part of the following discussion 3 months ago: http://mail.openjdk.java.net/pipermail/core-libs-dev/2013-October/021547.html It's just modified to be applied on top of JDK9 which already changed since then. Here's the webrev: http://cr.openjdk.java.net/~plevart/jdk9-dev/DirectBufferAlloc/webrev.01/ As Alan Bateman pointed out: "A related piece of work is the FileChannel map implementation where there is a gc + retry if mmap fails. This could be changed to have a similar back-off/retry." I'm planning to examine this code and come up with a similar patch for mapped memory. Regards, Peter From belliottsmith at datastax.com Wed Feb 5 09:02:39 2014 From: belliottsmith at datastax.com (Benedict Elliott Smith) Date: Wed, 5 Feb 2014 09:02:39 +0000 Subject: RFR - 6857566 (bf) DirectByteBuffer garbage creation can outpace reclamation In-Reply-To: <52F1F174.8070105@gmail.com> References: <52F1F174.8070105@gmail.com> Message-ID: Hi, On the topic of DirectByteBuffer, I encountered a slightly less extreme - but important if there are many floating around - memory issue that would be nice to clean up: When you duplicate() or slice() a DBB the att field is always set to the buffer you duplicated/sliced from. In some cases (e.g. a duplicate() followed by a slice()) this means you keep DBBs around that exist only in a chain linking to the original buffer. It seems that it would be preferable to directly reference the parent's att if it is not null, and the parent itself otherwise, as the memory cost mounts up if you create a lot of small slices from a larger buffer. On 5 February 2014 08:08, Peter Levart wrote: > Hi, > > I'm proposing a patch for the following issue: > > https://bugs.openjdk.java.net/browse/JDK-6857566 > > The patch is basically the same as developed and tested as part of the > following discussion 3 months ago: > > http://mail.openjdk.java.net/pipermail/core-libs-dev/2013- > October/021547.html > > It's just modified to be applied on top of JDK9 which already changed > since then. Here's the webrev: > > http://cr.openjdk.java.net/~plevart/jdk9-dev/DirectBufferAlloc/webrev.01/ > > > As Alan Bateman pointed out: "A related piece of work is the FileChannel > map implementation where there is a gc + retry if mmap fails. This could be > changed to have a similar back-off/retry." > > I'm planning to examine this code and come up with a similar patch for > mapped memory. > > Regards, Peter > > From Alan.Bateman at oracle.com Wed Feb 5 10:07:51 2014 From: Alan.Bateman at oracle.com (Alan Bateman) Date: Wed, 05 Feb 2014 10:07:51 +0000 Subject: RFR: Checking for JNI Exceptions in jni_util.c In-Reply-To: <52F1560F.9040003@oracle.com> References: <52F0FA55.9000301@oracle.com> <52F10371.8010306@oracle.com> <52F12D4F.6000801@oracle.com> <52F14F61.3040300@oracle.com> <52F1560F.9040003@oracle.com> Message-ID: <52F20D77.9030905@oracle.com> On 04/02/2014 21:05, roger riggs wrote: > Corrected; but I note in reading the jni specification for > EnsureLocalCapacity > that the VM ensures that there is capacity for 16 local references before > calling the native method so checking for smaller numbers is a noop. Yes, and my comment was just to point out that it just looked a bit odd to get a reference before EnsureLocalCapacity. I've looked through the latest webrev and it looks good to me. -Alan. From Alan.Bateman at oracle.com Wed Feb 5 10:21:08 2014 From: Alan.Bateman at oracle.com (Alan Bateman) Date: Wed, 05 Feb 2014 10:21:08 +0000 Subject: RFR - 6857566 (bf) DirectByteBuffer garbage creation can outpace reclamation In-Reply-To: <52F1F174.8070105@gmail.com> References: <52F1F174.8070105@gmail.com> Message-ID: <52F21094.3090400@oracle.com> On 05/02/2014 08:08, Peter Levart wrote: > Hi, > > I'm proposing a patch for the following issue: > > https://bugs.openjdk.java.net/browse/JDK-6857566 > > The patch is basically the same as developed and tested as part of the > following discussion 3 months ago: > > http://mail.openjdk.java.net/pipermail/core-libs-dev/2013-October/021547.html > > > It's just modified to be applied on top of JDK9 which already changed > since then. Here's the webrev: > > http://cr.openjdk.java.net/~plevart/jdk9-dev/DirectBufferAlloc/webrev.01/ Thanks for re-basing the changes and from the previous discussion then I think there was agreement on the approach. So I'd say this is ready to go into jdk9/dev. A minor comments on the usages of tryHandlePendingReferences in Bits that it might be a bit neater to not split the line. At least when looking at the patch then it makes it a bit more obvious what is condition vs. statement in the while loop. Another minor comment is that usually avoid System.exit in tests. It's probably okay here as it always runs in othervm mode but in other modes a System.exit(0) will be flagged by jtreg and will require it spin up another VM. I completely agree with separating the mapped buffers issue to a separate issue. Thanks again for running with this one (it goes back to original implementation in JDK 1.4). -Alan. From chris.hegarty at oracle.com Wed Feb 5 15:11:16 2014 From: chris.hegarty at oracle.com (Chris Hegarty) Date: Wed, 5 Feb 2014 15:11:16 +0000 Subject: ObjectIn/OutputStream improvements In-Reply-To: <9A31C421-CBDC-4196-A74C-0CB5E2447CB6@oracle.com> References: <91644C57-6E0D-48ED-8E31-6AC217DECCEF@gmx.de>, <1C96B28E-5E20-4D8D-B1FD-8FD5DF193EC8@oracle.com> <52EBB1B8.2030401@oracle.com> <9A31C421-CBDC-4196-A74C-0CB5E2447CB6@oracle.com> Message-ID: <264411B6-B1E3-4BCA-BC64-42673D5C6733@oracle.com> Thanks stuart, Mike, and Paul. > - Why not have getClassSignature() return an interned string? (that's if interning is actually essential. Are we sure it's not just overhead?) I didn?t want to change the existing use of interning here, just refactor the code a little to make it cleaner. > I think that would be a better way to spend our performance investigation budget. Thus, it might be better to remove the StringBuilder field from this patch and submit another one focusing on UTF decoding; the other changes in the patch look good. Agreed. This could be looked at separately. Latest webrev: http://cr.openjdk.java.net/~chegar/serial_stupp.01/ Thanks, -Chris. From peter.levart at gmail.com Wed Feb 5 16:39:58 2014 From: peter.levart at gmail.com (Peter Levart) Date: Wed, 05 Feb 2014 17:39:58 +0100 Subject: ObjectIn/OutputStream improvements In-Reply-To: <264411B6-B1E3-4BCA-BC64-42673D5C6733@oracle.com> References: <91644C57-6E0D-48ED-8E31-6AC217DECCEF@gmx.de>, <1C96B28E-5E20-4D8D-B1FD-8FD5DF193EC8@oracle.com> <52EBB1B8.2030401@oracle.com> <9A31C421-CBDC-4196-A74C-0CB5E2447CB6@oracle.com> <264411B6-B1E3-4BCA-BC64-42673D5C6733@oracle.com> Message-ID: <52F2695E.20204@gmail.com> On 02/05/2014 04:11 PM, Chris Hegarty wrote: > Thanks stuart, Mike, and Paul. > >> - Why not have getClassSignature() return an interned string? (that's if interning is actually essential. Are we sure it's not just overhead?) > I didn?t want to change the existing use of interning here, just refactor the code a little to make it cleaner. > >> I think that would be a better way to spend our performance investigation budget. Thus, it might be better to remove the StringBuilder field from this patch and submit another one focusing on UTF decoding; the other changes in the patch look good. > Agreed. This could be looked at separately. > > Latest webrev: > http://cr.openjdk.java.net/~chegar/serial_stupp.01/ > > Thanks, > -Chris. Hi Chris, What about the following even less garbage-producing-and-copying ObjectStreamClass.get[Class|Method]Signature combo: /** * Returns JVM type signature for given class. */ static String getClassSignature(Class cl) { if (cl.isPrimitive()) return getPrimitiveSignature(cl); else return appendClassSignature(new StringBuilder(), cl).toString(); } private static StringBuilder appendClassSignature(StringBuilder sbuf, Class cl) { while (cl.isArray()) { sbuf.append('['); cl = cl.getComponentType(); } if (cl.isPrimitive()) sbuf.append(getPrimitiveSignature(cl)); else sbuf.append('L').append(cl.getName().replace('.', '/')).append(';'); return sbuf; } /** * Returns JVM type signature for given list of parameters and return type. */ private static String getMethodSignature(Class[] paramTypes, Class retType) { StringBuilder sbuf = new StringBuilder(); sbuf.append('('); for (int i = 0; i < paramTypes.length; i++) { appendClassSignature(sbuf, paramTypes[i]); } sbuf.append(')'); appendClassSignature(sbuf, retType); return sbuf.toString(); } Regards, Peter From xueming.shen at oracle.com Wed Feb 5 17:58:51 2014 From: xueming.shen at oracle.com (Xueming Shen) Date: Wed, 05 Feb 2014 09:58:51 -0800 Subject: RFR: JDK-8032012, , String.toLowerCase/toUpperCase performance improvement In-Reply-To: <3ACAF47E-0B56-4D17-B322-506EAB79A168@oracle.com> References: <52D82013.7020009@oracle.com> <52DEEF45.7060506@oracle.com> <3ACAF47E-0B56-4D17-B322-506EAB79A168@oracle.com> Message-ID: <52F27BDB.6040705@oracle.com> Hi, Let's try to wrap it up, otherwise I may drop the ball somewhere :-) On 01/22/2014 07:20 AM, Paul Sandoz wrote: > > >> > if (lang == "tr" || lang == "az" || lang == "lt") { > // local dependent > return toLowerCaseEx(result, firstUpper, locale, true); > } > // otherwise false is passed to subsequent calls to toLowerCaseEx > > ? > toLowerCaseEx will also be invoked later (in your another suggestion next), which needs a "localeDependent". >>> - you might be able to optimize by doing (could depend on the answer to the next point): >>> >>> int c = (int)value[i]; >>> int lc = Character.toLowerCase(c); >>> if (.....) { result[i] = (char)lc; } else { return toLowerCaseEx(result, i, locale, localeDependent); } >>> >>> - Do you need to check ERROR for the result of toLowerCase? >>> >>> 2586 if (c == Character.ERROR || >>> >> Yes, Character.toLowerCase() should never return ERROR (while the package private >> Character.toUpperCaseEx() will). In theory there is no need to check if the return >> value of Character.toUpperCase(int)> min_supplementary_code_point in our loop, >> because there is no bmp character returns a supplementary code point as its lower >> case. But since it's a data driven mapping table, there is no guarantee the unicode >> data table is not going to change in the "future", so I still keep the check. The webrev >> has been updated to use one single "if" inside the loop. >> >> I have a "single if" implementation for the toUpperCase() as well, though don't sure >> which one is better/faster :-) >> > OK, i suppose one could measure :-) > > Not sure how much it is worth obsessing over but... > > int c = (int)value[i]; > if (c< '\u03A3') { > result[i] = (char)Character.toLowerCase(c); // Is that safe? > } else if (c< Character.MIN_HIGH_SURROGATE&& c != 'u03A3'&& (c = Character.toLowerCase(c))< Character.MIN_SUPPLEMENTARY_CODE_POINT)) { > result[i] = (char)c; > } else { > return toLowerCaseEx(result, i, locale, localeDependent); > } > > or: > > int c = (int)value[i]; > int lc = Character.toLowerCase(c); // is that safe? > if (c< '\u03A3') { > result[i] = (char)lc; > } else if (c< Character.MIN_HIGH_SURROGATE&& c != 'u03A3'&& lc< Character.MIN_SUPPLEMENTARY_CODE_POINT)) { > result[i] = (char)lc; > } else { > return toLowerCaseEx(result, i, locale, localeDependent); > } > > or: > > int c = (int)value[i]; > int lc = Character.toLowerCase(c); // is that safe? > if (c< '\u03A3' || (c< Character.MIN_HIGH_SURROGATE&& c != 'u03A3'&& lc< Character.MIN_SUPPLEMENTARY_CODE_POINT))) { > result[i] = (char)lc; > } else { > return toLowerCaseEx(result, i, locale, localeDependent); > } > > FWIW i personally find those solutions easier to read, if they are safe w.r.t. Character.toLowerCase and that annoying greek character. The existing code has the clear and explicit logic that (1) j.l.C.toLowerCase() can NOT handle surrogate(s) (2) j.l.C.toLowerCase() can NOT handle \u03A3 (3) if for some reasons j.l.C.toLowerCase() returns a supplementary character for a non-bmp character, the current "one-char" based result can NOT handle it, So in above cases, go to "ex" version. The suggested approach is kinda of hacky and the logic is not clear. For example it tries to optimize with c < \u03a3 cases, it works with the assumption that all lower cases of < \u03a3 characters are never a supplementary characters, which is true for now and probably will be true forever, but it actually changes the design and implementation logic (be driven by the backend unicode case mapping data, not an "assumption"). The (c< Character.MIN_HIGH_SURROGATE&& c != 'u03A3'&& lc< Character.MIN_SUPPLEMENTARY_CODE_POINT) part appears to be fine, but its logic is not as explicit as the existing one. Given the measurement indicates it does not bring in measurable improvement, personally I would prefer to keep the existing one, if you don't have a very strong opinion on this. http://cr.openjdk.java.net/~sherman/8032012/ Thanks! -Sherman From paul.sandoz at oracle.com Wed Feb 5 19:09:08 2014 From: paul.sandoz at oracle.com (Paul Sandoz) Date: Wed, 5 Feb 2014 20:09:08 +0100 Subject: RFR: JDK-8032012, , String.toLowerCase/toUpperCase performance improvement In-Reply-To: <52F27BDB.6040705@oracle.com> References: <52D82013.7020009@oracle.com> <52DEEF45.7060506@oracle.com> <3ACAF47E-0B56-4D17-B322-506EAB79A168@oracle.com> <52F27BDB.6040705@oracle.com> Message-ID: On Feb 5, 2014, at 6:58 PM, Xueming Shen wrote: > Hi, > > Let's try to wrap it up, otherwise I may drop the ball somewhere :-) > > On 01/22/2014 07:20 AM, Paul Sandoz wrote: >> >> >>> >> if (lang == "tr" || lang == "az" || lang == "lt") { >> // local dependent >> return toLowerCaseEx(result, firstUpper, locale, true); >> } >> // otherwise false is passed to subsequent calls to toLowerCaseEx >> >> ? >> > > toLowerCaseEx will also be invoked later (in your another suggestion next), which > needs a "localeDependent". > But is not the second call to toLowerCaseEx always invoked with a value of false? 2574 String lang = locale.getLanguage(); 2575 final boolean localeDependent = (lang == "tr" || lang == "az" || lang == "lt"); 2576 if (localeDependent) { 2577 return toLowerCaseEx(result, firstUpper, locale, localeDependent); // <-- localeDependent is true 2578 } 2579 for (int i = firstUpper; i < len; i++) { 2580 int c = (int)value[i]; 2581 if (c >= Character.MIN_HIGH_SURROGATE && c <= Character.MAX_HIGH_SURROGATE || 2582 c == '\u03A3' || // GREEK CAPITAL LETTER SIGMA 2583 (c = Character.toLowerCase(c)) >= Character.MIN_SUPPLEMENTARY_CODE_POINT) { 2584 return toLowerCaseEx(result, i, locale, localeDependent); // <-- localDependent is false 2585 } 2586 result[i] = (char)c; 2587 } 2588 return new String(result, true); > Given the measurement indicates it does not bring in measurable > improvement, personally I would prefer to keep the existing one, > if you don't have a very strong opinion on this. > No i don't have a strong opinion. Paul. > http://cr.openjdk.java.net/~sherman/8032012/ > > Thanks! > -Sherman > > From xueming.shen at oracle.com Wed Feb 5 18:43:24 2014 From: xueming.shen at oracle.com (Xueming Shen) Date: Wed, 05 Feb 2014 10:43:24 -0800 Subject: RFR: JDK-8032012, , String.toLowerCase/toUpperCase performance improvement In-Reply-To: References: <52D82013.7020009@oracle.com> <52DEEF45.7060506@oracle.com> <3ACAF47E-0B56-4D17-B322-506EAB79A168@oracle.com> <52F27BDB.6040705@oracle.com> Message-ID: <52F2864C.1030508@oracle.com> On 02/05/2014 11:09 AM, Paul Sandoz wrote: > On Feb 5, 2014, at 6:58 PM, Xueming Shen wrote: > >> Hi, >> >> Let's try to wrap it up, otherwise I may drop the ball somewhere :-) >> >> On 01/22/2014 07:20 AM, Paul Sandoz wrote: >>> >>> if (lang == "tr" || lang == "az" || lang == "lt") { >>> // local dependent >>> return toLowerCaseEx(result, firstUpper, locale, true); >>> } >>> // otherwise false is passed to subsequent calls to toLowerCaseEx >>> >>> ? >>> >> toLowerCaseEx will also be invoked later (in your another suggestion next), which >> needs a "localeDependent". >> > But is not the second call to toLowerCaseEx always invoked with a value of false? > > 2574 String lang = locale.getLanguage(); > 2575 final boolean localeDependent = (lang == "tr" || lang == "az" || lang == "lt"); > 2576 if (localeDependent) { > 2577 return toLowerCaseEx(result, firstUpper, locale, localeDependent); //<-- localeDependent is true > 2578 } > 2579 for (int i = firstUpper; i< len; i++) { > 2580 int c = (int)value[i]; > 2581 if (c>= Character.MIN_HIGH_SURROGATE&& c<= Character.MAX_HIGH_SURROGATE || > 2582 c == '\u03A3' || // GREEK CAPITAL LETTER SIGMA > 2583 (c = Character.toLowerCase(c))>= Character.MIN_SUPPLEMENTARY_CODE_POINT) { > 2584 return toLowerCaseEx(result, i, locale, localeDependent); //<-- localDependent is false > 2585 } > 2586 result[i] = (char)c; > 2587 } > 2588 return new String(result, true); > > You are absolutely right :-) I will update as suggested. -sherman From martinrb at google.com Wed Feb 5 19:39:40 2014 From: martinrb at google.com (Martin Buchholz) Date: Wed, 5 Feb 2014 11:39:40 -0800 Subject: ObjectIn/OutputStream improvements In-Reply-To: References: <91644C57-6E0D-48ED-8E31-6AC217DECCEF@gmx.de> <1C96B28E-5E20-4D8D-B1FD-8FD5DF193EC8@oracle.com> <52EBB1B8.2030401@oracle.com> Message-ID: High-level thoughts: Although we all love garbage collection, performance sensitive code should still avoid unnecessary allocation. Escape analysis has not fulfilled its promise, and seems unlikely to in the future. If and when value types/tuples come to Java, we can create such objects with hopefully more confidence that they will be stack allocated in the common case. Memory management continues to be the biggest concern of people managing large Java applications. It continues to be true that Java applications need much more memory than "equivalent" C++ applications, and much more "memory tuning". OTOH, CPU usage parity is much less of an issue. --- I have done some (incomplete) work on making faster UTF-8 implementations, both inside and outside the JDK, some of which could be brought into the JDK proper and built upon. Increasingly, UTF-8 is the only important charset worth optimizing for. http://code.google.com/p/guava-libraries/source/browse/guava/src/com/google/common/base/Utf8.java http://code.google.com/p/protobuf/source/browse/trunk/java/src/main/java/com/google/protobuf/Utf8.java On Fri, Jan 31, 2014 at 10:46 AM, Chris Hegarty wrote: > Forwarding to correct core-libs-dev list. > > -Chris. > > On 31 Jan 2014, at 14:22, Chris Hegarty wrote: > > > Hi Robert, > > > > I think your patch can be split into two logical, independent, parts. > The first is the use of unsafe to access the String UTF length. The seconds > is to reuse, where possible, the existing StringBuilder instances, skipping > of primitive/object reading/writing where applicable, and general cleanup. > > > > Since this is a very sensitive area I would like to consider these > separately. To that end, I have taken the changes that are applicable to > the latter, removed any subjective stylistic changes, and made some > additional cleanup improvements. > > > > http://cr.openjdk.java.net/~chegar/serial_stupp.00/webrev/ > > > > Specifically, > > * I think for clarify and readability SerialCallbackContext > > checkAndSetUsed() should be invoked directly. It makes it very > > clear what the intent is. > > * Skip primitive/object reading/writing if no primitives/objects in > > the stream/class. ( I personally don't see any benefit of rounding > > up the size of the array, it just seems to add unnecessary > > complexity ) > > * Move primitive types into getPrimitiveSignature for better reuse > > of code. This retains your change to not create the additional > > StringBuilder when it is not necessary. > > > > I am currently running tests on this change. > > > > -Chris. > > > > On 07/01/14 13:03, Robert Stupp wrote: > >> Hi, > >> I've attached the diff to the original email - it has been stripped. > >> Here's a second try (inline). > >> I've already signed the OCA and it has been approved :) > >> Robert > >> # HG changeset patch > >> # User snazy > >> # Date 1387101091 -3600 > >> # Sun Dec 15 10:51:31 2013 +0100 > >> # Node ID 6d46d99212453017c88417678d08dc8f10da9606 > >> # Parent 9e1be800420265e5dcf264a7ed4abb6f230dd19d > >> Removed some unnecessary variable assignments. > >> ObjectInputStream: > >> - skip primitive/object reading if no primitive/objects in class > >> - use shared StringBuilder for string reading (prevent superfluous > >> object allocations) > >> ObjectOutputStream: > >> - skip primitive/object writing if no primitive/objects in class > >> - use unsafe access to calculate UTF-length > >> - use unsafe access in readBytes() and writeChars() methods to access > >> String value field > >> - removed cbuf field > >> ObjectStreamClass/ObjectStreamField: > >> - minor improvement in getClassSignature ; share method code with > >> ObjectStreamField > >> diff --git a/src/share/classes/java/io/ObjectInputStream.java > >> b/src/share/classes/java/io/ObjectInputStream.java > >> --- a/src/share/classes/java/io/ObjectInputStream.java > >> +++ b/src/share/classes/java/io/ObjectInputStream.java > >> @@ -39,8 +39,8 @@ > >> import java.util.HashMap; > >> import java.util.concurrent.ConcurrentHashMap; > >> import java.util.concurrent.ConcurrentMap; > >> -import java.util.concurrent.atomic.AtomicBoolean; > >> import static java.io.ObjectStreamClass.processQueue; > >> + > >> import sun.reflect.misc.ReflectUtil; > >> > >> /** > >> @@ -534,7 +534,7 @@ > >> if (ctx == null) { > >> throw new NotActiveException("not in call to readObject"); > >> } > >> - Object curObj = ctx.getObj(); > >> + ctx.getObj(); > >> ObjectStreamClass curDesc = ctx.getDesc(); > >> bin.setBlockDataMode(false); > >> GetFieldImpl getField = new GetFieldImpl(curDesc); > >> @@ -1597,7 +1597,7 @@ > >> int descHandle = handles.assign(unshared ? unsharedMarker : > desc); > >> passHandle = NULL_HANDLE; > >> > >> - ObjectStreamClass readDesc = null; > >> + ObjectStreamClass readDesc; > >> try { > >> readDesc = readClassDescriptor(); > >> } catch (ClassNotFoundException ex) { > >> @@ -1976,29 +1976,34 @@ > >> } > >> > >> int primDataSize = desc.getPrimDataSize(); > >> - if (primVals == null || primVals.length < primDataSize) { > >> - primVals = new byte[primDataSize]; > >> - } > >> - bin.readFully(primVals, 0, primDataSize, false); > >> - if (obj != null) { > >> - desc.setPrimFieldValues(obj, primVals); > >> - } > >> - > >> - int objHandle = passHandle; > >> - ObjectStreamField[] fields = desc.getFields(false); > >> - Object[] objVals = new Object[desc.getNumObjFields()]; > >> - int numPrimFields = fields.length - objVals.length; > >> - for (int i = 0; i < objVals.length; i++) { > >> - ObjectStreamField f = fields[numPrimFields + i]; > >> - objVals[i] = readObject0(f.isUnshared()); > >> - if (f.getField() != null) { > >> - handles.markDependency(objHandle, passHandle); > >> + if (primDataSize > 0) { > >> + if (primVals == null || primVals.length < primDataSize) { > >> + primVals = new byte[ ((primDataSize>>4)+1)<<4 ]; > >> + } > >> + bin.readFully(primVals, 0, primDataSize, false); > >> + if (obj != null) { > >> + desc.setPrimFieldValues(obj, primVals); > >> } > >> } > >> - if (obj != null) { > >> - desc.setObjFieldValues(obj, objVals); > >> + > >> + int numObjFields = desc.getNumObjFields(); > >> + if (numObjFields > 0) { > >> + int objHandle = passHandle; > >> + ObjectStreamField[] fields = desc.getFields(false); > >> + Object[] objVals = new Object[numObjFields]; > >> + int numPrimFields = fields.length - objVals.length; > >> + for (int i = 0; i < objVals.length; i++) { > >> + ObjectStreamField f = fields[numPrimFields + i]; > >> + objVals[i] = readObject0(f.isUnshared()); > >> + if (f.getField() != null) { > >> + handles.markDependency(objHandle, passHandle); > >> + } > >> + } > >> + if (obj != null) { > >> + desc.setObjFieldValues(obj, objVals); > >> + } > >> + passHandle = objHandle; > >> } > >> - passHandle = objHandle; > >> } > >> > >> /** > >> @@ -2377,8 +2382,10 @@ > >> private final byte[] buf = new byte[MAX_BLOCK_SIZE]; > >> /** buffer for reading block data headers */ > >> private final byte[] hbuf = new byte[MAX_HEADER_SIZE]; > >> - /** char buffer for fast string reads */ > >> + /** char buffer for fast string reads - used by {@link > >> #readUTFSpan(long)} */ > >> private final char[] cbuf = new char[CHAR_BUF_SIZE]; > >> + /** shared string builder for less object allocations - used by > >> {@link #readUTFBody(long)}, {@link #readUTFChar(long)} and {@link > >> #readUTFSpan(long)} */ > >> + private final StringBuilder sbuf = new > >> StringBuilder(CHAR_BUF_SIZE); > >> > >> /** block data mode */ > >> private boolean blkmode = false; > >> @@ -3044,19 +3051,19 @@ > >> * utflen bytes. > >> */ > >> private String readUTFBody(long utflen) throws IOException { > >> - StringBuilder sbuf = new StringBuilder(); > >> if (!blkmode) { > >> end = pos = 0; > >> } > >> > >> + sbuf.setLength(0); > >> while (utflen > 0) { > >> int avail = end - pos; > >> if (avail >= 3 || (long) avail == utflen) { > >> - utflen -= readUTFSpan(sbuf, utflen); > >> + utflen -= readUTFSpan(utflen); > >> } else { > >> if (blkmode) { > >> // near block boundary, read one byte at a time > >> - utflen -= readUTFChar(sbuf, utflen); > >> + utflen -= readUTFChar(utflen); > >> } else { > >> // shift and refill buffer manually > >> if (avail > 0) { > >> @@ -3076,10 +3083,10 @@ > >> * Reads span of UTF-encoded characters out of internal buffer > >> * (starting at offset pos and ending at or before offset end), > >> * consuming no more than utflen bytes. Appends read > >> characters to > >> - * sbuf. Returns the number of bytes consumed. > >> + * {@link #sbuf}. Returns the number of bytes consumed. > >> */ > >> - private long readUTFSpan(StringBuilder sbuf, long utflen) > >> - throws IOException > >> + private long readUTFSpan(long utflen) > >> + throws IOException > >> { > >> int cpos = 0; > >> int start = pos; > >> @@ -3111,19 +3118,19 @@ > >> throw new UTFDataFormatException(); > >> } > >> cbuf[cpos++] = (char) (((b1 & 0x1F) << 6) | > >> - ((b2 & 0x3F) << 0)); > >> + (b2 & 0x3F)); > >> break; > >> > >> case 14: // 3 byte format: 1110xxxx 10xxxxxx > >> 10xxxxxx > >> b3 = buf[pos + 1]; > >> - b2 = buf[pos + 0]; > >> + b2 = buf[pos ]; > >> pos += 2; > >> if ((b2 & 0xC0) != 0x80 || (b3 & 0xC0) != > >> 0x80) { > >> throw new UTFDataFormatException(); > >> } > >> cbuf[cpos++] = (char) (((b1 & 0x0F) << 12) > | > >> - ((b2 & 0x3F) << 6) | > >> - ((b3 & 0x3F) << 0)); > >> + ((b2 & 0x3F) << 6) | > >> + (b3 & 0x3F)); > >> break; > >> > >> default: // 10xx xxxx, 1111 xxxx > >> @@ -3150,12 +3157,12 @@ > >> > >> /** > >> * Reads in single UTF-encoded character one byte at a time, > >> appends > >> - * the character to sbuf, and returns the number of bytes > consumed. > >> + * the character to {@link #sbuf}, and returns the number of > >> bytes consumed. > >> * This method is used when reading in UTF strings written in > >> block > >> * data mode to handle UTF-encoded characters which > (potentially) > >> * straddle block-data boundaries. > >> */ > >> - private int readUTFChar(StringBuilder sbuf, long utflen) > >> + private int readUTFChar(long utflen) > >> throws IOException > >> { > >> int b1, b2, b3; > >> @@ -3181,8 +3188,7 @@ > >> if ((b2 & 0xC0) != 0x80) { > >> throw new UTFDataFormatException(); > >> } > >> - sbuf.append((char) (((b1 & 0x1F) << 6) | > >> - ((b2 & 0x3F) << 0))); > >> + sbuf.append((char) (((b1 & 0x1F) << 6) | (b2 & > 0x3F))); > >> return 2; > >> > >> case 14: // 3 byte format: 1110xxxx 10xxxxxx > 10xxxxxx > >> @@ -3198,8 +3204,8 @@ > >> throw new UTFDataFormatException(); > >> } > >> sbuf.append((char) (((b1 & 0x0F) << 12) | > >> - ((b2 & 0x3F) << 6) | > >> - ((b3 & 0x3F) << 0))); > >> + ((b2 & 0x3F) << 6) | > >> + (b3 & 0x3F))); > >> return 3; > >> > >> default: // 10xx xxxx, 1111 xxxx > >> diff --git a/src/share/classes/java/io/ObjectOutputStream.java > >> b/src/share/classes/java/io/ObjectOutputStream.java > >> --- a/src/share/classes/java/io/ObjectOutputStream.java > >> +++ b/src/share/classes/java/io/ObjectOutputStream.java > >> @@ -35,7 +35,8 @@ > >> import java.util.concurrent.ConcurrentHashMap; > >> import java.util.concurrent.ConcurrentMap; > >> import static java.io.ObjectStreamClass.processQueue; > >> -import java.io.SerialCallbackContext; > >> + > >> +import sun.misc.Unsafe; > >> import sun.reflect.misc.ReflectUtil; > >> > >> /** > >> @@ -458,7 +459,7 @@ > >> if (ctx == null) { > >> throw new NotActiveException("not in call to > >> writeObject"); > >> } > >> - Object curObj = ctx.getObj(); > >> + ctx.getObj(); > >> ObjectStreamClass curDesc = ctx.getDesc(); > >> curPut = new PutFieldImpl(curDesc); > >> } > >> @@ -1300,7 +1301,7 @@ > >> */ > >> private void writeString(String str, boolean unshared) throws > >> IOException { > >> handles.assign(unshared ? null : str); > >> - long utflen = bout.getUTFLength(str); > >> + long utflen = BlockDataOutputStream.getUTFLength(str); > >> if (utflen <= 0xFFFF) { > >> bout.writeByte(TC_STRING); > >> bout.writeUTF(str, utflen); > >> @@ -1527,29 +1528,34 @@ > >> desc.checkDefaultSerialize(); > >> > >> int primDataSize = desc.getPrimDataSize(); > >> - if (primVals == null || primVals.length < primDataSize) { > >> - primVals = new byte[primDataSize]; > >> + if (primDataSize > 0) { > >> + if (primVals == null || primVals.length < primDataSize) { > >> + primVals = new byte[ ((primDataSize>>4)+1)<<4 ]; > >> + } > >> + desc.getPrimFieldValues(obj, primVals); > >> + bout.write(primVals, 0, primDataSize, false); > >> } > >> - desc.getPrimFieldValues(obj, primVals); > >> - bout.write(primVals, 0, primDataSize, false); > >> > >> - ObjectStreamField[] fields = desc.getFields(false); > >> - Object[] objVals = new Object[desc.getNumObjFields()]; > >> - int numPrimFields = fields.length - objVals.length; > >> - desc.getObjFieldValues(obj, objVals); > >> - for (int i = 0; i < objVals.length; i++) { > >> - if (extendedDebugInfo) { > >> - debugInfoStack.push( > >> - "field (class \"" + desc.getName() + "\", name: > \"" + > >> - fields[numPrimFields + i].getName() + "\", type: > \"" + > >> - fields[numPrimFields + i].getType() + "\")"); > >> - } > >> - try { > >> - writeObject0(objVals[i], > >> - fields[numPrimFields + i].isUnshared()); > >> - } finally { > >> + int numObjFields = desc.getNumObjFields(); > >> + if (numObjFields > 0) { > >> + ObjectStreamField[] fields = desc.getFields(false); > >> + Object[] objVals = new Object[numObjFields]; > >> + int numPrimFields = fields.length - objVals.length; > >> + desc.getObjFieldValues(obj, objVals); > >> + for (int i = 0; i < objVals.length; i++) { > >> if (extendedDebugInfo) { > >> - debugInfoStack.pop(); > >> + debugInfoStack.push( > >> + "field (class \"" + desc.getName() + "\", name: > >> \"" + > >> + fields[numPrimFields + i].getName() + "\", > >> type: \"" + > >> + fields[numPrimFields + i].getType() + "\")"); > >> + } > >> + try { > >> + writeObject0(objVals[i], > >> + fields[numPrimFields + > i].isUnshared()); > >> + } finally { > >> + if (extendedDebugInfo) { > >> + debugInfoStack.pop(); > >> + } > >> } > >> } > >> } > >> @@ -1743,15 +1749,11 @@ > >> private static final int MAX_BLOCK_SIZE = 1024; > >> /** maximum data block header length */ > >> private static final int MAX_HEADER_SIZE = 5; > >> - /** (tunable) length of char buffer (for writing strings) */ > >> - private static final int CHAR_BUF_SIZE = 256; > >> > >> /** buffer for writing general/block data */ > >> private final byte[] buf = new byte[MAX_BLOCK_SIZE]; > >> /** buffer for writing block data headers */ > >> private final byte[] hbuf = new byte[MAX_HEADER_SIZE]; > >> - /** char buffer for fast string writes */ > >> - private final char[] cbuf = new char[CHAR_BUF_SIZE]; > >> > >> /** block data mode */ > >> private boolean blkmode = false; > >> @@ -1763,6 +1765,18 @@ > >> /** loopback stream (for data writes that span data blocks) */ > >> private final DataOutputStream dout; > >> > >> + /** use unsafe to directly access value field in > >> java.lang.String */ > >> + private static final Unsafe unsafe = Unsafe.getUnsafe(); > >> + /** use field offset to directly access value field in > >> java.lang.String */ > >> + private static final long stringValueOffset; > >> + static { > >> + try { > >> + stringValueOffset = > >> unsafe.objectFieldOffset(String.class.getDeclaredField("value")); > >> + } catch (NoSuchFieldException e) { > >> + throw new InternalError(e); > >> + } > >> + } > >> + > >> /** > >> * Creates new BlockDataOutputStream on top of given > >> underlying stream. > >> * Block data mode is turned off by default. > >> @@ -1972,35 +1986,23 @@ > >> } > >> > >> public void writeBytes(String s) throws IOException { > >> - int endoff = s.length(); > >> - int cpos = 0; > >> - int csize = 0; > >> + char[] sChars = (char[])unsafe.getObject(s, > stringValueOffset); > >> + int endoff = sChars.length; > >> for (int off = 0; off < endoff; ) { > >> - if (cpos >= csize) { > >> - cpos = 0; > >> - csize = Math.min(endoff - off, CHAR_BUF_SIZE); > >> - s.getChars(off, off + csize, cbuf, 0); > >> - } > >> if (pos >= MAX_BLOCK_SIZE) { > >> drain(); > >> } > >> - int n = Math.min(csize - cpos, MAX_BLOCK_SIZE - pos); > >> + int n = Math.min(endoff - off, MAX_BLOCK_SIZE - pos); > >> int stop = pos + n; > >> while (pos < stop) { > >> - buf[pos++] = (byte) cbuf[cpos++]; > >> + buf[pos++] = (byte) sChars[off++]; > >> } > >> - off += n; > >> } > >> } > >> > >> public void writeChars(String s) throws IOException { > >> - int endoff = s.length(); > >> - for (int off = 0; off < endoff; ) { > >> - int csize = Math.min(endoff - off, CHAR_BUF_SIZE); > >> - s.getChars(off, off + csize, cbuf, 0); > >> - writeChars(cbuf, 0, csize); > >> - off += csize; > >> - } > >> + char[] sChars = (char[])unsafe.getObject(s, > stringValueOffset); > >> + writeChars(sChars, 0, sChars.length); > >> } > >> > >> public void writeUTF(String s) throws IOException { > >> @@ -2130,25 +2132,21 @@ > >> } > >> > >> /** > >> - * Returns the length in bytes of the UTF encoding of the given > >> string. > >> + * Returns the length in bytes of the UTF encoding of this > >> given string. > >> */ > >> - long getUTFLength(String s) { > >> - int len = s.length(); > >> + static long getUTFLength(String s) { > >> + char[] value = (char[])unsafe.getObject(s, > stringValueOffset); > >> + int len = value.length; > >> long utflen = 0; > >> for (int off = 0; off < len; ) { > >> - int csize = Math.min(len - off, CHAR_BUF_SIZE); > >> - s.getChars(off, off + csize, cbuf, 0); > >> - for (int cpos = 0; cpos < csize; cpos++) { > >> - char c = cbuf[cpos]; > >> - if (c >= 0x0001 && c <= 0x007F) { > >> - utflen++; > >> - } else if (c > 0x07FF) { > >> - utflen += 3; > >> - } else { > >> - utflen += 2; > >> - } > >> + char c = value[off++]; > >> + if (c >= 0x0001 && c <= 0x007F) { > >> + utflen++; > >> + } else if (c > 0x07FF) { > >> + utflen += 3; > >> + } else { > >> + utflen += 2; > >> } > >> - off += csize; > >> } > >> return utflen; > >> } > >> @@ -2198,40 +2196,36 @@ > >> * 8-byte length header) of the UTF encoding for the given > string. > >> */ > >> private void writeUTFBody(String s) throws IOException { > >> + char[] sChars = (char[])unsafe.getObject(s, > stringValueOffset); > >> + int len = sChars.length; > >> int limit = MAX_BLOCK_SIZE - 3; > >> - int len = s.length(); > >> for (int off = 0; off < len; ) { > >> - int csize = Math.min(len - off, CHAR_BUF_SIZE); > >> - s.getChars(off, off + csize, cbuf, 0); > >> - for (int cpos = 0; cpos < csize; cpos++) { > >> - char c = cbuf[cpos]; > >> - if (pos <= limit) { > >> - if (c <= 0x007F && c != 0) { > >> - buf[pos++] = (byte) c; > >> - } else if (c > 0x07FF) { > >> - buf[pos + 2] = (byte) (0x80 | ((c >> 0) & > >> 0x3F)); > >> - buf[pos + 1] = (byte) (0x80 | ((c >> 6) & > >> 0x3F)); > >> - buf[pos + 0] = (byte) (0xE0 | ((c >> 12) & > >> 0x0F)); > >> - pos += 3; > >> - } else { > >> - buf[pos + 1] = (byte) (0x80 | ((c >> 0) & > >> 0x3F)); > >> - buf[pos + 0] = (byte) (0xC0 | ((c >> 6) & > >> 0x1F)); > >> - pos += 2; > >> - } > >> - } else { // write one byte at a time to > >> normalize block > >> - if (c <= 0x007F && c != 0) { > >> - write(c); > >> - } else if (c > 0x07FF) { > >> - write(0xE0 | ((c >> 12) & 0x0F)); > >> - write(0x80 | ((c >> 6) & 0x3F)); > >> - write(0x80 | ((c >> 0) & 0x3F)); > >> - } else { > >> - write(0xC0 | ((c >> 6) & 0x1F)); > >> - write(0x80 | ((c >> 0) & 0x3F)); > >> - } > >> + char c = sChars[off++]; > >> + if (pos <= limit) { > >> + if (c <= 0x007F && c != 0) { > >> + buf[pos++] = (byte) c; > >> + } else if (c > 0x07FF) { > >> + buf[pos + 2] = (byte) (0x80 | ( c & > 0x3F)); > >> + buf[pos + 1] = (byte) (0x80 | ((c >> 6) & > 0x3F)); > >> + buf[pos ] = (byte) (0xE0 | ((c >> 12) & > 0x0F)); > >> + pos += 3; > >> + } else { > >> + buf[pos + 1] = (byte) (0x80 | ( c & > 0x3F)); > >> + buf[pos ] = (byte) (0xC0 | ((c >> 6) & > 0x1F)); > >> + pos += 2; > >> + } > >> + } else { // write one byte at a time to normalize > block > >> + if (c <= 0x007F && c != 0) { > >> + write(c); > >> + } else if (c > 0x07FF) { > >> + write(0xE0 | ((c >> 12) & 0x0F)); > >> + write(0x80 | ((c >> 6) & 0x3F)); > >> + write(0x80 | ( c & 0x3F)); > >> + } else { > >> + write(0xC0 | ((c >> 6) & 0x1F)); > >> + write(0x80 | ( c & 0x3F)); > >> } > >> } > >> - off += csize; > >> } > >> } > >> } > >> @@ -2464,7 +2458,10 @@ > >> StringBuilder buffer = new StringBuilder(); > >> if (!stack.isEmpty()) { > >> for(int i = stack.size(); i > 0; i-- ) { > >> - buffer.append(stack.get(i-1) + ((i != 1) ? "\n" : > "")); > >> + buffer.append(stack.get(i - 1)); > >> + if (i!=1) { > >> + buffer.append('\n'); > >> + } > >> } > >> } > >> return buffer.toString(); > >> diff --git a/src/share/classes/java/io/ObjectStreamClass.java > >> b/src/share/classes/java/io/ObjectStreamClass.java > >> --- a/src/share/classes/java/io/ObjectStreamClass.java > >> +++ b/src/share/classes/java/io/ObjectStreamClass.java > >> @@ -1474,7 +1474,28 @@ > >> /** > >> * Returns JVM type signature for given class. > >> */ > >> - private static String getClassSignature(Class cl) { > >> + static String getClassSignature(Class cl) { > >> + if (cl.isPrimitive()) > >> + if (cl == Integer.TYPE) { > >> + return "I"; > >> + } else if (cl == Byte.TYPE) { > >> + return "B"; > >> + } else if (cl == Long.TYPE) { > >> + return "J"; > >> + } else if (cl == Float.TYPE) { > >> + return "F"; > >> + } else if (cl == Double.TYPE) { > >> + return "D"; > >> + } else if (cl == Short.TYPE) { > >> + return "S"; > >> + } else if (cl == Character.TYPE) { > >> + return "C"; > >> + } else if (cl == Boolean.TYPE) { > >> + return "Z"; > >> + } else if (cl == Void.TYPE) { > >> + return "V"; > >> + } > >> + > >> StringBuilder sbuf = new StringBuilder(); > >> while (cl.isArray()) { > >> sbuf.append('['); > >> @@ -1503,7 +1524,7 @@ > >> throw new InternalError(); > >> } > >> } else { > >> - sbuf.append('L' + cl.getName().replace('.', '/') + ';'); > >> + sbuf.append('L').append(cl.getName().replace('.', > >> '/')).append(';'); > >> } > >> return sbuf.toString(); > >> } > >> diff --git a/src/share/classes/java/io/ObjectStreamField.java > >> b/src/share/classes/java/io/ObjectStreamField.java > >> --- a/src/share/classes/java/io/ObjectStreamField.java > >> +++ b/src/share/classes/java/io/ObjectStreamField.java > >> @@ -91,7 +91,7 @@ > >> this.name = name; > >> this.type = type; > >> this.unshared = unshared; > >> - signature = getClassSignature(type).intern(); > >> + signature = ObjectStreamClass.getClassSignature(type).intern(); > >> field = null; > >> } > >> > >> @@ -137,7 +137,7 @@ > >> name = field.getName(); > >> Class ftype = field.getType(); > >> type = (showType || ftype.isPrimitive()) ? ftype : > Object.class; > >> - signature = getClassSignature(ftype).intern(); > >> + signature = > ObjectStreamClass.getClassSignature(ftype).intern(); > >> } > >> > >> /** > >> @@ -286,41 +286,4 @@ > >> String getSignature() { > >> return signature; > >> } > >> - > >> - /** > >> - * Returns JVM type signature for given class. > >> - */ > >> - private static String getClassSignature(Class cl) { > >> - StringBuilder sbuf = new StringBuilder(); > >> - while (cl.isArray()) { > >> - sbuf.append('['); > >> - cl = cl.getComponentType(); > >> - } > >> - if (cl.isPrimitive()) { > >> - if (cl == Integer.TYPE) { > >> - sbuf.append('I'); > >> - } else if (cl == Byte.TYPE) { > >> - sbuf.append('B'); > >> - } else if (cl == Long.TYPE) { > >> - sbuf.append('J'); > >> - } else if (cl == Float.TYPE) { > >> - sbuf.append('F'); > >> - } else if (cl == Double.TYPE) { > >> - sbuf.append('D'); > >> - } else if (cl == Short.TYPE) { > >> - sbuf.append('S'); > >> - } else if (cl == Character.TYPE) { > >> - sbuf.append('C'); > >> - } else if (cl == Boolean.TYPE) { > >> - sbuf.append('Z'); > >> - } else if (cl == Void.TYPE) { > >> - sbuf.append('V'); > >> - } else { > >> - throw new InternalError(); > >> - } > >> - } else { > >> - sbuf.append('L' + cl.getName().replace('.', '/') + ';'); > >> - } > >> - return sbuf.toString(); > >> - } > >> } > >> *Gesendet:* Dienstag, 07. Januar 2014 um 10:05 Uhr > >> *Von:* "Chris Hegarty" > >> *An:* "Robert Stupp" > >> *Cc:* "core-libs-dev at openjdk.java.net Libs" < > core-libs-dev at openjdk.java.net> > >> *Betreff:* Re: ObjectIn/OutputStream improvements > >> On 15 Dec 2013, at 10:29, Robert Stupp wrote: > >> > >> > Hi, > >> > > >> > I digged through the object serialization code and found some lines > >> that could be optimized to reduce the number of calls to > >> System.arraycopy() and temporary object allocations especially during > >> string (de)serialization. > >> > In short sentences the changes are: > >> > ObjectInputStream: > >> > - skip primitive/object reading if no primitive/objects in class > >> (defaultReadFields method) > >> > - use shared StringBuilder for string reading (prevent superfluous > >> object allocations of one StingBuilder and one implicit char[] for each > >> string being deserialized) > >> > ObjectOutputStream: > >> > - skip primitive/object writing if no primitive/objects in class > >> (defaultWriteFields method) > >> > - use unsafe access to calculate UTF-length > >> > - use unsafe access in readBytes() and writeChars() methods to access > >> String value field > >> > - removed cbuf field > >> > ObjectStreamClass/ObjectStreamField: > >> > - minor improvement in getClassSignature ; share method code with > >> ObjectStreamField (return constant string for primitives) > >> > > >> > I have tested the changes in a big Java installation at a customer > >> (backported the Java8 ObjectIn/OutputStream including the changes to > >> Java6) and a long running real application performance test resulted in > >> reduced CPU usage (from about 60% per server to 50%). > >> > The changes I made in openjdk8 pass all tests. > >> > > >> > Since I have no experience how to contribute code to openjdk in form > >> of a push/changeset I have added the diff (hg export -g) to this email. > >> > >> Did you attach the diffs? I don?t see any attachment, it may be that the > >> attachment was stripped. Can you try resending inline? > >> > >> You should take a look at the OpenJDK How to Contribute page [1]. Paying > >> particular attention to the OCA, without it we will not be able to move > >> your patch forward. > >> > >> Thanks, > >> -Chris. > >> > >> [1] http://openjdk.java.net/contribute/ > >> > >> > > >> > Robert > >> > > >> > > > From xueming.shen at oracle.com Wed Feb 5 20:06:35 2014 From: xueming.shen at oracle.com (Xueming Shen) Date: Wed, 05 Feb 2014 12:06:35 -0800 Subject: RFR: JDK-8032012, , String.toLowerCase/toUpperCase performance improvement In-Reply-To: <52F2864C.1030508@oracle.com> References: <52D82013.7020009@oracle.com> <52DEEF45.7060506@oracle.com> <3ACAF47E-0B56-4D17-B322-506EAB79A168@oracle.com> <52F27BDB.6040705@oracle.com> <52F2864C.1030508@oracle.com> Message-ID: <52F299CB.4020006@oracle.com> Hi Paul, http://cr.openjdk.java.net/~sherman/8032012/webrev webrev has been updated accordingly. Thanks! -Sherman On 02/05/2014 10:43 AM, Xueming Shen wrote: > On 02/05/2014 11:09 AM, Paul Sandoz wrote: >> On Feb 5, 2014, at 6:58 PM, Xueming Shen wrote: >> >>> Hi, >>> >>> Let's try to wrap it up, otherwise I may drop the ball somewhere :-) >>> >>> On 01/22/2014 07:20 AM, Paul Sandoz wrote: >>>> >>>> if (lang == "tr" || lang == "az" || lang == "lt") { >>>> // local dependent >>>> return toLowerCaseEx(result, firstUpper, locale, true); >>>> } >>>> // otherwise false is passed to subsequent calls to toLowerCaseEx >>>> >>>> ? >>>> >>> toLowerCaseEx will also be invoked later (in your another suggestion next), which >>> needs a "localeDependent". >>> >> But is not the second call to toLowerCaseEx always invoked with a value of false? >> >> 2574 String lang = locale.getLanguage(); >> 2575 final boolean localeDependent = (lang == "tr" || lang == "az" || lang == "lt"); >> 2576 if (localeDependent) { >> 2577 return toLowerCaseEx(result, firstUpper, locale, localeDependent); //<-- localeDependent is true >> 2578 } >> 2579 for (int i = firstUpper; i< len; i++) { >> 2580 int c = (int)value[i]; >> 2581 if (c>= Character.MIN_HIGH_SURROGATE&& c<= Character.MAX_HIGH_SURROGATE || >> 2582 c == '\u03A3' || // GREEK CAPITAL LETTER SIGMA >> 2583 (c = Character.toLowerCase(c))>= Character.MIN_SUPPLEMENTARY_CODE_POINT) { >> 2584 return toLowerCaseEx(result, i, locale, localeDependent); //<-- localDependent is false >> 2585 } >> 2586 result[i] = (char)c; >> 2587 } >> 2588 return new String(result, true); >> >> > > You are absolutely right :-) I will update as suggested. > > -sherman From forax at univ-mlv.fr Wed Feb 5 21:00:32 2014 From: forax at univ-mlv.fr (Remi Forax) Date: Wed, 05 Feb 2014 22:00:32 +0100 Subject: RFR: JDK-8032012, , String.toLowerCase/toUpperCase performance improvement In-Reply-To: <52F2864C.1030508@oracle.com> References: <52D82013.7020009@oracle.com> <52DEEF45.7060506@oracle.com> <3ACAF47E-0B56-4D17-B322-506EAB79A168@oracle.com> <52F27BDB.6040705@oracle.com> <52F2864C.1030508@oracle.com> Message-ID: <52F2A670.60806@univ-mlv.fr> On 02/05/2014 07:43 PM, Xueming Shen wrote: > On 02/05/2014 11:09 AM, Paul Sandoz wrote: >> On Feb 5, 2014, at 6:58 PM, Xueming Shen >> wrote: >> >>> Hi, >>> >>> Let's try to wrap it up, otherwise I may drop the ball somewhere :-) >>> >>> On 01/22/2014 07:20 AM, Paul Sandoz wrote: >>>> >>>> if (lang == "tr" || lang == "az" || lang == "lt") { >>>> // local dependent >>>> return toLowerCaseEx(result, firstUpper, locale, true); >>>> } >>>> // otherwise false is passed to subsequent calls to toLowerCaseEx >>>> >>>> ? >>>> >>> toLowerCaseEx will also be invoked later (in your another suggestion >>> next), which >>> needs a "localeDependent". >>> >> But is not the second call to toLowerCaseEx always invoked with a >> value of false? >> >> 2574 String lang = locale.getLanguage(); >> 2575 final boolean localeDependent = (lang == "tr" || lang == >> "az" || lang == "lt"); >> 2576 if (localeDependent) { >> 2577 return toLowerCaseEx(result, firstUpper, locale, >> localeDependent); //<-- localeDependent is true >> 2578 } >> 2579 for (int i = firstUpper; i< len; i++) { >> 2580 int c = (int)value[i]; >> 2581 if (c>= Character.MIN_HIGH_SURROGATE&& c<= >> Character.MAX_HIGH_SURROGATE || >> 2582 c == '\u03A3' || // GREEK CAPITAL LETTER SIGMA >> 2583 (c = Character.toLowerCase(c))>= >> Character.MIN_SUPPLEMENTARY_CODE_POINT) { >> 2584 return toLowerCaseEx(result, i, locale, >> localeDependent); //<-- localDependent is false >> 2585 } >> 2586 result[i] = (char)c; >> 2587 } >> 2588 return new String(result, true); >> >> > > You are absolutely right :-) I will update as suggested. > > -sherman Hi Sherman, the code can be faster if the first loop call toLowerCaseEx in case of a surrogate instead of modifying srcCount because in that case the JIT will see that the increment is constant. Note that with the current code, the performance of toLowerCase if it was never called with a string that contains a surrogate are great but if toLowerCase is called once with a string that contains a surrogate, even if toLowercase is called after with strings that never contain a surrogate performance will be slower than it was. Calling toLowerCaseEx will make a toLowerCase with a surrogate slower but make toLowerCase faster in the common case. cheers, R?mi From xueming.shen at oracle.com Wed Feb 5 21:30:37 2014 From: xueming.shen at oracle.com (Xueming Shen) Date: Wed, 05 Feb 2014 13:30:37 -0800 Subject: RFR: JDK-8032012, , String.toLowerCase/toUpperCase performance improvement In-Reply-To: <52F2A670.60806@univ-mlv.fr> References: <52D82013.7020009@oracle.com> <52DEEF45.7060506@oracle.com> <3ACAF47E-0B56-4D17-B322-506EAB79A168@oracle.com> <52F27BDB.6040705@oracle.com> <52F2864C.1030508@oracle.com> <52F2A670.60806@univ-mlv.fr> Message-ID: <52F2AD7D.1050609@oracle.com> Hi Remi, Good suggestion. Now the "common case" path is much simple and faster :-) I'm seeing a 5%-10% boost for the normal-non-surrogates case. And it appears the bmp+surr mixed is getting faster as well. Though I would assume the it would get slower in case of "no-case-folding" + surrogates. But the common case wins here. http://cr.openjdk.java.net/~sherman/8032012/webrev/ Thanks! -Sherman On 02/05/2014 01:00 PM, Remi Forax wrote: > > Hi Sherman, > the code can be faster if the first loop call toLowerCaseEx in case of a surrogate instead of modifying srcCount because in that case the JIT will see that the increment is constant. > > Note that with the current code, the performance of toLowerCase if it was never called with a string that contains a surrogate are great but if toLowerCase is called once with a string that contains a surrogate, even if toLowercase is called after with strings that never contain a surrogate performance will be slower than it was. Calling toLowerCaseEx will make a toLowerCase with a surrogate slower but make toLowerCase faster in the common case. > > cheers, > R?mi > > > > From forax at univ-mlv.fr Wed Feb 5 22:46:44 2014 From: forax at univ-mlv.fr (Remi Forax) Date: Wed, 05 Feb 2014 23:46:44 +0100 Subject: RFR: JDK-8032012, , String.toLowerCase/toUpperCase performance improvement In-Reply-To: <52F2AD7D.1050609@oracle.com> References: <52D82013.7020009@oracle.com> <52DEEF45.7060506@oracle.com> <3ACAF47E-0B56-4D17-B322-506EAB79A168@oracle.com> <52F27BDB.6040705@oracle.com> <52F2864C.1030508@oracle.com> <52F2A670.60806@univ-mlv.fr> <52F2AD7D.1050609@oracle.com> Message-ID: <52F2BF54.3000804@univ-mlv.fr> On 02/05/2014 10:30 PM, Xueming Shen wrote: > Hi Remi, > > Good suggestion. Now the "common case" path is much simple and faster :-) > I'm seeing a 5%-10% boost for the normal-non-surrogates case. And it > appears > the bmp+surr mixed is getting faster as well. Though I would assume > the it > would get slower in case of "no-case-folding" + surrogates. But the > common > case wins here. > > http://cr.openjdk.java.net/~sherman/8032012/webrev/ > > Thanks! > -Sherman nice, cool ! R?mi > > On 02/05/2014 01:00 PM, Remi Forax wrote: >> >> Hi Sherman, >> the code can be faster if the first loop call toLowerCaseEx in case >> of a surrogate instead of modifying srcCount because in that case the >> JIT will see that the increment is constant. >> >> Note that with the current code, the performance of toLowerCase if it >> was never called with a string that contains a surrogate are great >> but if toLowerCase is called once with a string that contains a >> surrogate, even if toLowercase is called after with strings that >> never contain a surrogate performance will be slower than it was. >> Calling toLowerCaseEx will make a toLowerCase with a surrogate slower >> but make toLowerCase faster in the common case. >> >> cheers, >> R?mi >> >> >> >> > From roger.riggs at oracle.com Wed Feb 5 22:58:38 2014 From: roger.riggs at oracle.com (roger riggs) Date: Wed, 05 Feb 2014 17:58:38 -0500 Subject: RFR: 8033565: Remove unused nativeNewStringPlatform and nativeGetStringPlatformChars Message-ID: <52F2C21E.8030800@oracle.com> Please review this change to remove unused support in jni_util.c for functions nativeNewStringPlatform and nativeGetStringPlatformChars. On Windows, both were conditional on jvm.dll being installed on a path containing "kernel". On Solaris, Linux, and Mac, they had no-op implementations. Webrev: http://cr.openjdk.java.net/~rriggs/webrev-remove-nativenewstring-8033565/ Thanks, Roger From Ulf.Zibis at CoSoCo.de Wed Feb 5 23:28:39 2014 From: Ulf.Zibis at CoSoCo.de (Ulf Zibis) Date: Thu, 06 Feb 2014 00:28:39 +0100 Subject: RFR: JDK-8032012, , String.toLowerCase/toUpperCase performance improvement In-Reply-To: <52F2AD7D.1050609@oracle.com> References: <52D82013.7020009@oracle.com> <52DEEF45.7060506@oracle.com> <3ACAF47E-0B56-4D17-B322-506EAB79A168@oracle.com> <52F27BDB.6040705@oracle.com> <52F2864C.1030508@oracle.com> <52F2A670.60806@univ-mlv.fr> <52F2AD7D.1050609@oracle.com> Message-ID: <52F2C927.3030609@CoSoCo.de> Additionally you could use Character.isSurrogate() and Character.isSupplementaryCode Hi Remi, > > Good suggestion. Now the "common case" path is much simple and faster :-) > I'm seeing a 5%-10% boost for the normal-non-surrogates case. And it appears > the bmp+surr mixed is getting faster as well. Though I would assume the it > would get slower in case of "no-case-folding" + surrogates. But the common > case wins here. > > http://cr.openjdk.java.net/~sherman/8032012/webrev/ From xueming.shen at oracle.com Wed Feb 5 23:57:01 2014 From: xueming.shen at oracle.com (Xueming Shen) Date: Wed, 05 Feb 2014 15:57:01 -0800 Subject: RFR: JDK-8032012, , String.toLowerCase/toUpperCase performance improvement In-Reply-To: <52F2C927.3030609@CoSoCo.de> References: <52D82013.7020009@oracle.com> <52DEEF45.7060506@oracle.com> <3ACAF47E-0B56-4D17-B322-506EAB79A168@oracle.com> <52F27BDB.6040705@oracle.com> <52F2864C.1030508@oracle.com> <52F2A670.60806@univ-mlv.fr> <52F2AD7D.1050609@oracle.com> <52F2C927.3030609@CoSoCo.de> Message-ID: <52F2CFCD.7050001@oracle.com> On 02/05/2014 03:28 PM, Ulf Zibis wrote: > Additionally you could use Character.isSurrogate() and Character.isSupplementaryCode > -Ulf > > > Am 05.02.2014 22:30, schrieb Xueming Shen: >> Hi Remi, >> >> Good suggestion. Now the "common case" path is much simple and faster :-) >> I'm seeing a 5%-10% boost for the normal-non-surrogates case. And it appears >> the bmp+surr mixed is getting faster as well. Though I would assume the it >> would get slower in case of "no-case-folding" + surrogates. But the common >> case wins here. >> >> http://cr.openjdk.java.net/~sherman/8032012/webrev/ > From mandy.chung at oracle.com Thu Feb 6 01:28:29 2014 From: mandy.chung at oracle.com (Mandy Chung) Date: Wed, 05 Feb 2014 17:28:29 -0800 Subject: RFR: 8033565: Remove unused nativeNewStringPlatform and nativeGetStringPlatformChars In-Reply-To: <52F2C21E.8030800@oracle.com> References: <52F2C21E.8030800@oracle.com> Message-ID: <52F2E53D.70702@oracle.com> On 2/5/2014 2:58 PM, roger riggs wrote: > Please review this change to remove unused support in jni_util.c for > functions nativeNewStringPlatform and nativeGetStringPlatformChars. > > On Windows, both were conditional on jvm.dll being installed on a path > containing "kernel". > On Solaris, Linux, and Mac, they had no-op implementations. > > Webrev: > http://cr.openjdk.java.net/~rriggs/webrev-remove-nativenewstring-8033565/ > Looks good. It's good that you noticed this piece of old and unused code that was added for Java Kernel. Thanks Mandy From vitalyd at gmail.com Thu Feb 6 02:21:40 2014 From: vitalyd at gmail.com (Vitaly Davidovich) Date: Wed, 5 Feb 2014 21:21:40 -0500 Subject: RFR: JDK-8032012, , String.toLowerCase/toUpperCase performance improvement In-Reply-To: References: <52D82013.7020009@oracle.com> <52DEEF45.7060506@oracle.com> <3ACAF47E-0B56-4D17-B322-506EAB79A168@oracle.com> <52F27BDB.6040705@oracle.com> <52F2864C.1030508@oracle.com> <52F2A670.60806@univ-mlv.fr> <52F2AD7D.1050609@oracle.com> <52F2C927.3030609@CoSoCo.de> <52F2CFCD.7050001@oracle.com> Message-ID: i2c conversion should not cost anything; it'll just make jit use low 16 bits of the registers for (unsigned) comparisons. I haven't checked this though, but that's what I'd expect. Sent from my phone On Feb 5, 2014 7:27 PM, "Xueming Shen" wrote: On 02/05/2014 03:28 PM, Ulf Zibis wrote: > Additionally you could use Character.isSurrogate() and > Character.isSupplementaryCode better optimized for JIT. > j.l.C.isSupplementaryCodePoint() checks up boundary of supp, we probably don't need it here, as the returning code point is either a ERROR or a valid unicode code point. I'm not sure about the j.l.C.isSurrogate(), which takes a char and we have an int here. I would expect the javac will inline the constants for me, but I don't know whether jit can inline and then optimize away the explicit casting i2c. Not a big deal though. -sherman > -Ulf > > > Am 05.02.2014 22:30, schrieb Xueming Shen: > >> Hi Remi, >> >> Good suggestion. Now the "common case" path is much simple and faster :-) >> I'm seeing a 5%-10% boost for the normal-non-surrogates case. And it >> appears >> the bmp+surr mixed is getting faster as well. Though I would assume the it >> would get slower in case of "no-case-folding" + surrogates. But the common >> case wins here. >> >> http://cr.openjdk.java.net/~sherman/8032012/webrev/ >> > > From xueming.shen at oracle.com Thu Feb 6 04:37:35 2014 From: xueming.shen at oracle.com (Xueming Shen) Date: Wed, 05 Feb 2014 20:37:35 -0800 Subject: RFR: JDK-8032012, , String.toLowerCase/toUpperCase performance improvement In-Reply-To: References: <52D82013.7020009@oracle.com> <52DEEF45.7060506@oracle.com> <3ACAF47E-0B56-4D17-B322-506EAB79A168@oracle.com> <52F27BDB.6040705@oracle.com> <52F2864C.1030508@oracle.com> <52F2A670.60806@univ-mlv.fr> <52F2AD7D.1050609@oracle.com> <52F2C927.3030609@CoSoCo.de> <52F2CFCD.7050001@oracle.com> Message-ID: <52F3118F.4020902@oracle.com> Fair enough. I don't think it's going to be a measurable difference. I have updated the webrev to use the Character.isSurrogate() for better readability. http://cr.openjdk.java.net/~sherman/8032012/webrev -Sherman On 2/5/14 6:21 PM, Vitaly Davidovich wrote: > > i2c conversion should not cost anything; it'll just make jit use low > 16 bits of the registers for (unsigned) comparisons. I haven't > checked this though, but that's what I'd expect. > > Sent from my phone > > On Feb 5, 2014 7:27 PM, "Xueming Shen" > wrote: > > On 02/05/2014 03:28 PM, Ulf Zibis wrote: > > Additionally you could use Character.isSurrogate() and > Character.isSupplementaryCode Both are better optimized for JIT. > > > j.l.C.isSupplementaryCodePoint() checks up boundary of supp, we > probably don't need it > here, as the returning code point is either a ERROR or a valid > unicode code point. > > I'm not sure about the j.l.C.isSurrogate(), which takes a char and > we have an int here. > I would expect the javac will inline the constants for me, but I > don't know whether jit > can inline and then optimize away the explicit casting i2c. Not a > big deal though. > > -sherman > > > > > -Ulf > > > Am 05.02.2014 22:30, schrieb Xueming Shen: > > Hi Remi, > > Good suggestion. Now the "common case" path is much simple > and faster :-) > I'm seeing a 5%-10% boost for the normal-non-surrogates > case. And it appears > the bmp+surr mixed is getting faster as well. Though I > would assume the it > would get slower in case of "no-case-folding" + > surrogates. But the common > case wins here. > > http://cr.openjdk.java.net/~sherman/8032012/webrev/ > > > > From david.holmes at oracle.com Thu Feb 6 05:18:30 2014 From: david.holmes at oracle.com (David Holmes) Date: Thu, 06 Feb 2014 15:18:30 +1000 Subject: RFR: 8033565: Remove unused nativeNewStringPlatform and nativeGetStringPlatformChars In-Reply-To: <52F2C21E.8030800@oracle.com> References: <52F2C21E.8030800@oracle.com> Message-ID: <52F31B26.1040802@oracle.com> On 6/02/2014 8:58 AM, roger riggs wrote: > Please review this change to remove unused support in jni_util.c for > functions nativeNewStringPlatform and nativeGetStringPlatformChars. > > On Windows, both were conditional on jvm.dll being installed on a path > containing "kernel". > On Solaris, Linux, and Mac, they had no-op implementations. > > Webrev: > http://cr.openjdk.java.net/~rriggs/webrev-remove-nativenewstring-8033565/ Looks good to me too! Hopefully that is the last remnant of the Kernel VM. Cheers, David > Thanks, Roger > > From Alan.Bateman at oracle.com Thu Feb 6 07:58:19 2014 From: Alan.Bateman at oracle.com (Alan Bateman) Date: Thu, 06 Feb 2014 07:58:19 +0000 Subject: RFR: 8033565: Remove unused nativeNewStringPlatform and nativeGetStringPlatformChars In-Reply-To: <52F31B26.1040802@oracle.com> References: <52F2C21E.8030800@oracle.com> <52F31B26.1040802@oracle.com> Message-ID: <52F3409B.7090008@oracle.com> On 06/02/2014 05:18, David Holmes wrote: > On 6/02/2014 8:58 AM, roger riggs wrote: >> Please review this change to remove unused support in jni_util.c for >> functions nativeNewStringPlatform and nativeGetStringPlatformChars. >> >> On Windows, both were conditional on jvm.dll being installed on a path >> containing "kernel". >> On Solaris, Linux, and Mac, they had no-op implementations. >> >> Webrev: >> http://cr.openjdk.java.net/~rriggs/webrev-remove-nativenewstring-8033565/ >> > > Looks good to me too! Hopefully that is the last remnant of the Kernel > VM. Indeed. I thought the last of the remnants were removed in JDK 7 but it seems not. The changes look good to me too. -Alan. From tristan.yan at oracle.com Thu Feb 6 09:13:17 2014 From: tristan.yan at oracle.com (Tristan Yan) Date: Thu, 6 Feb 2014 17:13:17 +0800 Subject: RFR for JDK-8030844:sun/rmi/rmic/classpath/RMICClassPathTest.java timeout in same binaries run Message-ID: Hi All Please help to review a simple fix for https://bugs.openjdk.java.net/browse/JDK-8030844 http://cr.openjdk.java.net/~tyan/JDK-8030844/webrev.00/. Description: Change replace a ?/home/~user" folder with an test source path. Folder ?/home/~user? cause some problem whenever something wrong with the automount filesystem or an username lookup problem. Thank you Tristan From paul.sandoz at oracle.com Thu Feb 6 09:44:07 2014 From: paul.sandoz at oracle.com (Paul Sandoz) Date: Thu, 6 Feb 2014 10:44:07 +0100 Subject: RFR: JDK-8032012, , String.toLowerCase/toUpperCase performance improvement In-Reply-To: <52F3118F.4020902@oracle.com> References: <52D82013.7020009@oracle.com> <52DEEF45.7060506@oracle.com> <3ACAF47E-0B56-4D17-B322-506EAB79A168@oracle.com> <52F27BDB.6040705@oracle.com> <52F2864C.1030508@oracle.com> <52F2A670.60806@univ-mlv.fr> <52F2AD7D.1050609@oracle.com> <52F2C927.3030609@CoSoCo.de> <52F2CFCD.7050001@oracle.com> <52F3118F.4020902@oracle.com> Message-ID: On Feb 6, 2014, at 5:37 AM, Xueming Shen wrote: > Fair enough. I don't think it's going to be a measurable difference. I have updated the webrev > to use the Character.isSurrogate() for better readability. > > http://cr.openjdk.java.net/~sherman/8032012/webrev > One last point, sorry :-) Can you use Character.toUpperCase rather than Character.toUpperCaseEx ? [*]. Paul. [*] They both seem to point to the same code, so either Character.toUpperCase can incorrectly return Character.ERROR or Character.toUpperCaseEx never returns ERROR: public static int toUpperCase(int codePoint) { return CharacterData.of(codePoint).toUpperCase(codePoint); } static int toUpperCaseEx(int codePoint) { assert isValidCodePoint(codePoint); return CharacterData.of(codePoint).toUpperCaseEx(codePoint); } abstract class CharacterData { ... abstract int toUpperCase(int ch); ... //need to implement for JSR204 int toUpperCaseEx(int ch) { return toUpperCase(ch); } From chris.hegarty at oracle.com Thu Feb 6 12:07:37 2014 From: chris.hegarty at oracle.com (Chris Hegarty) Date: Thu, 6 Feb 2014 12:07:37 +0000 Subject: ObjectIn/OutputStream improvements In-Reply-To: <52F2695E.20204@gmail.com> References: <91644C57-6E0D-48ED-8E31-6AC217DECCEF@gmx.de>, <1C96B28E-5E20-4D8D-B1FD-8FD5DF193EC8@oracle.com> <52EBB1B8.2030401@oracle.com> <9A31C421-CBDC-4196-A74C-0CB5E2447CB6@oracle.com> <264411B6-B1E3-4BCA-BC64-42673D5C6733@oracle.com> <52F2695E.20204@gmail.com> Message-ID: <993C87EE-54F4-407D-B7B8-053D3391C2CA@oracle.com> Thanks Peter, this is a nice improvement. I?ll incorporate your changes before pushing. -Chris. On 5 Feb 2014, at 16:39, Peter Levart wrote: > On 02/05/2014 04:11 PM, Chris Hegarty wrote: >> Thanks stuart, Mike, and Paul. >> >>> - Why not have getClassSignature() return an interned string? (that's if interning is actually essential. Are we sure it's not just overhead?) >> I didn?t want to change the existing use of interning here, just refactor the code a little to make it cleaner. >> >>> I think that would be a better way to spend our performance investigation budget. Thus, it might be better to remove the StringBuilder field from this patch and submit another one focusing on UTF decoding; the other changes in the patch look good. >> Agreed. This could be looked at separately. >> >> Latest webrev: >> http://cr.openjdk.java.net/~chegar/serial_stupp.01/ >> >> Thanks, >> -Chris. > > Hi Chris, > > What about the following even less garbage-producing-and-copying ObjectStreamClass.get[Class|Method]Signature combo: > > /** > * Returns JVM type signature for given class. > */ > static String getClassSignature(Class cl) { > if (cl.isPrimitive()) > return getPrimitiveSignature(cl); > else > return appendClassSignature(new StringBuilder(), cl).toString(); > } > > private static StringBuilder appendClassSignature(StringBuilder sbuf, Class cl) { > while (cl.isArray()) { > sbuf.append('['); > cl = cl.getComponentType(); > } > > if (cl.isPrimitive()) > sbuf.append(getPrimitiveSignature(cl)); > else > sbuf.append('L').append(cl.getName().replace('.', '/')).append(';'); > > return sbuf; > } > > /** > * Returns JVM type signature for given list of parameters and return type. > */ > private static String getMethodSignature(Class[] paramTypes, > Class retType) > { > StringBuilder sbuf = new StringBuilder(); > sbuf.append('('); > for (int i = 0; i < paramTypes.length; i++) { > appendClassSignature(sbuf, paramTypes[i]); > } > sbuf.append(')'); > appendClassSignature(sbuf, retType); > return sbuf.toString(); > } > > > Regards, Peter > From daniel.fuchs at oracle.com Thu Feb 6 15:29:32 2014 From: daniel.fuchs at oracle.com (Daniel Fuchs) Date: Thu, 06 Feb 2014 16:29:32 +0100 Subject: RFR - 8029876: Add a test to verify that Logger.getLogger & LogManager.getLogManager don't throw NPE when System.out == null Message-ID: <52F3AA5C.4080006@oracle.com> Hi, Please find below a patch that adds a new test for logging in JDK 9. The test verifies an issue we had on JDK 7 (the test is already in 7) but since the issue was not present in 8 & 9 the fix (and test) went to 7 only. This bug is about adding the same test that was added to 7 to 9 (and then 8u). https://bugs.openjdk.java.net/browse/JDK-8029876 8029876: Add a test to verify that Logger.getLogger & LogManager.getLogManager don't throw NPE when System.out == null original fix + tests pushed in 7: http://hg.openjdk.java.net/jdk7u/jdk7u/jdk/rev/3ba8d15adade webrev (trivial copy of the test to jdk9-dev): http://cr.openjdk.java.net/~dfuchs/webrev_8029876/webrev.00/ best regards, -- daniel From mandy.chung at oracle.com Thu Feb 6 15:49:51 2014 From: mandy.chung at oracle.com (Mandy Chung) Date: Thu, 06 Feb 2014 07:49:51 -0800 Subject: RFR - 8029876: Add a test to verify that Logger.getLogger & LogManager.getLogManager don't throw NPE when System.out == null In-Reply-To: <52F3AA5C.4080006@oracle.com> References: <52F3AA5C.4080006@oracle.com> Message-ID: <52F3AF1F.5070604@oracle.com> Hi Daniel, On 2/6/2014 7:29 AM, Daniel Fuchs wrote: > https://bugs.openjdk.java.net/browse/JDK-8029876 > 8029876: Add a test to verify that Logger.getLogger > & LogManager.getLogManager don't throw NPE > when System.out == null > > webrev (trivial copy of the test to jdk9-dev): > http://cr.openjdk.java.net/~dfuchs/webrev_8029876/webrev.00/ Do you think this test can use the JavaAWTAccessStub as in other tests so that it doesn't depend on SunToolkit? TestMainAppContext.java also depends on SunTookit. It may be that the bug can only be duplicated in this way; if so, please add the new test in needstest/TEST.group. It would make it simpler to move the tests with AWT dependency to a separate directory. Mandy From Ulf.Zibis at CoSoCo.de Thu Feb 6 16:42:44 2014 From: Ulf.Zibis at CoSoCo.de (Ulf Zibis) Date: Thu, 06 Feb 2014 17:42:44 +0100 Subject: RFR: JDK-8032012, , String.toLowerCase/toUpperCase performance improvement In-Reply-To: <52F2CFCD.7050001@oracle.com> References: <52D82013.7020009@oracle.com> <52DEEF45.7060506@oracle.com> <3ACAF47E-0B56-4D17-B322-506EAB79A168@oracle.com> <52F27BDB.6040705@oracle.com> <52F2864C.1030508@oracle.com> <52F2A670.60806@univ-mlv.fr> <52F2AD7D.1050609@oracle.com> <52F2C927.3030609@CoSoCo.de> <52F2CFCD.7050001@oracle.com> Message-ID: <52F3BB84.5080300@CoSoCo.de> Hi, Am 06.02.2014 00:57, schrieb Xueming Shen: > On 02/05/2014 03:28 PM, Ulf Zibis wrote: >> Additionally you could use Character.isSurrogate() and Character.isSupplementaryCode> appropriate places. Both are better optimized for JIT. > > j.l.C.isSupplementaryCodePoint() checks up boundary of supp, we probably don't need it > here, as the returning code point is either a ERROR or a valid unicode code point. Sorry, I was in error. I meant using !isBmpCodePoint() which is codePoint >>> 16 != 0 which is should be slightly faster than codePoint >= Character.MIN_SUPPLEMENTARY_CODE_POINT as the latter needs a 32-bit value to be loaded. > I'm not sure about the j.l.C.isSurrogate(), which takes a char and we have an int here. > I would expect the javac will inline the constants for me, but I don't know whether jit > can inline and then optimize away the explicit casting i2c. Not a big deal though. Why you use (in) here, you could do the cast later? Yes, IIRC from my HSdis inspection, i2c is a Noop and yes, isSurrogate() is better to read. Additionally I remember, we had discussion if isSurrogate() would be faster with (byte)(ch >>> 11) == (byte)(0xD8 >>> 3) So if isSurrogate() will be optimised in future or even intrinsified, using isSurrogate() should be better than ch >= MIN_HIGH_SURROGATE&& c <= MAX_HIGH_SURROGATE Please note: ch >= MIN_SURROGATE && ch < (MAX_SURROGATE + 1) generally is better than: ch >= MIN_HIGH_SURROGATE&& c <= MAX_HIGH_SURROGATE See: http://bugs.java.com/bugdatabase/view_bug.do?bug_id=6984886 -Ulf From daniel.fuchs at oracle.com Thu Feb 6 17:04:04 2014 From: daniel.fuchs at oracle.com (Daniel Fuchs) Date: Thu, 06 Feb 2014 18:04:04 +0100 Subject: RFR - 8029876: Add a test to verify that Logger.getLogger & LogManager.getLogManager don't throw NPE when System.out == null In-Reply-To: <52F3AF1F.5070604@oracle.com> References: <52F3AA5C.4080006@oracle.com> <52F3AF1F.5070604@oracle.com> Message-ID: <52F3C084.5080502@oracle.com> On 2/6/14 4:49 PM, Mandy Chung wrote: > Hi Daniel, > > On 2/6/2014 7:29 AM, Daniel Fuchs wrote: >> https://bugs.openjdk.java.net/browse/JDK-8029876 >> 8029876: Add a test to verify that Logger.getLogger >> & LogManager.getLogManager don't throw NPE >> when System.out == null >> >> webrev (trivial copy of the test to jdk9-dev): >> http://cr.openjdk.java.net/~dfuchs/webrev_8029876/webrev.00/ > > Do you think this test can use the JavaAWTAccessStub as in other tests > so that it doesn't depend on SunToolkit? TestMainAppContext.java also > depends on SunTookit. Ah right! I remember we already spoke about that for the original test. Here is a new version that doesn't depend on AWT. http://cr.openjdk.java.net/~dfuchs/webrev_8029876/webrev.01/ -- daniel > > It may be that the bug can only be duplicated in this way; if so, please > add the new test in needstest/TEST.group. It would make it simpler to > move the tests with AWT dependency to a separate directory. > > Mandy From tbuktu at hotmail.com Thu Feb 6 17:11:21 2014 From: tbuktu at hotmail.com (Tim Buktu) Date: Thu, 6 Feb 2014 18:11:21 +0100 Subject: BigDecimal performance Message-ID: Hi, now that BigInteger deals better with large numbers, it would be nice for that to translate into an improvement in BigDecimal performance because BigDecimal is essentially a wrapper around BigInteger. Unfortunately, BigDecimal is still slower than BigInteger because it has to scale and round. I don't see a way to fix this without breaking the BigDecimal=BigInteger*10^n paradigm, but it could be done by introducing something like a BigFloat class that wraps a BigInteger such that BigFloat=BigInteger*2^n. I would expect the code to be less complex than BigDecimal because the only places it would have to deal with powers of ten would be conversion from and to String or BigDecimal. It would also be faster than BigDecimal for the same reason, but the downside is that it wouldn't accurately represent decimal fractions (just like float and double). Is this something that would be beneficial in the real world? I also did a little experiment to see how long a computation would take using BigDecimals vs the same computation using fixed-point BigInteger arithmetic. I wrote two programs that calculate pi to a million digits. The BigInteger version took 3 minutes; the BigDecimal version took 28 minutes (both single-threaded). Tim From henry.jen at oracle.com Thu Feb 6 17:16:11 2014 From: henry.jen at oracle.com (Henry Jen) Date: Thu, 06 Feb 2014 09:16:11 -0800 Subject: RFR: 8033590: java.util.Comparator thenComparing has unnecessary type restriction Message-ID: <52F3C35B.3040308@oracle.com> Hi, Please review a simple bug fix that loosen the generic type, the original proposed CCC for JDK-8009736 and webrev/specdiff has a discrepancy, I am not sure if another CCC is needed. https://bugs.openjdk.java.net/browse/JDK-8033590 http://cr.openjdk.java.net/~henryjen/tl/8033590.0/webrev/ Cheers, Henry From tbuktu at hotmail.com Thu Feb 6 17:22:53 2014 From: tbuktu at hotmail.com (Tim Buktu) Date: Thu, 6 Feb 2014 18:22:53 +0100 Subject: 8014320 ready for review Message-ID: Hi, 8014320 is a JDK9 enhancement and everybody is probably busy with JDK8, but I wanted to let the BigInteger people know the patch is finished. It consists of 3 patched files and 1 new file; they can be found at https://github.com/tbuktu/bigint/raw/5142a961252b2cd217377f32e4351640086c94e2/src/main/java/java/math/BigInteger.java https://github.com/tbuktu/bigint/raw/fe807bf92b58cf8a4556b3f8473f1b6dd35412b7/src/main/java/java/math/MutableModFn.java https://github.com/tbuktu/bigint/raw/3bd7329d93540a839582afb62c6ce40e243b3e47/src/main/java/java/math/MutableBigInteger.java https://github.com/tbuktu/bigint/raw/557584cba766507fe18d0cca5e8f03f45b483223/src/test/java/BigIntegerTest.java The SS and Barrett thresholds are from timings on an Intel CPU. An AMD CPU gave me lower values; I didn't try any other architectures. I used the program at the link below to find the thresholds: https://github.com/tbuktu/bigint/blob/master/src/main/java/Tune.java Tim From Ulf.Zibis at CoSoCo.de Thu Feb 6 17:59:42 2014 From: Ulf.Zibis at CoSoCo.de (Ulf Zibis) Date: Thu, 06 Feb 2014 18:59:42 +0100 Subject: RFR: JDK-8032012, , String.toLowerCase/toUpperCase performance improvement In-Reply-To: <52F299CB.4020006@oracle.com> References: <52D82013.7020009@oracle.com> <52DEEF45.7060506@oracle.com> <3ACAF47E-0B56-4D17-B322-506EAB79A168@oracle.com> <52F27BDB.6040705@oracle.com> <52F2864C.1030508@oracle.com> <52F299CB.4020006@oracle.com> Message-ID: <52F3CD8E.5000205@CoSoCo.de> Am 05.02.2014 21:06, schrieb Xueming Shen: > http://cr.openjdk.java.net/~sherman/8032012/webrev > > webrev has been updated accordingly. I still more like the break-to-label approach. It looks more logical and saves one comparison. This might count on very very short strings, but I would rename it to something like find1stUpperCase: or scanForUpperCase. As an alternative you could write "if (first >= len)" as counterpart to the last "first < len" operation in the for loop, which therefore should be merged by JIT. (Or use "first != len" in the for loop.) -Ulf From xueming.shen at oracle.com Thu Feb 6 17:40:44 2014 From: xueming.shen at oracle.com (Xueming Shen) Date: Thu, 06 Feb 2014 09:40:44 -0800 Subject: RFR: JDK-8032012, , String.toLowerCase/toUpperCase performance improvement In-Reply-To: References: <52D82013.7020009@oracle.com> <52DEEF45.7060506@oracle.com> <3ACAF47E-0B56-4D17-B322-506EAB79A168@oracle.com> <52F27BDB.6040705@oracle.com> <52F2864C.1030508@oracle.com> <52F2A670.60806@univ-mlv.fr> <52F2AD7D.1050609@oracle.com> <52F2C927.3030609@CoSoCo.de> <52F2CFCD.7050001@oracle.com> <52F3118F.4020902@oracle.com> Message-ID: <52F3C91C.1070800@oracle.com> Paul, toUpperCaseEx is overridden in CharacterData00/Latin1. Those two are under gensrc/java/lang. It might be possible to combine them some day (need to dig out some decade long history and probably there is compability concern...), but definitely is beyond the scope of this "improvement" :-) Ulf, webrev has been updated to use isBmpCodePoint() as suggested. Another benefit of using isBmpCodePoint() is that some Character.ERROR checks are no longer necessary if (lowerChar != Character.ERROR&& lowerChar< Character.MIN_SUPPLEMENTARY_CODE_POINT) { result[resultOffset++] = (char)lowerChar; } else { --> if (Character.isBmpCodePoint(lowerChar)) { result[resultOffset++] = (char)lowerChar; } else { Are we good enough to go? :-) While it took much longer than I would have expected, but I'm happy with the latest result. http://cr.openjdk.java.net/~sherman/8032012/webrev/ -Sherman On 02/06/2014 01:44 AM, Paul Sandoz wrote: > On Feb 6, 2014, at 5:37 AM, Xueming Shen wrote: > >> Fair enough. I don't think it's going to be a measurable difference. I have updated the webrev >> to use the Character.isSurrogate() for better readability. >> >> http://cr.openjdk.java.net/~sherman/8032012/webrev >> > One last point, sorry :-) > > Can you use Character.toUpperCase rather than Character.toUpperCaseEx ? [*]. > > Paul. > > [*] They both seem to point to the same code, so either Character.toUpperCase can incorrectly return Character.ERROR or Character.toUpperCaseEx never returns ERROR: > > public static int toUpperCase(int codePoint) { > return CharacterData.of(codePoint).toUpperCase(codePoint); > } > > static int toUpperCaseEx(int codePoint) { > assert isValidCodePoint(codePoint); > return CharacterData.of(codePoint).toUpperCaseEx(codePoint); > } > > > > abstract class CharacterData { > ... > abstract int toUpperCase(int ch); > ... > > //need to implement for JSR204 > int toUpperCaseEx(int ch) { > return toUpperCase(ch); > } > From Ulf.Zibis at CoSoCo.de Thu Feb 6 18:30:42 2014 From: Ulf.Zibis at CoSoCo.de (Ulf Zibis) Date: Thu, 06 Feb 2014 19:30:42 +0100 Subject: RFR: JDK-8032012, , String.toLowerCase/toUpperCase performance improvement In-Reply-To: <52F3C91C.1070800@oracle.com> References: <52D82013.7020009@oracle.com> <52DEEF45.7060506@oracle.com> <3ACAF47E-0B56-4D17-B322-506EAB79A168@oracle.com> <52F27BDB.6040705@oracle.com> <52F2864C.1030508@oracle.com> <52F2A670.60806@univ-mlv.fr> <52F2AD7D.1050609@oracle.com> <52F2C927.3030609@CoSoCo.de> <52F2CFCD.7050001@oracle.com> <52F3118F.4020902@oracle.com> <52F3C91C.1070800@oracle.com> Message-ID: <52F3D4D2.1070308@CoSoCo.de> Am 06.02.2014 18:40, schrieb Xueming Shen: > Ulf, > > webrev has been updated to use isBmpCodePoint() as suggested. Another > benefit of using isBmpCodePoint() is that some Character.ERROR checks > are no longer necessary Great, I didn't see that. But why not just coding: 2558 char ch = value[first]; 2559 if (Character.isSurrogate(ch) { 2560 hasSurr = true; 2561 break; 2562 } 2563 if (ch != Character.toLowerCase((int)ch)) { // no need to check Character.ERROR 2564 break; 2565 } -Ulf From xueming.shen at oracle.com Thu Feb 6 18:47:30 2014 From: xueming.shen at oracle.com (Xueming Shen) Date: Thu, 06 Feb 2014 10:47:30 -0800 Subject: RFR: JDK-8032012, , String.toLowerCase/toUpperCase performance improvement In-Reply-To: <52F3D4D2.1070308@CoSoCo.de> References: <52D82013.7020009@oracle.com> <52DEEF45.7060506@oracle.com> <3ACAF47E-0B56-4D17-B322-506EAB79A168@oracle.com> <52F27BDB.6040705@oracle.com> <52F2864C.1030508@oracle.com> <52F2A670.60806@univ-mlv.fr> <52F2AD7D.1050609@oracle.com> <52F2C927.3030609@CoSoCo.de> <52F2CFCD.7050001@oracle.com> <52F3118F.4020902@oracle.com> <52F3C91C.1070800@oracle.com> <52F3D4D2.1070308@CoSoCo.de> Message-ID: <52F3D8C2.6070008@oracle.com> On 02/06/2014 10:30 AM, Ulf Zibis wrote: > > Am 06.02.2014 18:40, schrieb Xueming Shen: >> Ulf, >> >> webrev has been updated to use isBmpCodePoint() as suggested. Another >> benefit of using isBmpCodePoint() is that some Character.ERROR checks >> are no longer necessary > > Great, I didn't see that. > > But why not just coding: > 2558 char ch = value[first]; > 2559 if (Character.isSurrogate(ch) { > 2560 hasSurr = true; > 2561 break; > 2562 } > 2563 if (ch != Character.toLowerCase((int)ch)) { // no need to check Character.ERROR > 2564 break; > 2565 } > > -Ulf > There is no measurable difference. I'm fine with either way. Given we are on a 32/64-bit platform, it might be better to just use the "word" size type? :-) -Sherman From henry.jen at oracle.com Thu Feb 6 18:34:09 2014 From: henry.jen at oracle.com (henry.jen at oracle.com) Date: Thu, 06 Feb 2014 18:34:09 +0000 Subject: hg: jdk8/tl/jdk: 8033590: java.util.Comparator::thenComparing has unnecessary type restriction Message-ID: <20140206183432.A898562A7F@hg.openjdk.java.net> Changeset: 7534523b4174 Author: henryjen Date: 2014-02-06 10:30 -0800 URL: http://hg.openjdk.java.net/jdk8/tl/jdk/rev/7534523b4174 8033590: java.util.Comparator::thenComparing has unnecessary type restriction Reviewed-by: psandoz ! src/share/classes/java/util/Comparator.java ! test/java/util/Comparator/TypeTest.java From Alan.Bateman at oracle.com Thu Feb 6 19:07:12 2014 From: Alan.Bateman at oracle.com (Alan Bateman) Date: Thu, 06 Feb 2014 19:07:12 +0000 Subject: RFR: 8033590: java.util.Comparator thenComparing has unnecessary type restriction In-Reply-To: <52F3C35B.3040308@oracle.com> References: <52F3C35B.3040308@oracle.com> Message-ID: <52F3DD60.4080002@oracle.com> On 06/02/2014 17:16, Henry Jen wrote: > Hi, > > Please review a simple bug fix that loosen the generic type, the > original proposed CCC for JDK-8009736 and webrev/specdiff has a > discrepancy, I am not sure if another CCC is needed. > > https://bugs.openjdk.java.net/browse/JDK-8033590 > http://cr.openjdk.java.net/~henryjen/tl/8033590.0/webrev/ It's good that this was caught and that it has been approved. Just a minor comment on the test is whether it might make sense to put the original issue into the @bug too, otherwise it looks like the test was added just for this specific bug. -Alan. From jeremymanson at google.com Thu Feb 6 19:25:42 2014 From: jeremymanson at google.com (Jeremy Manson) Date: Thu, 6 Feb 2014 11:25:42 -0800 Subject: Adding a link() Method to Method, Redux Message-ID: Hi folks, ( I sent this around in 2009, and it kind of got dropped. We're forward porting our patches to JDK8 now, so I've come across it again. It would be great not to have to keep forward porting it, and using reflection to invoke it. Original thread is here: http://mail.openjdk.java.net/pipermail/core-libs-dev/2009-August/002319.html Original method follows... ) I found myself in need of a method that determines whether a given native method is linked, and links it if it is present. My use case prevents me from calling the method directly and catching an UnsatisfiedLinkError, because the method has side effects. This is not the first time I've needed such a method, so I implemented it and dropped it into java.lang.reflect.Method. I thought I would ask if it was something that should be added to the platform. The prototype looks like this: /** * Attempts to link this method, per Section 5.6 of the Java * Virtual Machine Specification. Returns {@code true} if this is * a Java method, or if it is a native method and the Java Virtual * Machine succeeds in loading an appropriate native-language * definition of it; returns {@code false} otherwise. * * @return {@code true} if this is a Java method, or if it is a * native method and the Java Virtual Machine succeeds in * loading an appropriate native-language definition of it; returns * {@code false} otherwise. * */ public boolean tryLink(); This is basically useful in any case where you would otherwise catch UnsatisfiedLinkError, but can't invoke the method. For example, if you have a binary that lives on the system, but you aren't sure whether that shared library provides the implementation you want. (Another point in the design space would be to have it called link(), return void, and throw an UnsatisfiedLinkError if it fails. It is also worth pointing out that the JVMS calls it "binding", not "linking", so there is that possibility as well.) What do you all think? It seems equally as useful as the ability to catch UnsatisfiedLinkError from native method invocation. (2014 update: Alan thought I should add a versioning method to my native library. This is less direct, of course. Thoughts? Yes / no / maybe?) Jeremy From Ulf.Zibis at CoSoCo.de Thu Feb 6 19:47:33 2014 From: Ulf.Zibis at CoSoCo.de (Ulf Zibis) Date: Thu, 06 Feb 2014 20:47:33 +0100 Subject: RFR: JDK-8032012, , String.toLowerCase/toUpperCase performance improvement In-Reply-To: <52F3D8C2.6070008@oracle.com> References: <52D82013.7020009@oracle.com> <52DEEF45.7060506@oracle.com> <3ACAF47E-0B56-4D17-B322-506EAB79A168@oracle.com> <52F27BDB.6040705@oracle.com> <52F2864C.1030508@oracle.com> <52F2A670.60806@univ-mlv.fr> <52F2AD7D.1050609@oracle.com> <52F2C927.3030609@CoSoCo.de> <52F2CFCD.7050001@oracle.com> <52F3118F.4020902@oracle.com> <52F3C91C.1070800@oracle.com> <52F3D4D2.1070308@CoSoCo.de> <52F3D8C2.6070008@oracle.com> Message-ID: <52F3E6D5.8030704@CoSoCo.de> Me again ;-) > On 02/06/2014 10:30 AM, Ulf Zibis wrote: >> But why not just coding: >> 2558 char ch = value[first]; >> 2559 if (Character.isSurrogate(ch) { >> 2560 hasSurr = true; >> 2561 break; >> 2562 } >> 2563 if (ch != Character.toLowerCase((int)ch)) { // no need to check Character.ERROR >> 2564 break; >> 2565 } >> There already is Character.isUpperCase(). I don't see, why (ch != Character.toLowerCase((int)ch) should perform better. But there is a bug in javadoc ;-) Your Report (Review ID: JI-9010267) - Error in doc of Character.isLowerCase/UpperCase(char) -Ulf From xueming.shen at oracle.com Thu Feb 6 20:21:36 2014 From: xueming.shen at oracle.com (Xueming Shen) Date: Thu, 06 Feb 2014 12:21:36 -0800 Subject: RFR: JDK-8032012, , String.toLowerCase/toUpperCase performance improvement In-Reply-To: <52F3E57A.6000402@gmx.de> References: <52D82013.7020009@oracle.com> <52DEEF45.7060506@oracle.com> <3ACAF47E-0B56-4D17-B322-506EAB79A168@oracle.com> <52F27BDB.6040705@oracle.com> <52F2864C.1030508@oracle.com> <52F2A670.60806@univ-mlv.fr> <52F2AD7D.1050609@oracle.com> <52F2C927.3030609@CoSoCo.de> <52F2CFCD.7050001@oracle.com> <52F3118F.4020902@oracle.com> <52F3C91C.1070800@oracle.com> <52F3D4D2.1070308@CoSoCo.de> <52F3D8C2.6070008@oracle.com> <52F3E57A.6000402@gmx.de> Message-ID: <52F3EED0.9030002@oracle.com> On 02/06/2014 11:41 AM, Ulf Zibis wrote: > Me again ;-) > >> On 02/06/2014 10:30 AM, Ulf Zibis wrote: >>> But why not just coding: >>> 2558 char ch = value[first]; >>> 2559 if (Character.isSurrogate(ch) { >>> 2560 hasSurr = true; >>> 2561 break; >>> 2562 } >>> 2563 if (ch != Character.toLowerCase((int)ch)) { // no need to check Character.ERROR >>> 2564 break; >>> 2565 } >>> > > There already is Character.isUpperCase(). > I don't see, why (ch != Character.toLowerCase((int)ch) should perform better. > > But there is a bug in javadoc ;-) > Your Report (Review ID: JI-9010267) - Error in doc of Character.isLowerCase/UpperCase(char) > > -Ulf > "ch != Character.toLowerCase((int)ch)" and "Character.isUpperCase(ch)" are not equal for some code points...don't ask me why, please:-) -Sherman From Ulf.Zibis at CoSoCo.de Thu Feb 6 20:26:20 2014 From: Ulf.Zibis at CoSoCo.de (Ulf Zibis) Date: Thu, 06 Feb 2014 21:26:20 +0100 Subject: RFR: JDK-8032012, , String.toLowerCase/toUpperCase performance improvement In-Reply-To: <52F3CD8E.5000205@CoSoCo.de> References: <52D82013.7020009@oracle.com> <52DEEF45.7060506@oracle.com> <3ACAF47E-0B56-4D17-B322-506EAB79A168@oracle.com> <52F27BDB.6040705@oracle.com> <52F2864C.1030508@oracle.com> <52F299CB.4020006@oracle.com> <52F3CD8E.5000205@CoSoCo.de> Message-ID: <52F3EFEC.10205@CoSoCo.de> To cover this problem, I filed a new language suggestion: https://bugs.openjdk.java.net/browse/JDK-8033813 :-) -Ulf Am 06.02.2014 18:59, schrieb Ulf Zibis: > I still more like the break-to-label approach. It looks more logical and saves one comparison. > This might count on very very short strings, but I would rename it to something like > find1stUpperCase: or scanForUpperCase. > > As an alternative you could write "if (first >= len)" as counterpart to the last "first < len" > operation in the for loop, which therefore should be merged by JIT. > (Or use "first != len" in the for loop.) > > -Ulf > > > From snazy at snazy.de Thu Feb 6 22:36:36 2014 From: snazy at snazy.de (Robert Stupp) Date: Thu, 6 Feb 2014 23:36:36 +0100 Subject: Object allocation tracing / profilers Message-ID: <3486E1B1-8059-4362-B0B1-30A84E2C199D@snazy.de> Hi, I'm trying to trace/count object allocations - especially regarding the ObjectIn/OutputStream stuff. I know two API points: JVMTI and instrumentation. JVMTI fires an event for nearly everything except allocations from bytecode - instrumentation allows to transform bytecode (e.g. to issue a callback on object allocation). But I am not very satisfied - maybe I oversee something... What I need is the number of "real" allocations. I do not want to count allocations that Hotspot would normally eliminate - for example Hotspot seems to eliminate instances of ArrayList$Itr under some circumstances. I played around with this instrumentation code: https://code.google.com/p/java-allocation-instrumenter/ But this one and any other profiler (JProfiler) I tried always gave me the theoretical number of instances of ArrayList$Itr. This reason seems to be as follows: With instrumentation/transformation constructors get a piece of code "injected" which calls a "profile object allocation method". This method gets the new object instance, which is used to get the Class object and to inquire the shallow object size on the heap - and it gets the current thread's stack trace. My suggestion is, that Hotspot cannot eliminate object allocations (for example of ArrayList$Itr) because the profiler bytecode needs that object instance and does other things, that are "strange" to Hotspot. I hoped that I can get callbacks using JVMTI but due to the restrictions mentioned here (http://docs.oracle.com/javase/7/docs/platform/jvmti/jvmti.html#VMObjectAlloc) there are no callbacks for allocations caused by bytecode. VMObjectAlloc events occur during the start and end of the application - but not a single event is fired while the application is running (I've attached my JVMTI code). Am I doing anything wrong? Is there a possibility to count only the "real" instances? If not, wouldn't it be nice to have such a functionality in JVMTI or a callback functionality for http://download.java.net/jdk8/docs/api/java/lang/instrument/Instrumentation.html? My fear is that all profiler results that all developers see are inaccurate because the code injected by profilers influences Hotspot too much. - Robert From philip.hodges at bluewin.ch Wed Feb 5 20:29:30 2014 From: philip.hodges at bluewin.ch (Philip Hodges) Date: Wed, 5 Feb 2014 21:29:30 +0100 Subject: StringJoiner: detect or fix delimiter collision? In-Reply-To: <0F878BF8-2179-49F4-ADF0-95ECBE4FE727@bluewin.ch> References: <51673A45.8060908@oracle.com> <1374608627412-145039.post@n7.nabble.com> <52218FE6.7000804@bluewin.ch> <7916219F-E651-4939-BFB2-7D7CC38C9EEF@bluewin.ch> <5ACEEEB3-A1D3-4F45-9D78-FDFE567A65B0@bluewin.ch> <54010BC7-BDB8-4471-AC2F-20C9449FB85B@oracle.com> <0F878BF8-2179-49F4-ADF0-95ECBE4FE727@bluewin.ch> Message-ID: <9A55AE66-B475-45BD-B68C-15B9AD7F9254@bluewin.ch> String is probably the most commonly used class in the whole of Java. Please please please do not pollute it with this dangerous new method. Your Java-is-cool example is ridiculously trivial, of no practical use whatsoever, and absolutely *not* cool. How about a proper test where you have to double up the delimiter if it occurs in an element: assertEquals("'Java','isn''t','perfect'", "'"+String.join("','", javaIsNotPerfect)+"'"); I now have a working proof of concept, where newJoiner returns an UnsplittableJoiner which requires a delimiter handler to convert it back into a Joiner. Actually my joiner is an ObjectJoiner, unlike your StringJoiner that is just a CharSequenceJoiner, unlike AbstractCollections.toString and Arrays.toString which can join any elements that have a toString method. Please have the guts to pull String.join from the release now, before it becomes an antipattern, and you have to deprecate it. And tell everybody exactly why. Seriously. On 27 Jan 2014, at 21:31, Philip Hodges wrote: > I did not predict that it would be a "sprintf". It's not going to be consistently misused anything like so frequently. > I compared it to the unescaped XML generation antipattern. > > I have not seen any technical justifications whatsoever so far, just inexplicable enthusiasm. > > It is like giving young drivers a faster car with no seat belts. > > The trouble is, unlike in Google guava, this is the JDK, you can't just drop flawed beta interfaces two versions later. All you can do is add them to static review tool blacklists and deprecate them to cause a warning at compile time, and hope that people will give them a wide berth. Even if you hide them in an unoffical sun.com package. By the way, I'm looking forward to the proprietary Base64 being changed to simply call the new official one. You can't even modify an equals method in an undocumented reflection implementation in 1.7.0.51 without breaking production applications. Wow. > > I am raising doubt and you are ignoring it. > Don't you have the guts to say whoa, stop the bandwagon, there could just be a real problem with this, it really will make it easier to create bugs without any warning from the compiler and without making anyone's code better *in any way at all*? > > I am picking on String.join precisely because I have seen way too many delimiter collision bugs, not just in Java but in several other languages, and because this interface perpetuates a real world problem by preventing future interface changes to detect them. > > [I am not always a doomsayer. For example, I am not picking on JSR310 because Date and Calendar and SimpleDateFormat are well known disaster areas and the people working on their replacement have a deep understanding of the issues, have really taken their time, and nothing whatsoever that I have read about it jumped out at me and said this is *wrong*. It might not even be completely perfect. But I am confident it will be so much better than what we have now, and it's a shame that I won't have time to migrate existing apps to it.] > > The counterproposal, no, proposal refinement, wafting around inside my head, is to somehow compel programmers to make just one more method call before that final toString(). It will be difficult to find good names, especially ones that will be understood by programmers for whom English is not a first language. Something like: > > String.join(delimiter, joinables).assertNoUnescapedDelimiters().toString(); > String.join(delimiter, joinables).neverMindDelimiterCollisions().toString(); > String.join(delimiter, joinables).promiseNoUnescapedDelimiters().toString(); > String.join(delimiter, joinables).escapeDelimiters(escapeChar).toString(); > String.join(delimiter, joinables).quoteElements(quoteChar).toString(); > > > The vital thing is that String.join has to return an unjoinable, that needs an adapter method to make it safely joinable. If you get that right, then we can forgive this first shot being a little slow, and enjoy the triumph of CharSequence over immutability. > > Yes, ultimately the goal should be to add full support for at least the most popular csv generation recipes. > > I'm really sorry I couldn't carry on arguing the case before August. As a minority, I only have one person's quota of energy. I will try to get some more people to look at it. > > > On 27 Jan 2014, at 18:44, Mike Duigou wrote: > >> >> On Jan 26 2014, at 17:12 , Philip Hodges wrote: >> >>> Please please please drop StringJoiner from Java 1.8 before it is too late. >> >> It is well past "too late". The API has been frozen since August for all but the most exceptional cases. >> >>>>> At first I thought they were cool. Then I tried to use them in anger. >>>>> And was forced to roll my own. >>>> >>>> I would encourage you to share your implementation or the javadocs as grist for discussion. An actual alternative is the *only* thing that is likely to be persuasive. >> >> You seem to be very much in the minority on this issue by the silence from other responders on this list and elsewhere. The concerns you've highlighted with regard to delimiter management, while certainly valid, have not been of sufficient concern to suggest a change in the proposal. The prediction that using StringJoiner will become Java's sprintf seems unlikely to be be true. The reception we've seen thus far for StringJoiner has been otherwise exclusively enthusiastic and positive. >> >> Alternatives, refinements and counterproposals are always welcome in the discussion. Doomsaying unfortunately adds little. >> >> Mike > From snazy at snazy.de Thu Feb 6 22:50:53 2014 From: snazy at snazy.de (Robert Stupp) Date: Thu, 6 Feb 2014 23:50:53 +0100 Subject: ObjectIn/OutputStream improvements In-Reply-To: <993C87EE-54F4-407D-B7B8-053D3391C2CA@oracle.com> References: <91644C57-6E0D-48ED-8E31-6AC217DECCEF@gmx.de>, <1C96B28E-5E20-4D8D-B1FD-8FD5DF193EC8@oracle.com> <52EBB1B8.2030401@oracle.com> <9A31C421-CBDC-4196-A74C-0CB5E2447CB6@oracle.com> <264411B6-B1E3-4BCA-BC64-42673D5C6733@oracle.com> <52F2695E.20204@gmail.com> <993C87EE-54F4-407D-B7B8-053D3391C2CA@oracle.com> Message-ID: Hi, I looked into the UTF serialization and deserialization code - and compared it a bit with the code behind "new String(byte[], Charset)", "String.getBytes(Charset)". Just to find something that can be safely reused in Object*Stream classes to optimize String handling. The first thing I noticed is that the class sun.nio.cs.UTF_8 class uses one byte for (char)0 - Object*Stream use two 0 bytes to represent (char)0. Although this representation is incompatible to the representation that the class UTF_8 uses it is not a big issue. The other thing is that Object*Stream seem not to be able to (de)serialize 21 bit characters. Methods in UTF_8 class are sun.nio.cs.UTF_8.Decoder#decode and sun.nio.cs.UTF_8.Encoder#encode. Is it ok to add 21-bit UTF-8 representation in Object*Stream? - Robert Am 06.02.2014 um 13:07 schrieb Chris Hegarty : > Thanks Peter, this is a nice improvement. I?ll incorporate your changes before pushing. > > -Chris. > > On 5 Feb 2014, at 16:39, Peter Levart wrote: > >> On 02/05/2014 04:11 PM, Chris Hegarty wrote: >>> Thanks stuart, Mike, and Paul. >>> >>>> - Why not have getClassSignature() return an interned string? (that's if interning is actually essential. Are we sure it's not just overhead?) >>> I didn?t want to change the existing use of interning here, just refactor the code a little to make it cleaner. >>> >>>> I think that would be a better way to spend our performance investigation budget. Thus, it might be better to remove the StringBuilder field from this patch and submit another one focusing on UTF decoding; the other changes in the patch look good. >>> Agreed. This could be looked at separately. >>> >>> Latest webrev: >>> http://cr.openjdk.java.net/~chegar/serial_stupp.01/ >>> >>> Thanks, >>> -Chris. >> >> Hi Chris, >> >> What about the following even less garbage-producing-and-copying ObjectStreamClass.get[Class|Method]Signature combo: >> >> /** >> * Returns JVM type signature for given class. >> */ >> static String getClassSignature(Class cl) { >> if (cl.isPrimitive()) >> return getPrimitiveSignature(cl); >> else >> return appendClassSignature(new StringBuilder(), cl).toString(); >> } >> >> private static StringBuilder appendClassSignature(StringBuilder sbuf, Class cl) { >> while (cl.isArray()) { >> sbuf.append('['); >> cl = cl.getComponentType(); >> } >> >> if (cl.isPrimitive()) >> sbuf.append(getPrimitiveSignature(cl)); >> else >> sbuf.append('L').append(cl.getName().replace('.', '/')).append(';'); >> >> return sbuf; >> } >> >> /** >> * Returns JVM type signature for given list of parameters and return type. >> */ >> private static String getMethodSignature(Class[] paramTypes, >> Class retType) >> { >> StringBuilder sbuf = new StringBuilder(); >> sbuf.append('('); >> for (int i = 0; i < paramTypes.length; i++) { >> appendClassSignature(sbuf, paramTypes[i]); >> } >> sbuf.append(')'); >> appendClassSignature(sbuf, retType); >> return sbuf.toString(); >> } >> >> >> Regards, Peter >> > From martinrb at google.com Thu Feb 6 23:11:33 2014 From: martinrb at google.com (Martin Buchholz) Date: Thu, 6 Feb 2014 15:11:33 -0800 Subject: ObjectIn/OutputStream improvements In-Reply-To: References: <91644C57-6E0D-48ED-8E31-6AC217DECCEF@gmx.de> <1C96B28E-5E20-4D8D-B1FD-8FD5DF193EC8@oracle.com> <52EBB1B8.2030401@oracle.com> <9A31C421-CBDC-4196-A74C-0CB5E2447CB6@oracle.com> <264411B6-B1E3-4BCA-BC64-42673D5C6733@oracle.com> <52F2695E.20204@gmail.com> <993C87EE-54F4-407D-B7B8-053D3391C2CA@oracle.com> Message-ID: I had forgotten about the use of Modified UTF-8 in serialization. Leave it alone. http://en.wikipedia.org/wiki/UTF-8#Modified_UTF-8 On Thu, Feb 6, 2014 at 2:50 PM, Robert Stupp wrote: > Hi, > > I looked into the UTF serialization and deserialization code - and > compared it a bit with the code behind "new String(byte[], Charset)", > "String.getBytes(Charset)". > Just to find something that can be safely reused in Object*Stream classes > to optimize String handling. > > The first thing I noticed is that the class sun.nio.cs.UTF_8 class uses > one byte for (char)0 - Object*Stream use two 0 bytes to represent (char)0. > Although this representation is incompatible to the representation that the > class UTF_8 uses it is not a big issue. > The other thing is that Object*Stream seem not to be able to (de)serialize > 21 bit characters. Methods in UTF_8 class are > sun.nio.cs.UTF_8.Decoder#decode and sun.nio.cs.UTF_8.Encoder#encode. Is it > ok to add 21-bit UTF-8 representation in Object*Stream? > > - > Robert > > Am 06.02.2014 um 13:07 schrieb Chris Hegarty : > > > Thanks Peter, this is a nice improvement. I?ll incorporate your changes > before pushing. > > > > -Chris. > > > > On 5 Feb 2014, at 16:39, Peter Levart wrote: > > > >> On 02/05/2014 04:11 PM, Chris Hegarty wrote: > >>> Thanks stuart, Mike, and Paul. > >>> > >>>> - Why not have getClassSignature() return an interned string? (that's > if interning is actually essential. Are we sure it's not just overhead?) > >>> I didn?t want to change the existing use of interning here, just > refactor the code a little to make it cleaner. > >>> > >>>> I think that would be a better way to spend our performance > investigation budget. Thus, it might be better to remove the StringBuilder > field from this patch and submit another one focusing on UTF decoding; the > other changes in the patch look good. > >>> Agreed. This could be looked at separately. > >>> > >>> Latest webrev: > >>> http://cr.openjdk.java.net/~chegar/serial_stupp.01/ > >>> > >>> Thanks, > >>> -Chris. > >> > >> Hi Chris, > >> > >> What about the following even less garbage-producing-and-copying > ObjectStreamClass.get[Class|Method]Signature combo: > >> > >> /** > >> * Returns JVM type signature for given class. > >> */ > >> static String getClassSignature(Class cl) { > >> if (cl.isPrimitive()) > >> return getPrimitiveSignature(cl); > >> else > >> return appendClassSignature(new StringBuilder(), > cl).toString(); > >> } > >> > >> private static StringBuilder appendClassSignature(StringBuilder sbuf, > Class cl) { > >> while (cl.isArray()) { > >> sbuf.append('['); > >> cl = cl.getComponentType(); > >> } > >> > >> if (cl.isPrimitive()) > >> sbuf.append(getPrimitiveSignature(cl)); > >> else > >> sbuf.append('L').append(cl.getName().replace('.', '/')).append(';'); > >> > >> return sbuf; > >> } > >> > >> /** > >> * Returns JVM type signature for given list of parameters and return > type. > >> */ > >> private static String getMethodSignature(Class[] paramTypes, > >> Class retType) > >> { > >> StringBuilder sbuf = new StringBuilder(); > >> sbuf.append('('); > >> for (int i = 0; i < paramTypes.length; i++) { > >> appendClassSignature(sbuf, paramTypes[i]); > >> } > >> sbuf.append(')'); > >> appendClassSignature(sbuf, retType); > >> return sbuf.toString(); > >> } > >> > >> > >> Regards, Peter > >> > > > > From xueming.shen at oracle.com Thu Feb 6 23:15:10 2014 From: xueming.shen at oracle.com (Xueming Shen) Date: Thu, 06 Feb 2014 15:15:10 -0800 Subject: ObjectIn/OutputStream improvements In-Reply-To: References: <91644C57-6E0D-48ED-8E31-6AC217DECCEF@gmx.de>, <1C96B28E-5E20-4D8D-B1FD-8FD5DF193EC8@oracle.com> <52EBB1B8.2030401@oracle.com> <9A31C421-CBDC-4196-A74C-0CB5E2447CB6@oracle.com> <264411B6-B1E3-4BCA-BC64-42673D5C6733@oracle.com> <52F2695E.20204@gmail.com> <993C87EE-54F4-407D-B7B8-053D3391C2CA@oracle.com> Message-ID: <52F4177E.5090401@oracle.com> Modified UTF8 has to be used, as specified by the spec. http://download.java.net/jdk8/docs/api/java/io/DataInput.html#modified-utf-8 -Sherman On 02/06/2014 02:50 PM, Robert Stupp wrote: >> >>> -Chris. > >> From snazy at snazy.de Thu Feb 6 23:20:27 2014 From: snazy at snazy.de (Robert Stupp) Date: Fri, 7 Feb 2014 00:20:27 +0100 Subject: ObjectIn/OutputStream improvements In-Reply-To: <52F4177E.5090401@oracle.com> References: <91644C57-6E0D-48ED-8E31-6AC217DECCEF@gmx.de>, <1C96B28E-5E20-4D8D-B1FD-8FD5DF193EC8@oracle.com> <52EBB1B8.2030401@oracle.com> <9A31C421-CBDC-4196-A74C-0CB5E2447CB6@oracle.com> <264411B6-B1E3-4BCA-BC64-42673D5C6733@oracle.com> <52F2695E.20204@gmail.com> <993C87EE-54F4-407D-B7B8-053D3391C2CA@oracle.com> <52F4177E.5090401@oracle.com> Message-ID: eh - yes - ouch! 21 bit cannot occur (char = 16 bits) please, please ignore my comment about it. Am 07.02.2014 um 00:15 schrieb Xueming Shen : > Modified UTF8 has to be used, as specified by the spec. > > http://download.java.net/jdk8/docs/api/java/io/DataInput.html#modified-utf-8 > > -Sherman > > On 02/06/2014 02:50 PM, Robert Stupp wrote: >>> >>> -Chris. >> >> > From jeremymanson at google.com Fri Feb 7 01:13:16 2014 From: jeremymanson at google.com (Jeremy Manson) Date: Thu, 6 Feb 2014 17:13:16 -0800 Subject: Object allocation tracing / profilers In-Reply-To: <3486E1B1-8059-4362-B0B1-30A84E2C199D@snazy.de> References: <3486E1B1-8059-4362-B0B1-30A84E2C199D@snazy.de> Message-ID: You aren't doing anything wrong. More detail: - Anything bytecode based (like my instrumenter, which you tried) isn't going to let you know what has been escape analyzed away. You can only do bytecode rewriting before escape analysis runs, and before the escape analysis runs, there is no way to find out what will be escape analyzed away. - A bytecode-based instrumenter will also not pick up allocations that occur via JNI, which is the problem that VMObjectAlloc solves. - If you happen to be on a platform that supports it, and you've compiled the JDK correctly, you can use dtrace to capture Java allocations. - Because of its limitations and performance issues, we've actually stopped doing bytecode-based instrumentation at Google in favor of instrumenting the VM directly. We now use JVMTI-style callbacks whenever an allocation happens. If people are interested, I can provide the patch (in fact, I forward ported it to JDK8 yesterday!), but it is rather user unfriendly, so you have to be rather ambitious to use it. On the plus side, it can take into account escape analyzed objects (right now, it doesn't instrument them, but this could conceivably be changed). - On the down side, if you take this approach, you can't call back into Java from the JVM callbacks. This is because calling back into Java may require the JVM to come to a safepoint, which you can't do in the middle of allocation (which is where the callback would occur). Jeremy On Thu, Feb 6, 2014 at 2:36 PM, Robert Stupp wrote: > Hi, > > I'm trying to trace/count object allocations - especially regarding the > ObjectIn/OutputStream stuff. > > I know two API points: JVMTI and instrumentation. JVMTI fires an event for > nearly everything except allocations from bytecode - instrumentation allows > to transform bytecode (e.g. to issue a callback on object allocation). > > But I am not very satisfied - maybe I oversee something... > What I need is the number of "real" allocations. I do not want to count > allocations that Hotspot would normally eliminate - for example Hotspot > seems to eliminate instances of ArrayList$Itr under some circumstances. > > I played around with this instrumentation code: > https://code.google.com/p/java-allocation-instrumenter/ > But this one and any other profiler (JProfiler) I tried always gave me the > theoretical number of instances of ArrayList$Itr. This reason seems to be > as follows: > With instrumentation/transformation constructors get a piece of code > "injected" which calls a "profile object allocation method". This method > gets the new object instance, which is used to get the Class object and to > inquire the shallow object size on the heap - and it gets the current > thread's stack trace. > > My suggestion is, that Hotspot cannot eliminate object allocations (for > example of ArrayList$Itr) because the profiler bytecode needs that object > instance and does other things, that are "strange" to Hotspot. > > I hoped that I can get callbacks using JVMTI but due to the restrictions > mentioned here ( > http://docs.oracle.com/javase/7/docs/platform/jvmti/jvmti.html#VMObjectAlloc) > there are no callbacks for allocations caused by bytecode. > VMObjectAlloc events occur during the start and end of the application - > but not a single event is fired while the application is running (I've > attached my JVMTI code). > > Am I doing anything wrong? Is there a possibility to count only the "real" > instances? If not, wouldn't it be nice to have such a functionality in > JVMTI or a callback functionality for > http://download.java.net/jdk8/docs/api/java/lang/instrument/Instrumentation.html > ? > My fear is that all profiler results that all developers see are > inaccurate because the code injected by profilers influences Hotspot too > much. > > - > Robert > > From michael.cui at oracle.com Fri Feb 7 03:41:48 2014 From: michael.cui at oracle.com (michael cui) Date: Fri, 07 Feb 2014 11:41:48 +0800 Subject: RFR for JDK-8028711: TEST_BUG: Tests should pass through VM options: corelibs tests In-Reply-To: <52E5B4C3.6080405@oracle.com> References: <52E4C2E8.1060605@oracle.com> <52E4D1F2.4030406@oracle.com> <52E5B4C3.6080405@oracle.com> Message-ID: <52F455FC.3020009@oracle.com> Hi Alan, Based on current discussions we have had and existing usages in our code base, I would like to propose that the fix should ONLY make sure that ${TESTVMOPTS} will be passed in when launching java from shell script. For other options such as ${TESTJAVAOPTS} ${TESTTOOLVMOPTS} and ${TESTJAVACOPTS}, Since we are not very clear about needs to make them presents all the time when launching java or other java tools from shell, it is better to not make changes to current usages. Please let me know if you are ok with this approach. Thanks, Michael Cui On 01/27/2014 09:22 AM, michael cui wrote: > On 01/26/2014 05:14 PM, Alan Bateman wrote: >> On 26/01/2014 08:10, michael cui wrote: >>> Hi, >>> >>> Please review the following code changes for >>> >>> https://bugs.openjdk.java.net/browse/JDK-8028711 >>> >>> The bug and fix are for enforcing the following rules: >>> >>> launch java from shell script, ${TESTVMOPTS} and ${TESTJAVAOPTS} >>> should be passed in. >>> launch javac from shell script, ${TESTTOOLVMOPTS} and >>> ${TESTJAVACOPTS} should be passed in. >>> >>> So almost all the scripts who failed to follow above rules are >>> touched by this fix. >>> >>> There are very few exceptions since test cases explicitly state that >>> they are intended to launch java without any vm options. >>> >>> >>> webrev >>> http://cr.openjdk.java.net/~tyan/michael/JDK-8028711/webrev.01/ >> You might be aware of this area but we've had two previous efforts >> ([1] [2]) to update these tests to pass through options. > I am aware of efforts [1] JDK-8003890. Initially JDK-8028711 is opened > for double check the result of JDK-8003890 and fix them if there are > uncovered scripts. > > Initial list scripts need to touch are : > java/lang/Thread/UncaughtExceptions.sh > sun/tools/common/ApplicationSetup.sh > sun/tools/jrunscript/jrunscript-argsTest.sh > sun/tools/jrunscript/jrunscript-cpTest.sh > sun/tools/jrunscript/jrunscript-DTest.sh > sun/tools/jrunscript/jrunscript-eTest.sh > sun/tools/jrunscript/jrunscript-fTest.sh > sun/tools/jrunscript/jrunscriptTest.sh > > Later on, by reading http://openjdk.java.net/jtreg/vmoptions.html and > current last comments of JDK-8019502 (which is the parent of > JDK-8028711), also by exploring current existing usage (I mean large > part of scripts passing TESTJAVAOPTS or TESTJAVACOPTS.), I feel that > we might want to make the consistent usage of these two env variables > as well. > > I am not aware of efforts [2]. Thanks for the link. >> I have to admit that I wasn't aware of TESTJAVAOPTS or TESTJAVACOPTS. >> Looking at the" jtreg -help all" page then these seem to correspond >> to -javaoption and -javacoption. I see -vmoption used all the time to >> specify options, I don't think I've seen -javaoption being used but I >> see your point. One thing that might be helpful here is some guidance >> or examples as it confusing to have so many options that initially >> appear to do the same thing. > I just attached a very simple example to show the behaviours described > at http://openjdk.java.net/jtreg/vmoptions.html. > > Sample can download it from > https://bugs.openjdk.java.net/secure/attachment/18467/sample.zip > > My understanding of options is : > > When we start jtreg with "-vmoptions" , jtreg will set its value to > TESTVMOPTS and TESTTOOLVMOPTS before running shell script test (shell > scripts identified by jtreg tags @test and @run shell ). So if we call > java with TESTVMOPTS and call javac with TESTTOOLVMOPTS in shell > script, then it impact to both. > > If we want to set some options only for launching java without impact > other javac or other java tools, then we need to use "-javaoptions" > with jtreg. (Precondition : TESTJAVAOPTS was passed) > > Similarly, "-javacoptions" will only impact launching javac from all > shell scripts. (Precondition : TESTJAVACOPTS was passed) > > It would be less confusing if we could keep use them in a consistent way. > >> >> Mike Duigou might want to have a quick look to see if jdk/Makefile >> needs to be updated to support -javacoption. As these tests aren't >> really testing javac (except incidentally) then it might not be >> important but it's yet another place where you can loose a finger. >> >> I haven't looked at all the files but just notice that you've updated >> the genXXX scripts. They are used to generate tests, they aren't >> tests themselves and so aren't run by jtreg. > Then I will rollback those changes. I found three scripts belongs to > these category. > > test/java/nio/Buffer/genBasic.sh > test/java/nio/Buffer/genCopyDirectMemory.sh > test/java/util/Formatter/genBasic.sh >> >> So are you planning to also update the java tests that create VMs to >> pass through the options? This point came up during previous efforts >> (and some test infrastructure has been added) but it isn't clear to >> me where we are on that. >> > > I don't plan to cover that in this bug fix. I though it will be > covered by JDK-8019502. I could work on that bug if you want me to > cover that as well. > >> -Alan. >> >> [1] http://hg.openjdk.java.net/jdk8/tl/jdk/rev/ae5d04dbacd6 >> [2] http://hg.openjdk.java.net/jdk8/tl/jdk/rev/7da291690aa0 > > -Michael Cui From paul.sandoz at oracle.com Fri Feb 7 10:32:26 2014 From: paul.sandoz at oracle.com (Paul Sandoz) Date: Fri, 7 Feb 2014 11:32:26 +0100 Subject: RFR: JDK-8032012, , String.toLowerCase/toUpperCase performance improvement In-Reply-To: <52F3C91C.1070800@oracle.com> References: <52D82013.7020009@oracle.com> <52DEEF45.7060506@oracle.com> <3ACAF47E-0B56-4D17-B322-506EAB79A168@oracle.com> <52F27BDB.6040705@oracle.com> <52F2864C.1030508@oracle.com> <52F2A670.60806@univ-mlv.fr> <52F2AD7D.1050609@oracle.com> <52F2C927.3030609@CoSoCo.de> <52F2CFCD.7050001@oracle.com> <52F3118F.4020902@oracle.com> <52F3C91C.1070800@oracle.com> Message-ID: <89C37BCA-31B7-490F-97F3-6778E7D55DF8@oracle.com> On Feb 6, 2014, at 6:40 PM, Xueming Shen wrote: > Paul, > > toUpperCaseEx is overridden in CharacterData00/Latin1. Those two are > under gensrc/java/lang. It might be possible to combine them some day > (need to dig out some decade long history and probably there is compability > concern...), but definitely is beyond the scope of this "improvement" :-) > I was not suggesting combining, it was more a spin off to pondering of why Character.toUpperCase was not used (mirroring the use of toLowerCase). Anyway, this is minor. > Are we good enough to go? :-) While it took much longer than I would have > expected, but I'm happy with the latest result. > > http://cr.openjdk.java.net/~sherman/8032012/webrev/ > +1 from me. Paul. From forax at univ-mlv.fr Fri Feb 7 10:44:30 2014 From: forax at univ-mlv.fr (Remi Forax) Date: Fri, 07 Feb 2014 11:44:30 +0100 Subject: RFR: JDK-8032012, , String.toLowerCase/toUpperCase performance improvement In-Reply-To: <89C37BCA-31B7-490F-97F3-6778E7D55DF8@oracle.com> References: <52D82013.7020009@oracle.com> <52DEEF45.7060506@oracle.com> <3ACAF47E-0B56-4D17-B322-506EAB79A168@oracle.com> <52F27BDB.6040705@oracle.com> <52F2864C.1030508@oracle.com> <52F2A670.60806@univ-mlv.fr> <52F2AD7D.1050609@oracle.com> <52F2C927.3030609@CoSoCo.de> <52F2CFCD.7050001@oracle.com> <52F3118F.4020902@oracle.com> <52F3C91C.1070800@oracle.com> <89C37BCA-31B7-490F-97F3-6778E7D55DF8@oracle.com> Message-ID: <52F4B90E.5030902@univ-mlv.fr> On 02/07/2014 11:32 AM, Paul Sandoz wrote: > On Feb 6, 2014, at 6:40 PM, Xueming Shen wrote: > >> Paul, >> >> toUpperCaseEx is overridden in CharacterData00/Latin1. Those two are >> under gensrc/java/lang. It might be possible to combine them some day >> (need to dig out some decade long history and probably there is compability >> concern...), but definitely is beyond the scope of this "improvement" :-) >> > I was not suggesting combining, it was more a spin off to pondering of why Character.toUpperCase was not used (mirroring the use of toLowerCase). Anyway, this is minor. > > >> Are we good enough to go? :-) While it took much longer than I would have >> expected, but I'm happy with the latest result. >> >> http://cr.openjdk.java.net/~sherman/8032012/webrev/ >> > +1 from me. > > Paul. I'm happy with the latest code too. R?mi From dmitry.nadezhin at gmail.com Fri Feb 7 10:44:47 2014 From: dmitry.nadezhin at gmail.com (Dmitry Nadezhin) Date: Fri, 7 Feb 2014 14:44:47 +0400 Subject: BigDecimal performance In-Reply-To: References: Message-ID: I think that better name is BigBinary. Both BigDecimal and BigBinary are floating-point number with radix=10 and radix=2 respectively. More general approach is an abstract class or interface Rational which has implementation subclasses. Each nonzero rational number P/Q can be represented as P/Q = p/q * 2^e , where e is integer, p and q are odd integers and GCD(p,q) = 1. Then BigBinary is a subclass with q=1. Arithmetic operations on Rationals are implemented by general algorithm when arguments are true rationals (q!=1) and by specific algorithms when they are Binaries (q=1). This is elaborated here: http://interval.louisiana.edu/reliable-computing-journal/volume-19/reliable-computing-19-pp-229-247.pdf https://java.net/projects/jinterval/sources/svn/show/trunk/jinterval/jinterval-rational-java/src/main/java/net/java/jinterval/rational On Thu, Feb 6, 2014 at 9:11 PM, Tim Buktu wrote: > Hi, > > now that BigInteger deals better with large numbers, it would be nice > for that to translate into an improvement in BigDecimal performance > because BigDecimal is essentially a wrapper around BigInteger. > Unfortunately, BigDecimal is still slower than BigInteger because it has > to scale and round. > > I don't see a way to fix this without breaking the > BigDecimal=BigInteger*10^n paradigm, but it could be done by introducing > something like a BigFloat class that wraps a BigInteger such that > BigFloat=BigInteger*2^n. I would expect the code to be less complex than > BigDecimal because the only places it would have to deal with powers of > ten would be conversion from and to String or BigDecimal. It would also > be faster than BigDecimal for the same reason, but the downside is that > it wouldn't accurately represent decimal fractions (just like float and > double). > > Is this something that would be beneficial in the real world? > > I also did a little experiment to see how long a computation would take > using BigDecimals vs the same computation using fixed-point BigInteger > arithmetic. I wrote two programs that calculate pi to a million digits. > The BigInteger version took 3 minutes; the BigDecimal version took 28 > minutes (both single-threaded). > > Tim > From staffan.larsen at oracle.com Fri Feb 7 10:46:07 2014 From: staffan.larsen at oracle.com (Staffan Larsen) Date: Fri, 7 Feb 2014 11:46:07 +0100 Subject: RFR(S): JDK-8033911 Simplify instrumentation of FileInputStream and RandomAccessFile Message-ID: <2BA69765-5405-413A-BAC6-EDE77C176FFA@oracle.com> A few of the public read and write methods in FileInputStream and RandomAccessFile are declared native. This means that it is hard to instrument them using byte code instrumentation. Changing the public methods to be to non-native and instead calling private native methods simplifies instrumentation. webrev: http://cr.openjdk.java.net/~sla/8033911/webrev.00/ bug: https://bugs.openjdk.java.net/browse/JDK-8033911 Thanks, /Staffan From Alan.Bateman at oracle.com Fri Feb 7 10:56:30 2014 From: Alan.Bateman at oracle.com (Alan Bateman) Date: Fri, 07 Feb 2014 10:56:30 +0000 Subject: RFR(S): JDK-8033911 Simplify instrumentation of FileInputStream and RandomAccessFile In-Reply-To: <2BA69765-5405-413A-BAC6-EDE77C176FFA@oracle.com> References: <2BA69765-5405-413A-BAC6-EDE77C176FFA@oracle.com> Message-ID: <52F4BBDE.4010005@oracle.com> On 07/02/2014 10:46, Staffan Larsen wrote: > A few of the public read and write methods in FileInputStream and RandomAccessFile are declared native. This means that it is hard to instrument them using byte code instrumentation. Changing the public methods to be to non-native and instead calling private native methods simplifies instrumentation. > > webrev: http://cr.openjdk.java.net/~sla/8033911/webrev.00/ > bug: https://bugs.openjdk.java.net/browse/JDK-8033911 > I assume you know this already but both JVM TI and java.lang.instrument do have support for hooking into the resolution of native methods. It is of course more complicated and not for the fainthearted. The proposed changes look okay. The only thing is that it might not be complete but perhaps it's not too interesting to instrument methods such as skip or available. -Alan. From Alan.Bateman at oracle.com Fri Feb 7 11:03:23 2014 From: Alan.Bateman at oracle.com (Alan Bateman) Date: Fri, 07 Feb 2014 11:03:23 +0000 Subject: RFR for JDK-8028711: TEST_BUG: Tests should pass through VM options: corelibs tests In-Reply-To: <52F455FC.3020009@oracle.com> References: <52E4C2E8.1060605@oracle.com> <52E4D1F2.4030406@oracle.com> <52E5B4C3.6080405@oracle.com> <52F455FC.3020009@oracle.com> Message-ID: <52F4BD7B.8080006@oracle.com> On 07/02/2014 03:41, michael cui wrote: > Hi Alan, > > Based on current discussions we have had and existing usages in our > code base, > I would like to propose that the fix should ONLY make sure that > ${TESTVMOPTS} will be passed in when launching java from shell script. > > For other options such as ${TESTJAVAOPTS} ${TESTTOOLVMOPTS} and > ${TESTJAVACOPTS}, > Since we are not very clear about needs to make them presents all the > time when launching java or other java tools from shell, it is better > to not make changes to current usages. > > Please let me know if you are ok with this approach. If I understand correctly then you are proposing to address the remaining issues where the vmoptions aren't passed through from shell tests. That would be good. As I mentioned, there have been efforts on this already but we either missed some tests or we didn't continue it into newer tests. The other part is java tests that launch additional VMs as we do have a few that don't pass the options through. That might be a separate bug but would be good to address. As regards TESTJAVAOPTS then I think it might be worth starting a discussion on the jtreg list to see if there are opportunities to consolidate the number of variables that tests have to deal with. I don't have any objection to the original proposal, it's just that it makes it obvious that there are too many options and too easy to get it wrong. -Alan. From dmitry.samersoff at oracle.com Fri Feb 7 11:07:23 2014 From: dmitry.samersoff at oracle.com (Dmitry Samersoff) Date: Fri, 07 Feb 2014 15:07:23 +0400 Subject: RFR(S): JDK-8033911 Simplify instrumentation of FileInputStream and RandomAccessFile In-Reply-To: <2BA69765-5405-413A-BAC6-EDE77C176FFA@oracle.com> References: <2BA69765-5405-413A-BAC6-EDE77C176FFA@oracle.com> Message-ID: <52F4BE6B.8040109@oracle.com> Staffan, As far as you touching this. Is it possible to change all native methods in these two classes to have 0 at the end of name? i.e. readBytes => readBytes0 it's pure cosmetic, but fairly simplify core dump reading and later grep-ing. -Dmitry On 2014-02-07 14:46, Staffan Larsen wrote: > A few of the public read and write methods in FileInputStream and RandomAccessFile are declared native. This means that it is hard to instrument them using byte code instrumentation. Changing the public methods to be to non-native and instead calling private native methods simplifies instrumentation. > > webrev: http://cr.openjdk.java.net/~sla/8033911/webrev.00/ > bug: https://bugs.openjdk.java.net/browse/JDK-8033911 > > Thanks, > /Staffan > -- Dmitry Samersoff Oracle Java development team, Saint Petersburg, Russia * I would love to change the world, but they won't give me the sources. From dmitry.nadezhin at gmail.com Fri Feb 7 11:25:11 2014 From: dmitry.nadezhin at gmail.com (Dmitry Nadezhin) Date: Fri, 7 Feb 2014 15:25:11 +0400 Subject: 8014320 ready for review In-Reply-To: References: Message-ID: I'm curious to compare performance on huge numbers between this pure Java implementation of SS and some native implementation of SS (libgmp, for example). On Thu, Feb 6, 2014 at 9:22 PM, Tim Buktu wrote: > Hi, > > 8014320 is a JDK9 enhancement and everybody is probably busy with JDK8, > but I wanted to let the BigInteger people know the patch is finished. It > consists of 3 patched files and 1 new file; they can be found at > > > https://github.com/tbuktu/bigint/raw/5142a961252b2cd217377f32e4351640086c94e2/src/main/java/java/math/BigInteger.java > > https://github.com/tbuktu/bigint/raw/fe807bf92b58cf8a4556b3f8473f1b6dd35412b7/src/main/java/java/math/MutableModFn.java > > https://github.com/tbuktu/bigint/raw/3bd7329d93540a839582afb62c6ce40e243b3e47/src/main/java/java/math/MutableBigInteger.java > > https://github.com/tbuktu/bigint/raw/557584cba766507fe18d0cca5e8f03f45b483223/src/test/java/BigIntegerTest.java > > The SS and Barrett thresholds are from timings on an Intel CPU. An AMD > CPU gave me lower values; I didn't try any other architectures. I used > the program at the link below to find the thresholds: > > https://github.com/tbuktu/bigint/blob/master/src/main/java/Tune.java > > Tim > From staffan.larsen at oracle.com Fri Feb 7 11:27:46 2014 From: staffan.larsen at oracle.com (Staffan Larsen) Date: Fri, 7 Feb 2014 12:27:46 +0100 Subject: RFR(S): JDK-8033911 Simplify instrumentation of FileInputStream and RandomAccessFile In-Reply-To: <52F4BBDE.4010005@oracle.com> References: <2BA69765-5405-413A-BAC6-EDE77C176FFA@oracle.com> <52F4BBDE.4010005@oracle.com> Message-ID: On 7 feb 2014, at 11:56, Alan Bateman wrote: > On 07/02/2014 10:46, Staffan Larsen wrote: >> A few of the public read and write methods in FileInputStream and RandomAccessFile are declared native. This means that it is hard to instrument them using byte code instrumentation. Changing the public methods to be to non-native and instead calling private native methods simplifies instrumentation. >> >> webrev: http://cr.openjdk.java.net/~sla/8033911/webrev.00/ >> bug: https://bugs.openjdk.java.net/browse/JDK-8033911 >> > I assume you know this already but both JVM TI and java.lang.instrument do have support for hooking into the resolution of native methods. It is of course more complicated and not for the fainthearted. Yes, but a large drawback of using SetNativeMethodPrefix is that it requires schema changes to the class file (adding a method). This is not possible when dynamically attaching an instrumentation agent. > The proposed changes look okay. The only thing is that it might not be complete but perhaps it's not too interesting to instrument methods such as skip or available. That was my thinking, too. Thanks, /Staffan From staffan.larsen at oracle.com Fri Feb 7 11:28:12 2014 From: staffan.larsen at oracle.com (Staffan Larsen) Date: Fri, 7 Feb 2014 12:28:12 +0100 Subject: RFR(S): JDK-8033911 Simplify instrumentation of FileInputStream and RandomAccessFile In-Reply-To: <52F4BE6B.8040109@oracle.com> References: <2BA69765-5405-413A-BAC6-EDE77C176FFA@oracle.com> <52F4BE6B.8040109@oracle.com> Message-ID: I would prefer that to be a different change. Thanks, /Staffan On 7 feb 2014, at 12:07, Dmitry Samersoff wrote: > Staffan, > > As far as you touching this. > > Is it possible to change all native methods in these two classes to have > 0 at the end of name? > > i.e. readBytes => readBytes0 > > it's pure cosmetic, but fairly simplify core dump reading and later > grep-ing. > > -Dmitry > > On 2014-02-07 14:46, Staffan Larsen wrote: >> A few of the public read and write methods in FileInputStream and RandomAccessFile are declared native. This means that it is hard to instrument them using byte code instrumentation. Changing the public methods to be to non-native and instead calling private native methods simplifies instrumentation. >> >> webrev: http://cr.openjdk.java.net/~sla/8033911/webrev.00/ >> bug: https://bugs.openjdk.java.net/browse/JDK-8033911 >> >> Thanks, >> /Staffan >> > > > -- > Dmitry Samersoff > Oracle Java development team, Saint Petersburg, Russia > * I would love to change the world, but they won't give me the sources. From dmitry.samersoff at oracle.com Fri Feb 7 11:30:23 2014 From: dmitry.samersoff at oracle.com (Dmitry Samersoff) Date: Fri, 07 Feb 2014 15:30:23 +0400 Subject: RFR(S): JDK-8033911 Simplify instrumentation of FileInputStream and RandomAccessFile In-Reply-To: References: <2BA69765-5405-413A-BAC6-EDE77C176FFA@oracle.com> <52F4BE6B.8040109@oracle.com> Message-ID: <52F4C3CF.4090505@oracle.com> Staffan, OK! Looks good for me. -Dmitry On 2014-02-07 15:28, Staffan Larsen wrote: > I would prefer that to be a different change. > > Thanks, > /Staffan > > On 7 feb 2014, at 12:07, Dmitry Samersoff wrote: > >> Staffan, >> >> As far as you touching this. >> >> Is it possible to change all native methods in these two classes to have >> 0 at the end of name? >> >> i.e. readBytes => readBytes0 >> >> it's pure cosmetic, but fairly simplify core dump reading and later >> grep-ing. >> >> -Dmitry >> >> On 2014-02-07 14:46, Staffan Larsen wrote: >>> A few of the public read and write methods in FileInputStream and RandomAccessFile are declared native. This means that it is hard to instrument them using byte code instrumentation. Changing the public methods to be to non-native and instead calling private native methods simplifies instrumentation. >>> >>> webrev: http://cr.openjdk.java.net/~sla/8033911/webrev.00/ >>> bug: https://bugs.openjdk.java.net/browse/JDK-8033911 >>> >>> Thanks, >>> /Staffan >>> >> >> >> -- >> Dmitry Samersoff >> Oracle Java development team, Saint Petersburg, Russia >> * I would love to change the world, but they won't give me the sources. > -- Dmitry Samersoff Oracle Java development team, Saint Petersburg, Russia * I would love to change the world, but they won't give me the sources. From jaroslav.bachorik at oracle.com Fri Feb 7 11:38:59 2014 From: jaroslav.bachorik at oracle.com (Jaroslav Bachorik) Date: Fri, 07 Feb 2014 12:38:59 +0100 Subject: RFR(S): JDK-8033911 Simplify instrumentation of FileInputStream and RandomAccessFile In-Reply-To: <2BA69765-5405-413A-BAC6-EDE77C176FFA@oracle.com> References: <2BA69765-5405-413A-BAC6-EDE77C176FFA@oracle.com> Message-ID: <3d89f5ba-3c6e-4571-b7bf-19cbad4e5b5c@email.android.com> Looks good. Additional benefit is the compliance with the secure coding guide. -JB- On February 7, 2014 11:46:07 AM CET, Staffan Larsen wrote: >A few of the public read and write methods in FileInputStream and >RandomAccessFile are declared native. This means that it is hard to >instrument them using byte code instrumentation. Changing the public >methods to be to non-native and instead calling private native methods >simplifies instrumentation. > >webrev: http://cr.openjdk.java.net/~sla/8033911/webrev.00/ >bug: https://bugs.openjdk.java.net/browse/JDK-8033911 > >Thanks, >/Staffan -- Sent from my Android device with K-9 Mail. Please excuse my brevity. From staffan.larsen at oracle.com Fri Feb 7 12:07:04 2014 From: staffan.larsen at oracle.com (Staffan Larsen) Date: Fri, 7 Feb 2014 13:07:04 +0100 Subject: RFR: JDK-8033917 Keep track of file paths in file streams and channels for instrumentation purposes Message-ID: <25552F6F-0FFC-42EA-9B0B-49EC135A5EE6@oracle.com> Instrumentation agents that want to instrument FileInputStream/FileOutputStream to see which files are being accessed do not currently have access to the file system path of the stream. This is because the path is never stored in the stream class, only the file descriptor is. (This is also true for RandomAccessFile and FileChannel). An agent could instrument the respective constructors to store the path. The problem is where to store it. To add a field, the instrumentation agent needs to change the schema of the class. This is not possible at runtime but can be done at class-loading time. However for a j.l.instrument agent these classes are already defined when the agent is first called. For a native JVMTI agent the problem becomes parsing and modifying byte codes in a native agent which is error prone and requires a lot of code to maintain. If instead the stream classes were modified to store a reference to the path, it would be readily available for agents at a minimum of cost to the libraries. This is what this patch does. FileInputStream, FileOutputStream, RandomAccessFile and FileChannelImpl are modified to record the path they operate on in a private field. There are no accessors added to retrieve the path - it is purely stored for instrumentation purposes. The path is intentionally not resolved to be an absolute path since that would potentially add unwanted overhead. If a stream is created from a file descriptor, no path will be stored. The overhead for this path will be keeping the path String alive for a longer period of time. I hope this will not cause any problems. A consumer of this feature will be Java Flight Recorder, but the implementation is usable by other agents as well. webrev: http://cr.openjdk.java.net/~sla/8033917/webrev.00/ bug: https://bugs.openjdk.java.net/browse/JDK-8033917 Thanks, /Staffan From Alan.Bateman at oracle.com Fri Feb 7 12:27:53 2014 From: Alan.Bateman at oracle.com (Alan Bateman) Date: Fri, 07 Feb 2014 12:27:53 +0000 Subject: RFR: JDK-8033917 Keep track of file paths in file streams and channels for instrumentation purposes In-Reply-To: <25552F6F-0FFC-42EA-9B0B-49EC135A5EE6@oracle.com> References: <25552F6F-0FFC-42EA-9B0B-49EC135A5EE6@oracle.com> Message-ID: <52F4D149.8000900@oracle.com> On 07/02/2014 12:07, Staffan Larsen wrote: > Instrumentation agents that want to instrument FileInputStream/FileOutputStream to see which files are being accessed do not currently have access to the file system path of the stream. This is because the path is never stored in the stream class, only the file descriptor is. (This is also true for RandomAccessFile and FileChannel). > > An agent could instrument the respective constructors to store the path. The problem is where to store it. To add a field, the instrumentation agent needs to change the schema of the class. This is not possible at runtime but can be done at class-loading time. However for a j.l.instrument agent these classes are already defined when the agent is first called. For a native JVMTI agent the problem becomes parsing and modifying byte codes in a native agent which is error prone and requires a lot of code to maintain. > > If instead the stream classes were modified to store a reference to the path, it would be readily available for agents at a minimum of cost to the libraries. This is what this patch does. FileInputStream, FileOutputStream, RandomAccessFile and FileChannelImpl are modified to record the path they operate on in a private field. There are no accessors added to retrieve the path - it is purely stored for instrumentation purposes. The path is intentionally not resolved to be an absolute path since that would potentially add unwanted overhead. If a stream is created from a file descriptor, no path will be stored. > > The overhead for this path will be keeping the path String alive for a longer period of time. I hope this will not cause any problems. > > A consumer of this feature will be Java Flight Recorder, but the implementation is usable by other agents as well. > > webrev: http://cr.openjdk.java.net/~sla/8033917/webrev.00/ > bug: https://bugs.openjdk.java.net/browse/JDK-8033917 > I have reservations about doing this as hints of code making use of private fields which isn't good. For the comments in FileInputStream and other then it might be best to keep the line lengths consistent with the existing code if you can (it makes future side-by-side reviews a bit easier too). In WindowsChannelFactory then you've re-order and expand imports. The ordering of the import groups in this area has been Java SE, JDK-specific and then finally the JDK-internal. It's not a big deal of course. Personally I prefer the original static imports but I know some people don't. -Alan. From Ulf.Zibis at CoSoCo.de Fri Feb 7 12:30:42 2014 From: Ulf.Zibis at CoSoCo.de (Ulf Zibis) Date: Fri, 07 Feb 2014 13:30:42 +0100 Subject: RFR: JDK-8032012, , String.toLowerCase/toUpperCase performance improvement In-Reply-To: <52F3C91C.1070800@oracle.com> References: <52D82013.7020009@oracle.com> <52DEEF45.7060506@oracle.com> <3ACAF47E-0B56-4D17-B322-506EAB79A168@oracle.com> <52F27BDB.6040705@oracle.com> <52F2864C.1030508@oracle.com> <52F2A670.60806@univ-mlv.fr> <52F2AD7D.1050609@oracle.com> <52F2C927.3030609@CoSoCo.de> <52F2CFCD.7050001@oracle.com> <52F3118F.4020902@oracle.com> <52F3C91C.1070800@oracle.com> Message-ID: <52F4D1F2.8000701@CoSoCo.de> Am 06.02.2014 18:40, schrieb Xueming Shen: > Are we good enough to go? :-) While it took much longer than I would have > expected, but I'm happy with the latest result. > > http://cr.openjdk.java.net/~sherman/8032012/webrev/ Except "if (first >= len)" instead "if (first = len)" I'm with you. -Ulf From staffan.larsen at oracle.com Fri Feb 7 12:49:55 2014 From: staffan.larsen at oracle.com (Staffan Larsen) Date: Fri, 7 Feb 2014 13:49:55 +0100 Subject: RFR: JDK-8033917 Keep track of file paths in file streams and channels for instrumentation purposes In-Reply-To: <52F4D149.8000900@oracle.com> References: <25552F6F-0FFC-42EA-9B0B-49EC135A5EE6@oracle.com> <52F4D149.8000900@oracle.com> Message-ID: On 7 feb 2014, at 13:27, Alan Bateman wrote: > On 07/02/2014 12:07, Staffan Larsen wrote: >> Instrumentation agents that want to instrument FileInputStream/FileOutputStream to see which files are being accessed do not currently have access to the file system path of the stream. This is because the path is never stored in the stream class, only the file descriptor is. (This is also true for RandomAccessFile and FileChannel). >> >> An agent could instrument the respective constructors to store the path. The problem is where to store it. To add a field, the instrumentation agent needs to change the schema of the class. This is not possible at runtime but can be done at class-loading time. However for a j.l.instrument agent these classes are already defined when the agent is first called. For a native JVMTI agent the problem becomes parsing and modifying byte codes in a native agent which is error prone and requires a lot of code to maintain. >> >> If instead the stream classes were modified to store a reference to the path, it would be readily available for agents at a minimum of cost to the libraries. This is what this patch does. FileInputStream, FileOutputStream, RandomAccessFile and FileChannelImpl are modified to record the path they operate on in a private field. There are no accessors added to retrieve the path - it is purely stored for instrumentation purposes. The path is intentionally not resolved to be an absolute path since that would potentially add unwanted overhead. If a stream is created from a file descriptor, no path will be stored. >> >> The overhead for this path will be keeping the path String alive for a longer period of time. I hope this will not cause any problems. >> >> A consumer of this feature will be Java Flight Recorder, but the implementation is usable by other agents as well. >> >> webrev: http://cr.openjdk.java.net/~sla/8033917/webrev.00/ >> bug: https://bugs.openjdk.java.net/browse/JDK-8033917 >> > I have reservations about doing this as hints of code making use of private fields which isn't good. > > For the comments in FileInputStream and other then it might be best to keep the line lengths consistent with the existing code if you can (it makes future side-by-side reviews a bit easier too). I?ve updated the comments to have shorter lines and javadoc style comments. > In WindowsChannelFactory then you've re-order and expand imports. The ordering of the import groups in this area has been Java SE, JDK-specific and then finally the JDK-internal. It's not a big deal of course. Personally I prefer the original static imports but I know some people don?t. I?ve reverted to the original order, and only expanded the non-static imports. (The danger of IDEs). Updated webrev here: http://cr.openjdk.java.net/~sla/8033917/webrev.01/ Thanks, /Staffan From heiko.wagner at apis.de Fri Feb 7 12:53:53 2014 From: heiko.wagner at apis.de (Heiko Wagner) Date: Fri, 7 Feb 2014 13:53:53 +0100 Subject: java.net.URLClassloader: behaviour during OS suspend/resume on Window Message-ID: Hi, I hope this is the right list to post this question. I am currently facing some problems with class loading on Windows. When a Windows 7 laptop goes into power save mode (suspend) and wakes up again, my URLClassloader no longer does load any classes from my JAR file (ClassNotFoundException). This is propably caused by the fact, that after resume all network shares, where the JAR file for my app reside, and all file handles associated to network shares are no longer valid. While this arguably is a indication how horrible broken the OS/driver is, the only workaround is to listen for the OS "PBT_APMRESUMESUSPEND" event and try to manually fix the problem by trying to "reopen" all peviously opened file to obtain valid OS handles again. Is there any code in the JDK that tries to workaround the suspend/resume problem or what would you suggest as a feasable way to get the URLClassloader in a usable state again? Regards, Heiko P.S.: I similar probem is also triggered by badly written anti-virus software, as it scans JAR files it denies read-access to them and a ClassNotFoundException also occurs. From Alan.Bateman at oracle.com Fri Feb 7 13:22:15 2014 From: Alan.Bateman at oracle.com (Alan Bateman) Date: Fri, 07 Feb 2014 13:22:15 +0000 Subject: RFR: JDK-8033917 Keep track of file paths in file streams and channels for instrumentation purposes In-Reply-To: References: <25552F6F-0FFC-42EA-9B0B-49EC135A5EE6@oracle.com> <52F4D149.8000900@oracle.com> Message-ID: <52F4DE07.7090003@oracle.com> On 07/02/2014 12:49, Staffan Larsen wrote: > > Updated webrev here: > http://cr.openjdk.java.net/~sla/8033917/webrev.01/ > > Thanks for the updates, looks good. -Alan. From staffan.larsen at oracle.com Fri Feb 7 13:25:05 2014 From: staffan.larsen at oracle.com (Staffan Larsen) Date: Fri, 7 Feb 2014 14:25:05 +0100 Subject: RFR: JDK-8033917 Keep track of file paths in file streams and channels for instrumentation purposes In-Reply-To: <52F4DE07.7090003@oracle.com> References: <25552F6F-0FFC-42EA-9B0B-49EC135A5EE6@oracle.com> <52F4D149.8000900@oracle.com> <52F4DE07.7090003@oracle.com> Message-ID: Thanks Alan! On 7 feb 2014, at 14:22, Alan Bateman wrote: > On 07/02/2014 12:49, Staffan Larsen wrote: >> >> Updated webrev here: http://cr.openjdk.java.net/~sla/8033917/webrev.01/ >> > Thanks for the updates, looks good. > > -Alan. From staffan.larsen at oracle.com Fri Feb 7 13:25:43 2014 From: staffan.larsen at oracle.com (Staffan Larsen) Date: Fri, 7 Feb 2014 14:25:43 +0100 Subject: RFR(S): JDK-8033911 Simplify instrumentation of FileInputStream and RandomAccessFile In-Reply-To: <3d89f5ba-3c6e-4571-b7bf-19cbad4e5b5c@email.android.com> References: <2BA69765-5405-413A-BAC6-EDE77C176FFA@oracle.com> <3d89f5ba-3c6e-4571-b7bf-19cbad4e5b5c@email.android.com> Message-ID: Alan, Jaroslav, Dmitry: Thanks! On 7 feb 2014, at 12:38, Jaroslav Bachorik wrote: > Looks good. Additional benefit is the compliance with the secure coding guide. > > -JB- > > On February 7, 2014 11:46:07 AM CET, Staffan Larsen wrote: > A few of the public read and write methods in FileInputStream and RandomAccessFile are declared native. This means that it is hard to instrument them using byte code instrumentation. Changing the public methods to be to non-native and instead calling private native methods simplifies instrumentation. > > webrev: http://cr.openjdk.java.net/~sla/8033911/webrev.00/ > bug: https://bugs.openjdk.java.net/browse/JDK-8033911 > > Thanks, > /Staffan > > -- > Sent from my Android device with K-9 Mail. Please excuse my brevity. From forax at univ-mlv.fr Fri Feb 7 14:27:41 2014 From: forax at univ-mlv.fr (Remi Forax) Date: Fri, 07 Feb 2014 15:27:41 +0100 Subject: RFR: JDK-8032012, , String.toLowerCase/toUpperCase performance improvement In-Reply-To: <52F4D1F2.8000701@CoSoCo.de> References: <52D82013.7020009@oracle.com> <52DEEF45.7060506@oracle.com> <3ACAF47E-0B56-4D17-B322-506EAB79A168@oracle.com> <52F27BDB.6040705@oracle.com> <52F2864C.1030508@oracle.com> <52F2A670.60806@univ-mlv.fr> <52F2AD7D.1050609@oracle.com> <52F2C927.3030609@CoSoCo.de> <52F2CFCD.7050001@oracle.com> <52F3118F.4020902@oracle.com> <52F3C91C.1070800@oracle.com> <52F4D1F2.8000701@CoSoCo.de> Message-ID: <52F4ED5D.2070403@univ-mlv.fr> On 02/07/2014 01:30 PM, Ulf Zibis wrote: > Am 06.02.2014 18:40, schrieb Xueming Shen: >> Are we good enough to go? :-) While it took much longer than I would >> have >> expected, but I'm happy with the latest result. >> >> http://cr.openjdk.java.net/~sherman/8032012/webrev/ > > Except "if (first >= len)" instead "if (first = len)" I'm with you. > > -Ulf > Hi Ulf, if the JIT is not able to fold for(;i References: <52D82013.7020009@oracle.com> <52DEEF45.7060506@oracle.com> <3ACAF47E-0B56-4D17-B322-506EAB79A168@oracle.com> <52F27BDB.6040705@oracle.com> <52F2864C.1030508@oracle.com> <52F2A670.60806@univ-mlv.fr> <52F2AD7D.1050609@oracle.com> <52F2C927.3030609@CoSoCo.de> <52F2CFCD.7050001@oracle.com> <52F3118F.4020902@oracle.com> <52F3C91C.1070800@oracle.com> <52F4D1F2.8000701@CoSoCo.de> <52F4ED5D.2070403@univ-mlv.fr> Message-ID: I think JIT handles trivial loops with break fine. One issue here though is the overall method size and complexity (esp if some other methods are inlined into it). If the first loop represents a common/fast path (no surrogates and string already lower case) then I'd move the rest of the code out of line into a separate method. The only question is whether absence of upper case is really the common case; I've certainly seen a ton of code calling toLowerCase "just in case" where the vast majority, if not all, of the strings are already lower cased. Sent from my phone On 02/07/2014 01:30 PM, Ulf Zibis wrote: > Am 06.02.2014 18:40, schrieb Xueming Shen: > >> Are we good enough to go? :-) While it took much longer than I would have >> expected, but I'm happy with the latest result. >> >> http://cr.openjdk.java.net/~sherman/8032012/webrev/ >> > > Except "if (first >= len)" instead "if (first = len)" I'm with you. > > -Ulf > > Hi Ulf, if the JIT is not able to fold for(;i Since the topic of serialization has come up recently, I'll take it as an excuse to bring up a problem that I've run into a couple of times with the serialization specification, which has resulted in user problems. If you read section 2.3 [1] of the specification, it says: "The class's writeObject method, if implemented, is responsible for saving the state of the class. Either ObjectOutputStream's defaultWriteObject or writeFields method must be called once (and only once) before writing any optional data that will be needed by the corresponding readObject method to restore the state of the object; even if no optional data is written, defaultWriteObject or writeFields must still be invoked once. If defaultWriteObject or writeFields is not invoked once prior to the writing of optional data (if any), then the behavior of instance deserialization is undefined in cases where the ObjectInputStream cannot resolve the class which defined the writeObject method in question." If you go to section 3.4 [2] of the specification, it reads: "The readObject method of the class, if implemented, is responsible for restoring the state of the class. The values of every field of the object whether transient or not, static or not are set to the default value for the fields type. Either ObjectInputStream's defaultReadObject or readFields method must be called once (and only once) before reading any optional data written by the corresponding writeObject method; even if no optional data is read, defaultReadObject or readFields must still be invoked once." Now the problem: there are many classes in the wild which nevertheless do not write/read fields. We cause an exception in such cases rather than make up some undefined behavior. What I'm wondering is, is there some sensible behavior that could be specified for this case? The Oracle JDK seems to simply leave fields uninitialized in this case, maybe that can be a specified behavior? [1] http://docs.oracle.com/javase/7/docs/platform/serialization/spec/output.html#861 [2] http://docs.oracle.com/javase/7/docs/platform/serialization/spec/input.html#2971 -- - DML From dmitry.samersoff at oracle.com Fri Feb 7 15:24:00 2014 From: dmitry.samersoff at oracle.com (Dmitry Samersoff) Date: Fri, 07 Feb 2014 19:24:00 +0400 Subject: RFR: JDK-8033917 Keep track of file paths in file streams and channels for instrumentation purposes In-Reply-To: <25552F6F-0FFC-42EA-9B0B-49EC135A5EE6@oracle.com> References: <25552F6F-0FFC-42EA-9B0B-49EC135A5EE6@oracle.com> Message-ID: <52F4FA90.2080908@oracle.com> Staffan, FileInputStream.java 55: It's better to initialize path with null. 134: It's better to assign name at one of first lines, in this case we will be able to retrieve file name ever if open fails for some reason. 171: It's not necessary (the same is applicable to other files) I'm a bit scared changing signature of public methods of FileChannelImpl but if Alan says it's OK - lets go with it. -Dmitry On 2014-02-07 16:07, Staffan Larsen wrote: > Instrumentation agents that want to instrument > FileInputStream/FileOutputStream to see which files are being > accessed do not currently have access to the file system path of the > stream. This is because the path is never stored in the stream class, > only the file descriptor is. (This is also true for RandomAccessFile > and FileChannel). > > An agent could instrument the respective constructors to store the > path. The problem is where to store it. To add a field, the > instrumentation agent needs to change the schema of the class. This > is not possible at runtime but can be done at class-loading time. > However for a j.l.instrument agent these classes are already defined > when the agent is first called. For a native JVMTI agent the problem > becomes parsing and modifying byte codes in a native agent which is > error prone and requires a lot of code to maintain. > > If instead the stream classes were modified to store a reference to > the path, it would be readily available for agents at a minimum of > cost to the libraries. This is what this patch does. FileInputStream, > FileOutputStream, RandomAccessFile and FileChannelImpl are modified > to record the path they operate on in a private field. There are no > accessors added to retrieve the path - it is purely stored for > instrumentation purposes. The path is intentionally not resolved to > be an absolute path since that would potentially add unwanted > overhead. If a stream is created from a file descriptor, no path will > be stored. > > The overhead for this path will be keeping the path String alive for > a longer period of time. I hope this will not cause any problems. > > A consumer of this feature will be Java Flight Recorder, but the > implementation is usable by other agents as well. > > webrev: http://cr.openjdk.java.net/~sla/8033917/webrev.00/ bug: > https://bugs.openjdk.java.net/browse/JDK-8033917 > > Thanks, /Staffan > -- Dmitry Samersoff Oracle Java development team, Saint Petersburg, Russia * I would love to change the world, but they won't give me the sources. From Alan.Bateman at oracle.com Fri Feb 7 15:27:36 2014 From: Alan.Bateman at oracle.com (Alan Bateman) Date: Fri, 07 Feb 2014 15:27:36 +0000 Subject: RFR: JDK-8033917 Keep track of file paths in file streams and channels for instrumentation purposes In-Reply-To: <52F4FA90.2080908@oracle.com> References: <25552F6F-0FFC-42EA-9B0B-49EC135A5EE6@oracle.com> <52F4FA90.2080908@oracle.com> Message-ID: <52F4FB68.7040805@oracle.com> On 07/02/2014 15:24, Dmitry Samersoff wrote: > : > > I'm a bit scared changing signature of public methods of FileChannelImpl > but if Alan says it's OK - lets go with it. > This is sun.nio.ch and so JDK-internal, nothing outside of the JDK should be using it directly. -Alan From chris.hegarty at oracle.com Fri Feb 7 15:32:03 2014 From: chris.hegarty at oracle.com (Chris Hegarty) Date: Fri, 07 Feb 2014 15:32:03 +0000 Subject: RFR: JDK-8033917 Keep track of file paths in file streams and channels for instrumentation purposes In-Reply-To: <52F4FA90.2080908@oracle.com> References: <25552F6F-0FFC-42EA-9B0B-49EC135A5EE6@oracle.com> <52F4FA90.2080908@oracle.com> Message-ID: <52F4FC73.5080803@oracle.com> On 07/02/14 15:24, Dmitry Samersoff wrote: > Staffan, > > FileInputStream.java > > 55: It's better to initialize path with null. I'm afraid I disagree with this. The default value is already null, why set it to null again? I see this pattern all over the code, but it seems completely redundant to me. -Chris. > 134: It's better to assign name at one of first lines, in this case we > will be able to retrieve file name ever if open fails for some reason. > 171: It's not necessary > > (the same is applicable to other files) > > I'm a bit scared changing signature of public methods of FileChannelImpl > but if Alan says it's OK - lets go with it. > > -Dmitry > > > On 2014-02-07 16:07, Staffan Larsen wrote: >> Instrumentation agents that want to instrument >> FileInputStream/FileOutputStream to see which files are being >> accessed do not currently have access to the file system path of the >> stream. This is because the path is never stored in the stream class, >> only the file descriptor is. (This is also true for RandomAccessFile >> and FileChannel). >> >> An agent could instrument the respective constructors to store the >> path. The problem is where to store it. To add a field, the >> instrumentation agent needs to change the schema of the class. This >> is not possible at runtime but can be done at class-loading time. >> However for a j.l.instrument agent these classes are already defined >> when the agent is first called. For a native JVMTI agent the problem >> becomes parsing and modifying byte codes in a native agent which is >> error prone and requires a lot of code to maintain. >> >> If instead the stream classes were modified to store a reference to >> the path, it would be readily available for agents at a minimum of >> cost to the libraries. This is what this patch does. FileInputStream, >> FileOutputStream, RandomAccessFile and FileChannelImpl are modified >> to record the path they operate on in a private field. There are no >> accessors added to retrieve the path - it is purely stored for >> instrumentation purposes. The path is intentionally not resolved to >> be an absolute path since that would potentially add unwanted >> overhead. If a stream is created from a file descriptor, no path will >> be stored. >> >> The overhead for this path will be keeping the path String alive for >> a longer period of time. I hope this will not cause any problems. >> >> A consumer of this feature will be Java Flight Recorder, but the >> implementation is usable by other agents as well. >> >> webrev: http://cr.openjdk.java.net/~sla/8033917/webrev.00/ bug: >> https://bugs.openjdk.java.net/browse/JDK-8033917 >> >> Thanks, /Staffan >> > > From staffan.larsen at oracle.com Fri Feb 7 15:49:25 2014 From: staffan.larsen at oracle.com (Staffan Larsen) Date: Fri, 7 Feb 2014 16:49:25 +0100 Subject: RFR: JDK-8033917 Keep track of file paths in file streams and channels for instrumentation purposes In-Reply-To: <52F4FA90.2080908@oracle.com> References: <25552F6F-0FFC-42EA-9B0B-49EC135A5EE6@oracle.com> <52F4FA90.2080908@oracle.com> Message-ID: <3F1E5072-1745-4EA4-B783-93CAD2643B91@oracle.com> On 7 feb 2014, at 16:24, Dmitry Samersoff wrote: > Staffan, > > FileInputStream.java > > 55: It's better to initialize path with null. I agree with Chris here. The value should be explicitly initialized by all constructors. > 134: It's better to assign name at one of first lines, in this case we > will be able to retrieve file name ever if open fails for some reason. This is the constructor. If anything fails it will throw and exception, and there won?t be an object to look at. > 171: It's not necessary All constructors must initialize the value. Thanks, /Staffan > > (the same is applicable to other files) > > I'm a bit scared changing signature of public methods of FileChannelImpl > but if Alan says it's OK - lets go with it. > > -Dmitry > > > On 2014-02-07 16:07, Staffan Larsen wrote: >> Instrumentation agents that want to instrument >> FileInputStream/FileOutputStream to see which files are being >> accessed do not currently have access to the file system path of the >> stream. This is because the path is never stored in the stream class, >> only the file descriptor is. (This is also true for RandomAccessFile >> and FileChannel). >> >> An agent could instrument the respective constructors to store the >> path. The problem is where to store it. To add a field, the >> instrumentation agent needs to change the schema of the class. This >> is not possible at runtime but can be done at class-loading time. >> However for a j.l.instrument agent these classes are already defined >> when the agent is first called. For a native JVMTI agent the problem >> becomes parsing and modifying byte codes in a native agent which is >> error prone and requires a lot of code to maintain. >> >> If instead the stream classes were modified to store a reference to >> the path, it would be readily available for agents at a minimum of >> cost to the libraries. This is what this patch does. FileInputStream, >> FileOutputStream, RandomAccessFile and FileChannelImpl are modified >> to record the path they operate on in a private field. There are no >> accessors added to retrieve the path - it is purely stored for >> instrumentation purposes. The path is intentionally not resolved to >> be an absolute path since that would potentially add unwanted >> overhead. If a stream is created from a file descriptor, no path will >> be stored. >> >> The overhead for this path will be keeping the path String alive for >> a longer period of time. I hope this will not cause any problems. >> >> A consumer of this feature will be Java Flight Recorder, but the >> implementation is usable by other agents as well. >> >> webrev: http://cr.openjdk.java.net/~sla/8033917/webrev.00/ bug: >> https://bugs.openjdk.java.net/browse/JDK-8033917 >> >> Thanks, /Staffan >> > > > -- > Dmitry Samersoff > Oracle Java development team, Saint Petersburg, Russia > * I would love to change the world, but they won't give me the sources. From ivan.gerasimov at oracle.com Fri Feb 7 16:08:24 2014 From: ivan.gerasimov at oracle.com (Ivan Gerasimov) Date: Fri, 07 Feb 2014 20:08:24 +0400 Subject: RFR [8033943] Typo in the documentation for the class Arrays Message-ID: <52F504F8.2070804@oracle.com> Hello! I'm looking for the review of a trivial fix of a typo in the Java doc. s/briefs description/brief descriptions/ Bug: https://bugs.openjdk.java.net/browse/JDK-8033943 Webrev: http://cr.openjdk.java.net/~igerasim/8033943/0/webrev/ Sincerely yours, Ivan From martinrb at google.com Fri Feb 7 16:15:43 2014 From: martinrb at google.com (Martin Buchholz) Date: Fri, 7 Feb 2014 08:15:43 -0800 Subject: RFR [8033943] Typo in the documentation for the class Arrays In-Reply-To: <52F504F8.2070804@oracle.com> References: <52F504F8.2070804@oracle.com> Message-ID: Approved! On Fri, Feb 7, 2014 at 8:08 AM, Ivan Gerasimov wrote: > Hello! > > I'm looking for the review of a trivial fix of a typo in the Java doc. > > s/briefs description/brief descriptions/ > > Bug: https://bugs.openjdk.java.net/browse/JDK-8033943 > Webrev: http://cr.openjdk.java.net/~igerasim/8033943/0/webrev/ > > Sincerely yours, > Ivan > From dmitry.samersoff at oracle.com Fri Feb 7 16:18:07 2014 From: dmitry.samersoff at oracle.com (Dmitry Samersoff) Date: Fri, 07 Feb 2014 20:18:07 +0400 Subject: RFR: JDK-8033917 Keep track of file paths in file streams and channels for instrumentation purposes In-Reply-To: <52F4FC73.5080803@oracle.com> References: <25552F6F-0FFC-42EA-9B0B-49EC135A5EE6@oracle.com> <52F4FA90.2080908@oracle.com> <52F4FC73.5080803@oracle.com> Message-ID: <52F5073F.3090904@oracle.com> On 2014-02-07 19:32, Chris Hegarty wrote: > On 07/02/14 15:24, Dmitry Samersoff wrote: >> Staffan, >> >> FileInputStream.java >> >> 55: It's better to initialize path with null. > > I'm afraid I disagree with this. The default value is already null, why > set it to null again? I see this pattern all over the code, but it seems > completely redundant to me. Yes, It's NOOP but it makes readers and variety of "security" tools happy. I will not press for it, but as far as rest of the code (e.g. private FileChannel channel = null; ) uses this pattern and initialize variables explicitly, I think it's good to initialize this variable as well. -Dmitry > > -Chris. > >> 134: It's better to assign name at one of first lines, in this case we >> will be able to retrieve file name ever if open fails for some reason. >> 171: It's not necessary >> >> (the same is applicable to other files) >> >> I'm a bit scared changing signature of public methods of FileChannelImpl >> but if Alan says it's OK - lets go with it. >> >> -Dmitry >> >> >> On 2014-02-07 16:07, Staffan Larsen wrote: >>> Instrumentation agents that want to instrument >>> FileInputStream/FileOutputStream to see which files are being >>> accessed do not currently have access to the file system path of the >>> stream. This is because the path is never stored in the stream class, >>> only the file descriptor is. (This is also true for RandomAccessFile >>> and FileChannel). >>> >>> An agent could instrument the respective constructors to store the >>> path. The problem is where to store it. To add a field, the >>> instrumentation agent needs to change the schema of the class. This >>> is not possible at runtime but can be done at class-loading time. >>> However for a j.l.instrument agent these classes are already defined >>> when the agent is first called. For a native JVMTI agent the problem >>> becomes parsing and modifying byte codes in a native agent which is >>> error prone and requires a lot of code to maintain. >>> >>> If instead the stream classes were modified to store a reference to >>> the path, it would be readily available for agents at a minimum of >>> cost to the libraries. This is what this patch does. FileInputStream, >>> FileOutputStream, RandomAccessFile and FileChannelImpl are modified >>> to record the path they operate on in a private field. There are no >>> accessors added to retrieve the path - it is purely stored for >>> instrumentation purposes. The path is intentionally not resolved to >>> be an absolute path since that would potentially add unwanted >>> overhead. If a stream is created from a file descriptor, no path will >>> be stored. >>> >>> The overhead for this path will be keeping the path String alive for >>> a longer period of time. I hope this will not cause any problems. >>> >>> A consumer of this feature will be Java Flight Recorder, but the >>> implementation is usable by other agents as well. >>> >>> webrev: http://cr.openjdk.java.net/~sla/8033917/webrev.00/ bug: >>> https://bugs.openjdk.java.net/browse/JDK-8033917 >>> >>> Thanks, /Staffan >>> >> >> -- Dmitry Samersoff Oracle Java development team, Saint Petersburg, Russia * I would love to change the world, but they won't give me the sources. From dmitry.samersoff at oracle.com Fri Feb 7 16:19:11 2014 From: dmitry.samersoff at oracle.com (Dmitry Samersoff) Date: Fri, 07 Feb 2014 20:19:11 +0400 Subject: RFR: JDK-8033917 Keep track of file paths in file streams and channels for instrumentation purposes In-Reply-To: <3F1E5072-1745-4EA4-B783-93CAD2643B91@oracle.com> References: <25552F6F-0FFC-42EA-9B0B-49EC135A5EE6@oracle.com> <52F4FA90.2080908@oracle.com> <3F1E5072-1745-4EA4-B783-93CAD2643B91@oracle.com> Message-ID: <52F5077F.6000500@oracle.com> Staffan, OK! -Dmitry On 2014-02-07 19:49, Staffan Larsen wrote: > > On 7 feb 2014, at 16:24, Dmitry Samersoff wrote: > >> Staffan, >> >> FileInputStream.java >> >> 55: It's better to initialize path with null. > > I agree with Chris here. The value should be explicitly initialized by all constructors. > >> 134: It's better to assign name at one of first lines, in this case we >> will be able to retrieve file name ever if open fails for some reason. > > This is the constructor. If anything fails it will throw and exception, and there won?t be an object to look at. > >> 171: It's not necessary > > All constructors must initialize the value. > > Thanks, > /Staffan > >> >> (the same is applicable to other files) >> >> I'm a bit scared changing signature of public methods of FileChannelImpl >> but if Alan says it's OK - lets go with it. >> >> -Dmitry >> >> >> On 2014-02-07 16:07, Staffan Larsen wrote: >>> Instrumentation agents that want to instrument >>> FileInputStream/FileOutputStream to see which files are being >>> accessed do not currently have access to the file system path of the >>> stream. This is because the path is never stored in the stream class, >>> only the file descriptor is. (This is also true for RandomAccessFile >>> and FileChannel). >>> >>> An agent could instrument the respective constructors to store the >>> path. The problem is where to store it. To add a field, the >>> instrumentation agent needs to change the schema of the class. This >>> is not possible at runtime but can be done at class-loading time. >>> However for a j.l.instrument agent these classes are already defined >>> when the agent is first called. For a native JVMTI agent the problem >>> becomes parsing and modifying byte codes in a native agent which is >>> error prone and requires a lot of code to maintain. >>> >>> If instead the stream classes were modified to store a reference to >>> the path, it would be readily available for agents at a minimum of >>> cost to the libraries. This is what this patch does. FileInputStream, >>> FileOutputStream, RandomAccessFile and FileChannelImpl are modified >>> to record the path they operate on in a private field. There are no >>> accessors added to retrieve the path - it is purely stored for >>> instrumentation purposes. The path is intentionally not resolved to >>> be an absolute path since that would potentially add unwanted >>> overhead. If a stream is created from a file descriptor, no path will >>> be stored. >>> >>> The overhead for this path will be keeping the path String alive for >>> a longer period of time. I hope this will not cause any problems. >>> >>> A consumer of this feature will be Java Flight Recorder, but the >>> implementation is usable by other agents as well. >>> >>> webrev: http://cr.openjdk.java.net/~sla/8033917/webrev.00/ bug: >>> https://bugs.openjdk.java.net/browse/JDK-8033917 >>> >>> Thanks, /Staffan >>> >> >> >> -- >> Dmitry Samersoff >> Oracle Java development team, Saint Petersburg, Russia >> * I would love to change the world, but they won't give me the sources. > -- Dmitry Samersoff Oracle Java development team, Saint Petersburg, Russia * I would love to change the world, but they won't give me the sources. From Ulf.Zibis at CoSoCo.de Fri Feb 7 17:05:38 2014 From: Ulf.Zibis at CoSoCo.de (Ulf Zibis) Date: Fri, 07 Feb 2014 18:05:38 +0100 Subject: RFR: JDK-8032012, , String.toLowerCase/toUpperCase performance improvement In-Reply-To: <52F4ED5D.2070403@univ-mlv.fr> References: <52D82013.7020009@oracle.com> <52DEEF45.7060506@oracle.com> <3ACAF47E-0B56-4D17-B322-506EAB79A168@oracle.com> <52F27BDB.6040705@oracle.com> <52F2864C.1030508@oracle.com> <52F2A670.60806@univ-mlv.fr> <52F2AD7D.1050609@oracle.com> <52F2C927.3030609@CoSoCo.de> <52F2CFCD.7050001@oracle.com> <52F3118F.4020902@oracle.com> <52F3C91C.1070800@oracle.com> <52F4D1F2.8000701@CoSoCo.de> <52F4ED5D.2070403@univ-mlv.fr> Message-ID: <52F51262.9030302@CoSoCo.de> Am 07.02.2014 15:27, schrieb Remi Forax: > if the JIT is not able to fold > for(;i // may call break in the body > } > if (i == len) { > // ... > } > > given that it's a common pattern, > i think it's better to improve the JIT that to change the code. Then JIT must examine the complete for statement+block to prove that i will never become >len. I suspect such examination will be manageable. Am 07.02.2014 15:58, schrieb Vitaly Davidovich: > One issue here though is the overall method size and complexity (esp if some other methods are > inlined into it). To serve this, my favourite, even it may be less readable, is: find1stUpperCase: { for(;i I noticed recently that the javac scanner is making use of ArrayList.remove(0) when it consumes a buffer. Obviously this is an inefficient way to implement a buffer, so I thought I'd try to fix it [1]. ArrayDeque seems to provide just the behavior I need, with one fatal flaw: despite encoding its data with an array, the class exposes no random-access operations. For lookahead, I need to be able to call get(int). This seems to be a fairly common complaint [2][3]. I found an old bug requesting that ArrayDeque be enhanced to implement List [4], as well as a thread from 2010 that included a proof-of-concept ArrayDeque+List [5] and had a note from Martin Buchholz saying he regrets that ArrayDeque doesn't already implement List [6]. Is there any hope of ArrayDeque being enhanced in the near future to provide random access? There's some risk that any such initiative would be derailed by a quest for an uber-collection. I think a lot of people would be quite happy just to have a (trivial) 'get(int)' method added to ArrayDeque. ?Dan [1] https://bugs.openjdk.java.net/browse/JDK-8033158 [2] http://en.wikipedia.org/wiki/Deque#Language_support [3] https://www.google.com/#q=arraydeque+%22random+access%22 [4] https://bugs.openjdk.java.net/browse/JDK-6368844 [5] http://mail.openjdk.java.net/pipermail/core-libs-dev/2010-April/004038.html [6] http://mail.openjdk.java.net/pipermail/core-libs-dev/2010-April/004031.html From david.lloyd at redhat.com Fri Feb 7 18:07:42 2014 From: david.lloyd at redhat.com (David M. Lloyd) Date: Fri, 07 Feb 2014 12:07:42 -0600 Subject: Random access in ArrayDeque In-Reply-To: <231A3FB1-639F-4C06-823B-0CB9F1CB194A@oracle.com> References: <231A3FB1-639F-4C06-823B-0CB9F1CB194A@oracle.com> Message-ID: <52F520EE.70709@redhat.com> Just want to say that we've also had the need to implement an array-backed Deque+List; we found no surprises implementing it and thus I believe that extending ArrayDeque to implement List would be a positive change. Failing that, a combined ArrayListAndDeque class seems like a good idea. I think that calling it "Masters' thesis" material is perhaps overblowing the complexity a bit though. ;) Given that the basic algorithmic complexity of ArrayList is well-understood and is well-suited to many (some would say "most") applications, going for a O(sqrt(N)) middle insert/remove complexity would be something I would consider "scope creep"; LinkedList is usually a fine choice for these cases. On 02/07/2014 11:44 AM, Dan Smith wrote: > I noticed recently that the javac scanner is making use of ArrayList.remove(0) when it consumes a buffer. Obviously this is an inefficient way to implement a buffer, so I thought I'd try to fix it [1]. ArrayDeque seems to provide just the behavior I need, with one fatal flaw: despite encoding its data with an array, the class exposes no random-access operations. For lookahead, I need to be able to call get(int). > > This seems to be a fairly common complaint [2][3]. > > I found an old bug requesting that ArrayDeque be enhanced to implement List [4], as well as a thread from 2010 that included a proof-of-concept ArrayDeque+List [5] and had a note from Martin Buchholz saying he regrets that ArrayDeque doesn't already implement List [6]. > > Is there any hope of ArrayDeque being enhanced in the near future to provide random access? There's some risk that any such initiative would be derailed by a quest for an uber-collection. I think a lot of people would be quite happy just to have a (trivial) 'get(int)' method added to ArrayDeque. > > ?Dan > > [1] https://bugs.openjdk.java.net/browse/JDK-8033158 > [2] http://en.wikipedia.org/wiki/Deque#Language_support > [3] https://www.google.com/#q=arraydeque+%22random+access%22 > [4] https://bugs.openjdk.java.net/browse/JDK-6368844 > [5] http://mail.openjdk.java.net/pipermail/core-libs-dev/2010-April/004038.html > [6] http://mail.openjdk.java.net/pipermail/core-libs-dev/2010-April/004031.html > -- - DML From joe.darcy at oracle.com Fri Feb 7 18:23:34 2014 From: joe.darcy at oracle.com (Joe Darcy) Date: Fri, 07 Feb 2014 10:23:34 -0800 Subject: BigDecimal performance In-Reply-To: References: Message-ID: <52F524A6.4020700@oracle.com> Hello, FWIW, I filed and then eventually closed as will not fix JDK-4529368: RFE: Add a BigBinary class for arbitrary precision floating-point computation https://bugs.openjdk.java.net/browse/JDK-4529368 If such a BigBinary type existed, I would certainly use it when writing tests of double mathematical functions, but in the Java ecosystem, I'm not sure where else it would get used. The BigDecimal type supports both floating-point and fixed-point styles of computation. In floating-point computation, whatever the base, conceptually the number of digits of precision is fixed and a separate exponent field is used to represent values of different magnitudes. With BigDecimal, the precision can be varied by using the arithmetic methods which take a MathContext object as an argument. I'm not sure which algorithm you used to compute pi, but I would be very surprised by a ~8 X speed difference between BigInteger and BigDecimal if the same computation was being under on the BigDecimal values, meaning the BigIntegers held in the BigDecimals were the same as the BigIntegers in the BigInteger computation. A BigDecimal object is a fairly small wrapper around a BigInteger and updating the scale/exponent field isn't too much additional work. Cheers, -Joe On 02/07/2014 02:44 AM, Dmitry Nadezhin wrote: > I think that better name is BigBinary. > Both BigDecimal and BigBinary are floating-point number > with radix=10 and radix=2 respectively. > > More general approach is an abstract class or interface Rational > which has implementation subclasses. > Each nonzero rational number P/Q can be represented as > P/Q = p/q * 2^e , where e is integer, p and q are odd integers and GCD(p,q) > = 1. > Then BigBinary is a subclass with q=1. > Arithmetic operations on Rationals are implemented by general algorithm > when arguments are true > rationals (q!=1) and by specific algorithms when they are Binaries (q=1). > > This is elaborated here: > http://interval.louisiana.edu/reliable-computing-journal/volume-19/reliable-computing-19-pp-229-247.pdf > https://java.net/projects/jinterval/sources/svn/show/trunk/jinterval/jinterval-rational-java/src/main/java/net/java/jinterval/rational > > > > On Thu, Feb 6, 2014 at 9:11 PM, Tim Buktu wrote: > >> Hi, >> >> now that BigInteger deals better with large numbers, it would be nice >> for that to translate into an improvement in BigDecimal performance >> because BigDecimal is essentially a wrapper around BigInteger. >> Unfortunately, BigDecimal is still slower than BigInteger because it has >> to scale and round. >> >> I don't see a way to fix this without breaking the >> BigDecimal=BigInteger*10^n paradigm, but it could be done by introducing >> something like a BigFloat class that wraps a BigInteger such that >> BigFloat=BigInteger*2^n. I would expect the code to be less complex than >> BigDecimal because the only places it would have to deal with powers of >> ten would be conversion from and to String or BigDecimal. It would also >> be faster than BigDecimal for the same reason, but the downside is that >> it wouldn't accurately represent decimal fractions (just like float and >> double). >> >> Is this something that would be beneficial in the real world? >> >> I also did a little experiment to see how long a computation would take >> using BigDecimals vs the same computation using fixed-point BigInteger >> arithmetic. I wrote two programs that calculate pi to a million digits. >> The BigInteger version took 3 minutes; the BigDecimal version took 28 >> minutes (both single-threaded). >> >> Tim >> From dl at cs.oswego.edu Fri Feb 7 18:36:20 2014 From: dl at cs.oswego.edu (Doug Lea) Date: Fri, 07 Feb 2014 13:36:20 -0500 Subject: Draft JEP on enhanced volatiles Message-ID: <52F527A4.3030802@cs.oswego.edu> [To core-libs-dev; separately cross-posted to hotspot-dev and compiler-dev.] Now that the JMM update project is live (see http://openjdk.java.net/projects/jmm/), it would be helpful to get some preliminary feedback on one of its sure-to-be-needed counterparts: finally supporting a reasonable way to express atomic and/or ordered access operations beyond volatile-read/write. Some of you might know that the lack of these has often been discusses as a source of frustration and problems. One reason for nopt dealing with it sooner is the lack of a basis for even specifying such things, to be fixed with JMM update. But we also need some way for people to express them. The only good means for doing so appear to require a small amount of syntax support. Most people who have seen it seem to agree that the ".volatile" proposal below is the approach most likely to succeed. (We've considered a lot of others.) I'm hoping that members of libs, hotspot, and compiler groups can give it a sanity check before submitting (hopefully in a week or so). Thanks! Draft JEP proposal Title: Enhanced Volatiles Author: Doug Lea [...other metadata elided for now...] Summary ------- This JEP results in a means for programmers to invoke the equivalents of java.util.concurrent.atomic methods on volatile fields. Motivation ---------- As concurrent and parallel programming in Java continue to expand, programmers are increasingly frustrated by not being able to use Java constructions for arranging atomic operations for the fields of individual classes; for example atomically incrementing a "count" field. Until now the only ways to achieve these effects were to use a stand-alone AtomicInteger (adding both space overhead and additional concurrency issues to manage indirection) or, in some situations, to use atomic FieldUpdates (often encountering more overhead than the operation itself), or to use JVM Unsafe intrinsics. Because intrinsics are preferable on performance grounds, their use has been increasingly common, to the detriment of safety and portability. Without this JEP, these problems are expected to become worse as atomic APIs expand to cover additional access consistency policies (aligned with the recent C++11 memory model) as part of Java Memory Model revisions. Description ----------- The target solution requires a syntax enhancement, a few library enhancements, and compiler support. We model the extended operations on volatile integers via an interface VolatileInt, that also captures the functionality of AtomicInteger (which will also be updated to reflect Java Memory Model revisions as part of this JEP). A tentative version is below. Similar interfaces are needed for other primitive and reference types. We then enable access to corresponding methods for (typically volatile) fields using the ".volatile" prefix. For example: class Usage { volatile int count; int incrementCount() { return count.volatile.incrementAndGet(); } } The ".volatile" syntax is slightly unusual, but we are confident that it is syntactically unambiguous and semantically specifiable. New syntax is required to avoid ambiguities with existing usages, especially for volatile references -- invocations of methods on the reference versus the referent would be indistinguishable. The ".volatile" prefix introduces a scope for operations on these "L-values", not their retrieved contents. However, just using the prefix itself without a method invocation (as in "count.volatile;") would be meaningless and illegal. We also expect to allow volatile operations on array elements in addition to fields. The main task is to translate these calls into corresponding JVM intrinsics. The most likely option is for the source compiler to use method handles. This and other techniques are known to suffice, but are subject to further exploration. Minor enhancements to intrinsics and a few additional JDK library methods may also be needed. Here is a tentative VolatileInt interface. Those for other types are similar. The final released versions will surely differ in small ways. interface VolatileInt { int get(); int getRelaxed(); int getAcquire(); int getSequential(); void set(int x); void setRelaxed(int x); void setRelease(int x); void setSequential(int x); int getAndSet(int x); boolean compareAndSet(int e, int x); boolean compareAndSetAcquire(int e, int x); boolean compareAndSetRelease(int e, int x); boolean weakCompareAndSet(int e, int x); boolean weakCompareAndSetAcquire(int e, int x); boolean weakCompareAndSetRelease(int e, int x); int getAndAdd(int x); int addAndGet(int x); int getAndIncrement(); int incrementAndGet(); int getAndDecrement(); int decrementAndGet(); } Alternatives ------------ We considered instead introducing new forms of "value type" that support volatile operations. However, this would be inconsistent with properties of other types, and would also require more effort for programmers to use. We also considered expanding reliance on java.util.concurrent.atomic FieldUpdaters, but their dynamic overhead and usage limitations make them unsuitable. Several other alternatives (including those based on field references) have been raised and dismissed as unworkable on syntactic, efficiency, and/or usability grounds over the many years that these issues has been discussed. Risks and Assumptions --------------------- We are confident of feasibility. However, we expect that it will require more experimentation to arrive at compilation techniques that result in efficient enough implementation for routine use in the performance-critical contexts where these constructs are most often needed. The use of method handles may be impacted by and may impact JVM method handle support. Impact ------ A large number of usages in java.util.concurrent (and a few elsewhere in JDK) could be simplified and updated to use this support. From me at noctarius.com Fri Feb 7 18:43:49 2014 From: me at noctarius.com (Noctarius) Date: Fri, 07 Feb 2014 19:43:49 +0100 Subject: Draft JEP on enhanced volatiles In-Reply-To: <52F527A4.3030802@cs.oswego.edu> References: <52F527A4.3030802@cs.oswego.edu> Message-ID: <52F52965.3030503@noctarius.com> Hey Doug, The syntax is definitely weird and unusual but from users point I think I could live with that :) Thanks for this proposal, absolutely something we were waiting for. Chris Am 07.02.2014 19:36, schrieb Doug Lea: > > [To core-libs-dev; separately cross-posted to hotspot-dev and > compiler-dev.] > > Now that the JMM update project is live (see > http://openjdk.java.net/projects/jmm/), it would be helpful to > get some preliminary feedback on one of its sure-to-be-needed > counterparts: finally supporting a reasonable way to express > atomic and/or ordered access operations beyond > volatile-read/write. Some of you might know that the lack of > these has often been discusses as a source of frustration and > problems. One reason for nopt dealing with it sooner is the > lack of a basis for even specifying such things, to be fixed > with JMM update. But we also need some way for people to > express them. The only good means for doing so appear to > require a small amount of syntax support. Most people who have > seen it seem to agree that the ".volatile" proposal below is > the approach most likely to succeed. (We've considered a lot of > others.) I'm hoping that members of libs, hotspot, and compiler > groups can give it a sanity check before submitting (hopefully > in a week or so). Thanks! > > Draft JEP proposal Title: Enhanced Volatiles Author: Doug Lea > [...other metadata elided for now...] > > > Summary ------- > > This JEP results in a means for programmers to invoke the > equivalents of java.util.concurrent.atomic methods on volatile > fields. > > Motivation ---------- > > As concurrent and parallel programming in Java continue to > expand, programmers are increasingly frustrated by not being > able to use Java constructions for arranging atomic operations > for the fields of individual classes; for example atomically > incrementing a "count" field. Until now the only ways to > achieve these effects were to use a stand-alone AtomicInteger > (adding both space overhead and additional concurrency issues > to manage indirection) or, in some situations, to use atomic > FieldUpdates (often encountering more overhead than the > operation itself), or to use JVM Unsafe intrinsics. Because > intrinsics are preferable on performance grounds, their use has > been increasingly common, to the detriment of safety and > portability. Without this JEP, these problems are expected to > become worse as atomic APIs expand to cover additional access > consistency policies (aligned with the recent C++11 memory > model) as part of Java Memory Model revisions. > > Description ----------- > > The target solution requires a syntax enhancement, a few > library enhancements, and compiler support. > > We model the extended operations on volatile integers via an > interface VolatileInt, that also captures the functionality of > AtomicInteger (which will also be updated to reflect Java > Memory Model revisions as part of this JEP). A tentative > version is below. Similar interfaces are needed for other > primitive and reference types. > > We then enable access to corresponding methods for (typically > volatile) fields using the ".volatile" prefix. For example: > > class Usage { volatile int count; int incrementCount() { return > count.volatile.incrementAndGet(); } } > > The ".volatile" syntax is slightly unusual, but we are > confident that it is syntactically unambiguous and semantically > specifiable. New syntax is required to avoid ambiguities with > existing usages, especially for volatile references -- > invocations of methods on the reference versus the referent > would be indistinguishable. The ".volatile" prefix introduces > a scope for operations on these "L-values", not their retrieved > contents. However, just using the prefix itself without a > method invocation (as in "count.volatile;") would be > meaningless and illegal. We also expect to allow volatile > operations on array elements in addition to fields. > > The main task is to translate these calls into corresponding > JVM intrinsics. The most likely option is for the source > compiler to use method handles. This and other techniques are > known to suffice, but are subject to further exploration. Minor > enhancements to intrinsics and a few additional JDK library > methods may also be needed. > > Here is a tentative VolatileInt interface. Those for other > types are similar. The final released versions will surely > differ in small ways. > > interface VolatileInt { int get(); int getRelaxed(); int > getAcquire(); int getSequential(); > > void set(int x); void setRelaxed(int x); void setRelease(int > x); void setSequential(int x); > > int getAndSet(int x); boolean compareAndSet(int e, int x); > boolean compareAndSetAcquire(int e, int x); boolean > compareAndSetRelease(int e, int x); boolean > weakCompareAndSet(int e, int x); boolean > weakCompareAndSetAcquire(int e, int x); boolean > weakCompareAndSetRelease(int e, int x); > > int getAndAdd(int x); int addAndGet(int x); int > getAndIncrement(); int incrementAndGet(); int > getAndDecrement(); int decrementAndGet(); } > > > Alternatives ------------ > > We considered instead introducing new forms of "value type" > that support volatile operations. However, this would be > inconsistent with properties of other types, and would also > require more effort for programmers to use. We also considered > expanding reliance on java.util.concurrent.atomic > FieldUpdaters, but their dynamic overhead and usage limitations > make them unsuitable. Several other alternatives (including > those based on field references) have been raised and dismissed > as unworkable on syntactic, efficiency, and/or usability > grounds over the many years that these issues has been > discussed. > > > Risks and Assumptions --------------------- > > We are confident of feasibility. However, we expect that it > will require more experimentation to arrive at compilation > techniques that result in efficient enough implementation for > routine use in the performance-critical contexts where these > constructs are most often needed. The use of method handles may > be impacted by and may impact JVM method handle support. > > Impact ------ > > A large number of usages in java.util.concurrent (and a few > elsewhere in JDK) could be simplified and updated to use this > support. > From rob.mckenna at oracle.com Fri Feb 7 18:44:32 2014 From: rob.mckenna at oracle.com (Rob McKenna) Date: Fri, 07 Feb 2014 18:44:32 +0000 Subject: RFR: 7152892: some jtreg tests fail with permission denied Message-ID: <52F52990.5060200@oracle.com> Hi folks, When files are copied by our test harness the permissions are left unchanged. This can cause trouble for a few tests when the test folder is read only. http://cr.openjdk.java.net/~robm/7152892/webrev.01/ -Rob From david.lloyd at redhat.com Fri Feb 7 18:49:50 2014 From: david.lloyd at redhat.com (David M. Lloyd) Date: Fri, 07 Feb 2014 12:49:50 -0600 Subject: Draft JEP on enhanced volatiles In-Reply-To: <52F527A4.3030802@cs.oswego.edu> References: <52F527A4.3030802@cs.oswego.edu> Message-ID: <52F52ACE.6070107@redhat.com> I think making the description contingent on a specific syntax might be a mistake. I think that making required performance characteristics (e.g. "must be equivalent in performance to Unsafe counterparts") a part of acceptance criteria is also something that should be done. On 02/07/2014 12:36 PM, Doug Lea wrote: > > [To core-libs-dev; separately cross-posted to hotspot-dev and > compiler-dev.] > > Now that the JMM update project is live > (see http://openjdk.java.net/projects/jmm/), > it would be helpful to get some preliminary feedback on > one of its sure-to-be-needed counterparts: finally supporting > a reasonable way to express atomic and/or ordered access > operations beyond volatile-read/write. Some of you might > know that the lack of these has often been discusses as a > source of frustration and problems. One reason for nopt > dealing with it sooner is the lack of a basis for even > specifying such things, to be fixed with JMM update. > But we also need some way for people to express them. The only > good means for doing so appear to require a small amount of syntax > support. Most people who have seen it seem to agree that the > ".volatile" proposal below is the approach most likely to > succeed. (We've considered a lot of others.) I'm hoping that > members of libs, hotspot, and compiler groups can give it a > sanity check before submitting (hopefully in a week or so). > Thanks! > > Draft JEP proposal > Title: Enhanced Volatiles > Author: Doug Lea > [...other metadata elided for now...] > > > Summary > ------- > > This JEP results in a means for programmers to invoke the equivalents > of java.util.concurrent.atomic methods on volatile fields. > > Motivation > ---------- > > As concurrent and parallel programming in Java continue to expand, > programmers are increasingly frustrated by not being able to use Java > constructions for arranging atomic operations for the fields of > individual classes; for example atomically incrementing a "count" > field. Until now the only ways to achieve these effects were to use a > stand-alone AtomicInteger (adding both space overhead and additional > concurrency issues to manage indirection) or, in some situations, to > use atomic FieldUpdates (often encountering more overhead than the > operation itself), or to use JVM Unsafe intrinsics. Because intrinsics > are preferable on performance grounds, their use has been increasingly > common, to the detriment of safety and portability. Without this JEP, > these problems are expected to become worse as atomic APIs expand to > cover additional access consistency policies (aligned with the recent > C++11 memory model) as part of Java Memory Model revisions. > > Description > ----------- > > The target solution requires a syntax enhancement, a few > library enhancements, and compiler support. > > We model the extended operations on volatile integers via an interface > VolatileInt, that also captures the functionality of AtomicInteger > (which will also be updated to reflect Java Memory Model revisions as > part of this JEP). A tentative version is below. Similar interfaces > are needed for other primitive and reference types. > > We then enable access to corresponding methods for (typically > volatile) fields using the ".volatile" prefix. For example: > > class Usage { > volatile int count; > int incrementCount() { > return count.volatile.incrementAndGet(); > } > } > > The ".volatile" syntax is slightly unusual, but we are confident that > it is syntactically unambiguous and semantically specifiable. New > syntax is required to avoid ambiguities with existing usages, > especially for volatile references -- invocations of methods on the > reference versus the referent would be indistinguishable. The > ".volatile" prefix introduces a scope for operations on these > "L-values", not their retrieved contents. However, just using the > prefix itself without a method invocation (as in "count.volatile;") > would be meaningless and illegal. We also expect to allow volatile > operations on array elements in addition to fields. > > The main task is to translate these calls into corresponding JVM > intrinsics. The most likely option is for the source compiler to use > method handles. This and other techniques are known to suffice, but > are subject to further exploration. Minor enhancements to intrinsics > and a few additional JDK library methods may also be needed. > > Here is a tentative VolatileInt interface. Those for other types are > similar. The final released versions will surely differ in small > ways. > > interface VolatileInt { > int get(); > int getRelaxed(); > int getAcquire(); > int getSequential(); > > void set(int x); > void setRelaxed(int x); > void setRelease(int x); > void setSequential(int x); > > int getAndSet(int x); > boolean compareAndSet(int e, int x); > boolean compareAndSetAcquire(int e, int x); > boolean compareAndSetRelease(int e, int x); > boolean weakCompareAndSet(int e, int x); > boolean weakCompareAndSetAcquire(int e, int x); > boolean weakCompareAndSetRelease(int e, int x); > > int getAndAdd(int x); > int addAndGet(int x); > int getAndIncrement(); > int incrementAndGet(); > int getAndDecrement(); > int decrementAndGet(); > } > > > Alternatives > ------------ > > We considered instead introducing new forms of "value type" that > support volatile operations. However, this would be inconsistent with > properties of other types, and would also require more effort for > programmers to use. We also considered expanding reliance on > java.util.concurrent.atomic FieldUpdaters, but their dynamic overhead > and usage limitations make them unsuitable. Several other alternatives > (including those based on field references) have been raised and > dismissed as unworkable on syntactic, efficiency, and/or usability > grounds over the many years that these issues has been discussed. > > > Risks and Assumptions > --------------------- > > We are confident of feasibility. However, we expect that it will > require more experimentation to arrive at compilation techniques that > result in efficient enough implementation for routine use in the > performance-critical contexts where these constructs are most often > needed. The use of method handles may be impacted by and may impact > JVM method handle support. > > Impact > ------ > > A large number of usages in java.util.concurrent (and a few elsewhere > in JDK) could be simplified and updated to use this support. > -- - DML From martinrb at google.com Fri Feb 7 18:59:51 2014 From: martinrb at google.com (Martin Buchholz) Date: Fri, 7 Feb 2014 10:59:51 -0800 Subject: Random access in ArrayDeque In-Reply-To: <52F520EE.70709@redhat.com> References: <231A3FB1-639F-4C06-823B-0CB9F1CB194A@oracle.com> <52F520EE.70709@redhat.com> Message-ID: Sorry we're all such lamers. ArrayDeque should implement most of the methods in List, notably get(i). ArrayDeque should not actually implement List itself, because the change in contract/behavior for hashCode/equals would be too great. But we can provide a List asList() method. If we have agreement, we can do the relatively easy implementation. On Fri, Feb 7, 2014 at 10:07 AM, David M. Lloyd wrote: > Just want to say that we've also had the need to implement an array-backed > Deque+List; we found no surprises implementing it and thus I believe that > extending ArrayDeque to implement List would be a positive change. Failing > that, a combined ArrayListAndDeque class seems like a good idea. > > I think that calling it "Masters' thesis" material is perhaps overblowing > the complexity a bit though. ;) Given that the basic algorithmic > complexity of ArrayList is well-understood and is well-suited to many (some > would say "most") applications, going for a O(sqrt(N)) middle insert/remove > complexity would be something I would consider "scope creep"; LinkedList is > usually a fine choice for these cases. > > > On 02/07/2014 11:44 AM, Dan Smith wrote: > >> I noticed recently that the javac scanner is making use of >> ArrayList.remove(0) when it consumes a buffer. Obviously this is an >> inefficient way to implement a buffer, so I thought I'd try to fix it [1]. >> ArrayDeque seems to provide just the behavior I need, with one fatal flaw: >> despite encoding its data with an array, the class exposes no random-access >> operations. For lookahead, I need to be able to call get(int). >> >> This seems to be a fairly common complaint [2][3]. >> >> I found an old bug requesting that ArrayDeque be enhanced to implement >> List [4], as well as a thread from 2010 that included a proof-of-concept >> ArrayDeque+List [5] and had a note from Martin Buchholz saying he regrets >> that ArrayDeque doesn't already implement List [6]. >> >> Is there any hope of ArrayDeque being enhanced in the near future to >> provide random access? There's some risk that any such initiative would be >> derailed by a quest for an uber-collection. I think a lot of people would >> be quite happy just to have a (trivial) 'get(int)' method added to >> ArrayDeque. >> >> ?Dan >> >> [1] https://bugs.openjdk.java.net/browse/JDK-8033158 >> [2] http://en.wikipedia.org/wiki/Deque#Language_support >> [3] https://www.google.com/#q=arraydeque+%22random+access%22 >> [4] https://bugs.openjdk.java.net/browse/JDK-6368844 >> [5] http://mail.openjdk.java.net/pipermail/core-libs-dev/2010- >> April/004038.html >> [6] http://mail.openjdk.java.net/pipermail/core-libs-dev/2010- >> April/004031.html >> >> > > -- > - DML > From joe.darcy at oracle.com Fri Feb 7 19:01:28 2014 From: joe.darcy at oracle.com (Joe Darcy) Date: Fri, 07 Feb 2014 11:01:28 -0800 Subject: 8014320 ready for review In-Reply-To: References: Message-ID: <52F52D88.5060101@oracle.com> Hi Tim, Thanks for the patches. Brian and I spoke recently about possible math library projects for JDK 9. We were planning to prioritize solidifying the implementation of the algorithms currently in BigInteger (better tuning of thresholds, improve memory usage, etc.) over adding new algorithms which would make really large values run faster. Cheers, -Joe On 02/06/2014 09:22 AM, Tim Buktu wrote: > Hi, > > 8014320 is a JDK9 enhancement and everybody is probably busy with JDK8, > but I wanted to let the BigInteger people know the patch is finished. It > consists of 3 patched files and 1 new file; they can be found at > > https://github.com/tbuktu/bigint/raw/5142a961252b2cd217377f32e4351640086c94e2/src/main/java/java/math/BigInteger.java > https://github.com/tbuktu/bigint/raw/fe807bf92b58cf8a4556b3f8473f1b6dd35412b7/src/main/java/java/math/MutableModFn.java > https://github.com/tbuktu/bigint/raw/3bd7329d93540a839582afb62c6ce40e243b3e47/src/main/java/java/math/MutableBigInteger.java > https://github.com/tbuktu/bigint/raw/557584cba766507fe18d0cca5e8f03f45b483223/src/test/java/BigIntegerTest.java > > The SS and Barrett thresholds are from timings on an Intel CPU. An AMD > CPU gave me lower values; I didn't try any other architectures. I used > the program at the link below to find the thresholds: > > https://github.com/tbuktu/bigint/blob/master/src/main/java/Tune.java > > Tim From martinrb at google.com Fri Feb 7 19:07:52 2014 From: martinrb at google.com (Martin Buchholz) Date: Fri, 7 Feb 2014 11:07:52 -0800 Subject: Random access in ArrayDeque In-Reply-To: References: <231A3FB1-639F-4C06-823B-0CB9F1CB194A@oracle.com> <52F520EE.70709@redhat.com> Message-ID: Can't finish if we don't start. Here's v0.1: No rocket science here. Index: src/main/java/util/ArrayDeque.java =================================================================== RCS file: /export/home/jsr166/jsr166/jsr166/src/main/java/util/ArrayDeque.java,v retrieving revision 1.57 diff -u -r1.57 ArrayDeque.java --- src/main/java/util/ArrayDeque.java 18 Jul 2013 18:21:22 -0000 1.57 +++ src/main/java/util/ArrayDeque.java 7 Feb 2014 19:06:15 -0000 @@ -291,6 +291,17 @@ return result; } + /** + * Implements {@link List#get(int)}. + */ + public E get(int index) { + if (index < 0 || index >= size()) + throw new IndexOutOfBoundsException(); + @SuppressWarnings("unchecked") + E result = (E) elements[(head + index) & (elements.length - 1)]; + return result; + } + @SuppressWarnings("unchecked") public E peekFirst() { // elements[head] is null if deque empty On Fri, Feb 7, 2014 at 10:59 AM, Martin Buchholz wrote: > Sorry we're all such lamers. > > ArrayDeque should implement most of the methods in List, notably get(i). > > ArrayDeque should not actually implement List itself, because the change > in contract/behavior for hashCode/equals would be too great. But we can > provide a > List asList() > method. > > If we have agreement, we can do the relatively easy implementation. > > > On Fri, Feb 7, 2014 at 10:07 AM, David M. Lloyd wrote: > >> Just want to say that we've also had the need to implement an >> array-backed Deque+List; we found no surprises implementing it and thus I >> believe that extending ArrayDeque to implement List would be a positive >> change. Failing that, a combined ArrayListAndDeque class seems like a good >> idea. >> >> I think that calling it "Masters' thesis" material is perhaps overblowing >> the complexity a bit though. ;) Given that the basic algorithmic >> complexity of ArrayList is well-understood and is well-suited to many (some >> would say "most") applications, going for a O(sqrt(N)) middle insert/remove >> complexity would be something I would consider "scope creep"; LinkedList is >> usually a fine choice for these cases. >> >> >> On 02/07/2014 11:44 AM, Dan Smith wrote: >> >>> I noticed recently that the javac scanner is making use of >>> ArrayList.remove(0) when it consumes a buffer. Obviously this is an >>> inefficient way to implement a buffer, so I thought I'd try to fix it [1]. >>> ArrayDeque seems to provide just the behavior I need, with one fatal flaw: >>> despite encoding its data with an array, the class exposes no random-access >>> operations. For lookahead, I need to be able to call get(int). >>> >>> This seems to be a fairly common complaint [2][3]. >>> >>> I found an old bug requesting that ArrayDeque be enhanced to implement >>> List [4], as well as a thread from 2010 that included a proof-of-concept >>> ArrayDeque+List [5] and had a note from Martin Buchholz saying he regrets >>> that ArrayDeque doesn't already implement List [6]. >>> >>> Is there any hope of ArrayDeque being enhanced in the near future to >>> provide random access? There's some risk that any such initiative would be >>> derailed by a quest for an uber-collection. I think a lot of people would >>> be quite happy just to have a (trivial) 'get(int)' method added to >>> ArrayDeque. >>> >>> ?Dan >>> >>> [1] https://bugs.openjdk.java.net/browse/JDK-8033158 >>> [2] http://en.wikipedia.org/wiki/Deque#Language_support >>> [3] https://www.google.com/#q=arraydeque+%22random+access%22 >>> [4] https://bugs.openjdk.java.net/browse/JDK-6368844 >>> [5] http://mail.openjdk.java.net/pipermail/core-libs-dev/2010- >>> April/004038.html >>> [6] http://mail.openjdk.java.net/pipermail/core-libs-dev/2010- >>> April/004031.html >>> >>> >> >> -- >> - DML >> > > From david.lloyd at redhat.com Fri Feb 7 19:46:26 2014 From: david.lloyd at redhat.com (David M. Lloyd) Date: Fri, 07 Feb 2014 13:46:26 -0600 Subject: Random access in ArrayDeque In-Reply-To: References: <231A3FB1-639F-4C06-823B-0CB9F1CB194A@oracle.com> <52F520EE.70709@redhat.com> Message-ID: <52F53812.8050700@redhat.com> On 02/07/2014 12:59 PM, Martin Buchholz wrote: > Sorry we're all such lamers. ?? > ArrayDeque should implement most of the methods in List, notably get(i). > > ArrayDeque should not actually implement List itself, because the change > in contract/behavior for hashCode/equals would be too great. Extending ArrayList to implement Deque might avoid this? Though I suppose the chance of exciting new conflicts "in the wild" is higher given that ArrayList has been around since 1.2. > But we can provide a List asList() method. If this path is chosen, it could be perhaps interesting to have ArrayList and ArrayDeque act as mutual views for each other, though I'd hate to require the extra object when (as LinkedList shows) it is conceptually OK to implement List and Deque. > If we have agreement, we can do the relatively easy implementation. > > > On Fri, Feb 7, 2014 at 10:07 AM, David M. Lloyd > wrote: > > Just want to say that we've also had the need to implement an > array-backed Deque+List; we found no surprises implementing it and > thus I believe that extending ArrayDeque to implement List would be > a positive change. Failing that, a combined ArrayListAndDeque class > seems like a good idea. > > I think that calling it "Masters' thesis" material is perhaps > overblowing the complexity a bit though. ;) Given that the basic > algorithmic complexity of ArrayList is well-understood and is > well-suited to many (some would say "most") applications, going for > a O(sqrt(N)) middle insert/remove complexity would be something I > would consider "scope creep"; LinkedList is usually a fine choice > for these cases. > > > On 02/07/2014 11:44 AM, Dan Smith wrote: > > I noticed recently that the javac scanner is making use of > ArrayList.remove(0) when it consumes a buffer. Obviously this > is an inefficient way to implement a buffer, so I thought I'd > try to fix it [1]. ArrayDeque seems to provide just the > behavior I need, with one fatal flaw: despite encoding its data > with an array, the class exposes no random-access operations. > For lookahead, I need to be able to call get(int). > > This seems to be a fairly common complaint [2][3]. > > I found an old bug requesting that ArrayDeque be enhanced to > implement List [4], as well as a thread from 2010 that included > a proof-of-concept ArrayDeque+List [5] and had a note from > Martin Buchholz saying he regrets that ArrayDeque doesn't > already implement List [6]. > > Is there any hope of ArrayDeque being enhanced in the near > future to provide random access? There's some risk that any > such initiative would be derailed by a quest for an > uber-collection. I think a lot of people would be quite happy > just to have a (trivial) 'get(int)' method added to ArrayDeque. > > ?Dan > > [1] https://bugs.openjdk.java.net/__browse/JDK-8033158 > > [2] http://en.wikipedia.org/wiki/__Deque#Language_support > > [3] https://www.google.com/#q=__arraydeque+%22random+access%22 > > [4] https://bugs.openjdk.java.net/__browse/JDK-6368844 > > [5] > http://mail.openjdk.java.net/__pipermail/core-libs-dev/2010-__April/004038.html > > [6] > http://mail.openjdk.java.net/__pipermail/core-libs-dev/2010-__April/004031.html > > > > > -- > - DML > > -- - DML From martinrb at google.com Fri Feb 7 20:24:44 2014 From: martinrb at google.com (Martin Buchholz) Date: Fri, 7 Feb 2014 12:24:44 -0800 Subject: Random access in ArrayDeque In-Reply-To: <52F53812.8050700@redhat.com> References: <231A3FB1-639F-4C06-823B-0CB9F1CB194A@oracle.com> <52F520EE.70709@redhat.com> <52F53812.8050700@redhat.com> Message-ID: On Fri, Feb 7, 2014 at 11:46 AM, David M. Lloyd wrote: > On 02/07/2014 12:59 PM, Martin Buchholz wrote: > >> >> ArrayDeque should not actually implement List itself, because the change >> in contract/behavior for hashCode/equals would be too great. >> > > Extending ArrayList to implement Deque might avoid this? Though I suppose > the chance of exciting new conflicts "in the wild" is higher given that > ArrayList has been around since 1.2. > > I would have preferred a world where there was only one ArrayList class that implemented the circular array idea of ArrayDeque, but it seems too late. Even today I would support circular-array-ification of ArrayList, and ArrayList can compatibly implement Deque I think, but we're unlikely to be able to reach consensus on that. Applications have learned that ArrayList is append-only, and so adding circular array mechanics would add a small cost on all operations. Which is probably why it wasn't done back in 1.6. > > But we can provide a List asList() method. >> > > If this path is chosen, it could be perhaps interesting to have ArrayList > and ArrayDeque act as mutual views for each other, though I'd hate to > require the extra object when (as LinkedList shows) it is conceptually OK > to implement List and Deque. > > You can extend a List to implement Deque, but not the other way around. From brian.goetz at oracle.com Fri Feb 7 20:44:33 2014 From: brian.goetz at oracle.com (Brian Goetz) Date: Fri, 07 Feb 2014 15:44:33 -0500 Subject: Draft JEP on enhanced volatiles In-Reply-To: <52F527A4.3030802@cs.oswego.edu> References: <52F527A4.3030802@cs.oswego.edu> Message-ID: <52F545B1.2070706@oracle.com> It's about time we brought atomic ops out of the shadows of Unsafe and into the programming model proper! It was a good compromise ten years ago to stuff these into Unsafe, but its time for these features to grow up. Specific to this proposal, I like how the VolatileXxx interface approach allows us to add new fencing modes and/or atomic operations should we discover that the initial set is deficient. I also like how this potentially gives us a consistent story for volatile semantics on array elements. I can see a clean and straightforward implementation here at all the required touch points: language spec, memory model, type system, compiler, and VM runtime, and none of them require any special magic. (Having spent some time exploring this design space myself, I think the proposed surfacing is the least objectionable and most transparent of the options that I've seen.) Minor nits called out inline. Otherwise, Reviewed (as JDK reviewer) and Endorsed (as area lead). > We then enable access to corresponding methods for (typically > volatile) fields using the ".volatile" prefix. For example: Is this a prefix or a suffix? > The ".volatile" syntax is slightly unusual, but we are confident that > it is syntactically unambiguous and semantically specifiable. And, it is reasonably consistent with other similar uses of keywords for qualifying access to functionality, such as I.super or Outer.this. > We also expect to allow volatile > operations on array elements in addition to fields. This calls for the memory model to provide clear semantics for mixed accesses, such as volatile write / nonvolatile read. (The same would be true if we permitted the use of the .volatile modifier on nonvolatile fields.) > Several other alternatives > (including those based on field references) have been raised and > dismissed as unworkable on syntactic, efficiency, and/or usability > grounds or extensibility grounds. Many existing proposals might work for the operations in front of us today, but not scale as new fencing modes or atomic operations are discovered. > Risks and Assumptions > --------------------- > > We are confident of feasibility. However, we expect that it will > require more experimentation to arrive at compilation techniques that > result in efficient enough implementation for routine use in the > performance-critical contexts where these constructs are most often > needed. The use of method handles may be impacted by and may impact > JVM method handle support. As a limitation, I'd call out that its not obvious how this would scale to DCAS. > A large number of usages in java.util.concurrent (and a few elsewhere > in JDK) could be simplified and updated to use this support. Hooray! From guy.steele at oracle.com Fri Feb 7 21:09:07 2014 From: guy.steele at oracle.com (Guy Steele) Date: Fri, 7 Feb 2014 16:09:07 -0500 Subject: Draft JEP on enhanced volatiles In-Reply-To: <52F545B1.2070706@oracle.com> References: <52F527A4.3030802@cs.oswego.edu> <52F545B1.2070706@oracle.com> Message-ID: <3436632C-817E-4E83-B6B5-D3161EB0AD1C@oracle.com> On Feb 7, 2014, at 3:44 PM, Brian Goetz wrote: . . . > As a limitation, I'd call out that its not obvious how this would scale to DCAS. Just off the top of my head: (var1, var2).doubleCompareAndSet(e1, e2, x1, x2) thus providing access to a VolatileIntInt interface that provides the doubleCompareAndSet method. ?Guy From Gary.Frost at amd.com Fri Feb 7 21:25:40 2014 From: Gary.Frost at amd.com (Frost, Gary) Date: Fri, 7 Feb 2014 21:25:40 +0000 Subject: Draft JEP on enhanced volatiles In-Reply-To: <52F545B1.2070706@oracle.com> References: <52F527A4.3030802@cs.oswego.edu> <52F545B1.2070706@oracle.com> Message-ID: WRT to volatile. Is there anyway to extend the notion/contract of volatile to access to array elements. So of course volatile int[] arr = new int[2]; Just offers the contract to the array reference 'arr' What about multiple threads wanting some contract between their uses of arr[1] and arr[2]? Just thought I would throw this into the mix.. Gary -----Original Message----- From: core-libs-dev-bounces at openjdk.java.net [mailto:core-libs-dev-bounces at openjdk.java.net] On Behalf Of Brian Goetz Sent: Friday, February 07, 2014 2:45 PM To: Doug Lea; core-libs-dev Subject: Re: Draft JEP on enhanced volatiles It's about time we brought atomic ops out of the shadows of Unsafe and into the programming model proper! It was a good compromise ten years ago to stuff these into Unsafe, but its time for these features to grow up. Specific to this proposal, I like how the VolatileXxx interface approach allows us to add new fencing modes and/or atomic operations should we discover that the initial set is deficient. I also like how this potentially gives us a consistent story for volatile semantics on array elements. I can see a clean and straightforward implementation here at all the required touch points: language spec, memory model, type system, compiler, and VM runtime, and none of them require any special magic. (Having spent some time exploring this design space myself, I think the proposed surfacing is the least objectionable and most transparent of the options that I've seen.) Minor nits called out inline. Otherwise, Reviewed (as JDK reviewer) and Endorsed (as area lead). > We then enable access to corresponding methods for (typically > volatile) fields using the ".volatile" prefix. For example: Is this a prefix or a suffix? > The ".volatile" syntax is slightly unusual, but we are confident that > it is syntactically unambiguous and semantically specifiable. And, it is reasonably consistent with other similar uses of keywords for qualifying access to functionality, such as I.super or Outer.this. > We also expect to allow volatile > operations on array elements in addition to fields. This calls for the memory model to provide clear semantics for mixed accesses, such as volatile write / nonvolatile read. (The same would be true if we permitted the use of the .volatile modifier on nonvolatile fields.) > Several other alternatives > (including those based on field references) have been raised and > dismissed as unworkable on syntactic, efficiency, and/or usability > grounds or extensibility grounds. Many existing proposals might work for the operations in front of us today, but not scale as new fencing modes or atomic operations are discovered. > Risks and Assumptions > --------------------- > > We are confident of feasibility. However, we expect that it will > require more experimentation to arrive at compilation techniques that > result in efficient enough implementation for routine use in the > performance-critical contexts where these constructs are most often > needed. The use of method handles may be impacted by and may impact > JVM method handle support. As a limitation, I'd call out that its not obvious how this would scale to DCAS. > A large number of usages in java.util.concurrent (and a few elsewhere > in JDK) could be simplified and updated to use this support. Hooray! From snazy at snazy.de Fri Feb 7 21:35:32 2014 From: snazy at snazy.de (Robert Stupp) Date: Fri, 7 Feb 2014 22:35:32 +0100 Subject: Object allocation tracing / profilers In-Reply-To: References: <3486E1B1-8059-4362-B0B1-30A84E2C199D@snazy.de> Message-ID: <1AE39B78-ED80-4BD7-B949-B0103D3D1381@snazy.de> Jeremy, if you have a patch for JDK8 or 9 and some short instructions, i'd like to try it (OSX or Linux) :) I think it's ok if there's no way to do JDK callbacks (as long as the class, array length, thread and maybe (piece of) the its stack trace is available). But for now I only need the class and array length. Regarding the fact, that profiler output regarding allocated objects is wrong (compared to non-instrumented code) there should be a solution. I'm afraid that many others optimize their code by considering the output of a profiler as "the truth". Since all numbers (either object allocations or method runtimes) are based on instrumented code - they must be considered an at least airy estimate with a probability to be completely different compared to non-profiled/instrumented code. Maybe it is possible to extend the official JVMTI API to provide a new profiling API. IMO it would be faster and less intrusive and therefore better comparable against a real production system. That API would perform callbacks with information about the object, its class, array length, thread and stack trace (up to a configurable length, possibly empty). It should be considered that only a specific set of threads, set of classes is of interest. Such an API would need to provide a part to track method calls with similar filters - it would not need to do any callbacks for inlined methods. Stack trace information could also include information which method is compiled. The events could be placed by application threads in some queue and a separate thread could poll for events and perform the callbacks - means that the real profiling code is independent from the application code. Instrumentation itself could still be used for more complicated stuff like hunting for handle leaks (file, jdbc, etc). These are only some thoughts. Robert Am 07.02.2014 um 02:13 schrieb Jeremy Manson : > You aren't doing anything wrong. More detail: > > - Anything bytecode based (like my instrumenter, which you tried) isn't going to let you know what has been escape analyzed away. You can only do bytecode rewriting before escape analysis runs, and before the escape analysis runs, there is no way to find out what will be escape analyzed away. > > - A bytecode-based instrumenter will also not pick up allocations that occur via JNI, which is the problem that VMObjectAlloc solves. > > - If you happen to be on a platform that supports it, and you've compiled the JDK correctly, you can use dtrace to capture Java allocations. > > - Because of its limitations and performance issues, we've actually stopped doing bytecode-based instrumentation at Google in favor of instrumenting the VM directly. We now use JVMTI-style callbacks whenever an allocation happens. If people are interested, I can provide the patch (in fact, I forward ported it to JDK8 yesterday!), but it is rather user unfriendly, so you have to be rather ambitious to use it. On the plus side, it can take into account escape analyzed objects (right now, it doesn't instrument them, but this could conceivably be changed). > > - On the down side, if you take this approach, you can't call back into Java from the JVM callbacks. This is because calling back into Java may require the JVM to come to a safepoint, which you can't do in the middle of allocation (which is where the callback would occur). > > Jeremy > > > > On Thu, Feb 6, 2014 at 2:36 PM, Robert Stupp wrote: > Hi, > > I'm trying to trace/count object allocations - especially regarding the ObjectIn/OutputStream stuff. > > I know two API points: JVMTI and instrumentation. JVMTI fires an event for nearly everything except allocations from bytecode - instrumentation allows to transform bytecode (e.g. to issue a callback on object allocation). > > But I am not very satisfied - maybe I oversee something... > What I need is the number of "real" allocations. I do not want to count allocations that Hotspot would normally eliminate - for example Hotspot seems to eliminate instances of ArrayList$Itr under some circumstances. > > I played around with this instrumentation code: https://code.google.com/p/java-allocation-instrumenter/ > But this one and any other profiler (JProfiler) I tried always gave me the theoretical number of instances of ArrayList$Itr. This reason seems to be as follows: > With instrumentation/transformation constructors get a piece of code "injected" which calls a "profile object allocation method". This method gets the new object instance, which is used to get the Class object and to inquire the shallow object size on the heap - and it gets the current thread's stack trace. > > My suggestion is, that Hotspot cannot eliminate object allocations (for example of ArrayList$Itr) because the profiler bytecode needs that object instance and does other things, that are "strange" to Hotspot. > > I hoped that I can get callbacks using JVMTI but due to the restrictions mentioned here (http://docs.oracle.com/javase/7/docs/platform/jvmti/jvmti.html#VMObjectAlloc) there are no callbacks for allocations caused by bytecode. > VMObjectAlloc events occur during the start and end of the application - but not a single event is fired while the application is running (I've attached my JVMTI code). > > Am I doing anything wrong? Is there a possibility to count only the "real" instances? If not, wouldn't it be nice to have such a functionality in JVMTI or a callback functionality for http://download.java.net/jdk8/docs/api/java/lang/instrument/Instrumentation.html? > My fear is that all profiler results that all developers see are inaccurate because the code injected by profilers influences Hotspot too much. > > - > Robert > > From belliottsmith at datastax.com Fri Feb 7 21:41:11 2014 From: belliottsmith at datastax.com (Benedict Elliott Smith) Date: Fri, 7 Feb 2014 21:41:11 +0000 Subject: Draft JEP on enhanced volatiles In-Reply-To: References: <52F527A4.3030802@cs.oswego.edu> <52F545B1.2070706@oracle.com> Message-ID: > > Is there anyway to extend the notion/contract of volatile to access to > array elements. >From Doug's original email We also expect to allow volatile operations on array elements in addition > to fields. I'm quite excited about these changes, it looks like a great way to provide support outside of unsafe. On 7 February 2014 21:25, Frost, Gary wrote: > > > WRT to volatile. > > Is there anyway to extend the notion/contract of volatile to access to > array elements. > > So of course > > volatile int[] arr = new int[2]; > > Just offers the contract to the array reference 'arr' > > What about multiple threads wanting some contract between their uses of > arr[1] and arr[2]? > > Just thought I would throw this into the mix.. > > Gary > -----Original Message----- > From: core-libs-dev-bounces at openjdk.java.net [mailto: > core-libs-dev-bounces at openjdk.java.net] On Behalf Of Brian Goetz > Sent: Friday, February 07, 2014 2:45 PM > To: Doug Lea; core-libs-dev > Subject: Re: Draft JEP on enhanced volatiles > > It's about time we brought atomic ops out of the shadows of Unsafe and > into the programming model proper! It was a good compromise ten years ago > to stuff these into Unsafe, but its time for these features to grow up. > > Specific to this proposal, I like how the VolatileXxx interface approach > allows us to add new fencing modes and/or atomic operations should we > discover that the initial set is deficient. I also like how this > potentially gives us a consistent story for volatile semantics on array > elements. > > I can see a clean and straightforward implementation here at all the > required touch points: language spec, memory model, type system, compiler, > and VM runtime, and none of them require any special magic. > (Having spent some time exploring this design space myself, I think the > proposed surfacing is the least objectionable and most transparent of the > options that I've seen.) > > Minor nits called out inline. > > Otherwise, Reviewed (as JDK reviewer) and Endorsed (as area lead). > > > We then enable access to corresponding methods for (typically > > volatile) fields using the ".volatile" prefix. For example: > > Is this a prefix or a suffix? > > > The ".volatile" syntax is slightly unusual, but we are confident that > > it is syntactically unambiguous and semantically specifiable. > > And, it is reasonably consistent with other similar uses of keywords for > qualifying access to functionality, such as I.super or Outer.this. > > > We also expect to allow volatile > > operations on array elements in addition to fields. > > This calls for the memory model to provide clear semantics for mixed > accesses, such as volatile write / nonvolatile read. (The same would be > true if we permitted the use of the .volatile modifier on nonvolatile > fields.) > > Several other alternatives > > (including those based on field references) have been raised and > > dismissed as unworkable on syntactic, efficiency, and/or usability > > grounds > > or extensibility grounds. Many existing proposals might work for the > operations in front of us today, but not scale as new fencing modes or > atomic operations are discovered. > > > Risks and Assumptions > > --------------------- > > > > We are confident of feasibility. However, we expect that it will > > require more experimentation to arrive at compilation techniques that > > result in efficient enough implementation for routine use in the > > performance-critical contexts where these constructs are most often > > needed. The use of method handles may be impacted by and may impact > > JVM method handle support. > > As a limitation, I'd call out that its not obvious how this would scale to > DCAS. > > > A large number of usages in java.util.concurrent (and a few elsewhere > > in JDK) could be simplified and updated to use this support. > > Hooray! > > > > From snazy at snazy.de Fri Feb 7 21:47:48 2014 From: snazy at snazy.de (Robert Stupp) Date: Fri, 7 Feb 2014 22:47:48 +0100 Subject: Draft JEP on enhanced volatiles In-Reply-To: References: <52F527A4.3030802@cs.oswego.edu> <52F545B1.2070706@oracle.com> Message-ID: Am 07.02.2014 um 22:41 schrieb Benedict Elliott Smith : > > I'm quite excited about these changes, it looks like a great way to provide > support outside of unsafe. A great way to remove a lot of the Unsafe methods at all by providing their functionality to the world :) From jeremymanson at google.com Fri Feb 7 22:29:13 2014 From: jeremymanson at google.com (Jeremy Manson) Date: Fri, 7 Feb 2014 14:29:13 -0800 Subject: Object allocation tracing / profilers In-Reply-To: <1AE39B78-ED80-4BD7-B949-B0103D3D1381@snazy.de> References: <3486E1B1-8059-4362-B0B1-30A84E2C199D@snazy.de> <1AE39B78-ED80-4BD7-B949-B0103D3D1381@snazy.de> Message-ID: We have a patch for tracking method calls, too. It doesn't do callbacks on method invocation, but it does provide a list of method invocations. You can already do callbacks on method entry and exit with JVMTI, but it is expensive, which is why we built that. If there's interest, we can provide it. As far as the allocation tracking patch - you will have to feel comfortable patching and building your own JDK, and writing a blob of native code loaded by the JVM that will dlsym into libjvm.so to fish out the method to register your callback. Does that sound like something you feel comfortable doing? (I'm about to send the forward-port patch out to get code-reviewed for JDK8 before leaving for the first half of vacation, so if that doesn't scare you, then I can send it the week after next, or so.) Jeremy On Fri, Feb 7, 2014 at 1:35 PM, Robert Stupp wrote: > Jeremy, if you have a patch for JDK8 or 9 and some short instructions, i'd > like to try it (OSX or Linux) :) > I think it's ok if there's no way to do JDK callbacks (as long as the > class, array length, thread and maybe (piece of) the its stack trace is > available). > But for now I only need the class and array length. > > Regarding the fact, that profiler output regarding allocated objects is > wrong (compared to non-instrumented code) there should be a solution. I'm > afraid that many others optimize their code by considering the output of a > profiler as "the truth". Since all numbers (either object allocations or > method runtimes) are based on instrumented code - they must be considered > an at least airy estimate with a probability to be completely different > compared to non-profiled/instrumented code. > > Maybe it is possible to extend the official JVMTI API to provide a new > profiling API. IMO it would be faster and less intrusive and therefore > better comparable against a real production system. > That API would perform callbacks with information about the object, its > class, array length, thread and stack trace (up to a configurable length, > possibly empty). It should be considered that only a specific set of > threads, set of classes is of interest. Such an API would need to provide a > part to track method calls with similar filters - it would not need to do > any callbacks for inlined methods. Stack trace information could also > include information which method is compiled. The events could be placed by > application threads in some queue and a separate thread could poll for > events and perform the callbacks - means that the real profiling code is > independent from the application code. > > Instrumentation itself could still be used for more complicated stuff like > hunting for handle leaks (file, jdbc, etc). > > These are only some thoughts. > > Robert > > Am 07.02.2014 um 02:13 schrieb Jeremy Manson : > > You aren't doing anything wrong. More detail: > > - Anything bytecode based (like my instrumenter, which you tried) isn't > going to let you know what has been escape analyzed away. You can only do > bytecode rewriting before escape analysis runs, and before the escape > analysis runs, there is no way to find out what will be escape analyzed > away. > > - A bytecode-based instrumenter will also not pick up allocations that > occur via JNI, which is the problem that VMObjectAlloc solves. > > - If you happen to be on a platform that supports it, and you've compiled > the JDK correctly, you can use dtrace to capture Java allocations. > > - Because of its limitations and performance issues, we've actually > stopped doing bytecode-based instrumentation at Google in favor of > instrumenting the VM directly. We now use JVMTI-style callbacks whenever > an allocation happens. If people are interested, I can provide the patch > (in fact, I forward ported it to JDK8 yesterday!), but it is rather user > unfriendly, so you have to be rather ambitious to use it. On the plus > side, it can take into account escape analyzed objects (right now, it > doesn't instrument them, but this could conceivably be changed). > > - On the down side, if you take this approach, you can't call back into > Java from the JVM callbacks. This is because calling back into Java may > require the JVM to come to a safepoint, which you can't do in the middle of > allocation (which is where the callback would occur). > > Jeremy > > > > On Thu, Feb 6, 2014 at 2:36 PM, Robert Stupp wrote: > >> Hi, >> >> I'm trying to trace/count object allocations - especially regarding the >> ObjectIn/OutputStream stuff. >> >> I know two API points: JVMTI and instrumentation. JVMTI fires an event >> for nearly everything except allocations from bytecode - instrumentation >> allows to transform bytecode (e.g. to issue a callback on object >> allocation). >> >> But I am not very satisfied - maybe I oversee something... >> What I need is the number of "real" allocations. I do not want to count >> allocations that Hotspot would normally eliminate - for example Hotspot >> seems to eliminate instances of ArrayList$Itr under some circumstances. >> >> I played around with this instrumentation code: >> https://code.google.com/p/java-allocation-instrumenter/ >> But this one and any other profiler (JProfiler) I tried always gave me >> the theoretical number of instances of ArrayList$Itr. This reason seems to >> be as follows: >> With instrumentation/transformation constructors get a piece of code >> "injected" which calls a "profile object allocation method". This method >> gets the new object instance, which is used to get the Class object and to >> inquire the shallow object size on the heap - and it gets the current >> thread's stack trace. >> >> My suggestion is, that Hotspot cannot eliminate object allocations (for >> example of ArrayList$Itr) because the profiler bytecode needs that object >> instance and does other things, that are "strange" to Hotspot. >> >> I hoped that I can get callbacks using JVMTI but due to the restrictions >> mentioned here ( >> http://docs.oracle.com/javase/7/docs/platform/jvmti/jvmti.html#VMObjectAlloc) >> there are no callbacks for allocations caused by bytecode. >> VMObjectAlloc events occur during the start and end of the application - >> but not a single event is fired while the application is running (I've >> attached my JVMTI code). >> >> Am I doing anything wrong? Is there a possibility to count only the >> "real" instances? If not, wouldn't it be nice to have such a functionality >> in JVMTI or a callback functionality for >> http://download.java.net/jdk8/docs/api/java/lang/instrument/Instrumentation.html >> ? >> My fear is that all profiler results that all developers see are >> inaccurate because the code injected by profilers influences Hotspot too >> much. >> >> - >> Robert >> >> > > From snazy at snazy.de Fri Feb 7 22:39:19 2014 From: snazy at snazy.de (Robert Stupp) Date: Fri, 7 Feb 2014 23:39:19 +0100 Subject: Object allocation tracing / profilers In-Reply-To: References: <3486E1B1-8059-4362-B0B1-30A84E2C199D@snazy.de> <1AE39B78-ED80-4BD7-B949-B0103D3D1381@snazy.de> Message-ID: <667EB7D7-0DDC-4DAD-B789-673D8FBB209E@snazy.de> Am 07.02.2014 um 23:29 schrieb Jeremy Manson : > We have a patch for tracking method calls, too. It doesn't do callbacks on method invocation, but it does provide a list of method invocations. You can already do callbacks on method entry and exit with JVMTI, but it is expensive, which is why we built that. If there's interest, we can provide it. > > As far as the allocation tracking patch - you will have to feel comfortable patching and building your own JDK, and writing a blob of native code loaded by the JVM that will dlsym into libjvm.so to fish out the method to register your callback. Does that sound like something you feel comfortable doing? Hm - I interrupted coding C/C++ when Java was born. But I want to give it a try :) A nice project to "get back to the roots" and get a deeper understanding what's going on in that "wunderbox" JVM/Hotspot. > (I'm about to send the forward-port patch out to get code-reviewed for JDK8 before leaving for the first half of vacation, so if that doesn't scare you, then I can send it the week after next, or so.) Great! Robert From dl at cs.oswego.edu Sat Feb 8 00:49:45 2014 From: dl at cs.oswego.edu (Doug Lea) Date: Fri, 07 Feb 2014 19:49:45 -0500 Subject: Draft JEP on enhanced volatiles In-Reply-To: <52F527A4.3030802@cs.oswego.edu> References: <52F527A4.3030802@cs.oswego.edu> Message-ID: <52F57F29.30507@cs.oswego.edu> Thanks for comments! A few collected replies: On 02/07/2014 01:49 PM, David M. Lloyd wrote: > I think making the description contingent on a specific syntax might be a > mistake. I'm guessing that people who care are so tired of talking about problems without resolving them that they'll be happy to see implemented the only proposal so far that covers all the requirements and for which the strongest negative reaction has been: "it's a little bit funny looking" :-). (Well, I'm not counting reactions that seem too unrealistic. Like "let's add first-class L-values to Java" or "make people use method-handle-macros".) > I think that making required performance characteristics (e.g. "must be > equivalent in performance to Unsafe counterparts") a part of acceptance > criteria is also something that should be done. Yes. I don't think this needs writing down though. If people continue to use Unsafe in these cases for the sake of performance, we all lose. On 02/07/2014 03:18 PM, Robert Stupp wrote: > Would the syntax for array element updates look like this? > > long[][] foo; foo[7][42].volatile.compareAndSet(0, 42); > Tentatively yes. Details for arrays (as well as non-volatile fields) ultimately rely on whether, in JMM efforts, we can unify semantics of operations on non-volatiles with relaxed operations on volatiles. In C/C++11, these are subtly different, but my guess is that we can/will unify. If not, usages will be uglier. Also, as only vaguely mentioned in the JEP (because vague is the best I can do at the moment), for completeness, the .volatile forms will need to be augmented with some (static) methods that somehow export the standalone Unsafe fence method functionality added in JDK8. (One possibility is to add such methods to the java.util.Objects class. Or maybe hide them elsewhere.) On 02/07/2014 03:44 PM, Brian Goetz wrote: > >> We then enable access to corresponding methods for (typically volatile) >> fields using the ".volatile" prefix. For example: > > Is this a prefix or a suffix? I called it "prefix", since it sets the name resolution scope for what comes next. But we could just call it a "fix" :-) > As a limitation, I'd call out that its not obvious how this would scale to > DCAS. Right; thanks. Although the issues vary depending on whether you are dealing with the double-width sense of DCAS (side-by-side fields of the same object; possibly some sort of "wide" value type) versus "2CAS" (2 separate vars, a form of transaction). My current thinking about 2CAS is that for the sake of portable implementation, we would need to restrict the classes of objects that are 2CASable, so wouldn't be addressed by this JEP. But some new wide value type X, if ever supported, could be accommodated by creating a VolatileX class and using the same rules as for other cases. We'd want to spec this in a way that make such things is possible. -Doug From martinrb at google.com Sat Feb 8 01:21:17 2014 From: martinrb at google.com (Martin Buchholz) Date: Fri, 7 Feb 2014 17:21:17 -0800 Subject: Draft JEP on enhanced volatiles In-Reply-To: <52F57F29.30507@cs.oswego.edu> References: <52F527A4.3030802@cs.oswego.edu> <52F57F29.30507@cs.oswego.edu> Message-ID: On Fri, Feb 7, 2014 at 4:49 PM, Doug Lea
wrote: > > As a limitation, I'd call out that its not obvious how this would scale to >> DCAS. >> > > Right; thanks. Although the issues vary depending on whether > you are dealing with the double-width sense of DCAS (side-by-side > fields of the same object; possibly some sort of "wide" value type) > versus "2CAS" (2 separate vars, a form of transaction). My current > thinking about 2CAS is that for the sake of portable implementation, > we would need to restrict the classes of objects that are 2CASable, > so wouldn't be addressed by this JEP. But some new wide value > type X, if ever supported, could be accommodated by creating a > VolatileX class and using the same rules as for other cases. We'd > want to spec this in a way that make such things is possible. I rather like what C++11 did with atomic - it accepts any size TYPE, and falls back to locking if the TYPE is too large, and the programmer can query at compile time whether native atomic access is possible for that TYPE. If we get value types, it will be natural to do complex z; z.volatile.compareAndSet ... I think "wide" 2-CAS is completely separate from transactional 2-CAS. The former isn't really a 2-CAS at all; just a wide single-CAS. From weijun.wang at oracle.com Sat Feb 8 06:54:49 2014 From: weijun.wang at oracle.com (Wang Weijun) Date: Sat, 8 Feb 2014 14:54:49 +0800 Subject: native2ascii high CPU building jdk7u? Message-ID: <07647345-D186-4C24-A7EE-2F918C2DEB78@oracle.com> Hi All I am building jdk7u on linux and it's stuck in images build (for almost an hour). Turns out native2ascii uses too many CPU: PID USER PR NI VIRT RES SHR S %CPU %MEM TIME+ COMMAND 23569 ******* 20 0 9436 676 540 R 54.4 0.0 15:56.11 native2ascii The machine is Linux ******** 2.6.39-400.21.1.el6uek.x86_64 #1 SMP Thu Apr 4 03:49:00 PDT 2013 x86_64 x86_64 x86_64 GNU/Linux Red Hat Enterprise Linux Server release 6.4 (Santiago) The command (according to ps -f) is /java/re/jdk/1.6.0/archive/fcs/binaries/linux-amd64/bin/native2ascii -J-XX:-PrintVMOptions -J-XX:+UnlockDiagnosticVMOptions -J-XX:-LogVMOutput -J-Xmx512m -J-Xms512m -J-XX:PermSize=32m -J-XX:MaxPermSize=160m -reverse -encoding UTF-8 I can kill it (actually, twice) and the build goes on and succeeds finally. Has anyone seen the same problem? Thanks Max From Alan.Bateman at oracle.com Sat Feb 8 07:44:02 2014 From: Alan.Bateman at oracle.com (Alan Bateman) Date: Sat, 08 Feb 2014 07:44:02 +0000 Subject: native2ascii high CPU building jdk7u? In-Reply-To: <07647345-D186-4C24-A7EE-2F918C2DEB78@oracle.com> References: <07647345-D186-4C24-A7EE-2F918C2DEB78@oracle.com> Message-ID: <52F5E042.5030707@oracle.com> On 08/02/2014 06:54, Wang Weijun wrote: > : > > The command (according to ps -f) is > > /java/re/jdk/1.6.0/archive/fcs/binaries/linux-amd64/bin/native2ascii -J-XX:-PrintVMOptions -J-XX:+UnlockDiagnosticVMOptions -J-XX:-LogVMOutput -J-Xmx512m -J-Xms512m -J-XX:PermSize=32m -J-XX:MaxPermSize=160m -reverse -encoding UTF-8 > This looks like the original (Sun) JDK 6 from 2006 and so pre-dates OpenJDK. I think I'd start with jstack to see if I can get a thread dump, this might give you a hint as to what it is doing. -Alan. From weijun.wang at oracle.com Sat Feb 8 08:54:22 2014 From: weijun.wang at oracle.com (Wang Weijun) Date: Sat, 8 Feb 2014 16:54:22 +0800 Subject: native2ascii high CPU building jdk7u? In-Reply-To: <52F5E042.5030707@oracle.com> References: <07647345-D186-4C24-A7EE-2F918C2DEB78@oracle.com> <52F5E042.5030707@oracle.com> Message-ID: <7F541DF6-79A2-40FF-A867-41DFBE93EFC7@oracle.com> JDK 6 is the bootdir the build process automatically chooses. If I set ALT_BOOTDIR to a 7u there is no such problem. With JDK 6 it's always reproducible, "jstack -F -m -l nnnnn" also hungs. Neither the build nor jstack can be stopped by Ctrl-C and I have to kill native2ascii again. --Max On Feb 8, 2014, at 15:44, Alan Bateman wrote: > On 08/02/2014 06:54, Wang Weijun wrote: >> : >> >> The command (according to ps -f) is >> >> /java/re/jdk/1.6.0/archive/fcs/binaries/linux-amd64/bin/native2ascii -J-XX:-PrintVMOptions -J-XX:+UnlockDiagnosticVMOptions -J-XX:-LogVMOutput -J-Xmx512m -J-Xms512m -J-XX:PermSize=32m -J-XX:MaxPermSize=160m -reverse -encoding UTF-8 >> > This looks like the original (Sun) JDK 6 from 2006 and so pre-dates OpenJDK. I think I'd start with jstack to see if I can get a thread dump, this might give you a hint as to what it is doing. > > -Alan. From me at noctarius.com Sat Feb 8 08:58:37 2014 From: me at noctarius.com (Christoph Engelbert) Date: Sat, 8 Feb 2014 09:58:37 +0100 Subject: Draft JEP on enhanced volatiles In-Reply-To: <52F545B1.2070706@oracle.com> References: <52F527A4.3030802@cs.oswego.edu> <52F545B1.2070706@oracle.com> Message-ID: Am 07.02.2014 um 21:44 schrieb Brian Goetz : > And, it is reasonably consistent with other similar uses of keywords for qualifying access to functionality, such as I.super or Outer.this. Brian?s right, if you see it the same way as super and this it doesn?t even looks so unfamiliar :) From Alan.Bateman at oracle.com Sat Feb 8 09:23:38 2014 From: Alan.Bateman at oracle.com (Alan Bateman) Date: Sat, 08 Feb 2014 09:23:38 +0000 Subject: native2ascii high CPU building jdk7u? In-Reply-To: <7F541DF6-79A2-40FF-A867-41DFBE93EFC7@oracle.com> References: <07647345-D186-4C24-A7EE-2F918C2DEB78@oracle.com> <52F5E042.5030707@oracle.com> <7F541DF6-79A2-40FF-A867-41DFBE93EFC7@oracle.com> Message-ID: <52F5F79A.1080709@oracle.com> On 08/02/2014 08:54, Wang Weijun wrote: > JDK 6 is the bootdir the build process automatically chooses. If I set ALT_BOOTDIR to a 7u there is no such problem. > > With JDK 6 it's always reproducible, "jstack -F -m -l nnnnn" also hungs. Neither the build nor jstack can be stopped by Ctrl-C and I have to kill native2ascii again. > > --Max The boot JDK should also be N-1. Can you set ALT_BOOTDIR to a more recent 6u build and see if that helps? -Alan. From weijun.wang at oracle.com Sat Feb 8 09:39:48 2014 From: weijun.wang at oracle.com (Wang Weijun) Date: Sat, 8 Feb 2014 17:39:48 +0800 Subject: native2ascii high CPU building jdk7u? In-Reply-To: <52F5F79A.1080709@oracle.com> References: <07647345-D186-4C24-A7EE-2F918C2DEB78@oracle.com> <52F5E042.5030707@oracle.com> <7F541DF6-79A2-40FF-A867-41DFBE93EFC7@oracle.com> <52F5F79A.1080709@oracle.com> Message-ID: <08AC0551-8F89-43B8-A1EA-63279D8315AB@oracle.com> 6u45 is also fine. On Feb 8, 2014, at 17:23, Alan Bateman wrote: > On 08/02/2014 08:54, Wang Weijun wrote: >> JDK 6 is the bootdir the build process automatically chooses. If I set ALT_BOOTDIR to a 7u there is no such problem. >> >> With JDK 6 it's always reproducible, "jstack -F -m -l nnnnn" also hungs. Neither the build nor jstack can be stopped by Ctrl-C and I have to kill native2ascii again. >> >> --Max > The boot JDK should also be N-1. Can you set ALT_BOOTDIR to a more recent 6u build and see if that helps? > > -Alan. From sean.coffey at oracle.com Sat Feb 8 10:33:55 2014 From: sean.coffey at oracle.com (=?ISO-8859-1?Q?Se=E1n_Coffey?=) Date: Sat, 08 Feb 2014 10:33:55 +0000 Subject: RFR: 7152892: some jtreg tests fail with permission denied In-Reply-To: <52F52990.5060200@oracle.com> References: <52F52990.5060200@oracle.com> Message-ID: <52F60813.4080705@oracle.com> Looks fine to me Rob. regards, Sean. On 07/02/14 18:44, Rob McKenna wrote: > Hi folks, > > When files are copied by our test harness the permissions are left > unchanged. This can cause trouble for a few tests when the test folder > is read only. > > http://cr.openjdk.java.net/~robm/7152892/webrev.01/ > > -Rob > From forax at univ-mlv.fr Sat Feb 8 10:39:50 2014 From: forax at univ-mlv.fr (Remi Forax) Date: Sat, 08 Feb 2014 11:39:50 +0100 Subject: Draft JEP on enhanced volatiles In-Reply-To: <52F545B1.2070706@oracle.com> References: <52F527A4.3030802@cs.oswego.edu> <52F545B1.2070706@oracle.com> Message-ID: <52F60976.4060703@univ-mlv.fr> On 02/07/2014 09:44 PM, Brian Goetz wrote: > It's about time we brought atomic ops out of the shadows of Unsafe and > into the programming model proper! It was a good compromise ten years > ago to stuff these into Unsafe, but its time for these features to > grow up. > > Specific to this proposal, I like how the VolatileXxx interface > approach allows us to add new fencing modes and/or atomic operations > should we discover that the initial set is deficient. I also like how > this potentially gives us a consistent story for volatile semantics on > array elements. > > I can see a clean and straightforward implementation here at all the > required touch points: language spec, memory model, type system, > compiler, and VM runtime, and none of them require any special magic. > (Having spent some time exploring this design space myself, I think > the proposed surfacing is the least objectionable and most transparent > of the options that I've seen.) Brian, can you be a little more specific and provide the way foo.volatile.compareAndSet is compiled with this example: class Linked { volatile Node head; public void add(String element) { for(;;) { Node oldHead = head.volatile.get(); Node newNode = new Node(element, oldHead); if (head.volatile.compareAndSet(oldHead, newNode)) { return; } } } } I will be very happy to see how you solve the different issues because i've also spent some times thinking on how the .volatile syntax can be implemented: - what is the signature of the method compareAndSet, and how you fight the generics erasure ? - how do you remove the extra-level of indirection if head is a VolatileXXX, because accessing to the value is this.head.payload and not this.head ? - what LinkedClass.class.getDeclaredField("head").getType() returns ? regards, R?mi From forax at univ-mlv.fr Sat Feb 8 11:19:36 2014 From: forax at univ-mlv.fr (Remi Forax) Date: Sat, 08 Feb 2014 12:19:36 +0100 Subject: Draft JEP on enhanced volatiles In-Reply-To: <52F57F29.30507@cs.oswego.edu> References: <52F527A4.3030802@cs.oswego.edu> <52F57F29.30507@cs.oswego.edu> Message-ID: <52F612C8.1080603@univ-mlv.fr> On 02/08/2014 01:49 AM, Doug Lea wrote: > > Thanks for comments! A few collected replies: > > On 02/07/2014 01:49 PM, David M. Lloyd wrote: >> I think making the description contingent on a specific syntax might >> be a >> mistake. > > I'm guessing that people who care are so tired of talking > about problems without resolving them that they'll be happy to > see implemented the only proposal so far that covers all the > requirements and for which the strongest negative reaction has > been: "it's a little bit funny looking" :-). > > (Well, I'm not counting reactions that seem too unrealistic. > Like "let's add first-class L-values to Java" or > "make people use method-handle-macros".) It seems I have to answer. You don't need to have what you call 'method-handle-macros' to implement the .volatile syntax, it's just that having a general mechanism is IMO better for the future of Java that having a mechanism only tailored for the .volatile syntax given that by example the JNI-JEP of Charles Nutter also need the very same mechanism. Anyway, it's a separate discussion. Let me try to summarize the problems you mention earlier, there are two main issues with the current java.util.concurrent.atomic API, - erasure of generics, because of that, the API implementation does numerous of runtime checks which have a cost at runtime hence some users perfers to use the unsupported class sun.misc.Unsafe - no lvalue or & at language level, because of that the volatile field name is passed as a pair (Class, String) and at runtime For the former, erasure of generics, we had the same issue with the creation of a lambda because the runtime need the reified types in order to create the proxy class with the right bridges. This was solved by using invokedynamic because invokedynamic allows to pass out of band values representing the non-erased types. For the later, the .volatile syntax introduces a language level way to reference a field, this is exactly what we want, hooray as Brian wrote. That's true but there is a downside, not in the syntax by itself but in the way the syntax is coupled with a class VolatileXXX. Accessing to the field in order to modifying it will require to pass through an instance of VolatileXXX which introduce an extra level of indirection. Because of that level of indirection, users may still prefer to use Unsafe. This problem can be solved if VolatileXXX take a java.lang.reflect.Field at construction (or any values that morally represent a field) and not own the field itself. > > >> I think that making required performance characteristics (e.g. "must be >> equivalent in performance to Unsafe counterparts") a part of acceptance >> criteria is also something that should be done. > > Yes. I don't think this needs writing down though. If people > continue to use Unsafe in these cases for the sake of performance, > we all lose. cheers, R?mi From dl at cs.oswego.edu Sat Feb 8 12:50:39 2014 From: dl at cs.oswego.edu (Doug Lea) Date: Sat, 08 Feb 2014 07:50:39 -0500 Subject: Draft JEP on enhanced volatiles In-Reply-To: References: <52F527A4.3030802@cs.oswego.edu> <52F57F29.30507@cs.oswego.edu> Message-ID: <52F6281F.8050602@cs.oswego.edu> On 02/07/2014 08:21 PM, Martin Buchholz wrote: > I think "wide" 2-CAS is completely separate from transactional 2-CAS. The > former isn't really a 2-CAS at all; just a wide single-CAS. Right; thanks. I added: This proposal focuses on the control of atomicity and ordering for single variables. We expect the resulting specifications to be amenable for extension in natural ways for additional primitive-like value types, if they are ever defined for Java. However, it is not a general-purpose transaction mechanism for controlling accesses and updates to multiple variables. Alternative forms for expressing and implementing such constructions may be explored in the course of this or other JEPs, and/or may be the subject of further JEPs. From dl at cs.oswego.edu Sat Feb 8 15:50:37 2014 From: dl at cs.oswego.edu (Doug Lea) Date: Sat, 08 Feb 2014 10:50:37 -0500 Subject: Draft JEP on enhanced volatiles In-Reply-To: <52F612C8.1080603@univ-mlv.fr> References: <52F527A4.3030802@cs.oswego.edu> <52F57F29.30507@cs.oswego.edu> <52F612C8.1080603@univ-mlv.fr> Message-ID: <52F6524D.9070605@cs.oswego.edu> On 02/08/2014 06:19 AM, Remi Forax wrote: > It seems I have to answer. > You don't need to have what you call 'method-handle-macros' to implement the > .volatile syntax, (Aside: I love macros enough to want them to be done right someday, but they seem to be the wrong plan of attack for this problem.) > Let me try to summarize the problems you mention earlier, > there are two main issues with the current java.util.concurrent.atomic API, Another: There is a (very!) undesirable but available fallback for .volatile that doesn't apply to FieldUpdaters: If method handles don't work out, generate exactly the same Unsafe code that we manually write today, with some crazy further fallback if run on a JVM not supporting the intrinsics. -Doug > - erasure of generics, > because of that, the API implementation does numerous of runtime checks > which have a cost at runtime hence some users perfers to use the > unsupported class sun.misc.Unsafe > - no lvalue or & at language level, > because of that the volatile field name is passed as a pair (Class, String) > and at runtime > > For the former, erasure of generics, we had the same issue with the creation of > a lambda > because the runtime need the reified types in order to create the proxy class > with the right bridges. This was solved by using invokedynamic because > invokedynamic > allows to pass out of band values representing the non-erased types. > > For the later, the .volatile syntax introduces a language level way to reference > a field, > this is exactly what we want, hooray as Brian wrote. That's true but there is a > downside, > not in the syntax by itself but in the way the syntax is coupled with a class > VolatileXXX. > Accessing to the field in order to modifying it will require to pass through an > instance > of VolatileXXX which introduce an extra level of indirection. > Because of that level of indirection, users may still prefer to use Unsafe. > This problem can be solved if VolatileXXX take a java.lang.reflect.Field at > construction > (or any values that morally represent a field) and not own the field itself. > >> >> >>> I think that making required performance characteristics (e.g. "must be >>> equivalent in performance to Unsafe counterparts") a part of acceptance >>> criteria is also something that should be done. >> >> Yes. I don't think this needs writing down though. If people >> continue to use Unsafe in these cases for the sake of performance, >> we all lose. > > cheers, > R?mi > From brian.goetz at oracle.com Sat Feb 8 17:52:14 2014 From: brian.goetz at oracle.com (Brian Goetz) Date: Sat, 08 Feb 2014 12:52:14 -0500 Subject: Draft JEP on enhanced volatiles In-Reply-To: <52F60976.4060703@univ-mlv.fr> References: <52F527A4.3030802@cs.oswego.edu> <52F545B1.2070706@oracle.com> <52F60976.4060703@univ-mlv.fr> Message-ID: <52F66ECE.3040607@oracle.com> > can you be a little more specific and provide the way > foo.volatile.compareAndSet is compiled with this example: Hey, the JEP isn't even out of draft yet! Stop asking hard questions. The current strawman, though, is to expose direct (CP-ready) MH forms (Lookup API / constant pool format TBD) for field get/set with fence, array element get/set with fence, field CAS, and array element CAS. Then the static compiler translates foo.volatile.cAS(n,m) as LDC #MH[lookupFieldCASer(foo)] aload n aload m invokevirtual "invokeExact" > class Linked { > volatile Node head; > > public void add(String element) { > for(;;) { > Node oldHead = head.volatile.get(); > Node newNode = new Node(element, oldHead); > if (head.volatile.compareAndSet(oldHead, newNode)) { > return; > } > } > } > } > > I will be very happy to see how you solve the different issues because > i've also spent some times thinking on how the .volatile syntax can be > implemented: > - what is the signature of the method compareAndSet, and how you > fight the generics erasure ? > - how do you remove the extra-level of indirection if head is a > VolatileXXX, > because accessing to the value is this.head.payload and not > this.head ? > - what LinkedClass.class.getDeclaredField("head").getType() returns ? > > regards, > R?mi > From dl at cs.oswego.edu Sat Feb 8 19:33:15 2014 From: dl at cs.oswego.edu (Doug Lea) Date: Sat, 08 Feb 2014 14:33:15 -0500 Subject: Random access in ArrayDeque In-Reply-To: References: <231A3FB1-639F-4C06-823B-0CB9F1CB194A@oracle.com> <52F520EE.70709@redhat.com> Message-ID: <52F6867B.4060703@cs.oswego.edu> On 02/07/2014 01:59 PM, Martin Buchholz wrote: > ArrayDeque should implement most of the methods in List, notably get(i). > > ArrayDeque should not actually implement List itself, because the change in > contract/behavior for hashCode/equals would be too great. Right. My vague recollection is that these were among initial reasons for not implementing List. Also, the List.sublist method is questionable for a Queue. Too bad there is no interface with only get(index), set(index, x), and indexOf(x). Implementing only these would probably satisfy all candidate usages. > But we can > provide a > List asList() > method. This might be the best compromise. Possibly with the sublist method returning an unmodifiable list (which is legal and probably not too surprising). Maybe even similarly for listIterator. -Doug > > If we have agreement, we can do the relatively easy implementation. > > > On Fri, Feb 7, 2014 at 10:07 AM, David M. Lloyd wrote: > >> Just want to say that we've also had the need to implement an array-backed >> Deque+List; we found no surprises implementing it and thus I believe that >> extending ArrayDeque to implement List would be a positive change. Failing >> that, a combined ArrayListAndDeque class seems like a good idea. >> >> I think that calling it "Masters' thesis" material is perhaps overblowing >> the complexity a bit though. ;) Given that the basic algorithmic >> complexity of ArrayList is well-understood and is well-suited to many (some >> would say "most") applications, going for a O(sqrt(N)) middle insert/remove >> complexity would be something I would consider "scope creep"; LinkedList is >> usually a fine choice for these cases. >> >> >> On 02/07/2014 11:44 AM, Dan Smith wrote: >> >>> I noticed recently that the javac scanner is making use of >>> ArrayList.remove(0) when it consumes a buffer. Obviously this is an >>> inefficient way to implement a buffer, so I thought I'd try to fix it [1]. >>> ArrayDeque seems to provide just the behavior I need, with one fatal flaw: >>> despite encoding its data with an array, the class exposes no random-access >>> operations. For lookahead, I need to be able to call get(int). >>> >>> This seems to be a fairly common complaint [2][3]. >>> >>> I found an old bug requesting that ArrayDeque be enhanced to implement >>> List [4], as well as a thread from 2010 that included a proof-of-concept >>> ArrayDeque+List [5] and had a note from Martin Buchholz saying he regrets >>> that ArrayDeque doesn't already implement List [6]. >>> >>> Is there any hope of ArrayDeque being enhanced in the near future to >>> provide random access? There's some risk that any such initiative would be >>> derailed by a quest for an uber-collection. I think a lot of people would >>> be quite happy just to have a (trivial) 'get(int)' method added to >>> ArrayDeque. >>> >>> ?Dan >>> >>> [1] https://bugs.openjdk.java.net/browse/JDK-8033158 >>> [2] http://en.wikipedia.org/wiki/Deque#Language_support >>> [3] https://www.google.com/#q=arraydeque+%22random+access%22 >>> [4] https://bugs.openjdk.java.net/browse/JDK-6368844 >>> [5] http://mail.openjdk.java.net/pipermail/core-libs-dev/2010- >>> April/004038.html >>> [6] http://mail.openjdk.java.net/pipermail/core-libs-dev/2010- >>> April/004031.html >>> >>> >> >> -- >> - DML >> > From tbuktu at hotmail.com Sun Feb 9 10:00:05 2014 From: tbuktu at hotmail.com (Tim Buktu) Date: Sun, 9 Feb 2014 11:00:05 +0100 Subject: BigDecimal performance In-Reply-To: <52F524A6.4020700@oracle.com> References: <52F524A6.4020700@oracle.com> Message-ID: Hi Joe, > I'm not sure which algorithm you used to compute pi, but I would be > very surprised by a ~8 X speed difference between BigInteger and > BigDecimal if the same computation was being under on the BigDecimal > values, meaning the BigIntegers held in the BigDecimals were the same > as the BigIntegers in the BigInteger computation. The BigIntegers are not identical because one is rounded to some number of bits whereas the other is rounded to some number of decimal digits. They should be the same order of magnitude, though. It's possible I made a mistake, of course. The two programs are on GitHub: https://github.com/tbuktu/bigint/blob/master/src/main/java/Pi.java https://github.com/tbuktu/bigint/blob/master/src/main/java/PiBigInt.java > A BigDecimal object is a fairly small wrapper around a BigInteger and > updating the scale/exponent field isn't too much additional work. It's not the exponent, it's rounding decimal digits that slows BigDecimal down because it has to divide the BigInteger by a power of 10. Tim > > On 02/07/2014 02:44 AM, Dmitry Nadezhin wrote: >> I think that better name is BigBinary. >> Both BigDecimal and BigBinary are floating-point number >> with radix=10 and radix=2 respectively. >> >> More general approach is an abstract class or interface Rational >> which has implementation subclasses. >> Each nonzero rational number P/Q can be represented as >> P/Q = p/q * 2^e , where e is integer, p and q are odd integers and >> GCD(p,q) >> = 1. >> Then BigBinary is a subclass with q=1. >> Arithmetic operations on Rationals are implemented by general algorithm >> when arguments are true >> rationals (q!=1) and by specific algorithms when they are Binaries >> (q=1). >> >> This is elaborated here: >> http://interval.louisiana.edu/reliable-computing-journal/volume-19/reliable-computing-19-pp-229-247.pdf >> >> https://java.net/projects/jinterval/sources/svn/show/trunk/jinterval/jinterval-rational-java/src/main/java/net/java/jinterval/rational >> >> >> >> >> On Thu, Feb 6, 2014 at 9:11 PM, Tim Buktu wrote: >> >>> Hi, >>> >>> now that BigInteger deals better with large numbers, it would be nice >>> for that to translate into an improvement in BigDecimal performance >>> because BigDecimal is essentially a wrapper around BigInteger. >>> Unfortunately, BigDecimal is still slower than BigInteger because it >>> has >>> to scale and round. >>> >>> I don't see a way to fix this without breaking the >>> BigDecimal=BigInteger*10^n paradigm, but it could be done by >>> introducing >>> something like a BigFloat class that wraps a BigInteger such that >>> BigFloat=BigInteger*2^n. I would expect the code to be less complex >>> than >>> BigDecimal because the only places it would have to deal with powers of >>> ten would be conversion from and to String or BigDecimal. It would also >>> be faster than BigDecimal for the same reason, but the downside is that >>> it wouldn't accurately represent decimal fractions (just like float and >>> double). >>> >>> Is this something that would be beneficial in the real world? >>> >>> I also did a little experiment to see how long a computation would take >>> using BigDecimals vs the same computation using fixed-point BigInteger >>> arithmetic. I wrote two programs that calculate pi to a million digits. >>> The BigInteger version took 3 minutes; the BigDecimal version took 28 >>> minutes (both single-threaded). >>> >>> Tim >>> > > > From forax at univ-mlv.fr Sun Feb 9 12:11:17 2014 From: forax at univ-mlv.fr (Remi Forax) Date: Sun, 09 Feb 2014 13:11:17 +0100 Subject: Draft JEP on enhanced volatiles In-Reply-To: <52F6524D.9070605@cs.oswego.edu> References: <52F527A4.3030802@cs.oswego.edu> <52F57F29.30507@cs.oswego.edu> <52F612C8.1080603@univ-mlv.fr> <52F6524D.9070605@cs.oswego.edu> Message-ID: <52F77065.7050207@univ-mlv.fr> On 02/08/2014 04:50 PM, Doug Lea wrote: > On 02/08/2014 06:19 AM, Remi Forax wrote: > >> It seems I have to answer. >> You don't need to have what you call 'method-handle-macros' to >> implement the >> .volatile syntax, > > (Aside: I love macros enough to want them to be done right someday, > but they seem to be the wrong plan of attack for this problem.) What I propose is not macros but a way to write intrinsics in Java, which I think is the right way to tackle this problem, at API level, you can still see intrinsics has a kind of macro system, but I don't think it helps. > >> Let me try to summarize the problems you mention earlier, >> there are two main issues with the current >> java.util.concurrent.atomic API, > > Another: There is a (very!) undesirable but available fallback for > .volatile > that doesn't apply to FieldUpdaters: If method handles don't work > out, generate exactly the same Unsafe code that we manually write > today, with some crazy further fallback if run on a JVM not > supporting the intrinsics. I think you can not do that, because for that you need to publicly expose the methods of Unsafe, so someone may generate a bytecode that call these methods creating a security hole. That's why these methods were put in Unsafe after all. > > -Doug R?mi > > > >> - erasure of generics, >> because of that, the API implementation does numerous of runtime >> checks >> which have a cost at runtime hence some users perfers to use the >> unsupported class sun.misc.Unsafe >> - no lvalue or & at language level, >> because of that the volatile field name is passed as a pair >> (Class, String) >> and at runtime >> >> For the former, erasure of generics, we had the same issue with the >> creation of >> a lambda >> because the runtime need the reified types in order to create the >> proxy class >> with the right bridges. This was solved by using invokedynamic because >> invokedynamic >> allows to pass out of band values representing the non-erased types. >> >> For the later, the .volatile syntax introduces a language level way >> to reference >> a field, >> this is exactly what we want, hooray as Brian wrote. That's true but >> there is a >> downside, >> not in the syntax by itself but in the way the syntax is coupled with >> a class >> VolatileXXX. >> Accessing to the field in order to modifying it will require to pass >> through an >> instance >> of VolatileXXX which introduce an extra level of indirection. >> Because of that level of indirection, users may still prefer to use >> Unsafe. >> This problem can be solved if VolatileXXX take a >> java.lang.reflect.Field at >> construction >> (or any values that morally represent a field) and not own the field >> itself. >> >>> >>> >>>> I think that making required performance characteristics (e.g. >>>> "must be >>>> equivalent in performance to Unsafe counterparts") a part of >>>> acceptance >>>> criteria is also something that should be done. >>> >>> Yes. I don't think this needs writing down though. If people >>> continue to use Unsafe in these cases for the sake of performance, >>> we all lose. >> >> cheers, >> R?mi >> > > From forax at univ-mlv.fr Sun Feb 9 12:52:43 2014 From: forax at univ-mlv.fr (Remi Forax) Date: Sun, 09 Feb 2014 13:52:43 +0100 Subject: Draft JEP on enhanced volatiles In-Reply-To: <52F66ECE.3040607@oracle.com> References: <52F527A4.3030802@cs.oswego.edu> <52F545B1.2070706@oracle.com> <52F60976.4060703@univ-mlv.fr> <52F66ECE.3040607@oracle.com> Message-ID: <52F77A1B.2070804@univ-mlv.fr> On 02/08/2014 06:52 PM, Brian Goetz wrote: >> can you be a little more specific and provide the way >> foo.volatile.compareAndSet is compiled with this example: > > Hey, the JEP isn't even out of draft yet! Stop asking hard questions. :) > > The current strawman, though, is to expose direct (CP-ready) MH forms > (Lookup API / constant pool format TBD) for field get/set with fence, > array element get/set with fence, field CAS, and array element CAS. > Then the static compiler translates > > foo.volatile.cAS(n,m) > > as > > LDC #MH[lookupFieldCASer(foo)] > aload n > aload m > invokevirtual "invokeExact" wow, you bring out the big guns ! I see a big issue with this approach, it requires to change the JVM spec each time you want to add a new operation. Moreover, overloads of LDC are usually re-written by JVMs has private opcodes, so why not directly use new opcodes, it will take less space in the classfile too. That said, having new opcodes is an interesting idea because it can solve several issues that make by example Unsafe.compareAndSwap slower than the C equivalent, Unsafe.compareAndSwapObject update card marks even if it's not necessary and the pattern 'if (foo.volatile.compareAndSet())' should be recognized by the JIT and the information of the branch taken should be ignored (to avoid de-optimization on re-try). but updating the JVM spec each time you want a new operation seems a no go for me. R?mi > >> class Linked { >> volatile Node head; >> >> public void add(String element) { >> for(;;) { >> Node oldHead = head.volatile.get(); >> Node newNode = new Node(element, oldHead); >> if (head.volatile.compareAndSet(oldHead, newNode)) { >> return; >> } >> } >> } >> } >> >> I will be very happy to see how you solve the different issues because >> i've also spent some times thinking on how the .volatile syntax can be >> implemented: >> - what is the signature of the method compareAndSet, and how you >> fight the generics erasure ? >> - how do you remove the extra-level of indirection if head is a >> VolatileXXX, >> because accessing to the value is this.head.payload and not >> this.head ? >> - what LinkedClass.class.getDeclaredField("head").getType() returns ? >> >> regards, >> R?mi >> From ivan.gerasimov at oracle.com Sun Feb 9 20:03:10 2014 From: ivan.gerasimov at oracle.com (Ivan Gerasimov) Date: Mon, 10 Feb 2014 00:03:10 +0400 Subject: Random access in ArrayDeque In-Reply-To: References: <231A3FB1-639F-4C06-823B-0CB9F1CB194A@oracle.com> <52F520EE.70709@redhat.com> Message-ID: <52F7DEFE.5080204@oracle.com> Wouldn't it be convenient, if negative indices were allowed for the get() method here? So that get(-1) would return the last element, like in some other languages such as Perl or Python. I understand it violates the spec for the List#get(), but this is the 'Double Ended' queue, so we may want to access it from the both ends. Sincerely yours, Ivan On 07.02.2014 23:07, Martin Buchholz wrote: > Can't finish if we don't start. Here's v0.1: No rocket science here. > > Index: src/main/java/util/ArrayDeque.java > =================================================================== > RCS file: > /export/home/jsr166/jsr166/jsr166/src/main/java/util/ArrayDeque.java,v > retrieving revision 1.57 > diff -u -r1.57 ArrayDeque.java > --- src/main/java/util/ArrayDeque.java 18 Jul 2013 18:21:22 -0000 1.57 > +++ src/main/java/util/ArrayDeque.java 7 Feb 2014 19:06:15 -0000 > @@ -291,6 +291,17 @@ > return result; > } > > + /** > + * Implements {@link List#get(int)}. > + */ > + public E get(int index) { > + if (index < 0 || index >= size()) > + throw new IndexOutOfBoundsException(); > + @SuppressWarnings("unchecked") > + E result = (E) elements[(head + index) & (elements.length - 1)]; > + return result; > + } > + > @SuppressWarnings("unchecked") > public E peekFirst() { > // elements[head] is null if deque empty > > > > On Fri, Feb 7, 2014 at 10:59 AM, Martin Buchholz wrote: > >> Sorry we're all such lamers. >> >> ArrayDeque should implement most of the methods in List, notably get(i). >> >> ArrayDeque should not actually implement List itself, because the change >> in contract/behavior for hashCode/equals would be too great. But we can >> provide a >> List asList() >> method. >> >> If we have agreement, we can do the relatively easy implementation. >> >> >> On Fri, Feb 7, 2014 at 10:07 AM, David M. Lloyd wrote: >> >>> Just want to say that we've also had the need to implement an >>> array-backed Deque+List; we found no surprises implementing it and thus I >>> believe that extending ArrayDeque to implement List would be a positive >>> change. Failing that, a combined ArrayListAndDeque class seems like a good >>> idea. >>> >>> I think that calling it "Masters' thesis" material is perhaps overblowing >>> the complexity a bit though. ;) Given that the basic algorithmic >>> complexity of ArrayList is well-understood and is well-suited to many (some >>> would say "most") applications, going for a O(sqrt(N)) middle insert/remove >>> complexity would be something I would consider "scope creep"; LinkedList is >>> usually a fine choice for these cases. >>> >>> >>> On 02/07/2014 11:44 AM, Dan Smith wrote: >>> >>>> I noticed recently that the javac scanner is making use of >>>> ArrayList.remove(0) when it consumes a buffer. Obviously this is an >>>> inefficient way to implement a buffer, so I thought I'd try to fix it [1]. >>>> ArrayDeque seems to provide just the behavior I need, with one fatal flaw: >>>> despite encoding its data with an array, the class exposes no random-access >>>> operations. For lookahead, I need to be able to call get(int). >>>> >>>> This seems to be a fairly common complaint [2][3]. >>>> >>>> I found an old bug requesting that ArrayDeque be enhanced to implement >>>> List [4], as well as a thread from 2010 that included a proof-of-concept >>>> ArrayDeque+List [5] and had a note from Martin Buchholz saying he regrets >>>> that ArrayDeque doesn't already implement List [6]. >>>> >>>> Is there any hope of ArrayDeque being enhanced in the near future to >>>> provide random access? There's some risk that any such initiative would be >>>> derailed by a quest for an uber-collection. I think a lot of people would >>>> be quite happy just to have a (trivial) 'get(int)' method added to >>>> ArrayDeque. >>>> >>>> ?Dan >>>> >>>> [1] https://bugs.openjdk.java.net/browse/JDK-8033158 >>>> [2] http://en.wikipedia.org/wiki/Deque#Language_support >>>> [3] https://www.google.com/#q=arraydeque+%22random+access%22 >>>> [4] https://bugs.openjdk.java.net/browse/JDK-6368844 >>>> [5] http://mail.openjdk.java.net/pipermail/core-libs-dev/2010- >>>> April/004038.html >>>> [6] http://mail.openjdk.java.net/pipermail/core-libs-dev/2010- >>>> April/004031.html >>>> >>>> >>> -- >>> - DML >>> >> > From tom.hawtin at oracle.com Mon Feb 10 03:24:43 2014 From: tom.hawtin at oracle.com (Tom Hawtin) Date: Mon, 10 Feb 2014 03:24:43 +0000 Subject: Draft JEP on enhanced volatiles In-Reply-To: <52F527A4.3030802@cs.oswego.edu> References: <52F527A4.3030802@cs.oswego.edu> Message-ID: <52F8467B.3050004@oracle.com> I'm very concerned that this adds bulk to an already oversized language to support something which should be incredibly obscure. On 07/02/2014 18:36, Doug Lea wrote: > Alternatives > ------------ > > We considered instead introducing new forms of "value type" that > support volatile operations. However, this would be inconsistent with > properties of other types, and would also require more effort for > programmers to use. I assume by "value type" you mean a type with only constant fields of "value types" and no identity (==, System.identityHashCode, monitor or equivalents). Clearly volatiles don't fit well there. There's something that Joe-E calls a "selfless type" which is very similar. A selfless type has no identity but may reference reference types. It gains the benefits of being an inlineable object, like value types, but supports mutable interfaces. That's a perfect fit. Notionally you would have a selfless type, VolatileInt say, that could be implemented as having a reference to an AtomicInteger. As a quality of implementation matter, you'd want a JVM that either manages to inline the AtomicXxx (assuming selfless types cannot be violated by reflection) or intrisifies the whole implementation of VolatileXxx. > We also considered expanding reliance on > java.util.concurrent.atomic FieldUpdaters, but their dynamic overhead > and usage limitations make them unsuitable. Several other alternatives > (including those based on field references) have been raised and > dismissed as unworkable on syntactic, efficiency, and/or usability > grounds over the many years that these issues has been discussed. If the proposed language changes were to be implemented as method handles, is there any reason why the updaters couldn't be implemented in an equivalent fashion? Tom From michael.cui at oracle.com Mon Feb 10 07:00:22 2014 From: michael.cui at oracle.com (michael cui) Date: Mon, 10 Feb 2014 15:00:22 +0800 Subject: RFR for JDK-8028711: TEST_BUG: Tests should pass through VM options: corelibs tests In-Reply-To: <52F4BD7B.8080006@oracle.com> References: <52E4C2E8.1060605@oracle.com> <52E4D1F2.4030406@oracle.com> <52E5B4C3.6080405@oracle.com> <52F455FC.3020009@oracle.com> <52F4BD7B.8080006@oracle.com> Message-ID: <52F87906.5040702@oracle.com> On 02/07/2014 07:03 PM, Alan Bateman wrote: > On 07/02/2014 03:41, michael cui wrote: >> Hi Alan, >> >> Based on current discussions we have had and existing usages in our >> code base, >> I would like to propose that the fix should ONLY make sure that >> ${TESTVMOPTS} will be passed in when launching java from shell script. >> >> For other options such as ${TESTJAVAOPTS} ${TESTTOOLVMOPTS} and >> ${TESTJAVACOPTS}, >> Since we are not very clear about needs to make them presents all the >> time when launching java or other java tools from shell, it is better >> to not make changes to current usages. >> >> Please let me know if you are ok with this approach. > If I understand correctly then you are proposing to address the > remaining issues where the vmoptions aren't passed through from shell > tests. That would be good. As I mentioned, there have been efforts on > this already but we either missed some tests or we didn't continue it > into newer tests. The other part is java tests that launch additional > VMs as we do have a few that don't pass the options through. That > might be a separate bug but would be good to address. Yes, that's my suggestion. I made code changes based on this proposal and tested it locally. Please review the newest version at : http://cr.openjdk.java.net/~tyan/michael/JDK-8028711/webrev.03/ > > As regards TESTJAVAOPTS then I think it might be worth starting a > discussion on the jtreg list to see if there are opportunities to > consolidate the number of variables that tests have to deal with. I > don't have any objection to the original proposal, it's just that it > makes it obvious that there are too many options and too easy to get > it wrong. > > -Alan. Thanks, -Michael Cui From forax at univ-mlv.fr Mon Feb 10 08:06:08 2014 From: forax at univ-mlv.fr (Remi Forax) Date: Mon, 10 Feb 2014 09:06:08 +0100 Subject: Draft JEP on enhanced volatiles In-Reply-To: <52F8467B.3050004@oracle.com> References: <52F527A4.3030802@cs.oswego.edu> <52F8467B.3050004@oracle.com> Message-ID: <52F88870.2010105@univ-mlv.fr> On 02/10/2014 04:24 AM, Tom Hawtin wrote: [... > >> We also considered expanding reliance on >> java.util.concurrent.atomic FieldUpdaters, but their dynamic overhead >> and usage limitations make them unsuitable. Several other alternatives >> (including those based on field references) have been raised and >> dismissed as unworkable on syntactic, efficiency, and/or usability >> grounds over the many years that these issues has been discussed. > > If the proposed language changes were to be implemented as method > handles, is there any reason why the updaters couldn't be implemented > in an equivalent fashion? The updater API in case of objects (AtomicReferenceFieldUpdater) rely on generics to enforce type safety, because generics are erased at runtime the implementation has to do several runtime class checks in order to guarantee safety. and nobody wants reified generics because among other ugly details it will bound the JVM to the Java type system, hence the idea to have a new API. > > Tom R?mi From bourges.laurent at gmail.com Mon Feb 10 09:24:32 2014 From: bourges.laurent at gmail.com (=?ISO-8859-1?Q?Laurent_Bourg=E8s?=) Date: Mon, 10 Feb 2014 10:24:32 +0100 Subject: Improve large array allocation / gc & intrinsics Message-ID: Dear all, I would like to propose a JDK9 RFE to improve JVM efficiency when dealing with large arrays (allocation + gc). In several scientific applications (and my patched java2d pisces), many large arrays are needed, created on the fly and it becomes very painful to recycle them using an efficient array cache (concurrency, cache size tuning, clear + cache eviction issues). In this case, the GC overhead leads to a big performance penalty (several hundred megabytes per seconds). As such array cache are very efficient in an application context, I am wondering if that approach could be promoted at the JDK level itself: - provide a new array allocator for large arrays that can return larger arrays than expected (size = 4 or 8 multiples) using array caches (per thread ?) stored in a dedicated JVM large memory area (GC aware) and providing efficient cache eviction policies - may support for both clean arrays (zero filled) or dirty arrays because some algorithms does not need zero-filled arrays. - improve JVM intrinsics (array clear, fill) to achieve maximum performance ie take into account the data alignment (4 or 8 multiples) - upgrade GC to recycle such 'cached' arrays (clean), update usage statistics and manage cache eviction policy (avoid wasting memory) Please give me your feedback & opinion and evaluate if this feature seems possible to implement into the JDK (hotspot, gc, core-libs) ... What is the procedure to create such JDK9 RFE ? Best regards, Laurent Bourg?s From Alan.Bateman at oracle.com Mon Feb 10 09:36:22 2014 From: Alan.Bateman at oracle.com (Alan Bateman) Date: Mon, 10 Feb 2014 09:36:22 +0000 Subject: RFR for JDK-8028711: TEST_BUG: Tests should pass through VM options: corelibs tests In-Reply-To: <52F87906.5040702@oracle.com> References: <52E4C2E8.1060605@oracle.com> <52E4D1F2.4030406@oracle.com> <52E5B4C3.6080405@oracle.com> <52F455FC.3020009@oracle.com> <52F4BD7B.8080006@oracle.com> <52F87906.5040702@oracle.com> Message-ID: <52F89D96.7000601@oracle.com> On 10/02/2014 07:00, michael cui wrote: > : > > Please review the newest version at : > http://cr.openjdk.java.net/~tyan/michael/JDK-8028711/webrev.03/ This looks good to me. There are a couple of places where javac wasn't updated. A few that I noticed are: test/com/sun/corba/5036554/TestCorbaBug.sh test/sun/tools/jconsole/ResourceCheckTest.sh test/sun/tools/native2ascii/resources/ImmutableResourceTest.sh A minor comment on test/java/net/URLPermission/nstest/lookup.sh is that the resulting line length is 182 characters and this will likely be annoying for future side-by-side views. So I think I'd split this while you are there. -Alan. From tristan.yan at oracle.com Mon Feb 10 10:57:15 2014 From: tristan.yan at oracle.com (Tristan Yan) Date: Mon, 10 Feb 2014 18:57:15 +0800 Subject: RFR for JDK-8030844:sun/rmi/rmic/classpath/RMICClassPathTest.java timeout in same binaries run In-Reply-To: References: Message-ID: <13F6E908-26F2-420A-9E25-AD00DF8E3769@oracle.com> Ping: Can anyone give a review on this. Thank you Tristan On Feb 6, 2014, at 5:13 PM, Tristan Yan wrote: > Hi All > > Please help to review a simple fix for https://bugs.openjdk.java.net/browse/JDK-8030844 > > http://cr.openjdk.java.net/~tyan/JDK-8030844/webrev.00/. > > Description: > Change replace a ?/home/~user" folder with an test source path. Folder ?/home/~user? cause some problem whenever something wrong with the automount filesystem or an username lookup problem. > > Thank you > Tristan From tristan.yan at oracle.com Mon Feb 10 10:59:25 2014 From: tristan.yan at oracle.com (Tristan Yan) Date: Mon, 10 Feb 2014 18:59:25 +0800 Subject: RFR: JDK-8032050: TEST_BUG: java/rmi/activation/Activatable/shutdownGracefully/ShutdownGracefully.java fails intermittently In-Reply-To: <52EB60A8.7070307@oracle.com> References: <52E20974.7060706@oracle.com> <52E31F53.3070702@oracle.com> <52E4959C.3020907@oracle.com> <52E8AADE.6000300@oracle.com> <52EB60A8.7070307@oracle.com> Message-ID: <133FA08E-5731-437D-BB1F-AD9872CBDD36@oracle.com> Hi Stuart Could you help to review this. Thank you Tristan On Jan 31, 2014, at 4:36 PM, Tristan Yan wrote: > Thank you for fixing JDK-8023541. Then I leave ActivationLibrary.java for now. > I still did some clean up following your suggestion. > 1. I changed waitFor(long timeout) method, this method is going to use code like Process.waitFor(timeout, unit). This can be backported to JDK7. Also exitValue is kept as a return value. For making sure there is no Pipe leak, a cleanup thread will start when timeout happens. > 2. Change in ShutdownGracefully is a little tricky. I think we should just destroy JVM once exception is thrown. So I move the wait logic into try block instead keep them in finally block. > Can you receive it again. > http://cr.openjdk.java.net/~tyan/JDK-8032050/webrev.02/ > Thank you > Tristan > > On 01/29/2014 03:16 PM, Stuart Marks wrote: >> Hi Tristan, >> >> I don't want to put the workaround into ActivationLibrary.rmidRunning() for a null return from the lookup, because this is only a workaround for an actual bug in rmid initialization. See the review I just posted for JDK-8023541. >> >> Adding JavaVM.waitFor(timeout) is something that would be useful in general, but it needs to be handled carefully. It uses the new Process.waitFor(timeout, unit) which is new in Java SE 8; this makes backporting to 7 more complicated. Not clear whether we'll do so, but I don't want to forclose the opportunity without discussion. It's also not clear how one can get the vm's exit status after JavaVM.waitFor() has returned true. With the Process API it's possible simply to call waitFor() or exitValue(). With JavaVM, a new API needs to be created, or the rule has to be established that one must call JavaVM.waitFor() to collect the exit status as well as to close the pipes from the subprocess. If JavaVM.waitFor(timeout, unit) is called without subsequently calling JavaVM.waitFor(), the pipes are leaked. >> >> In ShutdownGracefully.java, the finally-block needs to check to see if rmid is still running, and if it is, to shut it down. Simply calling waitFor(timeout, unit) isn't sufficient, because if the rmid process is still running, it will be left running. >> >> The straightforward approach would be to call ActivationLibrary.rmidRunning() to test if it's still running. Unfortunately this isn't quite right, because rmidRunning() has a timeout loop in it -- which should probably be removed. (I think there's a bug for this.) Another approach would be simply to call rmid.destroy(). This calls rmid's shutdown() method first, which is reasonable, but I'm not sure it kills the process if that fails. In any case, this already has a timeout loop waiting for the process to die, so ShutdownGracefully.java needn't use a new waitFor(timeout, unit) call. >> >> Removing the commented-out code that starts with "no longer needed" is good, and removing the ShutdownDetectThread is also good, since that's unnecessary. >> >> There are some more cleanups I have in mind here but I'd like to see a revised webrev before proceeding. >> >> Thanks, >> >> s'marks >> >> On 1/25/14 8:57 PM, Tristan Yan wrote: >>> Hi Stuart >>> Thank you for your review and suggestion. >>> Yes, since this failure mode is very hard to be reproduced. I guess it's make sense to do some hack. And I also noticed in ActivationLibrary.rmidRunning. It does try to look up ActivationSystem but doesn't check if it's null. So I add the logic to make sure we will look up the non-null ActivationSystem. Also I did some cleanup if you don't mind. Add a waitFor(long timeout, TimeUnit unit) for JavaVM. Which we can have a better waitFor control. >>> I appreciate you can review the code again. >>> http://cr.openjdk.java.net/~tyan/JDK-8032050/webrev.01/ >>> Thank you >>> Tristan >>> >>> >>> On 01/25/2014 10:20 AM, Stuart Marks wrote: >>>> On 1/23/14 10:34 PM, Tristan Yan wrote: >>>>> Hi All >>>>> Could you review the bug fix for JDK-8032050. >>>>> >>>>> http://cr.openjdk.java.net/~tyan/JDK-8032050/webrev.00/ >>>>> >>>>> Description: >>>>> This rare happened failure caused because when RMID starts. It don't guarantee >>>>> sun.rmi.server.Activation.startActivation finishes. >>>>> Fix by adding a iterative getSystem with a 5 seconds timeout. >>>> >>>> Hi Tristan, >>>> >>>> Adding a timing/retry loop into this test isn't the correct approach for fixing this test. >>>> >>>> The timing loop isn't necessary because there is already a timing loop in RMID.start() in the RMI test library. (There's another timing loop in ActivationLibrary.rmidRunning() which should probably be removed.) So the intent of this library call is that it start rmid and wait for it to become ready. That logic doesn't need to be added to the test. >>>> >>>> In the bug report JDK-8032050 you had mentioned that the NullPointerException was suspicious. You're right! I took a look and it seemed like it was related to JDK-8023541, and I added a note to this effect to the bug report. The problem here is that rmid can come up and transiently return null instead of the stub of the activation system. That's what JDK-8023541 covers. I think that rmid itself needs to be fixed, though modifying the timing loop in the RMI test library to wait for rmid to come up *and* for the lookup to return non-null is an easy way to fix the problem. (Or at least cover it up.) >>>> >>>> The next step in the analysis is to determine, given that ActivationLibrary.getSystem can sometimes return null, whether this has actually caused this test failure. This is pretty easy to determine; just hack in a line "system = null" in the right place and run the test. I've done this, and the test times out and the output log is pretty much identical to the one in the bug report. (I recommend you try this yourself.) So I think it's fairly safe to say that the problem in JDK-8023541 has caused the failure listed in JDK-8032050. >>>> >>>> I can see a couple ways to proceed here. One way is just to close this out as a duplicate of JDK-8023541 since that bug caused this failure. >>>> >>>> Another is that this test could use some cleaning up. This bug certainly covers a failure, but the messages emitted are confusing and in some cases completely wrong. For example, the "rmid has shutdown" message at line 180 is incorrect, because in this case rmid is still running and the wait() call has timed out. Most of the code here can be replaced with calls to various bits of the RMI test library. There are a bunch of other things in this test that could be cleaned up as well. >>>> >>>> It's up to you how you'd like to proceed. >>>> >>>> s'marks >>> >> > From paul.sandoz at oracle.com Mon Feb 10 11:46:44 2014 From: paul.sandoz at oracle.com (Paul Sandoz) Date: Mon, 10 Feb 2014 12:46:44 +0100 Subject: Draft JEP on enhanced volatiles In-Reply-To: <52F77A1B.2070804@univ-mlv.fr> References: <52F527A4.3030802@cs.oswego.edu> <52F545B1.2070706@oracle.com> <52F60976.4060703@univ-mlv.fr> <52F66ECE.3040607@oracle.com> <52F77A1B.2070804@univ-mlv.fr> Message-ID: On Feb 9, 2014, at 1:52 PM, Remi Forax wrote: > On 02/08/2014 06:52 PM, Brian Goetz wrote: >>> can you be a little more specific and provide the way >>> foo.volatile.compareAndSet is compiled with this example: >> >> Hey, the JEP isn't even out of draft yet! Stop asking hard questions. > > :) > >> >> The current strawman, though, is to expose direct (CP-ready) MH forms (Lookup API / constant pool format TBD) for field get/set with fence, array element get/set with fence, field CAS, and array element CAS. Then the static compiler translates >> >> foo.volatile.cAS(n,m) >> >> as >> >> LDC #MH[lookupFieldCASer(foo)] >> aload n >> aload m >> invokevirtual "invokeExact" > > wow, you bring out the big guns ! > > I see a big issue with this approach, it requires to change the JVM spec each time you want to add a new operation. > Yes, although realistically i would expect new operations only to be added with each major release. > Moreover, overloads of LDC are usually re-written by JVMs has private opcodes, so why not directly use new opcodes, > it will take less space in the classfile too. > More work than expanding the kinds of method handle? I have not looked at this in detail but perhaps there might be a few new reference_kind items (for get/put/static fields) that can refer to a new constant pool entry consisting of a pair of CONSTANT_Fieldref_info and CONSTANT_InterfaceMethodref_info for VirtualXXX iface? Paul. From Alan.Bateman at oracle.com Mon Feb 10 11:47:17 2014 From: Alan.Bateman at oracle.com (Alan Bateman) Date: Mon, 10 Feb 2014 11:47:17 +0000 Subject: 8034043/8034026 - clean up of a few native code issues to align things with modules Message-ID: <52F8BC45.8040202@oracle.com> This patch is a clean-up to fix a number issues that arise trying to align native libraries with possible future modules. The first is the preferences API where the native code is currently linked into libjava for Windows/Linux/Solaris and libosx on OS X. This patch moves the native sources down to a prefs sub-directory (to follow long standing conventions) and links the resulting code into a new libprefs (all platforms). The rational for this is that the preferences API is a reasonable candidate for a module and it is odd to have its native methods in libraries that are likely to be in other modules. See also JDK-8034074 [1] for an opportunity for anyone so inclined to eliminate some of this code. The logging API is another case where it has native code linked into libjava. It's for the somewhat obscure case where the VM is running as a setuid process and the FileHandler is configured to include the user's home directory. For now, I've moved this to sun.misc.VM on the assumption that there may be other places where we need to know if we are running with setuid. We have to do a general clean-up of sun.misc and that may be the time to move it to a better home. In the security area then the OS X implementation of the native methods for sun.security.krb5.SCDynamicStoreConfig were put in the wrong location by the OS X port (src/macosx/native/java/util???) and also linked to the wrong library (libosx.dylib). This patch moves the source and links it into existing libosxkrb5.dylib along with the other OS X specific Kerberos code. This is code that is not exercised by the tests as it requires Open Directory to be configured. Thanks to Max Wang for trying out a preliminary version of this patch to ensure that it continues to work. One comment on SecurityLibraries.gmk is it's using hardcoded paths to the OS X frameworks. Erik Joelsson told me that this is on the build group's list of things to fix up in the future. The webrev with the proposed changes is here: http://cr.openjdk.java.net/~alanb/8034043%2b8034026/webrev/ -Alan. [1] https://bugs.openjdk.java.net/browse/JDK-8034074 From Alan.Bateman at oracle.com Mon Feb 10 12:08:38 2014 From: Alan.Bateman at oracle.com (Alan Bateman) Date: Mon, 10 Feb 2014 12:08:38 +0000 Subject: RFR for JDK-8030844:sun/rmi/rmic/classpath/RMICClassPathTest.java timeout in same binaries run In-Reply-To: <13F6E908-26F2-420A-9E25-AD00DF8E3769@oracle.com> References: <13F6E908-26F2-420A-9E25-AD00DF8E3769@oracle.com> Message-ID: <52F8C146.7060505@oracle.com> On 10/02/2014 10:57, Tristan Yan wrote: > Ping: Can anyone give a review on this. > Thank you > Tristan Changing the test so that it doesn't try to /home/~user seems reasonable to me. Stuart - I see you've been sponsoring Tristan's updates to the RMI tests. Are you going to take this one too? -Alan > > On Feb 6, 2014, at 5:13 PM, Tristan Yan wrote: > >> Hi All >> >> Please help to review a simple fix for https://bugs.openjdk.java.net/browse/JDK-8030844 >> >> http://cr.openjdk.java.net/~tyan/JDK-8030844/webrev.00/. >> >> Description: >> Change replace a ?/home/~user" folder with an test source path. Folder ?/home/~user? cause some problem whenever something wrong with the automount filesystem or an username lookup problem. >> >> Thank you >> Tristan From michael.cui at oracle.com Mon Feb 10 12:12:14 2014 From: michael.cui at oracle.com (michael cui) Date: Mon, 10 Feb 2014 20:12:14 +0800 Subject: RFR for JDK-8028711: TEST_BUG: Tests should pass through VM options: corelibs tests In-Reply-To: <52F89D96.7000601@oracle.com> References: <52E4C2E8.1060605@oracle.com> <52E4D1F2.4030406@oracle.com> <52E5B4C3.6080405@oracle.com> <52F455FC.3020009@oracle.com> <52F4BD7B.8080006@oracle.com> <52F87906.5040702@oracle.com> <52F89D96.7000601@oracle.com> Message-ID: <52F8C21E.10503@oracle.com> On 02/10/2014 05:36 PM, Alan Bateman wrote: > On 10/02/2014 07:00, michael cui wrote: >> : >> >> Please review the newest version at : >> http://cr.openjdk.java.net/~tyan/michael/JDK-8028711/webrev.03/ > This looks good to me. > > There are a couple of places where javac wasn't updated. A few that I > noticed are: > > test/com/sun/corba/5036554/TestCorbaBug.sh > test/sun/tools/jconsole/ResourceCheckTest.sh > test/sun/tools/native2ascii/resources/ImmutableResourceTest.sh > The original code change did adding ${TESTTOOLVMOPTS} to javac, but I dropped it from new proposal. My understanding is that vmoption for javac would not change the compiler's behaviour. I could be wrong. Do you think we really need to pass such vm options to javac? And if it is needed, than how about other java tools such as jar, rmiregistry, ... etc. > A minor comment on test/java/net/URLPermission/nstest/lookup.sh is > that the resulting line length is 182 characters and this will likely > be annoying for future side-by-side views. So I think I'd split this > while you are there. I will split it after finalize the changes that we want to make. > > -Alan. -Michael Cui From dl at cs.oswego.edu Mon Feb 10 12:33:00 2014 From: dl at cs.oswego.edu (Doug Lea) Date: Mon, 10 Feb 2014 07:33:00 -0500 Subject: Draft JEP on enhanced volatiles In-Reply-To: <52F8467B.3050004@oracle.com> References: <52F527A4.3030802@cs.oswego.edu> <52F8467B.3050004@oracle.com> Message-ID: <52F8C6FC.8050205@cs.oswego.edu> On 02/09/2014 10:24 PM, Tom Hawtin wrote: > I'm very concerned that this adds bulk to an already oversized language to > support something which should be incredibly obscure. A lot of us started out with the same concerns, but after a decade of exploration, we don't know of a defensible alternative. Back in JDK5, this functionality was needed by at most a dozen people writing code near the Java/JVM borderline. Using second-class cheats to work around lack of language support seemed acceptable. Barely acceptable, because they rely on Unsafe constructions that require special care to ensure safety otherwise guaranteed by JVMs, so increase the size of the trusted code base. Over the past decade the use of concurrent algorithms requiring fine-grained atomicity and ordering control intrinsics (which we periodically add to) has expanded to include cases not just isolated to low-level software. Enough so that other languages standards (including C/C++11) define them as intrinsics. Without better support, people needing them in Java will use Unsafe more. Or will use Java less. Among the constraints on solution is that several of these methods compile down to no-ops on many common platforms. Any further overhead encountered in arranging to do nothing has a good chance of measurably impairing some of the best concurrency support offered on any platform. > > On 07/02/2014 18:36, Doug Lea wrote: >> Alternatives >> ------------ >> >> We considered instead introducing new forms of "value type" that >> support volatile operations. However, this would be inconsistent with >> properties of other types, and would also require more effort for >> programmers to use. > > I assume by "value type" you mean a type with only constant fields of "value > types" and no identity (==, System.identityHashCode, monitor or equivalents). > Clearly volatiles don't fit well there. > > There's something that Joe-E calls a "selfless type" which is very similar. A > selfless type has no identity but may reference reference types. It gains the > benefits of being an inlineable object, like value types, but supports mutable > interfaces. That's a perfect fit. Not so perfect. For the memory semantics to be meaningful, the field MUST be a component, not MAY be. It is possible in theory to make a zillion JVM changes to correctly emulate this, but it is not a realistic option. More realistically, this might be approachable via "Packed" in a variant of the IBM Packed Object proposal. But this would be a more significant language change than adding .volatile expressions. -Doug From volker.simonis at gmail.com Mon Feb 10 13:30:29 2014 From: volker.simonis at gmail.com (Volker Simonis) Date: Mon, 10 Feb 2014 14:30:29 +0100 Subject: RFR(XS): 8033909: Objects.requireNonNull(T, Supplier) doesn't specify what happens if the passed supplier is null itself Message-ID: Hi, could you please review the following tiny change which fixes a problem in Objects.requireNonNull: http://cr.openjdk.java.net/~simonis/webrevs/8033909/ https://bugs.openjdk.java.net/browse/JDK-8033909 The problem is that Objects.requireNonNull(T, Supplier) does not check for the Supplier argument being null. Instead it relies on the fact, that the VM will implicitly throw a NullPointerException while trying to call .get on the Supplier argument during the creation of the explicit NullPointerException which is supposed to be thrown by Objects.requireNonNull(T, Supplier) for a null T argument. This makes the behavior of Objects.requireNonNull(T, Supplier) slightly different from the one of the overladed Objects.requireNonNull(T, String) version. The latter one always throws a NPE with a null message in the case where the String argument is null. For the first one, it depends on the implementation of the VM whether the trown NPE will have a null message or not (i.e. the VM is free to create a NPE with an arbitrary message). The bug report mentions that this behavior makes it hard to develop a conformance test for the scenario and suggests to tweak the JavaDoc of Objects.requireNonNull(T, Supplier) such that it allows NPE with an unspecified message. However, I think the easiest fix is to just check for the supplier object beeing null and explicitely creating a NPE with a null message in that case. This will align the behavior of Objects.requireNonNull(T, Supplier) with that of Objects.requireNonNull(T, String). Also notice that the extra null-check is only performed if the object argument T of Objects.requireNonNull(T, Supplier) is null, which should be the unusual case anyway. I therefore don't expect this change to have any negative performance impact. This webrev is for jdk9, but I think it should be also downported to jdk8. Thank you and best regards, Volker From jason_mehrens at hotmail.com Mon Feb 10 14:11:44 2014 From: jason_mehrens at hotmail.com (Jason Mehrens) Date: Mon, 10 Feb 2014 08:11:44 -0600 Subject: Random access in ArrayDeque In-Reply-To: <52F6867B.4060703@cs.oswego.edu> References: <231A3FB1-639F-4C06-823B-0CB9F1CB194A@oracle.com> <52F520EE.70709@redhat.com>, , <52F6867B.4060703@cs.oswego.edu> Message-ID: > Too bad there is no interface with only get(index), set(index, x), > and indexOf(x). Implementing only these would probably satisfy > all candidate usages. With default methods can't we modify the RandomAccess marker interface with hostile implementations? Jason From paul.sandoz at oracle.com Mon Feb 10 14:26:57 2014 From: paul.sandoz at oracle.com (Paul Sandoz) Date: Mon, 10 Feb 2014 15:26:57 +0100 Subject: Random access in ArrayDeque In-Reply-To: References: <231A3FB1-639F-4C06-823B-0CB9F1CB194A@oracle.com> <52F520EE.70709@redhat.com>, , <52F6867B.4060703@cs.oswego.edu> Message-ID: <3218EC3A-AFE1-445D-94E7-FBD7491304C4@oracle.com> On Feb 10, 2014, at 3:11 PM, Jason Mehrens wrote: >> Too bad there is no interface with only get(index), set(index, x), >> and indexOf(x). Implementing only these would probably satisfy >> all candidate usages. > > With default methods can't we modify the RandomAccess marker interface with hostile implementations? > I was pondering the same thing, but concluded to myself that the hostility was rather too hostile given that the interface is not a subtype of List. Paul. From chris.hegarty at oracle.com Mon Feb 10 14:53:59 2014 From: chris.hegarty at oracle.com (Chris Hegarty) Date: Mon, 10 Feb 2014 14:53:59 +0000 Subject: A hole in the serialization spec In-Reply-To: <52F4F6BB.2010207@redhat.com> References: <52F4F6BB.2010207@redhat.com> Message-ID: <52F8E807.2090403@oracle.com> David, " ... undefined in cases where the ObjectInputStream cannot resolve the class which defined the writeObject method in question." I'm not clear as to what this statement is about? I'm sure you already know this, and maybe in your environment do not care much about it, but having a read/writeObject not invoke the appropriate default read/write Object/Fields method is a serious impediment to evolving the serial form ( in a compatible way ). For example, if your class has no serializable fields in one revision, but adds a serializable field(s) in a subsequent revision. This could lead to a StreamCorruptedException, or some other undefined behavior. The OpenJDK sources do seem to be quite tolerant of this situation. I'm not entirely sure if this is a good or a bad thing. That said, I don't think we want to encourage this kind of behavior. -Chris. On 07/02/14 15:07, David M. Lloyd wrote: > Since the topic of serialization has come up recently, I'll take it as > an excuse to bring up a problem that I've run into a couple of times > with the serialization specification, which has resulted in user problems. > > If you read section 2.3 [1] of the specification, it says: > > "The class's writeObject method, if implemented, is responsible for > saving the state of the class. Either ObjectOutputStream's > defaultWriteObject or writeFields method must be called once (and only > once) before writing any optional data that will be needed by the > corresponding readObject method to restore the state of the object; even > if no optional data is written, defaultWriteObject or writeFields must > still be invoked once. If defaultWriteObject or writeFields is not > invoked once prior to the writing of optional data (if any), then the > behavior of instance deserialization is undefined in cases where the > ObjectInputStream cannot resolve the class which defined the writeObject > method in question." > > If you go to section 3.4 [2] of the specification, it reads: > > "The readObject method of the class, if implemented, is responsible for > restoring the state of the class. The values of every field of the > object whether transient or not, static or not are set to the default > value for the fields type. Either ObjectInputStream's defaultReadObject > or readFields method must be called once (and only once) before reading > any optional data written by the corresponding writeObject method; even > if no optional data is read, defaultReadObject or readFields must still > be invoked once." > > Now the problem: there are many classes in the wild which nevertheless > do not write/read fields. We cause an exception in such cases rather > than make up some undefined behavior. What I'm wondering is, is there > some sensible behavior that could be specified for this case? The > Oracle JDK seems to simply leave fields uninitialized in this case, > maybe that can be a specified behavior? > > [1] > http://docs.oracle.com/javase/7/docs/platform/serialization/spec/output.html#861 > > [2] > http://docs.oracle.com/javase/7/docs/platform/serialization/spec/input.html#2971 > > From rob.mckenna at oracle.com Mon Feb 10 14:50:15 2014 From: rob.mckenna at oracle.com (rob.mckenna at oracle.com) Date: Mon, 10 Feb 2014 14:50:15 +0000 Subject: hg: jdk8/tl/jdk: 7152892: some jtreg tests fail with permission denied Message-ID: <20140210145247.DD74462B24@hg.openjdk.java.net> Changeset: da4b0962ad11 Author: robm Date: 2014-02-10 14:35 +0000 URL: http://hg.openjdk.java.net/jdk8/tl/jdk/rev/da4b0962ad11 7152892: some jtreg tests fail with permission denied Reviewed-by: coffeys ! test/java/lang/ClassLoader/Assert.sh ! test/java/rmi/registry/readTest/readTest.sh ! test/java/util/zip/ZipFile/ReadZip.java ! test/sun/net/www/protocol/jar/jarbug/run.sh From david.lloyd at redhat.com Mon Feb 10 15:37:00 2014 From: david.lloyd at redhat.com (David M. Lloyd) Date: Mon, 10 Feb 2014 09:37:00 -0600 Subject: A hole in the serialization spec In-Reply-To: <52F8E807.2090403@oracle.com> References: <52F4F6BB.2010207@redhat.com> <52F8E807.2090403@oracle.com> Message-ID: <52F8F21C.80205@redhat.com> I agree that it's a problem; however it's also clear that there are many classes in the wild which have this problem. It would be nice if the behavior could _become_ defined *somehow* though. I can see at least four options: 1) do nothing :( 2) start throwing (or writing) an exception in write/readObject when stream ops are performed without reading fields (maybe can be disabled with a sys prop or something) 3) leave fields cleared and risk protocol issues 4) silently start reading/writing empty field information (risks protocol issues) Maybe there are better options I'm not thinking of. On 02/10/2014 08:53 AM, Chris Hegarty wrote: > David, > > " ... undefined in cases where the ObjectInputStream cannot resolve the > class which defined the writeObject method in question." > > I'm not clear as to what this statement is about? > > I'm sure you already know this, and maybe in your environment do not > care much about it, but having a read/writeObject not invoke the > appropriate default read/write Object/Fields method is a serious > impediment to evolving the serial form ( in a compatible way ). For > example, if your class has no serializable fields in one revision, but > adds a serializable field(s) in a subsequent revision. This could lead > to a StreamCorruptedException, or some other undefined behavior. > > The OpenJDK sources do seem to be quite tolerant of this situation. I'm > not entirely sure if this is a good or a bad thing. That said, I don't > think we want to encourage this kind of behavior. > > -Chris. > > On 07/02/14 15:07, David M. Lloyd wrote: >> Since the topic of serialization has come up recently, I'll take it as >> an excuse to bring up a problem that I've run into a couple of times >> with the serialization specification, which has resulted in user >> problems. >> >> If you read section 2.3 [1] of the specification, it says: >> >> "The class's writeObject method, if implemented, is responsible for >> saving the state of the class. Either ObjectOutputStream's >> defaultWriteObject or writeFields method must be called once (and only >> once) before writing any optional data that will be needed by the >> corresponding readObject method to restore the state of the object; even >> if no optional data is written, defaultWriteObject or writeFields must >> still be invoked once. If defaultWriteObject or writeFields is not >> invoked once prior to the writing of optional data (if any), then the >> behavior of instance deserialization is undefined in cases where the >> ObjectInputStream cannot resolve the class which defined the writeObject >> method in question." >> >> If you go to section 3.4 [2] of the specification, it reads: >> >> "The readObject method of the class, if implemented, is responsible for >> restoring the state of the class. The values of every field of the >> object whether transient or not, static or not are set to the default >> value for the fields type. Either ObjectInputStream's defaultReadObject >> or readFields method must be called once (and only once) before reading >> any optional data written by the corresponding writeObject method; even >> if no optional data is read, defaultReadObject or readFields must still >> be invoked once." >> >> Now the problem: there are many classes in the wild which nevertheless >> do not write/read fields. We cause an exception in such cases rather >> than make up some undefined behavior. What I'm wondering is, is there >> some sensible behavior that could be specified for this case? The >> Oracle JDK seems to simply leave fields uninitialized in this case, >> maybe that can be a specified behavior? >> >> [1] >> http://docs.oracle.com/javase/7/docs/platform/serialization/spec/output.html#861 >> >> >> [2] >> http://docs.oracle.com/javase/7/docs/platform/serialization/spec/input.html#2971 >> >> >> -- - DML From mandy.chung at oracle.com Mon Feb 10 15:43:17 2014 From: mandy.chung at oracle.com (Mandy Chung) Date: Mon, 10 Feb 2014 07:43:17 -0800 Subject: RFR: 7152892: some jtreg tests fail with permission denied In-Reply-To: <52F52990.5060200@oracle.com> References: <52F52990.5060200@oracle.com> Message-ID: <52F8F395.3070008@oracle.com> On 2/7/2014 10:44 AM, Rob McKenna wrote: > Hi folks, > > When files are copied by our test harness the permissions are left > unchanged. This can cause trouble for a few tests when the test folder > is read only. > > http://cr.openjdk.java.net/~robm/7152892/webrev.01/ AIX is missing in your change in Assert.sh. Assert.sh doesn't seem to be necessary. I think it can replace the shell test with @build to compile the classes. It may also need to change "java Assert" with the right classpath (I haven't checked in details) in Assert.java (which execs the java command). Do you mind looking into the possibility to remove Assert.sh? $ hg diff Assert.java diff --git a/test/java/lang/ClassLoader/Assert.java b/test/java/lang/ClassLoader/Assert.java --- a/test/java/lang/ClassLoader/Assert.java +++ b/test/java/lang/ClassLoader/Assert.java @@ -24,7 +24,8 @@ /* * @test * @bug 4290640 4785473 - * @run shell/timeout=300 Assert.sh + * @build package1.Class1 package2.Class2 package1.package3.Class3 Assert + * @run main/othervm Assert * @summary Test the assertion facility * @author Mike McCloskey */ Mandy From chris.hegarty at oracle.com Mon Feb 10 19:26:10 2014 From: chris.hegarty at oracle.com (Chris Hegarty) Date: Mon, 10 Feb 2014 19:26:10 +0000 Subject: JDK 9 RFR of JDK-8033527: Fix raw type lint warning in sun.nio.ch In-Reply-To: <52F08428.80100@oracle.com> References: <52F08428.80100@oracle.com> Message-ID: <673D2B79-ABF2-4637-A066-AC2C61ED7A5D@oracle.com> Looks good to me Joe. -Chris. On 4 Feb 2014, at 06:09, Joe Darcy wrote: > Hello, > > Please review this small fix to address > > JDK-803352: Fix raw type lint warning in sun.nio.ch > > --- a/src/share/classes/sun/nio/ch/Reflect.java Mon Feb 03 16:58:02 2014 -0500 > +++ b/src/share/classes/sun/nio/ch/Reflect.java Mon Feb 03 22:07:47 2014 -0800 > @@ -1,5 +1,5 @@ > /* > - * Copyright (c) 2000, 2011, Oracle and/or its affiliates. All rights reserved. > + * Copyright (c) 2000, 2014, Oracle and/or its affiliates. All rights reserved. > * DO NOT ALTER OR REMOVE COPYRIGHT NOTICES OR THIS FILE HEADER. > * > * This code is free software; you can redistribute it and/or modify it > @@ -75,7 +75,7 @@ > > static Method lookupMethod(String className, > String methodName, > - Class... paramTypes) > + Class... paramTypes) > { > try { > Class cl = Class.forName(className); > > The build succeeds with this change in place. > > Thanks, > > -Joe From mandy.chung at oracle.com Mon Feb 10 22:38:26 2014 From: mandy.chung at oracle.com (Mandy Chung) Date: Mon, 10 Feb 2014 14:38:26 -0800 Subject: RFR - 8029876: Add a test to verify that Logger.getLogger & LogManager.getLogManager don't throw NPE when System.out == null In-Reply-To: <52F3C084.5080502@oracle.com> References: <52F3AA5C.4080006@oracle.com> <52F3AF1F.5070604@oracle.com> <52F3C084.5080502@oracle.com> Message-ID: <52F954E2.9060009@oracle.com> On 2/6/2014 9:04 AM, Daniel Fuchs wrote: > On 2/6/14 4:49 PM, Mandy Chung wrote: >> Hi Daniel, >> >> On 2/6/2014 7:29 AM, Daniel Fuchs wrote: >>> https://bugs.openjdk.java.net/browse/JDK-8029876 >>> 8029876: Add a test to verify that Logger.getLogger >>> & LogManager.getLogManager don't throw NPE >>> when System.out == null >>> >>> webrev (trivial copy of the test to jdk9-dev): >>> http://cr.openjdk.java.net/~dfuchs/webrev_8029876/webrev.00/ >> >> Do you think this test can use the JavaAWTAccessStub as in other tests >> so that it doesn't depend on SunToolkit? TestMainAppContext.java also >> depends on SunTookit. > > Ah right! I remember we already spoke about that for the original test. > Here is a new version that doesn't depend on AWT. > > http://cr.openjdk.java.net/~dfuchs/webrev_8029876/webrev.01/ Looks good. Mandy From mike.duigou at oracle.com Mon Feb 10 23:24:45 2014 From: mike.duigou at oracle.com (Mike Duigou) Date: Mon, 10 Feb 2014 15:24:45 -0800 Subject: RFR(XS): 8033909: Objects.requireNonNull(T, Supplier) doesn't specify what happens if the passed supplier is null itself In-Reply-To: References: Message-ID: <412F60C4-6E43-483A-A87E-3384731A9483@oracle.com> I would prefer to leave the implementation as is and amend the documentation. The difference in behaviour between the overloads seems OK since it will still be valid for the Supplier to return null. The String overload reasonably allows null since the Throwable(String) constructor allows null message. (See Throwable.getMessage()). It seems like ignoring/masking the likely error of passing a null Supplier isn't really being helpful. YMMV, Mike On Feb 10 2014, at 05:30 , Volker Simonis wrote: > Hi, > > could you please review the following tiny change which fixes a > problem in Objects.requireNonNull: > > http://cr.openjdk.java.net/~simonis/webrevs/8033909/ > https://bugs.openjdk.java.net/browse/JDK-8033909 > > The problem is that Objects.requireNonNull(T, Supplier) does not check > for the Supplier argument being null. Instead it relies on the fact, > that the VM will implicitly throw a NullPointerException while trying > to call .get on the Supplier argument during the creation of the > explicit NullPointerException which is supposed to be thrown by > Objects.requireNonNull(T, Supplier) for a null T argument. > > This makes the behavior of Objects.requireNonNull(T, Supplier) > slightly different from the one of the overladed > Objects.requireNonNull(T, String) version. The latter one always > throws a NPE with a null message in the case where the String argument > is null. For the first one, it depends on the implementation of the VM > whether the trown NPE will have a null message or not (i.e. the VM is > free to create a NPE with an arbitrary message). > > The bug report mentions that this behavior makes it hard to develop a > conformance test for the scenario and suggests to tweak the JavaDoc of > Objects.requireNonNull(T, Supplier) such that it allows NPE with an > unspecified message. > > However, I think the easiest fix is to just check for the supplier > object beeing null and explicitely creating a NPE with a null message > in that case. This will align the behavior of > Objects.requireNonNull(T, Supplier) with that of > Objects.requireNonNull(T, String). Also notice that the extra > null-check is only performed if the object argument T of > Objects.requireNonNull(T, Supplier) is null, which should be the > unusual case anyway. I therefore don't expect this change to have any > negative performance impact. > > This webrev is for jdk9, but I think it should be also downported to jdk8. > > Thank you and best regards, > Volker From mike.duigou at oracle.com Mon Feb 10 23:43:43 2014 From: mike.duigou at oracle.com (Mike Duigou) Date: Mon, 10 Feb 2014 15:43:43 -0800 Subject: Draft JEP on enhanced volatiles In-Reply-To: <52F8C6FC.8050205@cs.oswego.edu> References: <52F527A4.3030802@cs.oswego.edu> <52F8467B.3050004@oracle.com> <52F8C6FC.8050205@cs.oswego.edu> Message-ID: On Feb 10 2014, at 04:33 , Doug Lea
wrote: > Among the constraints on solution is that several of these > methods compile down to no-ops on many common platforms. This is interpreted as a benefit. However, if the are no-ops only for the most common platform (x86) then we are likely to see a rough path for other platforms. Incorrect or inconsistent usage of these methods in user code will, instead of having no effect, result in unexpected and possibly mysterious runtime failures on other platforms. The JVM has been pretty good about hiding these differences thus far. It would be nice to see a "least common denominator" non-no-op mode as part of the VM implementation that provided the opportunity to test in the most pessimistic conditions. Mike From huizhe.wang at oracle.com Mon Feb 10 23:53:19 2014 From: huizhe.wang at oracle.com (huizhe wang) Date: Mon, 10 Feb 2014 15:53:19 -0800 Subject: RFR (JAXP): 8033980 : Xerces Update: datatype XMLGregorianCalendarImpl and DurationImpl Message-ID: <52F9666F.9010303@oracle.com> Hi, This is an update from Xerces for two datatype classes XMLGregorianCalendarImpl and DurationImpl. For details, please refer to: https://bugs.openjdk.java.net/browse/JDK-8033980 webrevs: http://cr.openjdk.java.net/~joehw/jdk9/8033980/webrev/ Tests included for: 1243, 1416, 1097 (Note that JDK did not have this bug) No test: 1343: remove unused code Existing tests: JAXP SQE and Unit tests passed. Thanks, Joe From stuart.marks at oracle.com Tue Feb 11 00:32:40 2014 From: stuart.marks at oracle.com (Stuart Marks) Date: Mon, 10 Feb 2014 16:32:40 -0800 Subject: RFR: JDK-8032050: TEST_BUG: java/rmi/activation/Activatable/shutdownGracefully/ShutdownGracefully.java fails intermittently In-Reply-To: <133FA08E-5731-437D-BB1F-AD9872CBDD36@oracle.com> References: <52E20974.7060706@oracle.com> <52E31F53.3070702@oracle.com> <52E4959C.3020907@oracle.com> <52E8AADE.6000300@oracle.com> <52EB60A8.7070307@oracle.com> <133FA08E-5731-437D-BB1F-AD9872CBDD36@oracle.com> Message-ID: <52F96FA8.3010102@oracle.com> Hi Tristan, Sorry about my recurring delays. Several comments on these changes. JavaVM.java -- The waitFor(timeout) method is mostly ok. The new thread started at line 208 and following seems unnecessary, though. This code is reached when a timeout occurs, so throwing TimeoutException is the only thing necessary in this case. Should the code to start the new thread be removed? There should be a similar check for vm == null as in the waitFor() [no args] method. ShutdownGracefully.java -- The condition that's checked after calling rmid.waitFor(SHUTDOWN_TIMEOUT) is incorrect. It's testing the exit status against zero. Offhand, when and if rmid exits, it might exit with a nonzero exit status. If rmid has exited at this point, then the test should succeed. Instead of testing against zero, the code should catch TimeoutException, which means that rmid is still running. It's probably reasonable to catch TimeoutException, print a message, and then let the finally-block destroy the rmid. Calling TestLibrary.bomb() from within the try-block is confusing, since that method throws an exception, which is then caught by the catch-block, when then calls TestLibrary.bomb() again. We should also make sure to test the success case properly. If rmid.waitFor() returns in a timely fashion without throwing TimeoutException, it doesn't matter what the exit status is. (It might be worth printing it out.) At that point we know that rmid *has* exited gracefully, so we need to set rmid to null so that the finally-block doesn't attempt to destroy rmid redundantly. Some additional messages about rmid having exited and the test passing are also warranted for this case. Some additional cleanup can be done here as well, over and above the changes you've proposed. (This stuff is left over from earlier RMI test messes.) In order to shut down an active object, the code here spawns a new thread that sleeps for a while and then deactivates this object. This isn't necessary. (It might have been necessary in the past.) It's sufficient simply to unexport this object and then deactivate it, directly within the shutdown() method. See test/java/rmi/activation/ActivationSystem/unregisterGroup/UnregisterGroup.java for an example of this. In addition, the run() method can be removed, and the "implements Runnable" declaration can also be removed from the ShutdownGracefully test class. Finally, revisiting some code farther up in the test, the try-block at lines 135-140 issues a registration request that the test expects to fail. If it succeeds, the message at line 139 isn't very clear; it should say that the registration request succeeded unexpectedly. This should cause the test to fail. We still probably want to go through the waitFor(timeout) path and eventual rmid cleanup, but a flag should be set here to ensure that the test indeed fails if the registration succeeds unexpectedly, and the messages should clearly indicate that this is going on. A good way to test this last case is to change rmid's security manager to the normal security manager java.lang.SecurityManager instead of TestSecurityManager. Thanks, s'marks On 2/10/14 2:59 AM, Tristan Yan wrote: > Hi Stuart > Could you help to review this. > Thank you > Tristan > > On Jan 31, 2014, at 4:36 PM, Tristan Yan > wrote: > >> Thank you for fixing JDK-8023541. Then I leave ActivationLibrary.java for now. >> I still did some clean up following your suggestion. >> 1. I changed waitFor(long timeout) method, this method is going to use code >> like Process.waitFor(timeout, unit). This can be backported to JDK7. Also >> exitValue is kept as a return value. For making sure there is no Pipe leak, a >> cleanup thread will start when timeout happens. >> 2. Change in ShutdownGracefully is a little tricky. I think we should just >> destroy JVM once exception is thrown. So I move the wait logic into try block >> instead keep them in finally block. >> Can you receive it again. >> http://cr.openjdk.java.net/~tyan/JDK-8032050/webrev.02/ >> Thank you >> Tristan >> >> On 01/29/2014 03:16 PM, Stuart Marks wrote: >>> Hi Tristan, >>> >>> I don't want to put the workaround into ActivationLibrary.rmidRunning() for a >>> null return from the lookup, because this is only a workaround for an actual >>> bug in rmid initialization. See the review I just posted for JDK-8023541. >>> >>> Adding JavaVM.waitFor(timeout) is something that would be useful in general, >>> but it needs to be handled carefully. It uses the new >>> Process.waitFor(timeout, unit) which is new in Java SE 8; this makes >>> backporting to 7 more complicated. Not clear whether we'll do so, but I don't >>> want to forclose the opportunity without discussion. It's also not clear how >>> one can get the vm's exit status after JavaVM.waitFor() has returned true. >>> With the Process API it's possible simply to call waitFor() or exitValue(). >>> With JavaVM, a new API needs to be created, or the rule has to be established >>> that one must call JavaVM.waitFor() to collect the exit status as well as to >>> close the pipes from the subprocess. If JavaVM.waitFor(timeout, unit) is >>> called without subsequently calling JavaVM.waitFor(), the pipes are leaked. >>> >>> In ShutdownGracefully.java, the finally-block needs to check to see if rmid >>> is still running, and if it is, to shut it down. Simply calling >>> waitFor(timeout, unit) isn't sufficient, because if the rmid process is still >>> running, it will be left running. >>> >>> The straightforward approach would be to call ActivationLibrary.rmidRunning() >>> to test if it's still running. Unfortunately this isn't quite right, because >>> rmidRunning() has a timeout loop in it -- which should probably be removed. >>> (I think there's a bug for this.) Another approach would be simply to call >>> rmid.destroy(). This calls rmid's shutdown() method first, which is >>> reasonable, but I'm not sure it kills the process if that fails. In any case, >>> this already has a timeout loop waiting for the process to die, so >>> ShutdownGracefully.java needn't use a new waitFor(timeout, unit) call. >>> >>> Removing the commented-out code that starts with "no longer needed" is good, >>> and removing the ShutdownDetectThread is also good, since that's unnecessary. >>> >>> There are some more cleanups I have in mind here but I'd like to see a >>> revised webrev before proceeding. >>> >>> Thanks, >>> >>> s'marks >>> >>> On 1/25/14 8:57 PM, Tristan Yan wrote: >>>> Hi Stuart >>>> Thank you for your review and suggestion. >>>> Yes, since this failure mode is very hard to be reproduced. I guess it's >>>> make sense to do some hack. And I also noticed in >>>> ActivationLibrary.rmidRunning. It does try to look up ActivationSystem but >>>> doesn't check if it's null. So I add the logic to make sure we will look up >>>> the non-null ActivationSystem. Also I did some cleanup if you don't mind. >>>> Add a waitFor(long timeout, TimeUnit unit) for JavaVM. Which we can have a >>>> better waitFor control. >>>> I appreciate you can review the code again. >>>> http://cr.openjdk.java.net/~tyan/JDK-8032050/webrev.01/ >>>> Thank you >>>> Tristan >>>> >>>> >>>> On 01/25/2014 10:20 AM, Stuart Marks wrote: >>>>> On 1/23/14 10:34 PM, Tristan Yan wrote: >>>>>> Hi All >>>>>> Could you review the bug fix for JDK-8032050. >>>>>> >>>>>> http://cr.openjdk.java.net/~tyan/JDK-8032050/webrev.00/ >>>>>> >>>>>> Description: >>>>>> This rare happened failure caused because when RMID starts. It don't >>>>>> guarantee >>>>>> sun.rmi.server.Activation.startActivation finishes. >>>>>> Fix by adding a iterative getSystem with a 5 seconds timeout. >>>>> >>>>> Hi Tristan, >>>>> >>>>> Adding a timing/retry loop into this test isn't the correct approach for >>>>> fixing this test. >>>>> >>>>> The timing loop isn't necessary because there is already a timing loop in >>>>> RMID.start() in the RMI test library. (There's another timing loop in >>>>> ActivationLibrary.rmidRunning() which should probably be removed.) So the >>>>> intent of this library call is that it start rmid and wait for it to become >>>>> ready. That logic doesn't need to be added to the test. >>>>> >>>>> In the bug report JDK-8032050 you had mentioned that the >>>>> NullPointerException was suspicious. You're right! I took a look and it >>>>> seemed like it was related to JDK-8023541, and I added a note to this >>>>> effect to the bug report. The problem here is that rmid can come up and >>>>> transiently return null instead of the stub of the activation system. >>>>> That's what JDK-8023541 covers. I think that rmid itself needs to be fixed, >>>>> though modifying the timing loop in the RMI test library to wait for rmid >>>>> to come up *and* for the lookup to return non-null is an easy way to fix >>>>> the problem. (Or at least cover it up.) >>>>> >>>>> The next step in the analysis is to determine, given that >>>>> ActivationLibrary.getSystem can sometimes return null, whether this has >>>>> actually caused this test failure. This is pretty easy to determine; just >>>>> hack in a line "system = null" in the right place and run the test. I've >>>>> done this, and the test times out and the output log is pretty much >>>>> identical to the one in the bug report. (I recommend you try this >>>>> yourself.) So I think it's fairly safe to say that the problem in >>>>> JDK-8023541 has caused the failure listed in JDK-8032050. >>>>> >>>>> I can see a couple ways to proceed here. One way is just to close this out >>>>> as a duplicate of JDK-8023541 since that bug caused this failure. >>>>> >>>>> Another is that this test could use some cleaning up. This bug certainly >>>>> covers a failure, but the messages emitted are confusing and in some cases >>>>> completely wrong. For example, the "rmid has shutdown" message at line 180 >>>>> is incorrect, because in this case rmid is still running and the wait() >>>>> call has timed out. Most of the code here can be replaced with calls to >>>>> various bits of the RMI test library. There are a bunch of other things in >>>>> this test that could be cleaned up as well. >>>>> >>>>> It's up to you how you'd like to proceed. >>>>> >>>>> s'marks >>>> >>> >> > From bourges.laurent at gmail.com Tue Feb 11 08:51:09 2014 From: bourges.laurent at gmail.com (=?ISO-8859-1?Q?Laurent_Bourg=E8s?=) Date: Tue, 11 Feb 2014 09:51:09 +0100 Subject: Fwd: Improve large array allocation / gc & intrinsics In-Reply-To: References: Message-ID: Please, could you give me your opinions on the following ideas for a JDK9 RFE ? Is it worth? Interesting or totally stupid with current hotspot / gc ? How hard / long would it be to study and make a prototype ? Any other ideas to improve array performance like improving again the array bound check elimination or fill / clear intrinsics ... Who would be interested by this topics ? Laurent ---------- Message transf?r? ---------- De : "Laurent Bourg?s" Date : 10 f?vr. 2014 10:24 Objet : Improve large array allocation / gc & intrinsics ? : "core-libs-dev" , "discuss" < discuss at openjdk.java.net> Cc : > Dear all, > > I would like to propose a JDK9 RFE to improve JVM efficiency when > dealing with large arrays (allocation + gc). > > In several scientific applications (and my patched java2d pisces), > many large arrays are needed, created on the fly and it becomes very > painful to recycle them using an efficient array cache (concurrency, > cache size tuning, clear + cache eviction issues). > > In this case, the GC overhead leads to a big performance penalty > (several hundred megabytes per seconds). > > As such array cache are very efficient in an application context, I am > wondering if that approach could be promoted at the JDK level itself: > > - provide a new array allocator for large arrays that can return > larger arrays than expected (size = 4 or 8 multiples) using array > caches (per thread ?) stored in a dedicated JVM large memory area (GC > aware) and providing efficient cache eviction policies > > - may support for both clean arrays (zero filled) or dirty arrays > because some algorithms does not need zero-filled arrays. > > - improve JVM intrinsics (array clear, fill) to achieve maximum > performance ie take into account the data alignment (4 or 8 multiples) > > - upgrade GC to recycle such 'cached' arrays (clean), update usage > statistics and manage cache eviction policy (avoid wasting memory) > > Please give me your feedback & opinion and evaluate if this feature > seems possible to implement into the JDK (hotspot, gc, core-libs) ... > > What is the procedure to create such JDK9 RFE ? > > Best regards, > > Laurent Bourg?s ---------- Message transf?r? ---------- De : "Laurent Bourg?s" Date : 10 f?vr. 2014 10:24 Objet : Improve large array allocation / gc & intrinsics ? : "core-libs-dev" , "discuss" < discuss at openjdk.java.net> Cc : Dear all, > > I would like to propose a JDK9 RFE to improve JVM efficiency when > dealing with large arrays (allocation + gc). > > In several scientific applications (and my patched java2d pisces), > many large arrays are needed, created on the fly and it becomes very > painful to recycle them using an efficient array cache (concurrency, > cache size tuning, clear + cache eviction issues). > > In this case, the GC overhead leads to a big performance penalty > (several hundred megabytes per seconds). > > As such array cache are very efficient in an application context, I am > wondering if that approach could be promoted at the JDK level itself: > > - provide a new array allocator for large arrays that can return > larger arrays than expected (size = 4 or 8 multiples) using array > caches (per thread ?) stored in a dedicated JVM large memory area (GC > aware) and providing efficient cache eviction policies > > - may support for both clean arrays (zero filled) or dirty arrays > because some algorithms does not need zero-filled arrays. > > - improve JVM intrinsics (array clear, fill) to achieve maximum > performance ie take into account the data alignment (4 or 8 multiples) > > - upgrade GC to recycle such 'cached' arrays (clean), update usage > statistics and manage cache eviction policy (avoid wasting memory) > > Please give me your feedback & opinion and evaluate if this feature > seems possible to implement into the JDK (hotspot, gc, core-libs) ... > > What is the procedure to create such JDK9 RFE ? > > Best regards, > > Laurent Bourg?s > From volker.simonis at gmail.com Tue Feb 11 09:03:27 2014 From: volker.simonis at gmail.com (Volker Simonis) Date: Tue, 11 Feb 2014 10:03:27 +0100 Subject: RFR(XS): 8033909: Objects.requireNonNull(T, Supplier) doesn't specify what happens if the passed supplier is null itself In-Reply-To: <412F60C4-6E43-483A-A87E-3384731A9483@oracle.com> References: <412F60C4-6E43-483A-A87E-3384731A9483@oracle.com> Message-ID: Hi Mike, as described in the bug comments, I still think that correctly handling a null Supplier is the cleanest and easiest solution. Without the change our VM will throw the following NPE: java.lang.NullPointerException: while trying to invoke the method java.util.function.Supplier.get() of a null object loaded from the second parameter of the method at java.util.Objects.requireNonNull(Objects.java:290) which is more descriptive and perfectly legal, but it may be not what the user expects (i.e. a NPE with a null message). Anyhow, I don't insist on my change. If everybody agrees that it would be better to change the documentation then please go ahead. But keep in mind that Joe also had some objections against the documentation changes in the bug comments. Regards, Volker On Tue, Feb 11, 2014 at 12:24 AM, Mike Duigou wrote: > I would prefer to leave the implementation as is and amend the documentation. The difference in behaviour between the overloads seems OK since it will still be valid for the Supplier to return null. The String overload reasonably allows null since the Throwable(String) constructor allows null message. (See Throwable.getMessage()). > > It seems like ignoring/masking the likely error of passing a null Supplier isn't really being helpful. > > YMMV, > > Mike > > On Feb 10 2014, at 05:30 , Volker Simonis wrote: > >> Hi, >> >> could you please review the following tiny change which fixes a >> problem in Objects.requireNonNull: >> >> http://cr.openjdk.java.net/~simonis/webrevs/8033909/ >> https://bugs.openjdk.java.net/browse/JDK-8033909 >> >> The problem is that Objects.requireNonNull(T, Supplier) does not check >> for the Supplier argument being null. Instead it relies on the fact, >> that the VM will implicitly throw a NullPointerException while trying >> to call .get on the Supplier argument during the creation of the >> explicit NullPointerException which is supposed to be thrown by >> Objects.requireNonNull(T, Supplier) for a null T argument. >> >> This makes the behavior of Objects.requireNonNull(T, Supplier) >> slightly different from the one of the overladed >> Objects.requireNonNull(T, String) version. The latter one always >> throws a NPE with a null message in the case where the String argument >> is null. For the first one, it depends on the implementation of the VM >> whether the trown NPE will have a null message or not (i.e. the VM is >> free to create a NPE with an arbitrary message). >> >> The bug report mentions that this behavior makes it hard to develop a >> conformance test for the scenario and suggests to tweak the JavaDoc of >> Objects.requireNonNull(T, Supplier) such that it allows NPE with an >> unspecified message. >> >> However, I think the easiest fix is to just check for the supplier >> object beeing null and explicitely creating a NPE with a null message >> in that case. This will align the behavior of >> Objects.requireNonNull(T, Supplier) with that of >> Objects.requireNonNull(T, String). Also notice that the extra >> null-check is only performed if the object argument T of >> Objects.requireNonNull(T, Supplier) is null, which should be the >> unusual case anyway. I therefore don't expect this change to have any >> negative performance impact. >> >> This webrev is for jdk9, but I think it should be also downported to jdk8. >> >> Thank you and best regards, >> Volker > From Alan.Bateman at oracle.com Tue Feb 11 10:31:34 2014 From: Alan.Bateman at oracle.com (Alan Bateman) Date: Tue, 11 Feb 2014 10:31:34 +0000 Subject: 8034175: Remove use of UseVMInterruptibleIO from tests Message-ID: <52F9FC06.2030101@oracle.com> Interruptible I/O was a (mostly Solaris only) mis-feature that we've been eradicating over a number of releases. In JDK 6 the VM option UseVMInterruptibleIO was added to allow it be disabled, in JDK 7 it was switched to being disabled by default, in JDK 8 we removed the dependency from the libraries (except classic networking, meaning java.net, that still needs to be done). Now in JDK 9, Frederic Parain is removing the VM support as part of JDK-6546236 [1]. As part of that discussion, Frederic pointed out that we still have 3 tests in the jdk repository that use specify this option. The attached (trivial) patch removes the option from the 3 tests. In each case then I've removed the @run line as there isn't any obvious that requires a new VM be spun up for each test. -Alan [1] http://mail.openjdk.java.net/pipermail/hotspot-runtime-dev/2014-February/010839.html diff --git a/test/java/nio/file/Files/InterruptCopy.java b/test/java/nio/file/Files/InterruptCopy.java --- a/test/java/nio/file/Files/InterruptCopy.java +++ b/test/java/nio/file/Files/InterruptCopy.java @@ -25,7 +25,6 @@ * @bug 4313887 6993267 * @summary Unit test for Sun-specific ExtendedCopyOption.INTERRUPTIBLE option * @library .. - * @run main/othervm -XX:-UseVMInterruptibleIO InterruptCopy */ import java.nio.file.*; diff --git a/test/java/util/concurrent/BlockingQueue/Interrupt.java b/test/java/util/concurrent/BlockingQueue/Interrupt.java --- a/test/java/util/concurrent/BlockingQueue/Interrupt.java +++ b/test/java/util/concurrent/BlockingQueue/Interrupt.java @@ -25,7 +25,6 @@ * @test * @bug 6384064 * @summary Check proper handling of interrupts - * @run main/othervm -XX:-UseVMInterruptibleIO Interrupt * @author Martin Buchholz */ diff --git a/test/java/util/concurrent/ThreadPoolExecutor/ThrowingTasks.java b/test/java/util/concurrent/ThreadPoolExecutor/ThrowingTasks.java --- a/test/java/util/concurrent/ThreadPoolExecutor/ThrowingTasks.java +++ b/test/java/util/concurrent/ThreadPoolExecutor/ThrowingTasks.java @@ -25,7 +25,6 @@ * @test * @bug 6450200 6450205 6450207 6450211 * @summary Test proper handling of tasks that terminate abruptly - * @run main/othervm -XX:-UseVMInterruptibleIO ThrowingTasks * @author Martin Buchholz */ From chris.hegarty at oracle.com Tue Feb 11 10:36:27 2014 From: chris.hegarty at oracle.com (Chris Hegarty) Date: Tue, 11 Feb 2014 10:36:27 +0000 Subject: 8034175: Remove use of UseVMInterruptibleIO from tests In-Reply-To: <52F9FC06.2030101@oracle.com> References: <52F9FC06.2030101@oracle.com> Message-ID: <575D7215-AFEA-432D-9744-1C5F015B9F30@oracle.com> Looks good to me Alan. -Chris. On 11 Feb 2014, at 10:31, Alan Bateman wrote: > > Interruptible I/O was a (mostly Solaris only) mis-feature that we've been eradicating over a number of releases. In JDK 6 the VM option UseVMInterruptibleIO was added to allow it be disabled, in JDK 7 it was switched to being disabled by default, in JDK 8 we removed the dependency from the libraries (except classic networking, meaning java.net, that still needs to be done). Now in JDK 9, Frederic Parain is removing the VM support as part of JDK-6546236 [1]. > > As part of that discussion, Frederic pointed out that we still have 3 tests in the jdk repository that use specify this option. The attached (trivial) patch removes the option from the 3 tests. In each case then I've removed the @run line as there isn't any obvious that requires a new VM be spun up for each test. > > -Alan > > [1] http://mail.openjdk.java.net/pipermail/hotspot-runtime-dev/2014-February/010839.html > > > diff --git a/test/java/nio/file/Files/InterruptCopy.java b/test/java/nio/file/Files/InterruptCopy.java > --- a/test/java/nio/file/Files/InterruptCopy.java > +++ b/test/java/nio/file/Files/InterruptCopy.java > @@ -25,7 +25,6 @@ > * @bug 4313887 6993267 > * @summary Unit test for Sun-specific ExtendedCopyOption.INTERRUPTIBLE option > * @library .. > - * @run main/othervm -XX:-UseVMInterruptibleIO InterruptCopy > */ > > import java.nio.file.*; > diff --git a/test/java/util/concurrent/BlockingQueue/Interrupt.java b/test/java/util/concurrent/BlockingQueue/Interrupt.java > --- a/test/java/util/concurrent/BlockingQueue/Interrupt.java > +++ b/test/java/util/concurrent/BlockingQueue/Interrupt.java > @@ -25,7 +25,6 @@ > * @test > * @bug 6384064 > * @summary Check proper handling of interrupts > - * @run main/othervm -XX:-UseVMInterruptibleIO Interrupt > * @author Martin Buchholz > */ > > diff --git a/test/java/util/concurrent/ThreadPoolExecutor/ThrowingTasks.java b/test/java/util/concurrent/ThreadPoolExecutor/ThrowingTasks.java > --- a/test/java/util/concurrent/ThreadPoolExecutor/ThrowingTasks.java > +++ b/test/java/util/concurrent/ThreadPoolExecutor/ThrowingTasks.java > @@ -25,7 +25,6 @@ > * @test > * @bug 6450200 6450205 6450207 6450211 > * @summary Test proper handling of tasks that terminate abruptly > - * @run main/othervm -XX:-UseVMInterruptibleIO ThrowingTasks > * @author Martin Buchholz > */ From Alan.Bateman at oracle.com Tue Feb 11 11:10:06 2014 From: Alan.Bateman at oracle.com (Alan Bateman) Date: Tue, 11 Feb 2014 11:10:06 +0000 Subject: 8034175: Remove use of UseVMInterruptibleIO from tests In-Reply-To: <575D7215-AFEA-432D-9744-1C5F015B9F30@oracle.com> References: <52F9FC06.2030101@oracle.com> <575D7215-AFEA-432D-9744-1C5F015B9F30@oracle.com> Message-ID: <52FA050E.5010501@oracle.com> On 11/02/2014 10:36, Chris Hegarty wrote: > Looks good to me Alan. > > -Chris. > I checked the jsr166 CVS and both of the j.u.concurrent tests also specify UseVMInterruptibleIO. I assume Martin will update these. -Alan From chris.hegarty at oracle.com Tue Feb 11 11:13:35 2014 From: chris.hegarty at oracle.com (Chris Hegarty) Date: Tue, 11 Feb 2014 11:13:35 +0000 Subject: 8034175: Remove use of UseVMInterruptibleIO from tests In-Reply-To: <52FA050E.5010501@oracle.com> References: <52F9FC06.2030101@oracle.com> <575D7215-AFEA-432D-9744-1C5F015B9F30@oracle.com> <52FA050E.5010501@oracle.com> Message-ID: <3D00C6E5-2742-483E-A121-D7C899DFF617@oracle.com> On 11 Feb 2014, at 11:10, Alan Bateman wrote: > On 11/02/2014 10:36, Chris Hegarty wrote: >> Looks good to me Alan. >> >> -Chris. >> > I checked the jsr166 CVS and both of the j.u.concurrent tests also specify UseVMInterruptibleIO. I assume Martin will update these. Yes, I see this too. Martin, Can you please update the 166 CVS as per Alan?s patch? -Chris. > > -Alan From Alan.Bateman at oracle.com Tue Feb 11 11:19:58 2014 From: Alan.Bateman at oracle.com (Alan Bateman) Date: Tue, 11 Feb 2014 11:19:58 +0000 Subject: Time to remove sun.misc.Service? Message-ID: <52FA075E.40109@oracle.com> It was never meant to be used by anything outside of the JDK and there has been a standard API (java.util.ServiceLoader) since JDK 6 (released in 2006). -Alan From lance.andersen at oracle.com Tue Feb 11 11:43:43 2014 From: lance.andersen at oracle.com (Lance @ Oracle) Date: Tue, 11 Feb 2014 06:43:43 -0500 Subject: Time to remove sun.misc.Service? In-Reply-To: <52FA075E.40109@oracle.com> References: <52FA075E.40109@oracle.com> Message-ID: <46D166B3-A7D1-4000-B5BC-B9C6CD743159@oracle.com> +1 Lance Andersen| Principal Member of Technical Staff | +1.781.442.2037 Oracle Java Engineering 1 Network Drive Burlington, MA 01803 Lance.Andersen at oracle.com Sent from my iPad On Feb 11, 2014, at 6:19 AM, Alan Bateman wrote: > > It was never meant to be used by anything outside of the JDK and there has been a standard API (java.util.ServiceLoader) since JDK 6 (released in 2006). > > -Alan From paul.sandoz at oracle.com Tue Feb 11 11:44:03 2014 From: paul.sandoz at oracle.com (Paul Sandoz) Date: Tue, 11 Feb 2014 12:44:03 +0100 Subject: Time to remove sun.misc.Service? In-Reply-To: <52FA075E.40109@oracle.com> References: <52FA075E.40109@oracle.com> Message-ID: <323EA730-D2E9-4122-8A76-0658F31C6CE7@oracle.com> On Feb 11, 2014, at 12:19 PM, Alan Bateman wrote: > > It was never meant to be used by anything outside of the JDK and there has been a standard API (java.util.ServiceLoader) since JDK 6 (released in 2006). > Scrub it! AFAICT it is not widely used (looking at the use of s.m.Service static methods on grep code there are only a handful of dependent artifacts). And the upgrade is quite easy. Paul. From volker.simonis at gmail.com Tue Feb 11 14:57:13 2014 From: volker.simonis at gmail.com (Volker Simonis) Date: Tue, 11 Feb 2014 15:57:13 +0100 Subject: RFR(S): 8034087: XML parser may overwrite element content if that content falls onto the border of an entity scanner buffer Message-ID: Hi, after opening this bug yesterday for an issue found by my colleague Steffen Schreiber we realized that this is actually a duplicate of "8027359: XML parser returns incorrect parsing results" (https://bugs.openjdk.java.net/browse/JDK-8027359). While there's no need now to submit a patch anymore, we'd nevertheless like to contribute at least our test case for this issue: http://cr.openjdk.java.net/~simonis/webrevs/8034087/ The webrev is against jdk9-client but we'd like to also downport this test to jdk7 and jdk8 to track that the fix for 8027359 will be correctly downported to these releases as well. I will sponsor this change if somebody would be so kind to review it. Thank you and best regards, Volker From martinrb at google.com Tue Feb 11 15:52:30 2014 From: martinrb at google.com (Martin Buchholz) Date: Tue, 11 Feb 2014 07:52:30 -0800 Subject: 8034175: Remove use of UseVMInterruptibleIO from tests In-Reply-To: <3D00C6E5-2742-483E-A121-D7C899DFF617@oracle.com> References: <52F9FC06.2030101@oracle.com> <575D7215-AFEA-432D-9744-1C5F015B9F30@oracle.com> <52FA050E.5010501@oracle.com> <3D00C6E5-2742-483E-A121-D7C899DFF617@oracle.com> Message-ID: Thanks for doing this cleanup. ThrowingTasks fiddles with uncaught exception handlers and security managers, so should remain otherVM. Interrupt does not, but could use a tiny bit of cleanup hygiene: Index: src/test/jtreg/util/concurrent/BlockingQueue/Interrupt.java =================================================================== RCS file: /export/home/jsr166/jsr166/jsr166/src/test/jtreg/util/concurrent/BlockingQueue/Interrupt.java,v retrieving revision 1.3 diff -u -r1.3 Interrupt.java --- src/test/jtreg/util/concurrent/BlockingQueue/Interrupt.java 27 Sep 2010 19:15:15 -0000 1.3 +++ src/test/jtreg/util/concurrent/BlockingQueue/Interrupt.java 11 Feb 2014 15:47:00 -0000 @@ -25,7 +25,6 @@ * @test * @bug 6384064 * @summary Check proper handling of interrupts - * @run main/othervm -XX:-UseVMInterruptibleIO Interrupt * @author Martin Buchholz */ @@ -49,7 +48,8 @@ } } - static void checkInterrupted(Iterable fs) { + static void checkInterrupted(Iterable fs) + throws InterruptedException { final Executor immediateExecutor = new Executor() { public void execute(Runnable r) { r.run(); }}; @@ -61,6 +61,7 @@ checkInterrupted0(fs, immediateExecutor); checkInterrupted0(fs, delayedExecutor); stpe.shutdown(); + check(stpe.awaitTermination(10, SECONDS)); } static void testQueue(final BlockingQueue q) { @@ -109,8 +110,10 @@ } checkInterrupted(fs); } catch (Throwable t) { - System.out.printf("Failed: %s%n", q.getClass().getSimpleName()); - unexpected(t); + System.out.printf("Failed: %s%n", q.getClass().getSimpleName()); + unexpected(t); + } finally { + Thread.interrupted(); // clear interrupts, just in case } } Index: src/test/jtreg/util/concurrent/ThreadPoolExecutor/ThrowingTasks.java =================================================================== RCS file: /export/home/jsr166/jsr166/jsr166/src/test/jtreg/util/concurrent/ThreadPoolExecutor/ThrowingTasks.java,v retrieving revision 1.4 diff -u -r1.4 ThrowingTasks.java --- src/test/jtreg/util/concurrent/ThreadPoolExecutor/ThrowingTasks.java 10 Jun 2013 17:36:39 -0000 1.4 +++ src/test/jtreg/util/concurrent/ThreadPoolExecutor/ThrowingTasks.java 11 Feb 2014 15:47:00 -0000 @@ -25,7 +25,7 @@ * @test * @bug 6450200 6450205 6450207 6450211 * @summary Test proper handling of tasks that terminate abruptly - * @run main/othervm -XX:-UseVMInterruptibleIO ThrowingTasks + * @run main/othervm ThrowingTasks * @author Martin Buchholz */ On Tue, Feb 11, 2014 at 3:13 AM, Chris Hegarty wrote: > On 11 Feb 2014, at 11:10, Alan Bateman wrote: > > > On 11/02/2014 10:36, Chris Hegarty wrote: > >> Looks good to me Alan. > >> > >> -Chris. > >> > > I checked the jsr166 CVS and both of the j.u.concurrent tests also > specify UseVMInterruptibleIO. I assume Martin will update these. > > Yes, I see this too. > > Martin, > Can you please update the 166 CVS as per Alan?s patch? > > -Chris. > > > > > -Alan > > From Alan.Bateman at oracle.com Tue Feb 11 16:00:44 2014 From: Alan.Bateman at oracle.com (Alan Bateman) Date: Tue, 11 Feb 2014 16:00:44 +0000 Subject: 8034175: Remove use of UseVMInterruptibleIO from tests In-Reply-To: References: <52F9FC06.2030101@oracle.com> <575D7215-AFEA-432D-9744-1C5F015B9F30@oracle.com> <52FA050E.5010501@oracle.com> <3D00C6E5-2742-483E-A121-D7C899DFF617@oracle.com> Message-ID: <52FA492C.30900@oracle.com> On 11/02/2014 15:52, Martin Buchholz wrote: > Thanks for doing this cleanup. > > ThrowingTasks fiddles with uncaught exception handlers and security > managers, so should remain otherVM. Interrupt does not, but could use > a tiny bit of cleanup hygiene: Probably best to create a separate issue for the cleanup. On tests setting a security manager then it's not strictly required tospin up a VM, jtreg does a better job at cleanup and restoring the environment. No problem keeping it for this test of course. -Alan From martinrb at google.com Tue Feb 11 16:09:00 2014 From: martinrb at google.com (Martin Buchholz) Date: Tue, 11 Feb 2014 08:09:00 -0800 Subject: 8034175: Remove use of UseVMInterruptibleIO from tests In-Reply-To: <52FA492C.30900@oracle.com> References: <52F9FC06.2030101@oracle.com> <575D7215-AFEA-432D-9744-1C5F015B9F30@oracle.com> <52FA050E.5010501@oracle.com> <3D00C6E5-2742-483E-A121-D7C899DFF617@oracle.com> <52FA492C.30900@oracle.com> Message-ID: It is possible to restore any pre-existing security manager setting, but I don't know what the current recommended practice for that is with jtreg. The uncaught exception handler is actually not a problem because it is just used with a thread factory (and could be eliminated completely). On Tue, Feb 11, 2014 at 8:00 AM, Alan Bateman wrote: > On 11/02/2014 15:52, Martin Buchholz wrote: > >> Thanks for doing this cleanup. >> >> ThrowingTasks fiddles with uncaught exception handlers and security >> managers, so should remain otherVM. Interrupt does not, but could use a >> tiny bit of cleanup hygiene: >> > Probably best to create a separate issue for the cleanup. On tests setting > a security manager then it's not strictly required tospin up a VM, jtreg > does a better job at cleanup and restoring the environment. No problem > keeping it for this test of course. > > -Alan > From martinrb at google.com Tue Feb 11 16:14:31 2014 From: martinrb at google.com (Martin Buchholz) Date: Tue, 11 Feb 2014 08:14:31 -0800 Subject: 8034175: Remove use of UseVMInterruptibleIO from tests In-Reply-To: <52FA492C.30900@oracle.com> References: <52F9FC06.2030101@oracle.com> <575D7215-AFEA-432D-9744-1C5F015B9F30@oracle.com> <52FA050E.5010501@oracle.com> <3D00C6E5-2742-483E-A121-D7C899DFF617@oracle.com> <52FA492C.30900@oracle.com> Message-ID: If you wanted to be super-conservative, you could simply remove the unwanted flag without changing the type of test, in this changeset. On Tue, Feb 11, 2014 at 8:00 AM, Alan Bateman wrote: > On 11/02/2014 15:52, Martin Buchholz wrote: > >> Thanks for doing this cleanup. >> >> ThrowingTasks fiddles with uncaught exception handlers and security >> managers, so should remain otherVM. Interrupt does not, but could use a >> tiny bit of cleanup hygiene: >> > Probably best to create a separate issue for the cleanup. On tests setting > a security manager then it's not strictly required tospin up a VM, jtreg > does a better job at cleanup and restoring the environment. No problem > keeping it for this test of course. > > -Alan > From eric.mccorkle at oracle.com Tue Feb 11 16:27:56 2014 From: eric.mccorkle at oracle.com (Eric McCorkle) Date: Tue, 11 Feb 2014 11:27:56 -0500 Subject: Java crypto libraries and large keys Message-ID: <52FA4F8C.8040903@oracle.com> I've been doing some upgrades on servers I run at home recently. One of the upgrades I'd planned was to increase the key size of my internal CA key and SSL keys to 8192 bits as a future-proofing measure (I use SSL with client certificates for all service-to-service communication). What I found was that apparently a number of server applications are not capable of handling keys of that size. I found a number of things stopped working, failing with error messages that suggest hard-coded limits ("excessive message size", etc). I have not gotten to any of the Java-based services I run yet, but I think it's worth looking in to whether the Java security and crypto libraries suffer from similar limitations. From Alan.Bateman at oracle.com Tue Feb 11 16:44:13 2014 From: Alan.Bateman at oracle.com (Alan Bateman) Date: Tue, 11 Feb 2014 16:44:13 +0000 Subject: RFR(S): 8034087: XML parser may overwrite element content if that content falls onto the border of an entity scanner buffer In-Reply-To: References: Message-ID: <52FA535D.70804@oracle.com> On 11/02/2014 14:57, Volker Simonis wrote: > Hi, > > after opening this bug yesterday for an issue found by my colleague > Steffen Schreiber we realized that this is actually a duplicate of > "8027359: XML parser returns incorrect parsing results" > (https://bugs.openjdk.java.net/browse/JDK-8027359). > > While there's no need now to submit a patch anymore, we'd > nevertheless like to contribute at least our test case for this issue: > > http://cr.openjdk.java.net/~simonis/webrevs/8034087/ > > The webrev is against jdk9-client but we'd like to also downport this > test to jdk7 and jdk8 to track that the fix for 8027359 will be > correctly downported to these releases as well. > > I will sponsor this change if somebody would be so kind to review it. > I'll leave it to Joe Wang to comment on the test but just to mention that jdk9/dev is probably a better forest to aim for because that is where the XML (and its tests) usually go. Also I wonder if it might be better to put it in the same directory as the test that Joe pushed with the change? If you are getting approval to push to jdk8u-dev and jdk7u-dev then it might be better to request a backport of Joe's change at the same time. -Alan. From lance.andersen at oracle.com Tue Feb 11 16:55:34 2014 From: lance.andersen at oracle.com (Lance @ Oracle) Date: Tue, 11 Feb 2014 11:55:34 -0500 Subject: RFR (JAXP): 8033980 : Xerces Update: datatype XMLGregorianCalendarImpl and DurationImpl In-Reply-To: <52F9666F.9010303@oracle.com> References: <52F9666F.9010303@oracle.com> Message-ID: <5111B487-22C1-4987-B9CC-ABF2F5CDD964@oracle.com> Hi joe It looks like you changed the serialversionuid in Durationimpl, did it get changed incorrectly previously? Lance Andersen| Principal Member of Technical Staff | +1.781.442.2037 Oracle Java Engineering 1 Network Drive Burlington, MA 01803 Lance.Andersen at oracle.com Sent from my iPad On Feb 10, 2014, at 6:53 PM, huizhe wang wrote: > Hi, > > This is an update from Xerces for two datatype classes XMLGregorianCalendarImpl and DurationImpl. For details, please refer to: > https://bugs.openjdk.java.net/browse/JDK-8033980 > > webrevs: http://cr.openjdk.java.net/~joehw/jdk9/8033980/webrev/ > > Tests included for: 1243, 1416, 1097 (Note that JDK did not have this bug) > > No test: > 1343: remove unused code > > Existing tests: JAXP SQE and Unit tests passed. > > Thanks, > Joe From volker.simonis at gmail.com Tue Feb 11 17:00:41 2014 From: volker.simonis at gmail.com (Volker Simonis) Date: Tue, 11 Feb 2014 18:00:41 +0100 Subject: RFR(S): 8034087: XML parser may overwrite element content if that content falls onto the border of an entity scanner buffer In-Reply-To: <52FA535D.70804@oracle.com> References: <52FA535D.70804@oracle.com> Message-ID: Hi Alan, you're right. I initially didn't saw the test because I just looked at the change in the jaxp repository. If it will be approved, I'll put the test in the same directory like the other test (i.e. test/javax/xml/jaxp/parsers/8027359). And yes, my plan was to get approval for both, the tests and the fix, when asking for the permission to downport to jdk8u-dev and jdk7u-dev. Thanks, Volker On Tue, Feb 11, 2014 at 5:44 PM, Alan Bateman wrote: > On 11/02/2014 14:57, Volker Simonis wrote: >> >> Hi, >> >> after opening this bug yesterday for an issue found by my colleague >> Steffen Schreiber we realized that this is actually a duplicate of >> "8027359: XML parser returns incorrect parsing results" >> (https://bugs.openjdk.java.net/browse/JDK-8027359). >> >> While there's no need now to submit a patch anymore, we'd >> nevertheless like to contribute at least our test case for this issue: >> >> http://cr.openjdk.java.net/~simonis/webrevs/8034087/ >> >> The webrev is against jdk9-client but we'd like to also downport this >> test to jdk7 and jdk8 to track that the fix for 8027359 will be >> correctly downported to these releases as well. >> >> I will sponsor this change if somebody would be so kind to review it. >> > I'll leave it to Joe Wang to comment on the test but just to mention that > jdk9/dev is probably a better forest to aim for because that is where the > XML (and its tests) usually go. Also I wonder if it might be better to put > it in the same directory as the test that Joe pushed with the change? > > If you are getting approval to push to jdk8u-dev and jdk7u-dev then it might > be better to request a backport of Joe's change at the same time. > > -Alan. From daniel.fuchs at oracle.com Tue Feb 11 17:05:25 2014 From: daniel.fuchs at oracle.com (Daniel Fuchs) Date: Tue, 11 Feb 2014 18:05:25 +0100 Subject: RFR (JAXP): 8033980 : Xerces Update: datatype XMLGregorianCalendarImpl and DurationImpl In-Reply-To: <5111B487-22C1-4987-B9CC-ABF2F5CDD964@oracle.com> References: <52F9666F.9010303@oracle.com> <5111B487-22C1-4987-B9CC-ABF2F5CDD964@oracle.com> Message-ID: <52FA5855.3020106@oracle.com> On 2/11/14 5:55 PM, Lance @ Oracle wrote: > Hi joe > > It looks like you changed the serialversionuid in Durationimpl, did it get changed incorrectly previously? I noticed that as well, but I'm not sure it matters since Duration uses writeReplace to serialize itself as an instance of SerializedDuration. Is that correct Joe? If I'm not mistaken it also means that the serialization forms (before & after the change) are not compatible - but is that an issue for such internal classes? best regards, -- daniel > > > Lance Andersen| Principal Member of Technical Staff | +1.781.442.2037 > Oracle Java Engineering > 1 Network Drive > Burlington, MA 01803 > Lance.Andersen at oracle.com > Sent from my iPad > > On Feb 10, 2014, at 6:53 PM, huizhe wang wrote: > >> Hi, >> >> This is an update from Xerces for two datatype classes XMLGregorianCalendarImpl and DurationImpl. For details, please refer to: >> https://bugs.openjdk.java.net/browse/JDK-8033980 >> >> webrevs: http://cr.openjdk.java.net/~joehw/jdk9/8033980/webrev/ >> >> Tests included for: 1243, 1416, 1097 (Note that JDK did not have this bug) >> >> No test: >> 1343: remove unused code >> >> Existing tests: JAXP SQE and Unit tests passed. >> >> Thanks, >> Joe From mark.reinhold at oracle.com Tue Feb 11 17:42:40 2014 From: mark.reinhold at oracle.com (mark.reinhold at oracle.com) Date: Tue, 11 Feb 2014 09:42:40 -0800 Subject: Changeset rolled back: jdk8/tl/jdk: 7152892: some jtreg tests fail with permission denied Message-ID: <20140211094240.954088@eggemoggin.niobe.net> This changeset was erroneously pushed to jdk8/tl/jdk: Changeset: da4b0962ad11 Author: robm Date: 2014-02-10 14:35 +0000 URL: http://hg.openjdk.java.net/jdk8/tl/jdk/rev/da4b0962ad11 7152892: some jtreg tests fail with permission denied Reviewed-by: coffeys ! test/java/lang/ClassLoader/Assert.sh ! test/java/rmi/registry/readTest/readTest.sh ! test/java/util/zip/ZipFile/ReadZip.java ! test/sun/net/www/protocol/jar/jarbug/run.sh It has been rolled back on the server and blacklisted in jcheck. If you have a clone of this repository then either strip this changeset or destroy the repository and re-clone it. - Mark From harvey at actenum.com Tue Feb 11 18:41:21 2014 From: harvey at actenum.com (Peter Harvey) Date: Tue, 11 Feb 2014 11:41:21 -0700 Subject: AtomicReference.compareAndSet allows multiple threads to succeed Message-ID: If multiple threads call AtomicReference.compareAndSet with the same pair of values, it appears that multiple threads may succeed. For example, if multiple threads call compareAndSet("Alpha", "Beta") then they may all succeed. Is this the correct behaviour? The documentation maybe should make this clear, as most developers and tutorials seem to assume that the compareAndSet methods of the Atomic classes guarantee that at most one thread will succeed. I've included full demonstration code at the very bottom of this email. In this demo, multiple threads are popping items from a stack and then pushing items back on to the stack. I discovered that it was possible for two threads to pop the same item from the stack, despite using compareAndSet to modify the head of the stack. Specifically, this code did not work as intended: AtomicReference stack = new AtomicReference(); ... // Pop an entry from the top of the queue StackEntry entry; while (true) { entry = stack.get(); if (entry == null) break; // Atomic, right? Only one thread can succeed... if (stack.compareAndSet(entry, entry.next)) break; } Peter. Full demo code below: -------------------------------------- public class AtomicReferenceTest { private static final class StackEntry { volatile Thread owner; volatile StackEntry next; } private static final AtomicReference stack = new AtomicReference(); private static class MyThread extends Thread { @Override public void run() { while (true) { /* * Pop an entry off the top of the stack. The only way to exit * this loop is to have found an empty stack, or to have * ATOMICALLY replaced the head of the stack with the next item * on the stack. In theory, no two threads should be able to * exit this loop with the SAME entry... */ StackEntry entry; while (true) { entry = stack.get(); if (entry == null) break; // Atomic, right? Only one thread can succeed... if (stack.compareAndSet(entry, entry.next)) break; } /* * If there was nothing on the stack, make a new entry. * Otherwise, just set the 'next' to null. This isn't required * but is good practice. */ if (entry == null) entry = new StackEntry(); else entry.next = null; /* * Check if the entry really was exclusively claimed. In theory, * no two threads should ever have a reference to the same * entry... */ Thread owner = entry.owner; if (owner != null) System.err.println("ALREADY CLAIMED BY " + owner); entry.owner = Thread.currentThread(); /* Push the entry back on to the queue. */ entry.owner = null; while (true) { entry.next = stack.get(); // Atomic, right? Only one thread can succeed... if (stack.compareAndSet(entry.next, entry)) break; } } } }; public static void main(String[] args) { Thread[] threads = new Thread[8]; for (int i = 0; i < threads.length; i++) threads[i] = new MyThread(); for (int i = 0; i < threads.length; i++) threads[i].start(); } } -- *Actenum Corporation* Peter Harvey | Cell: 780.729.8192 | harvey at actenum.com | www.actenum.com From huizhe.wang at oracle.com Tue Feb 11 18:56:48 2014 From: huizhe.wang at oracle.com (huizhe wang) Date: Tue, 11 Feb 2014 10:56:48 -0800 Subject: RFR (JAXP): 8033980 : Xerces Update: datatype XMLGregorianCalendarImpl and DurationImpl In-Reply-To: <52FA5855.3020106@oracle.com> References: <52F9666F.9010303@oracle.com> <5111B487-22C1-4987-B9CC-ABF2F5CDD964@oracle.com> <52FA5855.3020106@oracle.com> Message-ID: <52FA7270.4090709@oracle.com> On 2/11/2014 9:05 AM, Daniel Fuchs wrote: > On 2/11/14 5:55 PM, Lance @ Oracle wrote: >> Hi joe >> >> It looks like you changed the serialversionuid in Durationimpl, did >> it get changed incorrectly previously? > > I noticed that as well, but I'm not sure it matters since Duration > uses writeReplace to serialize itself as an instance of > SerializedDuration. Is that correct Joe? Yes, it's serialized as an instance of SerializedDuration, similar to the calendar where it's SerializedXMLGregorianCalendar. > > If I'm not mistaken it also means that the serialization forms > (before & after the change) are not compatible - but is that an > issue for such internal classes? Indeed, it's probably an issue since the binary is not compatible. It looks like this change should be rolled back. -Joe > > best regards, > > -- daniel > >> >> >> Lance Andersen| Principal Member of Technical Staff | +1.781.442.2037 >> Oracle Java Engineering >> 1 Network Drive >> Burlington, MA 01803 >> Lance.Andersen at oracle.com >> Sent from my iPad >> >> On Feb 10, 2014, at 6:53 PM, huizhe wang wrote: >> >>> Hi, >>> >>> This is an update from Xerces for two datatype classes >>> XMLGregorianCalendarImpl and DurationImpl. For details, please refer >>> to: >>> https://bugs.openjdk.java.net/browse/JDK-8033980 >>> >>> webrevs: http://cr.openjdk.java.net/~joehw/jdk9/8033980/webrev/ >>> >>> Tests included for: 1243, 1416, 1097 (Note that JDK did not have >>> this bug) >>> >>> No test: >>> 1343: remove unused code >>> >>> Existing tests: JAXP SQE and Unit tests passed. >>> >>> Thanks, >>> Joe > From philip.race at oracle.com Tue Feb 11 19:04:09 2014 From: philip.race at oracle.com (Phil Race) Date: Tue, 11 Feb 2014 11:04:09 -0800 Subject: RFR: (8031737) CHECK_NULL and CHECK_EXCEPTION macros cleanup In-Reply-To: <52F00EAC.8000900@oracle.com> References: <52ED36FD.5070507@oracle.com> <1243D3E7-C384-4AA8-80CA-9AA4E053F056@oracle.com> <52F00EAC.8000900@oracle.com> Message-ID: <52FA7429.5070806@oracle.com> Roger, Why not JDK 8u ? I've got a lot of changes that utilise these that will backport cleanly to JDK 8u only if 8u includes these macros. And since the changes are all over the place I don't fancy copy/pasting them everywhere. I suspect I am not the only one who would like these in 8u .. -phil. On 02/03/2014 01:48 PM, roger riggs wrote: > Hi Lance, > > The convenience macros are only intended for JDK 9. > > Roger > > On 2/1/2014 1:58 PM, Lance @ Oracle wrote: >> Looks fine >> >> Which releases are you think of including this in if any besides 9? >> >> >> Lance >> Andersen| Principal Member of Technical Staff | +1.781.442.2037 >> >> Oracle Java Engineering >> 1 Network Drive >> Burlington, MA 01803 >> Lance.Andersen at oracle.com >> Sent from my iPad >> >> On Feb 1, 2014, at 1:03 PM, roger riggs > > wrote: > From roger.riggs at oracle.com Tue Feb 11 19:14:18 2014 From: roger.riggs at oracle.com (roger riggs) Date: Tue, 11 Feb 2014 14:14:18 -0500 Subject: RFR: (8031737) CHECK_NULL and CHECK_EXCEPTION macros cleanup In-Reply-To: <52FA7429.5070806@oracle.com> References: <52ED36FD.5070507@oracle.com> <1243D3E7-C384-4AA8-80CA-9AA4E053F056@oracle.com> <52F00EAC.8000900@oracle.com> <52FA7429.5070806@oracle.com> Message-ID: <52FA768A.70005@oracle.com> Hi Phil, I see your point, there is nothing in the changes unique to 9. Do you want to take care of the back point? Roger On 2/11/2014 2:04 PM, Phil Race wrote: > Roger, > > Why not JDK 8u ? I've got a lot of changes that utilise these that will > backport cleanly to JDK 8u only if 8u includes these macros. And since > the changes are all over the place I don't fancy copy/pasting them > everywhere. I suspect I am not the only one who would like these in 8u .. > > -phil. > > > On 02/03/2014 01:48 PM, roger riggs wrote: >> Hi Lance, >> >> The convenience macros are only intended for JDK 9. >> >> Roger >> >> On 2/1/2014 1:58 PM, Lance @ Oracle wrote: >>> Looks fine >>> >>> Which releases are you think of including this in if any besides 9? >>> >>> >>> Lance >>> Andersen| Principal Member of Technical Staff | +1.781.442.2037 >>> >>> Oracle Java Engineering >>> 1 Network Drive >>> Burlington, MA 01803 >>> Lance.Andersen at oracle.com >>> Sent from my iPad >>> >>> On Feb 1, 2014, at 1:03 PM, roger riggs >> > wrote: >> > From dl at cs.oswego.edu Tue Feb 11 19:46:51 2014 From: dl at cs.oswego.edu (Doug Lea) Date: Tue, 11 Feb 2014 14:46:51 -0500 Subject: AtomicReference.compareAndSet allows multiple threads to succeed In-Reply-To: References: Message-ID: <52FA7E2B.7070802@cs.oswego.edu> On 02/11/2014 01:41 PM, Peter Harvey wrote: > If multiple threads call AtomicReference.compareAndSet with the same pair > of values, it appears that multiple threads may succeed. For example, if > multiple threads call compareAndSet("Alpha", "Beta") then they may all > succeed. Is this the correct behaviour? Your example has an ABA problem. See http://en.wikipedia.org/wiki/ABA_problem (that uses a variant of this example to illustrate). And/or try commenting out the // if (entry == null) entry = new StackEntry(); // else // entry.next = null; -Doug > > The documentation maybe should make this clear, as most developers and > tutorials seem to assume that the compareAndSet methods of the Atomic > classes guarantee that at most one thread will succeed. > > I've included full demonstration code at the very bottom of this email. In > this demo, multiple threads are popping items from a stack and then pushing > items back on to the stack. I discovered that it was possible for two > threads to pop the same item from the stack, despite using compareAndSet to > modify the head of the stack. Specifically, this code did not work as > intended: > > AtomicReference stack = new AtomicReference(); > > ... > > // Pop an entry from the top of the queue > StackEntry entry; > while (true) { > entry = stack.get(); > if (entry == null) > break; > > // Atomic, right? Only one thread can succeed... > if (stack.compareAndSet(entry, entry.next)) > break; > } > > Peter. > > Full demo code below: > -------------------------------------- > > public class AtomicReferenceTest { > private static final class StackEntry { > volatile Thread owner; > > volatile StackEntry next; > } > > private static final AtomicReference stack = new > AtomicReference(); > > private static class MyThread extends Thread { > @Override > public void run() { > while (true) { > > /* > * Pop an entry off the top of the stack. The only way to exit > * this loop is to have found an empty stack, or to have > * ATOMICALLY replaced the head of the stack with the next item > * on the stack. In theory, no two threads should be able to > * exit this loop with the SAME entry... > */ > StackEntry entry; > while (true) { > entry = stack.get(); > if (entry == null) > break; > > // Atomic, right? Only one thread can succeed... > if (stack.compareAndSet(entry, entry.next)) > break; > } > > /* > * If there was nothing on the stack, make a new entry. > * Otherwise, just set the 'next' to null. This isn't required > * but is good practice. > */ > if (entry == null) > entry = new StackEntry(); > else > entry.next = null; > > /* > * Check if the entry really was exclusively claimed. In theory, > * no two threads should ever have a reference to the same > * entry... > */ > Thread owner = entry.owner; > if (owner != null) > System.err.println("ALREADY CLAIMED BY " + owner); > entry.owner = Thread.currentThread(); > > /* Push the entry back on to the queue. */ > entry.owner = null; > while (true) { > entry.next = stack.get(); > > // Atomic, right? Only one thread can succeed... > if (stack.compareAndSet(entry.next, entry)) > break; > } > } > } > }; > > public static void main(String[] args) { > Thread[] threads = new Thread[8]; > for (int i = 0; i < threads.length; i++) > threads[i] = new MyThread(); > for (int i = 0; i < threads.length; i++) > threads[i].start(); > } > } > From philip.race at oracle.com Tue Feb 11 19:57:26 2014 From: philip.race at oracle.com (Phil Race) Date: Tue, 11 Feb 2014 11:57:26 -0800 Subject: RFR: (8031737) CHECK_NULL and CHECK_EXCEPTION macros cleanup In-Reply-To: <52FA768A.70005@oracle.com> References: <52ED36FD.5070507@oracle.com> <1243D3E7-C384-4AA8-80CA-9AA4E053F056@oracle.com> <52F00EAC.8000900@oracle.com> <52FA7429.5070806@oracle.com> <52FA768A.70005@oracle.com> Message-ID: <52FA80A6.2040509@oracle.com> Roger, Yes, I can do that. I see here http://cr.openjdk.java.net/~rriggs/webrev-check-cleanup-8031737/ that 1) There was a previous version of these macros. Looks like no need to worry about that I just need the latest version. 2) There was also a change to Version.c. I can include that if you think it appropriate .. or omit it if you think its not essential. -phil. On 2/11/2014 11:14 AM, roger riggs wrote: > Hi Phil, > > I see your point, there is nothing in the changes unique to 9. > Do you want to take care of the back point? > > Roger > > On 2/11/2014 2:04 PM, Phil Race wrote: >> Roger, >> >> Why not JDK 8u ? I've got a lot of changes that utilise these that will >> backport cleanly to JDK 8u only if 8u includes these macros. And since >> the changes are all over the place I don't fancy copy/pasting them >> everywhere. I suspect I am not the only one who would like these in >> 8u .. >> >> -phil. >> >> >> On 02/03/2014 01:48 PM, roger riggs wrote: >>> Hi Lance, >>> >>> The convenience macros are only intended for JDK 9. >>> >>> Roger >>> >>> On 2/1/2014 1:58 PM, Lance @ Oracle wrote: >>>> Looks fine >>>> >>>> Which releases are you think of including this in if any besides 9? >>>> >>>> >>>> Lance >>>> Andersen| Principal Member of Technical Staff | +1.781.442.2037 >>>> >>>> Oracle Java Engineering >>>> 1 Network Drive >>>> Burlington, MA 01803 >>>> Lance.Andersen at oracle.com >>>> Sent from my iPad >>>> >>>> On Feb 1, 2014, at 1:03 PM, roger riggs >>> > wrote: >>> >> > From Alan.Bateman at oracle.com Tue Feb 11 20:04:07 2014 From: Alan.Bateman at oracle.com (Alan Bateman) Date: Tue, 11 Feb 2014 20:04:07 +0000 Subject: RFR: (8031737) CHECK_NULL and CHECK_EXCEPTION macros cleanup In-Reply-To: <52FA80A6.2040509@oracle.com> References: <52ED36FD.5070507@oracle.com> <1243D3E7-C384-4AA8-80CA-9AA4E053F056@oracle.com> <52F00EAC.8000900@oracle.com> <52FA7429.5070806@oracle.com> <52FA768A.70005@oracle.com> <52FA80A6.2040509@oracle.com> Message-ID: <52FA8237.5040201@oracle.com> On 11/02/2014 19:57, Phil Race wrote: > Roger, > > Yes, I can do that. > > I see here > http://cr.openjdk.java.net/~rriggs/webrev-check-cleanup-8031737/ that > 1) There was a previous version of these macros. > Looks like no need to worry about that I just need the latest version. > 2) There was also a change to Version.c. I can include that if you > think it > appropriate .. or omit it if you think its not essential. > > -phil. If you "hg export" the change sets for 8030875 and 8031737 then do they "hg import" cleanly into 8u? That might be preferable to taking patches from webrevs. -Alan. From harvey at actenum.com Tue Feb 11 20:23:17 2014 From: harvey at actenum.com (Peter Harvey) Date: Tue, 11 Feb 2014 13:23:17 -0700 Subject: AtomicReference.compareAndSet allows multiple threads to succeed In-Reply-To: References: Message-ID: Thanks all for the responses. It is an ABA problem. I changed my code so that it creates new StackEntries if multiple threads attempt to pop the stack at the same time. This relies on getAndSet to obtain exclusive access to the stack contents, rather than using compareAndSet to update the stack. I'll look into educating myself more on lock-free programming when I can. For now, I'll be a lot more cautious when using compareAndSet. Thanks again. On Tue, Feb 11, 2014 at 11:41 AM, Peter Harvey wrote: > If multiple threads call AtomicReference.compareAndSet with the same pair > of values, it appears that multiple threads may succeed. For example, if > multiple threads call compareAndSet("Alpha", "Beta") then they may all > succeed. Is this the correct behaviour? > > The documentation maybe should make this clear, as most developers and > tutorials seem to assume that the compareAndSet methods of the Atomic > classes guarantee that at most one thread will succeed. > > I've included full demonstration code at the very bottom of this email. In > this demo, multiple threads are popping items from a stack and then pushing > items back on to the stack. I discovered that it was possible for two > threads to pop the same item from the stack, despite using compareAndSet to > modify the head of the stack. Specifically, this code did not work as > intended: > > AtomicReference stack = new AtomicReference(); > > ... > > // Pop an entry from the top of the queue > StackEntry entry; > while (true) { > entry = stack.get(); > if (entry == null) > break; > > // Atomic, right? Only one thread can succeed... > if (stack.compareAndSet(entry, entry.next)) > break; > } > > Peter. > > Full demo code below: > -------------------------------------- > > public class AtomicReferenceTest { > private static final class StackEntry { > volatile Thread owner; > > volatile StackEntry next; > } > > private static final AtomicReference stack = new > AtomicReference(); > > private static class MyThread extends Thread { > @Override > public void run() { > while (true) { > > /* > * Pop an entry off the top of the stack. The only way to exit > * this loop is to have found an empty stack, or to have > * ATOMICALLY replaced the head of the stack with the next item > * on the stack. In theory, no two threads should be able to > * exit this loop with the SAME entry... > */ > StackEntry entry; > while (true) { > entry = stack.get(); > if (entry == null) > break; > > // Atomic, right? Only one thread can succeed... > if (stack.compareAndSet(entry, entry.next)) > break; > } > > /* > * If there was nothing on the stack, make a new entry. > * Otherwise, just set the 'next' to null. This isn't required > * but is good practice. > */ > if (entry == null) > entry = new StackEntry(); > else > entry.next = null; > > /* > * Check if the entry really was exclusively claimed. In theory, > * no two threads should ever have a reference to the same > * entry... > */ > Thread owner = entry.owner; > if (owner != null) > System.err.println("ALREADY CLAIMED BY " + owner); > entry.owner = Thread.currentThread(); > > /* Push the entry back on to the queue. */ > entry.owner = null; > while (true) { > entry.next = stack.get(); > > // Atomic, right? Only one thread can succeed... > if (stack.compareAndSet(entry.next, entry)) > break; > } > } > } > }; > > public static void main(String[] args) { > Thread[] threads = new Thread[8]; > for (int i = 0; i < threads.length; i++) > threads[i] = new MyThread(); > for (int i = 0; i < threads.length; i++) > threads[i].start(); > } > } > > -- > *Actenum Corporation* > Peter Harvey | Cell: 780.729.8192 | harvey at actenum.com | > www.actenum.com > -- *Actenum Corporation* Peter Harvey | Cell: 780.729.8192 | harvey at actenum.com | www.actenum.com From roger.riggs at oracle.com Tue Feb 11 20:28:18 2014 From: roger.riggs at oracle.com (roger riggs) Date: Tue, 11 Feb 2014 15:28:18 -0500 Subject: RFR: (8031737) CHECK_NULL and CHECK_EXCEPTION macros cleanup In-Reply-To: <52FA80A6.2040509@oracle.com> References: <52ED36FD.5070507@oracle.com> <1243D3E7-C384-4AA8-80CA-9AA4E053F056@oracle.com> <52F00EAC.8000900@oracle.com> <52FA7429.5070806@oracle.com> <52FA768A.70005@oracle.com> <52FA80A6.2040509@oracle.com> Message-ID: <52FA87E2.9080009@oracle.com> Hi Phil, Yes, it ended up in two change sets in jdk 9, you should take both to be up to date. changeset: 9245:a09982d91fab user: rriggs date: Wed Feb 05 10:59:53 2014 -0500 files: src/share/native/common/jni_util.c description: 8030993: Check jdk/src/share/native/common/jni_util.c for JNI pending exceptions changeset: 9229:fb89dc4fe8da date: Mon Feb 03 16:58:02 2014 -0500 files: src/share/native/common/jni_util.h src/share/native/sun/misc/Version.c interrupted! description: 8031737: CHECK_NULL and CHECK_EXCEPTION macros cleanup Thanks, Roger On 2/11/2014 2:57 PM, Phil Race wrote: > Roger, > > Yes, I can do that. > > I see here > http://cr.openjdk.java.net/~rriggs/webrev-check-cleanup-8031737/ that > 1) There was a previous version of these macros. > Looks like no need to worry about that I just need the latest version. > 2) There was also a change to Version.c. I can include that if you > think it > appropriate .. or omit it if you think its not essential. > > -phil. > > On 2/11/2014 11:14 AM, roger riggs wrote: >> Hi Phil, >> >> I see your point, there is nothing in the changes unique to 9. >> Do you want to take care of the back point? >> >> Roger >> >> On 2/11/2014 2:04 PM, Phil Race wrote: >>> Roger, >>> >>> Why not JDK 8u ? I've got a lot of changes that utilise these that >>> will >>> backport cleanly to JDK 8u only if 8u includes these macros. And since >>> the changes are all over the place I don't fancy copy/pasting them >>> everywhere. I suspect I am not the only one who would like these in >>> 8u .. >>> >>> -phil. >>> >>> >>> On 02/03/2014 01:48 PM, roger riggs wrote: >>>> Hi Lance, >>>> >>>> The convenience macros are only intended for JDK 9. >>>> >>>> Roger >>>> >>>> On 2/1/2014 1:58 PM, Lance @ Oracle wrote: >>>>> Looks fine >>>>> >>>>> Which releases are you think of including this in if any besides 9? >>>>> >>>>> >>>>> Lance >>>>> Andersen| Principal Member of Technical Staff | +1.781.442.2037 >>>>> >>>>> Oracle Java Engineering >>>>> 1 Network Drive >>>>> Burlington, MA 01803 >>>>> Lance.Andersen at oracle.com >>>>> Sent from my iPad >>>>> >>>>> On Feb 1, 2014, at 1:03 PM, roger riggs >>>> > wrote: >>>> >>> >> > From philip.race at oracle.com Tue Feb 11 21:37:02 2014 From: philip.race at oracle.com (Phil Race) Date: Tue, 11 Feb 2014 13:37:02 -0800 Subject: RFR: (8031737) CHECK_NULL and CHECK_EXCEPTION macros cleanup In-Reply-To: <52FA8237.5040201@oracle.com> References: <52ED36FD.5070507@oracle.com> <1243D3E7-C384-4AA8-80CA-9AA4E053F056@oracle.com> <52F00EAC.8000900@oracle.com> <52FA7429.5070806@oracle.com> <52FA768A.70005@oracle.com> <52FA80A6.2040509@oracle.com> <52FA8237.5040201@oracle.com> Message-ID: <52FA97FE.9020904@oracle.com> 8030875 applied but 8031737 will not apply for me, presumably because Version.c had a number of changes under bug 8029007. And the version from 8031737 is the one that I want. -phil. On 2/11/14 12:04 PM, Alan Bateman wrote: > On 11/02/2014 19:57, Phil Race wrote: >> Roger, >> >> Yes, I can do that. >> >> I see here >> http://cr.openjdk.java.net/~rriggs/webrev-check-cleanup-8031737/ that >> 1) There was a previous version of these macros. >> Looks like no need to worry about that I just need the latest version. >> 2) There was also a change to Version.c. I can include that if you >> think it >> appropriate .. or omit it if you think its not essential. >> >> -phil. > If you "hg export" the change sets for 8030875 and 8031737 then do > they "hg import" cleanly into 8u? That might be preferable to taking > patches from webrevs. > > -Alan. From philip.race at oracle.com Tue Feb 11 21:39:52 2014 From: philip.race at oracle.com (Phil Race) Date: Tue, 11 Feb 2014 13:39:52 -0800 Subject: RFR: (8031737) CHECK_NULL and CHECK_EXCEPTION macros cleanup In-Reply-To: <52FA87E2.9080009@oracle.com> References: <52ED36FD.5070507@oracle.com> <1243D3E7-C384-4AA8-80CA-9AA4E053F056@oracle.com> <52F00EAC.8000900@oracle.com> <52FA7429.5070806@oracle.com> <52FA768A.70005@oracle.com> <52FA80A6.2040509@oracle.com> <52FA87E2.9080009@oracle.com> Message-ID: <52FA98A8.7090104@oracle.com> Roger, That later one seems to be using the macros. I don't see any update to the macros. So I'm not sure why I'm need it .. since I'm not using those calls and neither are the macros. -phil. On 2/11/14 12:28 PM, roger riggs wrote: > Hi Phil, > > Yes, it ended up in two change sets in jdk 9, you should take both to > be up to date. > > changeset: 9245:a09982d91fab > user: rriggs > date: Wed Feb 05 10:59:53 2014 -0500 > files: src/share/native/common/jni_util.c > description: > 8030993: Check jdk/src/share/native/common/jni_util.c for JNI pending > exceptions > > > changeset: 9229:fb89dc4fe8da > date: Mon Feb 03 16:58:02 2014 -0500 > files: src/share/native/common/jni_util.h > src/share/native/sun/misc/Version.c > interrupted! > description: > 8031737: CHECK_NULL and CHECK_EXCEPTION macros cleanup > > Thanks, Roger > > > On 2/11/2014 2:57 PM, Phil Race wrote: >> Roger, >> >> Yes, I can do that. >> >> I see here >> http://cr.openjdk.java.net/~rriggs/webrev-check-cleanup-8031737/ that >> 1) There was a previous version of these macros. >> Looks like no need to worry about that I just need the latest version. >> 2) There was also a change to Version.c. I can include that if you >> think it >> appropriate .. or omit it if you think its not essential. >> >> -phil. >> >> On 2/11/2014 11:14 AM, roger riggs wrote: >>> Hi Phil, >>> >>> I see your point, there is nothing in the changes unique to 9. >>> Do you want to take care of the back point? >>> >>> Roger >>> >>> On 2/11/2014 2:04 PM, Phil Race wrote: >>>> Roger, >>>> >>>> Why not JDK 8u ? I've got a lot of changes that utilise these that >>>> will >>>> backport cleanly to JDK 8u only if 8u includes these macros. And since >>>> the changes are all over the place I don't fancy copy/pasting them >>>> everywhere. I suspect I am not the only one who would like these in >>>> 8u .. >>>> >>>> -phil. >>>> >>>> >>>> On 02/03/2014 01:48 PM, roger riggs wrote: >>>>> Hi Lance, >>>>> >>>>> The convenience macros are only intended for JDK 9. >>>>> >>>>> Roger >>>>> >>>>> On 2/1/2014 1:58 PM, Lance @ Oracle wrote: >>>>>> Looks fine >>>>>> >>>>>> Which releases are you think of including this in if any besides 9? >>>>>> >>>>>> >>>>>> Lance >>>>>> Andersen| Principal Member of Technical Staff | +1.781.442.2037 >>>>>> >>>>>> Oracle Java Engineering >>>>>> 1 Network Drive >>>>>> Burlington, MA 01803 >>>>>> Lance.Andersen at oracle.com >>>>>> Sent from my iPad >>>>>> >>>>>> On Feb 1, 2014, at 1:03 PM, roger riggs >>>>> > wrote: >>>>> >>>> >>> >> > From lana.steuck at oracle.com Tue Feb 11 21:40:10 2014 From: lana.steuck at oracle.com (lana.steuck at oracle.com) Date: Tue, 11 Feb 2014 21:40:10 +0000 Subject: hg: jdk8/tl/corba: 7 new changesets Message-ID: <20140211214024.5EA1E62B9A@hg.openjdk.java.net> Changeset: 18c4d03cf516 Author: katleman Date: 2014-01-22 12:53 -0800 URL: http://hg.openjdk.java.net/jdk8/tl/corba/rev/18c4d03cf516 Added tag jdk8-b125 for changeset 7b45151c7a05 ! .hgtags Changeset: 8ceb68fd9e10 Author: katleman Date: 2014-01-22 14:06 -0800 URL: http://hg.openjdk.java.net/jdk8/tl/corba/rev/8ceb68fd9e10 Merge ! .hgtags Changeset: cfa04e69b115 Author: katleman Date: 2014-01-24 15:07 -0800 URL: http://hg.openjdk.java.net/jdk8/tl/corba/rev/cfa04e69b115 Added tag jdk8-b126 for changeset 8ceb68fd9e10 ! .hgtags Changeset: b8c71dae0557 Author: lana Date: 2014-01-29 11:11 -0800 URL: http://hg.openjdk.java.net/jdk8/tl/corba/rev/b8c71dae0557 Merge Changeset: 113e7569b49b Author: katleman Date: 2014-01-30 12:16 -0800 URL: http://hg.openjdk.java.net/jdk8/tl/corba/rev/113e7569b49b Added tag jdk8-b127 for changeset b8c71dae0557 ! .hgtags Changeset: 5c72d74c6805 Author: katleman Date: 2014-02-01 18:21 -0800 URL: http://hg.openjdk.java.net/jdk8/tl/corba/rev/5c72d74c6805 Added tag jdk8-b128 for changeset 113e7569b49b ! .hgtags Changeset: eea0d7dfcbe2 Author: katleman Date: 2014-02-06 17:34 -0800 URL: http://hg.openjdk.java.net/jdk8/tl/corba/rev/eea0d7dfcbe2 Added tag jdk8-b129 for changeset 5c72d74c6805 ! .hgtags From lana.steuck at oracle.com Tue Feb 11 21:40:17 2014 From: lana.steuck at oracle.com (lana.steuck at oracle.com) Date: Tue, 11 Feb 2014 21:40:17 +0000 Subject: hg: jdk8/tl/langtools: 7 new changesets Message-ID: <20140211214140.425E862B9E@hg.openjdk.java.net> Changeset: 9a4dbfe11ed1 Author: katleman Date: 2014-01-22 12:54 -0800 URL: http://hg.openjdk.java.net/jdk8/tl/langtools/rev/9a4dbfe11ed1 Added tag jdk8-b125 for changeset 436176151e85 ! .hgtags Changeset: ba24b6304362 Author: katleman Date: 2014-01-22 14:09 -0800 URL: http://hg.openjdk.java.net/jdk8/tl/langtools/rev/ba24b6304362 Merge ! .hgtags Changeset: 305b97f4651b Author: katleman Date: 2014-01-24 15:08 -0800 URL: http://hg.openjdk.java.net/jdk8/tl/langtools/rev/305b97f4651b Added tag jdk8-b126 for changeset ba24b6304362 ! .hgtags Changeset: bb69217ed812 Author: lana Date: 2014-01-29 11:12 -0800 URL: http://hg.openjdk.java.net/jdk8/tl/langtools/rev/bb69217ed812 Merge Changeset: 09cdd3b493c0 Author: katleman Date: 2014-01-30 12:17 -0800 URL: http://hg.openjdk.java.net/jdk8/tl/langtools/rev/09cdd3b493c0 Added tag jdk8-b127 for changeset bb69217ed812 ! .hgtags Changeset: 8fe7202d3c38 Author: katleman Date: 2014-02-01 18:21 -0800 URL: http://hg.openjdk.java.net/jdk8/tl/langtools/rev/8fe7202d3c38 Added tag jdk8-b128 for changeset 09cdd3b493c0 ! .hgtags Changeset: 9d81ae1c417a Author: katleman Date: 2014-02-06 17:35 -0800 URL: http://hg.openjdk.java.net/jdk8/tl/langtools/rev/9d81ae1c417a Added tag jdk8-b129 for changeset 8fe7202d3c38 ! .hgtags From lana.steuck at oracle.com Tue Feb 11 21:40:13 2014 From: lana.steuck at oracle.com (lana.steuck at oracle.com) Date: Tue, 11 Feb 2014 21:40:13 +0000 Subject: hg: jdk8/tl/jaxws: 7 new changesets Message-ID: <20140211214119.DF23C62B9C@hg.openjdk.java.net> Changeset: c0040f0b75e2 Author: katleman Date: 2014-01-22 12:53 -0800 URL: http://hg.openjdk.java.net/jdk8/tl/jaxws/rev/c0040f0b75e2 Added tag jdk8-b125 for changeset ef71ecbcd7bc ! .hgtags Changeset: 7193a007a159 Author: katleman Date: 2014-01-22 14:07 -0800 URL: http://hg.openjdk.java.net/jdk8/tl/jaxws/rev/7193a007a159 Merge ! .hgtags Changeset: 3f682f2ea376 Author: katleman Date: 2014-01-24 15:08 -0800 URL: http://hg.openjdk.java.net/jdk8/tl/jaxws/rev/3f682f2ea376 Added tag jdk8-b126 for changeset 7193a007a159 ! .hgtags Changeset: 8e46fe36e175 Author: lana Date: 2014-01-29 11:11 -0800 URL: http://hg.openjdk.java.net/jdk8/tl/jaxws/rev/8e46fe36e175 Merge Changeset: de172acc095b Author: katleman Date: 2014-01-30 12:16 -0800 URL: http://hg.openjdk.java.net/jdk8/tl/jaxws/rev/de172acc095b Added tag jdk8-b127 for changeset 8e46fe36e175 ! .hgtags Changeset: aabc90596123 Author: katleman Date: 2014-02-01 18:21 -0800 URL: http://hg.openjdk.java.net/jdk8/tl/jaxws/rev/aabc90596123 Added tag jdk8-b128 for changeset de172acc095b ! .hgtags Changeset: 4195c0956930 Author: katleman Date: 2014-02-06 17:35 -0800 URL: http://hg.openjdk.java.net/jdk8/tl/jaxws/rev/4195c0956930 Added tag jdk8-b129 for changeset aabc90596123 ! .hgtags From lana.steuck at oracle.com Tue Feb 11 21:40:17 2014 From: lana.steuck at oracle.com (lana.steuck at oracle.com) Date: Tue, 11 Feb 2014 21:40:17 +0000 Subject: hg: jdk8/tl/jaxp: 7 new changesets Message-ID: <20140211214123.063DA62B9D@hg.openjdk.java.net> Changeset: 6a5af8a36aaf Author: katleman Date: 2014-01-22 12:53 -0800 URL: http://hg.openjdk.java.net/jdk8/tl/jaxp/rev/6a5af8a36aaf Added tag jdk8-b125 for changeset 83bb924238f8 ! .hgtags Changeset: 390cc275c04c Author: katleman Date: 2014-01-22 14:07 -0800 URL: http://hg.openjdk.java.net/jdk8/tl/jaxp/rev/390cc275c04c Merge ! .hgtags Changeset: 573c261a2025 Author: katleman Date: 2014-01-24 15:08 -0800 URL: http://hg.openjdk.java.net/jdk8/tl/jaxp/rev/573c261a2025 Added tag jdk8-b126 for changeset 390cc275c04c ! .hgtags Changeset: b68cdb63a70b Author: lana Date: 2014-01-29 11:11 -0800 URL: http://hg.openjdk.java.net/jdk8/tl/jaxp/rev/b68cdb63a70b Merge Changeset: b1839922f10c Author: katleman Date: 2014-01-30 12:16 -0800 URL: http://hg.openjdk.java.net/jdk8/tl/jaxp/rev/b1839922f10c Added tag jdk8-b127 for changeset b68cdb63a70b ! .hgtags Changeset: b7752cea7c81 Author: katleman Date: 2014-02-01 18:21 -0800 URL: http://hg.openjdk.java.net/jdk8/tl/jaxp/rev/b7752cea7c81 Added tag jdk8-b128 for changeset b1839922f10c ! .hgtags Changeset: 0cb0cd015218 Author: katleman Date: 2014-02-06 17:34 -0800 URL: http://hg.openjdk.java.net/jdk8/tl/jaxp/rev/0cb0cd015218 Added tag jdk8-b129 for changeset b7752cea7c81 ! .hgtags From lana.steuck at oracle.com Tue Feb 11 21:40:24 2014 From: lana.steuck at oracle.com (lana.steuck at oracle.com) Date: Tue, 11 Feb 2014 21:40:24 +0000 Subject: hg: jdk8/tl/nashorn: 8 new changesets Message-ID: <20140211214101.632E862B9B@hg.openjdk.java.net> Changeset: d336209a0e45 Author: katleman Date: 2014-01-22 12:54 -0800 URL: http://hg.openjdk.java.net/jdk8/tl/nashorn/rev/d336209a0e45 Added tag jdk8-b125 for changeset 7346abe2ea03 ! .hgtags Changeset: 095263db862d Author: katleman Date: 2014-01-22 14:00 -0800 URL: http://hg.openjdk.java.net/jdk8/tl/nashorn/rev/095263db862d Merge ! .hgtags Changeset: e2522604c7c9 Author: katleman Date: 2014-01-24 15:08 -0800 URL: http://hg.openjdk.java.net/jdk8/tl/nashorn/rev/e2522604c7c9 Added tag jdk8-b126 for changeset 095263db862d ! .hgtags Changeset: fdfbb745caf0 Author: lana Date: 2014-01-29 11:12 -0800 URL: http://hg.openjdk.java.net/jdk8/tl/nashorn/rev/fdfbb745caf0 Merge Changeset: 7dfde83426d1 Author: katleman Date: 2014-01-30 12:17 -0800 URL: http://hg.openjdk.java.net/jdk8/tl/nashorn/rev/7dfde83426d1 Added tag jdk8-b127 for changeset fdfbb745caf0 ! .hgtags Changeset: 73cbad0c5d28 Author: lana Date: 2014-01-31 13:47 -0800 URL: http://hg.openjdk.java.net/jdk8/tl/nashorn/rev/73cbad0c5d28 Merge Changeset: 9cc3fd32fbab Author: katleman Date: 2014-02-01 18:21 -0800 URL: http://hg.openjdk.java.net/jdk8/tl/nashorn/rev/9cc3fd32fbab Added tag jdk8-b128 for changeset 73cbad0c5d28 ! .hgtags Changeset: f87eba70e9ee Author: katleman Date: 2014-02-06 17:35 -0800 URL: http://hg.openjdk.java.net/jdk8/tl/nashorn/rev/f87eba70e9ee Added tag jdk8-b129 for changeset 9cc3fd32fbab ! .hgtags From lana.steuck at oracle.com Tue Feb 11 21:40:17 2014 From: lana.steuck at oracle.com (lana.steuck at oracle.com) Date: Tue, 11 Feb 2014 21:40:17 +0000 Subject: hg: jdk8/tl/hotspot: 23 new changesets Message-ID: <20140211214227.12C4F62B9F@hg.openjdk.java.net> Changeset: 16e0c6c84a91 Author: amurillo Date: 2014-01-13 16:00 -0800 URL: http://hg.openjdk.java.net/jdk8/tl/hotspot/rev/16e0c6c84a91 8031553: new hotspot build - hs25-b67 Reviewed-by: jcoomes ! make/hotspot_version Changeset: 12ad8db39f76 Author: roland Date: 2014-01-14 09:44 +0100 URL: http://hg.openjdk.java.net/jdk8/tl/hotspot/rev/12ad8db39f76 8028764: dtrace/hotspot_jni/ALL/ALL001 crashes the vm on Solaris-amd64, SIGSEGV in MarkSweep::follow_stack()+0x8a Summary: C1 generates code to encode compressed oop into tmp register before runtime call for patching where GC may happen Reviewed-by: iveresov, twisti, kvn Contributed-by: mgerdin ! src/cpu/x86/vm/c1_LIRAssembler_x86.cpp Changeset: 8b81451dc7f7 Author: twisti Date: 2014-01-16 16:18 -0800 URL: http://hg.openjdk.java.net/jdk8/tl/hotspot/rev/8b81451dc7f7 8022395: java.util.zip.ZipException: Not in GZIP format in JT_JDK/test/java/util/zip/GZIP tests Reviewed-by: kvn, iveresov ! src/cpu/x86/vm/c1_LIRGenerator_x86.cpp Changeset: 3585183c191a Author: amurillo Date: 2014-01-17 20:24 -0800 URL: http://hg.openjdk.java.net/jdk8/tl/hotspot/rev/3585183c191a Merge Changeset: 5df2666e4573 Author: amurillo Date: 2014-01-17 20:24 -0800 URL: http://hg.openjdk.java.net/jdk8/tl/hotspot/rev/5df2666e4573 Added tag hs25-b67 for changeset 3585183c191a ! .hgtags Changeset: 55ff9170e27d Author: katleman Date: 2014-01-22 12:53 -0800 URL: http://hg.openjdk.java.net/jdk8/tl/hotspot/rev/55ff9170e27d Added tag jdk8-b125 for changeset df333ee12bba ! .hgtags Changeset: c8218f1072a0 Author: katleman Date: 2014-01-22 14:07 -0800 URL: http://hg.openjdk.java.net/jdk8/tl/hotspot/rev/c8218f1072a0 Merge ! .hgtags Changeset: 9a11d5e679cf Author: katleman Date: 2014-01-24 15:07 -0800 URL: http://hg.openjdk.java.net/jdk8/tl/hotspot/rev/9a11d5e679cf Added tag jdk8-b126 for changeset c8218f1072a0 ! .hgtags Changeset: c2106608358b Author: amurillo Date: 2014-01-17 20:30 -0800 URL: http://hg.openjdk.java.net/jdk8/tl/hotspot/rev/c2106608358b 8032015: new hotspot build - hs25-b68 Reviewed-by: jcoomes ! make/hotspot_version Changeset: 709018897c81 Author: vlivanov Date: 2014-01-23 01:23 +0400 URL: http://hg.openjdk.java.net/jdk8/tl/hotspot/rev/709018897c81 8031695: CHA ignores default methods during analysis leading to incorrect code generation Reviewed-by: jrose, acorn, hseigel, lfoltan ! src/share/vm/code/dependencies.cpp + test/compiler/inlining/DefaultAndConcreteMethodsCHA.java Changeset: f970454708b8 Author: iveresov Date: 2014-01-17 18:09 -0800 URL: http://hg.openjdk.java.net/jdk8/tl/hotspot/rev/f970454708b8 8032207: C2: assert(VerifyOops || MachNode::size(ra_) <= (3+1)*4) failed: bad fixed size Summary: Fix the sizing of loadUS2L_immI16 and loadI2L_immI Reviewed-by: kvn, azeemj ! src/cpu/sparc/vm/sparc.ad + test/compiler/codegen/LoadWithMask.java Changeset: 984401824c5e Author: iveresov Date: 2014-01-21 20:05 -0800 URL: http://hg.openjdk.java.net/jdk8/tl/hotspot/rev/984401824c5e 8031743: C2: loadI2L_immI broken for negative memory values Summary: Restrict loadI2L_imm optimizations to positive values of mask Reviewed-by: kvn, dlong ! src/cpu/sparc/vm/sparc.ad ! src/cpu/x86/vm/x86_32.ad ! src/cpu/x86/vm/x86_64.ad + test/compiler/codegen/LoadWithMask2.java Changeset: d45454002494 Author: amurillo Date: 2014-01-23 13:37 -0800 URL: http://hg.openjdk.java.net/jdk8/tl/hotspot/rev/d45454002494 Merge Changeset: 2c564e329c87 Author: amurillo Date: 2014-01-23 13:37 -0800 URL: http://hg.openjdk.java.net/jdk8/tl/hotspot/rev/2c564e329c87 Added tag hs25-b68 for changeset d45454002494 ! .hgtags Changeset: 58879cd9f8df Author: amurillo Date: 2014-01-28 09:51 -0800 URL: http://hg.openjdk.java.net/jdk8/tl/hotspot/rev/58879cd9f8df Merge ! .hgtags Changeset: 7e412f95e310 Author: amurillo Date: 2014-01-23 13:53 -0800 URL: http://hg.openjdk.java.net/jdk8/tl/hotspot/rev/7e412f95e310 8032608: new hotspot build - hs25-b69 Reviewed-by: jcoomes ! make/hotspot_version Changeset: 2185d483f5f8 Author: kvn Date: 2014-01-27 10:20 -0800 URL: http://hg.openjdk.java.net/jdk8/tl/hotspot/rev/2185d483f5f8 8032566: Crash in JIT when running Scala compiler (and compiling Scala std lib) Summary: Switch off EliminateAutoBox flag by default in jdk8 release. Reviewed-by: iveresov ! src/share/vm/opto/c2_globals.hpp Changeset: 32f017489ba5 Author: amurillo Date: 2014-01-28 15:00 -0800 URL: http://hg.openjdk.java.net/jdk8/tl/hotspot/rev/32f017489ba5 Merge Changeset: 1d8728efc05f Author: amurillo Date: 2014-01-28 15:00 -0800 URL: http://hg.openjdk.java.net/jdk8/tl/hotspot/rev/1d8728efc05f Added tag hs25-b69 for changeset 32f017489ba5 ! .hgtags Changeset: 35038da7bb9d Author: lana Date: 2014-01-29 11:11 -0800 URL: http://hg.openjdk.java.net/jdk8/tl/hotspot/rev/35038da7bb9d Merge Changeset: 874c0b4a946c Author: katleman Date: 2014-01-30 12:16 -0800 URL: http://hg.openjdk.java.net/jdk8/tl/hotspot/rev/874c0b4a946c Added tag jdk8-b127 for changeset 35038da7bb9d ! .hgtags Changeset: cb39165c4a65 Author: katleman Date: 2014-02-01 18:21 -0800 URL: http://hg.openjdk.java.net/jdk8/tl/hotspot/rev/cb39165c4a65 Added tag jdk8-b128 for changeset 874c0b4a946c ! .hgtags Changeset: 1dbaf664a611 Author: katleman Date: 2014-02-06 17:34 -0800 URL: http://hg.openjdk.java.net/jdk8/tl/hotspot/rev/1dbaf664a611 Added tag jdk8-b129 for changeset cb39165c4a65 ! .hgtags From lana.steuck at oracle.com Tue Feb 11 21:40:47 2014 From: lana.steuck at oracle.com (lana.steuck at oracle.com) Date: Tue, 11 Feb 2014 21:40:47 +0000 Subject: hg: jdk8/tl/jdk: 14 new changesets Message-ID: <20140211214445.3A9FE62BA0@hg.openjdk.java.net> Changeset: 75cf17ceb6d1 Author: katleman Date: 2014-01-22 12:54 -0800 URL: http://hg.openjdk.java.net/jdk8/tl/jdk/rev/75cf17ceb6d1 Added tag jdk8-b125 for changeset ae303640bc1c ! .hgtags Changeset: 95410515ba5f Author: katleman Date: 2014-01-22 14:08 -0800 URL: http://hg.openjdk.java.net/jdk8/tl/jdk/rev/95410515ba5f Merge ! .hgtags Changeset: 91bce40d0347 Author: alexsch Date: 2014-01-23 20:36 +0400 URL: http://hg.openjdk.java.net/jdk8/tl/jdk/rev/91bce40d0347 8032063: javax.swing.plaf.metal.MetalFileChooserUI$FilterComboBoxModel extends non-standard API Reviewed-by: pchelko, serb ! src/macosx/classes/com/apple/laf/AquaFileChooserUI.java ! src/share/classes/com/sun/java/swing/plaf/gtk/GTKFileChooserUI.java ! src/share/classes/com/sun/java/swing/plaf/motif/MotifFileChooserUI.java ! src/share/classes/com/sun/java/swing/plaf/windows/WindowsFileChooserUI.java ! src/share/classes/javax/swing/plaf/metal/MetalFileChooserUI.java - src/share/classes/sun/swing/AbstractFilterComboBoxModel.java ! src/share/classes/sun/swing/plaf/synth/SynthFileChooserUIImpl.java Changeset: 6935e7a3a7c9 Author: amurillo Date: 2014-01-23 14:46 -0800 URL: http://hg.openjdk.java.net/jdk8/tl/jdk/rev/6935e7a3a7c9 Merge Changeset: a9088d517f2f Author: amurillo Date: 2014-01-23 14:47 -0800 URL: http://hg.openjdk.java.net/jdk8/tl/jdk/rev/a9088d517f2f Merge Changeset: a635c394328c Author: katleman Date: 2014-01-24 15:08 -0800 URL: http://hg.openjdk.java.net/jdk8/tl/jdk/rev/a635c394328c Added tag jdk8-b126 for changeset a9088d517f2f ! .hgtags Changeset: fbf251b8ef8a Author: lana Date: 2014-01-29 11:11 -0800 URL: http://hg.openjdk.java.net/jdk8/tl/jdk/rev/fbf251b8ef8a Merge Changeset: f777d83e0433 Author: katleman Date: 2014-01-30 12:17 -0800 URL: http://hg.openjdk.java.net/jdk8/tl/jdk/rev/f777d83e0433 Added tag jdk8-b127 for changeset fbf251b8ef8a ! .hgtags Changeset: f644211c59fd Author: lana Date: 2014-01-31 13:47 -0800 URL: http://hg.openjdk.java.net/jdk8/tl/jdk/rev/f644211c59fd Merge Changeset: 3c9473004f38 Author: katleman Date: 2014-02-01 18:21 -0800 URL: http://hg.openjdk.java.net/jdk8/tl/jdk/rev/3c9473004f38 Added tag jdk8-b128 for changeset f644211c59fd ! .hgtags Changeset: ab6e7bb8ff9f Author: pchelko Date: 2014-01-22 16:15 +0400 URL: http://hg.openjdk.java.net/jdk8/tl/jdk/rev/ab6e7bb8ff9f 7155984: Security problems in regression test java/awt/PrintJob/Security/SecurityDialogTest.java Reviewed-by: anthony, serb ! src/macosx/classes/apple/laf/JRSUIUtils.java Changeset: eef10feca8ca Author: lana Date: 2014-02-06 13:28 -0800 URL: http://hg.openjdk.java.net/jdk8/tl/jdk/rev/eef10feca8ca Merge Changeset: 80568a19aab7 Author: lana Date: 2014-02-06 13:29 -0800 URL: http://hg.openjdk.java.net/jdk8/tl/jdk/rev/80568a19aab7 Merge Changeset: 43386cc9a017 Author: katleman Date: 2014-02-06 17:35 -0800 URL: http://hg.openjdk.java.net/jdk8/tl/jdk/rev/43386cc9a017 Added tag jdk8-b129 for changeset 80568a19aab7 ! .hgtags From martinrb at google.com Tue Feb 11 21:46:58 2014 From: martinrb at google.com (Martin Buchholz) Date: Tue, 11 Feb 2014 13:46:58 -0800 Subject: Draft JEP on enhanced volatiles In-Reply-To: References: <52F527A4.3030802@cs.oswego.edu> <52F8467B.3050004@oracle.com> <52F8C6FC.8050205@cs.oswego.edu> Message-ID: On Mon, Feb 10, 2014 at 3:43 PM, Mike Duigou wrote: > > On Feb 10 2014, at 04:33 , Doug Lea
wrote: > > > Among the constraints on solution is that several of these > > methods compile down to no-ops on many common platforms. > > This is interpreted as a benefit. However, if the are no-ops only for the > most common platform (x86) then we are likely to see a rough path for other > platforms. Incorrect or inconsistent usage of these methods in user code > will, instead of having no effect, result in unexpected and possibly > mysterious runtime failures on other platforms. We already have the problem that programs that succeed on x86 may fail on other platforms with a weaker memory model. The JVM has been pretty good about hiding these differences thus far. It > would be nice to see a "least common denominator" non-no-op mode as part of > the VM implementation that provided the opportunity to test in the most > pessimistic conditions. > If you wanted to be able enable robust testing, create an adversarial VM implementation (e.g. via hotspot flag) that did maximal compiler reordering, e.g. between synchronization points write values to main memory in reverse order. But that's a big job that's probably not going to happen? From roger.riggs at oracle.com Tue Feb 11 21:48:18 2014 From: roger.riggs at oracle.com (roger riggs) Date: Tue, 11 Feb 2014 16:48:18 -0500 Subject: RFR: (8031737) CHECK_NULL and CHECK_EXCEPTION macros cleanup In-Reply-To: <52FA98A8.7090104@oracle.com> References: <52ED36FD.5070507@oracle.com> <1243D3E7-C384-4AA8-80CA-9AA4E053F056@oracle.com> <52F00EAC.8000900@oracle.com> <52FA7429.5070806@oracle.com> <52FA768A.70005@oracle.com> <52FA80A6.2040509@oracle.com> <52FA87E2.9080009@oracle.com> <52FA98A8.7090104@oracle.com> Message-ID: <52FA9AA2.6090005@oracle.com> Hi Phil, The later changeset picked up the recommended style of implementing the macros but I don't think it was substantive. You can probably do without it. Version.c had some changes in a different changeset to address the omission of checking for exceptions after some JNI calls. Roger On 2/11/2014 4:39 PM, Phil Race wrote: > Roger, > > That later one seems to be using the macros. I don't see any update to > the macros. > So I'm not sure why I'm need it .. since I'm not using those calls and > neither > are the macros. > > -phil. > > On 2/11/14 12:28 PM, roger riggs wrote: >> Hi Phil, >> >> Yes, it ended up in two change sets in jdk 9, you should take both to >> be up to date. >> >> changeset: 9245:a09982d91fab >> user: rriggs >> date: Wed Feb 05 10:59:53 2014 -0500 >> files: src/share/native/common/jni_util.c >> description: >> 8030993: Check jdk/src/share/native/common/jni_util.c for JNI pending >> exceptions >> >> >> changeset: 9229:fb89dc4fe8da >> date: Mon Feb 03 16:58:02 2014 -0500 >> files: src/share/native/common/jni_util.h >> src/share/native/sun/misc/Version.c >> interrupted! >> description: >> 8031737: CHECK_NULL and CHECK_EXCEPTION macros cleanup >> >> Thanks, Roger >> >> >> On 2/11/2014 2:57 PM, Phil Race wrote: >>> Roger, >>> >>> Yes, I can do that. >>> >>> I see here >>> http://cr.openjdk.java.net/~rriggs/webrev-check-cleanup-8031737/ that >>> 1) There was a previous version of these macros. >>> Looks like no need to worry about that I just need the latest version. >>> 2) There was also a change to Version.c. I can include that if you >>> think it >>> appropriate .. or omit it if you think its not essential. >>> >>> -phil. >>> >>> On 2/11/2014 11:14 AM, roger riggs wrote: >>>> Hi Phil, >>>> >>>> I see your point, there is nothing in the changes unique to 9. >>>> Do you want to take care of the back point? >>>> >>>> Roger >>>> >>>> On 2/11/2014 2:04 PM, Phil Race wrote: >>>>> Roger, >>>>> >>>>> Why not JDK 8u ? I've got a lot of changes that utilise these >>>>> that will >>>>> backport cleanly to JDK 8u only if 8u includes these macros. And >>>>> since >>>>> the changes are all over the place I don't fancy copy/pasting them >>>>> everywhere. I suspect I am not the only one who would like these >>>>> in 8u .. >>>>> >>>>> -phil. >>>>> >>>>> >>>>> On 02/03/2014 01:48 PM, roger riggs wrote: >>>>>> Hi Lance, >>>>>> >>>>>> The convenience macros are only intended for JDK 9. >>>>>> >>>>>> Roger >>>>>> >>>>>> On 2/1/2014 1:58 PM, Lance @ Oracle wrote: >>>>>>> Looks fine >>>>>>> >>>>>>> Which releases are you think of including this in if any besides 9? >>>>>>> >>>>>>> >>>>>>> Lance >>>>>>> Andersen| Principal Member of Technical Staff | +1.781.442.2037 >>>>>>> >>>>>>> Oracle Java Engineering >>>>>>> 1 Network Drive >>>>>>> Burlington, MA 01803 >>>>>>> Lance.Andersen at oracle.com >>>>>>> Sent from my iPad >>>>>>> >>>>>>> On Feb 1, 2014, at 1:03 PM, roger riggs >>>>>> > wrote: >>>>>> >>>>> >>>> >>> >> > From huizhe.wang at oracle.com Tue Feb 11 21:57:25 2014 From: huizhe.wang at oracle.com (huizhe wang) Date: Tue, 11 Feb 2014 13:57:25 -0800 Subject: RFR(S): 8034087: XML parser may overwrite element content if that content falls onto the border of an entity scanner buffer In-Reply-To: References: <52FA535D.70804@oracle.com> Message-ID: <52FA9CC5.3010104@oracle.com> Hi Volker, I agree with the approach below and jdk9/dev is the better forest. For the test itself, I would suggest reducing the following loop to 1 or 2 cases: for (int i = 0; i < testString.length(); i++) { test(createDocument(testString.toString(), i), ""+ i); } when i=7, the problem starts to show. It's sufficient to demonstrate the issue then by just entering 7. It's unnecessary to run the test 43 times. Thanks, Joe On 2/11/2014 9:00 AM, Volker Simonis wrote: > Hi Alan, > > you're right. I initially didn't saw the test because I just looked at > the change in the jaxp repository. > > If it will be approved, I'll put the test in the same directory like > the other test (i.e. test/javax/xml/jaxp/parsers/8027359). > > And yes, my plan was to get approval for both, the tests and the fix, > when asking for the permission to downport to jdk8u-dev and jdk7u-dev. > > Thanks, > Volker > > > On Tue, Feb 11, 2014 at 5:44 PM, Alan Bateman wrote: >> On 11/02/2014 14:57, Volker Simonis wrote: >>> Hi, >>> >>> after opening this bug yesterday for an issue found by my colleague >>> Steffen Schreiber we realized that this is actually a duplicate of >>> "8027359: XML parser returns incorrect parsing results" >>> (https://bugs.openjdk.java.net/browse/JDK-8027359). >>> >>> While there's no need now to submit a patch anymore, we'd >>> nevertheless like to contribute at least our test case for this issue: >>> >>> http://cr.openjdk.java.net/~simonis/webrevs/8034087/ >>> >>> The webrev is against jdk9-client but we'd like to also downport this >>> test to jdk7 and jdk8 to track that the fix for 8027359 will be >>> correctly downported to these releases as well. >>> >>> I will sponsor this change if somebody would be so kind to review it. >>> >> I'll leave it to Joe Wang to comment on the test but just to mention that >> jdk9/dev is probably a better forest to aim for because that is where the >> XML (and its tests) usually go. Also I wonder if it might be better to put >> it in the same directory as the test that Joe pushed with the change? >> >> If you are getting approval to push to jdk8u-dev and jdk7u-dev then it might >> be better to request a backport of Joe's change at the same time. >> >> -Alan. From lana.steuck at oracle.com Tue Feb 11 21:40:08 2014 From: lana.steuck at oracle.com (lana.steuck at oracle.com) Date: Tue, 11 Feb 2014 21:40:08 +0000 Subject: hg: jdk8/tl: 8 new changesets Message-ID: <20140211214010.B63CD62B98@hg.openjdk.java.net> Changeset: 950921234b10 Author: katleman Date: 2014-01-22 12:53 -0800 URL: http://hg.openjdk.java.net/jdk8/tl/rev/950921234b10 Added tag jdk8-b125 for changeset 790bbd46b201 ! .hgtags Changeset: 1b5d578f93ef Author: katleman Date: 2014-01-22 14:06 -0800 URL: http://hg.openjdk.java.net/jdk8/tl/rev/1b5d578f93ef Merge ! .hgtags Changeset: 9ccce5bf1b0e Author: katleman Date: 2014-01-24 04:45 -0800 URL: http://hg.openjdk.java.net/jdk8/tl/rev/9ccce5bf1b0e Merge Changeset: 4f8fa4724c14 Author: katleman Date: 2014-01-24 15:07 -0800 URL: http://hg.openjdk.java.net/jdk8/tl/rev/4f8fa4724c14 Added tag jdk8-b126 for changeset 9ccce5bf1b0e ! .hgtags Changeset: 2e2ffb9e4b69 Author: lana Date: 2014-01-29 11:11 -0800 URL: http://hg.openjdk.java.net/jdk8/tl/rev/2e2ffb9e4b69 Merge Changeset: 101e42de4686 Author: katleman Date: 2014-01-30 12:16 -0800 URL: http://hg.openjdk.java.net/jdk8/tl/rev/101e42de4686 Added tag jdk8-b127 for changeset 2e2ffb9e4b69 ! .hgtags Changeset: 1e5fe8654913 Author: katleman Date: 2014-02-01 18:21 -0800 URL: http://hg.openjdk.java.net/jdk8/tl/rev/1e5fe8654913 Added tag jdk8-b128 for changeset 101e42de4686 ! .hgtags Changeset: 839546caab12 Author: katleman Date: 2014-02-06 17:34 -0800 URL: http://hg.openjdk.java.net/jdk8/tl/rev/839546caab12 Added tag jdk8-b129 for changeset 1e5fe8654913 ! .hgtags From philip.race at oracle.com Tue Feb 11 22:09:06 2014 From: philip.race at oracle.com (Phil Race) Date: Tue, 11 Feb 2014 14:09:06 -0800 Subject: RFR: (8031737) CHECK_NULL and CHECK_EXCEPTION macros cleanup In-Reply-To: <52FA9AA2.6090005@oracle.com> References: <52ED36FD.5070507@oracle.com> <1243D3E7-C384-4AA8-80CA-9AA4E053F056@oracle.com> <52F00EAC.8000900@oracle.com> <52FA7429.5070806@oracle.com> <52FA768A.70005@oracle.com> <52FA80A6.2040509@oracle.com> <52FA87E2.9080009@oracle.com> <52FA98A8.7090104@oracle.com> <52FA9AA2.6090005@oracle.com> Message-ID: <52FA9F82.90903@oracle.com> Are we talking about the same changesets ? a09982d91fab/8030993 has no change to the macros fb89dc4fe8da/8031737 is the one that reimplemented the macros and is the version I'd want. Its the last 'edit' of those macros in that file. c58c6b0fbe34/8030875 is the original addition of these :- ... changeset: 9229:fb89dc4fe8da user: rriggs date: Mon Feb 03 16:58:02 2014 -0500 summary: 8031737: CHECK_NULL and CHECK_EXCEPTION macros cleanup changeset: 9051:c58c6b0fbe34 user: rriggs date: Fri Jan 10 10:45:56 2014 -0500 summary: 8030875: Macros for checking and returning on exceptions ... -phil. On 2/11/14 1:48 PM, roger riggs wrote: > Hi Phil, > > The later changeset picked up the recommended style of implementing > the macros > but I don't think it was substantive. You can probably do without it. > > Version.c had some changes in a different changeset to address > the omission of checking for exceptions after some JNI calls. > > Roger > > On 2/11/2014 4:39 PM, Phil Race wrote: >> Roger, >> >> That later one seems to be using the macros. I don't see any update >> to the macros. >> So I'm not sure why I'm need it .. since I'm not using those calls >> and neither >> are the macros. >> >> -phil. >> >> On 2/11/14 12:28 PM, roger riggs wrote: >>> Hi Phil, >>> >>> Yes, it ended up in two change sets in jdk 9, you should take both >>> to be up to date. >>> >>> changeset: 9245:a09982d91fab >>> user: rriggs >>> date: Wed Feb 05 10:59:53 2014 -0500 >>> files: src/share/native/common/jni_util.c >>> description: >>> 8030993: Check jdk/src/share/native/common/jni_util.c for JNI >>> pending exceptions >>> >>> >>> changeset: 9229:fb89dc4fe8da >>> date: Mon Feb 03 16:58:02 2014 -0500 >>> files: src/share/native/common/jni_util.h >>> src/share/native/sun/misc/Version.c >>> interrupted! >>> description: >>> 8031737: CHECK_NULL and CHECK_EXCEPTION macros cleanup >>> >>> Thanks, Roger >>> >>> >>> On 2/11/2014 2:57 PM, Phil Race wrote: >>>> Roger, >>>> >>>> Yes, I can do that. >>>> >>>> I see here >>>> http://cr.openjdk.java.net/~rriggs/webrev-check-cleanup-8031737/ that >>>> 1) There was a previous version of these macros. >>>> Looks like no need to worry about that I just need the latest version. >>>> 2) There was also a change to Version.c. I can include that if you >>>> think it >>>> appropriate .. or omit it if you think its not essential. >>>> >>>> -phil. >>>> >>>> On 2/11/2014 11:14 AM, roger riggs wrote: >>>>> Hi Phil, >>>>> >>>>> I see your point, there is nothing in the changes unique to 9. >>>>> Do you want to take care of the back point? >>>>> >>>>> Roger >>>>> >>>>> On 2/11/2014 2:04 PM, Phil Race wrote: >>>>>> Roger, >>>>>> >>>>>> Why not JDK 8u ? I've got a lot of changes that utilise these >>>>>> that will >>>>>> backport cleanly to JDK 8u only if 8u includes these macros. And >>>>>> since >>>>>> the changes are all over the place I don't fancy copy/pasting them >>>>>> everywhere. I suspect I am not the only one who would like these >>>>>> in 8u .. >>>>>> >>>>>> -phil. >>>>>> >>>>>> >>>>>> On 02/03/2014 01:48 PM, roger riggs wrote: >>>>>>> Hi Lance, >>>>>>> >>>>>>> The convenience macros are only intended for JDK 9. >>>>>>> >>>>>>> Roger >>>>>>> >>>>>>> On 2/1/2014 1:58 PM, Lance @ Oracle wrote: >>>>>>>> Looks fine >>>>>>>> >>>>>>>> Which releases are you think of including this in if any >>>>>>>> besides 9? >>>>>>>> >>>>>>>> >>>>>>>> Lance >>>>>>>> Andersen| Principal Member of Technical Staff | +1.781.442.2037 >>>>>>>> >>>>>>>> Oracle Java Engineering >>>>>>>> 1 Network Drive >>>>>>>> Burlington, MA 01803 >>>>>>>> Lance.Andersen at oracle.com >>>>>>>> Sent from my iPad >>>>>>>> >>>>>>>> On Feb 1, 2014, at 1:03 PM, roger riggs >>>>>>> > wrote: >>>>>>> >>>>>> >>>>> >>>> >>> >> > From roger.riggs at oracle.com Tue Feb 11 22:20:01 2014 From: roger.riggs at oracle.com (roger riggs) Date: Tue, 11 Feb 2014 17:20:01 -0500 Subject: RFR: (8031737) CHECK_NULL and CHECK_EXCEPTION macros cleanup In-Reply-To: <52FA9F82.90903@oracle.com> References: <52ED36FD.5070507@oracle.com> <1243D3E7-C384-4AA8-80CA-9AA4E053F056@oracle.com> <52F00EAC.8000900@oracle.com> <52FA7429.5070806@oracle.com> <52FA768A.70005@oracle.com> <52FA80A6.2040509@oracle.com> <52FA87E2.9080009@oracle.com> <52FA98A8.7090104@oracle.com> <52FA9AA2.6090005@oracle.com> <52FA9F82.90903@oracle.com> Message-ID: <52FAA211.2070800@oracle.com> Hi Phil, On 2/11/2014 5:09 PM, Phil Race wrote: > Are we talking about the same changesets ? > a09982d91fab/8030993 has no change to the macros right (I didn't think this was topic of this conversation) > > fb89dc4fe8da/8031737 is the one that reimplemented the macros > and is the version I'd want. Its the last 'edit' of those macros in > that file. yes, > > c58c6b0fbe34/8030875 is the original addition of these :- Yes. Roger > > ... > > changeset: 9229:fb89dc4fe8da > user: rriggs > date: Mon Feb 03 16:58:02 2014 -0500 > summary: 8031737: CHECK_NULL and CHECK_EXCEPTION macros cleanup > > changeset: 9051:c58c6b0fbe34 > user: rriggs > date: Fri Jan 10 10:45:56 2014 -0500 > summary: 8030875: Macros for checking and returning on exceptions > > > ... > > -phil. > > On 2/11/14 1:48 PM, roger riggs wrote: >> Hi Phil, >> >> The later changeset picked up the recommended style of implementing >> the macros >> but I don't think it was substantive. You can probably do without it. >> >> Version.c had some changes in a different changeset to address >> the omission of checking for exceptions after some JNI calls. >> >> Roger >> >> On 2/11/2014 4:39 PM, Phil Race wrote: >>> Roger, >>> >>> That later one seems to be using the macros. I don't see any update >>> to the macros. >>> So I'm not sure why I'm need it .. since I'm not using those calls >>> and neither >>> are the macros. >>> >>> -phil. >>> >>> On 2/11/14 12:28 PM, roger riggs wrote: >>>> Hi Phil, >>>> >>>> Yes, it ended up in two change sets in jdk 9, you should take both >>>> to be up to date. >>>> >>>> changeset: 9245:a09982d91fab >>>> user: rriggs >>>> date: Wed Feb 05 10:59:53 2014 -0500 >>>> files: src/share/native/common/jni_util.c >>>> description: >>>> 8030993: Check jdk/src/share/native/common/jni_util.c for JNI >>>> pending exceptions >>>> >>>> >>>> changeset: 9229:fb89dc4fe8da >>>> date: Mon Feb 03 16:58:02 2014 -0500 >>>> files: src/share/native/common/jni_util.h >>>> src/share/native/sun/misc/Version.c >>>> interrupted! >>>> description: >>>> 8031737: CHECK_NULL and CHECK_EXCEPTION macros cleanup >>>> >>>> Thanks, Roger >>>> >>>> >>>> On 2/11/2014 2:57 PM, Phil Race wrote: >>>>> Roger, >>>>> >>>>> Yes, I can do that. >>>>> >>>>> I see here >>>>> http://cr.openjdk.java.net/~rriggs/webrev-check-cleanup-8031737/ that >>>>> 1) There was a previous version of these macros. >>>>> Looks like no need to worry about that I just need the latest >>>>> version. >>>>> 2) There was also a change to Version.c. I can include that if you >>>>> think it >>>>> appropriate .. or omit it if you think its not essential. >>>>> >>>>> -phil. >>>>> >>>>> On 2/11/2014 11:14 AM, roger riggs wrote: >>>>>> Hi Phil, >>>>>> >>>>>> I see your point, there is nothing in the changes unique to 9. >>>>>> Do you want to take care of the back point? >>>>>> >>>>>> Roger >>>>>> >>>>>> On 2/11/2014 2:04 PM, Phil Race wrote: >>>>>>> Roger, >>>>>>> >>>>>>> Why not JDK 8u ? I've got a lot of changes that utilise these >>>>>>> that will >>>>>>> backport cleanly to JDK 8u only if 8u includes these macros. And >>>>>>> since >>>>>>> the changes are all over the place I don't fancy copy/pasting them >>>>>>> everywhere. I suspect I am not the only one who would like these >>>>>>> in 8u .. >>>>>>> >>>>>>> -phil. >>>>>>> >>>>>>> >>>>>>> On 02/03/2014 01:48 PM, roger riggs wrote: >>>>>>>> Hi Lance, >>>>>>>> >>>>>>>> The convenience macros are only intended for JDK 9. >>>>>>>> >>>>>>>> Roger >>>>>>>> >>>>>>>> On 2/1/2014 1:58 PM, Lance @ Oracle wrote: >>>>>>>>> Looks fine >>>>>>>>> >>>>>>>>> Which releases are you think of including this in if any >>>>>>>>> besides 9? >>>>>>>>> >>>>>>>>> >>>>>>>>> Lance >>>>>>>>> Andersen| Principal Member of Technical Staff | >>>>>>>>> +1.781.442.2037 >>>>>>>>> Oracle Java Engineering >>>>>>>>> 1 Network Drive >>>>>>>>> Burlington, MA 01803 >>>>>>>>> Lance.Andersen at oracle.com >>>>>>>>> Sent from my iPad >>>>>>>>> >>>>>>>>> On Feb 1, 2014, at 1:03 PM, roger riggs >>>>>>>>> > wrote: >>>>>>>> >>>>>>> >>>>>> >>>>> >>>> >>> >> > From philip.race at oracle.com Tue Feb 11 22:28:04 2014 From: philip.race at oracle.com (Phil Race) Date: Tue, 11 Feb 2014 14:28:04 -0800 Subject: RFR: (8031737) CHECK_NULL and CHECK_EXCEPTION macros cleanup In-Reply-To: <52FAA211.2070800@oracle.com> References: <52ED36FD.5070507@oracle.com> <1243D3E7-C384-4AA8-80CA-9AA4E053F056@oracle.com> <52F00EAC.8000900@oracle.com> <52FA7429.5070806@oracle.com> <52FA768A.70005@oracle.com> <52FA80A6.2040509@oracle.com> <52FA87E2.9080009@oracle.com> <52FA98A8.7090104@oracle.com> <52FA9AA2.6090005@oracle.com> <52FA9F82.90903@oracle.com> <52FAA211.2070800@oracle.com> Message-ID: <52FAA3F4.6040702@oracle.com> So since hg export/import doesn't apply cleanly and the dependency chain seems, long and in order to have some consistency across the releases, I think I should prepare a webrev which essentially backports 8031737 including its small changes to Version.c, if only because otherwise I'd have to have a new bug ID that would not be forwarded ported (one source of confusion) or even worse re-use 8031737 but not fully implement it Agreed ? -phil. On 2/11/2014 2:20 PM, roger riggs wrote: > Hi Phil, > > > On 2/11/2014 5:09 PM, Phil Race wrote: >> Are we talking about the same changesets ? >> a09982d91fab/8030993 has no change to the macros > right (I didn't think this was topic of this conversation) >> >> fb89dc4fe8da/8031737 is the one that reimplemented the macros >> and is the version I'd want. Its the last 'edit' of those macros in >> that file. > yes, >> >> c58c6b0fbe34/8030875 is the original addition of these :- > Yes. > > Roger > >> >> ... >> >> changeset: 9229:fb89dc4fe8da >> user: rriggs >> date: Mon Feb 03 16:58:02 2014 -0500 >> summary: 8031737: CHECK_NULL and CHECK_EXCEPTION macros cleanup >> >> changeset: 9051:c58c6b0fbe34 >> user: rriggs >> date: Fri Jan 10 10:45:56 2014 -0500 >> summary: 8030875: Macros for checking and returning on exceptions >> >> >> ... >> >> -phil. >> >> On 2/11/14 1:48 PM, roger riggs wrote: >>> Hi Phil, >>> >>> The later changeset picked up the recommended style of implementing >>> the macros >>> but I don't think it was substantive. You can probably do without it. >>> >>> Version.c had some changes in a different changeset to address >>> the omission of checking for exceptions after some JNI calls. >>> >>> Roger >>> >>> On 2/11/2014 4:39 PM, Phil Race wrote: >>>> Roger, >>>> >>>> That later one seems to be using the macros. I don't see any update >>>> to the macros. >>>> So I'm not sure why I'm need it .. since I'm not using those calls >>>> and neither >>>> are the macros. >>>> >>>> -phil. >>>> >>>> On 2/11/14 12:28 PM, roger riggs wrote: >>>>> Hi Phil, >>>>> >>>>> Yes, it ended up in two change sets in jdk 9, you should take both >>>>> to be up to date. >>>>> >>>>> changeset: 9245:a09982d91fab >>>>> user: rriggs >>>>> date: Wed Feb 05 10:59:53 2014 -0500 >>>>> files: src/share/native/common/jni_util.c >>>>> description: >>>>> 8030993: Check jdk/src/share/native/common/jni_util.c for JNI >>>>> pending exceptions >>>>> >>>>> >>>>> changeset: 9229:fb89dc4fe8da >>>>> date: Mon Feb 03 16:58:02 2014 -0500 >>>>> files: src/share/native/common/jni_util.h >>>>> src/share/native/sun/misc/Version.c >>>>> interrupted! >>>>> description: >>>>> 8031737: CHECK_NULL and CHECK_EXCEPTION macros cleanup >>>>> >>>>> Thanks, Roger >>>>> >>>>> >>>>> On 2/11/2014 2:57 PM, Phil Race wrote: >>>>>> Roger, >>>>>> >>>>>> Yes, I can do that. >>>>>> >>>>>> I see here >>>>>> http://cr.openjdk.java.net/~rriggs/webrev-check-cleanup-8031737/ >>>>>> that >>>>>> 1) There was a previous version of these macros. >>>>>> Looks like no need to worry about that I just need the latest >>>>>> version. >>>>>> 2) There was also a change to Version.c. I can include that if >>>>>> you think it >>>>>> appropriate .. or omit it if you think its not essential. >>>>>> >>>>>> -phil. >>>>>> >>>>>> On 2/11/2014 11:14 AM, roger riggs wrote: >>>>>>> Hi Phil, >>>>>>> >>>>>>> I see your point, there is nothing in the changes unique to 9. >>>>>>> Do you want to take care of the back point? >>>>>>> >>>>>>> Roger >>>>>>> >>>>>>> On 2/11/2014 2:04 PM, Phil Race wrote: >>>>>>>> Roger, >>>>>>>> >>>>>>>> Why not JDK 8u ? I've got a lot of changes that utilise these >>>>>>>> that will >>>>>>>> backport cleanly to JDK 8u only if 8u includes these macros. >>>>>>>> And since >>>>>>>> the changes are all over the place I don't fancy copy/pasting them >>>>>>>> everywhere. I suspect I am not the only one who would like >>>>>>>> these in 8u .. >>>>>>>> >>>>>>>> -phil. >>>>>>>> >>>>>>>> >>>>>>>> On 02/03/2014 01:48 PM, roger riggs wrote: >>>>>>>>> Hi Lance, >>>>>>>>> >>>>>>>>> The convenience macros are only intended for JDK 9. >>>>>>>>> >>>>>>>>> Roger >>>>>>>>> >>>>>>>>> On 2/1/2014 1:58 PM, Lance @ Oracle wrote: >>>>>>>>>> Looks fine >>>>>>>>>> >>>>>>>>>> Which releases are you think of including this in if any >>>>>>>>>> besides 9? >>>>>>>>>> >>>>>>>>>> >>>>>>>>>> Lance Andersen| >>>>>>>>>> Principal Member of Technical Staff | +1.781.442.2037 >>>>>>>>>> >>>>>>>>>> Oracle Java Engineering >>>>>>>>>> 1 Network Drive >>>>>>>>>> Burlington, MA 01803 >>>>>>>>>> Lance.Andersen at oracle.com >>>>>>>>>> Sent from my iPad >>>>>>>>>> >>>>>>>>>> On Feb 1, 2014, at 1:03 PM, roger riggs >>>>>>>>>> > wrote: >>>>>>>>> >>>>>>>> >>>>>>> >>>>>> >>>>> >>>> >>> >> > From roger.riggs at oracle.com Tue Feb 11 22:26:52 2014 From: roger.riggs at oracle.com (roger riggs) Date: Tue, 11 Feb 2014 17:26:52 -0500 Subject: RFR: (8031737) CHECK_NULL and CHECK_EXCEPTION macros cleanup In-Reply-To: <52FAA3F4.6040702@oracle.com> References: <52ED36FD.5070507@oracle.com> <1243D3E7-C384-4AA8-80CA-9AA4E053F056@oracle.com> <52F00EAC.8000900@oracle.com> <52FA7429.5070806@oracle.com> <52FA768A.70005@oracle.com> <52FA80A6.2040509@oracle.com> <52FA87E2.9080009@oracle.com> <52FA98A8.7090104@oracle.com> <52FA9AA2.6090005@oracle.com> <52FA9F82.90903@oracle.com> <52FAA211.2070800@oracle.com> <52FAA3F4.6040702@oracle.com> Message-ID: <52FAA3AC.2020205@oracle.com> yes On 2/11/2014 5:28 PM, Phil Race wrote: > So since hg export/import doesn't apply cleanly and the dependency > chain seems, long and in order to have some consistency across the > releases, > I think I should prepare a webrev which essentially backports 8031737 > including its small changes to Version.c, if only because otherwise > I'd have to have a new bug ID that would not be forwarded ported > (one source of confusion) or even worse re-use 8031737 but not fully > implement it > > Agreed ? > > -phil. > > On 2/11/2014 2:20 PM, roger riggs wrote: >> Hi Phil, >> >> >> On 2/11/2014 5:09 PM, Phil Race wrote: >>> Are we talking about the same changesets ? >>> a09982d91fab/8030993 has no change to the macros >> right (I didn't think this was topic of this conversation) >>> >>> fb89dc4fe8da/8031737 is the one that reimplemented the macros >>> and is the version I'd want. Its the last 'edit' of those macros in >>> that file. >> yes, >>> >>> c58c6b0fbe34/8030875 is the original addition of these :- >> Yes. >> >> Roger >> >>> >>> ... >>> >>> changeset: 9229:fb89dc4fe8da >>> user: rriggs >>> date: Mon Feb 03 16:58:02 2014 -0500 >>> summary: 8031737: CHECK_NULL and CHECK_EXCEPTION macros cleanup >>> >>> changeset: 9051:c58c6b0fbe34 >>> user: rriggs >>> date: Fri Jan 10 10:45:56 2014 -0500 >>> summary: 8030875: Macros for checking and returning on exceptions >>> >>> >>> ... >>> >>> -phil. >>> >>> On 2/11/14 1:48 PM, roger riggs wrote: >>>> Hi Phil, >>>> >>>> The later changeset picked up the recommended style of implementing >>>> the macros >>>> but I don't think it was substantive. You can probably do without it. >>>> >>>> Version.c had some changes in a different changeset to address >>>> the omission of checking for exceptions after some JNI calls. >>>> >>>> Roger >>>> >>>> On 2/11/2014 4:39 PM, Phil Race wrote: >>>>> Roger, >>>>> >>>>> That later one seems to be using the macros. I don't see any >>>>> update to the macros. >>>>> So I'm not sure why I'm need it .. since I'm not using those calls >>>>> and neither >>>>> are the macros. >>>>> >>>>> -phil. >>>>> >>>>> On 2/11/14 12:28 PM, roger riggs wrote: >>>>>> Hi Phil, >>>>>> >>>>>> Yes, it ended up in two change sets in jdk 9, you should take >>>>>> both to be up to date. >>>>>> >>>>>> changeset: 9245:a09982d91fab >>>>>> user: rriggs >>>>>> date: Wed Feb 05 10:59:53 2014 -0500 >>>>>> files: src/share/native/common/jni_util.c >>>>>> description: >>>>>> 8030993: Check jdk/src/share/native/common/jni_util.c for JNI >>>>>> pending exceptions >>>>>> >>>>>> >>>>>> changeset: 9229:fb89dc4fe8da >>>>>> date: Mon Feb 03 16:58:02 2014 -0500 >>>>>> files: src/share/native/common/jni_util.h >>>>>> src/share/native/sun/misc/Version.c >>>>>> interrupted! >>>>>> description: >>>>>> 8031737: CHECK_NULL and CHECK_EXCEPTION macros cleanup >>>>>> >>>>>> Thanks, Roger >>>>>> >>>>>> >>>>>> On 2/11/2014 2:57 PM, Phil Race wrote: >>>>>>> Roger, >>>>>>> >>>>>>> Yes, I can do that. >>>>>>> >>>>>>> I see here >>>>>>> http://cr.openjdk.java.net/~rriggs/webrev-check-cleanup-8031737/ >>>>>>> that >>>>>>> 1) There was a previous version of these macros. >>>>>>> Looks like no need to worry about that I just need the latest >>>>>>> version. >>>>>>> 2) There was also a change to Version.c. I can include that if >>>>>>> you think it >>>>>>> appropriate .. or omit it if you think its not essential. >>>>>>> >>>>>>> -phil. >>>>>>> >>>>>>> On 2/11/2014 11:14 AM, roger riggs wrote: >>>>>>>> Hi Phil, >>>>>>>> >>>>>>>> I see your point, there is nothing in the changes unique to 9. >>>>>>>> Do you want to take care of the back point? >>>>>>>> >>>>>>>> Roger >>>>>>>> >>>>>>>> On 2/11/2014 2:04 PM, Phil Race wrote: >>>>>>>>> Roger, >>>>>>>>> >>>>>>>>> Why not JDK 8u ? I've got a lot of changes that utilise these >>>>>>>>> that will >>>>>>>>> backport cleanly to JDK 8u only if 8u includes these macros. >>>>>>>>> And since >>>>>>>>> the changes are all over the place I don't fancy copy/pasting >>>>>>>>> them >>>>>>>>> everywhere. I suspect I am not the only one who would like >>>>>>>>> these in 8u .. >>>>>>>>> >>>>>>>>> -phil. >>>>>>>>> >>>>>>>>> >>>>>>>>> On 02/03/2014 01:48 PM, roger riggs wrote: >>>>>>>>>> Hi Lance, >>>>>>>>>> >>>>>>>>>> The convenience macros are only intended for JDK 9. >>>>>>>>>> >>>>>>>>>> Roger >>>>>>>>>> >>>>>>>>>> On 2/1/2014 1:58 PM, Lance @ Oracle wrote: >>>>>>>>>>> Looks fine >>>>>>>>>>> >>>>>>>>>>> Which releases are you think of including this in if any >>>>>>>>>>> besides 9? >>>>>>>>>>> >>>>>>>>>>> >>>>>>>>>>> Lance >>>>>>>>>>> Andersen| Principal Member of Technical Staff | >>>>>>>>>>> +1.781.442.2037 >>>>>>>>>>> Oracle Java Engineering >>>>>>>>>>> 1 Network Drive >>>>>>>>>>> Burlington, MA 01803 >>>>>>>>>>> Lance.Andersen at oracle.com >>>>>>>>>>> Sent from my iPad >>>>>>>>>>> >>>>>>>>>>> On Feb 1, 2014, at 1:03 PM, roger riggs >>>>>>>>>>> > wrote: >>>>>>>>>> >>>>>>>>> >>>>>>>> >>>>>>> >>>>>> >>>>> >>>> >>> >> > From martinrb at google.com Tue Feb 11 23:16:42 2014 From: martinrb at google.com (Martin Buchholz) Date: Tue, 11 Feb 2014 15:16:42 -0800 Subject: Random access in ArrayDeque In-Reply-To: <52F6867B.4060703@cs.oswego.edu> References: <231A3FB1-639F-4C06-823B-0CB9F1CB194A@oracle.com> <52F520EE.70709@redhat.com> <52F6867B.4060703@cs.oswego.edu> Message-ID: On Sat, Feb 8, 2014 at 11:33 AM, Doug Lea
wrote: > On 02/07/2014 01:59 PM, Martin Buchholz wrote: > > ArrayDeque should implement most of the methods in List, notably get(i). >> >> ArrayDeque should not actually implement List itself, because the change >> in >> contract/behavior for hashCode/equals would be too great. >> > > Right. My vague recollection is that these were among initial reasons > for not implementing List. Also, the List.sublist method is questionable > for a Queue. > I don't think of ArrayDeque as a queue/deque at all. We just have two fundamental data structures here (ArrayList (ordinary resizable array) and ArrayDeque (circular resizable array)) both of which can implement List and Queue in reasonable ways, and ArrayDeque can additionally implement Deque. ArrayList.subList/listIterator implement "live" views with ConcurrentModificationException checking. I don't see why ArrayDeque couldn't do the same - that's what users will expect. You don't really need an asList method - subList(0, size()) would be equivalent. Although making an independent copy on asList is also attractive for avoiding overhead. I started some implementation work - mostly just very tedious in the way we're used to maintaining java.util collections. From philip.race at oracle.com Tue Feb 11 23:42:47 2014 From: philip.race at oracle.com (Phil Race) Date: Tue, 11 Feb 2014 15:42:47 -0800 Subject: RFR: (8031737) CHECK_NULL and CHECK_EXCEPTION macros cleanup In-Reply-To: <52FAA3F4.6040702@oracle.com> References: <52ED36FD.5070507@oracle.com> <1243D3E7-C384-4AA8-80CA-9AA4E053F056@oracle.com> <52F00EAC.8000900@oracle.com> <52FA7429.5070806@oracle.com> <52FA768A.70005@oracle.com> <52FA80A6.2040509@oracle.com> <52FA87E2.9080009@oracle.com> <52FA98A8.7090104@oracle.com> <52FA9AA2.6090005@oracle.com> <52FA9F82.90903@oracle.com> <52FAA211.2070800@oracle.com> <52FAA3F4.6040702@oracle.com> Message-ID: <52FAB577.3070205@oracle.com> Here's a JDk8u webrev : -http://cr.openjdk.java.net/~prr/8031737.8u/ -phil. On 2/11/14 2:28 PM, Phil Race wrote: > So since hg export/import doesn't apply cleanly and the dependency > chain seems, long and in order to have some consistency across the > releases, > I think I should prepare a webrev which essentially backports 8031737 > including its small changes to Version.c, if only because otherwise > I'd have to have a new bug ID that would not be forwarded ported > (one source of confusion) or even worse re-use 8031737 but not fully > implement it > > Agreed ? > > -phil. > > On 2/11/2014 2:20 PM, roger riggs wrote: >> Hi Phil, >> >> >> On 2/11/2014 5:09 PM, Phil Race wrote: >>> Are we talking about the same changesets ? >>> a09982d91fab/8030993 has no change to the macros >> right (I didn't think this was topic of this conversation) >>> >>> fb89dc4fe8da/8031737 is the one that reimplemented the macros >>> and is the version I'd want. Its the last 'edit' of those macros in >>> that file. >> yes, >>> >>> c58c6b0fbe34/8030875 is the original addition of these :- >> Yes. >> >> Roger >> >>> >>> ... >>> >>> changeset: 9229:fb89dc4fe8da >>> user: rriggs >>> date: Mon Feb 03 16:58:02 2014 -0500 >>> summary: 8031737: CHECK_NULL and CHECK_EXCEPTION macros cleanup >>> >>> changeset: 9051:c58c6b0fbe34 >>> user: rriggs >>> date: Fri Jan 10 10:45:56 2014 -0500 >>> summary: 8030875: Macros for checking and returning on exceptions >>> >>> >>> ... >>> >>> -phil. >>> >>> On 2/11/14 1:48 PM, roger riggs wrote: >>>> Hi Phil, >>>> >>>> The later changeset picked up the recommended style of implementing >>>> the macros >>>> but I don't think it was substantive. You can probably do without it. >>>> >>>> Version.c had some changes in a different changeset to address >>>> the omission of checking for exceptions after some JNI calls. >>>> >>>> Roger >>>> >>>> On 2/11/2014 4:39 PM, Phil Race wrote: >>>>> Roger, >>>>> >>>>> That later one seems to be using the macros. I don't see any >>>>> update to the macros. >>>>> So I'm not sure why I'm need it .. since I'm not using those calls >>>>> and neither >>>>> are the macros. >>>>> >>>>> -phil. >>>>> >>>>> On 2/11/14 12:28 PM, roger riggs wrote: >>>>>> Hi Phil, >>>>>> >>>>>> Yes, it ended up in two change sets in jdk 9, you should take >>>>>> both to be up to date. >>>>>> >>>>>> changeset: 9245:a09982d91fab >>>>>> user: rriggs >>>>>> date: Wed Feb 05 10:59:53 2014 -0500 >>>>>> files: src/share/native/common/jni_util.c >>>>>> description: >>>>>> 8030993: Check jdk/src/share/native/common/jni_util.c for JNI >>>>>> pending exceptions >>>>>> >>>>>> >>>>>> changeset: 9229:fb89dc4fe8da >>>>>> date: Mon Feb 03 16:58:02 2014 -0500 >>>>>> files: src/share/native/common/jni_util.h >>>>>> src/share/native/sun/misc/Version.c >>>>>> interrupted! >>>>>> description: >>>>>> 8031737: CHECK_NULL and CHECK_EXCEPTION macros cleanup >>>>>> >>>>>> Thanks, Roger >>>>>> >>>>>> >>>>>> On 2/11/2014 2:57 PM, Phil Race wrote: >>>>>>> Roger, >>>>>>> >>>>>>> Yes, I can do that. >>>>>>> >>>>>>> I see here >>>>>>> http://cr.openjdk.java.net/~rriggs/webrev-check-cleanup-8031737/ >>>>>>> that >>>>>>> 1) There was a previous version of these macros. >>>>>>> Looks like no need to worry about that I just need the latest >>>>>>> version. >>>>>>> 2) There was also a change to Version.c. I can include that if >>>>>>> you think it >>>>>>> appropriate .. or omit it if you think its not essential. >>>>>>> >>>>>>> -phil. >>>>>>> >>>>>>> On 2/11/2014 11:14 AM, roger riggs wrote: >>>>>>>> Hi Phil, >>>>>>>> >>>>>>>> I see your point, there is nothing in the changes unique to 9. >>>>>>>> Do you want to take care of the back point? >>>>>>>> >>>>>>>> Roger >>>>>>>> >>>>>>>> On 2/11/2014 2:04 PM, Phil Race wrote: >>>>>>>>> Roger, >>>>>>>>> >>>>>>>>> Why not JDK 8u ? I've got a lot of changes that utilise these >>>>>>>>> that will >>>>>>>>> backport cleanly to JDK 8u only if 8u includes these macros. >>>>>>>>> And since >>>>>>>>> the changes are all over the place I don't fancy copy/pasting >>>>>>>>> them >>>>>>>>> everywhere. I suspect I am not the only one who would like >>>>>>>>> these in 8u .. >>>>>>>>> >>>>>>>>> -phil. >>>>>>>>> >>>>>>>>> >>>>>>>>> On 02/03/2014 01:48 PM, roger riggs wrote: >>>>>>>>>> Hi Lance, >>>>>>>>>> >>>>>>>>>> The convenience macros are only intended for JDK 9. >>>>>>>>>> >>>>>>>>>> Roger >>>>>>>>>> >>>>>>>>>> On 2/1/2014 1:58 PM, Lance @ Oracle wrote: >>>>>>>>>>> Looks fine >>>>>>>>>>> >>>>>>>>>>> Which releases are you think of including this in if any >>>>>>>>>>> besides 9? >>>>>>>>>>> >>>>>>>>>>> >>>>>>>>>>> Lance >>>>>>>>>>> Andersen| Principal Member of Technical Staff | >>>>>>>>>>> +1.781.442.2037 >>>>>>>>>>> Oracle Java Engineering >>>>>>>>>>> 1 Network Drive >>>>>>>>>>> Burlington, MA 01803 >>>>>>>>>>> Lance.Andersen at oracle.com >>>>>>>>>>> Sent from my iPad >>>>>>>>>>> >>>>>>>>>>> On Feb 1, 2014, at 1:03 PM, roger riggs >>>>>>>>>>> > wrote: >>>>>>>>>> >>>>>>>>> >>>>>>>> >>>>>>> >>>>>> >>>>> >>>> >>> >> > From stuart.marks at oracle.com Wed Feb 12 02:06:16 2014 From: stuart.marks at oracle.com (Stuart Marks) Date: Tue, 11 Feb 2014 18:06:16 -0800 Subject: RFR for JDK-8030844:sun/rmi/rmic/classpath/RMICClassPathTest.java timeout in same binaries run In-Reply-To: <52F8C146.7060505@oracle.com> References: <13F6E908-26F2-420A-9E25-AD00DF8E3769@oracle.com> <52F8C146.7060505@oracle.com> Message-ID: <52FAD718.6090103@oracle.com> Hi, yes, I'll take this one. It's slightly odd that this is creating filenames that already have "/" in them (as opposed to File.separator) but since these files don't actually have to exist, I suppose it doesn't really matter. I'm not convinced that we actually have any evidence that /home/~user is really causing the hang/timeout (either caused by the automounter hanging on /home or LDAP or other nameservice lookup on ~user), but this is harmless, and it'll fix the problem on the off chance that this really is the root cause. Tristan, please update the test's @bug tag to add 8030844, create a changeset, and create a webrev with the changeset in it (as opposed to a bare patch). I'll then push it for you. s'marks On 2/10/14 4:08 AM, Alan Bateman wrote: > On 10/02/2014 10:57, Tristan Yan wrote: >> Ping: Can anyone give a review on this. >> Thank you >> Tristan > Changing the test so that it doesn't try to /home/~user seems reasonable to me. > > Stuart - I see you've been sponsoring Tristan's updates to the RMI tests. Are > you going to take this one too? > > -Alan > > >> >> On Feb 6, 2014, at 5:13 PM, Tristan Yan wrote: >> >>> Hi All >>> >>> Please help to review a simple fix for >>> https://bugs.openjdk.java.net/browse/JDK-8030844 >>> >>> http://cr.openjdk.java.net/~tyan/JDK-8030844/webrev.00/. >>> >>> Description: >>> Change replace a ?/home/~user" folder with an test source path. Folder >>> ?/home/~user? cause some problem whenever something wrong with the automount >>> filesystem or an username lookup problem. >>> >>> Thank you >>> Tristan > From tristan.yan at oracle.com Wed Feb 12 02:59:15 2014 From: tristan.yan at oracle.com (Tristan Yan) Date: Wed, 12 Feb 2014 10:59:15 +0800 Subject: RFR for JDK-8030844:sun/rmi/rmic/classpath/RMICClassPathTest.java timeout in same binaries run In-Reply-To: <52FAD718.6090103@oracle.com> References: <13F6E908-26F2-420A-9E25-AD00DF8E3769@oracle.com> <52F8C146.7060505@oracle.com> <52FAD718.6090103@oracle.com> Message-ID: <539F9CC8-77FA-43C6-A19A-C6623641CA6C@oracle.com> Thank you Stuart http://cr.openjdk.java.net/~tyan/JDK-8030844/webrev.01/ Regards Tristan On Feb 12, 2014, at 10:06 AM, Stuart Marks wrote: > Hi, yes, I'll take this one. > > It's slightly odd that this is creating filenames that already have "/" in them (as opposed to File.separator) but since these files don't actually have to exist, I suppose it doesn't really matter. > > I'm not convinced that we actually have any evidence that /home/~user is really causing the hang/timeout (either caused by the automounter hanging on /home or LDAP or other nameservice lookup on ~user), but this is harmless, and it'll fix the problem on the off chance that this really is the root cause. > > Tristan, please update the test's @bug tag to add 8030844, create a changeset, and create a webrev with the changeset in it (as opposed to a bare patch). I'll then push it for you. > > s'marks > > On 2/10/14 4:08 AM, Alan Bateman wrote: >> On 10/02/2014 10:57, Tristan Yan wrote: >>> Ping: Can anyone give a review on this. >>> Thank you >>> Tristan >> Changing the test so that it doesn't try to /home/~user seems reasonable to me. >> >> Stuart - I see you've been sponsoring Tristan's updates to the RMI tests. Are >> you going to take this one too? >> >> -Alan >> >> >>> >>> On Feb 6, 2014, at 5:13 PM, Tristan Yan wrote: >>> >>>> Hi All >>>> >>>> Please help to review a simple fix for >>>> https://bugs.openjdk.java.net/browse/JDK-8030844 >>>> >>>> http://cr.openjdk.java.net/~tyan/JDK-8030844/webrev.00/. >>>> >>>> Description: >>>> Change replace a ?/home/~user" folder with an test source path. Folder >>>> ?/home/~user? cause some problem whenever something wrong with the automount >>>> filesystem or an username lookup problem. >>>> >>>> Thank you >>>> Tristan >> From huizhe.wang at oracle.com Wed Feb 12 04:26:41 2014 From: huizhe.wang at oracle.com (huizhe wang) Date: Tue, 11 Feb 2014 20:26:41 -0800 Subject: RFR (JAXP): 8033980 : Xerces Update: datatype XMLGregorianCalendarImpl and DurationImpl In-Reply-To: <52FA7270.4090709@oracle.com> References: <52F9666F.9010303@oracle.com> <5111B487-22C1-4987-B9CC-ABF2F5CDD964@oracle.com> <52FA5855.3020106@oracle.com> <52FA7270.4090709@oracle.com> Message-ID: <52FAF801.5060300@oracle.com> Hi Lance, Daniel, I suggest we take this incompatibility and document it in the release notes (we'll likely have more). I reversed DurationImpl and then realized why the Xerces engineers made the incompatibility change when I started on XMLGregorianCalendarImpl. It was because XMLGregorianCalendarImpl did not implement what was recommended, using the lexical form for serialization. In the original implementation therefore, the impl between XMLGregorianCalendarImpl and DurationImpl was not consistent. Keeping this Xerces revision makes the serialization consistent in both classes and adds the benefit of having the same source as that of Xerces. Thanks, Joe On 2/11/2014 10:56 AM, huizhe wang wrote: > > On 2/11/2014 9:05 AM, Daniel Fuchs wrote: >> On 2/11/14 5:55 PM, Lance @ Oracle wrote: >>> Hi joe >>> >>> It looks like you changed the serialversionuid in Durationimpl, did >>> it get changed incorrectly previously? >> >> I noticed that as well, but I'm not sure it matters since Duration >> uses writeReplace to serialize itself as an instance of >> SerializedDuration. Is that correct Joe? > > Yes, it's serialized as an instance of SerializedDuration, similar to > the calendar where it's SerializedXMLGregorianCalendar. > >> >> If I'm not mistaken it also means that the serialization forms >> (before & after the change) are not compatible - but is that an >> issue for such internal classes? > > Indeed, it's probably an issue since the binary is not compatible. It > looks like this change should be rolled back. > > -Joe > >> >> best regards, >> >> -- daniel >> >>> >>> >>> Lance Andersen| Principal Member of Technical Staff | +1.781.442.2037 >>> Oracle Java Engineering >>> 1 Network Drive >>> Burlington, MA 01803 >>> Lance.Andersen at oracle.com >>> Sent from my iPad >>> >>> On Feb 10, 2014, at 6:53 PM, huizhe wang >>> wrote: >>> >>>> Hi, >>>> >>>> This is an update from Xerces for two datatype classes >>>> XMLGregorianCalendarImpl and DurationImpl. For details, please >>>> refer to: >>>> https://bugs.openjdk.java.net/browse/JDK-8033980 >>>> >>>> webrevs: http://cr.openjdk.java.net/~joehw/jdk9/8033980/webrev/ >>>> >>>> Tests included for: 1243, 1416, 1097 (Note that JDK did not have >>>> this bug) >>>> >>>> No test: >>>> 1343: remove unused code >>>> >>>> Existing tests: JAXP SQE and Unit tests passed. >>>> >>>> Thanks, >>>> Joe >> > From tristan.yan at oracle.com Wed Feb 12 05:39:19 2014 From: tristan.yan at oracle.com (Tristan Yan) Date: Wed, 12 Feb 2014 13:39:19 +0800 Subject: RFR: JDK-8032050: TEST_BUG: java/rmi/activation/Activatable/shutdownGracefully/ShutdownGracefully.java fails intermittently In-Reply-To: <52F96FA8.3010102@oracle.com> References: <52E20974.7060706@oracle.com> <52E31F53.3070702@oracle.com> <52E4959C.3020907@oracle.com> <52E8AADE.6000300@oracle.com> <52EB60A8.7070307@oracle.com> <133FA08E-5731-437D-BB1F-AD9872CBDD36@oracle.com> <52F96FA8.3010102@oracle.com> Message-ID: <0C67EF5A-7889-49C5-8079-BDA7FF4CF1F0@oracle.com> Thank you for your thorough mail. This is very educational. I took you advice and generated a new webrev for this. http://cr.openjdk.java.net/~tyan/JDK-8032050/webrev.03/ I appreciate you can review this again. Regards Tristan On Feb 11, 2014, at 8:32 AM, Stuart Marks wrote: > Hi Tristan, > > Sorry about my recurring delays. > > Several comments on these changes. > > JavaVM.java -- > > The waitFor(timeout) method is mostly ok. The new thread started at line 208 and following seems unnecessary, though. This code is reached when a timeout occurs, so throwing TimeoutException is the only thing necessary in this case. Should the code to start the new thread be removed? > > There should be a similar check for vm == null as in the waitFor() [no args] method. > > ShutdownGracefully.java -- > > The condition that's checked after calling rmid.waitFor(SHUTDOWN_TIMEOUT) is incorrect. It's testing the exit status against zero. Offhand, when and if rmid exits, it might exit with a nonzero exit status. If rmid has exited at this point, then the test should succeed. > > Instead of testing against zero, the code should catch TimeoutException, which means that rmid is still running. It's probably reasonable to catch TimeoutException, print a message, and then let the finally-block destroy the rmid. Calling TestLibrary.bomb() from within the try-block is confusing, since that method throws an exception, which is then caught by the catch-block, when then calls TestLibrary.bomb() again. > > We should also make sure to test the success case properly. If rmid.waitFor() returns in a timely fashion without throwing TimeoutException, it doesn't matter what the exit status is. (It might be worth printing it out.) At that point we know that rmid *has* exited gracefully, so we need to set rmid to null so that the finally-block doesn't attempt to destroy rmid redundantly. Some additional messages about rmid having exited and the test passing are also warranted for this case. > > Some additional cleanup can be done here as well, over and above the changes you've proposed. (This stuff is left over from earlier RMI test messes.) In order to shut down an active object, the code here spawns a new thread that sleeps for a while and then deactivates this object. This isn't necessary. (It might have been necessary in the past.) It's sufficient simply to unexport this object and then deactivate it, directly within the shutdown() method. See > > test/java/rmi/activation/ActivationSystem/unregisterGroup/UnregisterGroup.java > > for an example of this. In addition, the run() method can be removed, and the "implements Runnable" declaration can also be removed from the ShutdownGracefully test class. > > Finally, revisiting some code farther up in the test, the try-block at lines 135-140 issues a registration request that the test expects to fail. If it succeeds, the message at line 139 isn't very clear; it should say that the registration request succeeded unexpectedly. This should cause the test to fail. We still probably want to go through the waitFor(timeout) path and eventual rmid cleanup, but a flag should be set here to ensure that the test indeed fails if the registration succeeds unexpectedly, and the messages should clearly indicate that this is going on. > > A good way to test this last case is to change rmid's security manager to the normal security manager java.lang.SecurityManager instead of TestSecurityManager. > > Thanks, > > s'marks > > > > > On 2/10/14 2:59 AM, Tristan Yan wrote: >> Hi Stuart >> Could you help to review this. >> Thank you >> Tristan >> >> On Jan 31, 2014, at 4:36 PM, Tristan Yan > > wrote: >> >>> Thank you for fixing JDK-8023541. Then I leave ActivationLibrary.java for now. >>> I still did some clean up following your suggestion. >>> 1. I changed waitFor(long timeout) method, this method is going to use code >>> like Process.waitFor(timeout, unit). This can be backported to JDK7. Also >>> exitValue is kept as a return value. For making sure there is no Pipe leak, a >>> cleanup thread will start when timeout happens. >>> 2. Change in ShutdownGracefully is a little tricky. I think we should just >>> destroy JVM once exception is thrown. So I move the wait logic into try block >>> instead keep them in finally block. >>> Can you receive it again. >>> http://cr.openjdk.java.net/~tyan/JDK-8032050/webrev.02/ >>> Thank you >>> Tristan >>> >>> On 01/29/2014 03:16 PM, Stuart Marks wrote: >>>> Hi Tristan, >>>> >>>> I don't want to put the workaround into ActivationLibrary.rmidRunning() for a >>>> null return from the lookup, because this is only a workaround for an actual >>>> bug in rmid initialization. See the review I just posted for JDK-8023541. >>>> >>>> Adding JavaVM.waitFor(timeout) is something that would be useful in general, >>>> but it needs to be handled carefully. It uses the new >>>> Process.waitFor(timeout, unit) which is new in Java SE 8; this makes >>>> backporting to 7 more complicated. Not clear whether we'll do so, but I don't >>>> want to forclose the opportunity without discussion. It's also not clear how >>>> one can get the vm's exit status after JavaVM.waitFor() has returned true. >>>> With the Process API it's possible simply to call waitFor() or exitValue(). >>>> With JavaVM, a new API needs to be created, or the rule has to be established >>>> that one must call JavaVM.waitFor() to collect the exit status as well as to >>>> close the pipes from the subprocess. If JavaVM.waitFor(timeout, unit) is >>>> called without subsequently calling JavaVM.waitFor(), the pipes are leaked. >>>> >>>> In ShutdownGracefully.java, the finally-block needs to check to see if rmid >>>> is still running, and if it is, to shut it down. Simply calling >>>> waitFor(timeout, unit) isn't sufficient, because if the rmid process is still >>>> running, it will be left running. >>>> >>>> The straightforward approach would be to call ActivationLibrary.rmidRunning() >>>> to test if it's still running. Unfortunately this isn't quite right, because >>>> rmidRunning() has a timeout loop in it -- which should probably be removed. >>>> (I think there's a bug for this.) Another approach would be simply to call >>>> rmid.destroy(). This calls rmid's shutdown() method first, which is >>>> reasonable, but I'm not sure it kills the process if that fails. In any case, >>>> this already has a timeout loop waiting for the process to die, so >>>> ShutdownGracefully.java needn't use a new waitFor(timeout, unit) call. >>>> >>>> Removing the commented-out code that starts with "no longer needed" is good, >>>> and removing the ShutdownDetectThread is also good, since that's unnecessary. >>>> >>>> There are some more cleanups I have in mind here but I'd like to see a >>>> revised webrev before proceeding. >>>> >>>> Thanks, >>>> >>>> s'marks >>>> >>>> On 1/25/14 8:57 PM, Tristan Yan wrote: >>>>> Hi Stuart >>>>> Thank you for your review and suggestion. >>>>> Yes, since this failure mode is very hard to be reproduced. I guess it's >>>>> make sense to do some hack. And I also noticed in >>>>> ActivationLibrary.rmidRunning. It does try to look up ActivationSystem but >>>>> doesn't check if it's null. So I add the logic to make sure we will look up >>>>> the non-null ActivationSystem. Also I did some cleanup if you don't mind. >>>>> Add a waitFor(long timeout, TimeUnit unit) for JavaVM. Which we can have a >>>>> better waitFor control. >>>>> I appreciate you can review the code again. >>>>> http://cr.openjdk.java.net/~tyan/JDK-8032050/webrev.01/ >>>>> Thank you >>>>> Tristan >>>>> >>>>> >>>>> On 01/25/2014 10:20 AM, Stuart Marks wrote: >>>>>> On 1/23/14 10:34 PM, Tristan Yan wrote: >>>>>>> Hi All >>>>>>> Could you review the bug fix for JDK-8032050. >>>>>>> >>>>>>> http://cr.openjdk.java.net/~tyan/JDK-8032050/webrev.00/ >>>>>>> >>>>>>> Description: >>>>>>> This rare happened failure caused because when RMID starts. It don't >>>>>>> guarantee >>>>>>> sun.rmi.server.Activation.startActivation finishes. >>>>>>> Fix by adding a iterative getSystem with a 5 seconds timeout. >>>>>> >>>>>> Hi Tristan, >>>>>> >>>>>> Adding a timing/retry loop into this test isn't the correct approach for >>>>>> fixing this test. >>>>>> >>>>>> The timing loop isn't necessary because there is already a timing loop in >>>>>> RMID.start() in the RMI test library. (There's another timing loop in >>>>>> ActivationLibrary.rmidRunning() which should probably be removed.) So the >>>>>> intent of this library call is that it start rmid and wait for it to become >>>>>> ready. That logic doesn't need to be added to the test. >>>>>> >>>>>> In the bug report JDK-8032050 you had mentioned that the >>>>>> NullPointerException was suspicious. You're right! I took a look and it >>>>>> seemed like it was related to JDK-8023541, and I added a note to this >>>>>> effect to the bug report. The problem here is that rmid can come up and >>>>>> transiently return null instead of the stub of the activation system. >>>>>> That's what JDK-8023541 covers. I think that rmid itself needs to be fixed, >>>>>> though modifying the timing loop in the RMI test library to wait for rmid >>>>>> to come up *and* for the lookup to return non-null is an easy way to fix >>>>>> the problem. (Or at least cover it up.) >>>>>> >>>>>> The next step in the analysis is to determine, given that >>>>>> ActivationLibrary.getSystem can sometimes return null, whether this has >>>>>> actually caused this test failure. This is pretty easy to determine; just >>>>>> hack in a line "system = null" in the right place and run the test. I've >>>>>> done this, and the test times out and the output log is pretty much >>>>>> identical to the one in the bug report. (I recommend you try this >>>>>> yourself.) So I think it's fairly safe to say that the problem in >>>>>> JDK-8023541 has caused the failure listed in JDK-8032050. >>>>>> >>>>>> I can see a couple ways to proceed here. One way is just to close this out >>>>>> as a duplicate of JDK-8023541 since that bug caused this failure. >>>>>> >>>>>> Another is that this test could use some cleaning up. This bug certainly >>>>>> covers a failure, but the messages emitted are confusing and in some cases >>>>>> completely wrong. For example, the "rmid has shutdown" message at line 180 >>>>>> is incorrect, because in this case rmid is still running and the wait() >>>>>> call has timed out. Most of the code here can be replaced with calls to >>>>>> various bits of the RMI test library. There are a bunch of other things in >>>>>> this test that could be cleaned up as well. >>>>>> >>>>>> It's up to you how you'd like to proceed. >>>>>> >>>>>> s'marks >>>>> >>>> >>> >> From aph at redhat.com Wed Feb 12 09:59:51 2014 From: aph at redhat.com (Andrew Haley) Date: Wed, 12 Feb 2014 09:59:51 +0000 Subject: Draft JEP on enhanced volatiles In-Reply-To: References: <52F527A4.3030802@cs.oswego.edu> <52F8467B.3050004@oracle.com> <52F8C6FC.8050205@cs.oswego.edu> Message-ID: <52FB4617.1020509@redhat.com> On 02/11/2014 09:46 PM, Martin Buchholz wrote: > On Mon, Feb 10, 2014 at 3:43 PM, Mike Duigou wrote: > >> On Feb 10 2014, at 04:33 , Doug Lea
wrote: >> >>> Among the constraints on solution is that several of these >>> methods compile down to no-ops on many common platforms. >> >> This is interpreted as a benefit. However, if the are no-ops only for the >> most common platform (x86) then we are likely to see a rough path for other >> platforms. Incorrect or inconsistent usage of these methods in user code >> will, instead of having no effect, result in unexpected and possibly >> mysterious runtime failures on other platforms. > > We already have the problem that programs that succeed on x86 may fail on > other platforms with a weaker memory model. > > The JVM has been pretty good about hiding these differences thus far. It >> would be nice to see a "least common denominator" non-no-op mode as part of >> the VM implementation that provided the opportunity to test in the most >> pessimistic conditions. > > If you wanted to be able enable robust testing, create an adversarial VM > implementation (e.g. via hotspot flag) that did maximal compiler > reordering, e.g. between synchronization points write values to main memory > in reverse order. But that's a big job that's probably not going to happen? Valgrind has a race checker that can be used for this kind of thing, and it can probably be utilized for Java. I wouldn't be surprised if we found some races in HotSpot. Andrew. From Alan.Bateman at oracle.com Wed Feb 12 12:47:11 2014 From: Alan.Bateman at oracle.com (Alan Bateman) Date: Wed, 12 Feb 2014 12:47:11 +0000 Subject: Time to remove sun.misc.Service? In-Reply-To: <323EA730-D2E9-4122-8A76-0658F31C6CE7@oracle.com> References: <52FA075E.40109@oracle.com> <323EA730-D2E9-4122-8A76-0658F31C6CE7@oracle.com> Message-ID: <52FB6D4F.5020505@oracle.com> On 11/02/2014 11:44, Paul Sandoz wrote: > : > Scrub it! > > AFAICT it is not widely used (looking at the use of s.m.Service static methods on grep code there are only a handful of dependent artifacts). And the upgrade is quite easy. > I didn't see any more comments on this and I agree there isn't much of a case for keeping it around. Here's the webrev to remove it: http://cr.openjdk.java.net/~alanb/8034776/webrev/ -Alan. From sundararajan.athijegannathan at oracle.com Wed Feb 12 12:54:32 2014 From: sundararajan.athijegannathan at oracle.com (A. Sundararajan) Date: Wed, 12 Feb 2014 18:24:32 +0530 Subject: Time to remove sun.misc.Service? In-Reply-To: <52FB6D4F.5020505@oracle.com> References: <52FA075E.40109@oracle.com> <323EA730-D2E9-4122-8A76-0658F31C6CE7@oracle.com> <52FB6D4F.5020505@oracle.com> Message-ID: <52FB6F08.5090202@oracle.com> Looks good. -Sundar On Wednesday 12 February 2014 06:17 PM, Alan Bateman wrote: > On 11/02/2014 11:44, Paul Sandoz wrote: >> : >> Scrub it! >> >> AFAICT it is not widely used (looking at the use of s.m.Service >> static methods on grep code there are only a handful of dependent >> artifacts). And the upgrade is quite easy. >> > I didn't see any more comments on this and I agree there isn't much of > a case for keeping it around. > > Here's the webrev to remove it: > > http://cr.openjdk.java.net/~alanb/8034776/webrev/ > > -Alan. From michael.cui at oracle.com Wed Feb 12 12:59:37 2014 From: michael.cui at oracle.com (michael cui) Date: Wed, 12 Feb 2014 20:59:37 +0800 Subject: RFR for JDK-8028711: TEST_BUG: Tests should pass through VM options: corelibs tests In-Reply-To: <52F89D96.7000601@oracle.com> References: <52E4C2E8.1060605@oracle.com> <52E4D1F2.4030406@oracle.com> <52E5B4C3.6080405@oracle.com> <52F455FC.3020009@oracle.com> <52F4BD7B.8080006@oracle.com> <52F87906.5040702@oracle.com> <52F89D96.7000601@oracle.com> Message-ID: <52FB7039.3060907@oracle.com> On 02/10/2014 05:36 PM, Alan Bateman wrote: > A minor comment on test/java/net/URLPermission/nstest/lookup.sh is > that the resulting line length is 182 characters and this will likely > be annoying for future side-by-side views. So I think I'd split this > while you are there. Please review the updated version at http://cr.openjdk.java.net/~tyan/michael/JDK-8028711/webrev.04/ Changes includes : 1. split line if it longer than 80 characters. 2. merge the fix of JDK-8033897 3. add few missed scripts. If no further changes need to be made, I would like to find sponsor to push this fix. Michael Cui From Lance.Andersen at oracle.com Wed Feb 12 13:15:02 2014 From: Lance.Andersen at oracle.com (Lance Andersen) Date: Wed, 12 Feb 2014 08:15:02 -0500 Subject: Time to remove sun.misc.Service? In-Reply-To: <52FB6D4F.5020505@oracle.com> References: <52FA075E.40109@oracle.com> <323EA730-D2E9-4122-8A76-0658F31C6CE7@oracle.com> <52FB6D4F.5020505@oracle.com> Message-ID: +1 On Feb 12, 2014, at 7:47 AM, Alan Bateman wrote: > On 11/02/2014 11:44, Paul Sandoz wrote: >> : >> Scrub it! >> >> AFAICT it is not widely used (looking at the use of s.m.Service static methods on grep code there are only a handful of dependent artifacts). And the upgrade is quite easy. >> > I didn't see any more comments on this and I agree there isn't much of a case for keeping it around. > > Here's the webrev to remove it: > > http://cr.openjdk.java.net/~alanb/8034776/webrev/ > > -Alan. Lance Andersen| Principal Member of Technical Staff | +1.781.442.2037 Oracle Java Engineering 1 Network Drive Burlington, MA 01803 Lance.Andersen at oracle.com From Alan.Bateman at oracle.com Wed Feb 12 13:19:43 2014 From: Alan.Bateman at oracle.com (Alan Bateman) Date: Wed, 12 Feb 2014 13:19:43 +0000 Subject: 8034780: Remove used imports Message-ID: <52FB74EF.2090503@oracle.com> I need a reviewer for a trivial change to remove a tiny number of unused imports. The motive is experimental compilation of the JDK as modules rather than one big compilation unit. I've no doubt that there is other code that has unused imports but it's not the goal of this exercise to identify and remove these. The proposal patch is below. -Alan diff --git a/src/share/classes/java/lang/invoke/MethodHandle.java b/src/share/classes/java/lang/invoke/MethodHandle.java --- a/src/share/classes/java/lang/invoke/MethodHandle.java +++ b/src/share/classes/java/lang/invoke/MethodHandle.java @@ -31,8 +31,6 @@ import sun.misc.Unsafe; import static java.lang.invoke.MethodHandleStatics.*; -import java.util.logging.Level; -import java.util.logging.Logger; /** * A method handle is a typed, directly executable reference to an underlying method, diff --git a/src/share/classes/java/lang/invoke/SimpleMethodHandle.java b/src/share/classes/java/lang/invoke/SimpleMethodHandle.java --- a/src/share/classes/java/lang/invoke/SimpleMethodHandle.java +++ b/src/share/classes/java/lang/invoke/SimpleMethodHandle.java @@ -27,8 +27,6 @@ import static java.lang.invoke.LambdaForm.*; import static java.lang.invoke.MethodHandleNatives.Constants.*; -import java.util.logging.Level; -import java.util.logging.Logger; /** * A method handle whose behavior is determined only by its LambdaForm. diff --git a/src/share/classes/sun/applet/AppletViewerPanel.java b/src/share/classes/sun/applet/AppletViewerPanel.java --- a/src/share/classes/sun/applet/AppletViewerPanel.java +++ b/src/share/classes/sun/applet/AppletViewerPanel.java @@ -31,7 +31,6 @@ import java.net.MalformedURLException; import java.awt.*; import java.applet.*; -import sun.tools.jar.*; From chris.hegarty at oracle.com Wed Feb 12 13:25:27 2014 From: chris.hegarty at oracle.com (Chris Hegarty) Date: Wed, 12 Feb 2014 13:25:27 +0000 Subject: Time to remove sun.misc.Service? In-Reply-To: References: <52FA075E.40109@oracle.com> <323EA730-D2E9-4122-8A76-0658F31C6CE7@oracle.com> <52FB6D4F.5020505@oracle.com> Message-ID: <52FB7647.60007@oracle.com> ++1 -Chris. On 12/02/14 13:15, Lance Andersen wrote: > +1 > On Feb 12, 2014, at 7:47 AM, Alan Bateman wrote: > >> On 11/02/2014 11:44, Paul Sandoz wrote: >>> : >>> Scrub it! >>> >>> AFAICT it is not widely used (looking at the use of s.m.Service static methods on grep code there are only a handful of dependent artifacts). And the upgrade is quite easy. >>> >> I didn't see any more comments on this and I agree there isn't much of a case for keeping it around. >> >> Here's the webrev to remove it: >> >> http://cr.openjdk.java.net/~alanb/8034776/webrev/ >> >> -Alan. > > > > > Lance Andersen| Principal Member of Technical Staff | +1.781.442.2037 > Oracle Java Engineering > 1 Network Drive > Burlington, MA 01803 > Lance.Andersen at oracle.com > > > > From chris.hegarty at oracle.com Wed Feb 12 13:26:02 2014 From: chris.hegarty at oracle.com (Chris Hegarty) Date: Wed, 12 Feb 2014 13:26:02 +0000 Subject: 8034780: Remove used imports In-Reply-To: <52FB74EF.2090503@oracle.com> References: <52FB74EF.2090503@oracle.com> Message-ID: <52FB766A.8050802@oracle.com> Looks good to me Alan. -Chris. On 12/02/14 13:19, Alan Bateman wrote: > > I need a reviewer for a trivial change to remove a tiny number of unused > imports. The motive is experimental compilation of the JDK as modules > rather than one big compilation unit. I've no doubt that there is other > code that has unused imports but it's not the goal of this exercise to > identify and remove these. The proposal patch is below. > > -Alan > > > diff --git a/src/share/classes/java/lang/invoke/MethodHandle.java > b/src/share/classes/java/lang/invoke/MethodHandle.java > --- a/src/share/classes/java/lang/invoke/MethodHandle.java > +++ b/src/share/classes/java/lang/invoke/MethodHandle.java > @@ -31,8 +31,6 @@ > import sun.misc.Unsafe; > > import static java.lang.invoke.MethodHandleStatics.*; > -import java.util.logging.Level; > -import java.util.logging.Logger; > > /** > * A method handle is a typed, directly executable reference to an > underlying method, > diff --git a/src/share/classes/java/lang/invoke/SimpleMethodHandle.java > b/src/share/classes/java/lang/invoke/SimpleMethodHandle.java > --- a/src/share/classes/java/lang/invoke/SimpleMethodHandle.java > +++ b/src/share/classes/java/lang/invoke/SimpleMethodHandle.java > @@ -27,8 +27,6 @@ > > import static java.lang.invoke.LambdaForm.*; > import static java.lang.invoke.MethodHandleNatives.Constants.*; > -import java.util.logging.Level; > -import java.util.logging.Logger; > > /** > * A method handle whose behavior is determined only by its LambdaForm. > diff --git a/src/share/classes/sun/applet/AppletViewerPanel.java > b/src/share/classes/sun/applet/AppletViewerPanel.java > --- a/src/share/classes/sun/applet/AppletViewerPanel.java > +++ b/src/share/classes/sun/applet/AppletViewerPanel.java > @@ -31,7 +31,6 @@ > import java.net.MalformedURLException; > import java.awt.*; > import java.applet.*; > -import sun.tools.jar.*; From sundararajan.athijegannathan at oracle.com Wed Feb 12 13:25:00 2014 From: sundararajan.athijegannathan at oracle.com (A. Sundararajan) Date: Wed, 12 Feb 2014 18:55:00 +0530 Subject: 8034780: Remove used imports In-Reply-To: <52FB74EF.2090503@oracle.com> References: <52FB74EF.2090503@oracle.com> Message-ID: <52FB762C.1080803@oracle.com> +1 -Sundar On Wednesday 12 February 2014 06:49 PM, Alan Bateman wrote: > > I need a reviewer for a trivial change to remove a tiny number of > unused imports. The motive is experimental compilation of the JDK as > modules rather than one big compilation unit. I've no doubt that there > is other code that has unused imports but it's not the goal of this > exercise to identify and remove these. The proposal patch is below. > > -Alan > > > diff --git a/src/share/classes/java/lang/invoke/MethodHandle.java > b/src/share/classes/java/lang/invoke/MethodHandle.java > --- a/src/share/classes/java/lang/invoke/MethodHandle.java > +++ b/src/share/classes/java/lang/invoke/MethodHandle.java > @@ -31,8 +31,6 @@ > import sun.misc.Unsafe; > > import static java.lang.invoke.MethodHandleStatics.*; > -import java.util.logging.Level; > -import java.util.logging.Logger; > > /** > * A method handle is a typed, directly executable reference to an > underlying method, > diff --git > a/src/share/classes/java/lang/invoke/SimpleMethodHandle.java > b/src/share/classes/java/lang/invoke/SimpleMethodHandle.java > --- a/src/share/classes/java/lang/invoke/SimpleMethodHandle.java > +++ b/src/share/classes/java/lang/invoke/SimpleMethodHandle.java > @@ -27,8 +27,6 @@ > > import static java.lang.invoke.LambdaForm.*; > import static java.lang.invoke.MethodHandleNatives.Constants.*; > -import java.util.logging.Level; > -import java.util.logging.Logger; > > /** > * A method handle whose behavior is determined only by its LambdaForm. > diff --git a/src/share/classes/sun/applet/AppletViewerPanel.java > b/src/share/classes/sun/applet/AppletViewerPanel.java > --- a/src/share/classes/sun/applet/AppletViewerPanel.java > +++ b/src/share/classes/sun/applet/AppletViewerPanel.java > @@ -31,7 +31,6 @@ > import java.net.MalformedURLException; > import java.awt.*; > import java.applet.*; > -import sun.tools.jar.*; From Lance.Andersen at oracle.com Wed Feb 12 13:27:37 2014 From: Lance.Andersen at oracle.com (Lance Andersen) Date: Wed, 12 Feb 2014 08:27:37 -0500 Subject: 8034780: Remove used imports In-Reply-To: <52FB74EF.2090503@oracle.com> References: <52FB74EF.2090503@oracle.com> Message-ID: +1 On Feb 12, 2014, at 8:19 AM, Alan Bateman wrote: > > I need a reviewer for a trivial change to remove a tiny number of unused imports. The motive is experimental compilation of the JDK as modules rather than one big compilation unit. I've no doubt that there is other code that has unused imports but it's not the goal of this exercise to identify and remove these. The proposal patch is below. > > -Alan > > > diff --git a/src/share/classes/java/lang/invoke/MethodHandle.java b/src/share/classes/java/lang/invoke/MethodHandle.java > --- a/src/share/classes/java/lang/invoke/MethodHandle.java > +++ b/src/share/classes/java/lang/invoke/MethodHandle.java > @@ -31,8 +31,6 @@ > import sun.misc.Unsafe; > > import static java.lang.invoke.MethodHandleStatics.*; > -import java.util.logging.Level; > -import java.util.logging.Logger; > > /** > * A method handle is a typed, directly executable reference to an underlying method, > diff --git a/src/share/classes/java/lang/invoke/SimpleMethodHandle.java b/src/share/classes/java/lang/invoke/SimpleMethodHandle.java > --- a/src/share/classes/java/lang/invoke/SimpleMethodHandle.java > +++ b/src/share/classes/java/lang/invoke/SimpleMethodHandle.java > @@ -27,8 +27,6 @@ > > import static java.lang.invoke.LambdaForm.*; > import static java.lang.invoke.MethodHandleNatives.Constants.*; > -import java.util.logging.Level; > -import java.util.logging.Logger; > > /** > * A method handle whose behavior is determined only by its LambdaForm. > diff --git a/src/share/classes/sun/applet/AppletViewerPanel.java b/src/share/classes/sun/applet/AppletViewerPanel.java > --- a/src/share/classes/sun/applet/AppletViewerPanel.java > +++ b/src/share/classes/sun/applet/AppletViewerPanel.java > @@ -31,7 +31,6 @@ > import java.net.MalformedURLException; > import java.awt.*; > import java.applet.*; > -import sun.tools.jar.*; Lance Andersen| Principal Member of Technical Staff | +1.781.442.2037 Oracle Java Engineering 1 Network Drive Burlington, MA 01803 Lance.Andersen at oracle.com From daniel.fuchs at oracle.com Wed Feb 12 13:28:00 2014 From: daniel.fuchs at oracle.com (Daniel Fuchs) Date: Wed, 12 Feb 2014 14:28:00 +0100 Subject: 8034780: Remove used imports In-Reply-To: <52FB74EF.2090503@oracle.com> References: <52FB74EF.2090503@oracle.com> Message-ID: <52FB76E0.5030402@oracle.com> On 2/12/14 2:19 PM, Alan Bateman wrote: > > I need a reviewer for a trivial change to remove a tiny number of unused > imports. The motive is experimental compilation of the JDK as modules > rather than one big compilation unit. I've no doubt that there is other > code that has unused imports but it's not the goal of this exercise to > identify and remove these. The proposal patch is below. +1 I had noticed the unused imports in java.lang the other day when I looked at which packages were using logging - and I was intending to log an RFE about it :-) -- daniel > > -Alan > > > diff --git a/src/share/classes/java/lang/invoke/MethodHandle.java > b/src/share/classes/java/lang/invoke/MethodHandle.java > --- a/src/share/classes/java/lang/invoke/MethodHandle.java > +++ b/src/share/classes/java/lang/invoke/MethodHandle.java > @@ -31,8 +31,6 @@ > import sun.misc.Unsafe; > > import static java.lang.invoke.MethodHandleStatics.*; > -import java.util.logging.Level; > -import java.util.logging.Logger; > > /** > * A method handle is a typed, directly executable reference to an > underlying method, > diff --git a/src/share/classes/java/lang/invoke/SimpleMethodHandle.java > b/src/share/classes/java/lang/invoke/SimpleMethodHandle.java > --- a/src/share/classes/java/lang/invoke/SimpleMethodHandle.java > +++ b/src/share/classes/java/lang/invoke/SimpleMethodHandle.java > @@ -27,8 +27,6 @@ > > import static java.lang.invoke.LambdaForm.*; > import static java.lang.invoke.MethodHandleNatives.Constants.*; > -import java.util.logging.Level; > -import java.util.logging.Logger; > > /** > * A method handle whose behavior is determined only by its LambdaForm. > diff --git a/src/share/classes/sun/applet/AppletViewerPanel.java > b/src/share/classes/sun/applet/AppletViewerPanel.java > --- a/src/share/classes/sun/applet/AppletViewerPanel.java > +++ b/src/share/classes/sun/applet/AppletViewerPanel.java > @@ -31,7 +31,6 @@ > import java.net.MalformedURLException; > import java.awt.*; > import java.applet.*; > -import sun.tools.jar.*; From michael.cui at oracle.com Wed Feb 12 13:34:32 2014 From: michael.cui at oracle.com (michael cui) Date: Wed, 12 Feb 2014 21:34:32 +0800 Subject: RFR for JDK-8031563: TEST_BUG: java/nio/channels/Selector/ChangingInterests.java failed once In-Reply-To: References: <52FB60AB.4070708@oracle.com> Message-ID: <52FB7868.3080907@oracle.com> Hi, Please review two suggested fixes for https://bugs.openjdk.java.net/browse/JDK-8031563 Root cause: selectNow is called right after 50 milliseconds sleep in write1 method which may not be sufficient to make the socket channel readable in some situation. Since selectNow called only once and verification based on its result will fail if readable event wasn't ready at that time. Suggested fix A : simply increase the wait time from 50 to 1000 in write1() method. webrev link : http://cr.openjdk.java.net/~tyan/michael/JDK-8031563/webrev_v1/ Comments from Shura : Simply increasing the timeout will not fix the problem forever. Is it possible to re-write the tests so that the channel is waited to be readable instead? Suggested fix B : In testChange() , replacing original [select - verify] block by [wait - select - until verify or timeout] block. webrev link : http://cr.openjdk.java.net/~tyan/michael/JDK-8031563/webrev_v2/ Explanation : Test will fail intermittently when interest setting are (OP_READ | OP_WRITE). The goal is waiting until interested event coming or timeout. However, interest setting is OP_READ | OP_WRITE, so the blocking version api select() / select(long timeout) will return immediately as long as one of interested type event is ready (see selector class api). Moreover, SocketChannel is always writable if its internal buffer doesn't full. So all selection api will return immediately when interest setting contains OP_WRITE. So based on selector's api explanation and my testing, it is impossible to implement [wait - select - until verify or timeout] logic without using loop. The verification logic is adjusted slightly for while - loop. Another thing I would like to mention is the selector's selected-key set will be updated only by selection or manually remove. In other words, the selector.selectedKeys() will only reflect io events of last select operation even if a new interested event has arrived right after last selection. Comments from Shura : I have concern about two areas which might make your version and original version not equivalent. one is code changes in verification logic are different. the other is old version called selectNow() only once, your version replace it by calling it repeatedly in worst case. Both fixes were tested on 4 platforms (win, linux, mac, solaris)for 2000 runs without seeing any error. Any suggestion? Thanks, Michael Cui From weijun.wang at oracle.com Wed Feb 12 15:05:40 2014 From: weijun.wang at oracle.com (Wang Weijun) Date: Wed, 12 Feb 2014 23:05:40 +0800 Subject: methods that throws checked exceptions in a lambda block Message-ID: <68381B76-8459-4352-BDF1-CF37B4823517@oracle.com> This line does not compile Files.list(Paths.get("/secret")).forEach(x -> Files.write(x, new byte[Files.size(x)]); because Files.write() and Files.size() throw IOE. Then what's the best way to make it work? (I find some old threads on this on lambda-dev. Maybe we have a recommended way now?) Thanks Max From chris.hegarty at oracle.com Wed Feb 12 15:08:12 2014 From: chris.hegarty at oracle.com (Chris Hegarty) Date: Wed, 12 Feb 2014 15:08:12 +0000 Subject: A hole in the serialization spec In-Reply-To: <52F8F21C.80205@redhat.com> References: <52F4F6BB.2010207@redhat.com> <52F8E807.2090403@oracle.com> <52F8F21C.80205@redhat.com> Message-ID: <52FB8E5C.9020502@oracle.com> David, others? I'm afraid I'm still not clear what is mean by: " ... undefined in cases where the ObjectInputStream cannot resolve the class which defined the writeObject method in question." This does not seem directly related to the issue we are discussing ( whether to invoke default read/write object ). -Chris. On 10/02/14 15:37, David M. Lloyd wrote: > I agree that it's a problem; however it's also clear that there are many > classes in the wild which have this problem. It would be nice if the > behavior could _become_ defined *somehow* though. I can see at least > four options: > > 1) do nothing :( > 2) start throwing (or writing) an exception in write/readObject when > stream ops are performed without reading fields (maybe can be disabled > with a sys prop or something) > 3) leave fields cleared and risk protocol issues > 4) silently start reading/writing empty field information (risks > protocol issues) > > Maybe there are better options I'm not thinking of. > > On 02/10/2014 08:53 AM, Chris Hegarty wrote: >> David, >> >> " ... undefined in cases where the ObjectInputStream cannot resolve the >> class which defined the writeObject method in question." >> >> I'm not clear as to what this statement is about? >> >> I'm sure you already know this, and maybe in your environment do not >> care much about it, but having a read/writeObject not invoke the >> appropriate default read/write Object/Fields method is a serious >> impediment to evolving the serial form ( in a compatible way ). For >> example, if your class has no serializable fields in one revision, but >> adds a serializable field(s) in a subsequent revision. This could lead >> to a StreamCorruptedException, or some other undefined behavior. >> >> The OpenJDK sources do seem to be quite tolerant of this situation. I'm >> not entirely sure if this is a good or a bad thing. That said, I don't >> think we want to encourage this kind of behavior. >> >> -Chris. >> >> On 07/02/14 15:07, David M. Lloyd wrote: >>> Since the topic of serialization has come up recently, I'll take it as >>> an excuse to bring up a problem that I've run into a couple of times >>> with the serialization specification, which has resulted in user >>> problems. >>> >>> If you read section 2.3 [1] of the specification, it says: >>> >>> "The class's writeObject method, if implemented, is responsible for >>> saving the state of the class. Either ObjectOutputStream's >>> defaultWriteObject or writeFields method must be called once (and only >>> once) before writing any optional data that will be needed by the >>> corresponding readObject method to restore the state of the object; even >>> if no optional data is written, defaultWriteObject or writeFields must >>> still be invoked once. If defaultWriteObject or writeFields is not >>> invoked once prior to the writing of optional data (if any), then the >>> behavior of instance deserialization is undefined in cases where the >>> ObjectInputStream cannot resolve the class which defined the writeObject >>> method in question." >>> >>> If you go to section 3.4 [2] of the specification, it reads: >>> >>> "The readObject method of the class, if implemented, is responsible for >>> restoring the state of the class. The values of every field of the >>> object whether transient or not, static or not are set to the default >>> value for the fields type. Either ObjectInputStream's defaultReadObject >>> or readFields method must be called once (and only once) before reading >>> any optional data written by the corresponding writeObject method; even >>> if no optional data is read, defaultReadObject or readFields must still >>> be invoked once." >>> >>> Now the problem: there are many classes in the wild which nevertheless >>> do not write/read fields. We cause an exception in such cases rather >>> than make up some undefined behavior. What I'm wondering is, is there >>> some sensible behavior that could be specified for this case? The >>> Oracle JDK seems to simply leave fields uninitialized in this case, >>> maybe that can be a specified behavior? >>> >>> [1] >>> http://docs.oracle.com/javase/7/docs/platform/serialization/spec/output.html#861 >>> >>> >>> >>> [2] >>> http://docs.oracle.com/javase/7/docs/platform/serialization/spec/input.html#2971 >>> >>> >>> >>> > > From joe.darcy at oracle.com Wed Feb 12 15:14:26 2014 From: joe.darcy at oracle.com (Joe Darcy) Date: Wed, 12 Feb 2014 07:14:26 -0800 Subject: Time to remove sun.misc.Service? In-Reply-To: <52FB6D4F.5020505@oracle.com> References: <52FA075E.40109@oracle.com> <323EA730-D2E9-4122-8A76-0658F31C6CE7@oracle.com> <52FB6D4F.5020505@oracle.com> Message-ID: <52FB8FD2.9010700@oracle.com> On 02/12/2014 04:47 AM, Alan Bateman wrote: > On 11/02/2014 11:44, Paul Sandoz wrote: >> : >> Scrub it! >> >> AFAICT it is not widely used (looking at the use of s.m.Service >> static methods on grep code there are only a handful of dependent >> artifacts). And the upgrade is quite easy. >> > I didn't see any more comments on this and I agree there isn't much of > a case for keeping it around. > > Here's the webrev to remove it: > > http://cr.openjdk.java.net/~alanb/8034776/webrev/ > > -Alan. sun.misc.Service be gone! -Joe From david.lloyd at redhat.com Wed Feb 12 15:24:56 2014 From: david.lloyd at redhat.com (David M. Lloyd) Date: Wed, 12 Feb 2014 09:24:56 -0600 Subject: A hole in the serialization spec In-Reply-To: <52FB8E5C.9020502@oracle.com> References: <52F4F6BB.2010207@redhat.com> <52F8E807.2090403@oracle.com> <52F8F21C.80205@redhat.com> <52FB8E5C.9020502@oracle.com> Message-ID: <52FB9248.6010703@redhat.com> That's a quote from the serialization spec. I take it to mean, "Don't write fields and everything might go to hell". In practice, if the reading side doesn't read fields, things end up more or less OK, as evidenced by various classes in the wild. But it's not hard to imagine a scenario in which a class change could cause protocol corruption. I think the specifics of the quote relate to this kind of class change; in particular, if a class is deleted from the hierarchy on the read side, and that class corresponds to the class that had the misbehaving writeObject, I suspect that things will break at that point as the read side will probably try to consume and discard the field information for that class, which will be missing (it will start reading the next class' fields instead I think). On 02/12/2014 09:08 AM, Chris Hegarty wrote: > David, others? > > I'm afraid I'm still not clear what is mean by: > " ... undefined in cases where the ObjectInputStream cannot resolve > the class which defined the writeObject method in question." > > This does not seem directly related to the issue we are discussing ( > whether to invoke default read/write object ). > > -Chris. > > On 10/02/14 15:37, David M. Lloyd wrote: >> I agree that it's a problem; however it's also clear that there are many >> classes in the wild which have this problem. It would be nice if the >> behavior could _become_ defined *somehow* though. I can see at least >> four options: >> >> 1) do nothing :( >> 2) start throwing (or writing) an exception in write/readObject when >> stream ops are performed without reading fields (maybe can be disabled >> with a sys prop or something) >> 3) leave fields cleared and risk protocol issues >> 4) silently start reading/writing empty field information (risks >> protocol issues) >> >> Maybe there are better options I'm not thinking of. >> >> On 02/10/2014 08:53 AM, Chris Hegarty wrote: >>> David, >>> >>> " ... undefined in cases where the ObjectInputStream cannot resolve the >>> class which defined the writeObject method in question." >>> >>> I'm not clear as to what this statement is about? >>> >>> I'm sure you already know this, and maybe in your environment do not >>> care much about it, but having a read/writeObject not invoke the >>> appropriate default read/write Object/Fields method is a serious >>> impediment to evolving the serial form ( in a compatible way ). For >>> example, if your class has no serializable fields in one revision, but >>> adds a serializable field(s) in a subsequent revision. This could lead >>> to a StreamCorruptedException, or some other undefined behavior. >>> >>> The OpenJDK sources do seem to be quite tolerant of this situation. I'm >>> not entirely sure if this is a good or a bad thing. That said, I don't >>> think we want to encourage this kind of behavior. >>> >>> -Chris. >>> >>> On 07/02/14 15:07, David M. Lloyd wrote: >>>> Since the topic of serialization has come up recently, I'll take it as >>>> an excuse to bring up a problem that I've run into a couple of times >>>> with the serialization specification, which has resulted in user >>>> problems. >>>> >>>> If you read section 2.3 [1] of the specification, it says: >>>> >>>> "The class's writeObject method, if implemented, is responsible for >>>> saving the state of the class. Either ObjectOutputStream's >>>> defaultWriteObject or writeFields method must be called once (and only >>>> once) before writing any optional data that will be needed by the >>>> corresponding readObject method to restore the state of the object; >>>> even >>>> if no optional data is written, defaultWriteObject or writeFields must >>>> still be invoked once. If defaultWriteObject or writeFields is not >>>> invoked once prior to the writing of optional data (if any), then the >>>> behavior of instance deserialization is undefined in cases where the >>>> ObjectInputStream cannot resolve the class which defined the >>>> writeObject >>>> method in question." >>>> >>>> If you go to section 3.4 [2] of the specification, it reads: >>>> >>>> "The readObject method of the class, if implemented, is responsible for >>>> restoring the state of the class. The values of every field of the >>>> object whether transient or not, static or not are set to the default >>>> value for the fields type. Either ObjectInputStream's defaultReadObject >>>> or readFields method must be called once (and only once) before reading >>>> any optional data written by the corresponding writeObject method; even >>>> if no optional data is read, defaultReadObject or readFields must still >>>> be invoked once." >>>> >>>> Now the problem: there are many classes in the wild which nevertheless >>>> do not write/read fields. We cause an exception in such cases rather >>>> than make up some undefined behavior. What I'm wondering is, is there >>>> some sensible behavior that could be specified for this case? The >>>> Oracle JDK seems to simply leave fields uninitialized in this case, >>>> maybe that can be a specified behavior? >>>> >>>> [1] >>>> http://docs.oracle.com/javase/7/docs/platform/serialization/spec/output.html#861 >>>> >>>> >>>> >>>> >>>> [2] >>>> http://docs.oracle.com/javase/7/docs/platform/serialization/spec/input.html#2971 >>>> >>>> >>>> >>>> >>>> >> >> -- - DML From mandy.chung at oracle.com Wed Feb 12 16:07:17 2014 From: mandy.chung at oracle.com (Mandy Chung) Date: Wed, 12 Feb 2014 08:07:17 -0800 Subject: Time to remove sun.misc.Service? In-Reply-To: <52FB6D4F.5020505@oracle.com> References: <52FA075E.40109@oracle.com> <323EA730-D2E9-4122-8A76-0658F31C6CE7@oracle.com> <52FB6D4F.5020505@oracle.com> Message-ID: <52FB9C35.2060109@oracle.com> On 2/12/2014 4:47 AM, Alan Bateman wrote: > On 11/02/2014 11:44, Paul Sandoz wrote: >> : >> Scrub it! >> >> AFAICT it is not widely used (looking at the use of s.m.Service >> static methods on grep code there are only a handful of dependent >> artifacts). And the upgrade is quite easy. >> > I didn't see any more comments on this and I agree there isn't much of > a case for keeping it around. > > Here's the webrev to remove it: > > http://cr.openjdk.java.net/~alanb/8034776/webrev/ Thumbs up. Mandy From henry.jen at oracle.com Wed Feb 12 16:36:04 2014 From: henry.jen at oracle.com (Henry Jen) Date: Wed, 12 Feb 2014 08:36:04 -0800 Subject: methods that throws checked exceptions in a lambda block In-Reply-To: <68381B76-8459-4352-BDF1-CF37B4823517@oracle.com> References: <68381B76-8459-4352-BDF1-CF37B4823517@oracle.com> Message-ID: <52FBA2F4.7050302@oracle.com> On 02/12/2014 07:05 AM, Wang Weijun wrote: > This line does not compile > > Files.list(Paths.get("/secret")).forEach(x -> Files.write(x, new byte[Files.size(x)]); > > because Files.write() and Files.size() throw IOE. Then what's the best way to make it work? > > (I find some old threads on this on lambda-dev. Maybe we have a recommended way now?) > I am not sure there is a "recommended" way. Depends on how you need to handle those exceptions, I'll put the lambda in a private function to take care those exception, and surpress them in a UncheckedIOException. Cheers, Henry From paul.sandoz at oracle.com Wed Feb 12 16:48:40 2014 From: paul.sandoz at oracle.com (Paul Sandoz) Date: Wed, 12 Feb 2014 17:48:40 +0100 Subject: methods that throws checked exceptions in a lambda block In-Reply-To: <68381B76-8459-4352-BDF1-CF37B4823517@oracle.com> References: <68381B76-8459-4352-BDF1-CF37B4823517@oracle.com> Message-ID: <3C8A8743-42F7-4A66-BE32-6A089237E259@oracle.com> On Feb 12, 2014, at 4:05 PM, Wang Weijun wrote: > This line does not compile > > Files.list(Paths.get("/secret")).forEach(x -> Files.write(x, new byte[Files.size(x)]); > > because Files.write() and Files.size() throw IOE. Then what's the best way to make it work? > It depends... Sometimes i will shuffle things under a method (static one if there are no captures and use a method ref). If i get *really* fed up i have been know to write stuff like the following: private static interface S { T apply() throws E; } private static T r(S s) { try { return s.apply(); } catch (IOException t) { throw new UncheckedIOException(t); } } Paul. > (I find some old threads on this on lambda-dev. Maybe we have a recommended way now?) > From Alan.Bateman at oracle.com Wed Feb 12 17:22:14 2014 From: Alan.Bateman at oracle.com (Alan Bateman) Date: Wed, 12 Feb 2014 17:22:14 +0000 Subject: methods that throws checked exceptions in a lambda block In-Reply-To: <52FBA2F4.7050302@oracle.com> References: <68381B76-8459-4352-BDF1-CF37B4823517@oracle.com> <52FBA2F4.7050302@oracle.com> Message-ID: <52FBADC6.4020105@oracle.com> On 12/02/2014 16:36, Henry Jen wrote: > > > I am not sure there is a "recommended" way. Depends on how you need to > handle those exceptions, I'll put the lambda in a private function to > take care those exception, and surpress them in a UncheckedIOException. Yes, I think it depends on whether he wants to recover and continue to attempt to zero the remaining files or not. -Alan From mandy.chung at oracle.com Wed Feb 12 17:46:58 2014 From: mandy.chung at oracle.com (Mandy Chung) Date: Wed, 12 Feb 2014 09:46:58 -0800 Subject: Review request for JDK-8030010: cleanup native code warnings Message-ID: <52FBB392.4010505@oracle.com> This patch cleans up a few trivial native warnings (mainly remove local unreferenced variable) https://bugs.openjdk.java.net/browse/JDK-8030010 Webrev at: http://cr.openjdk.java.net/~mchung/jdk9/webrevs/8030010/webrev.00/ This patch was contributed by Francis Andre [1] and I fixed a couple other trivial ones. Mandy [1] http://mail.openjdk.java.net/pipermail/build-dev/2013-December/011332.html From Lance.Andersen at oracle.com Wed Feb 12 17:56:44 2014 From: Lance.Andersen at oracle.com (Lance Andersen - Oracle) Date: Wed, 12 Feb 2014 12:56:44 -0500 Subject: Review request for JDK-8030010: cleanup native code warnings In-Reply-To: <52FBB392.4010505@oracle.com> References: <52FBB392.4010505@oracle.com> Message-ID: <95636D0D-CF42-45D7-8E90-2A1959AC62F3@oracle.com> Hi Mandy This looks OK to me On Feb 12, 2014, at 12:46 PM, Mandy Chung wrote: > This patch cleans up a few trivial native warnings (mainly remove local unreferenced variable) > https://bugs.openjdk.java.net/browse/JDK-8030010 > > Webrev at: > http://cr.openjdk.java.net/~mchung/jdk9/webrevs/8030010/webrev.00/ > > This patch was contributed by Francis Andre [1] and I fixed a couple other trivial ones. > > Mandy > [1] http://mail.openjdk.java.net/pipermail/build-dev/2013-December/011332.html -------------- next part -------------- Lance Andersen| Principal Member of Technical Staff | +1.781.442.2037 Oracle Java Engineering 1 Network Drive Burlington, MA 01803 Lance.Andersen at oracle.com From roger.riggs at oracle.com Wed Feb 12 18:10:31 2014 From: roger.riggs at oracle.com (roger riggs) Date: Wed, 12 Feb 2014 13:10:31 -0500 Subject: RFR: (8031737) CHECK_NULL and CHECK_EXCEPTION macros cleanup In-Reply-To: <52FAB577.3070205@oracle.com> References: <52ED36FD.5070507@oracle.com> <1243D3E7-C384-4AA8-80CA-9AA4E053F056@oracle.com> <52F00EAC.8000900@oracle.com> <52FA7429.5070806@oracle.com> <52FA768A.70005@oracle.com> <52FA80A6.2040509@oracle.com> <52FA87E2.9080009@oracle.com> <52FA98A8.7090104@oracle.com> <52FA9AA2.6090005@oracle.com> <52FA9F82.90903@oracle.com> <52FAA211.2070800@oracle.com> <52FAA3F4.6040702@oracle.com> <52FAB577.3070205@oracle.com> Message-ID: <52FBB917.4080102@oracle.com> Hi Phil, The changes look fine. (Not a "Reviewer"). Thanks for doing the backport, Roger On 2/11/2014 6:42 PM, Phil Race wrote: > Here's a JDk8u webrev : -http://cr.openjdk.java.net/~prr/8031737.8u/ > > -phil. > > On 2/11/14 2:28 PM, Phil Race wrote: >> So since hg export/import doesn't apply cleanly and the dependency >> chain seems, long and in order to have some consistency across the >> releases, >> I think I should prepare a webrev which essentially backports 8031737 >> including its small changes to Version.c, if only because otherwise >> I'd have to have a new bug ID that would not be forwarded ported >> (one source of confusion) or even worse re-use 8031737 but not fully >> implement it >> >> Agreed ? >> >> -phil. >> >> On 2/11/2014 2:20 PM, roger riggs wrote: >>> Hi Phil, >>> >>> >>> On 2/11/2014 5:09 PM, Phil Race wrote: >>>> Are we talking about the same changesets ? >>>> a09982d91fab/8030993 has no change to the macros >>> right (I didn't think this was topic of this conversation) >>>> >>>> fb89dc4fe8da/8031737 is the one that reimplemented the macros >>>> and is the version I'd want. Its the last 'edit' of those macros in >>>> that file. >>> yes, >>>> >>>> c58c6b0fbe34/8030875 is the original addition of these :- >>> Yes. >>> >>> Roger >>> >>>> >>>> ... >>>> >>>> changeset: 9229:fb89dc4fe8da >>>> user: rriggs >>>> date: Mon Feb 03 16:58:02 2014 -0500 >>>> summary: 8031737: CHECK_NULL and CHECK_EXCEPTION macros cleanup >>>> >>>> changeset: 9051:c58c6b0fbe34 >>>> user: rriggs >>>> date: Fri Jan 10 10:45:56 2014 -0500 >>>> summary: 8030875: Macros for checking and returning on exceptions >>>> >>>> >>>> ... >>>> >>>> -phil. >>>> >>>> On 2/11/14 1:48 PM, roger riggs wrote: >>>>> Hi Phil, >>>>> >>>>> The later changeset picked up the recommended style of >>>>> implementing the macros >>>>> but I don't think it was substantive. You can probably do without >>>>> it. >>>>> >>>>> Version.c had some changes in a different changeset to address >>>>> the omission of checking for exceptions after some JNI calls. >>>>> >>>>> Roger >>>>> >>>>> On 2/11/2014 4:39 PM, Phil Race wrote: >>>>>> Roger, >>>>>> >>>>>> That later one seems to be using the macros. I don't see any >>>>>> update to the macros. >>>>>> So I'm not sure why I'm need it .. since I'm not using those >>>>>> calls and neither >>>>>> are the macros. >>>>>> >>>>>> -phil. >>>>>> >>>>>> On 2/11/14 12:28 PM, roger riggs wrote: >>>>>>> Hi Phil, >>>>>>> >>>>>>> Yes, it ended up in two change sets in jdk 9, you should take >>>>>>> both to be up to date. >>>>>>> >>>>>>> changeset: 9245:a09982d91fab >>>>>>> user: rriggs >>>>>>> date: Wed Feb 05 10:59:53 2014 -0500 >>>>>>> files: src/share/native/common/jni_util.c >>>>>>> description: >>>>>>> 8030993: Check jdk/src/share/native/common/jni_util.c for JNI >>>>>>> pending exceptions >>>>>>> >>>>>>> >>>>>>> changeset: 9229:fb89dc4fe8da >>>>>>> date: Mon Feb 03 16:58:02 2014 -0500 >>>>>>> files: src/share/native/common/jni_util.h >>>>>>> src/share/native/sun/misc/Version.c >>>>>>> interrupted! >>>>>>> description: >>>>>>> 8031737: CHECK_NULL and CHECK_EXCEPTION macros cleanup >>>>>>> >>>>>>> Thanks, Roger >>>>>>> >>>>>>> >>>>>>> On 2/11/2014 2:57 PM, Phil Race wrote: >>>>>>>> Roger, >>>>>>>> >>>>>>>> Yes, I can do that. >>>>>>>> >>>>>>>> I see here >>>>>>>> http://cr.openjdk.java.net/~rriggs/webrev-check-cleanup-8031737/ that >>>>>>>> >>>>>>>> 1) There was a previous version of these macros. >>>>>>>> Looks like no need to worry about that I just need the latest >>>>>>>> version. >>>>>>>> 2) There was also a change to Version.c. I can include that if >>>>>>>> you think it >>>>>>>> appropriate .. or omit it if you think its not essential. >>>>>>>> >>>>>>>> -phil. >>>>>>>> >>>>>>>> On 2/11/2014 11:14 AM, roger riggs wrote: >>>>>>>>> Hi Phil, >>>>>>>>> >>>>>>>>> I see your point, there is nothing in the changes unique to 9. >>>>>>>>> Do you want to take care of the back point? >>>>>>>>> >>>>>>>>> Roger >>>>>>>>> >>>>>>>>> On 2/11/2014 2:04 PM, Phil Race wrote: >>>>>>>>>> Roger, >>>>>>>>>> >>>>>>>>>> Why not JDK 8u ? I've got a lot of changes that utilise >>>>>>>>>> these that will >>>>>>>>>> backport cleanly to JDK 8u only if 8u includes these macros. >>>>>>>>>> And since >>>>>>>>>> the changes are all over the place I don't fancy copy/pasting >>>>>>>>>> them >>>>>>>>>> everywhere. I suspect I am not the only one who would like >>>>>>>>>> these in 8u .. >>>>>>>>>> >>>>>>>>>> -phil. >>>>>>>>>> >>>>>>>>>> >>>>>>>>>> On 02/03/2014 01:48 PM, roger riggs wrote: >>>>>>>>>>> Hi Lance, >>>>>>>>>>> >>>>>>>>>>> The convenience macros are only intended for JDK 9. >>>>>>>>>>> >>>>>>>>>>> Roger >>>>>>>>>>> >>>>>>>>>>> On 2/1/2014 1:58 PM, Lance @ Oracle wrote: >>>>>>>>>>>> Looks fine >>>>>>>>>>>> >>>>>>>>>>>> Which releases are you think of including this in if any >>>>>>>>>>>> besides 9? >>>>>>>>>>>> >>>>>>>>>>>> >>>>>>>>>>>> Lance >>>>>>>>>>>> Andersen| Principal Member of Technical Staff | >>>>>>>>>>>> +1.781.442.2037 >>>>>>>>>>>> Oracle Java Engineering >>>>>>>>>>>> 1 Network Drive >>>>>>>>>>>> Burlington, MA 01803 >>>>>>>>>>>> Lance.Andersen at oracle.com >>>>>>>>>>>> Sent from my iPad >>>>>>>>>>>> >>>>>>>>>>>> On Feb 1, 2014, at 1:03 PM, roger riggs >>>>>>>>>>>> > >>>>>>>>>>>> wrote: >>>>>>>>>>> >>>>>>>>>> >>>>>>>>> >>>>>>>> >>>>>>> >>>>>> >>>>> >>>> >>> >> > From Alan.Bateman at oracle.com Wed Feb 12 18:16:22 2014 From: Alan.Bateman at oracle.com (Alan Bateman) Date: Wed, 12 Feb 2014 18:16:22 +0000 Subject: Review request for JDK-8030010: cleanup native code warnings In-Reply-To: <52FBB392.4010505@oracle.com> References: <52FBB392.4010505@oracle.com> Message-ID: <52FBBA76.40700@oracle.com> On 12/02/2014 17:46, Mandy Chung wrote: > This patch cleans up a few trivial native warnings (mainly remove > local unreferenced variable) > https://bugs.openjdk.java.net/browse/JDK-8030010 > > Webrev at: > http://cr.openjdk.java.net/~mchung/jdk9/webrevs/8030010/webrev.00/ > > This patch was contributed by Francis Andre [1] and I fixed a couple > other trivial ones. This looks good and nice to see the native code warnings getting a bit attention. -Alan. From mike.duigou at oracle.com Wed Feb 12 18:21:22 2014 From: mike.duigou at oracle.com (Mike Duigou) Date: Wed, 12 Feb 2014 10:21:22 -0800 Subject: RFR: (8031737) CHECK_NULL and CHECK_EXCEPTION macros cleanup In-Reply-To: <52FAB577.3070205@oracle.com> References: <52ED36FD.5070507@oracle.com> <1243D3E7-C384-4AA8-80CA-9AA4E053F056@oracle.com> <52F00EAC.8000900@oracle.com> <52FA7429.5070806@oracle.com> <52FA768A.70005@oracle.com> <52FA80A6.2040509@oracle.com> <52FA87E2.9080009@oracle.com> <52FA98A8.7090104@oracle.com> <52FA9AA2.6090005@oracle.com> <52FA9F82.90903@oracle.com> <52FAA211.2070800@oracle.com> <52FAA3F4.6040702@oracle.com> <52FAB577.3070205@oracle.com> Message-ID: This looks fine. On Feb 11 2014, at 15:42 , Phil Race wrote: > Here's a JDk8u webrev : -http://cr.openjdk.java.net/~prr/8031737.8u/ > > -phil. > > On 2/11/14 2:28 PM, Phil Race wrote: >> So since hg export/import doesn't apply cleanly and the dependency >> chain seems, long and in order to have some consistency across the releases, >> I think I should prepare a webrev which essentially backports 8031737 >> including its small changes to Version.c, if only because otherwise >> I'd have to have a new bug ID that would not be forwarded ported >> (one source of confusion) or even worse re-use 8031737 but not fully implement it >> >> Agreed ? >> >> -phil. >> >> On 2/11/2014 2:20 PM, roger riggs wrote: >>> Hi Phil, >>> >>> >>> On 2/11/2014 5:09 PM, Phil Race wrote: >>>> Are we talking about the same changesets ? >>>> a09982d91fab/8030993 has no change to the macros >>> right (I didn't think this was topic of this conversation) >>>> >>>> fb89dc4fe8da/8031737 is the one that reimplemented the macros >>>> and is the version I'd want. Its the last 'edit' of those macros in that file. >>> yes, >>>> >>>> c58c6b0fbe34/8030875 is the original addition of these :- >>> Yes. >>> >>> Roger >>> >>>> >>>> ... >>>> >>>> changeset: 9229:fb89dc4fe8da >>>> user: rriggs >>>> date: Mon Feb 03 16:58:02 2014 -0500 >>>> summary: 8031737: CHECK_NULL and CHECK_EXCEPTION macros cleanup >>>> >>>> changeset: 9051:c58c6b0fbe34 >>>> user: rriggs >>>> date: Fri Jan 10 10:45:56 2014 -0500 >>>> summary: 8030875: Macros for checking and returning on exceptions >>>> >>>> >>>> ... >>>> >>>> -phil. >>>> >>>> On 2/11/14 1:48 PM, roger riggs wrote: >>>>> Hi Phil, >>>>> >>>>> The later changeset picked up the recommended style of implementing the macros >>>>> but I don't think it was substantive. You can probably do without it. >>>>> >>>>> Version.c had some changes in a different changeset to address >>>>> the omission of checking for exceptions after some JNI calls. >>>>> >>>>> Roger >>>>> >>>>> On 2/11/2014 4:39 PM, Phil Race wrote: >>>>>> Roger, >>>>>> >>>>>> That later one seems to be using the macros. I don't see any update to the macros. >>>>>> So I'm not sure why I'm need it .. since I'm not using those calls and neither >>>>>> are the macros. >>>>>> >>>>>> -phil. >>>>>> >>>>>> On 2/11/14 12:28 PM, roger riggs wrote: >>>>>>> Hi Phil, >>>>>>> >>>>>>> Yes, it ended up in two change sets in jdk 9, you should take both to be up to date. >>>>>>> >>>>>>> changeset: 9245:a09982d91fab >>>>>>> user: rriggs >>>>>>> date: Wed Feb 05 10:59:53 2014 -0500 >>>>>>> files: src/share/native/common/jni_util.c >>>>>>> description: >>>>>>> 8030993: Check jdk/src/share/native/common/jni_util.c for JNI pending exceptions >>>>>>> >>>>>>> >>>>>>> changeset: 9229:fb89dc4fe8da >>>>>>> date: Mon Feb 03 16:58:02 2014 -0500 >>>>>>> files: src/share/native/common/jni_util.h src/share/native/sun/misc/Version.c >>>>>>> interrupted! >>>>>>> description: >>>>>>> 8031737: CHECK_NULL and CHECK_EXCEPTION macros cleanup >>>>>>> >>>>>>> Thanks, Roger >>>>>>> >>>>>>> >>>>>>> On 2/11/2014 2:57 PM, Phil Race wrote: >>>>>>>> Roger, >>>>>>>> >>>>>>>> Yes, I can do that. >>>>>>>> >>>>>>>> I see here http://cr.openjdk.java.net/~rriggs/webrev-check-cleanup-8031737/ that >>>>>>>> 1) There was a previous version of these macros. >>>>>>>> Looks like no need to worry about that I just need the latest version. >>>>>>>> 2) There was also a change to Version.c. I can include that if you think it >>>>>>>> appropriate .. or omit it if you think its not essential. >>>>>>>> >>>>>>>> -phil. >>>>>>>> >>>>>>>> On 2/11/2014 11:14 AM, roger riggs wrote: >>>>>>>>> Hi Phil, >>>>>>>>> >>>>>>>>> I see your point, there is nothing in the changes unique to 9. >>>>>>>>> Do you want to take care of the back point? >>>>>>>>> >>>>>>>>> Roger >>>>>>>>> >>>>>>>>> On 2/11/2014 2:04 PM, Phil Race wrote: >>>>>>>>>> Roger, >>>>>>>>>> >>>>>>>>>> Why not JDK 8u ? I've got a lot of changes that utilise these that will >>>>>>>>>> backport cleanly to JDK 8u only if 8u includes these macros. And since >>>>>>>>>> the changes are all over the place I don't fancy copy/pasting them >>>>>>>>>> everywhere. I suspect I am not the only one who would like these in 8u .. >>>>>>>>>> >>>>>>>>>> -phil. >>>>>>>>>> >>>>>>>>>> >>>>>>>>>> On 02/03/2014 01:48 PM, roger riggs wrote: >>>>>>>>>>> Hi Lance, >>>>>>>>>>> >>>>>>>>>>> The convenience macros are only intended for JDK 9. >>>>>>>>>>> >>>>>>>>>>> Roger >>>>>>>>>>> >>>>>>>>>>> On 2/1/2014 1:58 PM, Lance @ Oracle wrote: >>>>>>>>>>>> Looks fine >>>>>>>>>>>> >>>>>>>>>>>> Which releases are you think of including this in if any besides 9? >>>>>>>>>>>> >>>>>>>>>>>> >>>>>>>>>>>> Lance Andersen| Principal Member of Technical Staff | +1.781.442.2037 >>>>>>>>>>>> Oracle Java Engineering >>>>>>>>>>>> 1 Network Drive >>>>>>>>>>>> Burlington, MA 01803 >>>>>>>>>>>> Lance.Andersen at oracle.com >>>>>>>>>>>> Sent from my iPad >>>>>>>>>>>> >>>>>>>>>>>> On Feb 1, 2014, at 1:03 PM, roger riggs > wrote: >>>>>>>>>>> >>>>>>>>>> >>>>>>>>> >>>>>>>> >>>>>>> >>>>>> >>>>> >>>> >>> >> > From mandy.chung at oracle.com Wed Feb 12 18:20:27 2014 From: mandy.chung at oracle.com (Mandy Chung) Date: Wed, 12 Feb 2014 10:20:27 -0800 Subject: RFR: (8031737) CHECK_NULL and CHECK_EXCEPTION macros cleanup In-Reply-To: <52FBB917.4080102@oracle.com> References: <52ED36FD.5070507@oracle.com> <1243D3E7-C384-4AA8-80CA-9AA4E053F056@oracle.com> <52F00EAC.8000900@oracle.com> <52FA7429.5070806@oracle.com> <52FA768A.70005@oracle.com> <52FA80A6.2040509@oracle.com> <52FA87E2.9080009@oracle.com> <52FA98A8.7090104@oracle.com> <52FA9AA2.6090005@oracle.com> <52FA9F82.90903@oracle.com> <52FAA211.2070800@oracle.com> <52FAA3F4.6040702@oracle.com> <52FAB577.3070205@oracle.com> <52FBB917.4080102@oracle.com> Message-ID: <52FBBB6B.9080101@oracle.com> This looks good to me too. Mandy On 2/12/2014 10:10 AM, roger riggs wrote: > Hi Phil, > > The changes look fine. (Not a "Reviewer"). > > Thanks for doing the backport, > > Roger > > On 2/11/2014 6:42 PM, Phil Race wrote: >> Here's a JDk8u webrev : -http://cr.openjdk.java.net/~prr/8031737.8u/ >> >> -phil. >> >> On 2/11/14 2:28 PM, Phil Race wrote: >>> So since hg export/import doesn't apply cleanly and the dependency >>> chain seems, long and in order to have some consistency across the >>> releases, >>> I think I should prepare a webrev which essentially backports 8031737 >>> including its small changes to Version.c, if only because otherwise >>> I'd have to have a new bug ID that would not be forwarded ported >>> (one source of confusion) or even worse re-use 8031737 but not >>> fully implement it >>> >>> Agreed ? >>> >>> -phil. >>> >>> On 2/11/2014 2:20 PM, roger riggs wrote: >>>> Hi Phil, >>>> >>>> >>>> On 2/11/2014 5:09 PM, Phil Race wrote: >>>>> Are we talking about the same changesets ? >>>>> a09982d91fab/8030993 has no change to the macros >>>> right (I didn't think this was topic of this conversation) >>>>> >>>>> fb89dc4fe8da/8031737 is the one that reimplemented the macros >>>>> and is the version I'd want. Its the last 'edit' of those macros >>>>> in that file. >>>> yes, >>>>> >>>>> c58c6b0fbe34/8030875 is the original addition of these :- >>>> Yes. >>>> >>>> Roger >>>> >>>>> >>>>> ... >>>>> >>>>> changeset: 9229:fb89dc4fe8da >>>>> user: rriggs >>>>> date: Mon Feb 03 16:58:02 2014 -0500 >>>>> summary: 8031737: CHECK_NULL and CHECK_EXCEPTION macros cleanup >>>>> >>>>> changeset: 9051:c58c6b0fbe34 >>>>> user: rriggs >>>>> date: Fri Jan 10 10:45:56 2014 -0500 >>>>> summary: 8030875: Macros for checking and returning on exceptions >>>>> >>>>> >>>>> ... >>>>> >>>>> -phil. >>>>> >>>>> On 2/11/14 1:48 PM, roger riggs wrote: >>>>>> Hi Phil, >>>>>> >>>>>> The later changeset picked up the recommended style of >>>>>> implementing the macros >>>>>> but I don't think it was substantive. You can probably do >>>>>> without it. >>>>>> >>>>>> Version.c had some changes in a different changeset to address >>>>>> the omission of checking for exceptions after some JNI calls. >>>>>> >>>>>> Roger >>>>>> >>>>>> On 2/11/2014 4:39 PM, Phil Race wrote: >>>>>>> Roger, >>>>>>> >>>>>>> That later one seems to be using the macros. I don't see any >>>>>>> update to the macros. >>>>>>> So I'm not sure why I'm need it .. since I'm not using those >>>>>>> calls and neither >>>>>>> are the macros. >>>>>>> >>>>>>> -phil. >>>>>>> >>>>>>> On 2/11/14 12:28 PM, roger riggs wrote: >>>>>>>> Hi Phil, >>>>>>>> >>>>>>>> Yes, it ended up in two change sets in jdk 9, you should take >>>>>>>> both to be up to date. >>>>>>>> >>>>>>>> changeset: 9245:a09982d91fab >>>>>>>> user: rriggs >>>>>>>> date: Wed Feb 05 10:59:53 2014 -0500 >>>>>>>> files: src/share/native/common/jni_util.c >>>>>>>> description: >>>>>>>> 8030993: Check jdk/src/share/native/common/jni_util.c for JNI >>>>>>>> pending exceptions >>>>>>>> >>>>>>>> >>>>>>>> changeset: 9229:fb89dc4fe8da >>>>>>>> date: Mon Feb 03 16:58:02 2014 -0500 >>>>>>>> files: src/share/native/common/jni_util.h >>>>>>>> src/share/native/sun/misc/Version.c >>>>>>>> interrupted! >>>>>>>> description: >>>>>>>> 8031737: CHECK_NULL and CHECK_EXCEPTION macros cleanup >>>>>>>> >>>>>>>> Thanks, Roger >>>>>>>> >>>>>>>> >>>>>>>> On 2/11/2014 2:57 PM, Phil Race wrote: >>>>>>>>> Roger, >>>>>>>>> >>>>>>>>> Yes, I can do that. >>>>>>>>> >>>>>>>>> I see here >>>>>>>>> http://cr.openjdk.java.net/~rriggs/webrev-check-cleanup-8031737/ >>>>>>>>> that >>>>>>>>> 1) There was a previous version of these macros. >>>>>>>>> Looks like no need to worry about that I just need the latest >>>>>>>>> version. >>>>>>>>> 2) There was also a change to Version.c. I can include that if >>>>>>>>> you think it >>>>>>>>> appropriate .. or omit it if you think its not essential. >>>>>>>>> >>>>>>>>> -phil. >>>>>>>>> >>>>>>>>> On 2/11/2014 11:14 AM, roger riggs wrote: >>>>>>>>>> Hi Phil, >>>>>>>>>> >>>>>>>>>> I see your point, there is nothing in the changes unique to 9. >>>>>>>>>> Do you want to take care of the back point? >>>>>>>>>> >>>>>>>>>> Roger >>>>>>>>>> >>>>>>>>>> On 2/11/2014 2:04 PM, Phil Race wrote: >>>>>>>>>>> Roger, >>>>>>>>>>> >>>>>>>>>>> Why not JDK 8u ? I've got a lot of changes that utilise >>>>>>>>>>> these that will >>>>>>>>>>> backport cleanly to JDK 8u only if 8u includes these macros. >>>>>>>>>>> And since >>>>>>>>>>> the changes are all over the place I don't fancy >>>>>>>>>>> copy/pasting them >>>>>>>>>>> everywhere. I suspect I am not the only one who would like >>>>>>>>>>> these in 8u .. >>>>>>>>>>> >>>>>>>>>>> -phil. >>>>>>>>>>> >>>>>>>>>>> >>>>>>>>>>> On 02/03/2014 01:48 PM, roger riggs wrote: >>>>>>>>>>>> Hi Lance, >>>>>>>>>>>> >>>>>>>>>>>> The convenience macros are only intended for JDK 9. >>>>>>>>>>>> >>>>>>>>>>>> Roger >>>>>>>>>>>> >>>>>>>>>>>> On 2/1/2014 1:58 PM, Lance @ Oracle wrote: >>>>>>>>>>>>> Looks fine >>>>>>>>>>>>> >>>>>>>>>>>>> Which releases are you think of including this in if any >>>>>>>>>>>>> besides 9? >>>>>>>>>>>>> >>>>>>>>>>>>> >>>>>>>>>>>>> Lance >>>>>>>>>>>>> Andersen| Principal Member of Technical Staff | >>>>>>>>>>>>> +1.781.442.2037 >>>>>>>>>>>>> Oracle Java Engineering >>>>>>>>>>>>> 1 Network Drive >>>>>>>>>>>>> Burlington, MA 01803 >>>>>>>>>>>>> Lance.Andersen at oracle.com >>>>>>>>>>>>> Sent from my iPad >>>>>>>>>>>>> >>>>>>>>>>>>> On Feb 1, 2014, at 1:03 PM, roger riggs >>>>>>>>>>>>> > >>>>>>>>>>>>> wrote: >>>>>>>>>>>> >>>>>>>>>>> >>>>>>>>>> >>>>>>>>> >>>>>>>> >>>>>>> >>>>>> >>>>> >>>> >>> >> > From huizhe.wang at oracle.com Wed Feb 12 18:25:59 2014 From: huizhe.wang at oracle.com (huizhe wang) Date: Wed, 12 Feb 2014 10:25:59 -0800 Subject: RFR(S): 8034087: XML parser may overwrite element content if that content falls onto the border of an entity scanner buffer In-Reply-To: <61C6FA0C43817B4FA6EB52E901E137E7729A7913@DEWDFEMB18B.global.corp.sap> References: <52FA535D.70804@oracle.com> <52FA9CC5.3010104@oracle.com> <61C6FA0C43817B4FA6EB52E901E137E7729A7913@DEWDFEMB18B.global.corp.sap> Message-ID: <52FBBCB7.9010907@oracle.com> Ok, understand. Thanks, Joe On 2/11/2014 11:40 PM, Schreiber, Steffen wrote: > Hi Joe, > > Yes, that's certainly true, if the test is only to show the issue. > We intended the test to work as a regression test as well and wanted to check the effects of the buffer boundaries on more possible places, i.e. before/at/after special characters like <, > or / as well as inside tags and content. > > We can easily adapt the loop, if runtime is a concern here. > > Regards, > Steffen > > -----Original Message----- > From: huizhe wang [mailto:huizhe.wang at oracle.com] > Sent: Dienstag, 11. Februar 2014 22:57 > To: Volker Simonis > Cc: Alan Bateman; Schreiber, Steffen; Java Core Libs > Subject: Re: RFR(S): 8034087: XML parser may overwrite element content if that content falls onto the border of an entity scanner buffer > > Hi Volker, > > I agree with the approach below and jdk9/dev is the better forest. > > For the test itself, I would suggest reducing the following loop to 1 or > 2 cases: > > for (int i = 0; i < testString.length(); i++) { > test(createDocument(testString.toString(), i), ""+ i); > } > > > when i=7, the problem starts to show. It's sufficient to demonstrate the > issue then by just entering 7. It's unnecessary to run the test 43 times. > > > Thanks, > Joe > > On 2/11/2014 9:00 AM, Volker Simonis wrote: >> Hi Alan, >> >> you're right. I initially didn't saw the test because I just looked at >> the change in the jaxp repository. >> >> If it will be approved, I'll put the test in the same directory like >> the other test (i.e. test/javax/xml/jaxp/parsers/8027359). >> >> And yes, my plan was to get approval for both, the tests and the fix, >> when asking for the permission to downport to jdk8u-dev and jdk7u-dev. >> >> Thanks, >> Volker >> >> >> On Tue, Feb 11, 2014 at 5:44 PM, Alan Bateman wrote: >>> On 11/02/2014 14:57, Volker Simonis wrote: >>>> Hi, >>>> >>>> after opening this bug yesterday for an issue found by my colleague >>>> Steffen Schreiber we realized that this is actually a duplicate of >>>> "8027359: XML parser returns incorrect parsing results" >>>> (https://bugs.openjdk.java.net/browse/JDK-8027359). >>>> >>>> While there's no need now to submit a patch anymore, we'd >>>> nevertheless like to contribute at least our test case for this issue: >>>> >>>> http://cr.openjdk.java.net/~simonis/webrevs/8034087/ >>>> >>>> The webrev is against jdk9-client but we'd like to also downport this >>>> test to jdk7 and jdk8 to track that the fix for 8027359 will be >>>> correctly downported to these releases as well. >>>> >>>> I will sponsor this change if somebody would be so kind to review it. >>>> >>> I'll leave it to Joe Wang to comment on the test but just to mention that >>> jdk9/dev is probably a better forest to aim for because that is where the >>> XML (and its tests) usually go. Also I wonder if it might be better to put >>> it in the same directory as the test that Joe pushed with the change? >>> >>> If you are getting approval to push to jdk8u-dev and jdk7u-dev then it might >>> be better to request a backport of Joe's change at the same time. >>> >>> -Alan. From Alan.Bateman at oracle.com Wed Feb 12 18:35:17 2014 From: Alan.Bateman at oracle.com (Alan Bateman) Date: Wed, 12 Feb 2014 18:35:17 +0000 Subject: RFR for JDK-8028711: TEST_BUG: Tests should pass through VM options: corelibs tests In-Reply-To: <52FB7039.3060907@oracle.com> References: <52E4C2E8.1060605@oracle.com> <52E4D1F2.4030406@oracle.com> <52E5B4C3.6080405@oracle.com> <52F455FC.3020009@oracle.com> <52F4BD7B.8080006@oracle.com> <52F87906.5040702@oracle.com> <52F89D96.7000601@oracle.com> <52FB7039.3060907@oracle.com> Message-ID: <52FBBEE5.7010302@oracle.com> On 12/02/2014 12:59, michael cui wrote: > On 02/10/2014 05:36 PM, Alan Bateman wrote: >> A minor comment on test/java/net/URLPermission/nstest/lookup.sh is >> that the resulting line length is 182 characters and this will likely >> be annoying for future side-by-side views. So I think I'd split this >> while you are there. > Please review the updated version at > http://cr.openjdk.java.net/~tyan/michael/JDK-8028711/webrev.04/ > > > Changes includes : > > 1. split line if it longer than 80 characters. > 2. merge the fix of JDK-8033897 > > 3. add few missed scripts. > > If no further changes need to be made, I would like to find sponsor to > push this fix. The changes look okay to me. I see you decided to ignore the javac usages but that is okay and can be done another time. Also thanks for fixing lookup.sh. I don't personally mind lines > 80 but that one was >180 which make it difficult to look at side-by-side changes. -Alan. From Alan.Bateman at oracle.com Wed Feb 12 19:45:42 2014 From: Alan.Bateman at oracle.com (Alan Bateman) Date: Wed, 12 Feb 2014 19:45:42 +0000 Subject: RFR (JAXP): 8033980 : Xerces Update: datatype XMLGregorianCalendarImpl and DurationImpl In-Reply-To: <52FAF801.5060300@oracle.com> References: <52F9666F.9010303@oracle.com> <5111B487-22C1-4987-B9CC-ABF2F5CDD964@oracle.com> <52FA5855.3020106@oracle.com> <52FA7270.4090709@oracle.com> <52FAF801.5060300@oracle.com> Message-ID: <52FBCF66.8060308@oracle.com> On 12/02/2014 04:26, huizhe wang wrote: > Hi Lance, Daniel, > > I suggest we take this incompatibility and document it in the release > notes (we'll likely have more). I reversed DurationImpl and then > realized why the Xerces engineers made the incompatibility change when > I started on XMLGregorianCalendarImpl. It was because > XMLGregorianCalendarImpl did not implement what was recommended, using > the lexical form for serialization. In the original implementation > therefore, the impl between XMLGregorianCalendarImpl and DurationImpl > was not consistent. > > Keeping this Xerces revision makes the serialization consistent in > both classes and adds the benefit of having the same source as that of > Xerces. I think we might need to hold the horses and understand this one and compatibility impact a bit more. As I understand, XMLGregorianCalendar is not Serializable but the Xerces implementation (XMLGregorianCalendarImpl) is. If someone is using the JDK and hasn't configured an alternative DatatypeFactory then it's an instance of com.sun.org.apache.xerces.internal.jaxp.datatype.XMLGregorianCalendarImpl that is serialized today. Do I have this right? I assume there is no serialization interop between this and someone using vanilla Xerces as the other end will be dealing with org.apache classes. With the proposed change then you've added a writeReplace so that going forward it will write a com.sun.org.apache.xerces.internal.jaxp.datatype.SerializedDuration into the stream. This isn't going to work if you've got a JDK 8 or older on the other end, right? Also I assume this doesn't help the Xerces interop because the fully qualified class names are still different. XMLGregorianCalendarImpl doesn't appear to have a readObject or other serialization methods so I assume that if you don't change the serialVersionUID then it would at least be possible to deserialize something that was serialized by an older JDK. As you've added a writeReplace then it makes me wonder why the serialVersionUID of XMLGregorianCalendarImpl has changed. Maybe that part could be reversed and leave the long standing value? One thing that might help is to develop a number serialization tests to help better understand the impact of the change. -Alan. From stuart.marks at oracle.com Wed Feb 12 19:48:54 2014 From: stuart.marks at oracle.com (Stuart Marks) Date: Wed, 12 Feb 2014 11:48:54 -0800 Subject: RFR for JDK-8030844:sun/rmi/rmic/classpath/RMICClassPathTest.java timeout in same binaries run In-Reply-To: <539F9CC8-77FA-43C6-A19A-C6623641CA6C@oracle.com> References: <13F6E908-26F2-420A-9E25-AD00DF8E3769@oracle.com> <52F8C146.7060505@oracle.com> <52FAD718.6090103@oracle.com> <539F9CC8-77FA-43C6-A19A-C6623641CA6C@oracle.com> Message-ID: <52FBD026.5030807@oracle.com> Hi Tristan, Changes look good. Unfortunately the webrev doesn't contain an actual changeset; it just contains a patch. In the webrev header there is the line "Patch of Changes". Instead it should say "Changeset". Like this one: http://cr.openjdk.java.net/~smarks/reviews/7900311/webrev.1/ Unfortunately webrev doesn't always produce an actual changeset, in particular it doesn't if you use webrev -r. (My example webrev above is a patch that changes webrev to generate the changeset even with -r. It hasn't been pushed yet; possibly it will later today. Or you can apply my webrev changeset to your local webrev.) Or, you can just run webrev without -r and it should check "hg outgoing" to determine the changesets used in generating the webrev, which does place the changeset into the webrev. Can you regenerate the webrev so that it includes the actual changeset? Sorry, this is a small thing, but as someone who is sponsoring changesets, I'd appreciate an actual changeset in the webrev instead of having to construct one manually. I think other sponsors would appreciate this too. s'marks On 2/11/14 6:59 PM, Tristan Yan wrote: > Thank you Stuart > http://cr.openjdk.java.net/~tyan/JDK-8030844/webrev.01/ > Regards > Tristan > > On Feb 12, 2014, at 10:06 AM, Stuart Marks > wrote: > >> Hi, yes, I'll take this one. >> >> It's slightly odd that this is creating filenames that already have "/" in >> them (as opposed to File.separator) but since these files don't actually have >> to exist, I suppose it doesn't really matter. >> >> I'm not convinced that we actually have any evidence that /home/~user is >> really causing the hang/timeout (either caused by the automounter hanging on >> /home or LDAP or other nameservice lookup on ~user), but this is harmless, and >> it'll fix the problem on the off chance that this really is the root cause. >> >> Tristan, please update the test's @bug tag to add 8030844, create a changeset, >> and create a webrev with the changeset in it (as opposed to a bare patch). >> I'll then push it for you. >> >> s'marks >> >> On 2/10/14 4:08 AM, Alan Bateman wrote: >>> On 10/02/2014 10:57, Tristan Yan wrote: >>>> Ping: Can anyone give a review on this. >>>> Thank you >>>> Tristan >>> Changing the test so that it doesn't try to /home/~user seems reasonable to me. >>> >>> Stuart - I see you've been sponsoring Tristan's updates to the RMI tests. Are >>> you going to take this one too? >>> >>> -Alan >>> >>> >>>> >>>> On Feb 6, 2014, at 5:13 PM, Tristan Yan>>> > wrote: >>>> >>>>> Hi All >>>>> >>>>> Please help to review a simple fix for >>>>> https://bugs.openjdk.java.net/browse/JDK-8030844 >>>>> >>>>> http://cr.openjdk.java.net/~tyan/JDK-8030844/webrev.00/. >>>>> >>>>> Description: >>>>> Change replace a ?/home/~user" folder with an test source path. Folder >>>>> ?/home/~user? cause some problem whenever something wrong with the automount >>>>> filesystem or an username lookup problem. >>>>> >>>>> Thank you >>>>> Tristan >>> > From stuart.marks at oracle.com Wed Feb 12 21:29:24 2014 From: stuart.marks at oracle.com (Stuart Marks) Date: Wed, 12 Feb 2014 13:29:24 -0800 Subject: RFR: JDK-8032050: TEST_BUG: java/rmi/activation/Activatable/shutdownGracefully/ShutdownGracefully.java fails intermittently In-Reply-To: <0C67EF5A-7889-49C5-8079-BDA7FF4CF1F0@oracle.com> References: <52E20974.7060706@oracle.com> <52E31F53.3070702@oracle.com> <52E4959C.3020907@oracle.com> <52E8AADE.6000300@oracle.com> <52EB60A8.7070307@oracle.com> <133FA08E-5731-437D-BB1F-AD9872CBDD36@oracle.com> <52F96FA8.3010102@oracle.com> <0C67EF5A-7889-49C5-8079-BDA7FF4CF1F0@oracle.com> Message-ID: <52FBE7B4.5060802@oracle.com> Hi Tristan, JavaVM.waitFor looks mostly fine. The indentation of the start of the waitFor(timeout) javadoc comment is a bit off, though; please fix. There are still some adjustments that need to be made in ShutdownGracefully.java. Both have to do with the case where the last registration succeeds unexpectedly -- this is the one that we expect to fail. First, if this registration has succeeded unexpectedly, that means rmid is still running. If that occurs, the call to rmid.waitFor(timeout) will inevitably time out. It may be worth calling rmid.destroy() directly at this point. Second, still in the succeeded-unexpectedly case, at line 154 TestLibrary.bomb() is called. This throws an exception, but it's caught by the catch-block at lines 157-158, which calls TestLibrary.bomb() again, saying "unexpected exception". Except that this is kind of expected, since it was thrown from an earlier call to TestLibrary.bomb(). This is quite confusing. There are several cases that need to be handled. 1. Normal case. Registration fails as expected, rmid has terminated gracefully. Test passes. 2. Rmid is still running and has processed the registration request successfully. Need to kill rmid and fail the test. 3. Rmid is still running but is in some bad state where the registration request failed. Need to kill rmid and fail the test. 4. Some other unexpected failure. This is what the catch and finally blocks at lines 157-161 are for. These four cases need to be handled independently. Ideally they should be separated from the cleanup code. As noted above, you don't want to throw an exception from the try-block, because it will get caught by your own catch block. Similarly, it's tempting to return from the midst of the try-block in the success case, but this still runs the finally-block. This can be quite confusing. A typical technique for dealing with this kind of issue is to record results of operations from within the try block, and then analyze the results outside, throwing a test failure (TestLibrary.bomb) at that point, where it won't be caught by the test's own catch block. Editoral: - line 153, there should be a space between 'if' and the opening paren - line 156, typo, "gracefuuly" Finally, it would be helpful if you could get webrev to generate the actual changeset instead of the plain patch, per my other review email. Thanks, s'marks On 2/11/14 9:39 PM, Tristan Yan wrote: > Thank you for your thorough mail. This is very educational. I took you advice > and generated a new webrev for this. > > http://cr.openjdk.java.net/~tyan/JDK-8032050/webrev.03/ > I appreciate you can review this again. > Regards > Tristan > > > On Feb 11, 2014, at 8:32 AM, Stuart Marks > wrote: > >> Hi Tristan, >> >> Sorry about my recurring delays. >> >> Several comments on these changes. >> >> JavaVM.java -- >> >> The waitFor(timeout) method is mostly ok. The new thread started at line 208 >> and following seems unnecessary, though. This code is reached when a timeout >> occurs, so throwing TimeoutException is the only thing necessary in this case. >> Should the code to start the new thread be removed? >> >> There should be a similar check for vm == null as in the waitFor() [no args] >> method. >> >> ShutdownGracefully.java -- >> >> The condition that's checked after calling rmid.waitFor(SHUTDOWN_TIMEOUT) is >> incorrect. It's testing the exit status against zero. Offhand, when and if >> rmid exits, it might exit with a nonzero exit status. If rmid has exited at >> this point, then the test should succeed. >> >> Instead of testing against zero, the code should catch TimeoutException, which >> means that rmid is still running. It's probably reasonable to catch >> TimeoutException, print a message, and then let the finally-block destroy the >> rmid. Calling TestLibrary.bomb() from within the try-block is confusing, since >> that method throws an exception, which is then caught by the catch-block, when >> then calls TestLibrary.bomb() again. >> >> We should also make sure to test the success case properly. If rmid.waitFor() >> returns in a timely fashion without throwing TimeoutException, it doesn't >> matter what the exit status is. (It might be worth printing it out.) At that >> point we know that rmid *has* exited gracefully, so we need to set rmid to >> null so that the finally-block doesn't attempt to destroy rmid redundantly. >> Some additional messages about rmid having exited and the test passing are >> also warranted for this case. >> >> Some additional cleanup can be done here as well, over and above the changes >> you've proposed. (This stuff is left over from earlier RMI test messes.) In >> order to shut down an active object, the code here spawns a new thread that >> sleeps for a while and then deactivates this object. This isn't necessary. (It >> might have been necessary in the past.) It's sufficient simply to unexport >> this object and then deactivate it, directly within the shutdown() method. See >> >> test/java/rmi/activation/ActivationSystem/unregisterGroup/UnregisterGroup.java >> >> for an example of this. In addition, the run() method can be removed, and the >> "implements Runnable" declaration can also be removed from the >> ShutdownGracefully test class. >> >> Finally, revisiting some code farther up in the test, the try-block at lines >> 135-140 issues a registration request that the test expects to fail. If it >> succeeds, the message at line 139 isn't very clear; it should say that the >> registration request succeeded unexpectedly. This should cause the test to >> fail. We still probably want to go through the waitFor(timeout) path and >> eventual rmid cleanup, but a flag should be set here to ensure that the test >> indeed fails if the registration succeeds unexpectedly, and the messages >> should clearly indicate that this is going on. >> >> A good way to test this last case is to change rmid's security manager to the >> normal security manager java.lang.SecurityManager instead of TestSecurityManager. >> >> Thanks, >> >> s'marks >> >> >> >> >> On 2/10/14 2:59 AM, Tristan Yan wrote: >>> Hi Stuart >>> Could you help to review this. >>> Thank you >>> Tristan >>> >>> On Jan 31, 2014, at 4:36 PM, Tristan Yan >> >>> > wrote: >>> >>>> Thank you for fixing JDK-8023541. Then I leave ActivationLibrary.java for now. >>>> I still did some clean up following your suggestion. >>>> 1. I changed waitFor(long timeout) method, this method is going to use code >>>> like Process.waitFor(timeout, unit). This can be backported to JDK7. Also >>>> exitValue is kept as a return value. For making sure there is no Pipe leak, a >>>> cleanup thread will start when timeout happens. >>>> 2. Change in ShutdownGracefully is a little tricky. I think we should just >>>> destroy JVM once exception is thrown. So I move the wait logic into try block >>>> instead keep them in finally block. >>>> Can you receive it again. >>>> http://cr.openjdk.java.net/~tyan/JDK-8032050/webrev.02/ >>>> Thank you >>>> Tristan >>>> >>>> On 01/29/2014 03:16 PM, Stuart Marks wrote: >>>>> Hi Tristan, >>>>> >>>>> I don't want to put the workaround into ActivationLibrary.rmidRunning() for a >>>>> null return from the lookup, because this is only a workaround for an actual >>>>> bug in rmid initialization. See the review I just posted for JDK-8023541. >>>>> >>>>> Adding JavaVM.waitFor(timeout) is something that would be useful in general, >>>>> but it needs to be handled carefully. It uses the new >>>>> Process.waitFor(timeout, unit) which is new in Java SE 8; this makes >>>>> backporting to 7 more complicated. Not clear whether we'll do so, but I don't >>>>> want to forclose the opportunity without discussion. It's also not clear how >>>>> one can get the vm's exit status after JavaVM.waitFor() has returned true. >>>>> With the Process API it's possible simply to call waitFor() or exitValue(). >>>>> With JavaVM, a new API needs to be created, or the rule has to be established >>>>> that one must call JavaVM.waitFor() to collect the exit status as well as to >>>>> close the pipes from the subprocess. If JavaVM.waitFor(timeout, unit) is >>>>> called without subsequently calling JavaVM.waitFor(), the pipes are leaked. >>>>> >>>>> In ShutdownGracefully.java, the finally-block needs to check to see if rmid >>>>> is still running, and if it is, to shut it down. Simply calling >>>>> waitFor(timeout, unit) isn't sufficient, because if the rmid process is still >>>>> running, it will be left running. >>>>> >>>>> The straightforward approach would be to call ActivationLibrary.rmidRunning() >>>>> to test if it's still running. Unfortunately this isn't quite right, because >>>>> rmidRunning() has a timeout loop in it -- which should probably be removed. >>>>> (I think there's a bug for this.) Another approach would be simply to call >>>>> rmid.destroy(). This calls rmid's shutdown() method first, which is >>>>> reasonable, but I'm not sure it kills the process if that fails. In any case, >>>>> this already has a timeout loop waiting for the process to die, so >>>>> ShutdownGracefully.java needn't use a new waitFor(timeout, unit) call. >>>>> >>>>> Removing the commented-out code that starts with "no longer needed" is good, >>>>> and removing the ShutdownDetectThread is also good, since that's unnecessary. >>>>> >>>>> There are some more cleanups I have in mind here but I'd like to see a >>>>> revised webrev before proceeding. >>>>> >>>>> Thanks, >>>>> >>>>> s'marks >>>>> >>>>> On 1/25/14 8:57 PM, Tristan Yan wrote: >>>>>> Hi Stuart >>>>>> Thank you for your review and suggestion. >>>>>> Yes, since this failure mode is very hard to be reproduced. I guess it's >>>>>> make sense to do some hack. And I also noticed in >>>>>> ActivationLibrary.rmidRunning. It does try to look up ActivationSystem but >>>>>> doesn't check if it's null. So I add the logic to make sure we will look up >>>>>> the non-null ActivationSystem. Also I did some cleanup if you don't mind. >>>>>> Add a waitFor(long timeout, TimeUnit unit) for JavaVM. Which we can have a >>>>>> better waitFor control. >>>>>> I appreciate you can review the code again. >>>>>> http://cr.openjdk.java.net/~tyan/JDK-8032050/webrev.01/ >>>>>> Thank you >>>>>> Tristan >>>>>> >>>>>> >>>>>> On 01/25/2014 10:20 AM, Stuart Marks wrote: >>>>>>> On 1/23/14 10:34 PM, Tristan Yan wrote: >>>>>>>> Hi All >>>>>>>> Could you review the bug fix for JDK-8032050. >>>>>>>> >>>>>>>> http://cr.openjdk.java.net/~tyan/JDK-8032050/webrev.00/ >>>>>>>> >>>>>>>> Description: >>>>>>>> This rare happened failure caused because when RMID starts. It don't >>>>>>>> guarantee >>>>>>>> sun.rmi.server.Activation.startActivation finishes. >>>>>>>> Fix by adding a iterative getSystem with a 5 seconds timeout. >>>>>>> >>>>>>> Hi Tristan, >>>>>>> >>>>>>> Adding a timing/retry loop into this test isn't the correct approach for >>>>>>> fixing this test. >>>>>>> >>>>>>> The timing loop isn't necessary because there is already a timing loop in >>>>>>> RMID.start() in the RMI test library. (There's another timing loop in >>>>>>> ActivationLibrary.rmidRunning() which should probably be removed.) So the >>>>>>> intent of this library call is that it start rmid and wait for it to become >>>>>>> ready. That logic doesn't need to be added to the test. >>>>>>> >>>>>>> In the bug report JDK-8032050 you had mentioned that the >>>>>>> NullPointerException was suspicious. You're right! I took a look and it >>>>>>> seemed like it was related to JDK-8023541, and I added a note to this >>>>>>> effect to the bug report. The problem here is that rmid can come up and >>>>>>> transiently return null instead of the stub of the activation system. >>>>>>> That's what JDK-8023541 covers. I think that rmid itself needs to be fixed, >>>>>>> though modifying the timing loop in the RMI test library to wait for rmid >>>>>>> to come up *and* for the lookup to return non-null is an easy way to fix >>>>>>> the problem. (Or at least cover it up.) >>>>>>> >>>>>>> The next step in the analysis is to determine, given that >>>>>>> ActivationLibrary.getSystem can sometimes return null, whether this has >>>>>>> actually caused this test failure. This is pretty easy to determine; just >>>>>>> hack in a line "system = null" in the right place and run the test. I've >>>>>>> done this, and the test times out and the output log is pretty much >>>>>>> identical to the one in the bug report. (I recommend you try this >>>>>>> yourself.) So I think it's fairly safe to say that the problem in >>>>>>> JDK-8023541 has caused the failure listed in JDK-8032050. >>>>>>> >>>>>>> I can see a couple ways to proceed here. One way is just to close this out >>>>>>> as a duplicate of JDK-8023541 since that bug caused this failure. >>>>>>> >>>>>>> Another is that this test could use some cleaning up. This bug certainly >>>>>>> covers a failure, but the messages emitted are confusing and in some cases >>>>>>> completely wrong. For example, the "rmid has shutdown" message at line 180 >>>>>>> is incorrect, because in this case rmid is still running and the wait() >>>>>>> call has timed out. Most of the code here can be replaced with calls to >>>>>>> various bits of the RMI test library. There are a bunch of other things in >>>>>>> this test that could be cleaned up as well. >>>>>>> >>>>>>> It's up to you how you'd like to proceed. >>>>>>> >>>>>>> s'marks >>>>>> >>>>> >>>> >>> > From forax at univ-mlv.fr Wed Feb 12 23:50:06 2014 From: forax at univ-mlv.fr (Remi Forax) Date: Thu, 13 Feb 2014 00:50:06 +0100 Subject: methods that throws checked exceptions in a lambda block In-Reply-To: <52FBA2F4.7050302@oracle.com> References: <68381B76-8459-4352-BDF1-CF37B4823517@oracle.com> <52FBA2F4.7050302@oracle.com> Message-ID: <52FC08AE.8010202@univ-mlv.fr> On 02/12/2014 05:36 PM, Henry Jen wrote: > > On 02/12/2014 07:05 AM, Wang Weijun wrote: >> This line does not compile >> >> Files.list(Paths.get("/secret")).forEach(x -> Files.write(x, new >> byte[Files.size(x)]); >> >> because Files.write() and Files.size() throw IOE. Then what's the >> best way to make it work? >> >> (I find some old threads on this on lambda-dev. Maybe we have a >> recommended way now?) >> > > I am not sure there is a "recommended" way. Depends on how you need to > handle those exceptions, I'll put the lambda in a private function to > take care those exception, and surpress them in a UncheckedIOException. > > Cheers, > Henry The other solution is to use wrappers like these ones: https://github.com/forax/blog/blob/master/src/com/github/forax/blog/UnsafeIO.java and rewrite your code like this: Files.list(Paths.get("/secret")).forEach(unsafeProc(x -> Files.write(x, new byte[Files.size(x)])); cheers, R?mi From michael.cui at oracle.com Thu Feb 13 01:15:36 2014 From: michael.cui at oracle.com (michael cui) Date: Thu, 13 Feb 2014 09:15:36 +0800 Subject: RFR for JDK-8028711: TEST_BUG: Tests should pass through VM options: corelibs tests In-Reply-To: <52FBBEE5.7010302@oracle.com> References: <52E4C2E8.1060605@oracle.com> <52E4D1F2.4030406@oracle.com> <52E5B4C3.6080405@oracle.com> <52F455FC.3020009@oracle.com> <52F4BD7B.8080006@oracle.com> <52F87906.5040702@oracle.com> <52F89D96.7000601@oracle.com> <52FB7039.3060907@oracle.com> <52FBBEE5.7010302@oracle.com> Message-ID: <52FC1CB8.90707@oracle.com> On 02/13/2014 02:35 AM, Alan Bateman wrote: > The changes look okay to me. I see you decided to ignore the javac > usages but that is okay and can be done another time. > > Also thanks for fixing lookup.sh. I don't personally mind lines > 80 > but that one was >180 which make it difficult to look at side-by-side > changes. Hi Alan, Thanks so much for reviewing this! Would you mind to sponsor this for me? Regrads, Michael Cui From tristan.yan at oracle.com Thu Feb 13 03:21:29 2014 From: tristan.yan at oracle.com (Tristan Yan) Date: Thu, 13 Feb 2014 11:21:29 +0800 Subject: RFR for JDK-8030844:sun/rmi/rmic/classpath/RMICClassPathTest.java timeout in same binaries run In-Reply-To: <52FBD026.5030807@oracle.com> References: <13F6E908-26F2-420A-9E25-AD00DF8E3769@oracle.com> <52F8C146.7060505@oracle.com> <52FAD718.6090103@oracle.com> <539F9CC8-77FA-43C6-A19A-C6623641CA6C@oracle.com> <52FBD026.5030807@oracle.com> Message-ID: <52FC3A39.6060307@oracle.com> Thank you Stuart This is a very nice tutorial. I did try both ways. Adopt your fix doesn't seem work for me. It still doesn't generate changeset. But without -r option works. http://cr.openjdk.java.net/~tyan/JDK-8030844/webrev.02/ Could you push it for me? Tristan On 02/13/2014 03:48 AM, Stuart Marks wrote: > Hi Tristan, > > Changes look good. Unfortunately the webrev doesn't contain an actual > changeset; it just contains a patch. In the webrev header there is the > line "Patch of Changes". Instead it should say "Changeset". Like this > one: > > http://cr.openjdk.java.net/~smarks/reviews/7900311/webrev.1/ > > Unfortunately webrev doesn't always produce an actual changeset, in > particular it doesn't if you use webrev -r. > > (My example webrev above is a patch that changes webrev to generate > the changeset even with -r. It hasn't been pushed yet; possibly it > will later today. Or you can apply my webrev changeset to your local > webrev.) > > Or, you can just run webrev without -r and it should check "hg > outgoing" to determine the changesets used in generating the webrev, > which does place the changeset into the webrev. > > Can you regenerate the webrev so that it includes the actual > changeset? Sorry, this is a small thing, but as someone who is > sponsoring changesets, I'd appreciate an actual changeset in the > webrev instead of having to construct one manually. I think other > sponsors would appreciate this too. > > s'marks > > On 2/11/14 6:59 PM, Tristan Yan wrote: >> Thank you Stuart >> http://cr.openjdk.java.net/~tyan/JDK-8030844/webrev.01/ >> Regards >> Tristan >> >> On Feb 12, 2014, at 10:06 AM, Stuart Marks > > wrote: >> >>> Hi, yes, I'll take this one. >>> >>> It's slightly odd that this is creating filenames that already have >>> "/" in >>> them (as opposed to File.separator) but since these files don't >>> actually have >>> to exist, I suppose it doesn't really matter. >>> >>> I'm not convinced that we actually have any evidence that >>> /home/~user is >>> really causing the hang/timeout (either caused by the automounter >>> hanging on >>> /home or LDAP or other nameservice lookup on ~user), but this is >>> harmless, and >>> it'll fix the problem on the off chance that this really is the root >>> cause. >>> >>> Tristan, please update the test's @bug tag to add 8030844, create a >>> changeset, >>> and create a webrev with the changeset in it (as opposed to a bare >>> patch). >>> I'll then push it for you. >>> >>> s'marks >>> >>> On 2/10/14 4:08 AM, Alan Bateman wrote: >>>> On 10/02/2014 10:57, Tristan Yan wrote: >>>>> Ping: Can anyone give a review on this. >>>>> Thank you >>>>> Tristan >>>> Changing the test so that it doesn't try to /home/~user seems >>>> reasonable to me. >>>> >>>> Stuart - I see you've been sponsoring Tristan's updates to the RMI >>>> tests. Are >>>> you going to take this one too? >>>> >>>> -Alan >>>> >>>> >>>>> >>>>> On Feb 6, 2014, at 5:13 PM, Tristan Yan>>>> > wrote: >>>>> >>>>>> Hi All >>>>>> >>>>>> Please help to review a simple fix for >>>>>> https://bugs.openjdk.java.net/browse/JDK-8030844 >>>>>> >>>>>> http://cr.openjdk.java.net/~tyan/JDK-8030844/webrev.00/. >>>>>> >>>>>> Description: >>>>>> Change replace a ?/home/~user" folder with an test source path. >>>>>> Folder >>>>>> ?/home/~user? cause some problem whenever something wrong with >>>>>> the automount >>>>>> filesystem or an username lookup problem. >>>>>> >>>>>> Thank you >>>>>> Tristan >>>> >> From stuart.marks at oracle.com Thu Feb 13 07:08:00 2014 From: stuart.marks at oracle.com (Stuart Marks) Date: Wed, 12 Feb 2014 23:08:00 -0800 Subject: RFR (S): JDK-8034000 lack of /othervm option can cause some RMI tests to fail Message-ID: <52FC6F50.6010901@oracle.com> Hi all, The RMI test directories were removed from TEST.ROOT's othervm.dirs by JDK-8031179 so that individual RMI tests could opt-in to othervm themselves. Unfortunately, some tests needed othervm but didn't get it, causing them to fail. This adds othervm to some failing tests, and adds a note to other tests that do not have othervm indicating specifically that they don't need othervm. Bug: https://bugs.openjdk.java.net/browse/jdk-8034000 Webrev: http://cr.openjdk.java.net/~smarks/reviews/8034000/webrev.0/ Thanks, s'marks From joe.darcy at oracle.com Thu Feb 13 07:31:14 2014 From: joe.darcy at oracle.com (Joe Darcy) Date: Wed, 12 Feb 2014 23:31:14 -0800 Subject: RFR (S): JDK-8034000 lack of /othervm option can cause some RMI tests to fail In-Reply-To: <52FC6F50.6010901@oracle.com> References: <52FC6F50.6010901@oracle.com> Message-ID: <52FC74C2.9000801@oracle.com> Look good Stuart, -Joe On 02/12/2014 11:08 PM, Stuart Marks wrote: > Hi all, > > The RMI test directories were removed from TEST.ROOT's othervm.dirs by > JDK-8031179 so that individual RMI tests could opt-in to othervm > themselves. Unfortunately, some tests needed othervm but didn't get > it, causing them to fail. > > This adds othervm to some failing tests, and adds a note to other > tests that do not have othervm indicating specifically that they don't > need othervm. > > Bug: > > https://bugs.openjdk.java.net/browse/jdk-8034000 > > Webrev: > > http://cr.openjdk.java.net/~smarks/reviews/8034000/webrev.0/ > > Thanks, > > s'marks From huizhe.wang at oracle.com Thu Feb 13 08:18:46 2014 From: huizhe.wang at oracle.com (huizhe wang) Date: Thu, 13 Feb 2014 00:18:46 -0800 Subject: RFR (JAXP): 8033980 : Xerces Update: datatype XMLGregorianCalendarImpl and DurationImpl In-Reply-To: <52FBCF66.8060308@oracle.com> References: <52F9666F.9010303@oracle.com> <5111B487-22C1-4987-B9CC-ABF2F5CDD964@oracle.com> <52FA5855.3020106@oracle.com> <52FA7270.4090709@oracle.com> <52FAF801.5060300@oracle.com> <52FBCF66.8060308@oracle.com> Message-ID: <52FC7FE6.2060400@oracle.com> Hi Alan, Lance, and Daniel, The Xerces serialization revision meant to create a serialization form that would help maintain future serialization compatibility. But in reality it itself is causing significant incompatibility as Alan pointed out below and we discussed previously. I've removed the revision from the patch as a result. Please see the new webrev here: http://cr.openjdk.java.net/~joehw/jdk9/8033980/webrev/ Thanks, Joe On 2/12/2014 11:45 AM, Alan Bateman wrote: > On 12/02/2014 04:26, huizhe wang wrote: >> Hi Lance, Daniel, >> >> I suggest we take this incompatibility and document it in the release >> notes (we'll likely have more). I reversed DurationImpl and then >> realized why the Xerces engineers made the incompatibility change >> when I started on XMLGregorianCalendarImpl. It was because >> XMLGregorianCalendarImpl did not implement what was recommended, >> using the lexical form for serialization. In the original >> implementation therefore, the impl between XMLGregorianCalendarImpl >> and DurationImpl was not consistent. >> >> Keeping this Xerces revision makes the serialization consistent in >> both classes and adds the benefit of having the same source as that >> of Xerces. > I think we might need to hold the horses and understand this one and > compatibility impact a bit more. > > As I understand, XMLGregorianCalendar is not Serializable but the > Xerces implementation (XMLGregorianCalendarImpl) is. If someone is > using the JDK and hasn't configured an alternative DatatypeFactory > then it's an instance of > com.sun.org.apache.xerces.internal.jaxp.datatype.XMLGregorianCalendarImpl > that is serialized today. Do I have this right? I assume there is no > serialization interop between this and someone using vanilla Xerces as > the other end will be dealing with org.apache classes. > > With the proposed change then you've added a writeReplace so that > going forward it will write a > com.sun.org.apache.xerces.internal.jaxp.datatype.SerializedDuration > into the stream. This isn't going to work if you've got a JDK 8 or > older on the other end, right? Also I assume this doesn't help the > Xerces interop because the fully qualified class names are still > different. > > XMLGregorianCalendarImpl doesn't appear to have a readObject or other > serialization methods so I assume that if you don't change the > serialVersionUID then it would at least be possible to deserialize > something that was serialized by an older JDK. As you've added a > writeReplace then it makes me wonder why the serialVersionUID of > XMLGregorianCalendarImpl has changed. Maybe that part could be > reversed and leave the long standing value? > > One thing that might help is to develop a number serialization tests > to help better understand the impact of the change. > > -Alan. > > From tristan.yan at oracle.com Thu Feb 13 08:25:28 2014 From: tristan.yan at oracle.com (Tristan Yan) Date: Thu, 13 Feb 2014 16:25:28 +0800 Subject: RFR: JDK-8032050: TEST_BUG: java/rmi/activation/Activatable/shutdownGracefully/ShutdownGracefully.java fails intermittently In-Reply-To: <52FBE7B4.5060802@oracle.com> References: <52E20974.7060706@oracle.com> <52E31F53.3070702@oracle.com> <52E4959C.3020907@oracle.com> <52E8AADE.6000300@oracle.com> <52EB60A8.7070307@oracle.com> <133FA08E-5731-437D-BB1F-AD9872CBDD36@oracle.com> <52F96FA8.3010102@oracle.com> <0C67EF5A-7889-49C5-8079-BDA7FF4CF1F0@oracle.com> <52FBE7B4.5060802@oracle.com> Message-ID: <52FC8178.9000500@oracle.com> Thank you Stuart I have fixed comment in JavaVM.java. Dealing with different cases in ShutdownGracefully.java, two variables were added. One is a flag indicate test passed or not. Other variable keeps the error message when test failed. I put TestLibrary.bomb in the bottom of the main method which only shows test fail message. Could you review it again http://cr.openjdk.java.net/~tyan/JDK-8032050/webrev.04/ Tristan On 02/13/2014 05:29 AM, Stuart Marks wrote: > Hi Tristan, > > JavaVM.waitFor looks mostly fine. The indentation of the start of the > waitFor(timeout) javadoc comment is a bit off, though; please fix. > > There are still some adjustments that need to be made in > ShutdownGracefully.java. Both have to do with the case where the last > registration succeeds unexpectedly -- this is the one that we expect > to fail. > > First, if this registration has succeeded unexpectedly, that means > rmid is still running. If that occurs, the call to > rmid.waitFor(timeout) will inevitably time out. It may be worth > calling rmid.destroy() directly at this point. > > Second, still in the succeeded-unexpectedly case, at line 154 > TestLibrary.bomb() is called. This throws an exception, but it's > caught by the catch-block at lines 157-158, which calls > TestLibrary.bomb() again, saying "unexpected exception". Except that > this is kind of expected, since it was thrown from an earlier call to > TestLibrary.bomb(). This is quite confusing. > > There are several cases that need to be handled. > > 1. Normal case. Registration fails as expected, rmid has terminated > gracefully. Test passes. > > 2. Rmid is still running and has processed the registration request > successfully. Need to kill rmid and fail the test. > > 3. Rmid is still running but is in some bad state where the > registration request failed. Need to kill rmid and fail the test. > > 4. Some other unexpected failure. This is what the catch and finally > blocks at lines 157-161 are for. > > These four cases need to be handled independently. Ideally they should > be separated from the cleanup code. As noted above, you don't want to > throw an exception from the try-block, because it will get caught by > your own catch block. Similarly, it's tempting to return from the > midst of the try-block in the success case, but this still runs the > finally-block. This can be quite confusing. > > A typical technique for dealing with this kind of issue is to record > results of operations from within the try block, and then analyze the > results outside, throwing a test failure (TestLibrary.bomb) at that > point, where it won't be caught by the test's own catch block. > > Editoral: > - line 153, there should be a space between 'if' and the opening paren > - line 156, typo, "gracefuuly" > > Finally, it would be helpful if you could get webrev to generate the > actual changeset instead of the plain patch, per my other review email. > > Thanks, > > s'marks > > > On 2/11/14 9:39 PM, Tristan Yan wrote: >> Thank you for your thorough mail. This is very educational. I took >> you advice >> and generated a new webrev for this. >> >> http://cr.openjdk.java.net/~tyan/JDK-8032050/webrev.03/ >> I appreciate you can review this again. >> Regards >> Tristan >> >> >> On Feb 11, 2014, at 8:32 AM, Stuart Marks > > wrote: >> >>> Hi Tristan, >>> >>> Sorry about my recurring delays. >>> >>> Several comments on these changes. >>> >>> JavaVM.java -- >>> >>> The waitFor(timeout) method is mostly ok. The new thread started at >>> line 208 >>> and following seems unnecessary, though. This code is reached when a >>> timeout >>> occurs, so throwing TimeoutException is the only thing necessary in >>> this case. >>> Should the code to start the new thread be removed? >>> >>> There should be a similar check for vm == null as in the waitFor() >>> [no args] >>> method. >>> >>> ShutdownGracefully.java -- >>> >>> The condition that's checked after calling >>> rmid.waitFor(SHUTDOWN_TIMEOUT) is >>> incorrect. It's testing the exit status against zero. Offhand, when >>> and if >>> rmid exits, it might exit with a nonzero exit status. If rmid has >>> exited at >>> this point, then the test should succeed. >>> >>> Instead of testing against zero, the code should catch >>> TimeoutException, which >>> means that rmid is still running. It's probably reasonable to catch >>> TimeoutException, print a message, and then let the finally-block >>> destroy the >>> rmid. Calling TestLibrary.bomb() from within the try-block is >>> confusing, since >>> that method throws an exception, which is then caught by the >>> catch-block, when >>> then calls TestLibrary.bomb() again. >>> >>> We should also make sure to test the success case properly. If >>> rmid.waitFor() >>> returns in a timely fashion without throwing TimeoutException, it >>> doesn't >>> matter what the exit status is. (It might be worth printing it out.) >>> At that >>> point we know that rmid *has* exited gracefully, so we need to set >>> rmid to >>> null so that the finally-block doesn't attempt to destroy rmid >>> redundantly. >>> Some additional messages about rmid having exited and the test >>> passing are >>> also warranted for this case. >>> >>> Some additional cleanup can be done here as well, over and above the >>> changes >>> you've proposed. (This stuff is left over from earlier RMI test >>> messes.) In >>> order to shut down an active object, the code here spawns a new >>> thread that >>> sleeps for a while and then deactivates this object. This isn't >>> necessary. (It >>> might have been necessary in the past.) It's sufficient simply to >>> unexport >>> this object and then deactivate it, directly within the shutdown() >>> method. See >>> >>> test/java/rmi/activation/ActivationSystem/unregisterGroup/UnregisterGroup.java >>> >>> >>> for an example of this. In addition, the run() method can be >>> removed, and the >>> "implements Runnable" declaration can also be removed from the >>> ShutdownGracefully test class. >>> >>> Finally, revisiting some code farther up in the test, the try-block >>> at lines >>> 135-140 issues a registration request that the test expects to fail. >>> If it >>> succeeds, the message at line 139 isn't very clear; it should say >>> that the >>> registration request succeeded unexpectedly. This should cause the >>> test to >>> fail. We still probably want to go through the waitFor(timeout) path >>> and >>> eventual rmid cleanup, but a flag should be set here to ensure that >>> the test >>> indeed fails if the registration succeeds unexpectedly, and the >>> messages >>> should clearly indicate that this is going on. >>> >>> A good way to test this last case is to change rmid's security >>> manager to the >>> normal security manager java.lang.SecurityManager instead of >>> TestSecurityManager. >>> >>> Thanks, >>> >>> s'marks >>> >>> >>> >>> >>> On 2/10/14 2:59 AM, Tristan Yan wrote: >>>> Hi Stuart >>>> Could you help to review this. >>>> Thank you >>>> Tristan >>>> >>>> On Jan 31, 2014, at 4:36 PM, Tristan Yan >>> >>>> > wrote: >>>> >>>>> Thank you for fixing JDK-8023541. Then I leave >>>>> ActivationLibrary.java for now. >>>>> I still did some clean up following your suggestion. >>>>> 1. I changed waitFor(long timeout) method, this method is going to >>>>> use code >>>>> like Process.waitFor(timeout, unit). This can be backported to >>>>> JDK7. Also >>>>> exitValue is kept as a return value. For making sure there is no >>>>> Pipe leak, a >>>>> cleanup thread will start when timeout happens. >>>>> 2. Change in ShutdownGracefully is a little tricky. I think we >>>>> should just >>>>> destroy JVM once exception is thrown. So I move the wait logic >>>>> into try block >>>>> instead keep them in finally block. >>>>> Can you receive it again. >>>>> http://cr.openjdk.java.net/~tyan/JDK-8032050/webrev.02/ >>>>> Thank you >>>>> Tristan >>>>> >>>>> On 01/29/2014 03:16 PM, Stuart Marks wrote: >>>>>> Hi Tristan, >>>>>> >>>>>> I don't want to put the workaround into >>>>>> ActivationLibrary.rmidRunning() for a >>>>>> null return from the lookup, because this is only a workaround >>>>>> for an actual >>>>>> bug in rmid initialization. See the review I just posted for >>>>>> JDK-8023541. >>>>>> >>>>>> Adding JavaVM.waitFor(timeout) is something that would be useful >>>>>> in general, >>>>>> but it needs to be handled carefully. It uses the new >>>>>> Process.waitFor(timeout, unit) which is new in Java SE 8; this makes >>>>>> backporting to 7 more complicated. Not clear whether we'll do so, >>>>>> but I don't >>>>>> want to forclose the opportunity without discussion. It's also >>>>>> not clear how >>>>>> one can get the vm's exit status after JavaVM.waitFor() has >>>>>> returned true. >>>>>> With the Process API it's possible simply to call waitFor() or >>>>>> exitValue(). >>>>>> With JavaVM, a new API needs to be created, or the rule has to be >>>>>> established >>>>>> that one must call JavaVM.waitFor() to collect the exit status as >>>>>> well as to >>>>>> close the pipes from the subprocess. If JavaVM.waitFor(timeout, >>>>>> unit) is >>>>>> called without subsequently calling JavaVM.waitFor(), the pipes >>>>>> are leaked. >>>>>> >>>>>> In ShutdownGracefully.java, the finally-block needs to check to >>>>>> see if rmid >>>>>> is still running, and if it is, to shut it down. Simply calling >>>>>> waitFor(timeout, unit) isn't sufficient, because if the rmid >>>>>> process is still >>>>>> running, it will be left running. >>>>>> >>>>>> The straightforward approach would be to call >>>>>> ActivationLibrary.rmidRunning() >>>>>> to test if it's still running. Unfortunately this isn't quite >>>>>> right, because >>>>>> rmidRunning() has a timeout loop in it -- which should probably >>>>>> be removed. >>>>>> (I think there's a bug for this.) Another approach would be >>>>>> simply to call >>>>>> rmid.destroy(). This calls rmid's shutdown() method first, which is >>>>>> reasonable, but I'm not sure it kills the process if that fails. >>>>>> In any case, >>>>>> this already has a timeout loop waiting for the process to die, so >>>>>> ShutdownGracefully.java needn't use a new waitFor(timeout, unit) >>>>>> call. >>>>>> >>>>>> Removing the commented-out code that starts with "no longer >>>>>> needed" is good, >>>>>> and removing the ShutdownDetectThread is also good, since that's >>>>>> unnecessary. >>>>>> >>>>>> There are some more cleanups I have in mind here but I'd like to >>>>>> see a >>>>>> revised webrev before proceeding. >>>>>> >>>>>> Thanks, >>>>>> >>>>>> s'marks >>>>>> >>>>>> On 1/25/14 8:57 PM, Tristan Yan wrote: >>>>>>> Hi Stuart >>>>>>> Thank you for your review and suggestion. >>>>>>> Yes, since this failure mode is very hard to be reproduced. I >>>>>>> guess it's >>>>>>> make sense to do some hack. And I also noticed in >>>>>>> ActivationLibrary.rmidRunning. It does try to look up >>>>>>> ActivationSystem but >>>>>>> doesn't check if it's null. So I add the logic to make sure we >>>>>>> will look up >>>>>>> the non-null ActivationSystem. Also I did some cleanup if you >>>>>>> don't mind. >>>>>>> Add a waitFor(long timeout, TimeUnit unit) for JavaVM. Which we >>>>>>> can have a >>>>>>> better waitFor control. >>>>>>> I appreciate you can review the code again. >>>>>>> http://cr.openjdk.java.net/~tyan/JDK-8032050/webrev.01/ >>>>>>> Thank you >>>>>>> Tristan >>>>>>> >>>>>>> >>>>>>> On 01/25/2014 10:20 AM, Stuart Marks wrote: >>>>>>>> On 1/23/14 10:34 PM, Tristan Yan wrote: >>>>>>>>> Hi All >>>>>>>>> Could you review the bug fix for JDK-8032050. >>>>>>>>> >>>>>>>>> http://cr.openjdk.java.net/~tyan/JDK-8032050/webrev.00/ >>>>>>>>> >>>>>>>>> Description: >>>>>>>>> This rare happened failure caused because when RMID starts. It >>>>>>>>> don't >>>>>>>>> guarantee >>>>>>>>> sun.rmi.server.Activation.startActivation finishes. >>>>>>>>> Fix by adding a iterative getSystem with a 5 seconds timeout. >>>>>>>> >>>>>>>> Hi Tristan, >>>>>>>> >>>>>>>> Adding a timing/retry loop into this test isn't the correct >>>>>>>> approach for >>>>>>>> fixing this test. >>>>>>>> >>>>>>>> The timing loop isn't necessary because there is already a >>>>>>>> timing loop in >>>>>>>> RMID.start() in the RMI test library. (There's another timing >>>>>>>> loop in >>>>>>>> ActivationLibrary.rmidRunning() which should probably be >>>>>>>> removed.) So the >>>>>>>> intent of this library call is that it start rmid and wait for >>>>>>>> it to become >>>>>>>> ready. That logic doesn't need to be added to the test. >>>>>>>> >>>>>>>> In the bug report JDK-8032050 you had mentioned that the >>>>>>>> NullPointerException was suspicious. You're right! I took a >>>>>>>> look and it >>>>>>>> seemed like it was related to JDK-8023541, and I added a note >>>>>>>> to this >>>>>>>> effect to the bug report. The problem here is that rmid can >>>>>>>> come up and >>>>>>>> transiently return null instead of the stub of the activation >>>>>>>> system. >>>>>>>> That's what JDK-8023541 covers. I think that rmid itself needs >>>>>>>> to be fixed, >>>>>>>> though modifying the timing loop in the RMI test library to >>>>>>>> wait for rmid >>>>>>>> to come up *and* for the lookup to return non-null is an easy >>>>>>>> way to fix >>>>>>>> the problem. (Or at least cover it up.) >>>>>>>> >>>>>>>> The next step in the analysis is to determine, given that >>>>>>>> ActivationLibrary.getSystem can sometimes return null, whether >>>>>>>> this has >>>>>>>> actually caused this test failure. This is pretty easy to >>>>>>>> determine; just >>>>>>>> hack in a line "system = null" in the right place and run the >>>>>>>> test. I've >>>>>>>> done this, and the test times out and the output log is pretty >>>>>>>> much >>>>>>>> identical to the one in the bug report. (I recommend you try this >>>>>>>> yourself.) So I think it's fairly safe to say that the problem in >>>>>>>> JDK-8023541 has caused the failure listed in JDK-8032050. >>>>>>>> >>>>>>>> I can see a couple ways to proceed here. One way is just to >>>>>>>> close this out >>>>>>>> as a duplicate of JDK-8023541 since that bug caused this failure. >>>>>>>> >>>>>>>> Another is that this test could use some cleaning up. This bug >>>>>>>> certainly >>>>>>>> covers a failure, but the messages emitted are confusing and in >>>>>>>> some cases >>>>>>>> completely wrong. For example, the "rmid has shutdown" message >>>>>>>> at line 180 >>>>>>>> is incorrect, because in this case rmid is still running and >>>>>>>> the wait() >>>>>>>> call has timed out. Most of the code here can be replaced with >>>>>>>> calls to >>>>>>>> various bits of the RMI test library. There are a bunch of >>>>>>>> other things in >>>>>>>> this test that could be cleaned up as well. >>>>>>>> >>>>>>>> It's up to you how you'd like to proceed. >>>>>>>> >>>>>>>> s'marks >>>>>>> >>>>>> >>>>> >>>> >> From Alan.Bateman at oracle.com Thu Feb 13 08:34:54 2014 From: Alan.Bateman at oracle.com (Alan Bateman) Date: Thu, 13 Feb 2014 08:34:54 +0000 Subject: RFR (S): JDK-8034000 lack of /othervm option can cause some RMI tests to fail In-Reply-To: <52FC6F50.6010901@oracle.com> References: <52FC6F50.6010901@oracle.com> Message-ID: <52FC83AE.7000100@oracle.com> On 13/02/2014 07:08, Stuart Marks wrote: > > > Webrev: > > http://cr.openjdk.java.net/~smarks/reviews/8034000/webrev.0/ Looks good to me too. -Alan From bourges.laurent at gmail.com Thu Feb 13 09:09:30 2014 From: bourges.laurent at gmail.com (=?ISO-8859-1?Q?Laurent_Bourg=E8s?=) Date: Thu, 13 Feb 2014 10:09:30 +0100 Subject: Fwd: Re: Fwd: Improve large array allocation / gc & intrinsics In-Reply-To: <1392114510.2707.33.camel@cirrus> References: <1392114510.2707.33.camel@cirrus> Message-ID: Comments & opinions from core-libs, please ? Laurent ---------- Message transf?r? ---------- De : "Thomas Schatzl" Date : 11 f?vr. 2014 11:28 Objet : Re: Fwd: Improve large array allocation / gc & intrinsics ? : "Laurent Bourg?s" Cc : > Hi, > > just a few comments... > > > De : "Laurent Bourg?s" > > Date : 10 f?vr. 2014 10:24 > > Objet : Improve large array allocation / gc & intrinsics > > ? : "core-libs-dev" , "discuss" < > > discuss at openjdk.java.net> > > Cc : > > > > > Dear all, > > > > > > I would like to propose a JDK9 RFE to improve JVM efficiency when > > > dealing with large arrays (allocation + gc). > > > > > > In several scientific applications (and my patched java2d pisces), > > > many large arrays are needed, created on the fly and it becomes very > > > painful to recycle them using an efficient array cache (concurrency, > > > cache size tuning, clear + cache eviction issues). > > Why do you think that a one-size fits all approach that any library in > the JDK will not have the same issues to deal with? How do you know that > a generic library in the JDK (as in your proposal) or hacking things > into the VM can deal with this problem better than you who knows the > application allocation patterns? > > Do you have a prototype (library) for that? > > > > In this case, the GC overhead leads to a big performance penalty > > > (several hundred megabytes per seconds). > > I do not understand what the problem is. Typically I would not specify > performance (throughput?) penalty in megabytes per seconds. > > Do the GCs take too long, or do you feel there too much memory wastage > somewhere? > > > > As such array cache are very efficient in an application context, I am > > > wondering if that approach could be promoted at the JDK level itself: > > > > > > - provide a new array allocator for large arrays that can return > > > larger arrays than expected (size = 4 or 8 multiples) using array > > > caches (per thread ?) stored in a dedicated JVM large memory area > > The behavior you propose seems very particular to your application. > Others may have completely different requirements. The mentioned > per-thread caches do not seem to be problematic to do in a library > either. > > > > (GC aware) and providing efficient cache eviction policies > > Did you every try one of the other available garbage collectors with > your application? Both CMS and G1 never move large objects around, i.e. > there is almost no direct GC overhead associated with them. > > Reclaiming them is almost zero cost in these collectors. Keeping them > alive seems to be best handled by logic in a library. > > Can you give examples where the VM has significant advantages over a > dedicated library, or a particular use case with measurements that show > this could be the case? > > > > - may support for both clean arrays (zero filled) or dirty arrays > > > because some algorithms does not need zero-filled arrays. > > > > > > - improve JVM intrinsics (array clear, fill) to achieve maximum > > > performance ie take into account the data alignment (4 or 8 multiples) > > I think the compiler already uses specialized methods for that, using > the "best" instructions that are available. It should also already be > able to detect "fill" loops, and vectorize them. > > Objects are always 8 byte aligned - I think you can force higher > alignments by setting ObjectAlignmentInBytes or so. > > Otherwise these changes could be simply added, i.e. seems to not need > any RFE. > > > > - upgrade GC to recycle such 'cached' arrays (clean), update usage > > > statistics and manage cache eviction policy (avoid wasting memory) > > The GCs already automatically recycle the freed space. Everything else > seems to be more complicated to implement at VM level than at library > level, with the added drawback that you add a VM dependency. > > > > Please give me your feedback & opinion and evaluate if this feature > > > seems possible to implement into the JDK (hotspot, gc, core-libs)... > > Thanks, > Thomas > > From daniel.fuchs at oracle.com Thu Feb 13 11:01:31 2014 From: daniel.fuchs at oracle.com (Daniel Fuchs) Date: Thu, 13 Feb 2014 12:01:31 +0100 Subject: RFR (JAXP): 8033980 : Xerces Update: datatype XMLGregorianCalendarImpl and DurationImpl In-Reply-To: <52FC7FE6.2060400@oracle.com> References: <52F9666F.9010303@oracle.com> <5111B487-22C1-4987-B9CC-ABF2F5CDD964@oracle.com> <52FA5855.3020106@oracle.com> <52FA7270.4090709@oracle.com> <52FAF801.5060300@oracle.com> <52FBCF66.8060308@oracle.com> <52FC7FE6.2060400@oracle.com> Message-ID: <52FCA60B.1010500@oracle.com> Hi Joe, Couldn't all the new orig_* fields in XMLGregorianCalendarImpl be made transient? It looks as if they shouldn't be serialized anyway. Should they? Not making them transient would change the serialization form, and I'm not sure where that would take us... best regards, -- daniel On 2/13/14 9:18 AM, huizhe wang wrote: > Hi Alan, Lance, and Daniel, > > The Xerces serialization revision meant to create a serialization form > that would help maintain future serialization compatibility. But in > reality it itself is causing significant incompatibility as Alan pointed > out below and we discussed previously. I've removed the revision from > the patch as a result. > > Please see the new webrev here: > http://cr.openjdk.java.net/~joehw/jdk9/8033980/webrev/ > > Thanks, > Joe > > On 2/12/2014 11:45 AM, Alan Bateman wrote: >> On 12/02/2014 04:26, huizhe wang wrote: >>> Hi Lance, Daniel, >>> >>> I suggest we take this incompatibility and document it in the release >>> notes (we'll likely have more). I reversed DurationImpl and then >>> realized why the Xerces engineers made the incompatibility change >>> when I started on XMLGregorianCalendarImpl. It was because >>> XMLGregorianCalendarImpl did not implement what was recommended, >>> using the lexical form for serialization. In the original >>> implementation therefore, the impl between XMLGregorianCalendarImpl >>> and DurationImpl was not consistent. >>> >>> Keeping this Xerces revision makes the serialization consistent in >>> both classes and adds the benefit of having the same source as that >>> of Xerces. >> I think we might need to hold the horses and understand this one and >> compatibility impact a bit more. >> >> As I understand, XMLGregorianCalendar is not Serializable but the >> Xerces implementation (XMLGregorianCalendarImpl) is. If someone is >> using the JDK and hasn't configured an alternative DatatypeFactory >> then it's an instance of >> com.sun.org.apache.xerces.internal.jaxp.datatype.XMLGregorianCalendarImpl >> that is serialized today. Do I have this right? I assume there is no >> serialization interop between this and someone using vanilla Xerces as >> the other end will be dealing with org.apache classes. >> >> With the proposed change then you've added a writeReplace so that >> going forward it will write a >> com.sun.org.apache.xerces.internal.jaxp.datatype.SerializedDuration >> into the stream. This isn't going to work if you've got a JDK 8 or >> older on the other end, right? Also I assume this doesn't help the >> Xerces interop because the fully qualified class names are still >> different. >> >> XMLGregorianCalendarImpl doesn't appear to have a readObject or other >> serialization methods so I assume that if you don't change the >> serialVersionUID then it would at least be possible to deserialize >> something that was serialized by an older JDK. As you've added a >> writeReplace then it makes me wonder why the serialVersionUID of >> XMLGregorianCalendarImpl has changed. Maybe that part could be >> reversed and leave the long standing value? >> >> One thing that might help is to develop a number serialization tests >> to help better understand the impact of the change. >> >> -Alan. >> >> > From fweimer at redhat.com Thu Feb 13 11:05:05 2014 From: fweimer at redhat.com (Florian Weimer) Date: Thu, 13 Feb 2014 12:05:05 +0100 Subject: JEP 187: Serialization 2.0 In-Reply-To: <9CBDDD37-0DA1-4B9B-BA2F-0391E568227A@oracle.com> References: <20140114002659.6AC0BD7C2@eggemoggin.niobe.net> <52DFCE40.4010108@redhat.com> <52DFDA07.2020307@oracle.com> <52DFE05A.40901@redhat.com> <9CBDDD37-0DA1-4B9B-BA2F-0391E568227A@oracle.com> Message-ID: <52FCA6E1.8030908@redhat.com> On 01/23/2014 03:30 PM, Chris Hegarty wrote: > On 22 Jan 2014, at 15:14, Florian Weimer wrote: > >> On 01/22/2014 03:47 PM, Chris Hegarty wrote: >>> On 22/01/14 13:57, Florian Weimer wrote: >>>> On 01/14/2014 01:26 AM, mark.reinhold at oracle.com wrote: >>>>> Posted: http://openjdk.java.net/jeps/187 >>>> >>>> There's another aspect of the current approach to serialization that is >>>> not mentioned: the type information does not come from the calling >>>> context, but exclusively from the input stream. >>> >>> Have you overlooked resolveClass [1], or are you looking for additional >>> context? >> >> I mean something slightly different, so here's a concrete example: Assume we are deserializing an instance of a class with a String field. The current framework deserializes whatever is in the input stream, and will bail out with a ClassCastException if the deserialized object turns out unusable. Contrast this with an alternative approach that uses the knowledge that the field String and will refuse to read anything else from the input stream. > > Sorry, but I may still be missing your point. From my reading of the code, CCE will only be thrown if the object being deserialized contains a field with a type that does not match the type of the similarly named field in the class loaded by the deserializing context. This is from the method ObjectStreamClass.FieldReflector::setObjFieldValues(Object, Object[]), called indirectly from ObjectInputStream::readObject0(boolean): Object val = vals[offsets[i]]; if (val != null && !types[i - numPrimFields].isInstance(val)) { Field f = fields[i].getField(); throw new ClassCastException( "cannot assign instance of " + val.getClass().getName() + " to field " + f.getDeclaringClass().getName() + "." + f.getName() + " of type " + f.getType().getName() + " in instance of " + obj.getClass().getName()); } So the exception is thrown after the object val has been read from the stream, which involves its construction and deserialization. This means that you always expose the full serialization functionality, no matter with which types you start. > Are you thinking specifically about corrupt/malicious streams, or evolving serializable types? Or a performance optimisation? Mostly the former. Sorry, I should have said so. There are serialization frameworks which restrict the types of deserializable objects based on the type of the top-level object being requested. Or in Java serialization terms, they won't load and instantiate arbitrary classes even if the programmer did not provide a customer, restrictive resolveClass() implementation. I get that type-erased generics make it complicated to achieve this in current Java. (But so is writing a correct resolveClass() because it breaks encapsulation.) Knowing in advance which types to expect in the stream would allow for a performance optimization, but something like that could be added to the current framework as well. -- Florian Weimer / Red Hat Product Security Team From Alan.Bateman at oracle.com Thu Feb 13 12:15:18 2014 From: Alan.Bateman at oracle.com (Alan Bateman) Date: Thu, 13 Feb 2014 12:15:18 +0000 Subject: RFR for JDK-8031563: TEST_BUG: java/nio/channels/Selector/ChangingInterests.java failed once In-Reply-To: <52FB7868.3080907@oracle.com> References: <52FB60AB.4070708@oracle.com> <52FB7868.3080907@oracle.com> Message-ID: <52FCB756.5080906@oracle.com> On 12/02/2014 13:34, michael cui wrote: > : > Comments from Shura : > I have concern about two areas which might make your version > and original version not equivalent. > one is code changes in verification logic are different. > the other is old version called selectNow() only once, your > version replace it by calling it repeatedly in worst case. > > Both fixes were tested on 4 platforms (win, linux, mac, solaris)for > 2000 runs without seeing any error. > > Any suggestion? > I plan to look at this soon but I have a concern that it is changing the test so that the change from OP_READ to OP_READ|OP_WRITE no longer tests that the key is both readable and writable after one select. -Alan From Alan.Bateman at oracle.com Thu Feb 13 14:23:55 2014 From: Alan.Bateman at oracle.com (Alan Bateman) Date: Thu, 13 Feb 2014 14:23:55 +0000 Subject: RFR (JAXP): 8033980 : Xerces Update: datatype XMLGregorianCalendarImpl and DurationImpl In-Reply-To: <52FC7FE6.2060400@oracle.com> References: <52F9666F.9010303@oracle.com> <5111B487-22C1-4987-B9CC-ABF2F5CDD964@oracle.com> <52FA5855.3020106@oracle.com> <52FA7270.4090709@oracle.com> <52FAF801.5060300@oracle.com> <52FBCF66.8060308@oracle.com> <52FC7FE6.2060400@oracle.com> Message-ID: <52FCD57B.8090807@oracle.com> On 13/02/2014 08:18, huizhe wang wrote: > Hi Alan, Lance, and Daniel, > > The Xerces serialization revision meant to create a serialization form > that would help maintain future serialization compatibility. But in > reality it itself is causing significant incompatibility as Alan > pointed out below and we discussed previously. I've removed the > revision from the patch as a result. > > Please see the new webrev here: > http://cr.openjdk.java.net/~joehw/jdk9/8033980/webrev/ Thanks for dropping the serialization change as it was just not going to work the way you had intended. I agree with Daniel's comment about all the new fields added to XMLGregorianCalendarImpl as it's not clear why they aren't transient. I have not studied the rest of the changes but I think Daniel and Lance are. -Alan From chris.hegarty at oracle.com Thu Feb 13 16:29:32 2014 From: chris.hegarty at oracle.com (Chris Hegarty) Date: Thu, 13 Feb 2014 16:29:32 +0000 Subject: A hole in the serialization spec In-Reply-To: <52FB9248.6010703@redhat.com> References: <52F4F6BB.2010207@redhat.com> <52F8E807.2090403@oracle.com> <52F8F21C.80205@redhat.com> <52FB8E5C.9020502@oracle.com> <52FB9248.6010703@redhat.com> Message-ID: <26199217-BAE6-47AC-AFC8-4DE90726B8C0@oracle.com> On 12 Feb 2014, at 15:24, David M. Lloyd wrote: > That's a quote from the serialization spec. I take it to mean, "Don't write fields and everything might go to hell". In practice, if the reading side doesn't read fields, things end up more or less OK, as evidenced by various classes in the wild. But it's not hard to imagine a scenario in which a class change could cause protocol corruption. > > I think the specifics of the quote relate to this kind of class change; in particular, if a class is deleted from the hierarchy on the read side, and that class corresponds to the class that had the misbehaving writeObject, I suspect that things will break at that point as the read side will probably try to consume and discard the field information for that class, which will be missing (it will start reading the next class' fields instead I think). Yes, possibly. And who knows what fields/values may be read and mistaken for the wrong object in the hierarchy. So ?undefined' behaviour seems right to me. A simple example throws StreamCorruptedException with Oracles JDK: public class NoFields { public static void main(String[] args) throws Exception { ByteArrayOutputStream baos = new ByteArrayOutputStream(); try (ObjectOutputStream oos = new ObjectOutputStream(baos)) { oos.writeObject(new B(5, 10)); } ByteArrayInputStream bais = new ByteArrayInputStream(baos.toByteArray()); ObjectInputStream ois = new ObjectInputStream(bais); B b = (B)ois.readObject(); System.out.println("aValue = " + b.aValue); System.out.println("bValue = " + b.bValue); } static class A implements Serializable { final int aValue; A(int value) { this.aValue = value; } private void writeObject(ObjectOutputStream oos) throws IOException { oos.defaultWriteObject(); // <<< comment out } } static class B extends A implements Serializable { final int bValue; B(int aValue, int bValue) { super(aValue); this.bValue = bValue; } } -Chris. > > On 02/12/2014 09:08 AM, Chris Hegarty wrote: >> David, others? >> >> I'm afraid I'm still not clear what is mean by: >> " ... undefined in cases where the ObjectInputStream cannot resolve >> the class which defined the writeObject method in question." >> >> This does not seem directly related to the issue we are discussing ( >> whether to invoke default read/write object ). >> >> -Chris. >> >> On 10/02/14 15:37, David M. Lloyd wrote: >>> I agree that it's a problem; however it's also clear that there are many >>> classes in the wild which have this problem. It would be nice if the >>> behavior could _become_ defined *somehow* though. I can see at least >>> four options: >>> >>> 1) do nothing :( >>> 2) start throwing (or writing) an exception in write/readObject when >>> stream ops are performed without reading fields (maybe can be disabled >>> with a sys prop or something) >>> 3) leave fields cleared and risk protocol issues >>> 4) silently start reading/writing empty field information (risks >>> protocol issues) >>> >>> Maybe there are better options I'm not thinking of. >>> >>> On 02/10/2014 08:53 AM, Chris Hegarty wrote: >>>> David, >>>> >>>> " ... undefined in cases where the ObjectInputStream cannot resolve the >>>> class which defined the writeObject method in question." >>>> >>>> I'm not clear as to what this statement is about? >>>> >>>> I'm sure you already know this, and maybe in your environment do not >>>> care much about it, but having a read/writeObject not invoke the >>>> appropriate default read/write Object/Fields method is a serious >>>> impediment to evolving the serial form ( in a compatible way ). For >>>> example, if your class has no serializable fields in one revision, but >>>> adds a serializable field(s) in a subsequent revision. This could lead >>>> to a StreamCorruptedException, or some other undefined behavior. >>>> >>>> The OpenJDK sources do seem to be quite tolerant of this situation. I'm >>>> not entirely sure if this is a good or a bad thing. That said, I don't >>>> think we want to encourage this kind of behavior. >>>> >>>> -Chris. >>>> >>>> On 07/02/14 15:07, David M. Lloyd wrote: >>>>> Since the topic of serialization has come up recently, I'll take it as >>>>> an excuse to bring up a problem that I've run into a couple of times >>>>> with the serialization specification, which has resulted in user >>>>> problems. >>>>> >>>>> If you read section 2.3 [1] of the specification, it says: >>>>> >>>>> "The class's writeObject method, if implemented, is responsible for >>>>> saving the state of the class. Either ObjectOutputStream's >>>>> defaultWriteObject or writeFields method must be called once (and only >>>>> once) before writing any optional data that will be needed by the >>>>> corresponding readObject method to restore the state of the object; >>>>> even >>>>> if no optional data is written, defaultWriteObject or writeFields must >>>>> still be invoked once. If defaultWriteObject or writeFields is not >>>>> invoked once prior to the writing of optional data (if any), then the >>>>> behavior of instance deserialization is undefined in cases where the >>>>> ObjectInputStream cannot resolve the class which defined the >>>>> writeObject >>>>> method in question." >>>>> >>>>> If you go to section 3.4 [2] of the specification, it reads: >>>>> >>>>> "The readObject method of the class, if implemented, is responsible for >>>>> restoring the state of the class. The values of every field of the >>>>> object whether transient or not, static or not are set to the default >>>>> value for the fields type. Either ObjectInputStream's defaultReadObject >>>>> or readFields method must be called once (and only once) before reading >>>>> any optional data written by the corresponding writeObject method; even >>>>> if no optional data is read, defaultReadObject or readFields must still >>>>> be invoked once." >>>>> >>>>> Now the problem: there are many classes in the wild which nevertheless >>>>> do not write/read fields. We cause an exception in such cases rather >>>>> than make up some undefined behavior. What I'm wondering is, is there >>>>> some sensible behavior that could be specified for this case? The >>>>> Oracle JDK seems to simply leave fields uninitialized in this case, >>>>> maybe that can be a specified behavior? >>>>> >>>>> [1] >>>>> http://docs.oracle.com/javase/7/docs/platform/serialization/spec/output.html#861 >>>>> >>>>> >>>>> >>>>> >>>>> [2] >>>>> http://docs.oracle.com/javase/7/docs/platform/serialization/spec/input.html#2971 >>>>> >>>>> >>>>> >>>>> >>>>> >>> >>> > > > -- > - DML From david.lloyd at redhat.com Thu Feb 13 17:38:48 2014 From: david.lloyd at redhat.com (David M. Lloyd) Date: Thu, 13 Feb 2014 11:38:48 -0600 Subject: A hole in the serialization spec In-Reply-To: <26199217-BAE6-47AC-AFC8-4DE90726B8C0@oracle.com> References: <52F4F6BB.2010207@redhat.com> <52F8E807.2090403@oracle.com> <52F8F21C.80205@redhat.com> <52FB8E5C.9020502@oracle.com> <52FB9248.6010703@redhat.com> <26199217-BAE6-47AC-AFC8-4DE90726B8C0@oracle.com> Message-ID: <52FD0328.3050304@redhat.com> On 02/13/2014 10:29 AM, Chris Hegarty wrote: > On 12 Feb 2014, at 15:24, David M. Lloyd > wrote: > >> That's a quote from the serialization spec. I take it to mean, >> "Don't write fields and everything might go to hell". In practice, >> if the reading side doesn't read fields, things end up more or less >> OK, as evidenced by various classes in the wild. But it's not hard >> to imagine a scenario in which a class change could cause protocol >> corruption. >> >> I think the specifics of the quote relate to this kind of class >> change; in particular, if a class is deleted from the hierarchy on >> the read side, and that class corresponds to the class that had the >> misbehaving writeObject, I suspect that things will break at that >> point as the read side will probably try to consume and discard the >> field information for that class, which will be missing (it will >> start reading the next class' fields instead I think). > > Yes, possibly. And who knows what fields/values may be read and > mistaken for the wrong object in the hierarchy. So ?undefined' > behaviour seems right to me. I think the behavior is well-defined, just "bad", which is my point. If the exact current is spec'd out as-is then at least we can be assured of the same bad behavior across implementations. If the behavior is changed such that fields are read/written but discarded, without updating the spec, then the "undefined" behavior at least becomes safer. If the behavior is changed, *and* the spec is updated, then we get both benefits, but at the cost that all previous implementations will not be compliant with the spec. All options seem to have a cost though. -- - DML From rob.mckenna at oracle.com Thu Feb 13 17:54:58 2014 From: rob.mckenna at oracle.com (Rob McKenna) Date: Thu, 13 Feb 2014 17:54:58 +0000 Subject: RFR: 7152892: some jtreg tests fail with permission denied In-Reply-To: <52F8F395.3070008@oracle.com> References: <52F52990.5060200@oracle.com> <52F8F395.3070008@oracle.com> Message-ID: <52FD06F2.7050104@oracle.com> Sorry Mandy, I totally missed your reply. I'll get this resolved separately. -Rob On 10/02/14 15:43, Mandy Chung wrote: > > On 2/7/2014 10:44 AM, Rob McKenna wrote: >> Hi folks, >> >> When files are copied by our test harness the permissions are left >> unchanged. This can cause trouble for a few tests when the test >> folder is read only. >> >> http://cr.openjdk.java.net/~robm/7152892/webrev.01/ > > AIX is missing in your change in Assert.sh. Assert.sh doesn't > seem to be necessary. I think it can replace the shell test > with @build to compile the classes. It may also need to change > "java Assert" with the right classpath (I haven't checked in details) > in Assert.java (which execs the java command). Do you mind looking > into the possibility to remove Assert.sh? > > $ hg diff Assert.java > diff --git a/test/java/lang/ClassLoader/Assert.java > b/test/java/lang/ClassLoader/Assert.java > --- a/test/java/lang/ClassLoader/Assert.java > +++ b/test/java/lang/ClassLoader/Assert.java > @@ -24,7 +24,8 @@ > /* > * @test > * @bug 4290640 4785473 > - * @run shell/timeout=300 Assert.sh > + * @build package1.Class1 package2.Class2 package1.package3.Class3 > Assert > + * @run main/othervm Assert > * @summary Test the assertion facility > * @author Mike McCloskey > */ > > > Mandy > From mandy.chung at oracle.com Thu Feb 13 18:20:18 2014 From: mandy.chung at oracle.com (Mandy Chung) Date: Thu, 13 Feb 2014 10:20:18 -0800 Subject: RFR: 7152892: some jtreg tests fail with permission denied In-Reply-To: <52FD06F2.7050104@oracle.com> References: <52F52990.5060200@oracle.com> <52F8F395.3070008@oracle.com> <52FD06F2.7050104@oracle.com> Message-ID: <52FD0CE2.5020200@oracle.com> On 2/13/14 9:54 AM, Rob McKenna wrote: > Sorry Mandy, I totally missed your reply. I'll get this resolved > separately. > No worries. It's fine to do it separately. Mandy > -Rob > > On 10/02/14 15:43, Mandy Chung wrote: >> >> On 2/7/2014 10:44 AM, Rob McKenna wrote: >>> Hi folks, >>> >>> When files are copied by our test harness the permissions are left >>> unchanged. This can cause trouble for a few tests when the test >>> folder is read only. >>> >>> http://cr.openjdk.java.net/~robm/7152892/webrev.01/ >> >> AIX is missing in your change in Assert.sh. Assert.sh doesn't >> seem to be necessary. I think it can replace the shell test >> with @build to compile the classes. It may also need to change >> "java Assert" with the right classpath (I haven't checked in details) >> in Assert.java (which execs the java command). Do you mind looking >> into the possibility to remove Assert.sh? >> >> $ hg diff Assert.java >> diff --git a/test/java/lang/ClassLoader/Assert.java >> b/test/java/lang/ClassLoader/Assert.java >> --- a/test/java/lang/ClassLoader/Assert.java >> +++ b/test/java/lang/ClassLoader/Assert.java >> @@ -24,7 +24,8 @@ >> /* >> * @test >> * @bug 4290640 4785473 >> - * @run shell/timeout=300 Assert.sh >> + * @build package1.Class1 package2.Class2 package1.package3.Class3 >> Assert >> + * @run main/othervm Assert >> * @summary Test the assertion facility >> * @author Mike McCloskey >> */ >> >> >> Mandy >> > From lance.andersen at oracle.com Thu Feb 13 19:59:26 2014 From: lance.andersen at oracle.com (Lance Andersen) Date: Thu, 13 Feb 2014 14:59:26 -0500 Subject: RFR 8034896, clean up typo in Clob.free Message-ID: Hi all, Need a reviewer for this trivial change to remove the redundant "the resources" from Clob.free. I will be going through all of the java[x]/sql classes at a later time to clean up the use of so I did not touch for now. Best Lance hg diff diff -r 4711a64b6a13 src/share/classes/java/sql/Clob.java --- a/src/share/classes/java/sql/Clob.java Thu Feb 13 17:40:43 2014 +0100 +++ b/src/share/classes/java/sql/Clob.java Thu Feb 13 14:55:33 2014 -0500 @@ -1,5 +1,5 @@ /* - * Copyright (c) 1998, 2013, Oracle and/or its affiliates. All rights reserved. + * Copyright (c) 1998, 2014, Oracle and/or its affiliates. All rights reserved. * DO NOT ALTER OR REMOVE COPYRIGHT NOTICES OR THIS FILE HEADER. * * This code is free software; you can redistribute it and/or modify it @@ -309,8 +309,8 @@ void truncate(long len) throws SQLException; /** - * This method frees the Clob object and releases the resources the resources - * that it holds. The object is invalid once the free method + * This method releases the resources that the Clob object + * holds. The object is invalid once the free method * is called. *

* After free has been called, any attempt to invoke a Best Lance From daniel.fuchs at oracle.com Thu Feb 13 20:59:58 2014 From: daniel.fuchs at oracle.com (Daniel Fuchs) Date: Thu, 13 Feb 2014 21:59:58 +0100 Subject: RFR 8034896, clean up typo in Clob.free In-Reply-To: References: Message-ID: <52FD324E.5050807@oracle.com> Hi Lance, Looks good, -- daniel On 2/13/14 8:59 PM, Lance Andersen wrote: > Hi all, > > Need a reviewer for this trivial change to remove the redundant "the resources" from Clob.free. > > > I will be going through all of the java[x]/sql classes at a later time to clean up the use of so I did not touch for now. > > Best > Lance > > > hg diff > diff -r 4711a64b6a13 src/share/classes/java/sql/Clob.java > --- a/src/share/classes/java/sql/Clob.java Thu Feb 13 17:40:43 2014 +0100 > +++ b/src/share/classes/java/sql/Clob.java Thu Feb 13 14:55:33 2014 -0500 > @@ -1,5 +1,5 @@ > /* > - * Copyright (c) 1998, 2013, Oracle and/or its affiliates. All rights reserved. > + * Copyright (c) 1998, 2014, Oracle and/or its affiliates. All rights reserved. > * DO NOT ALTER OR REMOVE COPYRIGHT NOTICES OR THIS FILE HEADER. > * > * This code is free software; you can redistribute it and/or modify it > @@ -309,8 +309,8 @@ > void truncate(long len) throws SQLException; > > /** > - * This method frees the Clob object and releases the resources the resources > - * that it holds. The object is invalid once the free method > + * This method releases the resources that the Clob object > + * holds. The object is invalid once the free method > * is called. > *

> * After free has been called, any attempt to invoke a > > > Best > Lance From roger.riggs at oracle.com Thu Feb 13 21:26:47 2014 From: roger.riggs at oracle.com (roger riggs) Date: Thu, 13 Feb 2014 16:26:47 -0500 Subject: RFR 9: 8034903: Add Logging of Process.start arguments and resulting pid Message-ID: <52FD3897.7070903@oracle.com> Hi, Having folks stumbling over process creation and problems of quoting, especially on windows, it seems useful to log the native commands and arguments. They are proposed to be logged using the PlatformLogger at Level.FINE which will not be logged by default. The environment is useful in some cases, but verbose, that it should be Logged at FINER. The pid of the spawned process logged as well for traceability. Please take a look at this first draft and comment on whether it is useful, a good idea and any improvements needed. Webrev: http://cr.openjdk.java.net/~rriggs/webrev-log-processcreate/ Thanks, Roger From pbenedict at apache.org Thu Feb 13 21:50:54 2014 From: pbenedict at apache.org (Paul Benedict) Date: Thu, 13 Feb 2014 15:50:54 -0600 Subject: RFR 9: 8034903: Add Logging of Process.start arguments and resulting pid In-Reply-To: <52FD3897.7070903@oracle.com> References: <52FD3897.7070903@oracle.com> Message-ID: Roger, I only have two suggested improvements: 1) solaris/../ProcessImpl.java should use Objects.toString() rather than the tenary operator for a string choice. You did this alright in /windows/../ProcessImpl.java 2) /windows/../ProcessImpl.java doesn't need to specify "null" for Objects.toString() since that is its default output for null input. Cheers, Paul On Thu, Feb 13, 2014 at 3:26 PM, roger riggs wrote: > Hi, > > Having folks stumbling over process creation and problems of quoting, > especially on windows, it seems useful to log the native commands and > arguments. > They are proposed to be logged using the PlatformLogger at Level.FINE > which will not be logged by default. The environment is useful in some > cases, > but verbose, that it should be Logged at FINER. The pid of the spawned > process > logged as well for traceability. > > Please take a look at this first draft and comment on whether it is useful, > a good idea and any improvements needed. > > Webrev: > http://cr.openjdk.java.net/~rriggs/webrev-log-processcreate/ > > Thanks, Roger > > -- Cheers, Paul From philip.race at oracle.com Thu Feb 13 21:50:23 2014 From: philip.race at oracle.com (Phil Race) Date: Thu, 13 Feb 2014 13:50:23 -0800 Subject: RFR: (8031737) CHECK_NULL and CHECK_EXCEPTION macros cleanup In-Reply-To: References: <52ED36FD.5070507@oracle.com> <1243D3E7-C384-4AA8-80CA-9AA4E053F056@oracle.com> <52F00EAC.8000900@oracle.com> <52FA7429.5070806@oracle.com> <52FA768A.70005@oracle.com> <52FA80A6.2040509@oracle.com> <52FA87E2.9080009@oracle.com> <52FA98A8.7090104@oracle.com> <52FA9AA2.6090005@oracle.com> <52FA9F82.90903@oracle.com> <52FAA211.2070800@oracle.com> <52FAA3F4.6040702@oracle.com> <52FAB577.3070205@oracle.com> Message-ID: <52FD3E1F.4070006@oracle.com> That worked on Mac but I just found it doesn't build on Linux because a macro-redefinition warning is treated as an error there. https://bugs.openjdk.java.net/browse/JDK-8034912 The early/original fix had removed the duplicate definition in src/share/native/java/net/net_util.h So the fix is to do the same in 8u :- ~/jdk8u-dev/jdk$ hg diff src/share/native/java/net/net_util.h diff --git a/src/share/native/java/net/net_util.h b/src/share/native/java/net/net_util.h --- a/src/share/native/java/net/net_util.h +++ b/src/share/native/java/net/net_util.h @@ -42,9 +42,6 @@ #define NET_ERROR(env, ex, msg) \ { if (!(*env)->ExceptionOccurred(env)) JNU_ThrowByName(env, ex, msg) } -#define CHECK_NULL(x) if ((x) == NULL) return; -#define CHECK_NULL_RETURN(x, y) if ((x) == NULL) return y; - /************************************************************************ * Cached field IDs * A little surprised no else else found this already (did they?) Anyway I need a review and Ok to push 8034912 to JDk 8 u -Phil. On 02/12/2014 10:21 AM, Mike Duigou wrote: > This looks fine. > > > On Feb 11 2014, at 15:42 , Phil Race wrote: > >> Here's a JDk8u webrev : -http://cr.openjdk.java.net/~prr/8031737.8u/ >> >> -phil. >> >> On 2/11/14 2:28 PM, Phil Race wrote: >>> So since hg export/import doesn't apply cleanly and the dependency >>> chain seems, long and in order to have some consistency across the releases, >>> I think I should prepare a webrev which essentially backports 8031737 >>> including its small changes to Version.c, if only because otherwise >>> I'd have to have a new bug ID that would not be forwarded ported >>> (one source of confusion) or even worse re-use 8031737 but not fully implement it >>> >>> Agreed ? >>> >>> -phil. >>> From stuart.marks at oracle.com Thu Feb 13 22:50:31 2014 From: stuart.marks at oracle.com (Stuart Marks) Date: Thu, 13 Feb 2014 14:50:31 -0800 Subject: RFR (S): JDK-8034000 lack of /othervm option can cause some RMI tests to fail In-Reply-To: <52FC74C2.9000801@oracle.com> References: <52FC6F50.6010901@oracle.com> <52FC74C2.9000801@oracle.com> Message-ID: <52FD4C37.1040507@oracle.com> Thanks guys. And Joe, you know that I *always* look good! :-) s'marks On 2/12/14 11:31 PM, Joe Darcy wrote: > Look good Stuart, > > -Joe > > On 02/12/2014 11:08 PM, Stuart Marks wrote: >> Hi all, >> >> The RMI test directories were removed from TEST.ROOT's othervm.dirs by >> JDK-8031179 so that individual RMI tests could opt-in to othervm themselves. >> Unfortunately, some tests needed othervm but didn't get it, causing them to fail. >> >> This adds othervm to some failing tests, and adds a note to other tests that >> do not have othervm indicating specifically that they don't need othervm. >> >> Bug: >> >> https://bugs.openjdk.java.net/browse/jdk-8034000 >> >> Webrev: >> >> http://cr.openjdk.java.net/~smarks/reviews/8034000/webrev.0/ >> >> Thanks, >> >> s'marks > From stuart.marks at oracle.com Thu Feb 13 23:23:08 2014 From: stuart.marks at oracle.com (Stuart Marks) Date: Thu, 13 Feb 2014 15:23:08 -0800 Subject: RFR for JDK-8030844:sun/rmi/rmic/classpath/RMICClassPathTest.java timeout in same binaries run In-Reply-To: <52FC3A39.6060307@oracle.com> References: <13F6E908-26F2-420A-9E25-AD00DF8E3769@oracle.com> <52F8C146.7060505@oracle.com> <52FAD718.6090103@oracle.com> <539F9CC8-77FA-43C6-A19A-C6623641CA6C@oracle.com> <52FBD026.5030807@oracle.com> <52FC3A39.6060307@oracle.com> Message-ID: <52FD53DC.7070801@oracle.com> Great, pushed: http://hg.openjdk.java.net/jdk9/dev/jdk/rev/2cecdf7fbc90 Having the full changeset in the webrev was quite convenient. I was able to "hg import" it directly. s'marks On 2/12/14 7:21 PM, Tristan Yan wrote: > Thank you Stuart > This is a very nice tutorial. I did try both ways. Adopt your fix doesn't seem > work for me. It still doesn't generate changeset. But without -r option works. > > http://cr.openjdk.java.net/~tyan/JDK-8030844/webrev.02/ > Could you push it for me? > > Tristan > > On 02/13/2014 03:48 AM, Stuart Marks wrote: >> Hi Tristan, >> >> Changes look good. Unfortunately the webrev doesn't contain an actual >> changeset; it just contains a patch. In the webrev header there is the line >> "Patch of Changes". Instead it should say "Changeset". Like this one: >> >> http://cr.openjdk.java.net/~smarks/reviews/7900311/webrev.1/ >> >> Unfortunately webrev doesn't always produce an actual changeset, in particular >> it doesn't if you use webrev -r. >> >> (My example webrev above is a patch that changes webrev to generate the >> changeset even with -r. It hasn't been pushed yet; possibly it will later >> today. Or you can apply my webrev changeset to your local webrev.) >> >> Or, you can just run webrev without -r and it should check "hg outgoing" to >> determine the changesets used in generating the webrev, which does place the >> changeset into the webrev. >> >> Can you regenerate the webrev so that it includes the actual changeset? Sorry, >> this is a small thing, but as someone who is sponsoring changesets, I'd >> appreciate an actual changeset in the webrev instead of having to construct >> one manually. I think other sponsors would appreciate this too. >> >> s'marks >> >> On 2/11/14 6:59 PM, Tristan Yan wrote: >>> Thank you Stuart >>> http://cr.openjdk.java.net/~tyan/JDK-8030844/webrev.01/ >>> Regards >>> Tristan >>> >>> On Feb 12, 2014, at 10:06 AM, Stuart Marks >> > wrote: >>> >>>> Hi, yes, I'll take this one. >>>> >>>> It's slightly odd that this is creating filenames that already have "/" in >>>> them (as opposed to File.separator) but since these files don't actually have >>>> to exist, I suppose it doesn't really matter. >>>> >>>> I'm not convinced that we actually have any evidence that /home/~user is >>>> really causing the hang/timeout (either caused by the automounter hanging on >>>> /home or LDAP or other nameservice lookup on ~user), but this is harmless, and >>>> it'll fix the problem on the off chance that this really is the root cause. >>>> >>>> Tristan, please update the test's @bug tag to add 8030844, create a changeset, >>>> and create a webrev with the changeset in it (as opposed to a bare patch). >>>> I'll then push it for you. >>>> >>>> s'marks >>>> >>>> On 2/10/14 4:08 AM, Alan Bateman wrote: >>>>> On 10/02/2014 10:57, Tristan Yan wrote: >>>>>> Ping: Can anyone give a review on this. >>>>>> Thank you >>>>>> Tristan >>>>> Changing the test so that it doesn't try to /home/~user seems reasonable to >>>>> me. >>>>> >>>>> Stuart - I see you've been sponsoring Tristan's updates to the RMI tests. Are >>>>> you going to take this one too? >>>>> >>>>> -Alan >>>>> >>>>> >>>>>> >>>>>> On Feb 6, 2014, at 5:13 PM, Tristan Yan>>>>> > wrote: >>>>>> >>>>>>> Hi All >>>>>>> >>>>>>> Please help to review a simple fix for >>>>>>> https://bugs.openjdk.java.net/browse/JDK-8030844 >>>>>>> >>>>>>> http://cr.openjdk.java.net/~tyan/JDK-8030844/webrev.00/. >>>>>>> >>>>>>> Description: >>>>>>> Change replace a ?/home/~user" folder with an test source path. Folder >>>>>>> ?/home/~user? cause some problem whenever something wrong with the automount >>>>>>> filesystem or an username lookup problem. >>>>>>> >>>>>>> Thank you >>>>>>> Tristan >>>>> >>> > From Roger.Riggs at Oracle.com Fri Feb 14 01:27:09 2014 From: Roger.Riggs at Oracle.com (Roger Riggs) Date: Thu, 13 Feb 2014 20:27:09 -0500 Subject: RFR: (8031737) CHECK_NULL and CHECK_EXCEPTION macros cleanup In-Reply-To: <52FD3E1F.4070006@oracle.com> References: <52ED36FD.5070507@oracle.com> <1243D3E7-C384-4AA8-80CA-9AA4E053F056@oracle.com> <52F00EAC.8000900@oracle.com> <52FA7429.5070806@oracle.com> <52FA768A.70005@oracle.com> <52FA80A6.2040509@oracle.com> <52FA87E2.9080009@oracle.com> <52FA98A8.7090104@oracle.com> <52FA9AA2.6090005@oracle.com> <52FA9F82.90903@oracle.com> <52FAA211.2070800@oracle.com> <52FAA3F4.6040702@oracle.com> <52FAB577.3070205@oracle.com> <52FD3E1F.4070006@oracle.com> Message-ID: <52FD70ED.5060102@Oracle.com> Hi Phil, There was an earlier commit that updated java.net 8030875: Macros for checking and returning on exceptions Maybe it will apply as a backport too? Roger On 2/13/14 4:50 PM, Phil Race wrote: > That worked on Mac but I just found it doesn't build on Linux because > a macro-redefinition > warning is treated as an error there. > > https://bugs.openjdk.java.net/browse/JDK-8034912 > > The early/original fix had removed the duplicate definition in > src/share/native/java/net/net_util.h > > So the fix is to do the same in 8u :- > > ~/jdk8u-dev/jdk$ hg diff src/share/native/java/net/net_util.h > diff --git a/src/share/native/java/net/net_util.h > b/src/share/native/java/net/net_util.h > --- a/src/share/native/java/net/net_util.h > +++ b/src/share/native/java/net/net_util.h > @@ -42,9 +42,6 @@ > #define NET_ERROR(env, ex, msg) \ > { if (!(*env)->ExceptionOccurred(env)) JNU_ThrowByName(env, ex, msg) } > > -#define CHECK_NULL(x) if ((x) == NULL) return; > -#define CHECK_NULL_RETURN(x, y) if ((x) == NULL) return y; > - > /************************************************************************ > > * Cached field IDs > * > > A little surprised no else else found this already (did they?) > Anyway I need a review and Ok to push 8034912 to JDk 8 u > > -Phil. > > On 02/12/2014 10:21 AM, Mike Duigou wrote: >> This looks fine. >> >> >> On Feb 11 2014, at 15:42 , Phil Race wrote: >> >>> Here's a JDk8u webrev : -http://cr.openjdk.java.net/~prr/8031737.8u/ >>> >>> -phil. >>> >>> On 2/11/14 2:28 PM, Phil Race wrote: >>>> So since hg export/import doesn't apply cleanly and the dependency >>>> chain seems, long and in order to have some consistency across the >>>> releases, >>>> I think I should prepare a webrev which essentially backports 8031737 >>>> including its small changes to Version.c, if only because otherwise >>>> I'd have to have a new bug ID that would not be forwarded ported >>>> (one source of confusion) or even worse re-use 8031737 but not >>>> fully implement it >>>> >>>> Agreed ? >>>> >>>> -phil. >>>> > From mandy.chung at oracle.com Fri Feb 14 01:36:29 2014 From: mandy.chung at oracle.com (Mandy Chung) Date: Thu, 13 Feb 2014 17:36:29 -0800 Subject: RFR: (8031737) CHECK_NULL and CHECK_EXCEPTION macros cleanup In-Reply-To: <52FD3E1F.4070006@oracle.com> References: <52ED36FD.5070507@oracle.com> <1243D3E7-C384-4AA8-80CA-9AA4E053F056@oracle.com> <52F00EAC.8000900@oracle.com> <52FA7429.5070806@oracle.com> <52FA768A.70005@oracle.com> <52FA80A6.2040509@oracle.com> <52FA87E2.9080009@oracle.com> <52FA98A8.7090104@oracle.com> <52FA9AA2.6090005@oracle.com> <52FA9F82.90903@oracle.com> <52FAA211.2070800@oracle.com> <52FAA3F4.6040702@oracle.com> <52FAB577.3070205@oracle.com> <52FD3E1F.4070006@oracle.com> Message-ID: <52FD731D.3030901@oracle.com> Looks good. Sorry I didn't catch this earlier and I was counting on the test build :) Mandy [1] http://hg.openjdk.java.net/jdk9/dev/jdk/rev/c58c6b0fbe34 On 2/13/2014 1:50 PM, Phil Race wrote: > That worked on Mac but I just found it doesn't build on Linux because > a macro-redefinition > warning is treated as an error there. > > https://bugs.openjdk.java.net/browse/JDK-8034912 > > The early/original fix had removed the duplicate definition in > src/share/native/java/net/net_util.h > > So the fix is to do the same in 8u :- > > ~/jdk8u-dev/jdk$ hg diff src/share/native/java/net/net_util.h > diff --git a/src/share/native/java/net/net_util.h > b/src/share/native/java/net/net_util.h > --- a/src/share/native/java/net/net_util.h > +++ b/src/share/native/java/net/net_util.h > @@ -42,9 +42,6 @@ > #define NET_ERROR(env, ex, msg) \ > { if (!(*env)->ExceptionOccurred(env)) JNU_ThrowByName(env, ex, msg) } > > -#define CHECK_NULL(x) if ((x) == NULL) return; > -#define CHECK_NULL_RETURN(x, y) if ((x) == NULL) return y; > - > /************************************************************************ > > * Cached field IDs > * > > A little surprised no else else found this already (did they?) > Anyway I need a review and Ok to push 8034912 to JDk 8 u > > -Phil. > > On 02/12/2014 10:21 AM, Mike Duigou wrote: >> This looks fine. >> >> >> On Feb 11 2014, at 15:42 , Phil Race wrote: >> >>> Here's a JDk8u webrev : -http://cr.openjdk.java.net/~prr/8031737.8u/ >>> >>> -phil. >>> >>> On 2/11/14 2:28 PM, Phil Race wrote: >>>> So since hg export/import doesn't apply cleanly and the dependency >>>> chain seems, long and in order to have some consistency across the >>>> releases, >>>> I think I should prepare a webrev which essentially backports 8031737 >>>> including its small changes to Version.c, if only because otherwise >>>> I'd have to have a new bug ID that would not be forwarded ported >>>> (one source of confusion) or even worse re-use 8031737 but not >>>> fully implement it >>>> >>>> Agreed ? >>>> >>>> -phil. >>>> > From philip.race at oracle.com Fri Feb 14 01:37:58 2014 From: philip.race at oracle.com (Phil Race) Date: Thu, 13 Feb 2014 17:37:58 -0800 Subject: RFR: (8031737) CHECK_NULL and CHECK_EXCEPTION macros cleanup In-Reply-To: <52FD70ED.5060102@Oracle.com> References: <52ED36FD.5070507@oracle.com> <1243D3E7-C384-4AA8-80CA-9AA4E053F056@oracle.com> <52F00EAC.8000900@oracle.com> <52FA7429.5070806@oracle.com> <52FA768A.70005@oracle.com> <52FA80A6.2040509@oracle.com> <52FA87E2.9080009@oracle.com> <52FA98A8.7090104@oracle.com> <52FA9AA2.6090005@oracle.com> <52FA9F82.90903@oracle.com> <52FAA211.2070800@oracle.com> <52FAA3F4.6040702@oracle.com> <52FAB577.3070205@oracle.com> <52FD3E1F.4070006@oracle.com> <52FD70ED.5060102@Oracle.com> Message-ID: <52FD7376.5@oracle.com> I did look closely at that today. It has 3 parts - removing the ones in net_util - moving these macros to jni_util - however these new versions are already superseded by the ones we have, so we can' t apply that. - updating some 'pack' code to use the new macros instead of its own ones (no clash here) So except for the last part I essentially have that fix here (and more) and at this point I just wanted to fix the build problem ... -phil. On 2/13/14 5:27 PM, Roger Riggs wrote: > Hi Phil, > > There was an earlier commit that updated java.net > > 8030875: Macros for checking and returning on exceptions > > Maybe it will apply as a backport too? > > Roger > > > > > > On 2/13/14 4:50 PM, Phil Race wrote: >> That worked on Mac but I just found it doesn't build on Linux because >> a macro-redefinition >> warning is treated as an error there. >> >> https://bugs.openjdk.java.net/browse/JDK-8034912 >> >> The early/original fix had removed the duplicate definition in >> src/share/native/java/net/net_util.h >> >> So the fix is to do the same in 8u :- >> >> ~/jdk8u-dev/jdk$ hg diff src/share/native/java/net/net_util.h >> diff --git a/src/share/native/java/net/net_util.h >> b/src/share/native/java/net/net_util.h >> --- a/src/share/native/java/net/net_util.h >> +++ b/src/share/native/java/net/net_util.h >> @@ -42,9 +42,6 @@ >> #define NET_ERROR(env, ex, msg) \ >> { if (!(*env)->ExceptionOccurred(env)) JNU_ThrowByName(env, ex, msg) } >> >> -#define CHECK_NULL(x) if ((x) == NULL) return; >> -#define CHECK_NULL_RETURN(x, y) if ((x) == NULL) return y; >> - >> /************************************************************************ >> >> * Cached field IDs >> * >> >> A little surprised no else else found this already (did they?) >> Anyway I need a review and Ok to push 8034912 to JDk 8 u >> >> -Phil. >> >> On 02/12/2014 10:21 AM, Mike Duigou wrote: >>> This looks fine. >>> >>> >>> On Feb 11 2014, at 15:42 , Phil Race wrote: >>> >>>> Here's a JDk8u webrev : -http://cr.openjdk.java.net/~prr/8031737.8u/ >>>> >>>> -phil. >>>> >>>> On 2/11/14 2:28 PM, Phil Race wrote: >>>>> So since hg export/import doesn't apply cleanly and the dependency >>>>> chain seems, long and in order to have some consistency across the >>>>> releases, >>>>> I think I should prepare a webrev which essentially backports 8031737 >>>>> including its small changes to Version.c, if only because otherwise >>>>> I'd have to have a new bug ID that would not be forwarded ported >>>>> (one source of confusion) or even worse re-use 8031737 but not >>>>> fully implement it >>>>> >>>>> Agreed ? >>>>> >>>>> -phil. >>>>> >> > From philip.race at oracle.com Fri Feb 14 01:38:45 2014 From: philip.race at oracle.com (Phil Race) Date: Thu, 13 Feb 2014 17:38:45 -0800 Subject: RFR: (8031737) CHECK_NULL and CHECK_EXCEPTION macros cleanup In-Reply-To: <52FD731D.3030901@oracle.com> References: <52ED36FD.5070507@oracle.com> <1243D3E7-C384-4AA8-80CA-9AA4E053F056@oracle.com> <52F00EAC.8000900@oracle.com> <52FA7429.5070806@oracle.com> <52FA768A.70005@oracle.com> <52FA80A6.2040509@oracle.com> <52FA87E2.9080009@oracle.com> <52FA98A8.7090104@oracle.com> <52FA9AA2.6090005@oracle.com> <52FA9F82.90903@oracle.com> <52FAA211.2070800@oracle.com> <52FAA3F4.6040702@oracle.com> <52FAB577.3070205@oracle.com> <52FD3E1F.4070006@oracle.com> <52FD731D.3030901@oracle.com> Message-ID: <52FD73A5.8040702@oracle.com> Yeah sorry about that. It built on Windows and Mac but not Linux as i found out today. -phil. On 2/13/14 5:36 PM, Mandy Chung wrote: > Looks good. Sorry I didn't catch this earlier and I was counting on > the test build :) > > Mandy > [1] http://hg.openjdk.java.net/jdk9/dev/jdk/rev/c58c6b0fbe34 > > On 2/13/2014 1:50 PM, Phil Race wrote: >> That worked on Mac but I just found it doesn't build on Linux because >> a macro-redefinition >> warning is treated as an error there. >> >> https://bugs.openjdk.java.net/browse/JDK-8034912 >> >> The early/original fix had removed the duplicate definition in >> src/share/native/java/net/net_util.h >> >> So the fix is to do the same in 8u :- >> >> ~/jdk8u-dev/jdk$ hg diff src/share/native/java/net/net_util.h >> diff --git a/src/share/native/java/net/net_util.h >> b/src/share/native/java/net/net_util.h >> --- a/src/share/native/java/net/net_util.h >> +++ b/src/share/native/java/net/net_util.h >> @@ -42,9 +42,6 @@ >> #define NET_ERROR(env, ex, msg) \ >> { if (!(*env)->ExceptionOccurred(env)) JNU_ThrowByName(env, ex, msg) } >> >> -#define CHECK_NULL(x) if ((x) == NULL) return; >> -#define CHECK_NULL_RETURN(x, y) if ((x) == NULL) return y; >> - >> /************************************************************************ >> >> * Cached field IDs >> * >> >> A little surprised no else else found this already (did they?) >> Anyway I need a review and Ok to push 8034912 to JDk 8 u >> >> -Phil. >> >> On 02/12/2014 10:21 AM, Mike Duigou wrote: >>> This looks fine. >>> >>> >>> On Feb 11 2014, at 15:42 , Phil Race wrote: >>> >>>> Here's a JDk8u webrev : -http://cr.openjdk.java.net/~prr/8031737.8u/ >>>> >>>> -phil. >>>> >>>> On 2/11/14 2:28 PM, Phil Race wrote: >>>>> So since hg export/import doesn't apply cleanly and the dependency >>>>> chain seems, long and in order to have some consistency across the >>>>> releases, >>>>> I think I should prepare a webrev which essentially backports 8031737 >>>>> including its small changes to Version.c, if only because otherwise >>>>> I'd have to have a new bug ID that would not be forwarded ported >>>>> (one source of confusion) or even worse re-use 8031737 but not >>>>> fully implement it >>>>> >>>>> Agreed ? >>>>> >>>>> -phil. >>>>> >> > From fweimer at redhat.com Fri Feb 14 10:26:20 2014 From: fweimer at redhat.com (Florian Weimer) Date: Fri, 14 Feb 2014 11:26:20 +0100 Subject: RFR 9: 8034903: Add Logging of Process.start arguments and resulting pid In-Reply-To: <52FD3897.7070903@oracle.com> References: <52FD3897.7070903@oracle.com> Message-ID: <52FDEF4C.9080202@redhat.com> On 02/13/2014 10:26 PM, roger riggs wrote: > Having folks stumbling over process creation and problems of quoting, > especially on windows, it seems useful to log the native commands and > arguments. > They are proposed to be logged using the PlatformLogger at Level.FINE > which will not be logged by default. The environment is useful in some > cases, > but verbose, that it should be Logged at FINER. The pid of the spawned > process > logged as well for traceability. It may make sense to expose something like getProcessID() to the public because it is independently useful on systems with /proc. In the src/solaris version, Arrays::toString(Object[]) does not quote the arguments, so the argument boundary is lost in the logging. I think something that performs quoting (using the Java language rules, not shell) would be helpful in this context. Although on non-Windows platforms, this information can easily be obtained with tools such as strace or truss. -- Florian Weimer / Red Hat Product Security Team From Alan.Bateman at oracle.com Fri Feb 14 10:43:45 2014 From: Alan.Bateman at oracle.com (Alan Bateman) Date: Fri, 14 Feb 2014 10:43:45 +0000 Subject: RFR 9: 8034903: Add Logging of Process.start arguments and resulting pid In-Reply-To: <52FDEF4C.9080202@redhat.com> References: <52FD3897.7070903@oracle.com> <52FDEF4C.9080202@redhat.com> Message-ID: <52FDF361.5070900@oracle.com> On 14/02/2014 10:26, Florian Weimer wrote: > > It may make sense to expose something like getProcessID() to the > public because it is independently useful on systems with /proc. Yes, this has been asked for many times. We have JEP 102 [1] that lists this and other Process topics that we'd like to get funded for JDK 9. -Alan [1] http://openjdk.java.net/jeps/102 From daniel.fuchs at oracle.com Fri Feb 14 11:06:06 2014 From: daniel.fuchs at oracle.com (Daniel Fuchs) Date: Fri, 14 Feb 2014 12:06:06 +0100 Subject: RFR 9: 8034903: Add Logging of Process.start arguments and resulting pid In-Reply-To: <52FD3897.7070903@oracle.com> References: <52FD3897.7070903@oracle.com> Message-ID: <52FDF89E.6050204@oracle.com> Hi Roger, I agree with Paul that using Objects.toString(environment) would be cleaner. Concerning the test, you may want to modify CheckHandler.publish to filter out any log record that doesn't come from your ProcessImpl classes. Something like: 175 @Override 176 public void publish(LogRecord lr) { // Only look at the records emitted by the // "java.lang.Process" logger if ("java.lang.Process".equals(lr.getLoggerName()) { 177 logRec = lr; } 178 } Also, since you're changing the Level of the root logger, you may want to either run your test in /othervm mode, or make sure that it puts back the root logger level to its original value and also removes the CheckHandler() instance from the root handlers with a try { } finally { } to avoid polluting the other tests that will follow. best regards, -- daniel On 2/13/14 10:26 PM, roger riggs wrote: > Hi, > > Having folks stumbling over process creation and problems of quoting, > especially on windows, it seems useful to log the native commands and > arguments. > They are proposed to be logged using the PlatformLogger at Level.FINE > which will not be logged by default. The environment is useful in some > cases, > but verbose, that it should be Logged at FINER. The pid of the spawned > process > logged as well for traceability. > > Please take a look at this first draft and comment on whether it is useful, > a good idea and any improvements needed. > > Webrev: > http://cr.openjdk.java.net/~rriggs/webrev-log-processcreate/ > > Thanks, Roger > From sean.coffey at oracle.com Fri Feb 14 11:10:16 2014 From: sean.coffey at oracle.com (=?ISO-8859-1?Q?Se=E1n_Coffey?=) Date: Fri, 14 Feb 2014 11:10:16 +0000 Subject: (8034912) Approval request : RFR: (8031737) CHECK_NULL and CHECK_EXCEPTION macros cleanup In-Reply-To: <52FD73A5.8040702@oracle.com> References: <52ED36FD.5070507@oracle.com> <1243D3E7-C384-4AA8-80CA-9AA4E053F056@oracle.com> <52F00EAC.8000900@oracle.com> <52FA7429.5070806@oracle.com> <52FA768A.70005@oracle.com> <52FA80A6.2040509@oracle.com> <52FA87E2.9080009@oracle.com> <52FA98A8.7090104@oracle.com> <52FA9AA2.6090005@oracle.com> <52FA9F82.90903@oracle.com> <52FAA211.2070800@oracle.com> <52FAA3F4.6040702@oracle.com> <52FAB577.3070205@oracle.com> <52FD3E1F.4070006@oracle.com> <52FD731D.3030901@oracle.com> <52FD73A5.8040702@oracle.com> Message-ID: <52FDF998.4020700@oracle.com> Approved for pushing to jdk8u-dev. regards, Sean. On 14/02/14 01:38, Phil Race wrote: > Yeah sorry about that. It built on Windows and Mac but not Linux as i > found out today. > > -phil. > > On 2/13/14 5:36 PM, Mandy Chung wrote: >> Looks good. Sorry I didn't catch this earlier and I was counting on >> the test build :) >> >> Mandy >> [1] http://hg.openjdk.java.net/jdk9/dev/jdk/rev/c58c6b0fbe34 >> >> On 2/13/2014 1:50 PM, Phil Race wrote: >>> That worked on Mac but I just found it doesn't build on Linux >>> because a macro-redefinition >>> warning is treated as an error there. >>> >>> https://bugs.openjdk.java.net/browse/JDK-8034912 >>> >>> The early/original fix had removed the duplicate definition in >>> src/share/native/java/net/net_util.h >>> >>> So the fix is to do the same in 8u :- >>> >>> ~/jdk8u-dev/jdk$ hg diff src/share/native/java/net/net_util.h >>> diff --git a/src/share/native/java/net/net_util.h >>> b/src/share/native/java/net/net_util.h >>> --- a/src/share/native/java/net/net_util.h >>> +++ b/src/share/native/java/net/net_util.h >>> @@ -42,9 +42,6 @@ >>> #define NET_ERROR(env, ex, msg) \ >>> { if (!(*env)->ExceptionOccurred(env)) JNU_ThrowByName(env, ex, msg) } >>> >>> -#define CHECK_NULL(x) if ((x) == NULL) return; >>> -#define CHECK_NULL_RETURN(x, y) if ((x) == NULL) return y; >>> - >>> /************************************************************************ >>> >>> * Cached field IDs >>> * >>> >>> A little surprised no else else found this already (did they?) >>> Anyway I need a review and Ok to push 8034912 to JDk 8 u >>> >>> -Phil. >>> >>> On 02/12/2014 10:21 AM, Mike Duigou wrote: >>>> This looks fine. >>>> >>>> >>>> On Feb 11 2014, at 15:42 , Phil Race wrote: >>>> >>>>> Here's a JDk8u webrev : -http://cr.openjdk.java.net/~prr/8031737.8u/ >>>>> >>>>> -phil. >>>>> >>>>> On 2/11/14 2:28 PM, Phil Race wrote: >>>>>> So since hg export/import doesn't apply cleanly and the dependency >>>>>> chain seems, long and in order to have some consistency across >>>>>> the releases, >>>>>> I think I should prepare a webrev which essentially backports >>>>>> 8031737 >>>>>> including its small changes to Version.c, if only because otherwise >>>>>> I'd have to have a new bug ID that would not be forwarded ported >>>>>> (one source of confusion) or even worse re-use 8031737 but not >>>>>> fully implement it >>>>>> >>>>>> Agreed ? >>>>>> >>>>>> -phil. >>>>>> >>> >> > From leventov at ya.ru Fri Feb 14 13:13:45 2014 From: leventov at ya.ru (Roman Leventov) Date: Fri, 14 Feb 2014 17:13:45 +0400 Subject: Why isEmpty(), retainAll() and containsAll() not supported with default implementations in JDK8? Message-ID: <258621392383625@web4m.yandex.ru> I'm sure there was a discussion somewhere in JDK mailing lists, but I couldn't find. Please, give me a link. I've tried to ask on SO: http://stackoverflow.com/questions/21758081/why-many-methods-in-jcf-interfaces-not-made-default-in-java-8, proved that JDK developers don't read SO :) From paul.sandoz at oracle.com Fri Feb 14 14:50:59 2014 From: paul.sandoz at oracle.com (Paul Sandoz) Date: Fri, 14 Feb 2014 15:50:59 +0100 Subject: Why isEmpty(), retainAll() and containsAll() not supported with default implementations in JDK8? In-Reply-To: <258621392383625@web4m.yandex.ru> References: <258621392383625@web4m.yandex.ru> Message-ID: <80FCA4EF-18A7-4D20-9A25-CB91543667BE@oracle.com> On Feb 14, 2014, at 2:13 PM, Roman Leventov wrote: > I'm sure there was a discussion somewhere in JDK mailing lists, but I couldn't find. Please, give me a link. > I cannot recall such discussion. Generally we have only converted existing abstract methods on an interface to non-abstract if there was a compelling reason to aid implementations, such as Iterator.remove. These are not new methods on Collection, and there are already implementations in AbstractCollection. The advantage of converting these abstract into non-abstract methods its not particularly compelling given one is most likely to inherit from AbstractCollection or provide more efficient implementations. It would be possible to move all non-abstract methods on AbstractCollection to Collection. If we were starting from a blank sheet of paper that is what we might have done. (Note one cannot do this with all non-abstract methods on AbstractList.) > I've tried to ask on SO: http://stackoverflow.com/questions/21758081/why-many-methods-in-jcf-interfaces-not-made-default-in-java-8, proved that JDK developers don't read SO :) I don't :-) i know others do, and Stuart replied: http://stackoverflow.com/a/21774137 Paul. From david.lloyd at redhat.com Fri Feb 14 15:56:34 2014 From: david.lloyd at redhat.com (David M. Lloyd) Date: Fri, 14 Feb 2014 09:56:34 -0600 Subject: A hole in the serialization spec In-Reply-To: <52FD0328.3050304@redhat.com> References: <52F4F6BB.2010207@redhat.com> <52F8E807.2090403@oracle.com> <52F8F21C.80205@redhat.com> <52FB8E5C.9020502@oracle.com> <52FB9248.6010703@redhat.com> <26199217-BAE6-47AC-AFC8-4DE90726B8C0@oracle.com> <52FD0328.3050304@redhat.com> Message-ID: <52FE3CB2.5060408@redhat.com> On 02/13/2014 11:38 AM, David M. Lloyd wrote: > On 02/13/2014 10:29 AM, Chris Hegarty wrote: >> On 12 Feb 2014, at 15:24, David M. Lloyd >> wrote: >> >>> That's a quote from the serialization spec. I take it to mean, >>> "Don't write fields and everything might go to hell". In practice, >>> if the reading side doesn't read fields, things end up more or less >>> OK, as evidenced by various classes in the wild. But it's not hard >>> to imagine a scenario in which a class change could cause protocol >>> corruption. >>> >>> I think the specifics of the quote relate to this kind of class >>> change; in particular, if a class is deleted from the hierarchy on >>> the read side, and that class corresponds to the class that had the >>> misbehaving writeObject, I suspect that things will break at that >>> point as the read side will probably try to consume and discard the >>> field information for that class, which will be missing (it will >>> start reading the next class' fields instead I think). >> >> Yes, possibly. And who knows what fields/values may be read and >> mistaken for the wrong object in the hierarchy. So ?undefined' >> behaviour seems right to me. > > I think the behavior is well-defined, just "bad", which is my point. If > the exact current is spec'd out as-is then at least we can be assured of > the same bad behavior across implementations. If the behavior is > changed such that fields are read/written but discarded, without > updating the spec, then the "undefined" behavior at least becomes safer. > If the behavior is changed, *and* the spec is updated, then we get > both benefits, but at the cost that all previous implementations will > not be compliant with the spec. > > All options seem to have a cost though. In the JDK, java.util.Date does not read/write fields. Perhaps others as well. Given that the behavior is presently undefined, that means the serialized representation of java.util.Date (and any other such non-conforming classes) are also undefined. -- - DML From roger.riggs at oracle.com Fri Feb 14 16:21:04 2014 From: roger.riggs at oracle.com (roger riggs) Date: Fri, 14 Feb 2014 11:21:04 -0500 Subject: RFR 9: 8034903: Add Logging of Process.start arguments and resulting pid In-Reply-To: <52FDF89E.6050204@oracle.com> References: <52FD3897.7070903@oracle.com> <52FDF89E.6050204@oracle.com> Message-ID: <52FE4270.2080602@oracle.com> Thanks Paul, Daniel, Thanks for the suggestions, I updated the webrev. /othervm does seem to be needed to prevent this test from interfering or being interfered wit. Roger On 2/14/2014 6:06 AM, Daniel Fuchs wrote: > Hi Roger, > > I agree with Paul that using Objects.toString(environment) would be > cleaner. > > Concerning the test, you may want to modify CheckHandler.publish to > filter out any log record that doesn't come from your ProcessImpl > classes. Something like: > > 175 @Override > 176 public void publish(LogRecord lr) { > // Only look at the records emitted by the > // "java.lang.Process" logger > if ("java.lang.Process".equals(lr.getLoggerName()) { > 177 logRec = lr; > } > 178 } > > Also, since you're changing the Level of the root logger, you may > want to either run your test in /othervm mode, or make sure that it > puts back the root logger level to its original value and also > removes the CheckHandler() instance from the root handlers with a try { > } finally { } to avoid polluting the other tests that will follow. > > best regards, > > -- daniel > > On 2/13/14 10:26 PM, roger riggs wrote: >> Hi, >> >> Having folks stumbling over process creation and problems of quoting, >> especially on windows, it seems useful to log the native commands and >> arguments. >> They are proposed to be logged using the PlatformLogger at Level.FINE >> which will not be logged by default. The environment is useful in some >> cases, >> but verbose, that it should be Logged at FINER. The pid of the spawned >> process >> logged as well for traceability. >> >> Please take a look at this first draft and comment on whether it is >> useful, >> a good idea and any improvements needed. >> >> Webrev: >> http://cr.openjdk.java.net/~rriggs/webrev-log-processcreate/ >> >> Thanks, Roger >> > From roger.riggs at oracle.com Fri Feb 14 16:26:16 2014 From: roger.riggs at oracle.com (roger riggs) Date: Fri, 14 Feb 2014 11:26:16 -0500 Subject: RFR 9: 8034903: Add Logging of Process.start arguments and resulting pid In-Reply-To: <52FDEF4C.9080202@redhat.com> References: <52FD3897.7070903@oracle.com> <52FDEF4C.9080202@redhat.com> Message-ID: <52FE43A8.1020506@oracle.com> Hi Florian, As for escaping, is there an existing API for encoding and decoding those? A human won't have trouble figuring out the few cases where there might be some ambiguity due to "," in the string. The test will be more complicated as well and I think added that the extra complexity is not a good investment. Thanks, Roger On 2/14/2014 5:26 AM, Florian Weimer wrote: > On 02/13/2014 10:26 PM, roger riggs wrote: > >> Having folks stumbling over process creation and problems of quoting, >> especially on windows, it seems useful to log the native commands and >> arguments. >> They are proposed to be logged using the PlatformLogger at Level.FINE >> which will not be logged by default. The environment is useful in some >> cases, >> but verbose, that it should be Logged at FINER. The pid of the spawned >> process >> logged as well for traceability. > > It may make sense to expose something like getProcessID() to the > public because it is independently useful on systems with /proc. > > In the src/solaris version, Arrays::toString(Object[]) does not quote > the arguments, so the argument boundary is lost in the logging. I > think something that performs quoting (using the Java language rules, > not shell) would be helpful in this context. Although on non-Windows > platforms, this information can easily be obtained with tools such as > strace or truss. > From roger.riggs at oracle.com Fri Feb 14 16:31:35 2014 From: roger.riggs at oracle.com (roger riggs) Date: Fri, 14 Feb 2014 11:31:35 -0500 Subject: A hole in the serialization spec In-Reply-To: <52FE3CB2.5060408@redhat.com> References: <52F4F6BB.2010207@redhat.com> <52F8E807.2090403@oracle.com> <52F8F21C.80205@redhat.com> <52FB8E5C.9020502@oracle.com> <52FB9248.6010703@redhat.com> <26199217-BAE6-47AC-AFC8-4DE90726B8C0@oracle.com> <52FD0328.3050304@redhat.com> <52FE3CB2.5060408@redhat.com> Message-ID: <52FE44E7.6060103@oracle.com> Hi David, I would quibble that the serialized form of java.util.Date is defined. The behavior is defined by the writeObject and readObject methods. They write and read a time computed from and restored to the state of the object. Roger On 2/14/2014 10:56 AM, David M. Lloyd wrote: > On 02/13/2014 11:38 AM, David M. Lloyd wrote: >> On 02/13/2014 10:29 AM, Chris Hegarty wrote: >>> On 12 Feb 2014, at 15:24, David M. Lloyd >>> wrote: >>> >>>> That's a quote from the serialization spec. I take it to mean, >>>> "Don't write fields and everything might go to hell". In practice, >>>> if the reading side doesn't read fields, things end up more or less >>>> OK, as evidenced by various classes in the wild. But it's not hard >>>> to imagine a scenario in which a class change could cause protocol >>>> corruption. >>>> >>>> I think the specifics of the quote relate to this kind of class >>>> change; in particular, if a class is deleted from the hierarchy on >>>> the read side, and that class corresponds to the class that had the >>>> misbehaving writeObject, I suspect that things will break at that >>>> point as the read side will probably try to consume and discard the >>>> field information for that class, which will be missing (it will >>>> start reading the next class' fields instead I think). >>> >>> Yes, possibly. And who knows what fields/values may be read and >>> mistaken for the wrong object in the hierarchy. So ?undefined' >>> behaviour seems right to me. >> >> I think the behavior is well-defined, just "bad", which is my point. If >> the exact current is spec'd out as-is then at least we can be assured of >> the same bad behavior across implementations. If the behavior is >> changed such that fields are read/written but discarded, without >> updating the spec, then the "undefined" behavior at least becomes safer. >> If the behavior is changed, *and* the spec is updated, then we get >> both benefits, but at the cost that all previous implementations will >> not be compliant with the spec. >> >> All options seem to have a cost though. > > In the JDK, java.util.Date does not read/write fields. Perhaps others > as well. Given that the behavior is presently undefined, that means > the serialized representation of java.util.Date (and any other such > non-conforming classes) are also undefined. > From chris.hegarty at oracle.com Fri Feb 14 16:36:09 2014 From: chris.hegarty at oracle.com (Chris Hegarty) Date: Fri, 14 Feb 2014 16:36:09 +0000 Subject: A hole in the serialization spec In-Reply-To: <52FE3CB2.5060408@redhat.com> References: <52F4F6BB.2010207@redhat.com> <52F8E807.2090403@oracle.com> <52F8F21C.80205@redhat.com> <52FB8E5C.9020502@oracle.com> <52FB9248.6010703@redhat.com> <26199217-BAE6-47AC-AFC8-4DE90726B8C0@oracle.com> <52FD0328.3050304@redhat.com> <52FE3CB2.5060408@redhat.com> Message-ID: <52FE45F9.9080304@oracle.com> On 14/02/14 15:56, David M. Lloyd wrote: > On 02/13/2014 11:38 AM, David M. Lloyd wrote: >> On 02/13/2014 10:29 AM, Chris Hegarty wrote: >>> On 12 Feb 2014, at 15:24, David M. Lloyd >>> wrote: >>> >>>> That's a quote from the serialization spec. I take it to mean, >>>> "Don't write fields and everything might go to hell". In practice, >>>> if the reading side doesn't read fields, things end up more or less >>>> OK, as evidenced by various classes in the wild. But it's not hard >>>> to imagine a scenario in which a class change could cause protocol >>>> corruption. >>>> >>>> I think the specifics of the quote relate to this kind of class >>>> change; in particular, if a class is deleted from the hierarchy on >>>> the read side, and that class corresponds to the class that had the >>>> misbehaving writeObject, I suspect that things will break at that >>>> point as the read side will probably try to consume and discard the >>>> field information for that class, which will be missing (it will >>>> start reading the next class' fields instead I think). >>> >>> Yes, possibly. And who knows what fields/values may be read and >>> mistaken for the wrong object in the hierarchy. So ?undefined' >>> behaviour seems right to me. >> >> I think the behavior is well-defined, just "bad", which is my point. If >> the exact current is spec'd out as-is then at least we can be assured of >> the same bad behavior across implementations. If the behavior is >> changed such that fields are read/written but discarded, without >> updating the spec, then the "undefined" behavior at least becomes safer. >> If the behavior is changed, *and* the spec is updated, then we get >> both benefits, but at the cost that all previous implementations will >> not be compliant with the spec. >> >> All options seem to have a cost though. > > In the JDK, java.util.Date does not read/write fields. Perhaps others > as well. Given that the behavior is presently undefined, that means the > serialized representation of java.util.Date (and any other such > non-conforming classes) are also undefined. True. j.u.Date is playing with fire ( and it should be fixed ). But since it has no serializable state itself, it doesn't fall foul ( at least on Oracles JDK ) of any problems. If it were to evolve in the future it may run into trouble. -Chris. From chris.hegarty at oracle.com Fri Feb 14 16:38:35 2014 From: chris.hegarty at oracle.com (Chris Hegarty) Date: Fri, 14 Feb 2014 16:38:35 +0000 Subject: A hole in the serialization spec In-Reply-To: <52FE44E7.6060103@oracle.com> References: <52F4F6BB.2010207@redhat.com> <52F8E807.2090403@oracle.com> <52F8F21C.80205@redhat.com> <52FB8E5C.9020502@oracle.com> <52FB9248.6010703@redhat.com> <26199217-BAE6-47AC-AFC8-4DE90726B8C0@oracle.com> <52FD0328.3050304@redhat.com> <52FE3CB2.5060408@redhat.com> <52FE44E7.6060103@oracle.com> Message-ID: <52FE468B.9020207@oracle.com> On 14/02/14 16:31, roger riggs wrote: > Hi David, > > I would quibble that the serialized form of java.util.Date is defined. > The behavior is defined by the writeObject and readObject methods. > They write and read a time computed from and restored to the state of > the object. Yes, but it falls foul if the MUST invoke default read/write fields in the Serialization spec if it has a writeObect. Evolving this class in the future ( if it were to happen ) would be problematic. If it were to add serializable state, for example. -Chris. > > Roger > > On 2/14/2014 10:56 AM, David M. Lloyd wrote: >> On 02/13/2014 11:38 AM, David M. Lloyd wrote: >>> On 02/13/2014 10:29 AM, Chris Hegarty wrote: >>>> On 12 Feb 2014, at 15:24, David M. Lloyd >>>> wrote: >>>> >>>>> That's a quote from the serialization spec. I take it to mean, >>>>> "Don't write fields and everything might go to hell". In practice, >>>>> if the reading side doesn't read fields, things end up more or less >>>>> OK, as evidenced by various classes in the wild. But it's not hard >>>>> to imagine a scenario in which a class change could cause protocol >>>>> corruption. >>>>> >>>>> I think the specifics of the quote relate to this kind of class >>>>> change; in particular, if a class is deleted from the hierarchy on >>>>> the read side, and that class corresponds to the class that had the >>>>> misbehaving writeObject, I suspect that things will break at that >>>>> point as the read side will probably try to consume and discard the >>>>> field information for that class, which will be missing (it will >>>>> start reading the next class' fields instead I think). >>>> >>>> Yes, possibly. And who knows what fields/values may be read and >>>> mistaken for the wrong object in the hierarchy. So ?undefined' >>>> behaviour seems right to me. >>> >>> I think the behavior is well-defined, just "bad", which is my point. If >>> the exact current is spec'd out as-is then at least we can be assured of >>> the same bad behavior across implementations. If the behavior is >>> changed such that fields are read/written but discarded, without >>> updating the spec, then the "undefined" behavior at least becomes safer. >>> If the behavior is changed, *and* the spec is updated, then we get >>> both benefits, but at the cost that all previous implementations will >>> not be compliant with the spec. >>> >>> All options seem to have a cost though. >> >> In the JDK, java.util.Date does not read/write fields. Perhaps others >> as well. Given that the behavior is presently undefined, that means >> the serialized representation of java.util.Date (and any other such >> non-conforming classes) are also undefined. >> > From huizhe.wang at oracle.com Fri Feb 14 17:35:05 2014 From: huizhe.wang at oracle.com (huizhe wang) Date: Fri, 14 Feb 2014 09:35:05 -0800 Subject: RFR (JAXP): 8033980 : Xerces Update: datatype XMLGregorianCalendarImpl and DurationImpl In-Reply-To: <52FCD57B.8090807@oracle.com> References: <52F9666F.9010303@oracle.com> <5111B487-22C1-4987-B9CC-ABF2F5CDD964@oracle.com> <52FA5855.3020106@oracle.com> <52FA7270.4090709@oracle.com> <52FAF801.5060300@oracle.com> <52FBCF66.8060308@oracle.com> <52FC7FE6.2060400@oracle.com> <52FCD57B.8090807@oracle.com> Message-ID: <52FE53C9.3010205@oracle.com> Hi All, I added a SerializationTest. The test contains a helper that can generate serialization files for XMLGregorianCalendar and Duration. I've created such files for jdk6,7,8 and 9, and manually run the test, that is, read them back with JDK6, 7, 8 and 9. The test worked fine. In the JDK9(or 10 in the future) repo and for an auto-run, it would use the current JDK9/10 build and test against JDK6, 7, 8 and 9. Past JDK10, we could consider add serialization files for JDK10. The new fields did not affect serialization compatibility. The above tests passed with/without the new fields being transient. But I added transient since it's the right thing to do. Adding fields is a compatible change in accordance with Java Object Serialization Spec . Thanks, Joe On 2/13/2014 6:23 AM, Alan Bateman wrote: > On 13/02/2014 08:18, huizhe wang wrote: >> Hi Alan, Lance, and Daniel, >> >> The Xerces serialization revision meant to create a serialization >> form that would help maintain future serialization compatibility. But >> in reality it itself is causing significant incompatibility as Alan >> pointed out below and we discussed previously. I've removed the >> revision from the patch as a result. >> >> Please see the new webrev here: >> http://cr.openjdk.java.net/~joehw/jdk9/8033980/webrev/ > Thanks for dropping the serialization change as it was just not going > to work the way you had intended. > > I agree with Daniel's comment about all the new fields added to > XMLGregorianCalendarImpl as it's not clear why they aren't transient. > > I have not studied the rest of the changes but I think Daniel and > Lance are. > > -Alan From michael.x.mcmahon at oracle.com Fri Feb 14 17:42:16 2014 From: michael.x.mcmahon at oracle.com (Michael McMahon) Date: Fri, 14 Feb 2014 17:42:16 +0000 Subject: RFR: 8034853 remove sun.misc.ClassLoaderUtil Message-ID: <52FE5578.9050006@oracle.com> Could I get the following change reviewed please? http://cr.openjdk.java.net/~michaelm/8034853/webrev.1/ The change is to remove the class sun.misc.ClassLoaderUtil. The functionality provided by this class is now in the public java.net.URLClassloader.close() method. Thanks Michael From david.lloyd at redhat.com Fri Feb 14 17:43:25 2014 From: david.lloyd at redhat.com (David M. Lloyd) Date: Fri, 14 Feb 2014 11:43:25 -0600 Subject: A hole in the serialization spec In-Reply-To: <52FE3CB2.5060408@redhat.com> References: <52F4F6BB.2010207@redhat.com> <52F8E807.2090403@oracle.com> <52F8F21C.80205@redhat.com> <52FB8E5C.9020502@oracle.com> <52FB9248.6010703@redhat.com> <26199217-BAE6-47AC-AFC8-4DE90726B8C0@oracle.com> <52FD0328.3050304@redhat.com> <52FE3CB2.5060408@redhat.com> Message-ID: <52FE55BD.2010201@redhat.com> On 02/14/2014 09:56 AM, David M. Lloyd wrote: > On 02/13/2014 11:38 AM, David M. Lloyd wrote: >> On 02/13/2014 10:29 AM, Chris Hegarty wrote: >>> On 12 Feb 2014, at 15:24, David M. Lloyd >>> wrote: >>> >>>> That's a quote from the serialization spec. I take it to mean, >>>> "Don't write fields and everything might go to hell". In practice, >>>> if the reading side doesn't read fields, things end up more or less >>>> OK, as evidenced by various classes in the wild. But it's not hard >>>> to imagine a scenario in which a class change could cause protocol >>>> corruption. >>>> >>>> I think the specifics of the quote relate to this kind of class >>>> change; in particular, if a class is deleted from the hierarchy on >>>> the read side, and that class corresponds to the class that had the >>>> misbehaving writeObject, I suspect that things will break at that >>>> point as the read side will probably try to consume and discard the >>>> field information for that class, which will be missing (it will >>>> start reading the next class' fields instead I think). >>> >>> Yes, possibly. And who knows what fields/values may be read and >>> mistaken for the wrong object in the hierarchy. So ?undefined' >>> behaviour seems right to me. >> >> I think the behavior is well-defined, just "bad", which is my point. If >> the exact current is spec'd out as-is then at least we can be assured of >> the same bad behavior across implementations. If the behavior is >> changed such that fields are read/written but discarded, without >> updating the spec, then the "undefined" behavior at least becomes safer. >> If the behavior is changed, *and* the spec is updated, then we get >> both benefits, but at the cost that all previous implementations will >> not be compliant with the spec. >> >> All options seem to have a cost though. > > In the JDK, java.util.Date does not read/write fields. Perhaps others > as well. Given that the behavior is presently undefined, that means the > serialized representation of java.util.Date (and any other such > non-conforming classes) are also undefined. An interesting detail here - since Date doesn't have any non-transient fields, this happens to work out OK for a second reason (that defaultReadFields() would read nothing anyway) - however it still would break if a non-transient field were to be added. -- - DML From david.lloyd at redhat.com Fri Feb 14 18:01:19 2014 From: david.lloyd at redhat.com (David M. Lloyd) Date: Fri, 14 Feb 2014 12:01:19 -0600 Subject: A hole in the serialization spec In-Reply-To: <52FE44E7.6060103@oracle.com> References: <52F4F6BB.2010207@redhat.com> <52F8E807.2090403@oracle.com> <52F8F21C.80205@redhat.com> <52FB8E5C.9020502@oracle.com> <52FB9248.6010703@redhat.com> <26199217-BAE6-47AC-AFC8-4DE90726B8C0@oracle.com> <52FD0328.3050304@redhat.com> <52FE3CB2.5060408@redhat.com> <52FE44E7.6060103@oracle.com> Message-ID: <52FE59EF.3030606@redhat.com> Yes, however given that the lack of reading/writing fields makes it undefined *in general*, you can only say "the behavior is defined as long as this undefined behavior is actually defined in this (risky) manner". Put another way, since the behavior of not reading/writing fields is not defined, this class is then by spec not portable between JVMs (after all, undefined behavior could mean anything at all). That's why I was saying that the spec should define exactly what will physically happen if you do this (from both the perspective of the "other end" of the serialization API, *and* the wire protocol), even if it's considered poor practice - because then at least you have guaranteed interoperability (which becomes even more important with the knowledge that JDK classes have this issue). On 02/14/2014 10:31 AM, roger riggs wrote: > Hi David, > > I would quibble that the serialized form of java.util.Date is defined. > The behavior is defined by the writeObject and readObject methods. > They write and read a time computed from and restored to the state of > the object. > > Roger > > On 2/14/2014 10:56 AM, David M. Lloyd wrote: >> On 02/13/2014 11:38 AM, David M. Lloyd wrote: >>> On 02/13/2014 10:29 AM, Chris Hegarty wrote: >>>> On 12 Feb 2014, at 15:24, David M. Lloyd >>>> wrote: >>>> >>>>> That's a quote from the serialization spec. I take it to mean, >>>>> "Don't write fields and everything might go to hell". In practice, >>>>> if the reading side doesn't read fields, things end up more or less >>>>> OK, as evidenced by various classes in the wild. But it's not hard >>>>> to imagine a scenario in which a class change could cause protocol >>>>> corruption. >>>>> >>>>> I think the specifics of the quote relate to this kind of class >>>>> change; in particular, if a class is deleted from the hierarchy on >>>>> the read side, and that class corresponds to the class that had the >>>>> misbehaving writeObject, I suspect that things will break at that >>>>> point as the read side will probably try to consume and discard the >>>>> field information for that class, which will be missing (it will >>>>> start reading the next class' fields instead I think). >>>> >>>> Yes, possibly. And who knows what fields/values may be read and >>>> mistaken for the wrong object in the hierarchy. So ?undefined' >>>> behaviour seems right to me. >>> >>> I think the behavior is well-defined, just "bad", which is my point. If >>> the exact current is spec'd out as-is then at least we can be assured of >>> the same bad behavior across implementations. If the behavior is >>> changed such that fields are read/written but discarded, without >>> updating the spec, then the "undefined" behavior at least becomes safer. >>> If the behavior is changed, *and* the spec is updated, then we get >>> both benefits, but at the cost that all previous implementations will >>> not be compliant with the spec. >>> >>> All options seem to have a cost though. >> >> In the JDK, java.util.Date does not read/write fields. Perhaps others >> as well. Given that the behavior is presently undefined, that means >> the serialized representation of java.util.Date (and any other such >> non-conforming classes) are also undefined. >> > -- - DML From joe.darcy at oracle.com Fri Feb 14 18:09:26 2014 From: joe.darcy at oracle.com (Joe Darcy) Date: Fri, 14 Feb 2014 10:09:26 -0800 Subject: RFR: 8034853 remove sun.misc.ClassLoaderUtil In-Reply-To: <52FE5578.9050006@oracle.com> References: <52FE5578.9050006@oracle.com> Message-ID: <52FE5BD6.4010903@oracle.com> Hi Michael, Good to see more of sun.misc go away :-) For the test, I recommend updating the @summary and removing the comment about the old API. Thanks, -Joe On 02/14/2014 09:42 AM, Michael McMahon wrote: > Could I get the following change reviewed please? > > http://cr.openjdk.java.net/~michaelm/8034853/webrev.1/ > > The change is to remove the class sun.misc.ClassLoaderUtil. > The functionality provided by this class is now in the public > java.net.URLClassloader.close() method. > > Thanks > Michael From martinrb at google.com Fri Feb 14 18:10:49 2014 From: martinrb at google.com (Martin Buchholz) Date: Fri, 14 Feb 2014 10:10:49 -0800 Subject: RFR 9: 8034903: Add Logging of Process.start arguments and resulting pid In-Reply-To: <52FD3897.7070903@oracle.com> References: <52FD3897.7070903@oracle.com> Message-ID: I'm not excited about having logging statements added throughout the jdk implementation. PlatformLogger is an implementation detail, so real users cannot benefit. Your OS should already provide facilities to let you trace process creation. E.g. on Linux you can do this with strace (and I have done this myself in the past to debug subprocess creation). So I'd say adding logging just to provide a small rarely used convenience for jdk developers is Not Worthwhile. On Thu, Feb 13, 2014 at 1:26 PM, roger riggs wrote: > Hi, > > Having folks stumbling over process creation and problems of quoting, > especially on windows, it seems useful to log the native commands and > arguments. > They are proposed to be logged using the PlatformLogger at Level.FINE > which will not be logged by default. The environment is useful in some > cases, > but verbose, that it should be Logged at FINER. The pid of the spawned > process > logged as well for traceability. > > Please take a look at this first draft and comment on whether it is useful, > a good idea and any improvements needed. > > Webrev: > http://cr.openjdk.java.net/~rriggs/webrev-log-processcreate/ > > Thanks, Roger > > From michael.x.mcmahon at oracle.com Fri Feb 14 18:11:33 2014 From: michael.x.mcmahon at oracle.com (Michael McMahon) Date: Fri, 14 Feb 2014 18:11:33 +0000 Subject: RFR: 8034853 remove sun.misc.ClassLoaderUtil In-Reply-To: <52FE5BD6.4010903@oracle.com> References: <52FE5578.9050006@oracle.com> <52FE5BD6.4010903@oracle.com> Message-ID: <52FE5C55.3080703@oracle.com> Thanks Joe. Will do that. Michael On 14/02/14 18:09, Joe Darcy wrote: > Hi Michael, > > Good to see more of sun.misc go away :-) > > For the test, I recommend updating the @summary and removing the > comment about the old API. > > Thanks, > > -Joe > > On 02/14/2014 09:42 AM, Michael McMahon wrote: >> Could I get the following change reviewed please? >> >> http://cr.openjdk.java.net/~michaelm/8034853/webrev.1/ >> >> The change is to remove the class sun.misc.ClassLoaderUtil. >> The functionality provided by this class is now in the public >> java.net.URLClassloader.close() method. >> >> Thanks >> Michael > From Alan.Bateman at oracle.com Fri Feb 14 18:20:12 2014 From: Alan.Bateman at oracle.com (Alan Bateman) Date: Fri, 14 Feb 2014 18:20:12 +0000 Subject: RFR: 8034853 remove sun.misc.ClassLoaderUtil In-Reply-To: <52FE5578.9050006@oracle.com> References: <52FE5578.9050006@oracle.com> Message-ID: <52FE5E5C.60709@oracle.com> On 14/02/2014 17:42, Michael McMahon wrote: > Could I get the following change reviewed please? > > http://cr.openjdk.java.net/~michaelm/8034853/webrev.1/ > > The change is to remove the class sun.misc.ClassLoaderUtil. > The functionality provided by this class is now in the public > java.net.URLClassloader.close() method. The removal is good. Maybe the test should be moved to URLClassLoader as it doesn't make sense to have it in ClassLoaderUtil anymore. -Alan. From michael.x.mcmahon at oracle.com Fri Feb 14 18:31:11 2014 From: michael.x.mcmahon at oracle.com (Michael McMahon) Date: Fri, 14 Feb 2014 18:31:11 +0000 Subject: RFR: 8034853 remove sun.misc.ClassLoaderUtil In-Reply-To: <52FE5E5C.60709@oracle.com> References: <52FE5578.9050006@oracle.com> <52FE5E5C.60709@oracle.com> Message-ID: <52FE60EF.5090909@oracle.com> On 14/02/14 18:20, Alan Bateman wrote: > On 14/02/2014 17:42, Michael McMahon wrote: >> Could I get the following change reviewed please? >> >> http://cr.openjdk.java.net/~michaelm/8034853/webrev.1/ >> >> The change is to remove the class sun.misc.ClassLoaderUtil. >> The functionality provided by this class is now in the public >> java.net.URLClassloader.close() method. > The removal is good. Maybe the test should be moved to URLClassLoader > as it doesn't make sense to have it in ClassLoaderUtil anymore. > > -Alan. I think I'd probably just delete the test since URLClassLoader.close has its own tests, I thought it might be useful as a point of reference in case there could be someone still using the sun.misc API. Also, I think mercurial won't delete the empty directory after the file is gone - not that that matters very much. Thanks Michael From david.lloyd at redhat.com Fri Feb 14 18:56:38 2014 From: david.lloyd at redhat.com (David M. Lloyd) Date: Fri, 14 Feb 2014 12:56:38 -0600 Subject: A hole in the serialization spec In-Reply-To: <52FE59EF.3030606@redhat.com> References: <52F4F6BB.2010207@redhat.com> <52F8E807.2090403@oracle.com> <52F8F21C.80205@redhat.com> <52FB8E5C.9020502@oracle.com> <52FB9248.6010703@redhat.com> <26199217-BAE6-47AC-AFC8-4DE90726B8C0@oracle.com> <52FD0328.3050304@redhat.com> <52FE3CB2.5060408@redhat.com> <52FE44E7.6060103@oracle.com> <52FE59EF.3030606@redhat.com> Message-ID: <52FE66E6.8040700@redhat.com> Interestingly, if the behavior in this case was to write empty values on the write side and read and discard on the read side, both the behavior and the serialized format of java.util.Date would be identical to today; though other pathological classes might be incompatible, they were already in undefined-land. On 02/14/2014 12:01 PM, David M. Lloyd wrote: > Yes, however given that the lack of reading/writing fields makes it > undefined *in general*, you can only say "the behavior is defined as > long as this undefined behavior is actually defined in this (risky) > manner". > > Put another way, since the behavior of not reading/writing fields is not > defined, this class is then by spec not portable between JVMs (after > all, undefined behavior could mean anything at all). That's why I was > saying that the spec should define exactly what will physically happen > if you do this (from both the perspective of the "other end" of the > serialization API, *and* the wire protocol), even if it's considered > poor practice - because then at least you have guaranteed > interoperability (which becomes even more important with the knowledge > that JDK classes have this issue). > > On 02/14/2014 10:31 AM, roger riggs wrote: >> Hi David, >> >> I would quibble that the serialized form of java.util.Date is defined. >> The behavior is defined by the writeObject and readObject methods. >> They write and read a time computed from and restored to the state of >> the object. >> >> Roger >> >> On 2/14/2014 10:56 AM, David M. Lloyd wrote: >>> On 02/13/2014 11:38 AM, David M. Lloyd wrote: >>>> On 02/13/2014 10:29 AM, Chris Hegarty wrote: >>>>> On 12 Feb 2014, at 15:24, David M. Lloyd >>>>> wrote: >>>>> >>>>>> That's a quote from the serialization spec. I take it to mean, >>>>>> "Don't write fields and everything might go to hell". In practice, >>>>>> if the reading side doesn't read fields, things end up more or less >>>>>> OK, as evidenced by various classes in the wild. But it's not hard >>>>>> to imagine a scenario in which a class change could cause protocol >>>>>> corruption. >>>>>> >>>>>> I think the specifics of the quote relate to this kind of class >>>>>> change; in particular, if a class is deleted from the hierarchy on >>>>>> the read side, and that class corresponds to the class that had the >>>>>> misbehaving writeObject, I suspect that things will break at that >>>>>> point as the read side will probably try to consume and discard the >>>>>> field information for that class, which will be missing (it will >>>>>> start reading the next class' fields instead I think). >>>>> >>>>> Yes, possibly. And who knows what fields/values may be read and >>>>> mistaken for the wrong object in the hierarchy. So ?undefined' >>>>> behaviour seems right to me. >>>> >>>> I think the behavior is well-defined, just "bad", which is my >>>> point. If >>>> the exact current is spec'd out as-is then at least we can be >>>> assured of >>>> the same bad behavior across implementations. If the behavior is >>>> changed such that fields are read/written but discarded, without >>>> updating the spec, then the "undefined" behavior at least becomes >>>> safer. >>>> If the behavior is changed, *and* the spec is updated, then we get >>>> both benefits, but at the cost that all previous implementations will >>>> not be compliant with the spec. >>>> >>>> All options seem to have a cost though. >>> >>> In the JDK, java.util.Date does not read/write fields. Perhaps others >>> as well. Given that the behavior is presently undefined, that means >>> the serialized representation of java.util.Date (and any other such >>> non-conforming classes) are also undefined. >>> >> > > -- - DML From chris.hegarty at oracle.com Fri Feb 14 19:15:16 2014 From: chris.hegarty at oracle.com (Chris Hegarty) Date: Fri, 14 Feb 2014 19:15:16 +0000 Subject: RFR: 8034853 remove sun.misc.ClassLoaderUtil In-Reply-To: <52FE5578.9050006@oracle.com> References: <52FE5578.9050006@oracle.com> Message-ID: Looks good. -Chris. > On 14 Feb 2014, at 17:42, Michael McMahon wrote: > > Could I get the following change reviewed please? > > http://cr.openjdk.java.net/~michaelm/8034853/webrev.1/ > > The change is to remove the class sun.misc.ClassLoaderUtil. > The functionality provided by this class is now in the public > java.net.URLClassloader.close() method. > > Thanks > Michael From ivan.gerasimov at oracle.com Fri Feb 14 19:48:18 2014 From: ivan.gerasimov at oracle.com (Ivan Gerasimov) Date: Fri, 14 Feb 2014 23:48:18 +0400 Subject: RFR [4682009] Typo in javadocs in javax/naming Message-ID: <52FE7302.1060406@oracle.com> Hello! May I please have a review of the fix? It's not meant to be a proof reading, I only fixed some obvious typos. Some of them were reported at the time of java 1.4, so it is probably the time to have them fixed. BUG: https://bugs.openjdk.java.net/browse/JDK-4682009 WEBREV: http://cr.openjdk.java.net/~igerasim/4682009/0/webrev/ Sincerely yours, Ivan From Lance.Andersen at oracle.com Fri Feb 14 20:13:05 2014 From: Lance.Andersen at oracle.com (Lance Andersen - Oracle) Date: Fri, 14 Feb 2014 15:13:05 -0500 Subject: RFR [4682009] Typo in javadocs in javax/naming In-Reply-To: <52FE7302.1060406@oracle.com> References: <52FE7302.1060406@oracle.com> Message-ID: <57AF68CB-C26B-4655-9E8A-D5425DC58040@oracle.com> Looks fine. assume the will be addressed as part of a full sweep of javax/naming On Feb 14, 2014, at 2:48 PM, Ivan Gerasimov wrote: > Hello! > > May I please have a review of the fix? > > It's not meant to be a proof reading, I only fixed some obvious typos. > Some of them were reported at the time of java 1.4, so it is probably the time to have them fixed. > > BUG: https://bugs.openjdk.java.net/browse/JDK-4682009 > WEBREV: http://cr.openjdk.java.net/~igerasim/4682009/0/webrev/ > > Sincerely yours, > Ivan -------------- next part -------------- Lance Andersen| Principal Member of Technical Staff | +1.781.442.2037 Oracle Java Engineering 1 Network Drive Burlington, MA 01803 Lance.Andersen at oracle.com From Alan.Bateman at oracle.com Fri Feb 14 20:14:41 2014 From: Alan.Bateman at oracle.com (Alan Bateman) Date: Fri, 14 Feb 2014 20:14:41 +0000 Subject: RFR 9: 8034903: Add Logging of Process.start arguments and resulting pid In-Reply-To: References: <52FD3897.7070903@oracle.com> Message-ID: <52FE7931.4020408@oracle.com> On 14/02/2014 18:10, Martin Buchholz wrote: > I'm not excited about having logging statements added throughout the jdk > implementation. PlatformLogger is an implementation detail, so real users > cannot benefit. Your OS should already provide facilities to let you trace > process creation. E.g. on Linux you can do this with strace (and I have > done this myself in the past to debug subprocess creation). So I'd say > adding logging just to provide a small rarely used convenience for jdk > developers is Not Worthwhile. > Just on PlatformLogger then it just forwards to j.u.logging when it is present. -Alan. From Alan.Bateman at oracle.com Fri Feb 14 20:17:04 2014 From: Alan.Bateman at oracle.com (Alan Bateman) Date: Fri, 14 Feb 2014 20:17:04 +0000 Subject: RFR: 8034853 remove sun.misc.ClassLoaderUtil In-Reply-To: <52FE60EF.5090909@oracle.com> References: <52FE5578.9050006@oracle.com> <52FE5E5C.60709@oracle.com> <52FE60EF.5090909@oracle.com> Message-ID: <52FE79C0.3080302@oracle.com> On 14/02/2014 18:31, Michael McMahon wrote: > > I think I'd probably just delete the test since URLClassLoader.close > has its own tests, > I thought it might be useful as a point of reference > in case there could be someone still using the sun.misc API. > Also, I think mercurial won't delete the empty directory after the > file is gone - not that that matters very much. If the updated test just duplicate the URLClassLoader tests then deleting the old test seems reasonable to me. I don't see a need to leave a test in a location that no longer corresponds to anything. -Alan From Alan.Bateman at oracle.com Fri Feb 14 20:20:49 2014 From: Alan.Bateman at oracle.com (Alan Bateman) Date: Fri, 14 Feb 2014 20:20:49 +0000 Subject: RFR [4682009] Typo in javadocs in javax/naming In-Reply-To: <52FE7302.1060406@oracle.com> References: <52FE7302.1060406@oracle.com> Message-ID: <52FE7AA1.3080800@oracle.com> On 14/02/2014 19:48, Ivan Gerasimov wrote: > Hello! > > May I please have a review of the fix? > > It's not meant to be a proof reading, I only fixed some obvious typos. > Some of them were reported at the time of java 1.4, so it is probably > the time to have them fixed. > > BUG: https://bugs.openjdk.java.net/browse/JDK-4682009 > WEBREV: http://cr.openjdk.java.net/~igerasim/4682009/0/webrev/ Looks good and nice to get these fixed after all these years. -Alan From ivan.gerasimov at oracle.com Fri Feb 14 20:27:10 2014 From: ivan.gerasimov at oracle.com (Ivan Gerasimov) Date: Sat, 15 Feb 2014 00:27:10 +0400 Subject: RFR [4682009] Typo in javadocs in javax/naming In-Reply-To: <57AF68CB-C26B-4655-9E8A-D5425DC58040@oracle.com> References: <52FE7302.1060406@oracle.com> <57AF68CB-C26B-4655-9E8A-D5425DC58040@oracle.com> Message-ID: <52FE7C1E.2030904@oracle.com> On 15.02.2014 0:13, Lance Andersen - Oracle wrote: > Looks fine. assume the will be addressed as part of a full sweep of javax/naming > On Feb 14, 2014, at 2:48 PM, Ivan Gerasimov wrote: Thanks! Yes, I didn't touch formatting, only typos. >> Hello! >> >> May I please have a review of the fix? >> >> It's not meant to be a proof reading, I only fixed some obvious typos. >> Some of them were reported at the time of java 1.4, so it is probably the time to have them fixed. >> >> BUG: https://bugs.openjdk.java.net/browse/JDK-4682009 >> WEBREV: http://cr.openjdk.java.net/~igerasim/4682009/0/webrev/ >> >> Sincerely yours, >> Ivan > > > Lance Andersen| Principal Member of Technical Staff | +1.781.442.2037 > Oracle Java Engineering > 1 Network Drive > Burlington, MA 01803 > Lance.Andersen at oracle.com > From mandy.chung at oracle.com Fri Feb 14 21:44:56 2014 From: mandy.chung at oracle.com (Mandy Chung) Date: Fri, 14 Feb 2014 13:44:56 -0800 Subject: RFR: 8034853 remove sun.misc.ClassLoaderUtil In-Reply-To: <52FE60EF.5090909@oracle.com> References: <52FE5578.9050006@oracle.com> <52FE5E5C.60709@oracle.com> <52FE60EF.5090909@oracle.com> Message-ID: <52FE8E58.8020907@oracle.com> On 2/14/14 10:31 AM, Michael McMahon wrote: > On 14/02/14 18:20, Alan Bateman wrote: >> On 14/02/2014 17:42, Michael McMahon wrote: >>> Could I get the following change reviewed please? >>> >>> http://cr.openjdk.java.net/~michaelm/8034853/webrev.1/ >>> >>> The change is to remove the class sun.misc.ClassLoaderUtil. >>> The functionality provided by this class is now in the public >>> java.net.URLClassloader.close() method. >> The removal is good. Maybe the test should be moved to URLClassLoader >> as it doesn't make sense to have it in ClassLoaderUtil anymore. >> >> -Alan. > > I think I'd probably just delete the test since URLClassLoader.close > has its own tests, The change looks good to me. I think URLClassLoader/closetest test covers this case already. I agree with Alan that you can simply delete this old test. Mandy From stuart.marks at oracle.com Fri Feb 14 22:24:32 2014 From: stuart.marks at oracle.com (Stuart Marks) Date: Fri, 14 Feb 2014 14:24:32 -0800 Subject: RFR: 8034853 remove sun.misc.ClassLoaderUtil In-Reply-To: <52FE60EF.5090909@oracle.com> References: <52FE5578.9050006@oracle.com> <52FE5E5C.60709@oracle.com> <52FE60EF.5090909@oracle.com> Message-ID: <52FE97A0.4030306@oracle.com> On 2/14/14 10:31 AM, Michael McMahon wrote: > Also, I think mercurial won't delete the empty directory after the file is gone > - not that that matters very much. If you delete the last file in a directory, the empty dir might stay around in your working copy. But the directory will be absent from new clones; Mercurial doesn't track empty directories. Go ahead and delete! s'marks From stuart.marks at oracle.com Sat Feb 15 01:47:24 2014 From: stuart.marks at oracle.com (Stuart Marks) Date: Fri, 14 Feb 2014 17:47:24 -0800 Subject: RFR: JDK-8032050: TEST_BUG: java/rmi/activation/Activatable/shutdownGracefully/ShutdownGracefully.java fails intermittently In-Reply-To: <52FC8178.9000500@oracle.com> References: <52E20974.7060706@oracle.com> <52E31F53.3070702@oracle.com> <52E4959C.3020907@oracle.com> <52E8AADE.6000300@oracle.com> <52EB60A8.7070307@oracle.com> <133FA08E-5731-437D-BB1F-AD9872CBDD36@oracle.com> <52F96FA8.3010102@oracle.com> <0C67EF5A-7889-49C5-8079-BDA7FF4CF1F0@oracle.com> <52FBE7B4.5060802@oracle.com> <52FC8178.9000500@oracle.com> Message-ID: <52FEC72C.1070704@oracle.com> Hi Tristan, OK, we're getting close. Just a couple things about ShutdownGracefully.java. It's a bit clumsy to have both a boolean and a message string to keep track of the state of the test, but by itself this isn't such a big deal. A bigger deal is the lack of exception chaining. If we catch an unexpected exception at line 162, its class and message are printed out, but its stack trace isn't. If this were to occur it might be fiendishly difficult to debug. The TimeoutException isn't chained up either, but this isn't so bad, since there's only one place the timeout can occur. Still, it's good to chain exceptions where possible. There's another failure case that doesn't have an exception at all, which is when the registration request we're expecting to fail actually succeeds. This doesn't have an exception, but we should consider creating one. Here's an approach to getting rid of the boolean+string and also chaining up exceptions. The key insight is that an exception can be created in a different place from where it's thrown. Instead of the boolean+stream, have a variable of type Exception that's initialized to null. Anyplace where we catch an exception that indicates failure, fill in this variable. The idea is that if this exception variable is non-null, a failure has occurred. The exception being non-null replaces the boolean, and the exception message replaces the failure string. In addition, the exception has a stack trace, which is essential for debugging. For the case where we don't get the expected exception (i.e., registration succeeds unexpectedly), simply set the exception variable to a newly created exception, but don't throw it yet. For example, exception = new RuntimeException( "The registration request succeeded unexpectedly"); At the place where we catch an unexpected exception, wrap the caught exception in a new RuntimeException with a message like "caught unexpected exception". The reason to do this is so we can add an additional message. The stack trace will also be a bit easier to follow. For the timeout, just assign the TimeoutException to the exception variable. Also, at each location where the exception variable is assigned to, print out a message at that point. It will at least show the state of the test to be a failure. The reason is that, if rmid.destroy() were to throw an exception from the finally-block, our carefully recorded exception state will be thrown away. (An alternative would be to put this into its own try-block, and add any exceptions caught from it to the suppressed exception list of the exception variable, but it's not clear to me that this is worth it.) At the end of the test, if the exception variable is non-null, call TestLibrary.bomb() with it to fail the test. Finally, one typo: "prcoess". Thanks for updating this webrev again. s'marks On 2/13/14 12:25 AM, Tristan Yan wrote: > Thank you Stuart > I have fixed comment in JavaVM.java. Dealing with different cases in > ShutdownGracefully.java, two variables were added. One is a flag indicate test > passed or not. Other variable keeps the error message when test failed. I put > TestLibrary.bomb in the bottom of the main method which only shows test fail > message. > Could you review it again > http://cr.openjdk.java.net/~tyan/JDK-8032050/webrev.04/ > Tristan > > On 02/13/2014 05:29 AM, Stuart Marks wrote: >> Hi Tristan, >> >> JavaVM.waitFor looks mostly fine. The indentation of the start of the >> waitFor(timeout) javadoc comment is a bit off, though; please fix. >> >> There are still some adjustments that need to be made in >> ShutdownGracefully.java. Both have to do with the case where the last >> registration succeeds unexpectedly -- this is the one that we expect to fail. >> >> First, if this registration has succeeded unexpectedly, that means rmid is >> still running. If that occurs, the call to rmid.waitFor(timeout) will >> inevitably time out. It may be worth calling rmid.destroy() directly at this >> point. >> >> Second, still in the succeeded-unexpectedly case, at line 154 >> TestLibrary.bomb() is called. This throws an exception, but it's caught by the >> catch-block at lines 157-158, which calls TestLibrary.bomb() again, saying >> "unexpected exception". Except that this is kind of expected, since it was >> thrown from an earlier call to TestLibrary.bomb(). This is quite confusing. >> >> There are several cases that need to be handled. >> >> 1. Normal case. Registration fails as expected, rmid has terminated >> gracefully. Test passes. >> >> 2. Rmid is still running and has processed the registration request >> successfully. Need to kill rmid and fail the test. >> >> 3. Rmid is still running but is in some bad state where the registration >> request failed. Need to kill rmid and fail the test. >> >> 4. Some other unexpected failure. This is what the catch and finally blocks at >> lines 157-161 are for. >> >> These four cases need to be handled independently. Ideally they should be >> separated from the cleanup code. As noted above, you don't want to throw an >> exception from the try-block, because it will get caught by your own catch >> block. Similarly, it's tempting to return from the midst of the try-block in >> the success case, but this still runs the finally-block. This can be quite >> confusing. >> >> A typical technique for dealing with this kind of issue is to record results >> of operations from within the try block, and then analyze the results outside, >> throwing a test failure (TestLibrary.bomb) at that point, where it won't be >> caught by the test's own catch block. >> >> Editoral: >> - line 153, there should be a space between 'if' and the opening paren >> - line 156, typo, "gracefuuly" >> >> Finally, it would be helpful if you could get webrev to generate the actual >> changeset instead of the plain patch, per my other review email. >> >> Thanks, >> >> s'marks >> >> >> On 2/11/14 9:39 PM, Tristan Yan wrote: >>> Thank you for your thorough mail. This is very educational. I took you advice >>> and generated a new webrev for this. >>> >>> http://cr.openjdk.java.net/~tyan/JDK-8032050/webrev.03/ >>> I appreciate you can review this again. >>> Regards >>> Tristan >>> >>> >>> On Feb 11, 2014, at 8:32 AM, Stuart Marks >> > wrote: >>> >>>> Hi Tristan, >>>> >>>> Sorry about my recurring delays. >>>> >>>> Several comments on these changes. >>>> >>>> JavaVM.java -- >>>> >>>> The waitFor(timeout) method is mostly ok. The new thread started at line 208 >>>> and following seems unnecessary, though. This code is reached when a timeout >>>> occurs, so throwing TimeoutException is the only thing necessary in this case. >>>> Should the code to start the new thread be removed? >>>> >>>> There should be a similar check for vm == null as in the waitFor() [no args] >>>> method. >>>> >>>> ShutdownGracefully.java -- >>>> >>>> The condition that's checked after calling rmid.waitFor(SHUTDOWN_TIMEOUT) is >>>> incorrect. It's testing the exit status against zero. Offhand, when and if >>>> rmid exits, it might exit with a nonzero exit status. If rmid has exited at >>>> this point, then the test should succeed. >>>> >>>> Instead of testing against zero, the code should catch TimeoutException, which >>>> means that rmid is still running. It's probably reasonable to catch >>>> TimeoutException, print a message, and then let the finally-block destroy the >>>> rmid. Calling TestLibrary.bomb() from within the try-block is confusing, since >>>> that method throws an exception, which is then caught by the catch-block, when >>>> then calls TestLibrary.bomb() again. >>>> >>>> We should also make sure to test the success case properly. If rmid.waitFor() >>>> returns in a timely fashion without throwing TimeoutException, it doesn't >>>> matter what the exit status is. (It might be worth printing it out.) At that >>>> point we know that rmid *has* exited gracefully, so we need to set rmid to >>>> null so that the finally-block doesn't attempt to destroy rmid redundantly. >>>> Some additional messages about rmid having exited and the test passing are >>>> also warranted for this case. >>>> >>>> Some additional cleanup can be done here as well, over and above the changes >>>> you've proposed. (This stuff is left over from earlier RMI test messes.) In >>>> order to shut down an active object, the code here spawns a new thread that >>>> sleeps for a while and then deactivates this object. This isn't necessary. (It >>>> might have been necessary in the past.) It's sufficient simply to unexport >>>> this object and then deactivate it, directly within the shutdown() method. See >>>> >>>> test/java/rmi/activation/ActivationSystem/unregisterGroup/UnregisterGroup.java >>>> >>>> for an example of this. In addition, the run() method can be removed, and the >>>> "implements Runnable" declaration can also be removed from the >>>> ShutdownGracefully test class. >>>> >>>> Finally, revisiting some code farther up in the test, the try-block at lines >>>> 135-140 issues a registration request that the test expects to fail. If it >>>> succeeds, the message at line 139 isn't very clear; it should say that the >>>> registration request succeeded unexpectedly. This should cause the test to >>>> fail. We still probably want to go through the waitFor(timeout) path and >>>> eventual rmid cleanup, but a flag should be set here to ensure that the test >>>> indeed fails if the registration succeeds unexpectedly, and the messages >>>> should clearly indicate that this is going on. >>>> >>>> A good way to test this last case is to change rmid's security manager to the >>>> normal security manager java.lang.SecurityManager instead of >>>> TestSecurityManager. >>>> >>>> Thanks, >>>> >>>> s'marks >>>> >>>> >>>> >>>> >>>> On 2/10/14 2:59 AM, Tristan Yan wrote: >>>>> Hi Stuart >>>>> Could you help to review this. >>>>> Thank you >>>>> Tristan >>>>> >>>>> On Jan 31, 2014, at 4:36 PM, Tristan Yan >>>> >>>>> > wrote: >>>>> >>>>>> Thank you for fixing JDK-8023541. Then I leave ActivationLibrary.java for >>>>>> now. >>>>>> I still did some clean up following your suggestion. >>>>>> 1. I changed waitFor(long timeout) method, this method is going to use code >>>>>> like Process.waitFor(timeout, unit). This can be backported to JDK7. Also >>>>>> exitValue is kept as a return value. For making sure there is no Pipe leak, a >>>>>> cleanup thread will start when timeout happens. >>>>>> 2. Change in ShutdownGracefully is a little tricky. I think we should just >>>>>> destroy JVM once exception is thrown. So I move the wait logic into try block >>>>>> instead keep them in finally block. >>>>>> Can you receive it again. >>>>>> http://cr.openjdk.java.net/~tyan/JDK-8032050/webrev.02/ >>>>>> Thank you >>>>>> Tristan >>>>>> >>>>>> On 01/29/2014 03:16 PM, Stuart Marks wrote: >>>>>>> Hi Tristan, >>>>>>> >>>>>>> I don't want to put the workaround into ActivationLibrary.rmidRunning() >>>>>>> for a >>>>>>> null return from the lookup, because this is only a workaround for an actual >>>>>>> bug in rmid initialization. See the review I just posted for JDK-8023541. >>>>>>> >>>>>>> Adding JavaVM.waitFor(timeout) is something that would be useful in general, >>>>>>> but it needs to be handled carefully. It uses the new >>>>>>> Process.waitFor(timeout, unit) which is new in Java SE 8; this makes >>>>>>> backporting to 7 more complicated. Not clear whether we'll do so, but I >>>>>>> don't >>>>>>> want to forclose the opportunity without discussion. It's also not clear how >>>>>>> one can get the vm's exit status after JavaVM.waitFor() has returned true. >>>>>>> With the Process API it's possible simply to call waitFor() or exitValue(). >>>>>>> With JavaVM, a new API needs to be created, or the rule has to be >>>>>>> established >>>>>>> that one must call JavaVM.waitFor() to collect the exit status as well as to >>>>>>> close the pipes from the subprocess. If JavaVM.waitFor(timeout, unit) is >>>>>>> called without subsequently calling JavaVM.waitFor(), the pipes are leaked. >>>>>>> >>>>>>> In ShutdownGracefully.java, the finally-block needs to check to see if rmid >>>>>>> is still running, and if it is, to shut it down. Simply calling >>>>>>> waitFor(timeout, unit) isn't sufficient, because if the rmid process is >>>>>>> still >>>>>>> running, it will be left running. >>>>>>> >>>>>>> The straightforward approach would be to call >>>>>>> ActivationLibrary.rmidRunning() >>>>>>> to test if it's still running. Unfortunately this isn't quite right, because >>>>>>> rmidRunning() has a timeout loop in it -- which should probably be removed. >>>>>>> (I think there's a bug for this.) Another approach would be simply to call >>>>>>> rmid.destroy(). This calls rmid's shutdown() method first, which is >>>>>>> reasonable, but I'm not sure it kills the process if that fails. In any >>>>>>> case, >>>>>>> this already has a timeout loop waiting for the process to die, so >>>>>>> ShutdownGracefully.java needn't use a new waitFor(timeout, unit) call. >>>>>>> >>>>>>> Removing the commented-out code that starts with "no longer needed" is good, >>>>>>> and removing the ShutdownDetectThread is also good, since that's >>>>>>> unnecessary. >>>>>>> >>>>>>> There are some more cleanups I have in mind here but I'd like to see a >>>>>>> revised webrev before proceeding. >>>>>>> >>>>>>> Thanks, >>>>>>> >>>>>>> s'marks >>>>>>> >>>>>>> On 1/25/14 8:57 PM, Tristan Yan wrote: >>>>>>>> Hi Stuart >>>>>>>> Thank you for your review and suggestion. >>>>>>>> Yes, since this failure mode is very hard to be reproduced. I guess it's >>>>>>>> make sense to do some hack. And I also noticed in >>>>>>>> ActivationLibrary.rmidRunning. It does try to look up ActivationSystem but >>>>>>>> doesn't check if it's null. So I add the logic to make sure we will look up >>>>>>>> the non-null ActivationSystem. Also I did some cleanup if you don't mind. >>>>>>>> Add a waitFor(long timeout, TimeUnit unit) for JavaVM. Which we can have a >>>>>>>> better waitFor control. >>>>>>>> I appreciate you can review the code again. >>>>>>>> http://cr.openjdk.java.net/~tyan/JDK-8032050/webrev.01/ >>>>>>>> Thank you >>>>>>>> Tristan >>>>>>>> >>>>>>>> >>>>>>>> On 01/25/2014 10:20 AM, Stuart Marks wrote: >>>>>>>>> On 1/23/14 10:34 PM, Tristan Yan wrote: >>>>>>>>>> Hi All >>>>>>>>>> Could you review the bug fix for JDK-8032050. >>>>>>>>>> >>>>>>>>>> http://cr.openjdk.java.net/~tyan/JDK-8032050/webrev.00/ >>>>>>>>>> >>>>>>>>>> Description: >>>>>>>>>> This rare happened failure caused because when RMID starts. It don't >>>>>>>>>> guarantee >>>>>>>>>> sun.rmi.server.Activation.startActivation finishes. >>>>>>>>>> Fix by adding a iterative getSystem with a 5 seconds timeout. >>>>>>>>> >>>>>>>>> Hi Tristan, >>>>>>>>> >>>>>>>>> Adding a timing/retry loop into this test isn't the correct approach for >>>>>>>>> fixing this test. >>>>>>>>> >>>>>>>>> The timing loop isn't necessary because there is already a timing loop in >>>>>>>>> RMID.start() in the RMI test library. (There's another timing loop in >>>>>>>>> ActivationLibrary.rmidRunning() which should probably be removed.) So the >>>>>>>>> intent of this library call is that it start rmid and wait for it to >>>>>>>>> become >>>>>>>>> ready. That logic doesn't need to be added to the test. >>>>>>>>> >>>>>>>>> In the bug report JDK-8032050 you had mentioned that the >>>>>>>>> NullPointerException was suspicious. You're right! I took a look and it >>>>>>>>> seemed like it was related to JDK-8023541, and I added a note to this >>>>>>>>> effect to the bug report. The problem here is that rmid can come up and >>>>>>>>> transiently return null instead of the stub of the activation system. >>>>>>>>> That's what JDK-8023541 covers. I think that rmid itself needs to be >>>>>>>>> fixed, >>>>>>>>> though modifying the timing loop in the RMI test library to wait for rmid >>>>>>>>> to come up *and* for the lookup to return non-null is an easy way to fix >>>>>>>>> the problem. (Or at least cover it up.) >>>>>>>>> >>>>>>>>> The next step in the analysis is to determine, given that >>>>>>>>> ActivationLibrary.getSystem can sometimes return null, whether this has >>>>>>>>> actually caused this test failure. This is pretty easy to determine; just >>>>>>>>> hack in a line "system = null" in the right place and run the test. I've >>>>>>>>> done this, and the test times out and the output log is pretty much >>>>>>>>> identical to the one in the bug report. (I recommend you try this >>>>>>>>> yourself.) So I think it's fairly safe to say that the problem in >>>>>>>>> JDK-8023541 has caused the failure listed in JDK-8032050. >>>>>>>>> >>>>>>>>> I can see a couple ways to proceed here. One way is just to close this out >>>>>>>>> as a duplicate of JDK-8023541 since that bug caused this failure. >>>>>>>>> >>>>>>>>> Another is that this test could use some cleaning up. This bug certainly >>>>>>>>> covers a failure, but the messages emitted are confusing and in some cases >>>>>>>>> completely wrong. For example, the "rmid has shutdown" message at line 180 >>>>>>>>> is incorrect, because in this case rmid is still running and the wait() >>>>>>>>> call has timed out. Most of the code here can be replaced with calls to >>>>>>>>> various bits of the RMI test library. There are a bunch of other things in >>>>>>>>> this test that could be cleaned up as well. >>>>>>>>> >>>>>>>>> It's up to you how you'd like to proceed. >>>>>>>>> >>>>>>>>> s'marks >>>>>>>> >>>>>>> >>>>>> >>>>> >>> > From tristan.yan at oracle.com Sat Feb 15 02:53:15 2014 From: tristan.yan at oracle.com (Tristan Yan) Date: Sat, 15 Feb 2014 10:53:15 +0800 Subject: RFR: JDK-8032050: TEST_BUG: java/rmi/activation/Activatable/shutdownGracefully/ShutdownGracefully.java fails intermittently In-Reply-To: <52FEC72C.1070704@oracle.com> References: <52E20974.7060706@oracle.com> <52E31F53.3070702@oracle.com> <52E4959C.3020907@oracle.com> <52E8AADE.6000300@oracle.com> <52EB60A8.7070307@oracle.com> <133FA08E-5731-437D-BB1F-AD9872CBDD36@oracle.com> <52F96FA8.3010102@oracle.com> <0C67EF5A-7889-49C5-8079-BDA7FF4CF1F0@oracle.com> <52FBE7B4.5060802@oracle.com> <52FC8178.9000500@oracle.com> <52FEC72C.1070704@oracle.com> Message-ID: <78E6DE88-531C-4548-B951-562C9BA98BE2@oracle.com> Thank you Stuart This is nice. I thought two variables were weird but I didn?t figure out the right solution. Also I wasn?t so sure why we print out so many messages now it?s clear to me. http://cr.openjdk.java.net/~tyan/JDK-8032050/webrev.05/ I?m sorry I have to ask you review this again. regards Tristan On Feb 15, 2014, at 9:47 AM, Stuart Marks wrote: > Hi Tristan, > > OK, we're getting close. Just a couple things about ShutdownGracefully.java. > > It's a bit clumsy to have both a boolean and a message string to keep track of the state of the test, but by itself this isn't such a big deal. > > A bigger deal is the lack of exception chaining. If we catch an unexpected exception at line 162, its class and message are printed out, but its stack trace isn't. If this were to occur it might be fiendishly difficult to debug. > > The TimeoutException isn't chained up either, but this isn't so bad, since there's only one place the timeout can occur. Still, it's good to chain exceptions where possible. > > There's another failure case that doesn't have an exception at all, which is when the registration request we're expecting to fail actually succeeds. This doesn't have an exception, but we should consider creating one. > > Here's an approach to getting rid of the boolean+string and also chaining up exceptions. The key insight is that an exception can be created in a different place from where it's thrown. > > Instead of the boolean+stream, have a variable of type Exception that's initialized to null. Anyplace where we catch an exception that indicates failure, fill in this variable. The idea is that if this exception variable is non-null, a failure has occurred. The exception being non-null replaces the boolean, and the exception message replaces the failure string. In addition, the exception has a stack trace, which is essential for debugging. > > For the case where we don't get the expected exception (i.e., registration succeeds unexpectedly), simply set the exception variable to a newly created exception, but don't throw it yet. For example, > > exception = new RuntimeException( > "The registration request succeeded unexpectedly"); > > At the place where we catch an unexpected exception, wrap the caught exception in a new RuntimeException with a message like "caught unexpected exception". The reason to do this is so we can add an additional message. The stack trace will also be a bit easier to follow. > > For the timeout, just assign the TimeoutException to the exception variable. > > Also, at each location where the exception variable is assigned to, print out a message at that point. It will at least show the state of the test to be a failure. The reason is that, if rmid.destroy() were to throw an exception from the finally-block, our carefully recorded exception state will be thrown away. (An alternative would be to put this into its own try-block, and add any exceptions caught from it to the suppressed exception list of the exception variable, but it's not clear to me that this is worth it.) > > At the end of the test, if the exception variable is non-null, call TestLibrary.bomb() with it to fail the test. > > Finally, one typo: "prcoess". > > Thanks for updating this webrev again. > > s'marks > > On 2/13/14 12:25 AM, Tristan Yan wrote: >> Thank you Stuart >> I have fixed comment in JavaVM.java. Dealing with different cases in >> ShutdownGracefully.java, two variables were added. One is a flag indicate test >> passed or not. Other variable keeps the error message when test failed. I put >> TestLibrary.bomb in the bottom of the main method which only shows test fail >> message. >> Could you review it again >> http://cr.openjdk.java.net/~tyan/JDK-8032050/webrev.04/ >> Tristan >> >> On 02/13/2014 05:29 AM, Stuart Marks wrote: >>> Hi Tristan, >>> >>> JavaVM.waitFor looks mostly fine. The indentation of the start of the >>> waitFor(timeout) javadoc comment is a bit off, though; please fix. >>> >>> There are still some adjustments that need to be made in >>> ShutdownGracefully.java. Both have to do with the case where the last >>> registration succeeds unexpectedly -- this is the one that we expect to fail. >>> >>> First, if this registration has succeeded unexpectedly, that means rmid is >>> still running. If that occurs, the call to rmid.waitFor(timeout) will >>> inevitably time out. It may be worth calling rmid.destroy() directly at this >>> point. >>> >>> Second, still in the succeeded-unexpectedly case, at line 154 >>> TestLibrary.bomb() is called. This throws an exception, but it's caught by the >>> catch-block at lines 157-158, which calls TestLibrary.bomb() again, saying >>> "unexpected exception". Except that this is kind of expected, since it was >>> thrown from an earlier call to TestLibrary.bomb(). This is quite confusing. >>> >>> There are several cases that need to be handled. >>> >>> 1. Normal case. Registration fails as expected, rmid has terminated >>> gracefully. Test passes. >>> >>> 2. Rmid is still running and has processed the registration request >>> successfully. Need to kill rmid and fail the test. >>> >>> 3. Rmid is still running but is in some bad state where the registration >>> request failed. Need to kill rmid and fail the test. >>> >>> 4. Some other unexpected failure. This is what the catch and finally blocks at >>> lines 157-161 are for. >>> >>> These four cases need to be handled independently. Ideally they should be >>> separated from the cleanup code. As noted above, you don't want to throw an >>> exception from the try-block, because it will get caught by your own catch >>> block. Similarly, it's tempting to return from the midst of the try-block in >>> the success case, but this still runs the finally-block. This can be quite >>> confusing. >>> >>> A typical technique for dealing with this kind of issue is to record results >>> of operations from within the try block, and then analyze the results outside, >>> throwing a test failure (TestLibrary.bomb) at that point, where it won't be >>> caught by the test's own catch block. >>> >>> Editoral: >>> - line 153, there should be a space between 'if' and the opening paren >>> - line 156, typo, "gracefuuly" >>> >>> Finally, it would be helpful if you could get webrev to generate the actual >>> changeset instead of the plain patch, per my other review email. >>> >>> Thanks, >>> >>> s'marks >>> >>> >>> On 2/11/14 9:39 PM, Tristan Yan wrote: >>>> Thank you for your thorough mail. This is very educational. I took you advice >>>> and generated a new webrev for this. >>>> >>>> http://cr.openjdk.java.net/~tyan/JDK-8032050/webrev.03/ >>>> I appreciate you can review this again. >>>> Regards >>>> Tristan >>>> >>>> >>>> On Feb 11, 2014, at 8:32 AM, Stuart Marks >>> > wrote: >>>> >>>>> Hi Tristan, >>>>> >>>>> Sorry about my recurring delays. >>>>> >>>>> Several comments on these changes. >>>>> >>>>> JavaVM.java -- >>>>> >>>>> The waitFor(timeout) method is mostly ok. The new thread started at line 208 >>>>> and following seems unnecessary, though. This code is reached when a timeout >>>>> occurs, so throwing TimeoutException is the only thing necessary in this case. >>>>> Should the code to start the new thread be removed? >>>>> >>>>> There should be a similar check for vm == null as in the waitFor() [no args] >>>>> method. >>>>> >>>>> ShutdownGracefully.java -- >>>>> >>>>> The condition that's checked after calling rmid.waitFor(SHUTDOWN_TIMEOUT) is >>>>> incorrect. It's testing the exit status against zero. Offhand, when and if >>>>> rmid exits, it might exit with a nonzero exit status. If rmid has exited at >>>>> this point, then the test should succeed. >>>>> >>>>> Instead of testing against zero, the code should catch TimeoutException, which >>>>> means that rmid is still running. It's probably reasonable to catch >>>>> TimeoutException, print a message, and then let the finally-block destroy the >>>>> rmid. Calling TestLibrary.bomb() from within the try-block is confusing, since >>>>> that method throws an exception, which is then caught by the catch-block, when >>>>> then calls TestLibrary.bomb() again. >>>>> >>>>> We should also make sure to test the success case properly. If rmid.waitFor() >>>>> returns in a timely fashion without throwing TimeoutException, it doesn't >>>>> matter what the exit status is. (It might be worth printing it out.) At that >>>>> point we know that rmid *has* exited gracefully, so we need to set rmid to >>>>> null so that the finally-block doesn't attempt to destroy rmid redundantly. >>>>> Some additional messages about rmid having exited and the test passing are >>>>> also warranted for this case. >>>>> >>>>> Some additional cleanup can be done here as well, over and above the changes >>>>> you've proposed. (This stuff is left over from earlier RMI test messes.) In >>>>> order to shut down an active object, the code here spawns a new thread that >>>>> sleeps for a while and then deactivates this object. This isn't necessary. (It >>>>> might have been necessary in the past.) It's sufficient simply to unexport >>>>> this object and then deactivate it, directly within the shutdown() method. See >>>>> >>>>> test/java/rmi/activation/ActivationSystem/unregisterGroup/UnregisterGroup.java >>>>> >>>>> for an example of this. In addition, the run() method can be removed, and the >>>>> "implements Runnable" declaration can also be removed from the >>>>> ShutdownGracefully test class. >>>>> >>>>> Finally, revisiting some code farther up in the test, the try-block at lines >>>>> 135-140 issues a registration request that the test expects to fail. If it >>>>> succeeds, the message at line 139 isn't very clear; it should say that the >>>>> registration request succeeded unexpectedly. This should cause the test to >>>>> fail. We still probably want to go through the waitFor(timeout) path and >>>>> eventual rmid cleanup, but a flag should be set here to ensure that the test >>>>> indeed fails if the registration succeeds unexpectedly, and the messages >>>>> should clearly indicate that this is going on. >>>>> >>>>> A good way to test this last case is to change rmid's security manager to the >>>>> normal security manager java.lang.SecurityManager instead of >>>>> TestSecurityManager. >>>>> >>>>> Thanks, >>>>> >>>>> s'marks >>>>> >>>>> >>>>> >>>>> >>>>> On 2/10/14 2:59 AM, Tristan Yan wrote: >>>>>> Hi Stuart >>>>>> Could you help to review this. >>>>>> Thank you >>>>>> Tristan >>>>>> >>>>>> On Jan 31, 2014, at 4:36 PM, Tristan Yan >>>>> >>>>>> > wrote: >>>>>> >>>>>>> Thank you for fixing JDK-8023541. Then I leave ActivationLibrary.java for >>>>>>> now. >>>>>>> I still did some clean up following your suggestion. >>>>>>> 1. I changed waitFor(long timeout) method, this method is going to use code >>>>>>> like Process.waitFor(timeout, unit). This can be backported to JDK7. Also >>>>>>> exitValue is kept as a return value. For making sure there is no Pipe leak, a >>>>>>> cleanup thread will start when timeout happens. >>>>>>> 2. Change in ShutdownGracefully is a little tricky. I think we should just >>>>>>> destroy JVM once exception is thrown. So I move the wait logic into try block >>>>>>> instead keep them in finally block. >>>>>>> Can you receive it again. >>>>>>> http://cr.openjdk.java.net/~tyan/JDK-8032050/webrev.02/ >>>>>>> Thank you >>>>>>> Tristan >>>>>>> >>>>>>> On 01/29/2014 03:16 PM, Stuart Marks wrote: >>>>>>>> Hi Tristan, >>>>>>>> >>>>>>>> I don't want to put the workaround into ActivationLibrary.rmidRunning() >>>>>>>> for a >>>>>>>> null return from the lookup, because this is only a workaround for an actual >>>>>>>> bug in rmid initialization. See the review I just posted for JDK-8023541. >>>>>>>> >>>>>>>> Adding JavaVM.waitFor(timeout) is something that would be useful in general, >>>>>>>> but it needs to be handled carefully. It uses the new >>>>>>>> Process.waitFor(timeout, unit) which is new in Java SE 8; this makes >>>>>>>> backporting to 7 more complicated. Not clear whether we'll do so, but I >>>>>>>> don't >>>>>>>> want to forclose the opportunity without discussion. It's also not clear how >>>>>>>> one can get the vm's exit status after JavaVM.waitFor() has returned true. >>>>>>>> With the Process API it's possible simply to call waitFor() or exitValue(). >>>>>>>> With JavaVM, a new API needs to be created, or the rule has to be >>>>>>>> established >>>>>>>> that one must call JavaVM.waitFor() to collect the exit status as well as to >>>>>>>> close the pipes from the subprocess. If JavaVM.waitFor(timeout, unit) is >>>>>>>> called without subsequently calling JavaVM.waitFor(), the pipes are leaked. >>>>>>>> >>>>>>>> In ShutdownGracefully.java, the finally-block needs to check to see if rmid >>>>>>>> is still running, and if it is, to shut it down. Simply calling >>>>>>>> waitFor(timeout, unit) isn't sufficient, because if the rmid process is >>>>>>>> still >>>>>>>> running, it will be left running. >>>>>>>> >>>>>>>> The straightforward approach would be to call >>>>>>>> ActivationLibrary.rmidRunning() >>>>>>>> to test if it's still running. Unfortunately this isn't quite right, because >>>>>>>> rmidRunning() has a timeout loop in it -- which should probably be removed. >>>>>>>> (I think there's a bug for this.) Another approach would be simply to call >>>>>>>> rmid.destroy(). This calls rmid's shutdown() method first, which is >>>>>>>> reasonable, but I'm not sure it kills the process if that fails. In any >>>>>>>> case, >>>>>>>> this already has a timeout loop waiting for the process to die, so >>>>>>>> ShutdownGracefully.java needn't use a new waitFor(timeout, unit) call. >>>>>>>> >>>>>>>> Removing the commented-out code that starts with "no longer needed" is good, >>>>>>>> and removing the ShutdownDetectThread is also good, since that's >>>>>>>> unnecessary. >>>>>>>> >>>>>>>> There are some more cleanups I have in mind here but I'd like to see a >>>>>>>> revised webrev before proceeding. >>>>>>>> >>>>>>>> Thanks, >>>>>>>> >>>>>>>> s'marks >>>>>>>> >>>>>>>> On 1/25/14 8:57 PM, Tristan Yan wrote: >>>>>>>>> Hi Stuart >>>>>>>>> Thank you for your review and suggestion. >>>>>>>>> Yes, since this failure mode is very hard to be reproduced. I guess it's >>>>>>>>> make sense to do some hack. And I also noticed in >>>>>>>>> ActivationLibrary.rmidRunning. It does try to look up ActivationSystem but >>>>>>>>> doesn't check if it's null. So I add the logic to make sure we will look up >>>>>>>>> the non-null ActivationSystem. Also I did some cleanup if you don't mind. >>>>>>>>> Add a waitFor(long timeout, TimeUnit unit) for JavaVM. Which we can have a >>>>>>>>> better waitFor control. >>>>>>>>> I appreciate you can review the code again. >>>>>>>>> http://cr.openjdk.java.net/~tyan/JDK-8032050/webrev.01/ >>>>>>>>> Thank you >>>>>>>>> Tristan >>>>>>>>> >>>>>>>>> >>>>>>>>> On 01/25/2014 10:20 AM, Stuart Marks wrote: >>>>>>>>>> On 1/23/14 10:34 PM, Tristan Yan wrote: >>>>>>>>>>> Hi All >>>>>>>>>>> Could you review the bug fix for JDK-8032050. >>>>>>>>>>> >>>>>>>>>>> http://cr.openjdk.java.net/~tyan/JDK-8032050/webrev.00/ >>>>>>>>>>> >>>>>>>>>>> Description: >>>>>>>>>>> This rare happened failure caused because when RMID starts. It don't >>>>>>>>>>> guarantee >>>>>>>>>>> sun.rmi.server.Activation.startActivation finishes. >>>>>>>>>>> Fix by adding a iterative getSystem with a 5 seconds timeout. >>>>>>>>>> >>>>>>>>>> Hi Tristan, >>>>>>>>>> >>>>>>>>>> Adding a timing/retry loop into this test isn't the correct approach for >>>>>>>>>> fixing this test. >>>>>>>>>> >>>>>>>>>> The timing loop isn't necessary because there is already a timing loop in >>>>>>>>>> RMID.start() in the RMI test library. (There's another timing loop in >>>>>>>>>> ActivationLibrary.rmidRunning() which should probably be removed.) So the >>>>>>>>>> intent of this library call is that it start rmid and wait for it to >>>>>>>>>> become >>>>>>>>>> ready. That logic doesn't need to be added to the test. >>>>>>>>>> >>>>>>>>>> In the bug report JDK-8032050 you had mentioned that the >>>>>>>>>> NullPointerException was suspicious. You're right! I took a look and it >>>>>>>>>> seemed like it was related to JDK-8023541, and I added a note to this >>>>>>>>>> effect to the bug report. The problem here is that rmid can come up and >>>>>>>>>> transiently return null instead of the stub of the activation system. >>>>>>>>>> That's what JDK-8023541 covers. I think that rmid itself needs to be >>>>>>>>>> fixed, >>>>>>>>>> though modifying the timing loop in the RMI test library to wait for rmid >>>>>>>>>> to come up *and* for the lookup to return non-null is an easy way to fix >>>>>>>>>> the problem. (Or at least cover it up.) >>>>>>>>>> >>>>>>>>>> The next step in the analysis is to determine, given that >>>>>>>>>> ActivationLibrary.getSystem can sometimes return null, whether this has >>>>>>>>>> actually caused this test failure. This is pretty easy to determine; just >>>>>>>>>> hack in a line "system = null" in the right place and run the test. I've >>>>>>>>>> done this, and the test times out and the output log is pretty much >>>>>>>>>> identical to the one in the bug report. (I recommend you try this >>>>>>>>>> yourself.) So I think it's fairly safe to say that the problem in >>>>>>>>>> JDK-8023541 has caused the failure listed in JDK-8032050. >>>>>>>>>> >>>>>>>>>> I can see a couple ways to proceed here. One way is just to close this out >>>>>>>>>> as a duplicate of JDK-8023541 since that bug caused this failure. >>>>>>>>>> >>>>>>>>>> Another is that this test could use some cleaning up. This bug certainly >>>>>>>>>> covers a failure, but the messages emitted are confusing and in some cases >>>>>>>>>> completely wrong. For example, the "rmid has shutdown" message at line 180 >>>>>>>>>> is incorrect, because in this case rmid is still running and the wait() >>>>>>>>>> call has timed out. Most of the code here can be replaced with calls to >>>>>>>>>> various bits of the RMI test library. There are a bunch of other things in >>>>>>>>>> this test that could be cleaned up as well. >>>>>>>>>> >>>>>>>>>> It's up to you how you'd like to proceed. >>>>>>>>>> >>>>>>>>>> s'marks >>>>>>>>> >>>>>>>> >>>>>>> >>>>>> >>>> >> From mikael.vidstedt at oracle.com Sat Feb 15 01:54:15 2014 From: mikael.vidstedt at oracle.com (Mikael Vidstedt) Date: Fri, 14 Feb 2014 17:54:15 -0800 Subject: RFR: Even more gcc warnings Message-ID: <52FEC8C7.9000700@oracle.com> All, A drive-by set of warning fixes: http://cr.openjdk.java.net/~mikael/webrevs/jdk-warnings/jdk-warnings/webrev.00/ Highlights: * src/share/native/com/sun/java/util/jar/pack/bands.cpp Set the size of the array explicitly to increase likelihood of enum and struct array being in sync. Arguably this should be changed to use the (new) [] = instead. Initialize all the fields in the (redundant) terminator struct explicitly. Remove unused macro. * src/share/native/sun/java2d/opengl/OGLContext.c Get the prototype for jio_snprintf from jvm.h to address an implicit declaration. * src/solaris/native/sun/awt/awt_Font.c Comparisons with string literals is undefined behavior - keep track of whether the string should be freed explicitly with a boolean instead. * src/solaris/native/sun/awt/awt_LoadLibrary.c The macro is supposed to expand to a void function declaration, but forgets to actually add the "void". Cheers, Mikael From martijnverburg at gmail.com Sat Feb 15 13:35:43 2014 From: martijnverburg at gmail.com (Martijn Verburg) Date: Sat, 15 Feb 2014 13:35:43 +0000 Subject: A hole in the serialization spec In-Reply-To: <52FE66E6.8040700@redhat.com> References: <52F4F6BB.2010207@redhat.com> <52F8E807.2090403@oracle.com> <52F8F21C.80205@redhat.com> <52FB8E5C.9020502@oracle.com> <52FB9248.6010703@redhat.com> <26199217-BAE6-47AC-AFC8-4DE90726B8C0@oracle.com> <52FD0328.3050304@redhat.com> <52FE3CB2.5060408@redhat.com> <52FE44E7.6060103@oracle.com> <52FE59EF.3030606@redhat.com> <52FE66E6.8040700@redhat.com> Message-ID: In the long term will it be possible to just mark j.u.Date as deprecated since the new Date and Time libs have come in? Avoids having to fix old behaviour that folks might be relying on. Cheers, Martijn On 14 February 2014 18:56, David M. Lloyd wrote: > Interestingly, if the behavior in this case was to write empty values on > the write side and read and discard on the read side, both the behavior and > the serialized format of java.util.Date would be identical to today; though > other pathological classes might be incompatible, they were already in > undefined-land. > > > On 02/14/2014 12:01 PM, David M. Lloyd wrote: > >> Yes, however given that the lack of reading/writing fields makes it >> undefined *in general*, you can only say "the behavior is defined as >> long as this undefined behavior is actually defined in this (risky) >> manner". >> >> Put another way, since the behavior of not reading/writing fields is not >> defined, this class is then by spec not portable between JVMs (after >> all, undefined behavior could mean anything at all). That's why I was >> saying that the spec should define exactly what will physically happen >> if you do this (from both the perspective of the "other end" of the >> serialization API, *and* the wire protocol), even if it's considered >> poor practice - because then at least you have guaranteed >> interoperability (which becomes even more important with the knowledge >> that JDK classes have this issue). >> >> On 02/14/2014 10:31 AM, roger riggs wrote: >> >>> Hi David, >>> >>> I would quibble that the serialized form of java.util.Date is defined. >>> The behavior is defined by the writeObject and readObject methods. >>> They write and read a time computed from and restored to the state of >>> the object. >>> >>> Roger >>> >>> On 2/14/2014 10:56 AM, David M. Lloyd wrote: >>> >>>> On 02/13/2014 11:38 AM, David M. Lloyd wrote: >>>> >>>>> On 02/13/2014 10:29 AM, Chris Hegarty wrote: >>>>> >>>>>> On 12 Feb 2014, at 15:24, David M. Lloyd >>>>>> wrote: >>>>>> >>>>>> That's a quote from the serialization spec. I take it to mean, >>>>>>> "Don't write fields and everything might go to hell". In practice, >>>>>>> if the reading side doesn't read fields, things end up more or less >>>>>>> OK, as evidenced by various classes in the wild. But it's not hard >>>>>>> to imagine a scenario in which a class change could cause protocol >>>>>>> corruption. >>>>>>> >>>>>>> I think the specifics of the quote relate to this kind of class >>>>>>> change; in particular, if a class is deleted from the hierarchy on >>>>>>> the read side, and that class corresponds to the class that had the >>>>>>> misbehaving writeObject, I suspect that things will break at that >>>>>>> point as the read side will probably try to consume and discard the >>>>>>> field information for that class, which will be missing (it will >>>>>>> start reading the next class' fields instead I think). >>>>>>> >>>>>> >>>>>> Yes, possibly. And who knows what fields/values may be read and >>>>>> mistaken for the wrong object in the hierarchy. So ?undefined' >>>>>> behaviour seems right to me. >>>>>> >>>>> >>>>> I think the behavior is well-defined, just "bad", which is my >>>>> point. If >>>>> the exact current is spec'd out as-is then at least we can be >>>>> assured of >>>>> the same bad behavior across implementations. If the behavior is >>>>> changed such that fields are read/written but discarded, without >>>>> updating the spec, then the "undefined" behavior at least becomes >>>>> safer. >>>>> If the behavior is changed, *and* the spec is updated, then we get >>>>> both benefits, but at the cost that all previous implementations will >>>>> not be compliant with the spec. >>>>> >>>>> All options seem to have a cost though. >>>>> >>>> >>>> In the JDK, java.util.Date does not read/write fields. Perhaps others >>>> as well. Given that the behavior is presently undefined, that means >>>> the serialized representation of java.util.Date (and any other such >>>> non-conforming classes) are also undefined. >>>> >>>> >>> >> >> > > -- > - DML > From martinrb at google.com Sat Feb 15 16:02:55 2014 From: martinrb at google.com (Martin Buchholz) Date: Sat, 15 Feb 2014 08:02:55 -0800 Subject: JDK 8: Second Release Candidate In-Reply-To: <52FE664F.8090308@gmail.com> References: <20140211143152.649634@eggemoggin.niobe.net> <52FE664F.8090308@gmail.com> Message-ID: [Moving this to core-libs-dev, where it's more likely to reach possible fixers] I didn't know about requireEnd, but your bug report looks persuasive. On Fri, Feb 14, 2014 at 10:54 AM, Brenden Towey wrote: > I'd like to make a bug report. Java has had for a while now a bug in its > regex system which I'd like to see fixed. > > The short of it is that the \z pattern does not return 'requiresEnd' and > it should. > > public void endTest() > { > Matcher m = Pattern.compile( "\\z" ).matcher( "" ); > m.find(); > System.out.println( m.requireEnd() ); > assert ( m.requireEnd() ); > } > > This prints 'false'. It shouldn't take much thought to convince yourself > that if the end of input is required, then 'requiresEnd()' should always be > true. There's never a case for the \z pattern that you want to match less > than all of input. The above code snippet would make a fine unit test for > this bug, btw. > > You can see the results of this bug if you use other parts of the API, for > example java.util.Scanner. Since the the requiresEnd() method always > returns false, the Scanner will match its own internal buffer (usually 1024 > characters) and not the end of input. > > > public void demo() > { > StringBuilder str = new StringBuilder( 4 * 1024 ); > for( int i = 0; i < 1024; i++ ) { > str.append( i ); > str.append( ',' ); > } > Scanner s = new Scanner( str.toString() ); > String result = s.useDelimiter( "\\z" ).next(); > String expected = str.toString(); > System.out.println( result.length()+", "+expected.length() ); > assert( expected.equals( result ) ); > } > > Output: > C:\Users\Brenden\Dev\proj\Test2\build\classes>java -version > java version "1.8.0" > Java(TM) SE Runtime Environment (build 1.8.0-b129) > Java HotSpot(TM) 64-Bit Server VM (build 25.0-b69, mixed mode) > > C:\Users\Brenden\Dev\proj\Test2\build\classes>java -cp . > quicktest.RegexBug > 1024, 4010 > > You can see that the length of the matched string is 1024, not 4010 from > the original string. > > Finally, if you need more convincing, you can test the \Z (capital Z) > pattern, which does essentially the same thing as \z. \Z always sets its > requiresEnd() flag, and it works as expected in the tests above. > > Summary: \z should always set its 'requiresEnd' flag. > > Thanks for taking the time to read this! > > > > > On 2/11/2014 2:31 PM, mark.reinhold at oracle.com wrote: > >> Last week a serious flaw in a new API was reported [1]. We decided to >> fix that bug, along with an unrelated JCK failure on Mac OS X [2], so >> we now have a second JDK 8 Release Candidate, build 129. >> >> Binaries available here, as usual: https://jdk8.java.net/download.html >> >> >> > > From xueming.shen at oracle.com Sat Feb 15 16:34:45 2014 From: xueming.shen at oracle.com (Xueming Shen) Date: Sat, 15 Feb 2014 08:34:45 -0800 Subject: JDK 8: Second Release Candidate In-Reply-To: References: <20140211143152.649634@eggemoggin.niobe.net> <52FE664F.8090308@gmail.com> Message-ID: <52FF9725.7060609@oracle.com> Created an issue for this one. https://bugs.openjdk.java.net/browse/JDK-8035042 Need go dig out some history. requireEnd is primarily designed for j.u.Scanner. -Sherman On 2/15/14 8:02 AM, Martin Buchholz wrote: > Java has had for a while now a bug in its regex system which I'd like > to see fixed. > > The short of it is that the \z pattern does not return 'requiresEnd' > and it should. > > public void endTest() > { > Matcher m = Pattern.compile( "\\z" ).matcher( "" ); > m.find(); > System.out.println( m.requireEnd() ); > assert ( m.requireEnd() ); > } > > This prints 'false'. It shouldn't take much thought to convince > yourself that if the end of input is required, then 'requiresEnd()' > should always be true. There's never a case for the \z pattern that > you want to match less than all of input. The above code snippet > would make a fine unit test for this bug, btw. > > You can see the results of this bug if you use other parts of the API, > for example java.util.Scanner. Since the the requiresEnd() method > always returns false, the Scanner will match its own internal buffer > (usually 1024 characters) and not the end of input. > > > public void demo() > { > StringBuilder str = new StringBuilder( 4 * 1024 ); > for( int i = 0; i < 1024; i++ ) { > str.append( i ); > str.append( ',' ); > } > Scanner s = new Scanner( str.toString() ); > String result = s.useDelimiter( "\\z" ).next(); > String expected = str.toString(); > System.out.println( result.length()+", "+expected.length() ); > assert( expected.equals( result ) ); > } > > Output: > C:\Users\Brenden\Dev\proj\Test2\build\classes>java -version > java version "1.8.0" > Java(TM) SE Runtime Environment (build 1.8.0-b129) > Java HotSpot(TM) 64-Bit Server VM (build 25.0-b69, mixed mode) > > C:\Users\Brenden\Dev\proj\Test2\build\classes>java -cp . > quicktest.RegexBug > 1024, 4010 > > You can see that the length of the matched string is 1024, not 4010 > from the original string. > > Finally, if you need more convincing, you can test the \Z (capital Z) > pattern, which does essentially the same thing as \z. \Z always sets > its requiresEnd() flag, and it works as expected in the tests above. From mikael.vidstedt at oracle.com Sat Feb 15 16:37:39 2014 From: mikael.vidstedt at oracle.com (Mikael Vidstedt) Date: Sat, 15 Feb 2014 08:37:39 -0800 Subject: RFR: Even more gcc warnings In-Reply-To: <52FEC8C7.9000700@oracle.com> References: <52FEC8C7.9000700@oracle.com> Message-ID: <41E8E647-4C1E-4DD6-86AF-40A498B503B7@oracle.com> Corrected link - this webrev is based on jdk9/client: http://cr.openjdk.java.net/~mikael/webrevs/jdk-warnings/webrev.01/webrev/ Cheers, Mikael > On Feb 14, 2014, at 17:54, Mikael Vidstedt wrote: > > > All, > > A drive-by set of warning fixes: > > http://cr.openjdk.java.net/~mikael/webrevs/jdk-warnings/jdk-warnings/webrev.00/ > > Highlights: > > * src/share/native/com/sun/java/util/jar/pack/bands.cpp > > Set the size of the array explicitly to increase likelihood of enum and struct array being in sync. Arguably this should be changed to use the (new) [] = instead. > > Initialize all the fields in the (redundant) terminator struct explicitly. > > Remove unused macro. > > * src/share/native/sun/java2d/opengl/OGLContext.c > > Get the prototype for jio_snprintf from jvm.h to address an implicit declaration. > > * src/solaris/native/sun/awt/awt_Font.c > > Comparisons with string literals is undefined behavior - keep track of whether the string should be freed explicitly with a boolean instead. > > * src/solaris/native/sun/awt/awt_LoadLibrary.c > > The macro is supposed to expand to a void function declaration, but forgets to actually add the "void". > > Cheers, > Mikael > From lance.andersen at oracle.com Sat Feb 15 16:40:22 2014 From: lance.andersen at oracle.com (Lance @ Oracle) Date: Sat, 15 Feb 2014 11:40:22 -0500 Subject: A hole in the serialization spec In-Reply-To: References: <52F4F6BB.2010207@redhat.com> <52F8E807.2090403@oracle.com> <52F8F21C.80205@redhat.com> <52FB8E5C.9020502@oracle.com> <52FB9248.6010703@redhat.com> <26199217-BAE6-47AC-AFC8-4DE90726B8C0@oracle.com> <52FD0328.3050304@redhat.com> <52FE3CB2.5060408@redhat.com> <52FE44E7.6060103@oracle.com> <52FE59EF.3030606@redhat.com> <52FE66E6.8040700@redhat.com> Message-ID: IMHO, I think that is a tall task. There is such a large base of code that uses it or java.sql.Date, that having warnings for deprecation will cause a lot of angst. That code is not going to migrate quickly, if at all to the new types. Best Lance Lance Andersen| Principal Member of Technical Staff | +1.781.442.2037 Oracle Java Engineering 1 Network Drive Burlington, MA 01803 Lance.Andersen at oracle.com Sent from my iPad On Feb 15, 2014, at 8:35 AM, Martijn Verburg wrote: > In the long term will it be possible to just mark j.u.Date as deprecated > since the new Date and Time libs have come in? Avoids having to fix old > behaviour that folks might be relying on. > > Cheers, > Martijn > > > On 14 February 2014 18:56, David M. Lloyd wrote: > >> Interestingly, if the behavior in this case was to write empty values on >> the write side and read and discard on the read side, both the behavior and >> the serialized format of java.util.Date would be identical to today; though >> other pathological classes might be incompatible, they were already in >> undefined-land. >> >> >> On 02/14/2014 12:01 PM, David M. Lloyd wrote: >> >>> Yes, however given that the lack of reading/writing fields makes it >>> undefined *in general*, you can only say "the behavior is defined as >>> long as this undefined behavior is actually defined in this (risky) >>> manner". >>> >>> Put another way, since the behavior of not reading/writing fields is not >>> defined, this class is then by spec not portable between JVMs (after >>> all, undefined behavior could mean anything at all). That's why I was >>> saying that the spec should define exactly what will physically happen >>> if you do this (from both the perspective of the "other end" of the >>> serialization API, *and* the wire protocol), even if it's considered >>> poor practice - because then at least you have guaranteed >>> interoperability (which becomes even more important with the knowledge >>> that JDK classes have this issue). >>> >>> On 02/14/2014 10:31 AM, roger riggs wrote: >>> >>>> Hi David, >>>> >>>> I would quibble that the serialized form of java.util.Date is defined. >>>> The behavior is defined by the writeObject and readObject methods. >>>> They write and read a time computed from and restored to the state of >>>> the object. >>>> >>>> Roger >>>> >>>> On 2/14/2014 10:56 AM, David M. Lloyd wrote: >>>> >>>>> On 02/13/2014 11:38 AM, David M. Lloyd wrote: >>>>> >>>>>> On 02/13/2014 10:29 AM, Chris Hegarty wrote: >>>>>> >>>>>>> On 12 Feb 2014, at 15:24, David M. Lloyd >>>>>>> wrote: >>>>>>> >>>>>>> That's a quote from the serialization spec. I take it to mean, >>>>>>>> "Don't write fields and everything might go to hell". In practice, >>>>>>>> if the reading side doesn't read fields, things end up more or less >>>>>>>> OK, as evidenced by various classes in the wild. But it's not hard >>>>>>>> to imagine a scenario in which a class change could cause protocol >>>>>>>> corruption. >>>>>>>> >>>>>>>> I think the specifics of the quote relate to this kind of class >>>>>>>> change; in particular, if a class is deleted from the hierarchy on >>>>>>>> the read side, and that class corresponds to the class that had the >>>>>>>> misbehaving writeObject, I suspect that things will break at that >>>>>>>> point as the read side will probably try to consume and discard the >>>>>>>> field information for that class, which will be missing (it will >>>>>>>> start reading the next class' fields instead I think). >>>>>>>> >>>>>>> >>>>>>> Yes, possibly. And who knows what fields/values may be read and >>>>>>> mistaken for the wrong object in the hierarchy. So ?undefined' >>>>>>> behaviour seems right to me. >>>>>>> >>>>>> >>>>>> I think the behavior is well-defined, just "bad", which is my >>>>>> point. If >>>>>> the exact current is spec'd out as-is then at least we can be >>>>>> assured of >>>>>> the same bad behavior across implementations. If the behavior is >>>>>> changed such that fields are read/written but discarded, without >>>>>> updating the spec, then the "undefined" behavior at least becomes >>>>>> safer. >>>>>> If the behavior is changed, *and* the spec is updated, then we get >>>>>> both benefits, but at the cost that all previous implementations will >>>>>> not be compliant with the spec. >>>>>> >>>>>> All options seem to have a cost though. >>>>>> >>>>> >>>>> In the JDK, java.util.Date does not read/write fields. Perhaps others >>>>> as well. Given that the behavior is presently undefined, that means >>>>> the serialized representation of java.util.Date (and any other such >>>>> non-conforming classes) are also undefined. >>>>> >>>>> >>>> >>> >>> >> >> -- >> - DML >> From roger.riggs at oracle.com Sat Feb 15 17:14:45 2014 From: roger.riggs at oracle.com (roger riggs) Date: Sat, 15 Feb 2014 12:14:45 -0500 Subject: A hole in the serialization spec In-Reply-To: References: <52F4F6BB.2010207@redhat.com> <52F8E807.2090403@oracle.com> <52F8F21C.80205@redhat.com> <52FB8E5C.9020502@oracle.com> <52FB9248.6010703@redhat.com> <26199217-BAE6-47AC-AFC8-4DE90726B8C0@oracle.com> <52FD0328.3050304@redhat.com> <52FE3CB2.5060408@redhat.com> <52FE44E7.6060103@oracle.com> <52FE59EF.3030606@redhat.com> <52FE66E6.8040700@redhat.com> Message-ID: <52FFA085.2030106@oracle.com> Hi, Not likely to deprecate Date due to the pervasive use. But it is highly unlikely that java.util.Date will be evolving, since java.time exists there is no need or desire to add any functionality to Date. But it is just one case of the serialized form issues that are being discussed. Roger On 2/15/2014 11:40 AM, Lance @ Oracle wrote: > IMHO, I think that is a tall task. There is such a large base of code that uses it or java.sql.Date, that having warnings for deprecation will cause a lot of angst. That code is not going to migrate quickly, if at all to the new types. > > Best > Lance > > > Lance Andersen| Principal Member of Technical Staff | +1.781.442.2037 > Oracle Java Engineering > 1 Network Drive > Burlington, MA 01803 > Lance.Andersen at oracle.com > Sent from my iPad > > On Feb 15, 2014, at 8:35 AM, Martijn Verburg wrote: > >> In the long term will it be possible to just mark j.u.Date as deprecated >> since the new Date and Time libs have come in? Avoids having to fix old >> behaviour that folks might be relying on. >> >> Cheers, >> Martijn >> >> >> On 14 February 2014 18:56, David M. Lloyd wrote: >> >>> Interestingly, if the behavior in this case was to write empty values on >>> the write side and read and discard on the read side, both the behavior and >>> the serialized format of java.util.Date would be identical to today; though >>> other pathological classes might be incompatible, they were already in >>> undefined-land. >>> >>> >>> On 02/14/2014 12:01 PM, David M. Lloyd wrote: >>> >>>> Yes, however given that the lack of reading/writing fields makes it >>>> undefined *in general*, you can only say "the behavior is defined as >>>> long as this undefined behavior is actually defined in this (risky) >>>> manner". >>>> >>>> Put another way, since the behavior of not reading/writing fields is not >>>> defined, this class is then by spec not portable between JVMs (after >>>> all, undefined behavior could mean anything at all). That's why I was >>>> saying that the spec should define exactly what will physically happen >>>> if you do this (from both the perspective of the "other end" of the >>>> serialization API, *and* the wire protocol), even if it's considered >>>> poor practice - because then at least you have guaranteed >>>> interoperability (which becomes even more important with the knowledge >>>> that JDK classes have this issue). >>>> >>>> On 02/14/2014 10:31 AM, roger riggs wrote: >>>> >>>>> Hi David, >>>>> >>>>> I would quibble that the serialized form of java.util.Date is defined. >>>>> The behavior is defined by the writeObject and readObject methods. >>>>> They write and read a time computed from and restored to the state of >>>>> the object. >>>>> >>>>> Roger >>>>> >>>>> On 2/14/2014 10:56 AM, David M. Lloyd wrote: >>>>> >>>>>> On 02/13/2014 11:38 AM, David M. Lloyd wrote: >>>>>> >>>>>>> On 02/13/2014 10:29 AM, Chris Hegarty wrote: >>>>>>> >>>>>>>> On 12 Feb 2014, at 15:24, David M. Lloyd >>>>>>>> wrote: >>>>>>>> >>>>>>>> That's a quote from the serialization spec. I take it to mean, >>>>>>>>> "Don't write fields and everything might go to hell". In practice, >>>>>>>>> if the reading side doesn't read fields, things end up more or less >>>>>>>>> OK, as evidenced by various classes in the wild. But it's not hard >>>>>>>>> to imagine a scenario in which a class change could cause protocol >>>>>>>>> corruption. >>>>>>>>> >>>>>>>>> I think the specifics of the quote relate to this kind of class >>>>>>>>> change; in particular, if a class is deleted from the hierarchy on >>>>>>>>> the read side, and that class corresponds to the class that had the >>>>>>>>> misbehaving writeObject, I suspect that things will break at that >>>>>>>>> point as the read side will probably try to consume and discard the >>>>>>>>> field information for that class, which will be missing (it will >>>>>>>>> start reading the next class' fields instead I think). >>>>>>>>> >>>>>>>> Yes, possibly. And who knows what fields/values may be read and >>>>>>>> mistaken for the wrong object in the hierarchy. So ?undefined' >>>>>>>> behaviour seems right to me. >>>>>>>> >>>>>>> I think the behavior is well-defined, just "bad", which is my >>>>>>> point. If >>>>>>> the exact current is spec'd out as-is then at least we can be >>>>>>> assured of >>>>>>> the same bad behavior across implementations. If the behavior is >>>>>>> changed such that fields are read/written but discarded, without >>>>>>> updating the spec, then the "undefined" behavior at least becomes >>>>>>> safer. >>>>>>> If the behavior is changed, *and* the spec is updated, then we get >>>>>>> both benefits, but at the cost that all previous implementations will >>>>>>> not be compliant with the spec. >>>>>>> >>>>>>> All options seem to have a cost though. >>>>>>> >>>>>> In the JDK, java.util.Date does not read/write fields. Perhaps others >>>>>> as well. Given that the behavior is presently undefined, that means >>>>>> the serialized representation of java.util.Date (and any other such >>>>>> non-conforming classes) are also undefined. >>>>>> >>>>>> >>>> >>> -- >>> - DML >>> From philip.race at oracle.com Sat Feb 15 17:31:27 2014 From: philip.race at oracle.com (Phil Race) Date: Sat, 15 Feb 2014 09:31:27 -0800 Subject: RFR: Even more gcc warnings In-Reply-To: <41E8E647-4C1E-4DD6-86AF-40A498B503B7@oracle.com> References: <52FEC8C7.9000700@oracle.com> <41E8E647-4C1E-4DD6-86AF-40A498B503B7@oracle.com> Message-ID: <52FFA46F.4000202@oracle.com> Looks OK to me although I just realised there's no bug ID here FWIW I develop on WIndows, Mac & Linux and I've noticed widely divergent things that the compilers on these platforms warn about. Warning free on Linux might not mean warning free on Windows. Also, assuming you develop on Linux might want to check if any of these make the VS compiler less happy about anything. > * src/solaris/native/sun/awt/awt_Font.c > > Comparisons with string literals is undefined behavior - keep track of whether the string should be freed explicitly with a boolean instead. Gosh .. that code must be from 1996 or thereabouts. -phil. On 2/15/14 8:37 AM, Mikael Vidstedt wrote: > Corrected link - this webrev is based on jdk9/client: > > http://cr.openjdk.java.net/~mikael/webrevs/jdk-warnings/webrev.01/webrev/ > > Cheers, > Mikael > >> On Feb 14, 2014, at 17:54, Mikael Vidstedt wrote: >> >> >> All, >> >> A drive-by set of warning fixes: >> >> http://cr.openjdk.java.net/~mikael/webrevs/jdk-warnings/jdk-warnings/webrev.00/ >> >> Highlights: >> >> * src/share/native/com/sun/java/util/jar/pack/bands.cpp >> >> Set the size of the array explicitly to increase likelihood of enum and struct array being in sync. Arguably this should be changed to use the (new) [] = instead. >> >> Initialize all the fields in the (redundant) terminator struct explicitly. >> >> Remove unused macro. >> >> * src/share/native/sun/java2d/opengl/OGLContext.c >> >> Get the prototype for jio_snprintf from jvm.h to address an implicit declaration. >> >> * src/solaris/native/sun/awt/awt_Font.c >> >> Comparisons with string literals is undefined behavior - keep track of whether the string should be freed explicitly with a boolean instead. >> >> * src/solaris/native/sun/awt/awt_LoadLibrary.c >> >> The macro is supposed to expand to a void function declaration, but forgets to actually add the "void". >> >> Cheers, >> Mikael >> From martinrb at google.com Sat Feb 15 18:27:36 2014 From: martinrb at google.com (Martin Buchholz) Date: Sat, 15 Feb 2014 10:27:36 -0800 Subject: RFR 9: 8034903: Add Logging of Process.start arguments and resulting pid In-Reply-To: <52FE7931.4020408@oracle.com> References: <52FD3897.7070903@oracle.com> <52FE7931.4020408@oracle.com> Message-ID: On Fri, Feb 14, 2014 at 12:14 PM, Alan Bateman wrote: > On 14/02/2014 18:10, Martin Buchholz wrote: > >> I'm not excited about having logging statements added throughout the jdk >> implementation. PlatformLogger is an implementation detail, so real users >> cannot benefit. Your OS should already provide facilities to let you >> trace >> process creation. E.g. on Linux you can do this with strace (and I have >> done this myself in the past to debug subprocess creation). So I'd say >> adding logging just to provide a small rarely used convenience for jdk >> developers is Not Worthwhile. >> >> Just on PlatformLogger then it just forwards to j.u.logging when it is > present. > > OK, so is this code for the benefit of "real" users or openjdk developers? If the latter, it doesn't pull its weight - "Dude, you forgot to remove your printf debugging statements". If the former, this feature needs to be explained somewhere - your JDK will randomly start doing more logging that users didn't ask for and is undocumented. It's very nuanced, because I am fully in favor of the similar feature "_JAVA_LAUNCHER_DEBUG". Maybe because strace/truss solved my need for subprocess invocation debugging on Linux/Solaris. From Alan.Bateman at oracle.com Sat Feb 15 21:55:02 2014 From: Alan.Bateman at oracle.com (Alan Bateman) Date: Sat, 15 Feb 2014 21:55:02 +0000 Subject: RFR 9: 8034903: Add Logging of Process.start arguments and resulting pid In-Reply-To: References: <52FD3897.7070903@oracle.com> <52FE7931.4020408@oracle.com> Message-ID: <52FFE236.9020301@oracle.com> On 15/02/2014 18:27, Martin Buchholz wrote: > : > > OK, so is this code for the benefit of "real" users or openjdk > developers? If the latter, it doesn't pull its weight - "Dude, you > forgot to remove your printf debugging statements". If the former, > this feature needs to be explained somewhere - your JDK will randomly > start doing more logging that users didn't ask for and is undocumented. > I think Roger needs to expands on the motivation a bit but if logging is being added here then I think we need to understand how it might fit into a possible larger effort to improve the overall supportability. -Alan. From Alan.Bateman at oracle.com Sun Feb 16 18:11:02 2014 From: Alan.Bateman at oracle.com (Alan Bateman) Date: Sun, 16 Feb 2014 18:11:02 +0000 Subject: 8034175: Remove use of UseVMInterruptibleIO from tests In-Reply-To: References: <52F9FC06.2030101@oracle.com> <575D7215-AFEA-432D-9744-1C5F015B9F30@oracle.com> <52FA050E.5010501@oracle.com> <3D00C6E5-2742-483E-A121-D7C899DFF617@oracle.com> <52FA492C.30900@oracle.com> Message-ID: <5300FF36.6030905@oracle.com> On 11/02/2014 16:14, Martin Buchholz wrote: > If you wanted to be super-conservative, you could simply remove the > unwanted flag without changing the type of test, in this changeset. I've created JDK-8034208 for the additional hygiene that you inlined in the last mail. The changes look good and I've assigned the bug (hope that it okay). On ThrowingTasks.java then one suggestion is to change it to have two @run tags so that it runs both with and without a security manager. That might be a bit better than the current randomness. -Alan From ivan.gerasimov at oracle.com Sun Feb 16 19:07:22 2014 From: ivan.gerasimov at oracle.com (Ivan Gerasimov) Date: Sun, 16 Feb 2014 23:07:22 +0400 Subject: RFR [8023022] Some more typos in javadoc Message-ID: <53010C6A.5080909@oracle.com> Hello! It was reported that there is a typo in the javadoc for the Thread class: s/If if were/If it were/ I found several other places with the same typo across the jdk. Would you please help review this trivial fix? http://cr.openjdk.java.net/~igerasim/8023022/0/webrev/ Thanks, Ivan From lance.andersen at oracle.com Sun Feb 16 22:03:02 2014 From: lance.andersen at oracle.com (Lance @ Oracle) Date: Sun, 16 Feb 2014 17:03:02 -0500 Subject: RFR [8023022] Some more typos in javadoc In-Reply-To: <53010C6A.5080909@oracle.com> References: <53010C6A.5080909@oracle.com> Message-ID: <20624D63-564D-4111-97E4-8F2147A701B9@oracle.com> Looks fine Lance Andersen| Principal Member of Technical Staff | +1.781.442.2037 Oracle Java Engineering 1 Network Drive Burlington, MA 01803 Lance.Andersen at oracle.com Sent from my iPad On Feb 16, 2014, at 2:07 PM, Ivan Gerasimov wrote: > Hello! > > It was reported that there is a typo in the javadoc for the Thread class: > s/If if were/If it were/ > > I found several other places with the same typo across the jdk. > > Would you please help review this trivial fix? > > http://cr.openjdk.java.net/~igerasim/8023022/0/webrev/ > > Thanks, > Ivan From stuart.marks at oracle.com Mon Feb 17 07:17:31 2014 From: stuart.marks at oracle.com (Stuart Marks) Date: Sun, 16 Feb 2014 23:17:31 -0800 Subject: A hole in the serialization spec In-Reply-To: <52FE55BD.2010201@redhat.com> References: <52F4F6BB.2010207@redhat.com> <52F8E807.2090403@oracle.com> <52F8F21C.80205@redhat.com> <52FB8E5C.9020502@oracle.com> <52FB9248.6010703@redhat.com> <26199217-BAE6-47AC-AFC8-4DE90726B8C0@oracle.com> <52FD0328.3050304@redhat.com> <52FE3CB2.5060408@redhat.com> <52FE55BD.2010201@redhat.com> Message-ID: <5301B78B.80801@oracle.com> On 2/14/14 9:43 AM, David M. Lloyd wrote: > On 02/14/2014 09:56 AM, David M. Lloyd wrote: >> In the JDK, java.util.Date does not read/write fields. Perhaps others >> as well. Given that the behavior is presently undefined, that means the >> serialized representation of java.util.Date (and any other such >> non-conforming classes) are also undefined. > > An interesting detail here - since Date doesn't have any non-transient fields, > this happens to work out OK for a second reason (that defaultReadFields() would > read nothing anyway) - however it still would break if a non-transient field > were to be added. Hi David, (coming late to this party) Thanks for pointing out these clauses in the serialization specification. I always knew that these methods "should" behave this way but I was unaware of the undefined qualification in the spec, and I was also unaware that even JDK classes like java.util.Date have readObject/writeObject methods that don't fulfil this requirement. I also think you're right that these problems are widespread. A recent blog post on serialization [1] has some sample code whose readObject/writeObject methods don't fulfil this requirement either. On the other hand, this requirement doesn't seem to appear in the javadoc anyplace that I can find. The class doc for java.io.Serializable is the most explicit, and it says, The writeObject method is responsible for writing the state of the object for its particular class so that the corresponding readObject method can restore it. The default mechanism for saving the Object's fields can be invoked by calling out.defaultWriteObject. The method does not need to concern itself with the state belonging to its superclasses or subclasses. State is saved by writing the individual fields to the ObjectOutputStream using the writeObject method or by using the methods for primitive data types supported by DataOutput. The readObject method is responsible for reading from the stream and restoring the classes fields. It may call in.defaultReadObject to invoke the default mechanism for restoring the object's non-static and non-transient fields. The defaultReadObject method uses information in the stream to assign the fields of the object saved in the stream with the correspondingly named fields in the current object. This handles the case when the class has evolved to add new fields. The method does not need to concern itself with the state belonging to its superclasses or subclasses. State is saved by writing the individual fields to the ObjectOutputStream using the writeObject method or by using the methods for primitive data types supported by DataOutput. The wording here seems to imply that calling defaultWriteObject and defaultReadObject is optional. It does look like the various bits of the specification could use some cleanup. In your initial post, you said that problems with the serialization specification that have caused user problems. Can you be more specific about what these problems were? In another message earlier in this thread, you had made a few suggestions: > 1) do nothing :( > 2) start throwing (or writing) an exception in write/readObject when stream ops are performed without reading fields (maybe can be disabled with a sys prop or something) > 3) leave fields cleared and risk protocol issues > 4) silently start reading/writing empty field information (risks protocol issues) I'd have to say that #2 is pretty close to a non-starter. Since the problem does appear to be widespread, a lot of software would start suffering this exception even if it otherwise seems to be behaving correctly. This is clearly a big behavioral incompatibility, and even if it could be mitigated with a system property, I'd question whether it was worthwhile. #4 also seems to be a fairly large incompatibility. If a class's writeObject method is missing a defaultWriteObject call, it has a fairly stable behavior, although one that's defined by the implementation as opposed to the specification. (Although the specification isn't self-consistent, per the above.) Silently changing the bytes emitted in these cases would certainly cause incompatibilities with existing readObject methods that are unprepared to deal with them. #3 leads me to mention another area of the serialization specification that *is* well-defined, which is what occurs if fields are added or removed from one object version to the next. This is covered in sections 5.6.1 and 5.6.2 of the spec. [2] Briefly, if the current object has fields for which values are not present in the serialization stream, those fields are initialized to their default values (zero, null, false). Does this have any bearing on the issues you're concerned about? (It doesn't say so very explicitly, but field data that appears in the serialized form is ignored if there is no corresponding field in the current class.) Finally, another suggestion that might help with these issues is not to change the JDK, but to use static analysis tools such as FindBugs to help programmers identify problem code. s'marks [1] http://marxsoftware.blogspot.com/2014/02/serializing-java-objects-with-non.html [2] http://docs.oracle.com/javase/7/docs/platform/serialization/spec/version.html#5172 From Alan.Bateman at oracle.com Mon Feb 17 09:10:46 2014 From: Alan.Bateman at oracle.com (Alan Bateman) Date: Mon, 17 Feb 2014 09:10:46 +0000 Subject: RFR [8023022] Some more typos in javadoc In-Reply-To: <53010C6A.5080909@oracle.com> References: <53010C6A.5080909@oracle.com> Message-ID: <5301D216.4060308@oracle.com> On 16/02/2014 19:07, Ivan Gerasimov wrote: > Hello! > > It was reported that there is a typo in the javadoc for the Thread class: > s/If if were/If it were/ > > I found several other places with the same typo across the jdk. > > Would you please help review this trivial fix? > > http://cr.openjdk.java.net/~igerasim/8023022/0/webrev/ Looks good. -Alan. From ivan.gerasimov at oracle.com Mon Feb 17 11:08:10 2014 From: ivan.gerasimov at oracle.com (Ivan Gerasimov) Date: Mon, 17 Feb 2014 15:08:10 +0400 Subject: RFR [8023022] Some more typos in javadoc In-Reply-To: <5301D216.4060308@oracle.com> References: <53010C6A.5080909@oracle.com> <5301D216.4060308@oracle.com> Message-ID: <5301ED9A.8050804@oracle.com> I was going to include even more typo fixes of a similar kind, but then realized that the following two commands return ~ 400 places: grep -r ' \([a-z][a-z]\) \1 ' . grep -r ' \([a-z][a-z][a-z]\) \1 ' . Not all these are typos, but many are. So I'm giving up - no more typo fixes :-) This should probably be done in some other systematic way. Sincerely yours, Ivan On 17.02.2014 13:10, Alan Bateman wrote: > On 16/02/2014 19:07, Ivan Gerasimov wrote: >> Hello! >> >> It was reported that there is a typo in the javadoc for the Thread >> class: >> s/If if were/If it were/ >> >> I found several other places with the same typo across the jdk. >> >> Would you please help review this trivial fix? >> >> http://cr.openjdk.java.net/~igerasim/8023022/0/webrev/ > Looks good. > > -Alan. > > From chris.hegarty at oracle.com Mon Feb 17 11:42:14 2014 From: chris.hegarty at oracle.com (Chris Hegarty) Date: Mon, 17 Feb 2014 11:42:14 +0000 Subject: A hole in the serialization spec In-Reply-To: <5301B78B.80801@oracle.com> References: <52F4F6BB.2010207@redhat.com> <52F8E807.2090403@oracle.com> <52F8F21C.80205@redhat.com> <52FB8E5C.9020502@oracle.com> <52FB9248.6010703@redhat.com> <26199217-BAE6-47AC-AFC8-4DE90726B8C0@oracle.com> <52FD0328.3050304@redhat.com> <52FE3CB2.5060408@redhat.com> <52FE55BD.2010201@redhat.com> <5301B78B.80801@oracle.com> Message-ID: <5301F596.1000300@oracle.com> Let me try to summarize the issue, as I see it. Circling back to David's original quote from the Serialization spec [1] "The class's writeObject method, if implemented, is responsible for saving the state of the class. Either ObjectOutputStream's defaultWriteObject or writeFields method must be called once (and only once) before writing any optional data that will be needed by the corresponding readObject method to restore the state of the object; even if no optional data is written, defaultWriteObject or writeFields must still be invoked once. If defaultWriteObject or writeFields is not invoked once prior to the writing of optional data (if any), then the behavior of instance deserialization i_s undefined in cases where the ObjectInputStream cannot resolve the class which defined the writeObject method in question._" The underlying section above is most relevant. It is a qualification of the scenario where the behavior is undefined. I read it to mean; the behavior is undefined if, and only if, the OIS cannot resolve the class which defined the writeObject. And this seems in line with David's description [2] (which I agree). "I think the specifics of the quote relate to this kind of class change; in particular, if a class is deleted from the hierarchy on the read side, and that class corresponds to the class that had the misbehaving writeObject, I suspect that things will break at that point as the read side will probably try to consume and discard the field information for that class, which will be missing (it will start reading the next class' fields instead I think)." My take on this is that the above writeObject undefined qualification is referring to a compatibility issue. Since removing a class from the hierarchy is a compatible change [3], then default read/write Object/Fields must be called, otherwise, if a class is removed from the hierarchy the behavior is undefined. In my testing I get StreamCorruptException, but I can see how this could behave differently, depending on the class hierarchy and actual serialization state. If the class defining the writeObject is resolvable, then the behavior is *not* undefined. Do we agree on what is actually undefined, and what is not? -Chris. [1] http://docs.oracle.com/javase/7/docs/platform/serialization/spec/output.html#861 [2] http://mail.openjdk.java.net/pipermail/core-libs-dev/2014-February/025069.html [3] http://docs.oracle.com/javase/7/docs/platform/serialization/spec/version.html#6754 On 17/02/14 07:17, Stuart Marks wrote: > On 2/14/14 9:43 AM, David M. Lloyd wrote: >> On 02/14/2014 09:56 AM, David M. Lloyd wrote: >>> In the JDK, java.util.Date does not read/write fields. Perhaps others >>> as well. Given that the behavior is presently undefined, that means >>> the >>> serialized representation of java.util.Date (and any other such >>> non-conforming classes) are also undefined. >> >> An interesting detail here - since Date doesn't have any >> non-transient fields, >> this happens to work out OK for a second reason (that >> defaultReadFields() would >> read nothing anyway) - however it still would break if a >> non-transient field >> were to be added. > > Hi David, > > (coming late to this party) > > Thanks for pointing out these clauses in the serialization > specification. I always knew that these methods "should" behave this > way but I was unaware of the undefined qualification in the spec, and > I was also unaware that even JDK classes like java.util.Date have > readObject/writeObject methods that don't fulfil this requirement. > > I also think you're right that these problems are widespread. A recent > blog post on serialization [1] has some sample code whose > readObject/writeObject methods don't fulfil this requirement either. > > On the other hand, this requirement doesn't seem to appear in the > javadoc anyplace that I can find. The class doc for > java.io.Serializable is the most explicit, and it says, > > The writeObject method is responsible for writing the state of the > object for its particular class so that the corresponding readObject > method can restore it. The default mechanism for saving the Object's > fields can be invoked by calling out.defaultWriteObject. The method > does not need to concern itself with the state belonging to its > superclasses or subclasses. State is saved by writing the individual > fields to the ObjectOutputStream using the writeObject method or by > using the methods for primitive data types supported by DataOutput. > > The readObject method is responsible for reading from the stream and > restoring the classes fields. It may call in.defaultReadObject to > invoke the default mechanism for restoring the object's non-static > and non-transient fields. The defaultReadObject method uses > information in the stream to assign the fields of the object saved > in the stream with the correspondingly named fields in the current > object. This handles the case when the class has evolved to add new > fields. The method does not need to concern itself with the state > belonging to its superclasses or subclasses. State is saved by > writing the individual fields to the ObjectOutputStream using the > writeObject method or by using the methods for primitive data types > supported by DataOutput. > > The wording here seems to imply that calling defaultWriteObject and > defaultReadObject is optional. > > It does look like the various bits of the specification could use some > cleanup. > > In your initial post, you said that problems with the serialization > specification that have caused user problems. Can you be more specific > about what these problems were? > > In another message earlier in this thread, you had made a few > suggestions: > >> 1) do nothing :( >> 2) start throwing (or writing) an exception in write/readObject when >> stream ops are performed without reading fields (maybe can be >> disabled with a sys prop or something) >> 3) leave fields cleared and risk protocol issues >> 4) silently start reading/writing empty field information (risks >> protocol issues) > > I'd have to say that #2 is pretty close to a non-starter. Since the > problem does appear to be widespread, a lot of software would start > suffering this exception even if it otherwise seems to be behaving > correctly. This is clearly a big behavioral incompatibility, and even > if it could be mitigated with a system property, I'd question whether > it was worthwhile. > > #4 also seems to be a fairly large incompatibility. If a class's > writeObject method is missing a defaultWriteObject call, it has a > fairly stable behavior, although one that's defined by the > implementation as opposed to the specification. (Although the > specification isn't self-consistent, per the above.) Silently changing > the bytes emitted in these cases would certainly cause > incompatibilities with existing readObject methods that are unprepared > to deal with them. > > #3 leads me to mention another area of the serialization specification > that *is* well-defined, which is what occurs if fields are added or > removed from one object version to the next. This is covered in > sections 5.6.1 and 5.6.2 of the spec. [2] Briefly, if the current > object has fields for which values are not present in the > serialization stream, those fields are initialized to their default > values (zero, null, false). Does this have any bearing on the issues > you're concerned about? (It doesn't say so very explicitly, but field > data that appears in the serialized form is ignored if there is no > corresponding field in the current class.) > > Finally, another suggestion that might help with these issues is not > to change the JDK, but to use static analysis tools such as FindBugs > to help programmers identify problem code. > > s'marks > > > [1] > http://marxsoftware.blogspot.com/2014/02/serializing-java-objects-with-non.html > > [2] > http://docs.oracle.com/javase/7/docs/platform/serialization/spec/version.html#5172 > From david.lloyd at redhat.com Mon Feb 17 15:36:19 2014 From: david.lloyd at redhat.com (David M. Lloyd) Date: Mon, 17 Feb 2014 09:36:19 -0600 Subject: A hole in the serialization spec In-Reply-To: <5301B78B.80801@oracle.com> References: <52F4F6BB.2010207@redhat.com> <52F8E807.2090403@oracle.com> <52F8F21C.80205@redhat.com> <52FB8E5C.9020502@oracle.com> <52FB9248.6010703@redhat.com> <26199217-BAE6-47AC-AFC8-4DE90726B8C0@oracle.com> <52FD0328.3050304@redhat.com> <52FE3CB2.5060408@redhat.com> <52FE55BD.2010201@redhat.com> <5301B78B.80801@oracle.com> Message-ID: <53022C73.6070107@redhat.com> On 02/17/2014 01:17 AM, Stuart Marks wrote: > On 2/14/14 9:43 AM, David M. Lloyd wrote: >> On 02/14/2014 09:56 AM, David M. Lloyd wrote: >>> In the JDK, java.util.Date does not read/write fields. Perhaps others >>> as well. Given that the behavior is presently undefined, that means the >>> serialized representation of java.util.Date (and any other such >>> non-conforming classes) are also undefined. >> >> An interesting detail here - since Date doesn't have any non-transient >> fields, >> this happens to work out OK for a second reason (that >> defaultReadFields() would >> read nothing anyway) - however it still would break if a non-transient >> field >> were to be added. > > Hi David, > > (coming late to this party) Thanks for replying, comments below. > Thanks for pointing out these clauses in the serialization > specification. I always knew that these methods "should" behave this way > but I was unaware of the undefined qualification in the spec, and I was > also unaware that even JDK classes like java.util.Date have > readObject/writeObject methods that don't fulfil this requirement. That's the case that blew up for us first, so that's how I knew about it. For a little background, we (JBoss) have a custom serialization wire protocol which nevertheless follows the serialization spec as exactly as possible from an API point of view. When I was studying the specification, I noted this bit and decided our action would be the #2 option (throw an exception when this happens), which effectively let me decide what to do later. This was when we discovered that java.util.Date has this issue. However I noted at the time that since this class has no non-transient fields, allowing this case is actually a relatively low risk. So we began allowing classes with no fields to skip the put/getFields step, while still throwing an exception if there are fields defined (at either end). This got us past the java.util.Date case (which is a particularly problematic case given how pervasive it appears to be in user code, particularly Java EE code). Recently I "fixed" the remaining cases in our implementation by doing the read/discard stuff I describe in the #4 option. I had realized that doing so is backwards compatible with what we had been doing (basically replacing the exception with "safe" behavior). This works out very well for us, as you can imagine. :) The problem with doing this in the JDK is, as you know, that there are/will be serialized blobs out there that would not include field information that would then be deserialized with implementations that expect it, and vice-versa. I wonder if perhaps it would be a good idea to make this behavior configurable though, for cases where both ends are under the same control. Or, perhaps it could be a protocol version thing (though IIRC there isn't a great facility for forwards-compatibility in the event that an "old" deserializer reads a "new" stream - how was this done for enums I wonder?). > I also think you're right that these problems are widespread. A recent > blog post on serialization [1] has some sample code whose > readObject/writeObject methods don't fulfil this requirement either. Yeah I wouldn't be surprised. We recently encountered at least one class in one of the Bouncy Castle security provider packages as well with this issue - except that class has fields. > On the other hand, this requirement doesn't seem to appear in the > javadoc anyplace that I can find. Yeah this is quite unfortunate. > In your initial post, you said that problems with the serialization > specification that have caused user problems. Can you be more specific > about what these problems were? As I outlined above. We want to provide a performant, and more importantly, a safe experience for users; our custom protocol enabled the former and our conservative behavior allowed the latter. We have what turns out to be a quite good solution for our own case, but this doesn't help the general community much. > In another message earlier in this thread, you had made a few suggestions: > >> 1) do nothing :( >> 2) start throwing (or writing) an exception in write/readObject when >> stream ops are performed without reading fields (maybe can be disabled >> with a sys prop or something) >> 3) leave fields cleared and risk protocol issues >> 4) silently start reading/writing empty field information (risks >> protocol issues) > [trim response for #2 and #4 which I agree with] > > #3 leads me to mention another area of the serialization specification > that *is* well-defined, which is what occurs if fields are added or > removed from one object version to the next. This is covered in sections > 5.6.1 and 5.6.2 of the spec. [2] Briefly, if the current object has > fields for which values are not present in the serialization stream, > those fields are initialized to their default values (zero, null, > false). Does this have any bearing on the issues you're concerned about? > (It doesn't say so very explicitly, but field data that appears in the > serialized form is ignored if there is no corresponding field in the > current class.) Yeah this is consistent with my understanding of the specification. I think it is also implied in 3.4, "If the class being restored is not present in the stream being read, then its readObjectNoData method, if defined, is invoked (instead of readObject); otherwise, its fields are initialized to the appropriate default values. For further detail, see section 3.5." Wire protocol aside, this requirement is what led to my #4 proposal: from an API perspective, the behavior is no different. The fields are initialized to default zero values. But it enables users to add get/put field operations later on without protocol corruption. Of course this change would be essentially protocol-incompatible with the current Oracle/OpenJDK implementation, as was pointed out already in various ways, unless as I said there is some clever way that I can't immediately see to let this work. > Finally, another suggestion that might help with these issues is not to > change the JDK, but to use static analysis tools such as FindBugs to > help programmers identify problem code. Yeah, this is basically the status quo I think. It would however be extra nice if the compiler could emit a warning at the least, instead of relying on external static analysis tools, given how central serialization is. The necessary static analysis should be relatively simple, from what little I recall of the compiler internals. One possible bad side-effect would be that people might "fix" their broken classes without realizing they've just broken their wire compatibility though. Such a tool should probably emit a message to the effect of "classes without OOS.dWO/wF should only have transient fields" instead of "this class should have a OOS.dWO/wF call". While adding transient to fields of such classes would technically change the serialized form of the class, that form is never used in any event, so it's an "okay" change, and decreases the serialized class descriptor size besides... and in the long term may even allow adding a dWO/dRO pair in the future, depending on the compatibility needs of the user for that class. -- - DML From fweimer at redhat.com Mon Feb 17 15:43:05 2014 From: fweimer at redhat.com (Florian Weimer) Date: Mon, 17 Feb 2014 16:43:05 +0100 Subject: Minor com.sun.jndi.dns cleanup Message-ID: <53022E09.2070002@redhat.com> This patch removes an unnecessary allocation from DnsClient and adds explicit detection for DNS label compression loops in ResourceRecord. Previously, the code relied on a StackOverflowError for exiting the parsing loop. -- Florian Weimer / Red Hat Product Security Team From david.lloyd at redhat.com Mon Feb 17 15:50:47 2014 From: david.lloyd at redhat.com (David M. Lloyd) Date: Mon, 17 Feb 2014 09:50:47 -0600 Subject: A hole in the serialization spec In-Reply-To: <5301F596.1000300@oracle.com> References: <52F4F6BB.2010207@redhat.com> <52F8E807.2090403@oracle.com> <52F8F21C.80205@redhat.com> <52FB8E5C.9020502@oracle.com> <52FB9248.6010703@redhat.com> <26199217-BAE6-47AC-AFC8-4DE90726B8C0@oracle.com> <52FD0328.3050304@redhat.com> <52FE3CB2.5060408@redhat.com> <52FE55BD.2010201@redhat.com> <5301B78B.80801@oracle.com> <5301F596.1000300@oracle.com> Message-ID: <53022FD7.8040602@redhat.com> On 02/17/2014 05:42 AM, Chris Hegarty wrote: > Let me try to summarize the issue, as I see it. > > Circling back to David's original quote from the Serialization spec [1] > > "The class's writeObject method, if implemented, is responsible for > saving the state of the class. Either ObjectOutputStream's > defaultWriteObject or writeFields method must be called once (and only > once) before writing any optional data that will be needed by the > corresponding readObject method to restore the state of the object; even > if no optional data is written, defaultWriteObject or writeFields must > still be invoked once. If defaultWriteObject or writeFields is not > invoked once prior to the writing of optional data (if any), then the > behavior of instance deserialization i_s undefined in cases where the > ObjectInputStream cannot resolve the class which defined the writeObject > method in question._" > > The underlying section above is most relevant. It is a qualification of > the scenario where the behavior is undefined. I read it to mean; the > behavior is undefined if, and only if, the OIS cannot resolve the class > which defined the writeObject. And this seems in line with David's > description [2] (which I agree). I think I see what you're getting at, or rather, what you're trying to get out of it. You are trying to say that not persisting fields is only undefined if the other side's class goes away, based on ?2.3. Unfortunately I don't think we can get away with this interpretation, as the read side of the specification at ?3.4 isn't any less valid and it flat-out says that not reading fields is completely undefined, so having defined behavior on one end doesn't really help here. But I think it does leave a little latitude for improvement. If the current behavior could be specified, this will not affect any existing code, but will better inform those users who read this specification. In particular, making it clear that skipping these methods is semantically equivalent to writing a class with no non-transient fields would be a particularly useful clarification; I am coming to think that any class which does this should only have transient fields as this allows the default*/get/putFields methods to be added in later without a risk to wire compatibility. With the specification clarified in this regard, it becomes possible to do two important things: 1) The API JavaDoc can be updated to be not only completely consistent with the specification, but also with the long-standing behavior of (at least) Sun and Oracle JDKs and OpenJDK, and also (I believe) other JDKs. 2) It allows all implementations to be able to guarantee interoperability for cases which are presently undefined yet are definitely pervasive, even within the JDK itself. > "I think the specifics of the quote relate to this kind of class change; > in particular, if a class is deleted from the hierarchy on the read > side, and that class corresponds to the class that had the misbehaving > writeObject, I suspect that things will break at that point as the read > side will probably try to consume and discard the field information for > that class, which will be missing (it will start reading the next class' > fields instead I think)." > > My take on this is that the above writeObject undefined qualification is > referring to a compatibility issue. Since removing a class from the > hierarchy is a compatible change [3], then default read/write > Object/Fields must be called, otherwise, if a class is removed from the > hierarchy the behavior is undefined. In my testing I get > StreamCorruptException, but I can see how this could behave differently, > depending on the class hierarchy and actual serialization state. > > If the class defining the writeObject is resolvable, then the behavior > is *not* undefined. > > Do we agree on what is actually undefined, and what is not? Also, if the class which did not write fields is not resolvable, but its descriptor defines no fields anyway (either they're all transient or there is an empty serialPersistentFields defined), then the behavior is still predictable, "safe", and well-understood, though still technically undefined by spec. Honestly I'm coming to the conclusion that the spec should just be updated to match the existing behavior. At least this way, it is possible to establish and explain exactly what is happening and why, and what the effect is on the wire protocol, and how to cope with changes to the affected class hierarchies in a compatible way. -- - DML From fweimer at redhat.com Mon Feb 17 16:22:36 2014 From: fweimer at redhat.com (Florian Weimer) Date: Mon, 17 Feb 2014 17:22:36 +0100 Subject: Minor com.sun.jndi.dns cleanup In-Reply-To: <53022E09.2070002@redhat.com> References: <53022E09.2070002@redhat.com> Message-ID: <5302374C.4090409@redhat.com> On 02/17/2014 04:43 PM, Florian Weimer wrote: > This patch removes an unnecessary allocation from DnsClient and adds > explicit detection for DNS label compression loops in ResourceRecord. > Previously, the code relied on a StackOverflowError for exiting the > parsing loop. Mailman ate the attachment, so I put it up here: Note that other implementations fixed this as CVE-2000-0333 a long time ago, but due to the lack of tail call optimization and reliable stack overflow detection, this is currently not a security vulnerability in OpenJDK (not even an endless loop). -- Florian Weimer / Red Hat Product Security Team From daniel.fuchs at oracle.com Mon Feb 17 16:23:44 2014 From: daniel.fuchs at oracle.com (Daniel Fuchs) Date: Mon, 17 Feb 2014 17:23:44 +0100 Subject: RFR (JAXP): 8033980 : Xerces Update: datatype XMLGregorianCalendarImpl and DurationImpl In-Reply-To: <52FE53C9.3010205@oracle.com> References: <52F9666F.9010303@oracle.com> <5111B487-22C1-4987-B9CC-ABF2F5CDD964@oracle.com> <52FA5855.3020106@oracle.com> <52FA7270.4090709@oracle.com> <52FAF801.5060300@oracle.com> <52FBCF66.8060308@oracle.com> <52FC7FE6.2060400@oracle.com> <52FCD57B.8090807@oracle.com> <52FE53C9.3010205@oracle.com> Message-ID: <53023790.4000708@oracle.com> Hi Joe, Sorry for the late reply... On 2/14/14 6:35 PM, huizhe wang wrote: > Hi All, > > I added a SerializationTest. The test contains a helper that can > generate serialization files for XMLGregorianCalendar and Duration. I've > created such files for jdk6,7,8 and 9, and manually run the test, that > is, read them back with JDK6, 7, 8 and 9. The test worked fine. In the > JDK9(or 10 in the future) repo and for an auto-run, it would use the > current JDK9/10 build and test against JDK6, 7, 8 and 9. Past JDK10, we > could consider add serialization files for JDK10. > > The new fields did not affect serialization compatibility. The above > tests passed with/without the new fields being transient. But I added > transient since it's the right thing to do. > > Adding fields is a compatible change in accordance with Java Object > Serialization Spec > . Concerning the new test I think it would be much more reliable to open the .ser files as a resource, using something like: InputStream streamIn = SerializationTest.class.getResourceAsStream( javaVersion + FILENAME_CAL); ObjectInputStream objectinputstream = new ObjectInputStream(streamIn); The reason is that URL.getPath() is an URL path - not a file path, so it can contain encoded characters (such as %20) that FileInputStream will not understand. I suspect that if there's a white space or some other special character in the path somewhere your current mechanism will not work. Also it would be good to verify what is the value of the transient fields after deserialization: I see that XMLGregorianCalendarImpl initializes them with non-default values - and you may need to write a readObject() to reinitialize those fields to these non default values: the deserialization framework may simply leave them with the global default for fields of that type (null for objects, 0 for numerical values, false for boolean etc...) [I am not 100% sure but it will not hurt to check] best regards, -- daniel > > > Thanks, > Joe > > On 2/13/2014 6:23 AM, Alan Bateman wrote: >> On 13/02/2014 08:18, huizhe wang wrote: >>> Hi Alan, Lance, and Daniel, >>> >>> The Xerces serialization revision meant to create a serialization >>> form that would help maintain future serialization compatibility. But >>> in reality it itself is causing significant incompatibility as Alan >>> pointed out below and we discussed previously. I've removed the >>> revision from the patch as a result. >>> >>> Please see the new webrev here: >>> http://cr.openjdk.java.net/~joehw/jdk9/8033980/webrev/ >> Thanks for dropping the serialization change as it was just not going >> to work the way you had intended. >> >> I agree with Daniel's comment about all the new fields added to >> XMLGregorianCalendarImpl as it's not clear why they aren't transient. >> >> I have not studied the rest of the changes but I think Daniel and >> Lance are. >> >> -Alan > From michael.cui at oracle.com Mon Feb 17 16:51:30 2014 From: michael.cui at oracle.com (michael cui) Date: Tue, 18 Feb 2014 00:51:30 +0800 Subject: Discussion on root cause analysis of JDK-7052625 : com/sun/net/httpserver/bugs/6725892/Test.java fails intermittently Message-ID: <53023E12.1060206@oracle.com> Hi, I would like to discuss my current root cause analysis of JDK-7052625 : com/sun/net/httpserver/bugs/6725892/Test.java fails intermittently As JDK-6725892 stated, the purpose of this regression test is testing bad http connections can be handled correctly which including + send no request + send an incomplete request + fail to read the response completely. test3() method will start 20 threads for each type listed above at same time. So totally 60 threads started in test3(). Each thread will open connection to httpserver and simulate the normal or bad http request to see if http server can handle them correctly. (20 threads for incomplete read, 20 threads for incomplete write, 20 threads for read/write normal case) Those threads will be started at same time. Among them, 40 threads using sleep to simulate bad request. The http server created by the following api call : s1 = HttpServer.create (addr, 0); According API doc and ServerSocket.java source code, the second parameter is backlog of socket which is the maximum number of queued incoming connections to allow on the listening socket. Queued TCP connections exceeding this limit may be rejected by the TCP implementation.. The default value 50 will be used if it was set to zero (See api doc and ServerSocket.java )). Since in test3(), 40 threads out of total 60 threads will simulate bad http request by sleeping either at reading or writing, there could be a very little possibility that httpserver 's socket connection queue reach his limit (50 for default value) and some tcp connection will be rest at that situation. This could be the root cause of this intermittently failure. Test result of the original version : 0 failure on Linux for 10000 runs. 0 failure on solaris for 10000 runs. 6 failure on windows for 10000 runs 28 failures on mac for 10000 runs By increasing the thread number of bad request, we can observe that the frequency of failure will be increased. Test result of fix version in which backlog of httpserver was changed from 0 to 100. 0 failure on Linux for 10000 runs. 0 failure on solaris for 10000 runs. 0 failure on windows for 10000 runs 0 failures on mac for 10000 runs It seems to me that using default 0 for backlog of httpserver could be root cause of this intermittently failure. Are we comfortable with this analysis? If it is the root cause, could setting backlog as 100 be a suggest fix? Thanks, Michael Cui From Alan.Bateman at oracle.com Mon Feb 17 19:47:16 2014 From: Alan.Bateman at oracle.com (Alan Bateman) Date: Mon, 17 Feb 2014 19:47:16 +0000 Subject: Minor com.sun.jndi.dns cleanup In-Reply-To: <5302374C.4090409@redhat.com> References: <53022E09.2070002@redhat.com> <5302374C.4090409@redhat.com> Message-ID: <53026744.4030500@oracle.com> On 17/02/2014 16:22, Florian Weimer wrote: > > Mailman ate the attachment, so I put it up here: > > > > Note that other implementations fixed this as CVE-2000-0333 a long > time ago, but due to the lack of tail call optimization and reliable > stack overflow detection, this is currently not a security > vulnerability in OpenJDK (not even an endless loop). > This looks good to me. I just wonder if InvalidNameException is the right NamingException for this case. Would CommunicationException with an IOException as cause be more suitable? For the test then we need to add a @bug line with a bug for this (I'll create a bug). A the test is a negative test then maybe ParsingErrors might be be a better name. -Alan. From fweimer at redhat.com Mon Feb 17 20:01:12 2014 From: fweimer at redhat.com (Florian Weimer) Date: Mon, 17 Feb 2014 21:01:12 +0100 Subject: Minor com.sun.jndi.dns cleanup In-Reply-To: <53026744.4030500@oracle.com> References: <53022E09.2070002@redhat.com> <5302374C.4090409@redhat.com> <53026744.4030500@oracle.com> Message-ID: <53026A88.9030107@redhat.com> On 02/17/2014 08:47 PM, Alan Bateman wrote: > On 17/02/2014 16:22, Florian Weimer wrote: >> >> Mailman ate the attachment, so I put it up here: >> >> >> >> Note that other implementations fixed this as CVE-2000-0333 a long >> time ago, but due to the lack of tail call optimization and reliable >> stack overflow detection, this is currently not a security >> vulnerability in OpenJDK (not even an endless loop). >> > This looks good to me. I just wonder if InvalidNameException is the > right NamingException for this case. Would CommunicationException with > an IOException as cause be more suitable? DnsName throws InvalidNameException for names that exceed the 255 octet limit of DNS, and I followed that precedent. Looking at the InvalidNameException documentation, I understand why you think another exception might be better here. In the DnsName case, the same exception is used for parsing user-supplied strings and data from the wire, and strictly speaking, InvalidNameException should be used only in the former case. If we are picky about exceptions, we should also wrap those ArrayIndexOutOfBoundsExceptions. > For the test then we need to add a @bug line with a bug for this (I'll > create a bug). A the test is a negative test then maybe ParsingErrors > might be be a better name. It contains positive tests as well, to rule out that the change hasn't complete broken things. Should I split this test into two different files? -- Florian Weimer / Red Hat Product Security Team From Alan.Bateman at oracle.com Mon Feb 17 21:17:47 2014 From: Alan.Bateman at oracle.com (Alan Bateman) Date: Mon, 17 Feb 2014 21:17:47 +0000 Subject: Minor com.sun.jndi.dns cleanup In-Reply-To: <53026A88.9030107@redhat.com> References: <53022E09.2070002@redhat.com> <5302374C.4090409@redhat.com> <53026744.4030500@oracle.com> <53026A88.9030107@redhat.com> Message-ID: <53027C7B.2060506@oracle.com> On 17/02/2014 20:01, Florian Weimer wrote: > In the DnsName case, the same exception is used for parsing > user-supplied strings and data from the wire, and strictly speaking, > InvalidNameException should be used only in the former case. Yes, I think the exceptions in DnsName should be re-examined too (doesn't have to be this patch of course). For RecordRecord then it would be good to see if there is a better NamingException, the closest seems to be CommunicationException with an appropriate cause but maybe there are more suitable choices. > > If we are picky about exceptions, we should also wrap those > ArrayIndexOutOfBoundsExceptions. That one is really odd and would be good to re-examine sometime (doesn't have to be this patch of course). > >> For the test then we need to add a @bug line with a bug for this (I'll >> create a bug). A the test is a negative test then maybe ParsingErrors >> might be be a better name. > > It contains positive tests as well, to rule out that the change hasn't > complete broken things. Should I split this test into two different > files? It's okay as is (I initially thought it was mostly negative tests but this isn't so). -Alan From huizhe.wang at oracle.com Mon Feb 17 22:17:41 2014 From: huizhe.wang at oracle.com (huizhe wang) Date: Mon, 17 Feb 2014 14:17:41 -0800 Subject: RFR (JAXP): 8033980 : Xerces Update: datatype XMLGregorianCalendarImpl and DurationImpl In-Reply-To: <53023790.4000708@oracle.com> References: <52F9666F.9010303@oracle.com> <5111B487-22C1-4987-B9CC-ABF2F5CDD964@oracle.com> <52FA5855.3020106@oracle.com> <52FA7270.4090709@oracle.com> <52FAF801.5060300@oracle.com> <52FBCF66.8060308@oracle.com> <52FC7FE6.2060400@oracle.com> <52FCD57B.8090807@oracle.com> <52FE53C9.3010205@oracle.com> <53023790.4000708@oracle.com> Message-ID: <53028A85.3010807@oracle.com> Thanks Daniel. I updated the test to call class.getResourceAsStream (see fromFile method) instead at the runtime. The filePath now is only used for the helper method to generate test file. Also added readObject() to XMLGregorianCalendarImpl (at the bottom). The method calls the save() method to initialize the orig_* fields as done in the constructors. http://cr.openjdk.java.net/~joehw/jdk9/8033980/webrev/ Thanks, Joe On 2/17/2014 8:23 AM, Daniel Fuchs wrote: > Hi Joe, > > Sorry for the late reply... > > On 2/14/14 6:35 PM, huizhe wang wrote: >> Hi All, >> >> I added a SerializationTest. The test contains a helper that can >> generate serialization files for XMLGregorianCalendar and Duration. I've >> created such files for jdk6,7,8 and 9, and manually run the test, that >> is, read them back with JDK6, 7, 8 and 9. The test worked fine. In the >> JDK9(or 10 in the future) repo and for an auto-run, it would use the >> current JDK9/10 build and test against JDK6, 7, 8 and 9. Past JDK10, we >> could consider add serialization files for JDK10. >> >> The new fields did not affect serialization compatibility. The above >> tests passed with/without the new fields being transient. But I added >> transient since it's the right thing to do. >> >> Adding fields is a compatible change in accordance with Java Object >> Serialization Spec >> . >> > > Concerning the new test I think it would be much more > reliable to open the .ser files as a resource, using > something like: > > InputStream streamIn = SerializationTest.class.getResourceAsStream( > javaVersion + FILENAME_CAL); > ObjectInputStream objectinputstream = new ObjectInputStream(streamIn); > > The reason is that URL.getPath() is an URL path - not a file path, so it > can contain encoded characters (such as %20) that FileInputStream will > not understand. I suspect that if there's a white space or some other > special character in the path somewhere your current mechanism > will not work. > > Also it would be good to verify what is the value of the transient > fields after deserialization: I see that XMLGregorianCalendarImpl > initializes them with non-default values - and you may need to > write a readObject() to reinitialize those fields to these non > default values: the deserialization framework may simply leave > them with the global default for fields of that type (null > for objects, 0 for numerical values, false for boolean etc...) > [I am not 100% sure but it will not hurt to check] > > best regards, > > -- daniel > >> >> >> Thanks, >> Joe >> >> On 2/13/2014 6:23 AM, Alan Bateman wrote: >>> On 13/02/2014 08:18, huizhe wang wrote: >>>> Hi Alan, Lance, and Daniel, >>>> >>>> The Xerces serialization revision meant to create a serialization >>>> form that would help maintain future serialization compatibility. But >>>> in reality it itself is causing significant incompatibility as Alan >>>> pointed out below and we discussed previously. I've removed the >>>> revision from the patch as a result. >>>> >>>> Please see the new webrev here: >>>> http://cr.openjdk.java.net/~joehw/jdk9/8033980/webrev/ >>> Thanks for dropping the serialization change as it was just not going >>> to work the way you had intended. >>> >>> I agree with Daniel's comment about all the new fields added to >>> XMLGregorianCalendarImpl as it's not clear why they aren't transient. >>> >>> I have not studied the rest of the changes but I think Daniel and >>> Lance are. >>> >>> -Alan >> > From michael.cui at oracle.com Tue Feb 18 05:57:56 2014 From: michael.cui at oracle.com (michael cui) Date: Tue, 18 Feb 2014 13:57:56 +0800 Subject: RFR for JDK-8031563: TEST_BUG: java/nio/channels/Selector/ChangingInterests.java failed once In-Reply-To: <52FCB756.5080906@oracle.com> References: <52FB60AB.4070708@oracle.com> <52FB7868.3080907@oracle.com> <52FCB756.5080906@oracle.com> Message-ID: <5302F664.1060504@oracle.com> Hi Alan, On 02/13/2014 08:15 PM, Alan Bateman wrote: > I plan to look at this soon but I have a concern that it is changing > the test so that the change from OP_READ to OP_READ|OP_WRITE no longer > tests that the key is both readable and writable after one select. Have you got chance to review the code change of proposed solution A and B? For solution B, it seems to me that code change isn't a change to test. I would like to see if you still have same concern about any cases after reviewing the code. Thanks, Michael Cui From volker.simonis at gmail.com Tue Feb 18 09:43:08 2014 From: volker.simonis at gmail.com (Volker Simonis) Date: Tue, 18 Feb 2014 10:43:08 +0100 Subject: RFR(S): 8034087: XML parser may overwrite element content if that content falls onto the border of an entity scanner buffer In-Reply-To: <52FBBCB7.9010907@oracle.com> References: <52FA535D.70804@oracle.com> <52FA9CC5.3010104@oracle.com> <61C6FA0C43817B4FA6EB52E901E137E7729A7913@DEWDFEMB18B.global.corp.sap> <52FBBCB7.9010907@oracle.com> Message-ID: Hi, I've moved the test to test/javax/xml/jaxp/parsers/8027359 as requested: http://cr.openjdk.java.net/~simonis/webrevs/8034087_1/ Can I please get a review for this change. Thank you and best regards, Volker On Wed, Feb 12, 2014 at 7:25 PM, huizhe wang wrote: > Ok, understand. > > Thanks, > Joe > > > On 2/11/2014 11:40 PM, Schreiber, Steffen wrote: >> >> Hi Joe, >> >> Yes, that's certainly true, if the test is only to show the issue. >> We intended the test to work as a regression test as well and wanted to >> check the effects of the buffer boundaries on more possible places, i.e. >> before/at/after special characters like <, > or / as well as inside tags and >> content. >> >> We can easily adapt the loop, if runtime is a concern here. >> >> Regards, >> Steffen >> >> -----Original Message----- >> From: huizhe wang [mailto:huizhe.wang at oracle.com] >> Sent: Dienstag, 11. Februar 2014 22:57 >> To: Volker Simonis >> Cc: Alan Bateman; Schreiber, Steffen; Java Core Libs >> Subject: Re: RFR(S): 8034087: XML parser may overwrite element content if >> that content falls onto the border of an entity scanner buffer >> >> Hi Volker, >> >> I agree with the approach below and jdk9/dev is the better forest. >> >> For the test itself, I would suggest reducing the following loop to 1 or >> 2 cases: >> >> for (int i = 0; i < testString.length(); i++) { >> test(createDocument(testString.toString(), i), ""+ i); >> } >> >> >> when i=7, the problem starts to show. It's sufficient to demonstrate the >> issue then by just entering 7. It's unnecessary to run the test 43 times. >> >> >> Thanks, >> Joe >> >> On 2/11/2014 9:00 AM, Volker Simonis wrote: >>> >>> Hi Alan, >>> >>> you're right. I initially didn't saw the test because I just looked at >>> the change in the jaxp repository. >>> >>> If it will be approved, I'll put the test in the same directory like >>> the other test (i.e. test/javax/xml/jaxp/parsers/8027359). >>> >>> And yes, my plan was to get approval for both, the tests and the fix, >>> when asking for the permission to downport to jdk8u-dev and jdk7u-dev. >>> >>> Thanks, >>> Volker >>> >>> >>> On Tue, Feb 11, 2014 at 5:44 PM, Alan Bateman >>> wrote: >>>> >>>> On 11/02/2014 14:57, Volker Simonis wrote: >>>>> >>>>> Hi, >>>>> >>>>> after opening this bug yesterday for an issue found by my colleague >>>>> Steffen Schreiber we realized that this is actually a duplicate of >>>>> "8027359: XML parser returns incorrect parsing results" >>>>> (https://bugs.openjdk.java.net/browse/JDK-8027359). >>>>> >>>>> While there's no need now to submit a patch anymore, we'd >>>>> nevertheless like to contribute at least our test case for this issue: >>>>> >>>>> http://cr.openjdk.java.net/~simonis/webrevs/8034087/ >>>>> >>>>> The webrev is against jdk9-client but we'd like to also downport this >>>>> test to jdk7 and jdk8 to track that the fix for 8027359 will be >>>>> correctly downported to these releases as well. >>>>> >>>>> I will sponsor this change if somebody would be so kind to review it. >>>>> >>>> I'll leave it to Joe Wang to comment on the test but just to mention >>>> that >>>> jdk9/dev is probably a better forest to aim for because that is where >>>> the >>>> XML (and its tests) usually go. Also I wonder if it might be better to >>>> put >>>> it in the same directory as the test that Joe pushed with the change? >>>> >>>> If you are getting approval to push to jdk8u-dev and jdk7u-dev then it >>>> might >>>> be better to request a backport of Joe's change at the same time. >>>> >>>> -Alan. > > From daniel.fuchs at oracle.com Tue Feb 18 11:39:30 2014 From: daniel.fuchs at oracle.com (Daniel Fuchs) Date: Tue, 18 Feb 2014 12:39:30 +0100 Subject: RFR (JAXP): 8033980 : Xerces Update: datatype XMLGregorianCalendarImpl and DurationImpl In-Reply-To: <53028A85.3010807@oracle.com> References: <52F9666F.9010303@oracle.com> <5111B487-22C1-4987-B9CC-ABF2F5CDD964@oracle.com> <52FA5855.3020106@oracle.com> <52FA7270.4090709@oracle.com> <52FAF801.5060300@oracle.com> <52FBCF66.8060308@oracle.com> <52FC7FE6.2060400@oracle.com> <52FCD57B.8090807@oracle.com> <52FE53C9.3010205@oracle.com> <53023790.4000708@oracle.com> <53028A85.3010807@oracle.com> Message-ID: <53034672.6030701@oracle.com> On 2/17/14 11:17 PM, huizhe wang wrote: > Thanks Daniel. > > I updated the test to call class.getResourceAsStream (see fromFile > method) instead at the runtime. The filePath now is only used for the > helper method to generate test file. Also added readObject() to > XMLGregorianCalendarImpl (at the bottom). The method calls the save() > method to initialize the orig_* fields as done in the constructors. > > http://cr.openjdk.java.net/~joehw/jdk9/8033980/webrev/ Hi Joe, The changes look good. In the serialization test I wonder if it would be better to simply replace: 53 { 54 filePath = SerializationTest.class.getResource("SerializationTest.java").getPath(); 55 int pos = filePath.lastIndexOf('/'); 56 filePath = filePath.substring(0, pos + 1); 57 } with: { filePath = System.getProperty("test.src", "."); } best regards, -- daniel > > Thanks, > Joe > > On 2/17/2014 8:23 AM, Daniel Fuchs wrote: >> Hi Joe, >> >> Sorry for the late reply... >> >> On 2/14/14 6:35 PM, huizhe wang wrote: >>> Hi All, >>> >>> I added a SerializationTest. The test contains a helper that can >>> generate serialization files for XMLGregorianCalendar and Duration. I've >>> created such files for jdk6,7,8 and 9, and manually run the test, that >>> is, read them back with JDK6, 7, 8 and 9. The test worked fine. In the >>> JDK9(or 10 in the future) repo and for an auto-run, it would use the >>> current JDK9/10 build and test against JDK6, 7, 8 and 9. Past JDK10, we >>> could consider add serialization files for JDK10. >>> >>> The new fields did not affect serialization compatibility. The above >>> tests passed with/without the new fields being transient. But I added >>> transient since it's the right thing to do. >>> >>> Adding fields is a compatible change in accordance with Java Object >>> Serialization Spec >>> . >>> >> >> Concerning the new test I think it would be much more >> reliable to open the .ser files as a resource, using >> something like: >> >> InputStream streamIn = SerializationTest.class.getResourceAsStream( >> javaVersion + FILENAME_CAL); >> ObjectInputStream objectinputstream = new ObjectInputStream(streamIn); >> >> The reason is that URL.getPath() is an URL path - not a file path, so it >> can contain encoded characters (such as %20) that FileInputStream will >> not understand. I suspect that if there's a white space or some other >> special character in the path somewhere your current mechanism >> will not work. >> >> Also it would be good to verify what is the value of the transient >> fields after deserialization: I see that XMLGregorianCalendarImpl >> initializes them with non-default values - and you may need to >> write a readObject() to reinitialize those fields to these non >> default values: the deserialization framework may simply leave >> them with the global default for fields of that type (null >> for objects, 0 for numerical values, false for boolean etc...) >> [I am not 100% sure but it will not hurt to check] >> >> best regards, >> >> -- daniel >> >>> >>> >>> Thanks, >>> Joe >>> >>> On 2/13/2014 6:23 AM, Alan Bateman wrote: >>>> On 13/02/2014 08:18, huizhe wang wrote: >>>>> Hi Alan, Lance, and Daniel, >>>>> >>>>> The Xerces serialization revision meant to create a serialization >>>>> form that would help maintain future serialization compatibility. But >>>>> in reality it itself is causing significant incompatibility as Alan >>>>> pointed out below and we discussed previously. I've removed the >>>>> revision from the patch as a result. >>>>> >>>>> Please see the new webrev here: >>>>> http://cr.openjdk.java.net/~joehw/jdk9/8033980/webrev/ >>>> Thanks for dropping the serialization change as it was just not going >>>> to work the way you had intended. >>>> >>>> I agree with Daniel's comment about all the new fields added to >>>> XMLGregorianCalendarImpl as it's not clear why they aren't transient. >>>> >>>> I have not studied the rest of the changes but I think Daniel and >>>> Lance are. >>>> >>>> -Alan >>> >> > From fweimer at redhat.com Tue Feb 18 12:45:54 2014 From: fweimer at redhat.com (Florian Weimer) Date: Tue, 18 Feb 2014 13:45:54 +0100 Subject: Minor com.sun.jndi.dns cleanup In-Reply-To: <53027C7B.2060506@oracle.com> References: <53022E09.2070002@redhat.com> <5302374C.4090409@redhat.com> <53026744.4030500@oracle.com> <53026A88.9030107@redhat.com> <53027C7B.2060506@oracle.com> Message-ID: <53035602.6080504@redhat.com> On 02/17/2014 10:17 PM, Alan Bateman wrote: > On 17/02/2014 20:01, Florian Weimer wrote: >> In the DnsName case, the same exception is used for parsing >> user-supplied strings and data from the wire, and strictly speaking, >> InvalidNameException should be used only in the former case. > Yes, I think the exceptions in DnsName should be re-examined too > (doesn't have to be this patch of course). For RecordRecord then it > would be good to see if there is a better NamingException, the closest > seems to be CommunicationException with an appropriate cause but maybe > there are more suitable choices. Thanks for the bug number and these suggestions. The new version is here: I had to adjust the checked exceptions, so I wrapped the InvalidNameException from DnsName in a CommunicationException as well. I eliminated the recursion and from the name parser and added yet another check for invalid label types. (Other implementations treat extended label types as errors as well, so there is no need to implement them here.) Is this code actually used? (I discovered it because I looked at DatagramSocket users in the JDK.) If it is, there are some other fixes besides the ArrayIndexOutOfBoundsException handling that would make sense. -- Florian Weimer / Red Hat Product Security Team From huizhe.wang at oracle.com Tue Feb 18 17:42:37 2014 From: huizhe.wang at oracle.com (huizhe wang) Date: Tue, 18 Feb 2014 09:42:37 -0800 Subject: RFR (JAXP): 8033980 : Xerces Update: datatype XMLGregorianCalendarImpl and DurationImpl In-Reply-To: <53034672.6030701@oracle.com> References: <52F9666F.9010303@oracle.com> <5111B487-22C1-4987-B9CC-ABF2F5CDD964@oracle.com> <52FA5855.3020106@oracle.com> <52FA7270.4090709@oracle.com> <52FAF801.5060300@oracle.com> <52FBCF66.8060308@oracle.com> <52FC7FE6.2060400@oracle.com> <52FCD57B.8090807@oracle.com> <52FE53C9.3010205@oracle.com> <53023790.4000708@oracle.com> <53028A85.3010807@oracle.com> <53034672.6030701@oracle.com> Message-ID: <53039B8D.7060606@oracle.com> On 2/18/2014 3:39 AM, Daniel Fuchs wrote: > On 2/17/14 11:17 PM, huizhe wang wrote: >> Thanks Daniel. >> >> I updated the test to call class.getResourceAsStream (see fromFile >> method) instead at the runtime. The filePath now is only used for the >> helper method to generate test file. Also added readObject() to >> XMLGregorianCalendarImpl (at the bottom). The method calls the save() >> method to initialize the orig_* fields as done in the constructors. >> >> http://cr.openjdk.java.net/~joehw/jdk9/8033980/webrev/ > > Hi Joe, > > The changes look good. > > In the serialization test I wonder if it would be better > to simply replace: > > 53 { > 54 filePath = > SerializationTest.class.getResource("SerializationTest.java").getPath(); > 55 int pos = filePath.lastIndexOf('/'); > 56 filePath = filePath.substring(0, pos + 1); > 57 } > > with: > > { > filePath = System.getProperty("test.src", "."); > } Ok, I changed that to: 54 filePath = System.getProperty("test.src"); 55 if (filePath == null) { 56 //current directory 57 filePath = System.getProperty("user.dir"); 58 } 59 filePath += File.separator; Thanks, Joe > > best regards, > > -- daniel > >> >> Thanks, >> Joe >> >> On 2/17/2014 8:23 AM, Daniel Fuchs wrote: >>> Hi Joe, >>> >>> Sorry for the late reply... >>> >>> On 2/14/14 6:35 PM, huizhe wang wrote: >>>> Hi All, >>>> >>>> I added a SerializationTest. The test contains a helper that can >>>> generate serialization files for XMLGregorianCalendar and Duration. >>>> I've >>>> created such files for jdk6,7,8 and 9, and manually run the test, that >>>> is, read them back with JDK6, 7, 8 and 9. The test worked fine. In the >>>> JDK9(or 10 in the future) repo and for an auto-run, it would use the >>>> current JDK9/10 build and test against JDK6, 7, 8 and 9. Past >>>> JDK10, we >>>> could consider add serialization files for JDK10. >>>> >>>> The new fields did not affect serialization compatibility. The above >>>> tests passed with/without the new fields being transient. But I added >>>> transient since it's the right thing to do. >>>> >>>> Adding fields is a compatible change in accordance with Java Object >>>> Serialization Spec >>>> . >>>> >>>> >>> >>> Concerning the new test I think it would be much more >>> reliable to open the .ser files as a resource, using >>> something like: >>> >>> InputStream streamIn = SerializationTest.class.getResourceAsStream( >>> javaVersion + FILENAME_CAL); >>> ObjectInputStream objectinputstream = new >>> ObjectInputStream(streamIn); >>> >>> The reason is that URL.getPath() is an URL path - not a file path, >>> so it >>> can contain encoded characters (such as %20) that FileInputStream will >>> not understand. I suspect that if there's a white space or some other >>> special character in the path somewhere your current mechanism >>> will not work. >>> >>> Also it would be good to verify what is the value of the transient >>> fields after deserialization: I see that XMLGregorianCalendarImpl >>> initializes them with non-default values - and you may need to >>> write a readObject() to reinitialize those fields to these non >>> default values: the deserialization framework may simply leave >>> them with the global default for fields of that type (null >>> for objects, 0 for numerical values, false for boolean etc...) >>> [I am not 100% sure but it will not hurt to check] >>> >>> best regards, >>> >>> -- daniel >>> >>>> >>>> >>>> Thanks, >>>> Joe >>>> >>>> On 2/13/2014 6:23 AM, Alan Bateman wrote: >>>>> On 13/02/2014 08:18, huizhe wang wrote: >>>>>> Hi Alan, Lance, and Daniel, >>>>>> >>>>>> The Xerces serialization revision meant to create a serialization >>>>>> form that would help maintain future serialization compatibility. >>>>>> But >>>>>> in reality it itself is causing significant incompatibility as Alan >>>>>> pointed out below and we discussed previously. I've removed the >>>>>> revision from the patch as a result. >>>>>> >>>>>> Please see the new webrev here: >>>>>> http://cr.openjdk.java.net/~joehw/jdk9/8033980/webrev/ >>>>> Thanks for dropping the serialization change as it was just not going >>>>> to work the way you had intended. >>>>> >>>>> I agree with Daniel's comment about all the new fields added to >>>>> XMLGregorianCalendarImpl as it's not clear why they aren't transient. >>>>> >>>>> I have not studied the rest of the changes but I think Daniel and >>>>> Lance are. >>>>> >>>>> -Alan >>>> >>> >> > From huizhe.wang at oracle.com Tue Feb 18 17:48:13 2014 From: huizhe.wang at oracle.com (huizhe wang) Date: Tue, 18 Feb 2014 09:48:13 -0800 Subject: RFR(S): 8034087: XML parser may overwrite element content if that content falls onto the border of an entity scanner buffer In-Reply-To: References: <52FA535D.70804@oracle.com> <52FA9CC5.3010104@oracle.com> <61C6FA0C43817B4FA6EB52E901E137E7729A7913@DEWDFEMB18B.global.corp.sap> <52FBBCB7.9010907@oracle.com> Message-ID: <53039CDD.7000504@oracle.com> Looks good to me. Thanks, Joe On 2/18/2014 1:43 AM, Volker Simonis wrote: > Hi, > > I've moved the test to test/javax/xml/jaxp/parsers/8027359 as requested: > > http://cr.openjdk.java.net/~simonis/webrevs/8034087_1/ > > Can I please get a review for this change. > > Thank you and best regards, > Volker > > > On Wed, Feb 12, 2014 at 7:25 PM, huizhe wang wrote: >> Ok, understand. >> >> Thanks, >> Joe >> >> >> On 2/11/2014 11:40 PM, Schreiber, Steffen wrote: >>> Hi Joe, >>> >>> Yes, that's certainly true, if the test is only to show the issue. >>> We intended the test to work as a regression test as well and wanted to >>> check the effects of the buffer boundaries on more possible places, i.e. >>> before/at/after special characters like <, > or / as well as inside tags and >>> content. >>> >>> We can easily adapt the loop, if runtime is a concern here. >>> >>> Regards, >>> Steffen >>> >>> -----Original Message----- >>> From: huizhe wang [mailto:huizhe.wang at oracle.com] >>> Sent: Dienstag, 11. Februar 2014 22:57 >>> To: Volker Simonis >>> Cc: Alan Bateman; Schreiber, Steffen; Java Core Libs >>> Subject: Re: RFR(S): 8034087: XML parser may overwrite element content if >>> that content falls onto the border of an entity scanner buffer >>> >>> Hi Volker, >>> >>> I agree with the approach below and jdk9/dev is the better forest. >>> >>> For the test itself, I would suggest reducing the following loop to 1 or >>> 2 cases: >>> >>> for (int i = 0; i < testString.length(); i++) { >>> test(createDocument(testString.toString(), i), ""+ i); >>> } >>> >>> >>> when i=7, the problem starts to show. It's sufficient to demonstrate the >>> issue then by just entering 7. It's unnecessary to run the test 43 times. >>> >>> >>> Thanks, >>> Joe >>> >>> On 2/11/2014 9:00 AM, Volker Simonis wrote: >>>> Hi Alan, >>>> >>>> you're right. I initially didn't saw the test because I just looked at >>>> the change in the jaxp repository. >>>> >>>> If it will be approved, I'll put the test in the same directory like >>>> the other test (i.e. test/javax/xml/jaxp/parsers/8027359). >>>> >>>> And yes, my plan was to get approval for both, the tests and the fix, >>>> when asking for the permission to downport to jdk8u-dev and jdk7u-dev. >>>> >>>> Thanks, >>>> Volker >>>> >>>> >>>> On Tue, Feb 11, 2014 at 5:44 PM, Alan Bateman >>>> wrote: >>>>> On 11/02/2014 14:57, Volker Simonis wrote: >>>>>> Hi, >>>>>> >>>>>> after opening this bug yesterday for an issue found by my colleague >>>>>> Steffen Schreiber we realized that this is actually a duplicate of >>>>>> "8027359: XML parser returns incorrect parsing results" >>>>>> (https://bugs.openjdk.java.net/browse/JDK-8027359). >>>>>> >>>>>> While there's no need now to submit a patch anymore, we'd >>>>>> nevertheless like to contribute at least our test case for this issue: >>>>>> >>>>>> http://cr.openjdk.java.net/~simonis/webrevs/8034087/ >>>>>> >>>>>> The webrev is against jdk9-client but we'd like to also downport this >>>>>> test to jdk7 and jdk8 to track that the fix for 8027359 will be >>>>>> correctly downported to these releases as well. >>>>>> >>>>>> I will sponsor this change if somebody would be so kind to review it. >>>>>> >>>>> I'll leave it to Joe Wang to comment on the test but just to mention >>>>> that >>>>> jdk9/dev is probably a better forest to aim for because that is where >>>>> the >>>>> XML (and its tests) usually go. Also I wonder if it might be better to >>>>> put >>>>> it in the same directory as the test that Joe pushed with the change? >>>>> >>>>> If you are getting approval to push to jdk8u-dev and jdk7u-dev then it >>>>> might >>>>> be better to request a backport of Joe's change at the same time. >>>>> >>>>> -Alan. >> From daniel.fuchs at oracle.com Tue Feb 18 18:05:54 2014 From: daniel.fuchs at oracle.com (Daniel Fuchs) Date: Tue, 18 Feb 2014 19:05:54 +0100 Subject: RFR (JAXP): 8033980 : Xerces Update: datatype XMLGregorianCalendarImpl and DurationImpl In-Reply-To: <53039B8D.7060606@oracle.com> References: <52F9666F.9010303@oracle.com> <5111B487-22C1-4987-B9CC-ABF2F5CDD964@oracle.com> <52FA5855.3020106@oracle.com> <52FA7270.4090709@oracle.com> <52FAF801.5060300@oracle.com> <52FBCF66.8060308@oracle.com> <52FC7FE6.2060400@oracle.com> <52FCD57B.8090807@oracle.com> <52FE53C9.3010205@oracle.com> <53023790.4000708@oracle.com> <53028A85.3010807@oracle.com> <53034672.6030701@oracle.com> <53039B8D.7060606@oracle.com> Message-ID: <5303A102.4000406@oracle.com> Thanks Joe, +1 There's a small alignment issue in SerializationTest.java line 152. Also the if (condition) { } else { } formatting in the source files is bizarre, but if it can help with future merges I have no issues. No need to regenerate a webrev. -- daniel On 2/18/14 6:42 PM, huizhe wang wrote: > > On 2/18/2014 3:39 AM, Daniel Fuchs wrote: >> On 2/17/14 11:17 PM, huizhe wang wrote: >>> Thanks Daniel. >>> >>> I updated the test to call class.getResourceAsStream (see fromFile >>> method) instead at the runtime. The filePath now is only used for the >>> helper method to generate test file. Also added readObject() to >>> XMLGregorianCalendarImpl (at the bottom). The method calls the save() >>> method to initialize the orig_* fields as done in the constructors. >>> >>> http://cr.openjdk.java.net/~joehw/jdk9/8033980/webrev/ >> >> Hi Joe, >> >> The changes look good. >> >> In the serialization test I wonder if it would be better >> to simply replace: >> >> 53 { >> 54 filePath = >> SerializationTest.class.getResource("SerializationTest.java").getPath(); >> 55 int pos = filePath.lastIndexOf('/'); >> 56 filePath = filePath.substring(0, pos + 1); >> 57 } >> >> with: >> >> { >> filePath = System.getProperty("test.src", "."); >> } > > Ok, I changed that to: > > 54 filePath = System.getProperty("test.src"); > 55 if (filePath == null) { > 56 //current directory > 57 filePath = System.getProperty("user.dir"); > 58 } > 59 filePath += File.separator; > > > Thanks, > Joe > >> >> best regards, >> >> -- daniel >> >>> >>> Thanks, >>> Joe >>> >>> On 2/17/2014 8:23 AM, Daniel Fuchs wrote: >>>> Hi Joe, >>>> >>>> Sorry for the late reply... >>>> >>>> On 2/14/14 6:35 PM, huizhe wang wrote: >>>>> Hi All, >>>>> >>>>> I added a SerializationTest. The test contains a helper that can >>>>> generate serialization files for XMLGregorianCalendar and Duration. >>>>> I've >>>>> created such files for jdk6,7,8 and 9, and manually run the test, that >>>>> is, read them back with JDK6, 7, 8 and 9. The test worked fine. In the >>>>> JDK9(or 10 in the future) repo and for an auto-run, it would use the >>>>> current JDK9/10 build and test against JDK6, 7, 8 and 9. Past >>>>> JDK10, we >>>>> could consider add serialization files for JDK10. >>>>> >>>>> The new fields did not affect serialization compatibility. The above >>>>> tests passed with/without the new fields being transient. But I added >>>>> transient since it's the right thing to do. >>>>> >>>>> Adding fields is a compatible change in accordance with Java Object >>>>> Serialization Spec >>>>> . >>>>> >>>>> >>>> >>>> Concerning the new test I think it would be much more >>>> reliable to open the .ser files as a resource, using >>>> something like: >>>> >>>> InputStream streamIn = SerializationTest.class.getResourceAsStream( >>>> javaVersion + FILENAME_CAL); >>>> ObjectInputStream objectinputstream = new >>>> ObjectInputStream(streamIn); >>>> >>>> The reason is that URL.getPath() is an URL path - not a file path, >>>> so it >>>> can contain encoded characters (such as %20) that FileInputStream will >>>> not understand. I suspect that if there's a white space or some other >>>> special character in the path somewhere your current mechanism >>>> will not work. >>>> >>>> Also it would be good to verify what is the value of the transient >>>> fields after deserialization: I see that XMLGregorianCalendarImpl >>>> initializes them with non-default values - and you may need to >>>> write a readObject() to reinitialize those fields to these non >>>> default values: the deserialization framework may simply leave >>>> them with the global default for fields of that type (null >>>> for objects, 0 for numerical values, false for boolean etc...) >>>> [I am not 100% sure but it will not hurt to check] >>>> >>>> best regards, >>>> >>>> -- daniel >>>> >>>>> >>>>> >>>>> Thanks, >>>>> Joe >>>>> >>>>> On 2/13/2014 6:23 AM, Alan Bateman wrote: >>>>>> On 13/02/2014 08:18, huizhe wang wrote: >>>>>>> Hi Alan, Lance, and Daniel, >>>>>>> >>>>>>> The Xerces serialization revision meant to create a serialization >>>>>>> form that would help maintain future serialization compatibility. >>>>>>> But >>>>>>> in reality it itself is causing significant incompatibility as Alan >>>>>>> pointed out below and we discussed previously. I've removed the >>>>>>> revision from the patch as a result. >>>>>>> >>>>>>> Please see the new webrev here: >>>>>>> http://cr.openjdk.java.net/~joehw/jdk9/8033980/webrev/ >>>>>> Thanks for dropping the serialization change as it was just not going >>>>>> to work the way you had intended. >>>>>> >>>>>> I agree with Daniel's comment about all the new fields added to >>>>>> XMLGregorianCalendarImpl as it's not clear why they aren't transient. >>>>>> >>>>>> I have not studied the rest of the changes but I think Daniel and >>>>>> Lance are. >>>>>> >>>>>> -Alan >>>>> >>>> >>> >> > From huizhe.wang at oracle.com Tue Feb 18 18:30:32 2014 From: huizhe.wang at oracle.com (huizhe wang) Date: Tue, 18 Feb 2014 10:30:32 -0800 Subject: RFR (JAXP): 8033980 : Xerces Update: datatype XMLGregorianCalendarImpl and DurationImpl In-Reply-To: <5303A102.4000406@oracle.com> References: <52F9666F.9010303@oracle.com> <5111B487-22C1-4987-B9CC-ABF2F5CDD964@oracle.com> <52FA5855.3020106@oracle.com> <52FA7270.4090709@oracle.com> <52FAF801.5060300@oracle.com> <52FBCF66.8060308@oracle.com> <52FC7FE6.2060400@oracle.com> <52FCD57B.8090807@oracle.com> <52FE53C9.3010205@oracle.com> <53023790.4000708@oracle.com> <53028A85.3010807@oracle.com> <53034672.6030701@oracle.com> <53039B8D.7060606@oracle.com> <5303A102.4000406@oracle.com> Message-ID: <5303A6C8.4010907@oracle.com> On 2/18/2014 10:05 AM, Daniel Fuchs wrote: > Thanks Joe, > > +1 > > There's a small alignment issue in SerializationTest.java line 152. That's a debug I added a moment ago, removed. > > Also the > > if (condition) { > } > else { > } > > formatting in the source files is bizarre, but if it can help with > future merges I have no issues. That's from Xerces (they use Eclipse). I've seen them do this in pure format changesets, that is, they'd change "} else {" to the above. NetBeans format would have been: if (condition) { } else { } I personally like The NetBeans format. But it will take a while before we could get jaxp code formatted properly since in many cases the relevant files are very big (over 3000 lines in this case). Hitting NetBeans format can generate a lot of noises to the webrev. > > No need to regenerate a webrev. Thanks, Joe > > -- daniel > > On 2/18/14 6:42 PM, huizhe wang wrote: >> >> On 2/18/2014 3:39 AM, Daniel Fuchs wrote: >>> On 2/17/14 11:17 PM, huizhe wang wrote: >>>> Thanks Daniel. >>>> >>>> I updated the test to call class.getResourceAsStream (see fromFile >>>> method) instead at the runtime. The filePath now is only used for the >>>> helper method to generate test file. Also added readObject() to >>>> XMLGregorianCalendarImpl (at the bottom). The method calls the save() >>>> method to initialize the orig_* fields as done in the constructors. >>>> >>>> http://cr.openjdk.java.net/~joehw/jdk9/8033980/webrev/ >>> >>> Hi Joe, >>> >>> The changes look good. >>> >>> In the serialization test I wonder if it would be better >>> to simply replace: >>> >>> 53 { >>> 54 filePath = >>> SerializationTest.class.getResource("SerializationTest.java").getPath(); >>> >>> 55 int pos = filePath.lastIndexOf('/'); >>> 56 filePath = filePath.substring(0, pos + 1); >>> 57 } >>> >>> with: >>> >>> { >>> filePath = System.getProperty("test.src", "."); >>> } >> >> Ok, I changed that to: >> >> 54 filePath = System.getProperty("test.src"); >> 55 if (filePath == null) { >> 56 //current directory >> 57 filePath = System.getProperty("user.dir"); >> 58 } >> 59 filePath += File.separator; >> >> >> Thanks, >> Joe >> >>> >>> best regards, >>> >>> -- daniel >>> >>>> >>>> Thanks, >>>> Joe >>>> >>>> On 2/17/2014 8:23 AM, Daniel Fuchs wrote: >>>>> Hi Joe, >>>>> >>>>> Sorry for the late reply... >>>>> >>>>> On 2/14/14 6:35 PM, huizhe wang wrote: >>>>>> Hi All, >>>>>> >>>>>> I added a SerializationTest. The test contains a helper that can >>>>>> generate serialization files for XMLGregorianCalendar and Duration. >>>>>> I've >>>>>> created such files for jdk6,7,8 and 9, and manually run the test, >>>>>> that >>>>>> is, read them back with JDK6, 7, 8 and 9. The test worked fine. >>>>>> In the >>>>>> JDK9(or 10 in the future) repo and for an auto-run, it would use the >>>>>> current JDK9/10 build and test against JDK6, 7, 8 and 9. Past >>>>>> JDK10, we >>>>>> could consider add serialization files for JDK10. >>>>>> >>>>>> The new fields did not affect serialization compatibility. The above >>>>>> tests passed with/without the new fields being transient. But I >>>>>> added >>>>>> transient since it's the right thing to do. >>>>>> >>>>>> Adding fields is a compatible change in accordance with Java Object >>>>>> Serialization Spec >>>>>> . >>>>>> >>>>>> >>>>>> >>>>> >>>>> Concerning the new test I think it would be much more >>>>> reliable to open the .ser files as a resource, using >>>>> something like: >>>>> >>>>> InputStream streamIn = >>>>> SerializationTest.class.getResourceAsStream( >>>>> javaVersion + FILENAME_CAL); >>>>> ObjectInputStream objectinputstream = new >>>>> ObjectInputStream(streamIn); >>>>> >>>>> The reason is that URL.getPath() is an URL path - not a file path, >>>>> so it >>>>> can contain encoded characters (such as %20) that FileInputStream >>>>> will >>>>> not understand. I suspect that if there's a white space or some other >>>>> special character in the path somewhere your current mechanism >>>>> will not work. >>>>> >>>>> Also it would be good to verify what is the value of the transient >>>>> fields after deserialization: I see that XMLGregorianCalendarImpl >>>>> initializes them with non-default values - and you may need to >>>>> write a readObject() to reinitialize those fields to these non >>>>> default values: the deserialization framework may simply leave >>>>> them with the global default for fields of that type (null >>>>> for objects, 0 for numerical values, false for boolean etc...) >>>>> [I am not 100% sure but it will not hurt to check] >>>>> >>>>> best regards, >>>>> >>>>> -- daniel >>>>> >>>>>> >>>>>> >>>>>> Thanks, >>>>>> Joe >>>>>> >>>>>> On 2/13/2014 6:23 AM, Alan Bateman wrote: >>>>>>> On 13/02/2014 08:18, huizhe wang wrote: >>>>>>>> Hi Alan, Lance, and Daniel, >>>>>>>> >>>>>>>> The Xerces serialization revision meant to create a serialization >>>>>>>> form that would help maintain future serialization compatibility. >>>>>>>> But >>>>>>>> in reality it itself is causing significant incompatibility as >>>>>>>> Alan >>>>>>>> pointed out below and we discussed previously. I've removed the >>>>>>>> revision from the patch as a result. >>>>>>>> >>>>>>>> Please see the new webrev here: >>>>>>>> http://cr.openjdk.java.net/~joehw/jdk9/8033980/webrev/ >>>>>>> Thanks for dropping the serialization change as it was just not >>>>>>> going >>>>>>> to work the way you had intended. >>>>>>> >>>>>>> I agree with Daniel's comment about all the new fields added to >>>>>>> XMLGregorianCalendarImpl as it's not clear why they aren't >>>>>>> transient. >>>>>>> >>>>>>> I have not studied the rest of the changes but I think Daniel and >>>>>>> Lance are. >>>>>>> >>>>>>> -Alan >>>>>> >>>>> >>>> >>> >> > From mikael.vidstedt at oracle.com Wed Feb 19 01:30:01 2014 From: mikael.vidstedt at oracle.com (Mikael Vidstedt) Date: Tue, 18 Feb 2014 17:30:01 -0800 Subject: RFR: Even more gcc warnings (8035287) In-Reply-To: <52FFA46F.4000202@oracle.com> References: <52FEC8C7.9000700@oracle.com> <41E8E647-4C1E-4DD6-86AF-40A498B503B7@oracle.com> <52FFA46F.4000202@oracle.com> Message-ID: <53040919.3050400@oracle.com> On 2014-02-15 09:31, Phil Race wrote: > Looks OK to me although I just realised there's no bug ID here Bug created: https://bugs.openjdk.java.net/browse/JDK-8035287 > FWIW I develop on WIndows, Mac & Linux and I've noticed widely divergent > things that the compilers on these platforms warn about. Warning > free on Linux might not mean warning free on Windows. > Also, assuming you develop on Linux might want to check if any of > these make > the VS compiler less happy about anything. Acknowledged - not all platforms/compilers complain about the same thing(s). I tried my best to manually verify that no new warnings are introduced by building on the usual suspect platforms and grep through the warnings. > >> * src/solaris/native/sun/awt/awt_Font.c >> >> Comparisons with string literals is undefined behavior - keep track >> of whether the string should be freed explicitly with a boolean instead. > Gosh .. that code must be from 1996 or thereabouts. I hope touching it doesn't mean I own it ;) Anything else I should do/test? Cheers, Mikael > > -phil. > > On 2/15/14 8:37 AM, Mikael Vidstedt wrote: >> Corrected link - this webrev is based on jdk9/client: >> >> http://cr.openjdk.java.net/~mikael/webrevs/jdk-warnings/webrev.01/webrev/ >> >> >> Cheers, >> Mikael >> >>> On Feb 14, 2014, at 17:54, Mikael Vidstedt >>> wrote: >>> >>> >>> All, >>> >>> A drive-by set of warning fixes: >>> >>> http://cr.openjdk.java.net/~mikael/webrevs/jdk-warnings/jdk-warnings/webrev.00/ >>> >>> >>> Highlights: >>> >>> * src/share/native/com/sun/java/util/jar/pack/bands.cpp >>> >>> Set the size of the array explicitly to increase likelihood of enum >>> and struct array being in sync. Arguably this should be changed to >>> use the (new) [] = instead. >>> >>> Initialize all the fields in the (redundant) terminator struct >>> explicitly. >>> >>> Remove unused macro. >>> >>> * src/share/native/sun/java2d/opengl/OGLContext.c >>> >>> Get the prototype for jio_snprintf from jvm.h to address an implicit >>> declaration. >>> >>> * src/solaris/native/sun/awt/awt_Font.c >>> >>> Comparisons with string literals is undefined behavior - keep track >>> of whether the string should be freed explicitly with a boolean >>> instead. >>> >>> * src/solaris/native/sun/awt/awt_LoadLibrary.c >>> >>> The macro is supposed to expand to a void function declaration, but >>> forgets to actually add the "void". >>> >>> Cheers, >>> Mikael >>> > From philip.race at oracle.com Wed Feb 19 01:37:08 2014 From: philip.race at oracle.com (Phil Race) Date: Tue, 18 Feb 2014 17:37:08 -0800 Subject: RFR: Even more gcc warnings (8035287) In-Reply-To: <53040919.3050400@oracle.com> References: <52FEC8C7.9000700@oracle.com> <41E8E647-4C1E-4DD6-86AF-40A498B503B7@oracle.com> <52FFA46F.4000202@oracle.com> <53040919.3050400@oracle.com> Message-ID: <53040AC4.2020403@oracle.com> > Anything else I should do/test? No. Sounds fine. -phil. On 2/18/14 5:30 PM, Mikael Vidstedt wrote: > > On 2014-02-15 09:31, Phil Race wrote: >> Looks OK to me although I just realised there's no bug ID here > > Bug created: > > https://bugs.openjdk.java.net/browse/JDK-8035287 > >> FWIW I develop on WIndows, Mac & Linux and I've noticed widely divergent >> things that the compilers on these platforms warn about. Warning >> free on Linux might not mean warning free on Windows. >> Also, assuming you develop on Linux might want to check if any of >> these make >> the VS compiler less happy about anything. > > Acknowledged - not all platforms/compilers complain about the same > thing(s). I tried my best to manually verify that no new warnings are > introduced by building on the usual suspect platforms and grep through > the warnings. > >> >>> * src/solaris/native/sun/awt/awt_Font.c >>> >>> Comparisons with string literals is undefined behavior - keep track >>> of whether the string should be freed explicitly with a boolean >>> instead. >> Gosh .. that code must be from 1996 or thereabouts. > > I hope touching it doesn't mean I own it ;) > > > Anything else I should do/test? > > Cheers, > Mikael > >> >> -phil. >> >> On 2/15/14 8:37 AM, Mikael Vidstedt wrote: >>> Corrected link - this webrev is based on jdk9/client: >>> >>> http://cr.openjdk.java.net/~mikael/webrevs/jdk-warnings/webrev.01/webrev/ >>> >>> >>> Cheers, >>> Mikael >>> >>>> On Feb 14, 2014, at 17:54, Mikael Vidstedt >>>> wrote: >>>> >>>> >>>> All, >>>> >>>> A drive-by set of warning fixes: >>>> >>>> http://cr.openjdk.java.net/~mikael/webrevs/jdk-warnings/jdk-warnings/webrev.00/ >>>> >>>> >>>> Highlights: >>>> >>>> * src/share/native/com/sun/java/util/jar/pack/bands.cpp >>>> >>>> Set the size of the array explicitly to increase likelihood of enum >>>> and struct array being in sync. Arguably this should be changed to >>>> use the (new) [] = instead. >>>> >>>> Initialize all the fields in the (redundant) terminator struct >>>> explicitly. >>>> >>>> Remove unused macro. >>>> >>>> * src/share/native/sun/java2d/opengl/OGLContext.c >>>> >>>> Get the prototype for jio_snprintf from jvm.h to address an >>>> implicit declaration. >>>> >>>> * src/solaris/native/sun/awt/awt_Font.c >>>> >>>> Comparisons with string literals is undefined behavior - keep track >>>> of whether the string should be freed explicitly with a boolean >>>> instead. >>>> >>>> * src/solaris/native/sun/awt/awt_LoadLibrary.c >>>> >>>> The macro is supposed to expand to a void function declaration, but >>>> forgets to actually add the "void". >>>> >>>> Cheers, >>>> Mikael >>>> >> > From mikael.vidstedt at oracle.com Wed Feb 19 01:57:18 2014 From: mikael.vidstedt at oracle.com (Mikael Vidstedt) Date: Tue, 18 Feb 2014 17:57:18 -0800 Subject: RFR: Even more gcc warnings (8035287) In-Reply-To: <53040AC4.2020403@oracle.com> References: <52FEC8C7.9000700@oracle.com> <41E8E647-4C1E-4DD6-86AF-40A498B503B7@oracle.com> <52FFA46F.4000202@oracle.com> <53040919.3050400@oracle.com> <53040AC4.2020403@oracle.com> Message-ID: <53040F7E.1060808@oracle.com> Thanks for the review Phil! Cheers, Mikael On 2014-02-18 17:37, Phil Race wrote: > > Anything else I should do/test? > > No. Sounds fine. > > -phil. > > On 2/18/14 5:30 PM, Mikael Vidstedt wrote: >> >> On 2014-02-15 09:31, Phil Race wrote: >>> Looks OK to me although I just realised there's no bug ID here >> >> Bug created: >> >> https://bugs.openjdk.java.net/browse/JDK-8035287 >> >>> FWIW I develop on WIndows, Mac & Linux and I've noticed widely >>> divergent >>> things that the compilers on these platforms warn about. Warning >>> free on Linux might not mean warning free on Windows. >>> Also, assuming you develop on Linux might want to check if any of >>> these make >>> the VS compiler less happy about anything. >> >> Acknowledged - not all platforms/compilers complain about the same >> thing(s). I tried my best to manually verify that no new warnings are >> introduced by building on the usual suspect platforms and grep >> through the warnings. >> >>> >>>> * src/solaris/native/sun/awt/awt_Font.c >>>> >>>> Comparisons with string literals is undefined behavior - keep track >>>> of whether the string should be freed explicitly with a boolean >>>> instead. >>> Gosh .. that code must be from 1996 or thereabouts. >> >> I hope touching it doesn't mean I own it ;) >> >> >> Anything else I should do/test? >> >> Cheers, >> Mikael >> >>> >>> -phil. >>> >>> On 2/15/14 8:37 AM, Mikael Vidstedt wrote: >>>> Corrected link - this webrev is based on jdk9/client: >>>> >>>> http://cr.openjdk.java.net/~mikael/webrevs/jdk-warnings/webrev.01/webrev/ >>>> >>>> >>>> Cheers, >>>> Mikael >>>> >>>>> On Feb 14, 2014, at 17:54, Mikael Vidstedt >>>>> wrote: >>>>> >>>>> >>>>> All, >>>>> >>>>> A drive-by set of warning fixes: >>>>> >>>>> http://cr.openjdk.java.net/~mikael/webrevs/jdk-warnings/jdk-warnings/webrev.00/ >>>>> >>>>> >>>>> Highlights: >>>>> >>>>> * src/share/native/com/sun/java/util/jar/pack/bands.cpp >>>>> >>>>> Set the size of the array explicitly to increase likelihood of >>>>> enum and struct array being in sync. Arguably this should be >>>>> changed to use the (new) [] = instead. >>>>> >>>>> Initialize all the fields in the (redundant) terminator struct >>>>> explicitly. >>>>> >>>>> Remove unused macro. >>>>> >>>>> * src/share/native/sun/java2d/opengl/OGLContext.c >>>>> >>>>> Get the prototype for jio_snprintf from jvm.h to address an >>>>> implicit declaration. >>>>> >>>>> * src/solaris/native/sun/awt/awt_Font.c >>>>> >>>>> Comparisons with string literals is undefined behavior - keep >>>>> track of whether the string should be freed explicitly with a >>>>> boolean instead. >>>>> >>>>> * src/solaris/native/sun/awt/awt_LoadLibrary.c >>>>> >>>>> The macro is supposed to expand to a void function declaration, >>>>> but forgets to actually add the "void". >>>>> >>>>> Cheers, >>>>> Mikael >>>>> >>> >> > From stuart.marks at oracle.com Wed Feb 19 02:47:49 2014 From: stuart.marks at oracle.com (Stuart Marks) Date: Tue, 18 Feb 2014 18:47:49 -0800 Subject: RFR(s): 8034999 change rmidRunning to a simple lookup (RMI test library) Message-ID: <53041B55.20904@oracle.com> Hi all, Please review this change to remove a redundant timing-retry loop from the rmidRunning() routine of the RMI test library. It is replaced with a simple rmid registry lookup that returns status immediately, without retries. Bug: https://bugs.openjdk.java.net/browse/JDK-8034999 Webrev: http://cr.openjdk.java.net/~smarks/reviews/8034999/webrev.0/ Thanks, s'marks From scolebourne at joda.org Wed Feb 19 10:54:10 2014 From: scolebourne at joda.org (Stephen Colebourne) Date: Wed, 19 Feb 2014 10:54:10 +0000 Subject: RFR: JDK-8034906 Fix typos, errors and Javadoc differences in java.time Message-ID: A set of minor wording fixes in Javadoc: https://gist.github.com/jodastephen/8984256 Comments welcome. Stephen From david.holmes at oracle.com Wed Feb 19 11:08:07 2014 From: david.holmes at oracle.com (David Holmes) Date: Wed, 19 Feb 2014 21:08:07 +1000 Subject: RFR: JDK-8034906 Fix typos, errors and Javadoc differences in java.time In-Reply-To: References: Message-ID: <53049097.1080005@oracle.com> Hi Stephen, This could be construed as a spec-change, even if a typo :( - * which are too large to fit in an {@code int} and throw a {@code DateTimeException}. + * which are too large to fit in an {@code int} and throw an {@code UnsupportedTemporalTypeException}. David On 19/02/2014 8:54 PM, Stephen Colebourne wrote: > A set of minor wording fixes in Javadoc: > > https://gist.github.com/jodastephen/8984256 > > Comments welcome. > Stephen > From scolebourne at joda.org Wed Feb 19 11:29:56 2014 From: scolebourne at joda.org (Stephen Colebourne) Date: Wed, 19 Feb 2014 11:29:56 +0000 Subject: RFR: JDK-8034906 Fix typos, errors and Javadoc differences in java.time In-Reply-To: <53049097.1080005@oracle.com> References: <53049097.1080005@oracle.com> Message-ID: Could or would? Who decides... Stephen On 19 February 2014 11:08, David Holmes wrote: > Hi Stephen, > > This could be construed as a spec-change, even if a typo :( > > - * which are too large to fit in an {@code int} and throw a {@code > DateTimeException}. > + * which are too large to fit in an {@code int} and throw an {@code > UnsupportedTemporalTypeException}. > > David > > > On 19/02/2014 8:54 PM, Stephen Colebourne wrote: >> >> A set of minor wording fixes in Javadoc: >> >> https://gist.github.com/jodastephen/8984256 >> >> Comments welcome. >> Stephen >> > From Alan.Bateman at oracle.com Wed Feb 19 11:39:23 2014 From: Alan.Bateman at oracle.com (Alan Bateman) Date: Wed, 19 Feb 2014 11:39:23 +0000 Subject: RFR: JDK-8034906 Fix typos, errors and Javadoc differences in java.time In-Reply-To: <53049097.1080005@oracle.com> References: <53049097.1080005@oracle.com> Message-ID: <530497EB.6090308@oracle.com> On 19/02/2014 11:08, David Holmes wrote: > Hi Stephen, > > This could be construed as a spec-change, even if a typo :( > > - * which are too large to fit in an {@code int} and throw a {@code > DateTimeException}. > + * which are too large to fit in an {@code int} and throw an {@code > UnsupportedTemporalTypeException}. > > David From what I can tell, this is LocalDate.get(TemporalField) and the @throws already declares the specific exception for the out of range case. So I think this one is okay and seems to just adjusting the method description so that it matches declaration (is that right?). -Alan. From david.holmes at oracle.com Wed Feb 19 11:42:30 2014 From: david.holmes at oracle.com (David Holmes) Date: Wed, 19 Feb 2014 21:42:30 +1000 Subject: RFR: JDK-8034906 Fix typos, errors and Javadoc differences in java.time In-Reply-To: References: <53049097.1080005@oracle.com> Message-ID: <530498A6.6040902@oracle.com> On 19/02/2014 9:29 PM, Stephen Colebourne wrote: > Could or would? Who decides... Officially I have no idea. I would think the criteria would be to consider: a) what the code actually does b) whether there is a TCK test for it Though if (b) holds then it must be that the code does what is written rather than what is intended. Conversely if the code does as intended rather than as documented then either there is no test or whomever wrote the test didn't report the error in the documentation. :( In this case I see the code throws the intended exception, so that adds weight to the case that this was a simple typo. But once this has shipped (and JDK 8 is untying from the dock) the distinction may be moot - we have a spec and code that disagree so one must be changed. David > Stephen > > > On 19 February 2014 11:08, David Holmes wrote: >> Hi Stephen, >> >> This could be construed as a spec-change, even if a typo :( >> >> - * which are too large to fit in an {@code int} and throw a {@code >> DateTimeException}. >> + * which are too large to fit in an {@code int} and throw an {@code >> UnsupportedTemporalTypeException}. >> >> David >> >> >> On 19/02/2014 8:54 PM, Stephen Colebourne wrote: >>> >>> A set of minor wording fixes in Javadoc: >>> >>> https://gist.github.com/jodastephen/8984256 >>> >>> Comments welcome. >>> Stephen >>> >> From ivan.gerasimov at oracle.com Wed Feb 19 11:58:58 2014 From: ivan.gerasimov at oracle.com (Ivan Gerasimov) Date: Wed, 19 Feb 2014 15:58:58 +0400 Subject: RFR: JDK-8034906 Fix typos, errors and Javadoc differences in java.time In-Reply-To: References: Message-ID: <53049C82.2000101@oracle.com> Hi Stephen! May I ask you to add some more typo fixes to your patch? Sincerely yours, Ivan diff a/src/share/classes/java/time/OffsetTime.java b/src/share/classes/java/time/OffsetTime.java - * This will result in the old and new objects representing the same instant an an implied day. + * This will result in the old and new objects representing the same instant on an implied day. diff a/src/share/classes/java/time/ZonedDateTime.java b/src/share/classes/java/time/ZonedDateTime.java - * the the date-time and offset of the zoned date-time will differ from those specified. + * the date-time and offset of the zoned date-time will differ from those specified. diff a/src/share/classes/java/time/chrono/Chronology.java b/src/share/classes/java/time/chrono/Chronology.java - * The default implementation behaves as the the formatter was used to + * The default implementation behaves as the formatter was used to diff a/src/share/classes/java/time/format/DateTimeFormatterBuilder.java b/src/share/classes/java/time/format/DateTimeFormatterBuilder.java - * Note that this method is is identical to {@code appendZoneId()} except + * Note that this method is identical to {@code appendZoneId()} except - * Note that this method is is identical to {@code appendZoneId()} except + * Note that this method is identical to {@code appendZoneId()} except diff a/src/share/classes/java/time/temporal/TemporalField.java b/src/share/classes/java/time/temporal/TemporalField.java - * If this returns false, the the temporal cannot be queried for this field. + * If this returns false, the temporal cannot be queried for this field. diff a/src/share/classes/java/time/zone/ZoneOffsetTransition.java b/src/share/classes/java/time/zone/ZoneOffsetTransition.java - * Gaps occur where there are local date-times that simply do not not exist. + * Gaps occur where there are local date-times that simply do not exist. - * Gaps occur where there are local date-times that simply do not not exist. + * Gaps occur where there are local date-times that simply do not exist. From scolebourne at joda.org Wed Feb 19 12:04:09 2014 From: scolebourne at joda.org (Stephen Colebourne) Date: Wed, 19 Feb 2014 12:04:09 +0000 Subject: RFR: JDK-8034906 Fix typos, errors and Javadoc differences in java.time In-Reply-To: <530497EB.6090308@oracle.com> References: <53049097.1080005@oracle.com> <530497EB.6090308@oracle.com> Message-ID: On 19 February 2014 11:39, Alan Bateman wrote: > On 19/02/2014 11:08, David Holmes wrote: >> >> Hi Stephen, >> >> This could be construed as a spec-change, even if a typo :( >> >> - * which are too large to fit in an {@code int} and throw a {@code >> DateTimeException}. >> + * which are too large to fit in an {@code int} and throw an {@code >> UnsupportedTemporalTypeException}. >> >> David > > From what I can tell, this is LocalDate.get(TemporalField) and the @throws > already declares the specific exception for the out of range case. So I > think this one is okay and seems to just adjusting the method description so > that it matches declaration (is that right?). The throws clause says "UnsupportedTemporalTypeException - if the field is not supported or the range of values exceeds an int", so I think this change can be justified under that. The code follows the throws clause. UnsupportedTemporalTypeException extends DateTimeException, so the description is not broken, just less specific than it could be, but the throws clause is correctly specific. Stephen From Alan.Bateman at oracle.com Wed Feb 19 12:05:01 2014 From: Alan.Bateman at oracle.com (Alan Bateman) Date: Wed, 19 Feb 2014 12:05:01 +0000 Subject: Minor com.sun.jndi.dns cleanup In-Reply-To: <53035602.6080504@redhat.com> References: <53022E09.2070002@redhat.com> <5302374C.4090409@redhat.com> <53026744.4030500@oracle.com> <53026A88.9030107@redhat.com> <53027C7B.2060506@oracle.com> <53035602.6080504@redhat.com> Message-ID: <53049DED.2040608@oracle.com> On 18/02/2014 12:45, Florian Weimer wrote: > > Thanks for the bug number and these suggestions. The new version is > here: > > I had to adjust the checked exceptions, so I wrapped the > InvalidNameException from DnsName in a CommunicationException as well. > > I eliminated the recursion and from the name parser and added yet > another check for invalid label types. (Other implementations treat > extended label types as errors as well, so there is no need to > implement them here.) > > Is this code actually used? (I discovered it because I looked at > DatagramSocket users in the JDK.) If it is, there are some other > fixes besides the ArrayIndexOutOfBoundsException handling that would > make sense. > Thanks for the update. It's good to replace the recursion. Do you have the regular label case right? It looks like the type is being used by the length. Also is it necessary to catch InvalidNameException at line 363 as I don't think this is throw anymore. One thing about communication exceptions is that the DNS client will retry. I think this is okay here but potentially it means that we fail with too many compression references a number of times. As to whether the DNS provider is widely used then I don't have data on that. I think most of the JNDI usages are LDAP but probably a small market beyond that. -Alan. From david.holmes at oracle.com Wed Feb 19 12:10:54 2014 From: david.holmes at oracle.com (David Holmes) Date: Wed, 19 Feb 2014 22:10:54 +1000 Subject: RFR: JDK-8034906 Fix typos, errors and Javadoc differences in java.time In-Reply-To: References: <53049097.1080005@oracle.com> <530497EB.6090308@oracle.com> Message-ID: <53049F4E.1090603@oracle.com> On 19/02/2014 10:04 PM, Stephen Colebourne wrote: > On 19 February 2014 11:39, Alan Bateman wrote: >> On 19/02/2014 11:08, David Holmes wrote: >>> >>> Hi Stephen, >>> >>> This could be construed as a spec-change, even if a typo :( >>> >>> - * which are too large to fit in an {@code int} and throw a {@code >>> DateTimeException}. >>> + * which are too large to fit in an {@code int} and throw an {@code >>> UnsupportedTemporalTypeException}. >>> >>> David >> >> From what I can tell, this is LocalDate.get(TemporalField) and the @throws >> already declares the specific exception for the out of range case. So I >> think this one is okay and seems to just adjusting the method description so >> that it matches declaration (is that right?). > > The throws clause says "UnsupportedTemporalTypeException - if the > field is not supported or the range of values exceeds an int", so I > think this change can be justified under that. The code follows the > throws clause. > > UnsupportedTemporalTypeException extends DateTimeException, so the > description is not broken, just less specific than it could be, but > the throws clause is correctly specific. That sounds fine to me. Thanks, David > Stephen > From fweimer at redhat.com Wed Feb 19 12:56:10 2014 From: fweimer at redhat.com (Florian Weimer) Date: Wed, 19 Feb 2014 13:56:10 +0100 Subject: Minor com.sun.jndi.dns cleanup In-Reply-To: <53049DED.2040608@oracle.com> References: <53022E09.2070002@redhat.com> <5302374C.4090409@redhat.com> <53026744.4030500@oracle.com> <53026A88.9030107@redhat.com> <53027C7B.2060506@oracle.com> <53035602.6080504@redhat.com> <53049DED.2040608@oracle.com> Message-ID: <5304A9EA.6020900@redhat.com> On 02/19/2014 01:05 PM, Alan Bateman wrote: > On 18/02/2014 12:45, Florian Weimer wrote: >> >> Thanks for the bug number and these suggestions. The new version is >> here: >> >> I had to adjust the checked exceptions, so I wrapped the >> InvalidNameException from DnsName in a CommunicationException as well. >> >> I eliminated the recursion and from the name parser and added yet >> another check for invalid label types. (Other implementations treat >> extended label types as errors as well, so there is no need to >> implement them here.) >> >> Is this code actually used? (I discovered it because I looked at >> DatagramSocket users in the JDK.) If it is, there are some other >> fixes besides the ArrayIndexOutOfBoundsException handling that would >> make sense. >> > Thanks for the update. > > It's good to replace the recursion. Do you have the regular label case > right? It looks like the type is being used by the length. It's encoded as a bit field, like this: 0000 0000 end of name 00xx xxxx regular label of length xxxxxx 11yy yyyy upper 6 bits of a compression reference (followed by another byte containing the lower 8 bits) 01zz zzzz used to be invalid, then extended type, then invalid again 10zz zzzz invalid I could rename the variable to typeAndLength. > Also is it > necessary to catch InvalidNameException at line 363 as I don't think > this is throw anymore. It is thrown by DNSName#add(int, String). > One thing about communication exceptions is that the DNS client will > retry. I think this is okay here but potentially it means that we fail > with too many compression references a number of times. It's difficult to tell whether the old or new behavior is more appropriate. If the packet is corrupted in transit for some reason, even retrying the same server is helpful. Trying a different server would be reasonable as well. The downside is that if there is a consistent error in the data, then it takes a longer time until the error is reported. -- Florian Weimer / Red Hat Product Security Team From scolebourne at joda.org Wed Feb 19 13:15:12 2014 From: scolebourne at joda.org (Stephen Colebourne) Date: Wed, 19 Feb 2014 13:15:12 +0000 Subject: RFR: JDK-8034906 Fix typos, errors and Javadoc differences in java.time In-Reply-To: <53049C82.2000101@oracle.com> References: <53049C82.2000101@oracle.com> Message-ID: Patch updated: https://gist.github.com/jodastephen/8984256 with differences: https://gist.github.com/jodastephen/8984256/revisions thanks for the spots, Stephen On 19 February 2014 11:58, Ivan Gerasimov wrote: > Hi Stephen! > > May I ask you to add some more typo fixes to your patch? > > Sincerely yours, > Ivan > > > diff a/src/share/classes/java/time/OffsetTime.java > b/src/share/classes/java/time/OffsetTime.java > - * This will result in the old and new objects representing the same > instant an an implied day. > + * This will result in the old and new objects representing the same > instant on an implied day. > diff a/src/share/classes/java/time/ZonedDateTime.java > b/src/share/classes/java/time/ZonedDateTime.java > - * the the date-time and offset of the zoned date-time will differ from > those specified. > + * the date-time and offset of the zoned date-time will differ from > those specified. > diff a/src/share/classes/java/time/chrono/Chronology.java > b/src/share/classes/java/time/chrono/Chronology.java > - * The default implementation behaves as the the formatter was used to > + * The default implementation behaves as the formatter was used to > diff a/src/share/classes/java/time/format/DateTimeFormatterBuilder.java > b/src/share/classes/java/time/format/DateTimeFormatterBuilder.java > - * Note that this method is is identical to {@code appendZoneId()} > except > + * Note that this method is identical to {@code appendZoneId()} except > - * Note that this method is is identical to {@code appendZoneId()} > except > + * Note that this method is identical to {@code appendZoneId()} except > diff a/src/share/classes/java/time/temporal/TemporalField.java > b/src/share/classes/java/time/temporal/TemporalField.java > - * If this returns false, the the temporal cannot be queried for this > field. > + * If this returns false, the temporal cannot be queried for this > field. > diff a/src/share/classes/java/time/zone/ZoneOffsetTransition.java > b/src/share/classes/java/time/zone/ZoneOffsetTransition.java > - * Gaps occur where there are local date-times that simply do not not > exist. > + * Gaps occur where there are local date-times that simply do not exist. > - * Gaps occur where there are local date-times that simply do not not > exist. > + * Gaps occur where there are local date-times that simply do not > exist. > From roger.riggs at oracle.com Wed Feb 19 13:51:12 2014 From: roger.riggs at oracle.com (roger riggs) Date: Wed, 19 Feb 2014 08:51:12 -0500 Subject: RFR: JDK-8034906 Fix typos, errors and Javadoc differences in java.time In-Reply-To: References: Message-ID: <5304B6D0.2040203@oracle.com> Hi, These would be fine for 9 but I would not backport them to 8 because they do not add significant value and do create differences between the specification and the javadoc. Any difference creates an overhead that a person has to resolve even if it turns out to be a no-op. I would update the spec language, *if* it a clarification that is within the spec so that it matched the implementation. Roger On 2/19/2014 5:54 AM, Stephen Colebourne wrote: > A set of minor wording fixes in Javadoc: > > https://gist.github.com/jodastephen/8984256 > > Comments welcome. > Stephen From Alan.Bateman at oracle.com Wed Feb 19 17:43:46 2014 From: Alan.Bateman at oracle.com (Alan Bateman) Date: Wed, 19 Feb 2014 17:43:46 +0000 Subject: Minor com.sun.jndi.dns cleanup In-Reply-To: <5304A9EA.6020900@redhat.com> References: <53022E09.2070002@redhat.com> <5302374C.4090409@redhat.com> <53026744.4030500@oracle.com> <53026A88.9030107@redhat.com> <53027C7B.2060506@oracle.com> <53035602.6080504@redhat.com> <53049DED.2040608@oracle.com> <5304A9EA.6020900@redhat.com> Message-ID: <5304ED52.9080204@oracle.com> On 19/02/2014 12:56, Florian Weimer wrote: > > It's encoded as a bit field, like this: > > 0000 0000 end of name > 00xx xxxx regular label of length xxxxxx > 11yy yyyy upper 6 bits of a compression reference > (followed by another byte containing the lower 8 bits) > 01zz zzzz used to be invalid, then extended type, then invalid again > 10zz zzzz invalid > > I could rename the variable to typeAndLength. Right, just too easy to mis-read compared to the original code. Using typeAndLen would be okay, so would "int len = type" and using len when constructing the String. Anything to make it easier for future maintainers. > : > > It is thrown by DNSName#add(int, String). Okay, I guess that is okay (I was mostly just wondering whether DnsName.add is throwing the most appropriate exception). So I think we are almost done and this and I can push it for you. -Alan From joe.darcy at oracle.com Wed Feb 19 18:25:12 2014 From: joe.darcy at oracle.com (Joe Darcy) Date: Wed, 19 Feb 2014 10:25:12 -0800 Subject: Serial warnings in java.util.ArrayPrefixHelpers Message-ID: <5304F708.5080800@oracle.com> Hello, The file java.util.ArrayPrefixHelpers seems to be part of the 166-alpha set of files in the JDK. There are a number of package-private static final and serializable classes in this file which neither define serialVersionUIDs nor have a @SuppressWarnings("serial") annotation. So, this file adds a number of serial warnings to the JDK and efforts are underway to clear the jdk repo of serial warnings, which are mostly in the client area: JDK-8032976: Fix serial lint warnings in client libraries Can the maintainers of ArrayPrefixHelpers "do the right thing" with respect to the serial warnings in this file? From a quick look at the file, it wasn't clear to me if SUIDs should be added or warnings suppressed. Thanks, -Joe From fweimer at redhat.com Wed Feb 19 19:16:29 2014 From: fweimer at redhat.com (Florian Weimer) Date: Wed, 19 Feb 2014 20:16:29 +0100 Subject: Minor com.sun.jndi.dns cleanup In-Reply-To: <5304ED52.9080204@oracle.com> References: <53022E09.2070002@redhat.com> <5302374C.4090409@redhat.com> <53026744.4030500@oracle.com> <53026A88.9030107@redhat.com> <53027C7B.2060506@oracle.com> <53035602.6080504@redhat.com> <53049DED.2040608@oracle.com> <5304A9EA.6020900@redhat.com> <5304ED52.9080204@oracle.com> Message-ID: <5305030D.1000205@redhat.com> On 02/19/2014 06:43 PM, Alan Bateman wrote: > On 19/02/2014 12:56, Florian Weimer wrote: >> >> It's encoded as a bit field, like this: >> >> 0000 0000 end of name >> 00xx xxxx regular label of length xxxxxx >> 11yy yyyy upper 6 bits of a compression reference >> (followed by another byte containing the lower 8 bits) >> 01zz zzzz used to be invalid, then extended type, then invalid again >> 10zz zzzz invalid >> >> I could rename the variable to typeAndLength. > Right, just too easy to mis-read compared to the original code. Using > typeAndLen would be okay, so would "int len = type" and using len when > constructing the String. Anything to make it easier for future maintainers. Updated webrev: > So I think we are almost done and this and I can push it for you. Thanks. -- Florian Weimer / Red Hat Product Security Team From Alan.Bateman at oracle.com Wed Feb 19 19:42:13 2014 From: Alan.Bateman at oracle.com (Alan Bateman) Date: Wed, 19 Feb 2014 19:42:13 +0000 Subject: RFR(S): 8034087: XML parser may overwrite element content if that content falls onto the border of an entity scanner buffer In-Reply-To: <53039CDD.7000504@oracle.com> References: <52FA535D.70804@oracle.com> <52FA9CC5.3010104@oracle.com> <61C6FA0C43817B4FA6EB52E901E137E7729A7913@DEWDFEMB18B.global.corp.sap> <52FBBCB7.9010907@oracle.com> <53039CDD.7000504@oracle.com> Message-ID: <53050915.7090207@oracle.com> On 18/02/2014 17:48, huizhe wang wrote: > Looks good to me. > > Thanks, > Joe > The test looks okay to me too. -Alan. From martinrb at google.com Wed Feb 19 20:16:38 2014 From: martinrb at google.com (Martin Buchholz) Date: Wed, 19 Feb 2014 12:16:38 -0800 Subject: Serial warnings in java.util.ArrayPrefixHelpers In-Reply-To: <5304F708.5080800@oracle.com> References: <5304F708.5080800@oracle.com> Message-ID: The jsr166 CVS version of this file already has serialVersionUIDs added. jsr166 CVS src/main has been jdk8/jdk9 warning-clean for a while now. http://gee.cs.oswego.edu/cgi-bin/viewcvs.cgi/jsr166/src/main/java/util/ArrayPrefixHelpers.java?view=markup yeah, we need to do a better job of down-streaming... On Wed, Feb 19, 2014 at 10:25 AM, Joe Darcy wrote: > Hello, > > The file java.util.ArrayPrefixHelpers seems to be part of the 166-alpha > set of files in the JDK. > > There are a number of package-private static final and serializable > classes in this file which neither define serialVersionUIDs nor have a > @SuppressWarnings("serial") annotation. So, this file adds a number of > serial warnings to the JDK and efforts are underway to clear the jdk repo > of serial warnings, which are mostly in the client area: > > JDK-8032976: Fix serial lint warnings in client libraries > > Can the maintainers of ArrayPrefixHelpers "do the right thing" with > respect to the serial warnings in this file? From a quick look at the file, > it wasn't clear to me if SUIDs should be added or warnings suppressed. > > Thanks, > > -Joe > From brian.burkhalter at oracle.com Wed Feb 19 21:44:47 2014 From: brian.burkhalter at oracle.com (Brian Burkhalter) Date: Wed, 19 Feb 2014 13:44:47 -0800 Subject: JDK 9 RFR of JDK-8035279: Clean up internal deprecations in BigInteger Message-ID: This patch concerns cleaning up some ugly internal deprecations. Issue: https://bugs.openjdk.java.net/browse/JDK-8035279 Webrev: http://cr.openjdk.java.net/~bpb/8035279/webrev.00/ All JTREG BigInteger tests pass, and the serialized form has been unaltered as verified by bidirectional read-write testing between Java 7 and this version. I would appreciate scrutiny of the arithmetic to make sure that I've made no silly errors. Also I would be interested in opinions as to whether the "volatile" keyword should in fact be used for the affected instance variables. Thank you for your comments. Brian From vladimir.x.ivanov at oracle.com Wed Feb 19 21:46:10 2014 From: vladimir.x.ivanov at oracle.com (Vladimir Ivanov) Date: Thu, 20 Feb 2014 01:46:10 +0400 Subject: [9] RFR (M): 8027827: Improve performance of catchException combinator Message-ID: <53052622.6010507@oracle.com> http://cr.openjdk.java.net/~vlivanov/8027827/final/webrev.00 https://jbs.oracle.com/bugs/browse/JDK-8027827 354 lines changed: 193 ins; 91 del; 70 mod OVERVIEW MethodHandles.catchException combinator implementation is based on generic invokers (MethodHandleImpl$GuardWithCatch.invoke_*). It is significantly slower than a Java equivalent (try-catch). Future Nashorn performance improvements require catchException combinator speed to be on par with try-catch in Java. So, it should be represented in a more efficient form. I chose the following lambda form representation: t_{i}:L=ValueConversions.array(a1:L,...,a_{k}:L); t_{i+1}:L=MethodHandleImpl.guardWithCatch(t_{p1}, t_{p2}, t_{p3}, t_{i}:L); t_{i+2}:I=ValueConversions.unbox(t7:L); OR :L=ValueConversions.identity(t_{n+1}) OR :V=ValueConversions.ignore(t_{n+1}) where: a1, ..., a_{k} - arguments t_{p1}, t_{p2}, t_{p3} - target method handle, exception class, catcher method handle respectively; passed as bounded parameters; During lambda form compilation it is converted into bytecode equivalent of the following Java code: try { return target.invokeBasic(...); } catch(Throwable e) { if (!exClass.isInstance(e)) throw e; return catcher.invokeBasic(e, ...); } There's a set of microbenchmarks (attached to the bug) I wrote to verify performance characteristics of new implementation. FURTHER WORK What is missing is lambda form caching. The plan is to have a single lambda form per basic type, but it needs more work - current representation is suitable for sharing on bytecode level, but lambda form interpretation doesn't work well (arguments boxing + result unboxing are problematic). TESTING Tests: microbenchmarks, jdk/java/lang/invoke/, nashorn with optimistic types (unit tests, octane). Tested in 2 modes: * COMPILE_THRESHOLD=30 * COMPILE_THRESHOLD=0 -Xverify:all OTHER 1) Update of cached name and member in LF compilation loop (see InvokerBytecodeGenerator.generateCustomizedCodeBytes) fixes a bug during compilation of selectAlternative when running with COMPILE_THRESHOLD=0. 2) As part of this change, I fix existing bug [1], so I add regression test for it. Thanks! Best regards, Vladimir Ivanov [1] https://bugs.openjdk.java.net/browse/JDK-8034120 From xueming.shen at oracle.com Wed Feb 19 22:31:23 2014 From: xueming.shen at oracle.com (Xueming Shen) Date: Wed, 19 Feb 2014 14:31:23 -0800 Subject: JDK 9 RFR of JDK-8035076: Pattern$BnMS never used due to bug in Pattern$BnM.optimize Message-ID: <530530BB.5010406@oracle.com> Hi, Please help codereview the change for JDK-8035076. Issue: https://bugs.openjdk.java.net/browse/JDK-8035076 Webrev: http://cr.openjdk.java.net/~sherman/8035067/webrev This is regression caused by the change we made back to jdk7 to support case insensitive match, in which a base class SliceNode for Slice family was added and we mistakenly subclass the SliceS class to this newly added class, instead of the original Slice class. The BnM optimization for supplementary support, which is based on the "instanceof Slice", is therefor disabled. Below is the original webrev for that changeset. http://cr.openjdk.java.net/~sherman/6486934_6233084_6504326_6436458 The proposed change is to subclass SliceS from Slice node. The change has been verified via the test case attached in the issue report. However I decided not to include this test as a regression test as it is too "implementation detail dependent". Thanks! -Sherman From joe.darcy at oracle.com Wed Feb 19 23:44:33 2014 From: joe.darcy at oracle.com (Joseph Darcy) Date: Wed, 19 Feb 2014 15:44:33 -0800 Subject: Serial warnings in java.util.ArrayPrefixHelpers In-Reply-To: References: <5304F708.5080800@oracle.com> Message-ID: <530541E1.2000703@oracle.com> On 2/19/2014 12:16 PM, Martin Buchholz wrote: > The jsr166 CVS version of this file already has serialVersionUIDs added. > jsr166 CVS src/main has been jdk8/jdk9 warning-clean for a while now. Thanks for the update Martin; looking forward to the next sync to get rid of a few more warnings :-) -Joe > > http://gee.cs.oswego.edu/cgi-bin/viewcvs.cgi/jsr166/src/main/java/util/ArrayPrefixHelpers.java?view=markup > > yeah, we need to do a better job of down-streaming... > > > On Wed, Feb 19, 2014 at 10:25 AM, Joe Darcy > wrote: > > Hello, > > The file java.util.ArrayPrefixHelpers seems to be part of the > 166-alpha set of files in the JDK. > > There are a number of package-private static final and > serializable classes in this file which neither define > serialVersionUIDs nor have a @SuppressWarnings("serial") > annotation. So, this file adds a number of serial warnings to the > JDK and efforts are underway to clear the jdk repo of serial > warnings, which are mostly in the client area: > > JDK-8032976: Fix serial lint warnings in client libraries > > Can the maintainers of ArrayPrefixHelpers "do the right thing" > with respect to the serial warnings in this file? From a quick > look at the file, it wasn't clear to me if SUIDs should be added > or warnings suppressed. > > Thanks, > > -Joe > > From stuart.marks at oracle.com Thu Feb 20 00:44:47 2014 From: stuart.marks at oracle.com (Stuart Marks) Date: Wed, 19 Feb 2014 16:44:47 -0800 Subject: RFR: JDK-8032050: TEST_BUG: java/rmi/activation/Activatable/shutdownGracefully/ShutdownGracefully.java fails intermittently In-Reply-To: <78E6DE88-531C-4548-B951-562C9BA98BE2@oracle.com> References: <52E20974.7060706@oracle.com> <52E31F53.3070702@oracle.com> <52E4959C.3020907@oracle.com> <52E8AADE.6000300@oracle.com> <52EB60A8.7070307@oracle.com> <133FA08E-5731-437D-BB1F-AD9872CBDD36@oracle.com> <52F96FA8.3010102@oracle.com> <0C67EF5A-7889-49C5-8079-BDA7FF4CF1F0@oracle.com> <52FBE7B4.5060802@oracle.com> <52FC8178.9000500@oracle.com> <52FEC72C.1070704@oracle.com> <78E6DE88-531C-4548-B951-562C9BA98BE2@oracle.com> Message-ID: <53054FFF.80604@oracle.com> Hi Tristan, Thanks for the updates. Pushed: http://hg.openjdk.java.net/jdk9/dev/jdk/rev/d24297425fd0 It's good to see all that unnecessary thread creation stuff removed. s'marks On 2/14/14 6:53 PM, Tristan Yan wrote: > Thank you Stuart > This is nice. I thought two variables were weird but I didn?t figure out the > right solution. Also I wasn?t so sure why we print out so many messages now > it?s clear to me. > http://cr.openjdk.java.net/~tyan/JDK-8032050/webrev.05/ > I?m sorry I have to ask you review this again. > regards > Tristan > > On Feb 15, 2014, at 9:47 AM, Stuart Marks > wrote: > >> Hi Tristan, >> >> OK, we're getting close. Just a couple things about ShutdownGracefully.java. >> >> It's a bit clumsy to have both a boolean and a message string to keep track of >> the state of the test, but by itself this isn't such a big deal. >> >> A bigger deal is the lack of exception chaining. If we catch an unexpected >> exception at line 162, its class and message are printed out, but its stack >> trace isn't. If this were to occur it might be fiendishly difficult to debug. >> >> The TimeoutException isn't chained up either, but this isn't so bad, since >> there's only one place the timeout can occur. Still, it's good to chain >> exceptions where possible. >> >> There's another failure case that doesn't have an exception at all, which is >> when the registration request we're expecting to fail actually succeeds. This >> doesn't have an exception, but we should consider creating one. >> >> Here's an approach to getting rid of the boolean+string and also chaining up >> exceptions. The key insight is that an exception can be created in a different >> place from where it's thrown. >> >> Instead of the boolean+stream, have a variable of type Exception that's >> initialized to null. Anyplace where we catch an exception that indicates >> failure, fill in this variable. The idea is that if this exception variable is >> non-null, a failure has occurred. The exception being non-null replaces the >> boolean, and the exception message replaces the failure string. In addition, >> the exception has a stack trace, which is essential for debugging. >> >> For the case where we don't get the expected exception (i.e., registration >> succeeds unexpectedly), simply set the exception variable to a newly created >> exception, but don't throw it yet. For example, >> >> exception = new RuntimeException( >> "The registration request succeeded unexpectedly"); >> >> At the place where we catch an unexpected exception, wrap the caught exception >> in a new RuntimeException with a message like "caught unexpected exception". >> The reason to do this is so we can add an additional message. The stack trace >> will also be a bit easier to follow. >> >> For the timeout, just assign the TimeoutException to the exception variable. >> >> Also, at each location where the exception variable is assigned to, print out >> a message at that point. It will at least show the state of the test to be a >> failure. The reason is that, if rmid.destroy() were to throw an exception from >> the finally-block, our carefully recorded exception state will be thrown away. >> (An alternative would be to put this into its own try-block, and add any >> exceptions caught from it to the suppressed exception list of the exception >> variable, but it's not clear to me that this is worth it.) >> >> At the end of the test, if the exception variable is non-null, call >> TestLibrary.bomb() with it to fail the test. >> >> Finally, one typo: "prcoess". >> >> Thanks for updating this webrev again. >> >> s'marks >> >> On 2/13/14 12:25 AM, Tristan Yan wrote: >>> Thank you Stuart >>> I have fixed comment in JavaVM.java. Dealing with different cases in >>> ShutdownGracefully.java, two variables were added. One is a flag indicate test >>> passed or not. Other variable keeps the error message when test failed. I put >>> TestLibrary.bomb in the bottom of the main method which only shows test fail >>> message. >>> Could you review it again >>> http://cr.openjdk.java.net/~tyan/JDK-8032050/webrev.04/ >>> Tristan >>> >>> On 02/13/2014 05:29 AM, Stuart Marks wrote: >>>> Hi Tristan, >>>> >>>> JavaVM.waitFor looks mostly fine. The indentation of the start of the >>>> waitFor(timeout) javadoc comment is a bit off, though; please fix. >>>> >>>> There are still some adjustments that need to be made in >>>> ShutdownGracefully.java. Both have to do with the case where the last >>>> registration succeeds unexpectedly -- this is the one that we expect to fail. >>>> >>>> First, if this registration has succeeded unexpectedly, that means rmid is >>>> still running. If that occurs, the call to rmid.waitFor(timeout) will >>>> inevitably time out. It may be worth calling rmid.destroy() directly at this >>>> point. >>>> >>>> Second, still in the succeeded-unexpectedly case, at line 154 >>>> TestLibrary.bomb() is called. This throws an exception, but it's caught by the >>>> catch-block at lines 157-158, which calls TestLibrary.bomb() again, saying >>>> "unexpected exception". Except that this is kind of expected, since it was >>>> thrown from an earlier call to TestLibrary.bomb(). This is quite confusing. >>>> >>>> There are several cases that need to be handled. >>>> >>>> 1. Normal case. Registration fails as expected, rmid has terminated >>>> gracefully. Test passes. >>>> >>>> 2. Rmid is still running and has processed the registration request >>>> successfully. Need to kill rmid and fail the test. >>>> >>>> 3. Rmid is still running but is in some bad state where the registration >>>> request failed. Need to kill rmid and fail the test. >>>> >>>> 4. Some other unexpected failure. This is what the catch and finally blocks at >>>> lines 157-161 are for. >>>> >>>> These four cases need to be handled independently. Ideally they should be >>>> separated from the cleanup code. As noted above, you don't want to throw an >>>> exception from the try-block, because it will get caught by your own catch >>>> block. Similarly, it's tempting to return from the midst of the try-block in >>>> the success case, but this still runs the finally-block. This can be quite >>>> confusing. >>>> >>>> A typical technique for dealing with this kind of issue is to record results >>>> of operations from within the try block, and then analyze the results outside, >>>> throwing a test failure (TestLibrary.bomb) at that point, where it won't be >>>> caught by the test's own catch block. >>>> >>>> Editoral: >>>> - line 153, there should be a space between 'if' and the opening paren >>>> - line 156, typo, "gracefuuly" >>>> >>>> Finally, it would be helpful if you could get webrev to generate the actual >>>> changeset instead of the plain patch, per my other review email. >>>> >>>> Thanks, >>>> >>>> s'marks >>>> >>>> >>>> On 2/11/14 9:39 PM, Tristan Yan wrote: >>>>> Thank you for your thorough mail. This is very educational. I took you advice >>>>> and generated a new webrev for this. >>>>> >>>>> http://cr.openjdk.java.net/~tyan/JDK-8032050/webrev.03/ >>>>> I appreciate you can review this again. >>>>> Regards >>>>> Tristan >>>>> >>>>> >>>>> On Feb 11, 2014, at 8:32 AM, Stuart Marks >>>> > wrote: >>>>> >>>>>> Hi Tristan, >>>>>> >>>>>> Sorry about my recurring delays. >>>>>> >>>>>> Several comments on these changes. >>>>>> >>>>>> JavaVM.java -- >>>>>> >>>>>> The waitFor(timeout) method is mostly ok. The new thread started at line 208 >>>>>> and following seems unnecessary, though. This code is reached when a timeout >>>>>> occurs, so throwing TimeoutException is the only thing necessary in this case. >>>>>> Should the code to start the new thread be removed? >>>>>> >>>>>> There should be a similar check for vm == null as in the waitFor() [no args] >>>>>> method. >>>>>> >>>>>> ShutdownGracefully.java -- >>>>>> >>>>>> The condition that's checked after calling rmid.waitFor(SHUTDOWN_TIMEOUT) is >>>>>> incorrect. It's testing the exit status against zero. Offhand, when and if >>>>>> rmid exits, it might exit with a nonzero exit status. If rmid has exited at >>>>>> this point, then the test should succeed. >>>>>> >>>>>> Instead of testing against zero, the code should catch TimeoutException, which >>>>>> means that rmid is still running. It's probably reasonable to catch >>>>>> TimeoutException, print a message, and then let the finally-block destroy the >>>>>> rmid. Calling TestLibrary.bomb() from within the try-block is confusing, since >>>>>> that method throws an exception, which is then caught by the catch-block, when >>>>>> then calls TestLibrary.bomb() again. >>>>>> >>>>>> We should also make sure to test the success case properly. If rmid.waitFor() >>>>>> returns in a timely fashion without throwing TimeoutException, it doesn't >>>>>> matter what the exit status is. (It might be worth printing it out.) At that >>>>>> point we know that rmid *has* exited gracefully, so we need to set rmid to >>>>>> null so that the finally-block doesn't attempt to destroy rmid redundantly. >>>>>> Some additional messages about rmid having exited and the test passing are >>>>>> also warranted for this case. >>>>>> >>>>>> Some additional cleanup can be done here as well, over and above the changes >>>>>> you've proposed. (This stuff is left over from earlier RMI test messes.) In >>>>>> order to shut down an active object, the code here spawns a new thread that >>>>>> sleeps for a while and then deactivates this object. This isn't necessary. (It >>>>>> might have been necessary in the past.) It's sufficient simply to unexport >>>>>> this object and then deactivate it, directly within the shutdown() method. See >>>>>> >>>>>> test/java/rmi/activation/ActivationSystem/unregisterGroup/UnregisterGroup.java >>>>>> >>>>>> for an example of this. In addition, the run() method can be removed, and the >>>>>> "implements Runnable" declaration can also be removed from the >>>>>> ShutdownGracefully test class. >>>>>> >>>>>> Finally, revisiting some code farther up in the test, the try-block at lines >>>>>> 135-140 issues a registration request that the test expects to fail. If it >>>>>> succeeds, the message at line 139 isn't very clear; it should say that the >>>>>> registration request succeeded unexpectedly. This should cause the test to >>>>>> fail. We still probably want to go through the waitFor(timeout) path and >>>>>> eventual rmid cleanup, but a flag should be set here to ensure that the test >>>>>> indeed fails if the registration succeeds unexpectedly, and the messages >>>>>> should clearly indicate that this is going on. >>>>>> >>>>>> A good way to test this last case is to change rmid's security manager to the >>>>>> normal security manager java.lang.SecurityManager instead of >>>>>> TestSecurityManager. >>>>>> >>>>>> Thanks, >>>>>> >>>>>> s'marks >>>>>> >>>>>> >>>>>> >>>>>> >>>>>> On 2/10/14 2:59 AM, Tristan Yan wrote: >>>>>>> Hi Stuart >>>>>>> Could you help to review this. >>>>>>> Thank you >>>>>>> Tristan >>>>>>> >>>>>>> On Jan 31, 2014, at 4:36 PM, Tristan Yan >>>>>> >>>>>>> >>>>>>> > wrote: >>>>>>> >>>>>>>> Thank you for fixing JDK-8023541. Then I leave ActivationLibrary.java for >>>>>>>> now. >>>>>>>> I still did some clean up following your suggestion. >>>>>>>> 1. I changed waitFor(long timeout) method, this method is going to use code >>>>>>>> like Process.waitFor(timeout, unit). This can be backported to JDK7. Also >>>>>>>> exitValue is kept as a return value. For making sure there is no Pipe >>>>>>>> leak, a >>>>>>>> cleanup thread will start when timeout happens. >>>>>>>> 2. Change in ShutdownGracefully is a little tricky. I think we should just >>>>>>>> destroy JVM once exception is thrown. So I move the wait logic into try >>>>>>>> block >>>>>>>> instead keep them in finally block. >>>>>>>> Can you receive it again. >>>>>>>> http://cr.openjdk.java.net/~tyan/JDK-8032050/webrev.02/ >>>>>>>> Thank you >>>>>>>> Tristan >>>>>>>> >>>>>>>> On 01/29/2014 03:16 PM, Stuart Marks wrote: >>>>>>>>> Hi Tristan, >>>>>>>>> >>>>>>>>> I don't want to put the workaround into ActivationLibrary.rmidRunning() >>>>>>>>> for a >>>>>>>>> null return from the lookup, because this is only a workaround for an >>>>>>>>> actual >>>>>>>>> bug in rmid initialization. See the review I just posted for JDK-8023541. >>>>>>>>> >>>>>>>>> Adding JavaVM.waitFor(timeout) is something that would be useful in >>>>>>>>> general, >>>>>>>>> but it needs to be handled carefully. It uses the new >>>>>>>>> Process.waitFor(timeout, unit) which is new in Java SE 8; this makes >>>>>>>>> backporting to 7 more complicated. Not clear whether we'll do so, but I >>>>>>>>> don't >>>>>>>>> want to forclose the opportunity without discussion. It's also not >>>>>>>>> clear how >>>>>>>>> one can get the vm's exit status after JavaVM.waitFor() has returned true. >>>>>>>>> With the Process API it's possible simply to call waitFor() or exitValue(). >>>>>>>>> With JavaVM, a new API needs to be created, or the rule has to be >>>>>>>>> established >>>>>>>>> that one must call JavaVM.waitFor() to collect the exit status as well >>>>>>>>> as to >>>>>>>>> close the pipes from the subprocess. If JavaVM.waitFor(timeout, unit) is >>>>>>>>> called without subsequently calling JavaVM.waitFor(), the pipes are leaked. >>>>>>>>> >>>>>>>>> In ShutdownGracefully.java, the finally-block needs to check to see if rmid >>>>>>>>> is still running, and if it is, to shut it down. Simply calling >>>>>>>>> waitFor(timeout, unit) isn't sufficient, because if the rmid process is >>>>>>>>> still >>>>>>>>> running, it will be left running. >>>>>>>>> >>>>>>>>> The straightforward approach would be to call >>>>>>>>> ActivationLibrary.rmidRunning() >>>>>>>>> to test if it's still running. Unfortunately this isn't quite right, >>>>>>>>> because >>>>>>>>> rmidRunning() has a timeout loop in it -- which should probably be removed. >>>>>>>>> (I think there's a bug for this.) Another approach would be simply to call >>>>>>>>> rmid.destroy(). This calls rmid's shutdown() method first, which is >>>>>>>>> reasonable, but I'm not sure it kills the process if that fails. In any >>>>>>>>> case, >>>>>>>>> this already has a timeout loop waiting for the process to die, so >>>>>>>>> ShutdownGracefully.java needn't use a new waitFor(timeout, unit) call. >>>>>>>>> >>>>>>>>> Removing the commented-out code that starts with "no longer needed" is >>>>>>>>> good, >>>>>>>>> and removing the ShutdownDetectThread is also good, since that's >>>>>>>>> unnecessary. >>>>>>>>> >>>>>>>>> There are some more cleanups I have in mind here but I'd like to see a >>>>>>>>> revised webrev before proceeding. >>>>>>>>> >>>>>>>>> Thanks, >>>>>>>>> >>>>>>>>> s'marks >>>>>>>>> >>>>>>>>> On 1/25/14 8:57 PM, Tristan Yan wrote: >>>>>>>>>> Hi Stuart >>>>>>>>>> Thank you for your review and suggestion. >>>>>>>>>> Yes, since this failure mode is very hard to be reproduced. I guess it's >>>>>>>>>> make sense to do some hack. And I also noticed in >>>>>>>>>> ActivationLibrary.rmidRunning. It does try to look up ActivationSystem but >>>>>>>>>> doesn't check if it's null. So I add the logic to make sure we will >>>>>>>>>> look up >>>>>>>>>> the non-null ActivationSystem. Also I did some cleanup if you don't mind. >>>>>>>>>> Add a waitFor(long timeout, TimeUnit unit) for JavaVM. Which we can have a >>>>>>>>>> better waitFor control. >>>>>>>>>> I appreciate you can review the code again. >>>>>>>>>> http://cr.openjdk.java.net/~tyan/JDK-8032050/webrev.01/ >>>>>>>>>> Thank you >>>>>>>>>> Tristan >>>>>>>>>> >>>>>>>>>> >>>>>>>>>> On 01/25/2014 10:20 AM, Stuart Marks wrote: >>>>>>>>>>> On 1/23/14 10:34 PM, Tristan Yan wrote: >>>>>>>>>>>> Hi All >>>>>>>>>>>> Could you review the bug fix for JDK-8032050. >>>>>>>>>>>> >>>>>>>>>>>> http://cr.openjdk.java.net/~tyan/JDK-8032050/webrev.00/ >>>>>>>>>>>> >>>>>>>>>>>> Description: >>>>>>>>>>>> This rare happened failure caused because when RMID starts. It don't >>>>>>>>>>>> guarantee >>>>>>>>>>>> sun.rmi.server.Activation.startActivation finishes. >>>>>>>>>>>> Fix by adding a iterative getSystem with a 5 seconds timeout. >>>>>>>>>>> >>>>>>>>>>> Hi Tristan, >>>>>>>>>>> >>>>>>>>>>> Adding a timing/retry loop into this test isn't the correct approach for >>>>>>>>>>> fixing this test. >>>>>>>>>>> >>>>>>>>>>> The timing loop isn't necessary because there is already a timing loop in >>>>>>>>>>> RMID.start() in the RMI test library. (There's another timing loop in >>>>>>>>>>> ActivationLibrary.rmidRunning() which should probably be removed.) So the >>>>>>>>>>> intent of this library call is that it start rmid and wait for it to >>>>>>>>>>> become >>>>>>>>>>> ready. That logic doesn't need to be added to the test. >>>>>>>>>>> >>>>>>>>>>> In the bug report JDK-8032050 you had mentioned that the >>>>>>>>>>> NullPointerException was suspicious. You're right! I took a look and it >>>>>>>>>>> seemed like it was related to JDK-8023541, and I added a note to this >>>>>>>>>>> effect to the bug report. The problem here is that rmid can come up and >>>>>>>>>>> transiently return null instead of the stub of the activation system. >>>>>>>>>>> That's what JDK-8023541 covers. I think that rmid itself needs to be >>>>>>>>>>> fixed, >>>>>>>>>>> though modifying the timing loop in the RMI test library to wait for rmid >>>>>>>>>>> to come up *and* for the lookup to return non-null is an easy way to fix >>>>>>>>>>> the problem. (Or at least cover it up.) >>>>>>>>>>> >>>>>>>>>>> The next step in the analysis is to determine, given that >>>>>>>>>>> ActivationLibrary.getSystem can sometimes return null, whether this has >>>>>>>>>>> actually caused this test failure. This is pretty easy to determine; just >>>>>>>>>>> hack in a line "system = null" in the right place and run the test. I've >>>>>>>>>>> done this, and the test times out and the output log is pretty much >>>>>>>>>>> identical to the one in the bug report. (I recommend you try this >>>>>>>>>>> yourself.) So I think it's fairly safe to say that the problem in >>>>>>>>>>> JDK-8023541 has caused the failure listed in JDK-8032050. >>>>>>>>>>> >>>>>>>>>>> I can see a couple ways to proceed here. One way is just to close >>>>>>>>>>> this out >>>>>>>>>>> as a duplicate of JDK-8023541 since that bug caused this failure. >>>>>>>>>>> >>>>>>>>>>> Another is that this test could use some cleaning up. This bug certainly >>>>>>>>>>> covers a failure, but the messages emitted are confusing and in some >>>>>>>>>>> cases >>>>>>>>>>> completely wrong. For example, the "rmid has shutdown" message at >>>>>>>>>>> line 180 >>>>>>>>>>> is incorrect, because in this case rmid is still running and the wait() >>>>>>>>>>> call has timed out. Most of the code here can be replaced with calls to >>>>>>>>>>> various bits of the RMI test library. There are a bunch of other >>>>>>>>>>> things in >>>>>>>>>>> this test that could be cleaned up as well. >>>>>>>>>>> >>>>>>>>>>> It's up to you how you'd like to proceed. >>>>>>>>>>> >>>>>>>>>>> s'marks >>>>>>>>>> >>>>>>>>> >>>>>>>> >>>>>>> >>>>> >>> > From luchsh at linux.vnet.ibm.com Thu Feb 20 03:15:30 2014 From: luchsh at linux.vnet.ibm.com (Jonathan Lu) Date: Thu, 20 Feb 2014 11:15:30 +0800 Subject: Add IDN support to existing net APIs In-Reply-To: <53048751.8070000@oracle.com> References: <53048751.8070000@oracle.com> Message-ID: Hi Michael, Thanks a lot for your comments. On Wed, Feb 19, 2014 at 6:28 PM, Michael McMahon < michael.x.mcmahon at oracle.com> wrote: > I think it's a good idea. Before changing anything though, > we would need to: > > 1. identify all APIs that could potentially be affected and figure out > what is > needed for each. For instance: > 1. InetAddress > 2. SocketPermission > 3. URLPermission > 4. HttpURLConnection > 5. URL/URI > 6. ... > > Right, I assume all APIs which accept string-represented host names would be affected. It that may not be a big change from point of view. > > 1. > 1. understand how it relates to URIs and IRIs. I haven't looked > into this much > but it seems that there might be potential for confusion given that > these entities > have their own encoding schemes already. I'm sure the issues are all > well thrashed out > already, but I'd like to see exactly how it will work in Java before > we start the work > > For the potential confusion, were you meaning a confusion between URI encoding (RFC-2396 [1]) and IDN ? > > > Maybe, the work is within the scope of a small JEP, if nothing else to > define the scope > of the work. > > Michael. > Copying core-libs-dev and security-dev to get more comments. Many thanks - Jonathan [1] http://www.ietf.org/rfc/rfc2396.txt > > > On 19/02/14 04:00, Jonathan Lu wrote: > > Hi net-dev, > > If a Java application tries to support International Domain Names (IDN) > [1], which was firstly brought in Java6, > it has to write additional code or wrap java.net.IDN API [2] to make the > conversion each time it tries to resolve > a non-ASCII domain name, and use the converted punycode to make > connections to remote host, like: > > java.net.IDN.toASCII("??.info "); > > It is easier for newly writen applications, since a wrappers can be made > in early design stage, but if > it comes to existing Java applications and third-party libraries, IDN > support would involve much more effort > like modifying the existing code base, inspecting libraries > interfaces/implementations, or even > re-implement some of the functions. > > I'm wondering if it is possible to add IDN support into existing Java > APIs, like > java.net.InetAddress.getByName(), if JDK itself supports IDN, any existing > applications or libraries would > benefit from supporting IDN automatically without any source code > modifications. > > Of course there's security risks regarding IDN homograph attacks [1][2], > so it may not be a > good idea to change the default behavior right now. But it would still > work if a simple switch can be > added into current JDK. > For example, by defining following the property in command like options > like > > -Djava.net.AutoIDN=true > > Any existing Java applications who wishes to support IDN feature will be > able to support IDN > without any changes to its source code or re-compilation. > > What's your opinion on this ? any comment is welcome. > > Thank you > > - Jonathan Lu > > ----------- > > [1] http://en.wikipedia.org/wiki/Internationalized_domain_name > [2] http://download.java.net/jdk8/docs/api/java/net/IDN.html > [3] http://www.unicode.org/reports/tr36/ > [4] http://en.wikipedia.org/wiki/IDN_homograph_attack > > > From martinrb at google.com Thu Feb 20 03:47:48 2014 From: martinrb at google.com (Martin Buchholz) Date: Wed, 19 Feb 2014 19:47:48 -0800 Subject: JDK 9 RFR of JDK-8035076: Pattern$BnMS never used due to bug in Pattern$BnM.optimize In-Reply-To: <530530BB.5010406@oracle.com> References: <530530BB.5010406@oracle.com> Message-ID: I don't see an actual test added here: http://cr.openjdk.java.net/~sherman/8035067/webrev/test/java/util/regex/RegExTest.java.udiff.html On Wed, Feb 19, 2014 at 2:31 PM, Xueming Shen wrote: > Hi, > > Please help codereview the change for JDK-8035076. > > Issue: https://bugs.openjdk.java.net/browse/JDK-8035076 > Webrev: http://cr.openjdk.java.net/~sherman/8035067/webrev > > This is regression caused by the change we made back to jdk7 to support > case > insensitive match, in which a base class SliceNode for Slice family was > added and > we mistakenly subclass the SliceS class to this newly added class, instead > of > the original Slice class. The BnM optimization for supplementary support, > which > is based on the "instanceof Slice", is therefor disabled. Below is the > original > webrev for that changeset. > > http://cr.openjdk.java.net/~sherman/6486934_6233084_6504326_6436458 > > The proposed change is to subclass SliceS from Slice node. The change has > been > verified via the test case attached in the issue report. However I decided > not to > include this test as a regression test as it is too "implementation detail > dependent". > > Thanks! > -Sherman > > > From xueming.shen at oracle.com Thu Feb 20 04:57:55 2014 From: xueming.shen at oracle.com (Xueming Shen) Date: Wed, 19 Feb 2014 20:57:55 -0800 Subject: JDK 9 RFR of JDK-8035076: Pattern$BnMS never used due to bug in Pattern$BnM.optimize In-Reply-To: References: <530530BB.5010406@oracle.com> Message-ID: <53058B53.4050203@oracle.com> On 2/19/2014 7:47 PM, Martin Buchholz wrote: > I don't see an actual test added here: > > http://cr.openjdk.java.net/~sherman/8035067/webrev/test/java/util/regex/RegExTest.java.udiff.html > No actually test added, just added the bugid. The existing tests should cover all functional testing for the supplementary characters. The fix is to enable the optimization path, black box type test does not work. As I mentioned in the email, I did verify the fix by using the provided test case which uses reflect/Field to go through regex's node chain. I feel uncomfortable to add that as a unit/regression test, as it is too implementation detail dependent. But if the consensus is that type of test is OK to serve as a unit/regression, I can surely add it in. -Sherman > > On Wed, Feb 19, 2014 at 2:31 PM, Xueming Shen > wrote: > > Hi, > > Please help codereview the change for JDK-8035076. > > Issue: https://bugs.openjdk.java.net/browse/JDK-8035076 > Webrev: http://cr.openjdk.java.net/~sherman/8035067/webrev > > This is regression caused by the change we made back to jdk7 to support case > insensitive match, in which a base class SliceNode for Slice family was added and > we mistakenly subclass the SliceS class to this newly added class, instead of > the original Slice class. The BnM optimization for supplementary support, which > is based on the "instanceof Slice", is therefor disabled. Below is the original > webrev for that changeset. > > http://cr.openjdk.java.net/~sherman/6486934_6233084_6504326_6436458 > > The proposed change is to subclass SliceS from Slice node. The change has been > verified via the test case attached in the issue report. However I decided not to > include this test as a regression test as it is too "implementation detail dependent". > > Thanks! > -Sherman > > > From martinrb at google.com Thu Feb 20 05:29:56 2014 From: martinrb at google.com (Martin Buchholz) Date: Wed, 19 Feb 2014 21:29:56 -0800 Subject: JDK 9 RFR of JDK-8035076: Pattern$BnMS never used due to bug in Pattern$BnM.optimize In-Reply-To: <53058B53.4050203@oracle.com> References: <530530BB.5010406@oracle.com> <53058B53.4050203@oracle.com> Message-ID: OK, understood - looks good. I might be tempted to add some reflective debugging capabilities and use those in a test, so we can ensure that the "compiled form" of a regex conforms to our expectations. On Wed, Feb 19, 2014 at 8:57 PM, Xueming Shen wrote: > > On 2/19/2014 7:47 PM, Martin Buchholz wrote: > > I don't see an actual test added here: > > > http://cr.openjdk.java.net/~sherman/8035067/webrev/test/java/util/regex/RegExTest.java.udiff.html > > > No actually test added, just added the bugid. The existing tests should > cover all functional testing for the > supplementary characters. The fix is to enable the optimization path, > black box type test does not work. > > As I mentioned in the email, I did verify the fix by using the provided > test case which uses reflect/Field > to go through regex's node chain. I feel uncomfortable to add that as a > unit/regression test, as it is > too implementation detail dependent. But if the consensus is that type of > test is OK to serve as a > unit/regression, I can surely add it in. > > -Sherman > > > > On Wed, Feb 19, 2014 at 2:31 PM, Xueming Shen wrote: > >> Hi, >> >> Please help codereview the change for JDK-8035076. >> >> Issue: https://bugs.openjdk.java.net/browse/JDK-8035076 >> Webrev: http://cr.openjdk.java.net/~sherman/8035067/webrev >> >> This is regression caused by the change we made back to jdk7 to support >> case >> insensitive match, in which a base class SliceNode for Slice family was >> added and >> we mistakenly subclass the SliceS class to this newly added class, >> instead of >> the original Slice class. The BnM optimization for supplementary support, >> which >> is based on the "instanceof Slice", is therefor disabled. Below is the >> original >> webrev for that changeset. >> >> http://cr.openjdk.java.net/~sherman/6486934_6233084_6504326_6436458 >> >> The proposed change is to subclass SliceS from Slice node. The change has >> been >> verified via the test case attached in the issue report. However I >> decided not to >> include this test as a regression test as it is too "implementation >> detail dependent". >> >> Thanks! >> -Sherman >> >> >> > > From michael.cui at oracle.com Thu Feb 20 06:11:03 2014 From: michael.cui at oracle.com (michael cui) Date: Thu, 20 Feb 2014 14:11:03 +0800 Subject: Discussion on root cause analysis of JDK-7052625 : com/sun/net/httpserver/bugs/6725892/Test.java fails intermittently In-Reply-To: <53023E12.1060206@oracle.com> References: <53023E12.1060206@oracle.com> Message-ID: <53059C77.3030003@oracle.com> On 02/18/2014 12:51 AM, michael cui wrote: > Hi, > > I would like to discuss my current root cause analysis of JDK-7052625 > : com/sun/net/httpserver/bugs/6725892/Test.java fails intermittently > > As JDK-6725892 > stated, the purpose of this regression test is testing bad http > connections can be handled correctly which including > + send no request > + send an incomplete request > + fail to read the response completely. > > test3() method will start 20 threads for each type listed above at > same time. So totally 60 threads started in test3(). Each thread will > open connection to httpserver and simulate the normal or bad http > request to see if http server can handle them correctly. (20 threads > for incomplete read, 20 threads for incomplete write, 20 threads for > read/write normal case) > > Those threads will be started at same time. Among them, 40 threads > using sleep to simulate bad request. > > The http server created by the following api call : > s1 = HttpServer.create (addr, 0); > > According API doc > > and ServerSocket.java source code, the second parameter is backlog of > socket which is the maximum number of queued incoming connections to > allow on the listening socket. Queued TCP connections exceeding this > limit may be rejected by the TCP implementation.. The default value 50 > will be used if it was set to zero (See api doc > > and ServerSocket.java )). > > Since in test3(), 40 threads out of total 60 threads will simulate bad > http request by sleeping either at reading or writing, there could be > a very little possibility that httpserver 's socket connection queue > reach his limit (50 for default value) and some tcp connection will be > rest at that situation. > > This could be the root cause of this intermittently failure. > > Test result of the original version : > 0 failure on Linux for 10000 runs. > 0 failure on solaris for 10000 runs. > 6 failure on windows for 10000 runs > 28 failures on mac for 10000 runs > > By increasing the thread number of bad request, we can observe that > the frequency of failure will be increased. > > Test result of fix version in which backlog of httpserver was changed > from 0 to 100. > 0 failure on Linux for 10000 runs. > 0 failure on solaris for 10000 runs. > 0 failure on windows for 10000 runs > 0 failures on mac for 10000 runs > > It seems to me that using default 0 for backlog of httpserver could be > root cause of this intermittently failure. > Are we comfortable with this analysis? If it is the root cause, could > setting backlog as 100 be a suggest fix? > > Thanks, > Michael Cui > Could anyone provide some insight on this analysis? Michael Cui From Alan.Bateman at oracle.com Thu Feb 20 08:13:08 2014 From: Alan.Bateman at oracle.com (Alan Bateman) Date: Thu, 20 Feb 2014 08:13:08 +0000 Subject: JDK 9 RFR of JDK-8035279: Clean up internal deprecations in BigInteger In-Reply-To: References: Message-ID: <5305B914.7000901@oracle.com> On 19/02/2014 21:44, Brian Burkhalter wrote: > This patch concerns cleaning up some ugly internal deprecations. > > Issue: https://bugs.openjdk.java.net/browse/JDK-8035279 > Webrev: http://cr.openjdk.java.net/~bpb/8035279/webrev.00/ > > All JTREG BigInteger tests pass, and the serialized form has been unaltered as verified by bidirectional read-write testing between Java 7 and this version. > > I would appreciate scrutiny of the arithmetic to make sure that I've made no silly errors. Also I would be interested in opinions as to whether the "volatile" keyword should in fact be used for the affected instance variables. > It's not clear (to me) that these should be volatile. I also wonder about setting the initial value too as volatile could make this expensive in some environments. Are you sure there isn't any serialization impact? Zeros has also meant these are uninitialized so I just wonder about deserializing on an older JDK. -Alan. From Alan.Bateman at oracle.com Thu Feb 20 08:21:55 2014 From: Alan.Bateman at oracle.com (Alan Bateman) Date: Thu, 20 Feb 2014 08:21:55 +0000 Subject: JDK 9 RFR of JDK-8035076: Pattern$BnMS never used due to bug in Pattern$BnM.optimize In-Reply-To: <530530BB.5010406@oracle.com> References: <530530BB.5010406@oracle.com> Message-ID: <5305BB23.5020304@oracle.com> On 19/02/2014 22:31, Xueming Shen wrote: > Hi, > > Please help codereview the change for JDK-8035076. > > Issue: https://bugs.openjdk.java.net/browse/JDK-8035076 > Webrev: http://cr.openjdk.java.net/~sherman/8035067/webrev > > This is regression caused by the change we made back to jdk7 to > support case > insensitive match, in which a base class SliceNode for Slice family > was added and > we mistakenly subclass the SliceS class to this newly added class, > instead of > the original Slice class. The BnM optimization for supplementary > support, which > is based on the "instanceof Slice", is therefor disabled. I wonder if the submitter observed a performance regression or whether it was just observed by reading the code. In any case, it looks good to me. -Alan. From michael.cui at oracle.com Thu Feb 20 08:33:43 2014 From: michael.cui at oracle.com (michael cui) Date: Thu, 20 Feb 2014 16:33:43 +0800 Subject: RFR for JDK-8031563: TEST_BUG: java/nio/channels/Selector/ChangingInterests.java failed once In-Reply-To: <5302F664.1060504@oracle.com> References: <52FB60AB.4070708@oracle.com> <52FB7868.3080907@oracle.com> <52FCB756.5080906@oracle.com> <5302F664.1060504@oracle.com> Message-ID: <5305BDE7.4080501@oracle.com> On 02/18/2014 01:57 PM, michael cui wrote: > Hi Alan, > > On 02/13/2014 08:15 PM, Alan Bateman wrote: >> I plan to look at this soon but I have a concern that it is changing >> the test so that the change from OP_READ to OP_READ|OP_WRITE no >> longer tests that the key is both readable and writable after one >> select. > Have you got chance to review the code change of proposed solution A > and B? > > For solution B, it seems to me that code change isn't a change to > test. I would like to see if you still have same concern about any > cases after reviewing the code. Any chance to review the code? Michael Cui From Alan.Bateman at oracle.com Thu Feb 20 08:37:21 2014 From: Alan.Bateman at oracle.com (Alan Bateman) Date: Thu, 20 Feb 2014 08:37:21 +0000 Subject: RFR for JDK-8031563: TEST_BUG: java/nio/channels/Selector/ChangingInterests.java failed once In-Reply-To: <5305BDE7.4080501@oracle.com> References: <52FB60AB.4070708@oracle.com> <52FB7868.3080907@oracle.com> <52FCB756.5080906@oracle.com> <5302F664.1060504@oracle.com> <5305BDE7.4080501@oracle.com> Message-ID: <5305BEC1.5070109@oracle.com> On 20/02/2014 08:33, michael cui wrote: > Any chance to review the code? I will try to get back to you by early next week with suggestions. -Alan. From paul.sandoz at oracle.com Thu Feb 20 09:42:49 2014 From: paul.sandoz at oracle.com (Paul Sandoz) Date: Thu, 20 Feb 2014 10:42:49 +0100 Subject: JDK 9 RFR of JDK-8035279: Clean up internal deprecations in BigInteger In-Reply-To: <5305B914.7000901@oracle.com> References: <5305B914.7000901@oracle.com> Message-ID: <264E7F8B-E421-4D91-88D9-A22EF0366F2A@oracle.com> On Feb 20, 2014, at 9:13 AM, Alan Bateman wrote: > On 19/02/2014 21:44, Brian Burkhalter wrote: >> This patch concerns cleaning up some ugly internal deprecations. >> >> Issue: https://bugs.openjdk.java.net/browse/JDK-8035279 >> Webrev: http://cr.openjdk.java.net/~bpb/8035279/webrev.00/ >> >> All JTREG BigInteger tests pass, and the serialized form has been unaltered as verified by bidirectional read-write testing between Java 7 and this version. >> >> I would appreciate scrutiny of the arithmetic to make sure that I've made no silly errors. Also I would be interested in opinions as to whether the "volatile" keyword should in fact be used for the affected instance variables. >> > It's not clear (to me) that these should be volatile. I also wonder about setting the initial value too as volatile could make this expensive in some environments. > Yes, IIUC only atomic access is required. Here be dragons! Not sure the static powerCache field, in the original code, needs to be volatile either: 1137 private static volatile BigInteger[][] powerCache; If the recently added volatile on the fields are removed the setting of them for deserialization should be moved above the putMag call to preserve construction semantics (although see Alan's point on serialization compat): 4272 // Calculate mag field from magnitude and discard magnitude 4273 UnsafeHolder.putMag(this, mag); 4274 if (mag.length >= MAX_MAG_LENGTH) { 4275 try { 4276 checkRange(); 4277 } catch (ArithmeticException e) { 4278 throw new java.io.StreamCorruptedException("BigInteger: Out of the supported range"); 4279 } 4280 } 4281 4282 // Initialize cached fields to indicate not computed yet. 4283 bitCount = Integer.MIN_VALUE; 4284 bitLength = Integer.MIN_VALUE; 4285 lowestSetBit = Integer.MIN_VALUE; 4286 firstNonzeroIntNum = Integer.MIN_VALUE; In fact the UnsafeHolder.putSign impl could be changed to use putObject rather than putObjectVolatile, since we just need to stamp a fence in after all fields have been set in readObject (alternatively Unsafe.fullFence could be used as the last call before readObject returns). Paul. > Are you sure there isn't any serialization impact? Zeros has also meant these are uninitialized so I just wonder about deserializing on an older JDK. > > -Alan. > From david.holmes at oracle.com Thu Feb 20 10:20:27 2014 From: david.holmes at oracle.com (David Holmes) Date: Thu, 20 Feb 2014 20:20:27 +1000 Subject: JDK 9 RFR of JDK-8035279: Clean up internal deprecations in BigInteger In-Reply-To: References: Message-ID: <5305D6EB.30806@oracle.com> Hi Brian, On 20/02/2014 7:44 AM, Brian Burkhalter wrote: > This patch concerns cleaning up some ugly internal deprecations. > > Issue: https://bugs.openjdk.java.net/browse/JDK-8035279 > Webrev: http://cr.openjdk.java.net/~bpb/8035279/webrev.00/ > > All JTREG BigInteger tests pass, and the serialized form has been unaltered as verified by bidirectional read-write testing between Java 7 and this version. > > I would appreciate scrutiny of the arithmetic to make sure that I've made no silly errors. Also I would be interested in opinions as to whether the "volatile" keyword should in fact be used for the affected instance variables. re: volatile The basic approach being taken with these primitive fields is lazy initialization to a value that will always be the same if recomputed. If the variables are not volatile then in the worst-case (this is theoretical only) they will be recomputed by each thread that accesses them. If they are volatile then they will only be recomputed by threads that truly race on the initialization logic, but you will thereafter pay the price of a volatile read on each access. Either way you will be functionally correct so it all comes down to performance. Practically speaking I would expect you to be better off overall without the volatile. As Paul notes the powerCache array does not need to be volatile as it is initialized as part of class initialization. Cheers, David > Thank you for your comments. > > Brian > From david.holmes at oracle.com Thu Feb 20 10:25:11 2014 From: david.holmes at oracle.com (David Holmes) Date: Thu, 20 Feb 2014 20:25:11 +1000 Subject: Discussion on root cause analysis of JDK-7052625 : com/sun/net/httpserver/bugs/6725892/Test.java fails intermittently In-Reply-To: <53059C77.3030003@oracle.com> References: <53023E12.1060206@oracle.com> <53059C77.3030003@oracle.com> Message-ID: <5305D807.8010904@oracle.com> Sounds like a question for net-dev more than core-libs - cc'd. David On 20/02/2014 4:11 PM, michael cui wrote: > On 02/18/2014 12:51 AM, michael cui wrote: >> Hi, >> >> I would like to discuss my current root cause analysis of JDK-7052625 >> : com/sun/net/httpserver/bugs/6725892/Test.java fails intermittently >> >> As JDK-6725892 >> stated, the purpose of this regression test is testing bad http >> connections can be handled correctly which including >> + send no request >> + send an incomplete request >> + fail to read the response completely. >> >> test3() method will start 20 threads for each type listed above at >> same time. So totally 60 threads started in test3(). Each thread will >> open connection to httpserver and simulate the normal or bad http >> request to see if http server can handle them correctly. (20 threads >> for incomplete read, 20 threads for incomplete write, 20 threads for >> read/write normal case) >> >> Those threads will be started at same time. Among them, 40 threads >> using sleep to simulate bad request. >> >> The http server created by the following api call : >> s1 = HttpServer.create (addr, 0); >> >> According API doc >> >> and ServerSocket.java source code, the second parameter is backlog of >> socket which is the maximum number of queued incoming connections to >> allow on the listening socket. Queued TCP connections exceeding this >> limit may be rejected by the TCP implementation.. The default value 50 >> will be used if it was set to zero (See api doc >> >> and ServerSocket.java )). >> >> Since in test3(), 40 threads out of total 60 threads will simulate bad >> http request by sleeping either at reading or writing, there could be >> a very little possibility that httpserver 's socket connection queue >> reach his limit (50 for default value) and some tcp connection will be >> rest at that situation. >> >> This could be the root cause of this intermittently failure. >> >> Test result of the original version : >> 0 failure on Linux for 10000 runs. >> 0 failure on solaris for 10000 runs. >> 6 failure on windows for 10000 runs >> 28 failures on mac for 10000 runs >> >> By increasing the thread number of bad request, we can observe that >> the frequency of failure will be increased. >> >> Test result of fix version in which backlog of httpserver was changed >> from 0 to 100. >> 0 failure on Linux for 10000 runs. >> 0 failure on solaris for 10000 runs. >> 0 failure on windows for 10000 runs >> 0 failures on mac for 10000 runs >> >> It seems to me that using default 0 for backlog of httpserver could be >> root cause of this intermittently failure. >> Are we comfortable with this analysis? If it is the root cause, could >> setting backlog as 100 be a suggest fix? >> >> Thanks, >> Michael Cui >> > Could anyone provide some insight on this analysis? > > Michael Cui > From chris.hegarty at oracle.com Thu Feb 20 12:42:21 2014 From: chris.hegarty at oracle.com (Chris Hegarty) Date: Thu, 20 Feb 2014 12:42:21 +0000 Subject: Serial warnings in java.util.ArrayPrefixHelpers In-Reply-To: <530541E1.2000703@oracle.com> References: <5304F708.5080800@oracle.com> <530541E1.2000703@oracle.com> Message-ID: <83AFBC4C-EB90-4918-9E2D-1250851F89E9@oracle.com> On 19 Feb 2014, at 23:44, Joseph Darcy wrote: > On 2/19/2014 12:16 PM, Martin Buchholz wrote: >> The jsr166 CVS version of this file already has serialVersionUIDs added. >> jsr166 CVS src/main has been jdk8/jdk9 warning-clean for a while now. > > Thanks for the update Martin; looking forward to the next sync to get rid of a few more warnings :-) http://hg.openjdk.java.net/jdk9/dev/jdk/rev/2f85b69d1d3f -Chris. > > -Joe > >> >> http://gee.cs.oswego.edu/cgi-bin/viewcvs.cgi/jsr166/src/main/java/util/ArrayPrefixHelpers.java?view=markup >> >> yeah, we need to do a better job of down-streaming... >> >> >> On Wed, Feb 19, 2014 at 10:25 AM, Joe Darcy > wrote: >> >> Hello, >> >> The file java.util.ArrayPrefixHelpers seems to be part of the >> 166-alpha set of files in the JDK. >> >> There are a number of package-private static final and >> serializable classes in this file which neither define >> serialVersionUIDs nor have a @SuppressWarnings("serial") >> annotation. So, this file adds a number of serial warnings to the >> JDK and efforts are underway to clear the jdk repo of serial >> warnings, which are mostly in the client area: >> >> JDK-8032976: Fix serial lint warnings in client libraries >> >> Can the maintainers of ArrayPrefixHelpers "do the right thing" >> with respect to the serial warnings in this file? From a quick >> look at the file, it wasn't clear to me if SUIDs should be added >> or warnings suppressed. >> >> Thanks, >> >> -Joe >> >> > From aleksej.efimov at oracle.com Thu Feb 20 14:54:55 2014 From: aleksej.efimov at oracle.com (Aleksej Efimov) Date: Thu, 20 Feb 2014 18:54:55 +0400 Subject: RFR: 8032909: XSLT string-length returns incorrect length when string includes complementary chars Message-ID: <5306173F.7050608@oracle.com> Hi, There is a problem in XSLT string-length function: When string passed to this function contains complementary chars the current implementation of XPath treates this sequence as two characters (the string-length returns 2). But according to the w3 spec it should be treated as a single XPath character [1]. Please, review the proposed fix for this problem [2]. It replaces the 'length' call in xslt compiler with a call to 'codePointCount'. Also new regression test was added. To verify the fix following tests were executed: 1. JCK test sets: api/xsl api/xinclude api/javax_xml api/org_xml xml_schema 2. Regression tests from 'jdk_other' testset (includes added regression test) Thanks in advance, Aleksej [1] http://www.w3.org/TR/xpath/#strings [2] http://cr.openjdk.java.net/~aefimov/8032909/webrev.00 From brian.burkhalter at oracle.com Thu Feb 20 15:24:44 2014 From: brian.burkhalter at oracle.com (Brian Burkhalter) Date: Thu, 20 Feb 2014 07:24:44 -0800 Subject: JDK 9 RFR of JDK-8035279: Clean up internal deprecations in BigInteger In-Reply-To: <5305B914.7000901@oracle.com> References: <5305B914.7000901@oracle.com> Message-ID: <1D87886B-63FD-458E-A1FA-2E5AF8642B46@oracle.com> On Feb 20, 2014, at 12:13 AM, Alan Bateman wrote: > Are you sure there isn't any serialization impact? Zeros has also meant these are uninitialized so I just wonder about deserializing on an older JDK. I tested the following: 1) serialize with 9, deserialize with 7 2) serialize with 7, deserialize with 9 and observed no errors unless of course my test was incorrect. Thanks, Brian From marcus.lagergren at oracle.com Thu Feb 20 15:31:30 2014 From: marcus.lagergren at oracle.com (Marcus Lagergren) Date: Thu, 20 Feb 2014 16:31:30 +0100 Subject: [9] RFR (M): 8027827: Improve performance of catchException combinator In-Reply-To: <53052622.6010507@oracle.com> References: <53052622.6010507@oracle.com> Message-ID: This looks good, and we have done a significant number of test runs to verify its integrity. I say ship it. +1 We know that there are some issues with sun.misc.ValueConversion.castReference and similar internal methods not being inlined, but as far as I can understand this is a separate issue that will be addressed. By rewriting a guard for Nashorn to not use castReference in the fast case, I get record indy performance with your catch combinator. /M (jdk9 reviewer) On 19 Feb 2014, at 22:46, Vladimir Ivanov wrote: > http://cr.openjdk.java.net/~vlivanov/8027827/final/webrev.00 > https://jbs.oracle.com/bugs/browse/JDK-8027827 > 354 lines changed: 193 ins; 91 del; 70 mod > > OVERVIEW > > MethodHandles.catchException combinator implementation is based on generic invokers (MethodHandleImpl$GuardWithCatch.invoke_*). It is significantly slower than a Java equivalent (try-catch). > > Future Nashorn performance improvements require catchException combinator speed to be on par with try-catch in Java. > > So, it should be represented in a more efficient form. > > I chose the following lambda form representation: > > t_{i}:L=ValueConversions.array(a1:L,...,a_{k}:L); > t_{i+1}:L=MethodHandleImpl.guardWithCatch(t_{p1}, t_{p2}, t_{p3}, t_{i}:L); > t_{i+2}:I=ValueConversions.unbox(t7:L); > OR :L=ValueConversions.identity(t_{n+1}) > OR :V=ValueConversions.ignore(t_{n+1}) > > where: > a1, ..., a_{k} - arguments > t_{p1}, t_{p2}, t_{p3} - target method handle, exception class, catcher method handle respectively; passed as bounded parameters; > > During lambda form compilation it is converted into bytecode equivalent of the following Java code: > try { > return target.invokeBasic(...); > } catch(Throwable e) { > if (!exClass.isInstance(e)) throw e; > return catcher.invokeBasic(e, ...); > } > > There's a set of microbenchmarks (attached to the bug) I wrote to verify performance characteristics of new implementation. > > FURTHER WORK > > What is missing is lambda form caching. The plan is to have a single lambda form per basic type, but it needs more work - current representation is suitable for sharing on bytecode level, but lambda form interpretation doesn't work well (arguments boxing + result unboxing are problematic). > > TESTING > > Tests: microbenchmarks, jdk/java/lang/invoke/, nashorn with optimistic types (unit tests, octane). > > Tested in 2 modes: > * COMPILE_THRESHOLD=30 > * COMPILE_THRESHOLD=0 -Xverify:all > > OTHER > > 1) Update of cached name and member in LF compilation loop (see InvokerBytecodeGenerator.generateCustomizedCodeBytes) fixes a bug during compilation of selectAlternative when running with COMPILE_THRESHOLD=0. > > 2) As part of this change, I fix existing bug [1], so I add regression test for it. > > Thanks! > > Best regards, > Vladimir Ivanov > > [1] https://bugs.openjdk.java.net/browse/JDK-8034120 From vladimir.x.ivanov at oracle.com Thu Feb 20 15:37:42 2014 From: vladimir.x.ivanov at oracle.com (Vladimir Ivanov) Date: Thu, 20 Feb 2014 19:37:42 +0400 Subject: [9] RFR (M): 8027827: Improve performance of catchException combinator In-Reply-To: References: <53052622.6010507@oracle.com> Message-ID: <53062146.6020507@oracle.com> Thanks, Marcus! Best regards, Vladimir Ivanov On 2/20/14 7:31 PM, Marcus Lagergren wrote: > This looks good, and we have done a significant number of test runs to verify its integrity. > > I say ship it. +1 > > We know that there are some issues with sun.misc.ValueConversion.castReference and similar internal methods not being inlined, but as far as I can understand this is a separate issue that will be addressed. By rewriting a guard for Nashorn to not use castReference in the fast case, I get record indy performance with your catch combinator. > > /M (jdk9 reviewer) > > On 19 Feb 2014, at 22:46, Vladimir Ivanov wrote: > >> http://cr.openjdk.java.net/~vlivanov/8027827/final/webrev.00 >> https://jbs.oracle.com/bugs/browse/JDK-8027827 >> 354 lines changed: 193 ins; 91 del; 70 mod >> >> OVERVIEW >> >> MethodHandles.catchException combinator implementation is based on generic invokers (MethodHandleImpl$GuardWithCatch.invoke_*). It is significantly slower than a Java equivalent (try-catch). >> >> Future Nashorn performance improvements require catchException combinator speed to be on par with try-catch in Java. >> >> So, it should be represented in a more efficient form. >> >> I chose the following lambda form representation: >> >> t_{i}:L=ValueConversions.array(a1:L,...,a_{k}:L); >> t_{i+1}:L=MethodHandleImpl.guardWithCatch(t_{p1}, t_{p2}, t_{p3}, t_{i}:L); >> t_{i+2}:I=ValueConversions.unbox(t7:L); >> OR :L=ValueConversions.identity(t_{n+1}) >> OR :V=ValueConversions.ignore(t_{n+1}) >> >> where: >> a1, ..., a_{k} - arguments >> t_{p1}, t_{p2}, t_{p3} - target method handle, exception class, catcher method handle respectively; passed as bounded parameters; >> >> During lambda form compilation it is converted into bytecode equivalent of the following Java code: >> try { >> return target.invokeBasic(...); >> } catch(Throwable e) { >> if (!exClass.isInstance(e)) throw e; >> return catcher.invokeBasic(e, ...); >> } >> >> There's a set of microbenchmarks (attached to the bug) I wrote to verify performance characteristics of new implementation. >> >> FURTHER WORK >> >> What is missing is lambda form caching. The plan is to have a single lambda form per basic type, but it needs more work - current representation is suitable for sharing on bytecode level, but lambda form interpretation doesn't work well (arguments boxing + result unboxing are problematic). >> >> TESTING >> >> Tests: microbenchmarks, jdk/java/lang/invoke/, nashorn with optimistic types (unit tests, octane). >> >> Tested in 2 modes: >> * COMPILE_THRESHOLD=30 >> * COMPILE_THRESHOLD=0 -Xverify:all >> >> OTHER >> >> 1) Update of cached name and member in LF compilation loop (see InvokerBytecodeGenerator.generateCustomizedCodeBytes) fixes a bug during compilation of selectAlternative when running with COMPILE_THRESHOLD=0. >> >> 2) As part of this change, I fix existing bug [1], so I add regression test for it. >> >> Thanks! >> >> Best regards, >> Vladimir Ivanov >> >> [1] https://bugs.openjdk.java.net/browse/JDK-8034120 > From paul.sandoz at oracle.com Thu Feb 20 15:51:33 2014 From: paul.sandoz at oracle.com (Paul Sandoz) Date: Thu, 20 Feb 2014 16:51:33 +0100 Subject: [9] RFR (M): 8027827: Improve performance of catchException combinator In-Reply-To: <53052622.6010507@oracle.com> References: <53052622.6010507@oracle.com> Message-ID: <0386E0D6-D515-4C2C-9D6E-DD3B53E8B3E2@oracle.com> Hi Vladimir, I know just enough about this area to be dangerous.... src/share/classes/java/lang/invoke/BoundMethodHandle.java 865 private static final SpeciesData[] SPECIES_DATA_CACHE = new SpeciesData[4]; Only 3 elements are stored in the above array? src/share/classes/java/lang/invoke/MethodHandleImpl.java 634 // t_{i+2}:L=ValueConversions.unbox(t_{i+1}) OR ValueConversions.identity(t_{i+1}) 635 if (type.returnType().isPrimitive()) { 636 names[UNBOX_RESULT] = new Name(ValueConversions.unbox(type.returnType()), 637 names[TRY_CATCH]); 638 } else { 639 names[UNBOX_RESULT] = new Name(ValueConversions.identity(), 640 names[TRY_CATCH]); 641 } You could create the form without the identity transform for the non-primitive case? final int UNBOX_RESULT = type.returnType().isPrimitive() ? nameCursor++ : 0; ... if (UNBOX_RESULT > 0) { ... names[UNBOX_RESULT] = new Name(ValueConversions.unbox(type.returnType()), names[TRY_CATCH]); } 699 try { 700 GUARD_WITH_CATCH 701 = IMPL_LOOKUP.findStatic(MethodHandleImpl.class, "guardWithCatch", 702 MethodType.methodType(Object.class, MethodHandle.class, Class.class, MethodHandle.class, Object[].class)); 703 } catch (ReflectiveOperationException ex) { 704 throw new RuntimeException(ex); 705 } 706 return GUARD_WITH_CATCH; Should #704 be: throw newInternalError(ex); ? test/java/lang/invoke/MethodHandles/TestCatchException.java 100 Object o = new Object(); 101 Object[] obj1 = new Object[] { "str" }; 102 103 Object r1 = target.invokeExact(o, o, o, o, o, o, o, o, obj1); 104 Object r2 = gwc.invokeExact(o, o, o, o, o, o, o, o, obj1); 105 assertTrue(r1 != null); 106 assertTrue(r1.equals(r2)); To be on the safe side should probably assert as follows: assertEquals(r1, obj); assertEquals(r2, obj); Paul. On Feb 19, 2014, at 10:46 PM, Vladimir Ivanov wrote: > http://cr.openjdk.java.net/~vlivanov/8027827/final/webrev.00 > https://jbs.oracle.com/bugs/browse/JDK-8027827 > 354 lines changed: 193 ins; 91 del; 70 mod > > OVERVIEW > > MethodHandles.catchException combinator implementation is based on generic invokers (MethodHandleImpl$GuardWithCatch.invoke_*). It is significantly slower than a Java equivalent (try-catch). > > Future Nashorn performance improvements require catchException combinator speed to be on par with try-catch in Java. > > So, it should be represented in a more efficient form. > > I chose the following lambda form representation: > > t_{i}:L=ValueConversions.array(a1:L,...,a_{k}:L); > t_{i+1}:L=MethodHandleImpl.guardWithCatch(t_{p1}, t_{p2}, t_{p3}, t_{i}:L); > t_{i+2}:I=ValueConversions.unbox(t7:L); > OR :L=ValueConversions.identity(t_{n+1}) > OR :V=ValueConversions.ignore(t_{n+1}) > > where: > a1, ..., a_{k} - arguments > t_{p1}, t_{p2}, t_{p3} - target method handle, exception class, catcher method handle respectively; passed as bounded parameters; > > During lambda form compilation it is converted into bytecode equivalent of the following Java code: > try { > return target.invokeBasic(...); > } catch(Throwable e) { > if (!exClass.isInstance(e)) throw e; > return catcher.invokeBasic(e, ...); > } > > There's a set of microbenchmarks (attached to the bug) I wrote to verify performance characteristics of new implementation. > > FURTHER WORK > > What is missing is lambda form caching. The plan is to have a single lambda form per basic type, but it needs more work - current representation is suitable for sharing on bytecode level, but lambda form interpretation doesn't work well (arguments boxing + result unboxing are problematic). > > TESTING > > Tests: microbenchmarks, jdk/java/lang/invoke/, nashorn with optimistic types (unit tests, octane). > > Tested in 2 modes: > * COMPILE_THRESHOLD=30 > * COMPILE_THRESHOLD=0 -Xverify:all > > OTHER > > 1) Update of cached name and member in LF compilation loop (see InvokerBytecodeGenerator.generateCustomizedCodeBytes) fixes a bug during compilation of selectAlternative when running with COMPILE_THRESHOLD=0. > > 2) As part of this change, I fix existing bug [1], so I add regression test for it. > > Thanks! > > Best regards, > Vladimir Ivanov > > [1] https://bugs.openjdk.java.net/browse/JDK-8034120 From peter.levart at gmail.com Thu Feb 20 16:59:10 2014 From: peter.levart at gmail.com (Peter Levart) Date: Thu, 20 Feb 2014 17:59:10 +0100 Subject: RFR - 6857566 (bf) DirectByteBuffer garbage creation can outpace reclamation In-Reply-To: <52F21094.3090400@oracle.com> References: <52F1F174.8070105@gmail.com> <52F21094.3090400@oracle.com> Message-ID: <5306345E.3060302@gmail.com> On 02/05/2014 11:21 AM, Alan Bateman wrote: > On 05/02/2014 08:08, Peter Levart wrote: >> Hi, >> >> I'm proposing a patch for the following issue: >> >> https://bugs.openjdk.java.net/browse/JDK-6857566 >> >> The patch is basically the same as developed and tested as part of >> the following discussion 3 months ago: >> >> http://mail.openjdk.java.net/pipermail/core-libs-dev/2013-October/021547.html >> >> >> It's just modified to be applied on top of JDK9 which already changed >> since then. Here's the webrev: >> >> http://cr.openjdk.java.net/~plevart/jdk9-dev/DirectBufferAlloc/webrev.01/ >> Hi Alan, I made another webrev with minor changes based on your feedback... http://cr.openjdk.java.net/~plevart/jdk9-dev/DirectBufferAlloc/webrev.03/ > Thanks for re-basing the changes and from the previous discussion then > I think there was agreement on the approach. So I'd say this is ready > to go into jdk9/dev. > > A minor comments on the usages of tryHandlePendingReferences in Bits > that it might be a bit neater to not split the line. At least when > looking at the patch then it makes it a bit more obvious what is > condition vs. statement in the while loop. I did that by extracting the reference to JavaLangRefAccess into a local variable. > > Another minor comment is that usually avoid System.exit in tests. It's > probably okay here as it always runs in othervm mode but in other > modes a System.exit(0) will be flagged by jtreg and will require it > spin up another VM. No System.exit in this webrev any more. The test should still run in othervm though, since it uses special VM option "-XX:MaxDirectMemorySize=128m". Without it there might be to much direct memory available by default on some machine (like mine with 16GB of RAM) to provoke an error in 5 seconds time... > > I completely agree with separating the mapped buffers issue to a > separate issue. > > Thanks again for running with this one (it goes back to original > implementation in JDK 1.4). > > -Alan. I think I'll need another official review before pushing this. Regards, Peter From scolebourne at joda.org Thu Feb 20 17:38:52 2014 From: scolebourne at joda.org (Stephen Colebourne) Date: Thu, 20 Feb 2014 17:38:52 +0000 Subject: Type of Class Message-ID: In JDK 5, three methods were added to java.lang.Class [1] - isAnonymousClass() - isLocalClass() - isMemberClass() Unfortunately, these do not cover the complete range of possible types of class. Would it be reasonable to add the following methods: - isNestedClass() - isInnerClass() - isTopLevelClass() While JVM/spec experts may know the difference, it is a surprisingly tricky area. For example, the current isMemberClass() method could do with much better Javadoc to explain what a member class actually is (as it seemed hard to google). FWIW, I was just trying to tell the difference between a nested class and an inner class, in order to determine which classes can be instantiated without reference to a surrounding object. The answer seems to be (cls.isMemberClass() && Modifiers.isStatic(cls.getModifiers()) which is not the most obvious code. In addition, I suffered from the absence of an isNormalClass() - probably a better name for this. Currently, you can determine if a class is an interface, annotation, primitive or array, leaving the "normal" case as a quadruple negative. This leaves such a user-written method vulnerable to any new type of class that gets added in a future JDK: boolean isNormalClass(Class cls) { return !cls.isInterface() && !cls.isAnnotation() && !cls.isPrimitive() && !cls.isArray(); } Stephen [1] https://blogs.oracle.com/darcy/entry/nested_inner_member_and_top From peter.levart at gmail.com Thu Feb 20 17:55:05 2014 From: peter.levart at gmail.com (Peter Levart) Date: Thu, 20 Feb 2014 18:55:05 +0100 Subject: JDK 9 RFR of JDK-8035279: Clean up internal deprecations in BigInteger In-Reply-To: <5305D6EB.30806@oracle.com> References: <5305D6EB.30806@oracle.com> Message-ID: <53064179.2060703@gmail.com> On 02/20/2014 11:20 AM, David Holmes wrote: > Hi Brian, > > On 20/02/2014 7:44 AM, Brian Burkhalter wrote: >> This patch concerns cleaning up some ugly internal deprecations. >> >> Issue: https://bugs.openjdk.java.net/browse/JDK-8035279 >> Webrev: http://cr.openjdk.java.net/~bpb/8035279/webrev.00/ >> >> All JTREG BigInteger tests pass, and the serialized form has been >> unaltered as verified by bidirectional read-write testing between >> Java 7 and this version. >> >> I would appreciate scrutiny of the arithmetic to make sure that I've >> made no silly errors. Also I would be interested in opinions as to >> whether the "volatile" keyword should in fact be used for the >> affected instance variables. > > re: volatile > > The basic approach being taken with these primitive fields is lazy > initialization to a value that will always be the same if recomputed. > If the variables are not volatile then in the worst-case (this is > theoretical only) they will be recomputed by each thread that accesses > them. If they are volatile then they will only be recomputed by > threads that truly race on the initialization logic, but you will > thereafter pay the price of a volatile read on each access. > > Either way you will be functionally correct so it all comes down to > performance. Practically speaking I would expect you to be better off > overall without the volatile. > > As Paul notes the powerCache array does not need to be volatile as it > is initialized as part of class initialization. > > Cheers, > David Hi Brian, I would also be concerned about correctness here. You changed fields to be volatile and assigned them with different that default (zero) values in field initializers. Take for example the field bitCount: | 156 private volatile int bitCount = Integer.MIN_VALUE;| In code you check this initial value and decide whether to lazily compute the real value or not based on this check: 3298 public int bitCount() { 3299 // This computation has a known, acceptable non-critical race condition. 3300 if (bitCount == Integer.MIN_VALUE) { // bitCount not initialized yet There has recently been a discussion on concurrency-dev mailing list about whether this is safe: http://cs.oswego.edu/pipermail/concurrency-interest/2013-November/011951.html ...in cases where the instance of such constructed object escapes to non-constructing thread via data-race. I think the conclusion was that such semantics is desirable and might be guaranteed in a future updated JMM, but currently it is not, although most JVMs are implemented in such a way that this is safe. Citing Doug Lea: " Right. To summarize: * Programmers do not expect that even though final fields are specifically publication-safe, volatile fields are not always so. * For various implementation reasons, JVMs arrange that volatile fields are publication safe anyway, at least in cases we know about. * Actually updating the JMM/JLS to mandate this is not easy (no small tweak that I know applies). But now is a good time to be considering a full revision for JDK9. * In the mean time, it would make sense to further test and validate JVMs as meeting this likely future spec. -Doug " Is there a special reason you changed the "uninitialized" value from default (zero) to non-zero? Regards, Peter From vladimir.x.ivanov at oracle.com Thu Feb 20 17:57:31 2014 From: vladimir.x.ivanov at oracle.com (Vladimir Ivanov) Date: Thu, 20 Feb 2014 21:57:31 +0400 Subject: [9] RFR (M): 8027827: Improve performance of catchException combinator In-Reply-To: <0386E0D6-D515-4C2C-9D6E-DD3B53E8B3E2@oracle.com> References: <53052622.6010507@oracle.com> <0386E0D6-D515-4C2C-9D6E-DD3B53E8B3E2@oracle.com> Message-ID: <5306420B.9040200@oracle.com> Paul, Thanks for the feedback! See my answers inline. Updated webrev: http://cr.openjdk.java.net/~vlivanov/8027827/final/webrev.01/ I finally figured out how to make caching work. This webrev contains these changes. I changed LF representation a bit and added 2 auxiliary method handles - argument boxing and wrapping into Object[] and result unboxing. These operations depend on actual type and can't be shared among arbitrary combinators with the same basic type. They are used only during LF interpretation and are completely ignored in compiled LFs. On 2/20/14 7:51 PM, Paul Sandoz wrote: > Hi Vladimir, > > I know just enough about this area to be dangerous.... > > > src/share/classes/java/lang/invoke/BoundMethodHandle.java > > 865 private static final SpeciesData[] SPECIES_DATA_CACHE = new SpeciesData[4]; > > > Only 3 elements are stored in the above array? Fixed. > > > > src/share/classes/java/lang/invoke/MethodHandleImpl.java > > 634 // t_{i+2}:L=ValueConversions.unbox(t_{i+1}) OR ValueConversions.identity(t_{i+1}) > 635 if (type.returnType().isPrimitive()) { > 636 names[UNBOX_RESULT] = new Name(ValueConversions.unbox(type.returnType()), > 637 names[TRY_CATCH]); > 638 } else { > 639 names[UNBOX_RESULT] = new Name(ValueConversions.identity(), > 640 names[TRY_CATCH]); > 641 } > > > You could create the form without the identity transform for the non-primitive case? > > final int UNBOX_RESULT = type.returnType().isPrimitive() ? nameCursor++ : 0; > ... > > if (UNBOX_RESULT > 0) { ... > names[UNBOX_RESULT] = new Name(ValueConversions.unbox(type.returnType()), names[TRY_CATCH]); > } I can, but it complicates matching and compiling the pattern in InvokerBytecodeGenerator. I decided to keep the shape uniform for all cases. > > > 699 try { > 700 GUARD_WITH_CATCH > 701 = IMPL_LOOKUP.findStatic(MethodHandleImpl.class, "guardWithCatch", > 702 MethodType.methodType(Object.class, MethodHandle.class, Class.class, MethodHandle.class, Object[].class)); > 703 } catch (ReflectiveOperationException ex) { > 704 throw new RuntimeException(ex); > 705 } > 706 return GUARD_WITH_CATCH; > > > Should #704 be: > > throw newInternalError(ex); > > ? Fixed. Also updated other places where RuntimeException was thrown. > > > test/java/lang/invoke/MethodHandles/TestCatchException.java > > 100 Object o = new Object(); > 101 Object[] obj1 = new Object[] { "str" }; > 102 > 103 Object r1 = target.invokeExact(o, o, o, o, o, o, o, o, obj1); > 104 Object r2 = gwc.invokeExact(o, o, o, o, o, o, o, o, obj1); > 105 assertTrue(r1 != null); > 106 assertTrue(r1.equals(r2)); > > To be on the safe side should probably assert as follows: > > assertEquals(r1, obj); > assertEquals(r2, obj); Fixed. Best regards, Vladimir Ivanov > Paul. > > > On Feb 19, 2014, at 10:46 PM, Vladimir Ivanov wrote: > >> http://cr.openjdk.java.net/~vlivanov/8027827/final/webrev.00 >> https://jbs.oracle.com/bugs/browse/JDK-8027827 >> 354 lines changed: 193 ins; 91 del; 70 mod >> >> OVERVIEW >> >> MethodHandles.catchException combinator implementation is based on generic invokers (MethodHandleImpl$GuardWithCatch.invoke_*). It is significantly slower than a Java equivalent (try-catch). >> >> Future Nashorn performance improvements require catchException combinator speed to be on par with try-catch in Java. >> >> So, it should be represented in a more efficient form. >> >> I chose the following lambda form representation: >> >> t_{i}:L=ValueConversions.array(a1:L,...,a_{k}:L); >> t_{i+1}:L=MethodHandleImpl.guardWithCatch(t_{p1}, t_{p2}, t_{p3}, t_{i}:L); >> t_{i+2}:I=ValueConversions.unbox(t7:L); >> OR :L=ValueConversions.identity(t_{n+1}) >> OR :V=ValueConversions.ignore(t_{n+1}) >> >> where: >> a1, ..., a_{k} - arguments >> t_{p1}, t_{p2}, t_{p3} - target method handle, exception class, catcher method handle respectively; passed as bounded parameters; >> >> During lambda form compilation it is converted into bytecode equivalent of the following Java code: >> try { >> return target.invokeBasic(...); >> } catch(Throwable e) { >> if (!exClass.isInstance(e)) throw e; >> return catcher.invokeBasic(e, ...); >> } >> >> There's a set of microbenchmarks (attached to the bug) I wrote to verify performance characteristics of new implementation. >> >> FURTHER WORK >> >> What is missing is lambda form caching. The plan is to have a single lambda form per basic type, but it needs more work - current representation is suitable for sharing on bytecode level, but lambda form interpretation doesn't work well (arguments boxing + result unboxing are problematic). >> >> TESTING >> >> Tests: microbenchmarks, jdk/java/lang/invoke/, nashorn with optimistic types (unit tests, octane). >> >> Tested in 2 modes: >> * COMPILE_THRESHOLD=30 >> * COMPILE_THRESHOLD=0 -Xverify:all >> >> OTHER >> >> 1) Update of cached name and member in LF compilation loop (see InvokerBytecodeGenerator.generateCustomizedCodeBytes) fixes a bug during compilation of selectAlternative when running with COMPILE_THRESHOLD=0. >> >> 2) As part of this change, I fix existing bug [1], so I add regression test for it. >> >> Thanks! >> >> Best regards, >> Vladimir Ivanov >> >> [1] https://bugs.openjdk.java.net/browse/JDK-8034120 > From huizhe.wang at oracle.com Thu Feb 20 18:44:29 2014 From: huizhe.wang at oracle.com (huizhe wang) Date: Thu, 20 Feb 2014 10:44:29 -0800 Subject: RFR: 8032909: XSLT string-length returns incorrect length when string includes complementary chars In-Reply-To: <5306173F.7050608@oracle.com> References: <5306173F.7050608@oracle.com> Message-ID: <53064D0D.5030804@oracle.com> Hi Aleksej, The change looks good. It's good to consolidate test cases into a single test for each encoding. Thanks, Joe On 2/20/2014 6:54 AM, Aleksej Efimov wrote: > Hi, > > There is a problem in XSLT string-length function: > When string passed to this function contains complementary chars the > current implementation of XPath treates this sequence as two > characters (the string-length returns 2). But according to the w3 spec > it should be treated as a single XPath character [1]. > > Please, review the proposed fix for this problem [2]. It replaces the > 'length' call in xslt compiler with a call to 'codePointCount'. Also > new regression test was added. > > To verify the fix following tests were executed: > 1. JCK test sets: api/xsl api/xinclude api/javax_xml api/org_xml > xml_schema > 2. Regression tests from 'jdk_other' testset (includes added > regression test) > > Thanks in advance, > Aleksej > > [1] http://www.w3.org/TR/xpath/#strings > [2] http://cr.openjdk.java.net/~aefimov/8032909/webrev.00 > From martinrb at google.com Thu Feb 20 18:50:29 2014 From: martinrb at google.com (Martin Buchholz) Date: Thu, 20 Feb 2014 10:50:29 -0800 Subject: Serial warnings in java.util.ArrayPrefixHelpers In-Reply-To: <83AFBC4C-EB90-4918-9E2D-1250851F89E9@oracle.com> References: <5304F708.5080800@oracle.com> <530541E1.2000703@oracle.com> <83AFBC4C-EB90-4918-9E2D-1250851F89E9@oracle.com> Message-ID: Chris, Thanks as always for your help! On Thu, Feb 20, 2014 at 4:42 AM, Chris Hegarty wrote: > > On 19 Feb 2014, at 23:44, Joseph Darcy wrote: > > > On 2/19/2014 12:16 PM, Martin Buchholz wrote: > >> The jsr166 CVS version of this file already has serialVersionUIDs added. > >> jsr166 CVS src/main has been jdk8/jdk9 warning-clean for a while now. > > > > Thanks for the update Martin; looking forward to the next sync to get > rid of a few more warnings :-) > > http://hg.openjdk.java.net/jdk9/dev/jdk/rev/2f85b69d1d3f > > -Chris. > > > > > -Joe > > > >> > >> > http://gee.cs.oswego.edu/cgi-bin/viewcvs.cgi/jsr166/src/main/java/util/ArrayPrefixHelpers.java?view=markup > >> > >> yeah, we need to do a better job of down-streaming... > >> > >> > >> On Wed, Feb 19, 2014 at 10:25 AM, Joe Darcy joe.darcy at oracle.com>> wrote: > >> > >> Hello, > >> > >> The file java.util.ArrayPrefixHelpers seems to be part of the > >> 166-alpha set of files in the JDK. > >> > >> There are a number of package-private static final and > >> serializable classes in this file which neither define > >> serialVersionUIDs nor have a @SuppressWarnings("serial") > >> annotation. So, this file adds a number of serial warnings to the > >> JDK and efforts are underway to clear the jdk repo of serial > >> warnings, which are mostly in the client area: > >> > >> JDK-8032976: Fix serial lint warnings in client libraries > >> > >> Can the maintainers of ArrayPrefixHelpers "do the right thing" > >> with respect to the serial warnings in this file? From a quick > >> look at the file, it wasn't clear to me if SUIDs should be added > >> or warnings suppressed. > >> > >> Thanks, > >> > >> -Joe > >> > >> > > > > From stuart.marks at oracle.com Thu Feb 20 20:04:05 2014 From: stuart.marks at oracle.com (Stuart Marks) Date: Thu, 20 Feb 2014 12:04:05 -0800 Subject: JDK 9 RFR of JDK-8035279: Clean up internal deprecations in BigInteger In-Reply-To: <53064179.2060703@gmail.com> References: <5305D6EB.30806@oracle.com> <53064179.2060703@gmail.com> Message-ID: <53065FB5.7010207@oracle.com> On 2/20/14 9:55 AM, Peter Levart wrote: > On 02/20/2014 11:20 AM, David Holmes wrote: >> re: volatile >> >> The basic approach being taken with these primitive fields is lazy >> initialization to a value that will always be the same if recomputed. If the >> variables are not volatile then in the worst-case (this is theoretical only) >> they will be recomputed by each thread that accesses them. If they are >> volatile then they will only be recomputed by threads that truly race on the >> initialization logic, but you will thereafter pay the price of a volatile read >> on each access. >> >> Either way you will be functionally correct so it all comes down to >> performance. Practically speaking I would expect you to be better off overall >> without the volatile. > > I would also be concerned about correctness here. You changed fields to be > volatile and assigned them with different that default (zero) values in field > initializers. Take for example the field bitCount: > > | 156 private volatile int bitCount = Integer.MIN_VALUE;| > > In code you check this initial value and decide whether to lazily compute the > real value or not based on this check: > > 3298 public int bitCount() { > 3299 // This computation has a known, acceptable non-critical race > condition. > 3300 if (bitCount == Integer.MIN_VALUE) { // bitCount not initialized yet > > There has recently been a discussion on concurrency-dev mailing list about > whether this is safe: > > http://cs.oswego.edu/pipermail/concurrency-interest/2013-November/011951.html > > ...in cases where the instance of such constructed object escapes to > non-constructing thread via data-race. I think the conclusion was that such > semantics is desirable and might be guaranteed in a future updated JMM, but > currently it is not, although most JVMs are implemented in such a way that this > is safe. Arrrgh. Yet another hairy JMM discussion .... :-) Brian and I had a discussion about this internally, and I recommended volatile, but mainly because I couldn't convince myself that it was correct in the absence of volatile. But my understanding of the JMM could be wrong (I think I'm in good company here) so I appreciate the additional discussion. In the context of the November concurrency-interest discussion, the phrase "escape ... via data race" implies to me that the object is *not* safely published. My assumption is that if the BigInteger is safely published, there is no issue. If it's not safely published, then other threads could see the default value (zero) of bitCount instead of the sentinel value MIN_VALUE, rely on zero improperly, and end up computing an incorrect result. That would be bad. Making the field volatile would mitigate this issue, or so I thought. > Citing Doug Lea: > > " > Right. To summarize: > > * Programmers do not expect that even though final fields are specifically > publication-safe, volatile fields are not always so. Yes -- this is a surprise to me. Two more data points: - String.hash is not volatile but is potentially written by any thread. This is a similar data race. But it's not initialized at construction time, and the code uses the default value zero as a special case. - Joe Darcy updated the bug report with a pointer to some discussion that occurred several years ago on core-libs-dev, which was the code review of the changes that introduced the code Brian is proposing to change. I'll include the link here for convenience: http://mail.openjdk.java.net/pipermail/core-libs-dev/2009-February/001105.html (I haven't read it yet.) It may shed some light on the current discussion. > Is there a special reason you changed the "uninitialized" value from default > (zero) to non-zero? Zero is a valid value for some (all?) of the fields, so zero can't be used directly as an "uninitialized" sentinel. The previous code offset the value space to allow zero as a sentinel, at a cost of sprinkling +1's, -1's, +2's, and -2's throughout the code. s'marks From david.holmes at oracle.com Thu Feb 20 20:09:30 2014 From: david.holmes at oracle.com (David Holmes) Date: Fri, 21 Feb 2014 06:09:30 +1000 Subject: JDK 9 RFR of JDK-8035279: Clean up internal deprecations in BigInteger In-Reply-To: <53064179.2060703@gmail.com> References: <5305D6EB.30806@oracle.com> <53064179.2060703@gmail.com> Message-ID: <530660FA.1060407@oracle.com> Brian, Peter is absolutely right - I realized my mistake lying in bed last night. If the sentinel values were the default zero values there would be no issue but they are initialized to a different value. If the instances are then shared without a safe publication mechanism then it is possible that threads will see the zero default value and mistake that for an actual field value. So the fields have to be volatile to be guaranteed that their initialized value will be seen. In practice, because there are also final fields in these instances implementations will most likely perform a storestore barrier after construction and prior to setting the reference to the created object. That would make non-volatile appear to work, but it would not be guaranteed to do so and the final-field "freeze" action only guarantees that unsafe publication will see the correct value of the final fields (and for final references anything reachable via that reference). Peter: > * For various implementation reasons, JVMs arrange that > volatile fields are publication safe anyway, at least in > cases we know about. Note that this part of the discussion was shown to be false - it is certainly false in hotspot (except for the new AIX/PPC64 port). Thanks, David On 21/02/2014 3:55 AM, Peter Levart wrote: > > On 02/20/2014 11:20 AM, David Holmes wrote: >> Hi Brian, >> >> On 20/02/2014 7:44 AM, Brian Burkhalter wrote: >>> This patch concerns cleaning up some ugly internal deprecations. >>> >>> Issue: https://bugs.openjdk.java.net/browse/JDK-8035279 >>> Webrev: http://cr.openjdk.java.net/~bpb/8035279/webrev.00/ >>> >>> All JTREG BigInteger tests pass, and the serialized form has been >>> unaltered as verified by bidirectional read-write testing between >>> Java 7 and this version. >>> >>> I would appreciate scrutiny of the arithmetic to make sure that I've >>> made no silly errors. Also I would be interested in opinions as to >>> whether the "volatile" keyword should in fact be used for the >>> affected instance variables. >> >> re: volatile >> >> The basic approach being taken with these primitive fields is lazy >> initialization to a value that will always be the same if recomputed. >> If the variables are not volatile then in the worst-case (this is >> theoretical only) they will be recomputed by each thread that accesses >> them. If they are volatile then they will only be recomputed by >> threads that truly race on the initialization logic, but you will >> thereafter pay the price of a volatile read on each access. >> >> Either way you will be functionally correct so it all comes down to >> performance. Practically speaking I would expect you to be better off >> overall without the volatile. >> >> As Paul notes the powerCache array does not need to be volatile as it >> is initialized as part of class initialization. >> >> Cheers, >> David > > Hi Brian, > > I would also be concerned about correctness here. You changed fields to > be volatile and assigned them with different that default (zero) values > in field initializers. Take for example the field bitCount: > > | 156 private volatile int bitCount = Integer.MIN_VALUE;| > > > In code you check this initial value and decide whether to lazily > compute the real value or not based on this check: > > 3298 public int bitCount() { > 3299 // This computation has a known, acceptable non-critical race condition. > 3300 if (bitCount == Integer.MIN_VALUE) { // bitCount not initialized yet > > > There has recently been a discussion on concurrency-dev mailing list > about whether this is safe: > > http://cs.oswego.edu/pipermail/concurrency-interest/2013-November/011951.html > > ...in cases where the instance of such constructed object escapes to > non-constructing thread via data-race. I think the conclusion was that > such semantics is desirable and might be guaranteed in a future updated > JMM, but currently it is not, although most JVMs are implemented in such > a way that this is safe. Citing Doug Lea: > > " > Right. To summarize: > > * Programmers do not expect that even though final fields are specifically > publication-safe, volatile fields are not always so. > > * For various implementation reasons, JVMs arrange that > volatile fields are publication safe anyway, at least in > cases we know about. > > * Actually updating the JMM/JLS to mandate this is not easy > (no small tweak that I know applies). But now is a good time > to be considering a full revision for JDK9. > > * In the mean time, it would make sense to further test > and validate JVMs as meeting this likely future spec. > > -Doug > " > > Is there a special reason you changed the "uninitialized" value from > default (zero) to non-zero? > > > Regards, Peter > > > > > > > From joe.darcy at oracle.com Thu Feb 20 20:33:41 2014 From: joe.darcy at oracle.com (Joe Darcy) Date: Thu, 20 Feb 2014 12:33:41 -0800 Subject: JDK 9 RFR of JDK-8035452: Fix serial lint warnings in core libs Message-ID: <530666A5.8050607@oracle.com> Hello, Please review the patch below which will address the handful of remaining serial lint warning in the core libraries. Thanks, -Joe diff -r e5a09bc70308 src/share/classes/java/util/EnumSet.java --- a/src/share/classes/java/util/EnumSet.java Thu Feb 20 13:03:36 2014 +0000 +++ b/src/share/classes/java/util/EnumSet.java Thu Feb 20 12:32:52 2014 -0800 @@ -80,6 +80,8 @@ public abstract class EnumSet> extends AbstractSet implements Cloneable, java.io.Serializable { + private static final long serialVersionUID = 1009687484059888093L; + /** * The class of all the elements of this set. */ diff -r e5a09bc70308 src/share/classes/java/util/TreeMap.java --- a/src/share/classes/java/util/TreeMap.java Thu Feb 20 13:03:36 2014 +0000 +++ b/src/share/classes/java/util/TreeMap.java Thu Feb 20 12:32:52 2014 -0800 @@ -1333,6 +1333,7 @@ /** * @serial include */ + @SuppressWarnings("serial") // Abstract class abstract static class NavigableSubMap extends AbstractMap implements NavigableMap, java.io.Serializable { /** diff -r e5a09bc70308 src/share/classes/sun/reflect/annotation/ExceptionProxy.java --- a/src/share/classes/sun/reflect/annotation/ExceptionProxy.java Thu Feb 20 13:03:36 2014 +0000 +++ b/src/share/classes/sun/reflect/annotation/ExceptionProxy.java Thu Feb 20 12:33:21 2014 -0800 @@ -37,5 +37,6 @@ * @since 1.5 */ public abstract class ExceptionProxy implements java.io.Serializable { + private static final long serialVersionUID = 7241930048386631401L; protected abstract RuntimeException generateException(); } From pbenedict at apache.org Thu Feb 20 20:49:33 2014 From: pbenedict at apache.org (Paul Benedict) Date: Thu, 20 Feb 2014 14:49:33 -0600 Subject: JDK 9 RFR of JDK-8035452: Fix serial lint warnings in core libs In-Reply-To: <530666A5.8050607@oracle.com> References: <530666A5.8050607@oracle.com> Message-ID: Joe, I find it interesting that you suppressed the serial warning on an abstract class. I'd like to know more about that. Is this a universal rule? Are serial uids not important for abstract classes? On Thu, Feb 20, 2014 at 2:33 PM, Joe Darcy wrote: > Hello, > > Please review the patch below which will address the handful of remaining > serial lint warning in the core libraries. > > Thanks, > > -Joe > > diff -r e5a09bc70308 src/share/classes/java/util/EnumSet.java > --- a/src/share/classes/java/util/EnumSet.java Thu Feb 20 13:03:36 > 2014 +0000 > +++ b/src/share/classes/java/util/EnumSet.java Thu Feb 20 12:32:52 > 2014 -0800 > @@ -80,6 +80,8 @@ > public abstract class EnumSet> extends AbstractSet > implements Cloneable, java.io.Serializable > { > + private static final long serialVersionUID = 1009687484059888093L; > + > /** > * The class of all the elements of this set. > */ > diff -r e5a09bc70308 src/share/classes/java/util/TreeMap.java > --- a/src/share/classes/java/util/TreeMap.java Thu Feb 20 13:03:36 > 2014 +0000 > +++ b/src/share/classes/java/util/TreeMap.java Thu Feb 20 12:32:52 > 2014 -0800 > @@ -1333,6 +1333,7 @@ > /** > * @serial include > */ > + @SuppressWarnings("serial") // Abstract class > abstract static class NavigableSubMap extends AbstractMap > implements NavigableMap, java.io.Serializable { > /** > diff -r e5a09bc70308 src/share/classes/sun/reflect/ > annotation/ExceptionProxy.java > --- a/src/share/classes/sun/reflect/annotation/ExceptionProxy.java Thu > Feb 20 13:03:36 2014 +0000 > +++ b/src/share/classes/sun/reflect/annotation/ExceptionProxy.java Thu > Feb 20 12:33:21 2014 -0800 > @@ -37,5 +37,6 @@ > * @since 1.5 > */ > public abstract class ExceptionProxy implements java.io.Serializable { > + private static final long serialVersionUID = 7241930048386631401L; > protected abstract RuntimeException generateException(); > } > > -- Cheers, Paul From joe.darcy at oracle.com Thu Feb 20 21:14:59 2014 From: joe.darcy at oracle.com (Joe Darcy) Date: Thu, 20 Feb 2014 13:14:59 -0800 Subject: JDK 9 RFR of JDK-8035453: Fix serial lint warnings in com.sun.tools and elsewhere Message-ID: <53067053.2060501@oracle.com> Hello, Please review my proposed changes for: JDK-8035453: Fix serial lint warnings in com.sun.tools and elsewhere http://cr.openjdk.java.net/~darcy/8035453.0/webrev/ -Joe --- old/src/share/classes/com/sun/java/browser/dom/DOMAccessException.java 2014-02-20 13:03:06.000000000 -0800 +++ new/src/share/classes/com/sun/java/browser/dom/DOMAccessException.java 2014-02-20 13:03:06.000000000 -0800 @@ -1,5 +1,5 @@ /* - * Copyright (c) 2000, Oracle and/or its affiliates. All rights reserved. + * Copyright (c) 2000, 2014, Oracle and/or its affiliates. All rights reserved. * DO NOT ALTER OR REMOVE COPYRIGHT NOTICES OR THIS FILE HEADER. * * This code is free software; you can redistribute it and/or modify it @@ -25,6 +25,7 @@ package com.sun.java.browser.dom; + at SuppressWarnings("serial") // JDK implementation class public class DOMAccessException extends Exception { /** --- old/src/share/classes/com/sun/java/browser/dom/DOMUnsupportedException.java 2014-02-20 13:03:06.000000000 -0800 +++ new/src/share/classes/com/sun/java/browser/dom/DOMUnsupportedException.java 2014-02-20 13:03:06.000000000 -0800 @@ -1,5 +1,5 @@ /* - * Copyright (c) 2000, Oracle and/or its affiliates. All rights reserved. + * Copyright (c) 2000, 2014, Oracle and/or its affiliates. All rights reserved. * DO NOT ALTER OR REMOVE COPYRIGHT NOTICES OR THIS FILE HEADER. * * This code is free software; you can redistribute it and/or modify it @@ -25,7 +25,7 @@ package com.sun.java.browser.dom; - + at SuppressWarnings("serial") // JDK implementation class public class DOMUnsupportedException extends Exception { /** --- old/src/share/classes/com/sun/tools/example/debug/expr/ExpressionParser.java 2014-02-20 13:03:07.000000000 -0800 +++ new/src/share/classes/com/sun/tools/example/debug/expr/ExpressionParser.java 2014-02-20 13:03:07.000000000 -0800 @@ -1,5 +1,5 @@ /* - * Copyright (c) 1999, 2013, Oracle and/or its affiliates. All rights reserved. + * Copyright (c) 1999, 2014, Oracle and/or its affiliates. All rights reserved. * DO NOT ALTER OR REMOVE COPYRIGHT NOTICES OR THIS FILE HEADER. * * This code is free software; you can redistribute it and/or modify it @@ -2172,6 +2172,7 @@ throw generateParseException(); } + @SuppressWarnings("serial") // JDK implementation class static private final class LookaheadSuccess extends java.lang.Error { } final private LookaheadSuccess jj_ls = new LookaheadSuccess(); private boolean jj_scan_token(int kind) { --- old/src/share/classes/com/sun/tools/hat/internal/oql/OQLException.java 2014-02-20 13:03:07.000000000 -0800 +++ new/src/share/classes/com/sun/tools/hat/internal/oql/OQLException.java 2014-02-20 13:03:07.000000000 -0800 @@ -1,5 +1,5 @@ /* - * Copyright (c) 1997, 2008, Oracle and/or its affiliates. All rights reserved. + * Copyright (c) 1997, 2014, Oracle and/or its affiliates. All rights reserved. * DO NOT ALTER OR REMOVE COPYRIGHT NOTICES OR THIS FILE HEADER. * * This code is free software; you can redistribute it and/or modify it @@ -36,6 +36,7 @@ * OQLException is thrown if OQL execution results in error * */ + at SuppressWarnings("serial") // JDK implementation class public class OQLException extends Exception { public OQLException(String msg) { super(msg); --- old/src/share/classes/com/sun/tools/jdi/ConnectorImpl.java 2014-02-20 13:03:08.000000000 -0800 +++ new/src/share/classes/com/sun/tools/jdi/ConnectorImpl.java 2014-02-20 13:03:08.000000000 -0800 @@ -144,6 +144,7 @@ return string; } + @SuppressWarnings("serial") // JDK implementation class abstract class ArgumentImpl implements Connector.Argument, Cloneable, Serializable { private String name; private String label; --- old/src/share/classes/jdk/internal/org/objectweb/asm/tree/MethodNode.java 2014-02-20 13:03:08.000000000 -0800 +++ new/src/share/classes/jdk/internal/org/objectweb/asm/tree/MethodNode.java 2014-02-20 13:03:08.000000000 -0800 @@ -351,6 +351,7 @@ } @Override + @SuppressWarnings("serial") // Anonymous class public AnnotationVisitor visitAnnotationDefault() { return new AnnotationNode(new ArrayList(0) { @Override --- old/src/share/classes/jdk/internal/org/objectweb/asm/tree/analysis/AnalyzerException.java 2014-02-20 13:03:09.000000000 -0800 +++ new/src/share/classes/jdk/internal/org/objectweb/asm/tree/analysis/AnalyzerException.java 2014-02-20 13:03:09.000000000 -0800 @@ -66,6 +66,7 @@ * @author Bing Ran * @author Eric Bruneton */ + at SuppressWarnings("serial") // JDK-implementation class public class AnalyzerException extends Exception { public final AbstractInsnNode node; --- old/src/share/classes/sun/jvmstat/monitor/MonitorException.java 2014-02-20 13:03:10.000000000 -0800 +++ new/src/share/classes/sun/jvmstat/monitor/MonitorException.java 2014-02-20 13:03:10.000000000 -0800 @@ -1,5 +1,5 @@ /* - * Copyright (c) 2004, Oracle and/or its affiliates. All rights reserved. + * Copyright (c) 2004, 2014, Oracle and/or its affiliates. All rights reserved. * DO NOT ALTER OR REMOVE COPYRIGHT NOTICES OR THIS FILE HEADER. * * This code is free software; you can redistribute it and/or modify it @@ -32,6 +32,7 @@ * @author Brian Doherty * @since 1.5 */ + at SuppressWarnings("serial") // JDK implementation class public class MonitorException extends Exception { /** --- old/src/share/classes/sun/jvmstat/monitor/event/HostEvent.java 2014-02-20 13:03:10.000000000 -0800 +++ new/src/share/classes/sun/jvmstat/monitor/event/HostEvent.java 2014-02-20 13:03:10.000000000 -0800 @@ -1,5 +1,5 @@ /* - * Copyright (c) 2004, Oracle and/or its affiliates. All rights reserved. + * Copyright (c) 2004, 2014, Oracle and/or its affiliates. All rights reserved. * DO NOT ALTER OR REMOVE COPYRIGHT NOTICES OR THIS FILE HEADER. * * This code is free software; you can redistribute it and/or modify it @@ -34,6 +34,7 @@ * @author Brian Doherty * @since 1.5 */ + at SuppressWarnings("serial") // JDK implementation class public class HostEvent extends EventObject { /** --- old/src/share/classes/sun/jvmstat/monitor/event/MonitorStatusChangeEvent.java 2014-02-20 13:03:11.000000000 -0800 +++ new/src/share/classes/sun/jvmstat/monitor/event/MonitorStatusChangeEvent.java 2014-02-20 13:03:11.000000000 -0800 @@ -1,5 +1,5 @@ /* - * Copyright (c) 2004, Oracle and/or its affiliates. All rights reserved. + * Copyright (c) 2004, 2014, Oracle and/or its affiliates. All rights reserved. * DO NOT ALTER OR REMOVE COPYRIGHT NOTICES OR THIS FILE HEADER. * * This code is free software; you can redistribute it and/or modify it @@ -35,6 +35,7 @@ * @author Brian Doherty * @since 1.5 */ + at SuppressWarnings("serial") // JDK implementation class public class MonitorStatusChangeEvent extends VmEvent { /** --- old/src/share/classes/sun/jvmstat/monitor/event/VmEvent.java 2014-02-20 13:03:12.000000000 -0800 +++ new/src/share/classes/sun/jvmstat/monitor/event/VmEvent.java 2014-02-20 13:03:12.000000000 -0800 @@ -1,5 +1,5 @@ /* - * Copyright (c) 2004, Oracle and/or its affiliates. All rights reserved. + * Copyright (c) 2004, 2014, Oracle and/or its affiliates. All rights reserved. * DO NOT ALTER OR REMOVE COPYRIGHT NOTICES OR THIS FILE HEADER. * * This code is free software; you can redistribute it and/or modify it @@ -34,6 +34,7 @@ * @author Brian Doherty * @since 1.5 */ + at SuppressWarnings("serial") // JDK implementation class public class VmEvent extends EventObject { /** --- old/src/share/classes/sun/jvmstat/monitor/event/VmStatusChangeEvent.java 2014-02-20 13:03:12.000000000 -0800 +++ new/src/share/classes/sun/jvmstat/monitor/event/VmStatusChangeEvent.java 2014-02-20 13:03:12.000000000 -0800 @@ -1,5 +1,5 @@ /* - * Copyright (c) 2004, Oracle and/or its affiliates. All rights reserved. + * Copyright (c) 2004, 2014, Oracle and/or its affiliates. All rights reserved. * DO NOT ALTER OR REMOVE COPYRIGHT NOTICES OR THIS FILE HEADER. * * This code is free software; you can redistribute it and/or modify it @@ -35,6 +35,7 @@ * @author Brian Doherty * @since 1.5 */ + at SuppressWarnings("serial") // JDK implementation class public class VmStatusChangeEvent extends HostEvent { /** --- old/src/share/classes/sun/jvmstat/perfdata/monitor/MonitorDataException.java 2014-02-20 13:03:13.000000000 -0800 +++ new/src/share/classes/sun/jvmstat/perfdata/monitor/MonitorDataException.java 2014-02-20 13:03:13.000000000 -0800 @@ -1,5 +1,5 @@ /* - * Copyright (c) 2004, Oracle and/or its affiliates. All rights reserved. + * Copyright (c) 2004, 2014, Oracle and/or its affiliates. All rights reserved. * DO NOT ALTER OR REMOVE COPYRIGHT NOTICES OR THIS FILE HEADER. * * This code is free software; you can redistribute it and/or modify it @@ -34,6 +34,7 @@ * @author Brian Doherty * @since 1.5 */ + at SuppressWarnings("serial") // JDK implementation class public class MonitorDataException extends MonitorException { /** --- old/src/share/classes/sun/jvmstat/perfdata/monitor/MonitorStructureException.java 2014-02-20 13:03:14.000000000 -0800 +++ new/src/share/classes/sun/jvmstat/perfdata/monitor/MonitorStructureException.java 2014-02-20 13:03:14.000000000 -0800 @@ -1,5 +1,5 @@ /* - * Copyright (c) 2004, Oracle and/or its affiliates. All rights reserved. + * Copyright (c) 2004, 2014, Oracle and/or its affiliates. All rights reserved. * DO NOT ALTER OR REMOVE COPYRIGHT NOTICES OR THIS FILE HEADER. * * This code is free software; you can redistribute it and/or modify it @@ -34,6 +34,7 @@ * @author Brian Doherty * @since 1.5 */ + at SuppressWarnings("serial") // JDK implementation class public class MonitorStructureException extends MonitorException { /** --- old/src/share/classes/sun/jvmstat/perfdata/monitor/MonitorTypeException.java 2014-02-20 13:03:14.000000000 -0800 +++ new/src/share/classes/sun/jvmstat/perfdata/monitor/MonitorTypeException.java 2014-02-20 13:03:14.000000000 -0800 @@ -1,5 +1,5 @@ /* - * Copyright (c) 2004, Oracle and/or its affiliates. All rights reserved. + * Copyright (c) 2004, 2014, Oracle and/or its affiliates. All rights reserved. * DO NOT ALTER OR REMOVE COPYRIGHT NOTICES OR THIS FILE HEADER. * * This code is free software; you can redistribute it and/or modify it @@ -34,6 +34,7 @@ * @author Brian Doherty * @since 1.5 */ + at SuppressWarnings("serial") // JDK implementation class public class MonitorTypeException extends MonitorException { /** --- old/src/share/classes/sun/jvmstat/perfdata/monitor/MonitorVersionException.java 2014-02-20 13:03:15.000000000 -0800 +++ new/src/share/classes/sun/jvmstat/perfdata/monitor/MonitorVersionException.java 2014-02-20 13:03:15.000000000 -0800 @@ -1,5 +1,5 @@ /* - * Copyright (c) 2004, Oracle and/or its affiliates. All rights reserved. + * Copyright (c) 2004, 2014, Oracle and/or its affiliates. All rights reserved. * DO NOT ALTER OR REMOVE COPYRIGHT NOTICES OR THIS FILE HEADER. * * This code is free software; you can redistribute it and/or modify it @@ -35,6 +35,7 @@ * @author Brian Doherty * @since 1.5 */ + at SuppressWarnings("serial") // JDK implementation class public class MonitorVersionException extends MonitorException { /** --- old/src/share/classes/sun/jvmstat/perfdata/monitor/SyntaxException.java 2014-02-20 13:03:16.000000000 -0800 +++ new/src/share/classes/sun/jvmstat/perfdata/monitor/SyntaxException.java 2014-02-20 13:03:16.000000000 -0800 @@ -1,5 +1,5 @@ /* - * Copyright (c) 2004, Oracle and/or its affiliates. All rights reserved. + * Copyright (c) 2004, 2014, Oracle and/or its affiliates. All rights reserved. * DO NOT ALTER OR REMOVE COPYRIGHT NOTICES OR THIS FILE HEADER. * * This code is free software; you can redistribute it and/or modify it @@ -36,6 +36,7 @@ * @author Brian Doherty * @since 1.5 */ + at SuppressWarnings("serial") // JDK implementation class public class SyntaxException extends Exception { int lineno; From paul.sandoz at oracle.com Thu Feb 20 21:20:07 2014 From: paul.sandoz at oracle.com (Paul Sandoz) Date: Thu, 20 Feb 2014 22:20:07 +0100 Subject: JDK 9 RFR of JDK-8035279: Clean up internal deprecations in BigInteger In-Reply-To: <530660FA.1060407@oracle.com> References: <5305D6EB.30806@oracle.com> <53064179.2060703@gmail.com> <530660FA.1060407@oracle.com> Message-ID: On Feb 20, 2014, at 9:09 PM, David Holmes wrote: > In practice, because there are also final fields in these instances implementations will most likely perform a storestore barrier after construction and prior to setting the reference to the created object. Yes, that is what i was banking on. That is how hotspot behaves, right? But, you are saying other VMs might not do so? Paul. > That would make non-volatile appear to work, but it would not be guaranteed to do so and the final-field "freeze" action only guarantees that unsafe publication will see the correct value of the final fields (and for final references anything reachable via that reference). > From david.holmes at oracle.com Thu Feb 20 22:11:16 2014 From: david.holmes at oracle.com (David Holmes) Date: Fri, 21 Feb 2014 08:11:16 +1000 Subject: JDK 9 RFR of JDK-8035279: Clean up internal deprecations in BigInteger In-Reply-To: References: <5305D6EB.30806@oracle.com> <53064179.2060703@gmail.com> <530660FA.1060407@oracle.com> Message-ID: <53067D84.5060500@oracle.com> On 21/02/2014 7:20 AM, Paul Sandoz wrote: > On Feb 20, 2014, at 9:09 PM, David Holmes wrote: >> In practice, because there are also final fields in these instances implementations will most likely perform a storestore barrier after construction and prior to setting the reference to the created object. > > Yes, that is what i was banking on. > > That is how hotspot behaves, right? Yes. > But, you are saying other VMs might not do so? Right. A storestore at the end of construction is a simple and obvious way to implement the final-field semantics, but there may be other ways. The visibility guarantees for final fields extends only to those fields, and objects accessed through them. For illustrative purposes only: you could move all the lazy-initialized fields into a separate State object, which is accessed via a final field, and then they could be safely initialized and would not need to be volatile. David ----- > Paul. > >> That would make non-volatile appear to work, but it would not be guaranteed to do so and the final-field "freeze" action only guarantees that unsafe publication will see the correct value of the final fields (and for final references anything reachable via that reference). >> > > From dl at cs.oswego.edu Thu Feb 20 23:52:28 2014 From: dl at cs.oswego.edu (Doug Lea) Date: Thu, 20 Feb 2014 18:52:28 -0500 Subject: JDK 9 RFR of JDK-8035279: Clean up internal deprecations in BigInteger In-Reply-To: <53067D84.5060500@oracle.com> References: <5305D6EB.30806@oracle.com> <53064179.2060703@gmail.com> <530660FA.1060407@oracle.com> <53067D84.5060500@oracle.com> Message-ID: <5306953C.4030007@cs.oswego.edu> On 02/20/2014 05:11 PM, David Holmes wrote: > On 21/02/2014 7:20 AM, Paul Sandoz wrote: >> On Feb 20, 2014, at 9:09 PM, David Holmes wrote: >>> In practice, because there are also final fields in these instances >>> implementations will most likely perform a storestore barrier after >>> construction and prior to setting the reference to the created object. >> >> Yes, that is what i was banking on. >> >> That is how hotspot behaves, right? > > Yes. > >> But, you are saying other VMs might not do so? > > Right. A storestore at the end of construction is a simple and obvious way to > implement the final-field semantics, but there may be other ways. The visibility > guarantees for final fields extends only to those fields, and objects accessed > through them. Which has turned out in practice to be the only plausible way to do this, so the spec might as well have said so. Paul: If you'd like to be pedantically correct at the expense of a lot more overhead, you could use AtomicXFieldUpdater.lazySet in the constructor. But since these changes are targeting JDK9, it is very likely that the memory model will be updated to confirm that the current approach is legal by the time it ships. Or maybe David has some alternative idea that won't slow things down a lot for no good reason :-) -Doug From john.r.rose at oracle.com Fri Feb 21 00:28:46 2014 From: john.r.rose at oracle.com (John Rose) Date: Thu, 20 Feb 2014 16:28:46 -0800 Subject: JDK 9 RFR of JDK-8035279: Clean up internal deprecations in BigInteger In-Reply-To: <530660FA.1060407@oracle.com> References: <5305D6EB.30806@oracle.com> <53064179.2060703@gmail.com> <530660FA.1060407@oracle.com> Message-ID: <96CF3236-4E65-4D64-BCF3-1E9660D29BAA@oracle.com> On Feb 20, 2014, at 12:09 PM, David Holmes wrote: > If the sentinel values were the default zero values there would be no issue This is an instance of the "stable value" pattern; see the javadoc for java/lang/invoke/Stable. Using zero for the sentinel in this pattern is the safest option. If I'm reading the JMM argument right, using a non-default value (perhaps even an explicitly set zero value) relies on accidental effects from nearby final variables, which is fragile even if correct on HotSpot or in a future JMM. If zero is also a valid result, and the "+1" values everywhere are odious, make a wrapper method that returns the corrected value, and applies the correction in just one place. That is much preferable to risking an initial race condition on your stable value. ? John From stuart.marks at oracle.com Fri Feb 21 02:02:14 2014 From: stuart.marks at oracle.com (Stuart Marks) Date: Thu, 20 Feb 2014 18:02:14 -0800 Subject: RFR(s): 8027810 RMI CheckUsage tests should be deleted Message-ID: <5306B3A6.7090904@oracle.com> Hi all, There are a couple of RMI tests that do nothing except check the usage message emitted by the 'rmiregistry' and 'rmid' commands when given invalid command-line options. Since two bugs have occurred in these tests, I have determined that the overhead of maintaining these tests outweighs their usefulness, and I'm therefore removing them. Bug: https://bugs.openjdk.java.net/browse/jdk-8027810 Webrev: http://cr.openjdk.java.net/~smarks/reviews/8027810/webrev.0/ Thanks, s'marks From david.holmes at oracle.com Fri Feb 21 02:28:32 2014 From: david.holmes at oracle.com (David Holmes) Date: Fri, 21 Feb 2014 12:28:32 +1000 Subject: JDK 9 RFR of JDK-8035279: Clean up internal deprecations in BigInteger In-Reply-To: <5306953C.4030007@cs.oswego.edu> References: <5305D6EB.30806@oracle.com> <53064179.2060703@gmail.com> <530660FA.1060407@oracle.com> <53067D84.5060500@oracle.com> <5306953C.4030007@cs.oswego.edu> Message-ID: <5306B9D0.1090906@oracle.com> On 21/02/2014 9:52 AM, Doug Lea wrote: > On 02/20/2014 05:11 PM, David Holmes wrote: >> On 21/02/2014 7:20 AM, Paul Sandoz wrote: >>> On Feb 20, 2014, at 9:09 PM, David Holmes >>> wrote: >>>> In practice, because there are also final fields in these instances >>>> implementations will most likely perform a storestore barrier after >>>> construction and prior to setting the reference to the created object. >>> >>> Yes, that is what i was banking on. >>> >>> That is how hotspot behaves, right? >> >> Yes. >> >>> But, you are saying other VMs might not do so? >> >> Right. A storestore at the end of construction is a simple and obvious >> way to >> implement the final-field semantics, but there may be other ways. The >> visibility >> guarantees for final fields extends only to those fields, and objects >> accessed >> through them. > > Which has turned out in practice to be the only plausible way to do > this, so the spec might as well have said so. Hindsight is a wonderful thing. :) > > Paul: If you'd like to be pedantically correct at the > expense of a lot more overhead, you could use > AtomicXFieldUpdater.lazySet in the constructor. ??? The fields have to be volatile to use that. If they are volatile then there is no problem. > But since these changes are targeting JDK9, it is very > likely that the memory model will be updated to > confirm that the current approach is legal by the time > it ships. By "current approach" I assume you mean the non-volatile variant? So we expect that for JDK 9 either: a) all constructors complete with a storestore; or b) all constructors with final field writes complete with a storestore; ? > Or maybe David has some alternative idea that > won't slow things down a lot for no good reason :-) volatile reads will slow things down a lot? David ----- > -Doug > > From mandy.chung at oracle.com Fri Feb 21 02:36:16 2014 From: mandy.chung at oracle.com (Mandy Chung) Date: Thu, 20 Feb 2014 18:36:16 -0800 Subject: RFR(s): 8027810 RMI CheckUsage tests should be deleted In-Reply-To: <5306B3A6.7090904@oracle.com> References: <5306B3A6.7090904@oracle.com> Message-ID: <5306BBA0.1090301@oracle.com> Thumbs up. FYI - jdeps follows the langtools convention to define a static int run(String... args) method that doesn't exit but instead returns the error exit code (whereas the main method will exit with an error code) so that the test can test the tool with different options in the same VM and avoid launching another VM. In any case, I agree these 2 tests are not as useful. Mandy On 2/20/2014 6:02 PM, Stuart Marks wrote: > Hi all, > > There are a couple of RMI tests that do nothing except check the usage > message emitted by the 'rmiregistry' and 'rmid' commands when given > invalid command-line options. Since two bugs have occurred in these > tests, I have determined that the overhead of maintaining these tests > outweighs their usefulness, and I'm therefore removing them. > > Bug: > > https://bugs.openjdk.java.net/browse/jdk-8027810 > > Webrev: > > http://cr.openjdk.java.net/~smarks/reviews/8027810/webrev.0/ > > Thanks, > > s'marks From david.holmes at oracle.com Fri Feb 21 02:39:07 2014 From: david.holmes at oracle.com (David Holmes) Date: Fri, 21 Feb 2014 12:39:07 +1000 Subject: JDK 9 RFR of JDK-8035279: Clean up internal deprecations in BigInteger In-Reply-To: <96CF3236-4E65-4D64-BCF3-1E9660D29BAA@oracle.com> References: <5305D6EB.30806@oracle.com> <53064179.2060703@gmail.com> <530660FA.1060407@oracle.com> <96CF3236-4E65-4D64-BCF3-1E9660D29BAA@oracle.com> Message-ID: <5306BC4B.4090909@oracle.com> On 21/02/2014 10:28 AM, John Rose wrote: > On Feb 20, 2014, at 12:09 PM, David Holmes > wrote: > >> If the sentinel values were the default zero values there would be no >> issue > > This is an instance of the "stable value" pattern; see the javadoc for > java/lang/invoke/Stable. > > Using zero for the sentinel in this pattern is the safest option. By the definition of the "pattern" it seems to be the only option. *

* Since all fields begin with a default value of null for references * (resp., zero for primitives), it follows that this annotation indicates * that the first non-null (resp., non-zero) value stored in the field * will never be changed. > If I'm reading the JMM argument right, using a non-default value > (perhaps even an explicitly set zero value) relies on accidental effects > from nearby final variables, which is fragile even if correct on HotSpot > or in a future JMM. A zero would be fine, even if explicitly set. The basic issue is that under the JMM, given that over time there will be three potential values for these fields, in the absence of any synchronization actions, the initial read of the field by any thread is allowed to return any of the values written up to that point in time. In practice the final-field related storestore barrier will reduce the set of potential values to two (dropping the default zero value). > If zero is also a valid result, and the "+1" values everywhere are > odious, make a wrapper method that returns the corrected value, and > applies the correction in just one place. That is much preferable to > risking an initial race condition on your stable value. Not clear what you mean by this. David ----- > ? John From stuart.marks at oracle.com Fri Feb 21 05:31:08 2014 From: stuart.marks at oracle.com (Stuart Marks) Date: Thu, 20 Feb 2014 21:31:08 -0800 Subject: RFR(s): 8027810 RMI CheckUsage tests should be deleted In-Reply-To: <5306BBA0.1090301@oracle.com> References: <5306B3A6.7090904@oracle.com> <5306BBA0.1090301@oracle.com> Message-ID: <5306E49C.9060308@oracle.com> OK, thanks. Unfortunately I don't think the "static int run()" technique can be applied to rmiregistry and rmid, since a significant part of testing them -- apart from their usage messages :-) -- involves operating within a fairly strict security policy. But it's good to keep this in mind. s'marks On 2/20/14 6:36 PM, Mandy Chung wrote: > Thumbs up. > > FYI - jdeps follows the langtools convention to define a static int > run(String... args) method that doesn't exit but instead returns the error exit > code (whereas the main method will exit with an error code) so that the test can > test the tool with different options in the same VM and avoid launching another > VM. In any case, I agree these 2 tests are not as useful. > > Mandy > > On 2/20/2014 6:02 PM, Stuart Marks wrote: >> Hi all, >> >> There are a couple of RMI tests that do nothing except check the usage message >> emitted by the 'rmiregistry' and 'rmid' commands when given invalid >> command-line options. Since two bugs have occurred in these tests, I have >> determined that the overhead of maintaining these tests outweighs their >> usefulness, and I'm therefore removing them. >> >> Bug: >> >> https://bugs.openjdk.java.net/browse/jdk-8027810 >> >> Webrev: >> >> http://cr.openjdk.java.net/~smarks/reviews/8027810/webrev.0/ >> >> Thanks, >> >> s'marks > From marcus.lagergren at oracle.com Fri Feb 21 06:53:38 2014 From: marcus.lagergren at oracle.com (Marcus Lagergren) Date: Fri, 21 Feb 2014 07:53:38 +0100 Subject: [9] RFR (M): 8027827: Improve performance of catchException combinator In-Reply-To: <5306420B.9040200@oracle.com> References: <53052622.6010507@oracle.com> <0386E0D6-D515-4C2C-9D6E-DD3B53E8B3E2@oracle.com> <5306420B.9040200@oracle.com> Message-ID: <4CB451F9-872B-40BE-AE17-11B9EF3F874A@oracle.com> Got catches, Paul! /M On 20 Feb 2014, at 18:57, Vladimir Ivanov wrote: > Paul, > > Thanks for the feedback! See my answers inline. > > Updated webrev: > http://cr.openjdk.java.net/~vlivanov/8027827/final/webrev.01/ > > I finally figured out how to make caching work. This webrev contains these changes. > > I changed LF representation a bit and added 2 auxiliary method handles - argument boxing and wrapping into Object[] and result unboxing. These operations depend on actual type and can't be shared among arbitrary combinators with the same basic type. They are used only during LF interpretation and are completely ignored in compiled LFs. > > On 2/20/14 7:51 PM, Paul Sandoz wrote: >> Hi Vladimir, >> >> I know just enough about this area to be dangerous.... >> >> >> src/share/classes/java/lang/invoke/BoundMethodHandle.java >> >> 865 private static final SpeciesData[] SPECIES_DATA_CACHE = new SpeciesData[4]; >> >> >> Only 3 elements are stored in the above array? > Fixed. > >> >> >> >> src/share/classes/java/lang/invoke/MethodHandleImpl.java >> >> 634 // t_{i+2}:L=ValueConversions.unbox(t_{i+1}) OR ValueConversions.identity(t_{i+1}) >> 635 if (type.returnType().isPrimitive()) { >> 636 names[UNBOX_RESULT] = new Name(ValueConversions.unbox(type.returnType()), >> 637 names[TRY_CATCH]); >> 638 } else { >> 639 names[UNBOX_RESULT] = new Name(ValueConversions.identity(), >> 640 names[TRY_CATCH]); >> 641 } >> >> >> You could create the form without the identity transform for the non-primitive case? >> >> final int UNBOX_RESULT = type.returnType().isPrimitive() ? nameCursor++ : 0; >> ... >> >> if (UNBOX_RESULT > 0) { ... >> names[UNBOX_RESULT] = new Name(ValueConversions.unbox(type.returnType()), names[TRY_CATCH]); >> } > I can, but it complicates matching and compiling the pattern in InvokerBytecodeGenerator. I decided to keep the shape uniform for all cases. > >> >> >> 699 try { >> 700 GUARD_WITH_CATCH >> 701 = IMPL_LOOKUP.findStatic(MethodHandleImpl.class, "guardWithCatch", >> 702 MethodType.methodType(Object.class, MethodHandle.class, Class.class, MethodHandle.class, Object[].class)); >> 703 } catch (ReflectiveOperationException ex) { >> 704 throw new RuntimeException(ex); >> 705 } >> 706 return GUARD_WITH_CATCH; >> >> >> Should #704 be: >> >> throw newInternalError(ex); >> >> ? > Fixed. Also updated other places where RuntimeException was thrown. > >> >> >> test/java/lang/invoke/MethodHandles/TestCatchException.java >> >> 100 Object o = new Object(); >> 101 Object[] obj1 = new Object[] { "str" }; >> 102 >> 103 Object r1 = target.invokeExact(o, o, o, o, o, o, o, o, obj1); >> 104 Object r2 = gwc.invokeExact(o, o, o, o, o, o, o, o, obj1); >> 105 assertTrue(r1 != null); >> 106 assertTrue(r1.equals(r2)); >> >> To be on the safe side should probably assert as follows: >> >> assertEquals(r1, obj); >> assertEquals(r2, obj); > Fixed. > > Best regards, > Vladimir Ivanov > >> Paul. >> >> >> On Feb 19, 2014, at 10:46 PM, Vladimir Ivanov wrote: >> >>> http://cr.openjdk.java.net/~vlivanov/8027827/final/webrev.00 >>> https://jbs.oracle.com/bugs/browse/JDK-8027827 >>> 354 lines changed: 193 ins; 91 del; 70 mod >>> >>> OVERVIEW >>> >>> MethodHandles.catchException combinator implementation is based on generic invokers (MethodHandleImpl$GuardWithCatch.invoke_*). It is significantly slower than a Java equivalent (try-catch). >>> >>> Future Nashorn performance improvements require catchException combinator speed to be on par with try-catch in Java. >>> >>> So, it should be represented in a more efficient form. >>> >>> I chose the following lambda form representation: >>> >>> t_{i}:L=ValueConversions.array(a1:L,...,a_{k}:L); >>> t_{i+1}:L=MethodHandleImpl.guardWithCatch(t_{p1}, t_{p2}, t_{p3}, t_{i}:L); >>> t_{i+2}:I=ValueConversions.unbox(t7:L); >>> OR :L=ValueConversions.identity(t_{n+1}) >>> OR :V=ValueConversions.ignore(t_{n+1}) >>> >>> where: >>> a1, ..., a_{k} - arguments >>> t_{p1}, t_{p2}, t_{p3} - target method handle, exception class, catcher method handle respectively; passed as bounded parameters; >>> >>> During lambda form compilation it is converted into bytecode equivalent of the following Java code: >>> try { >>> return target.invokeBasic(...); >>> } catch(Throwable e) { >>> if (!exClass.isInstance(e)) throw e; >>> return catcher.invokeBasic(e, ...); >>> } >>> >>> There's a set of microbenchmarks (attached to the bug) I wrote to verify performance characteristics of new implementation. >>> >>> FURTHER WORK >>> >>> What is missing is lambda form caching. The plan is to have a single lambda form per basic type, but it needs more work - current representation is suitable for sharing on bytecode level, but lambda form interpretation doesn't work well (arguments boxing + result unboxing are problematic). >>> >>> TESTING >>> >>> Tests: microbenchmarks, jdk/java/lang/invoke/, nashorn with optimistic types (unit tests, octane). >>> >>> Tested in 2 modes: >>> * COMPILE_THRESHOLD=30 >>> * COMPILE_THRESHOLD=0 -Xverify:all >>> >>> OTHER >>> >>> 1) Update of cached name and member in LF compilation loop (see InvokerBytecodeGenerator.generateCustomizedCodeBytes) fixes a bug during compilation of selectAlternative when running with COMPILE_THRESHOLD=0. >>> >>> 2) As part of this change, I fix existing bug [1], so I add regression test for it. >>> >>> Thanks! >>> >>> Best regards, >>> Vladimir Ivanov >>> >>> [1] https://bugs.openjdk.java.net/browse/JDK-8034120 From david.holmes at oracle.com Fri Feb 21 08:14:58 2014 From: david.holmes at oracle.com (David Holmes) Date: Fri, 21 Feb 2014 18:14:58 +1000 Subject: Type of Class In-Reply-To: References: Message-ID: <53070B02.5020104@oracle.com> On 21/02/2014 3:38 AM, Stephen Colebourne wrote: > In JDK 5, three methods were added to java.lang.Class [1] > - isAnonymousClass() > - isLocalClass() > - isMemberClass() > > Unfortunately, these do not cover the complete range of possible types of class. I think they do once you realize that the static modifier is also needed information. But it isn't pretty :) > Would it be reasonable to add the following methods: > - isNestedClass() This would be !isTopLevelClass() but otherwise isAnonymousClass() || isLocalClass() || isMemberClass() > - isInnerClass() isAnonymousClass() || isLocalClass() || (isMemberClass() && !static) > - isTopLevelClass() ! (isAnonymousClass() || isLocalClass() || isMemberClass()) and for completeness: - isStaticNestedClass() == isMemberClass() && static > While JVM/spec experts may know the difference, it is a surprisingly > tricky area. For example, the current isMemberClass() method could do > with much better Javadoc to explain what a member class actually is > (as it seemed hard to google). I agree. And I have to confess that I don't even mention member classes in my treatise on nested types in Chapter 5 of JPL 4e; and worse I give the wrong definition for isMemberClass in the reflection chapter (equating it to nested class)! But I do also comment on the lack of ability to ask whether it is a static nested class or an inner class. > FWIW, I was just trying to tell the difference between a nested class > and an inner class, in order to determine which classes can be > instantiated without reference to a surrounding object. The answer > seems to be (cls.isMemberClass() && > Modifiers.isStatic(cls.getModifiers()) which is not the most obvious > code. Actually it is more complex than that. You can define an inner class in a static context (static method, static initializer) and in that case there is no enclosing instance. Not sure how to detect that: innerClass() && getEnclosingMethod/Constructor() == null || (getEnclosingMethod().isStatic()) ? Cheers, David > In addition, I suffered from the absence of an isNormalClass() - > probably a better name for this. > > Currently, you can determine if a class is an interface, annotation, > primitive or array, leaving the "normal" case as a quadruple negative. > This leaves such a user-written method vulnerable to any new type of > class that gets added in a future JDK: > > boolean isNormalClass(Class cls) { > return !cls.isInterface() && !cls.isAnnotation() && > !cls.isPrimitive() && !cls.isArray(); > } > > Stephen > [1] https://blogs.oracle.com/darcy/entry/nested_inner_member_and_top > From paul.sandoz at oracle.com Fri Feb 21 09:06:38 2014 From: paul.sandoz at oracle.com (Paul Sandoz) Date: Fri, 21 Feb 2014 10:06:38 +0100 Subject: JDK 9 RFR of JDK-8035279: Clean up internal deprecations in BigInteger In-Reply-To: <5306953C.4030007@cs.oswego.edu> References: <5305D6EB.30806@oracle.com> <53064179.2060703@gmail.com> <530660FA.1060407@oracle.com> <53067D84.5060500@oracle.com> <5306953C.4030007@cs.oswego.edu> Message-ID: <2D2BE3C7-02AB-4651-AD72-B8DE08EC427E@oracle.com> On Feb 21, 2014, at 12:52 AM, Doug Lea

wrote: > On 02/20/2014 05:11 PM, David Holmes wrote: >> On 21/02/2014 7:20 AM, Paul Sandoz wrote: >>> On Feb 20, 2014, at 9:09 PM, David Holmes wrote: >>>> In practice, because there are also final fields in these instances >>>> implementations will most likely perform a storestore barrier after >>>> construction and prior to setting the reference to the created object. >>> >>> Yes, that is what i was banking on. >>> >>> That is how hotspot behaves, right? >> >> Yes. >> I could not resist an little experiment: static class X { String a; final String f; String b; public X() { a = "A"; f = "S"; b = "B"; } } 067 B3: # N79 <- B5 B2 Freq: 1 067 movl [RAX + #12 (8-bit)], narrowoop: java/lang/String:exact * # compressed ptr 06e movl [RAX + #16 (8-bit)], narrowoop: java/lang/String:exact * # compressed ptr 075 movl [RAX + #20 (8-bit)], narrowoop: java/lang/String:exact * # compressed ptr 07c 07c MEMBAR-storestore (empty encoding) 07c # checkcastPP of RAX 07c movq R10, RAX # ptr -> long 07f shrq R10, #9 083 movq R11, 0x0000000192569000 # ptr 08d movb [R11 + R10], R12 # short/char (R12_heapbase==0) 091 MEMBAR-release ! (empty encoding) 091 addq rsp, 16 # Destroy frame That "storestore" is for the object header IIUC (which just so happens to be placed after all fields have been set) and the "release" is due to the final field, both of which are noops on x86. >>> But, you are saying other VMs might not do so? >> >> Right. A storestore at the end of construction is a simple and obvious way to >> implement the final-field semantics, but there may be other ways. The visibility >> guarantees for final fields extends only to those fields, and objects accessed >> through them. > > Which has turned out in practice to be the only plausible way to do > this, so the spec might as well have said so. > Yeah, i was wondering what VMs out there would do it differently. It would be a shame to take the 3x volatile hit to cover less common environments. (Another recent issue related to initialization of volatiles on AbstractMap indicated some awkward interactions with G1.) I think we should try and use zero, as John says (alas @Stable is package private to j.l.invoke), and replace 0 with another value such as MIN_VALUE if one is unsure of the bounds: int lsb; if ((lsb = lowestSetBit) == 0) { ... lowestSetBit = lsb != 0 ? lsb : Integer.MIN_VALUE; } return lsb != Integer.MIN_VALUE : lsb ? 0; > Paul: If you'd like to be pedantically correct at the > expense of a lot more overhead, you could use > AtomicXFieldUpdater.lazySet in the constructor. Role on "lowestSetBit.volatile..." ! > But since these changes are targeting JDK9, it is very > likely that the memory model will be updated to > confirm that the current approach is legal by the time > it ships. > Looking forward to that :-) Paul. From nhahtdh at gmail.com Fri Feb 21 10:05:02 2014 From: nhahtdh at gmail.com (Hong Dai Thanh) Date: Fri, 21 Feb 2014 18:05:02 +0800 Subject: RFR: 7171390 - The "bug" describe is a typo Message-ID: I request that JDK-7171390 be closed, since there is a typo in the bug report Copied from the bug report > 1 - String r1="blah - * ^% $^& [gf]".replaceAll("[a-zA-Z0-9]", "-"); > 2 - String r2="blah - * ^% $^& [gf]".replaceAll("[0-9a-zA-z]", "-"); The regex from String r2 specifies A-z, instead of A-Z. The range A-z includes [ ] \ ^ _ ` This explains why more characters are replaced in the output of r2. Best regards. From paul.sandoz at oracle.com Fri Feb 21 10:11:43 2014 From: paul.sandoz at oracle.com (Paul Sandoz) Date: Fri, 21 Feb 2014 11:11:43 +0100 Subject: [9] RFR (M): 8027827: Improve performance of catchException combinator In-Reply-To: <5306420B.9040200@oracle.com> References: <53052622.6010507@oracle.com> <0386E0D6-D515-4C2C-9D6E-DD3B53E8B3E2@oracle.com> <5306420B.9040200@oracle.com> Message-ID: On Feb 20, 2014, at 6:57 PM, Vladimir Ivanov wrote: > Paul, > > Thanks for the feedback! See my answers inline. > > Updated webrev: > http://cr.openjdk.java.net/~vlivanov/8027827/final/webrev.01/ > > I finally figured out how to make caching work. This webrev contains these changes. > +1 > I changed LF representation a bit and added 2 auxiliary method handles - argument boxing and wrapping into Object[] and result unboxing. These operations depend on actual type and can't be shared among arbitrary combinators with the same basic type. They are used only during LF interpretation and are completely ignored in compiled LFs. > >> src/share/classes/java/lang/invoke/MethodHandleImpl.java >> >> 634 // t_{i+2}:L=ValueConversions.unbox(t_{i+1}) OR ValueConversions.identity(t_{i+1}) >> 635 if (type.returnType().isPrimitive()) { >> 636 names[UNBOX_RESULT] = new Name(ValueConversions.unbox(type.returnType()), >> 637 names[TRY_CATCH]); >> 638 } else { >> 639 names[UNBOX_RESULT] = new Name(ValueConversions.identity(), >> 640 names[TRY_CATCH]); >> 641 } >> >> >> You could create the form without the identity transform for the non-primitive case? >> >> final int UNBOX_RESULT = type.returnType().isPrimitive() ? nameCursor++ : 0; >> ... >> >> if (UNBOX_RESULT > 0) { ... >> names[UNBOX_RESULT] = new Name(ValueConversions.unbox(type.returnType()), names[TRY_CATCH]); >> } > I can, but it complicates matching and compiling the pattern in InvokerBytecodeGenerator. I decided to keep the shape uniform for all cases. > Ah, yes i see now, the code is simpler being kept uniform. Paul. From scolebourne at joda.org Fri Feb 21 10:27:19 2014 From: scolebourne at joda.org (Stephen Colebourne) Date: Fri, 21 Feb 2014 10:27:19 +0000 Subject: Type of Class In-Reply-To: <53070B02.5020104@oracle.com> References: <53070B02.5020104@oracle.com> Message-ID: On 21 February 2014 08:14, David Holmes wrote: >> Would it be reasonable to add the following methods: >> - isNestedClass() > This would be !isTopLevelClass() but otherwise > isAnonymousClass() || isLocalClass() || isMemberClass() >> - isInnerClass() > isAnonymousClass() || isLocalClass() || (isMemberClass() && !static) >> - isTopLevelClass() > ! (isAnonymousClass() || isLocalClass() || isMemberClass()) > and for completeness: > - isStaticNestedClass() == isMemberClass() && static I think the next step is a JIRA unless anyone objects. Stephen From peter.levart at gmail.com Fri Feb 21 11:48:51 2014 From: peter.levart at gmail.com (Peter Levart) Date: Fri, 21 Feb 2014 12:48:51 +0100 Subject: JDK 9 RFR of JDK-8035279: Clean up internal deprecations in BigInteger In-Reply-To: <2D2BE3C7-02AB-4651-AD72-B8DE08EC427E@oracle.com> References: <5305D6EB.30806@oracle.com> <53064179.2060703@gmail.com> <530660FA.1060407@oracle.com> <53067D84.5060500@oracle.com> <5306953C.4030007@cs.oswego.edu> <2D2BE3C7-02AB-4651-AD72-B8DE08EC427E@oracle.com> Message-ID: <53073D23.3060002@gmail.com> On 02/21/2014 10:06 AM, Paul Sandoz wrote: > I think we should try and use zero, as John says (alas @Stable is package private to j.l.invoke), and replace 0 with another value such as MIN_VALUE if one is unsure of the bounds: > > int lsb; > if ((lsb = lowestSetBit) == 0) { > ... > lowestSetBit = lsb != 0 ? lsb : Integer.MIN_VALUE; > } > return lsb != Integer.MIN_VALUE : lsb ? 0; Hm, would the following be any better or worse? int lsb = lowestSetBit; if (lsb <= 0) { if (lsb == Integer.MIN_VALUE) { lsb = 0; } else { ... lowestSetBit = lsb != 0 ? lsb : Integer.MIN_VALUE; } } return lsb; I wonder if any of the above is better than current code that offsets the value space. Current code only contains a single compare+conditional-branch on fast-path while both of above variants contain more than one. Does eliminating a single add instruction on fast-path pay up for that? What do measurements show? Regards, Peter From vladimir.x.ivanov at oracle.com Fri Feb 21 11:59:10 2014 From: vladimir.x.ivanov at oracle.com (Vladimir Ivanov) Date: Fri, 21 Feb 2014 15:59:10 +0400 Subject: [9] RFR (M): 8027827: Improve performance of catchException combinator In-Reply-To: References: <53052622.6010507@oracle.com> <0386E0D6-D515-4C2C-9D6E-DD3B53E8B3E2@oracle.com> <5306420B.9040200@oracle.com> Message-ID: <53073F8E.80707@oracle.com> Paul, thanks! Best regards, Vladimir Ivanov On 2/21/14 2:11 PM, Paul Sandoz wrote: > > On Feb 20, 2014, at 6:57 PM, Vladimir Ivanov wrote: > >> Paul, >> >> Thanks for the feedback! See my answers inline. >> >> Updated webrev: >> http://cr.openjdk.java.net/~vlivanov/8027827/final/webrev.01/ >> >> I finally figured out how to make caching work. This webrev contains these changes. >> > > +1 > > >> I changed LF representation a bit and added 2 auxiliary method handles - argument boxing and wrapping into Object[] and result unboxing. These operations depend on actual type and can't be shared among arbitrary combinators with the same basic type. They are used only during LF interpretation and are completely ignored in compiled LFs. >> > > >>> src/share/classes/java/lang/invoke/MethodHandleImpl.java >>> >>> 634 // t_{i+2}:L=ValueConversions.unbox(t_{i+1}) OR ValueConversions.identity(t_{i+1}) >>> 635 if (type.returnType().isPrimitive()) { >>> 636 names[UNBOX_RESULT] = new Name(ValueConversions.unbox(type.returnType()), >>> 637 names[TRY_CATCH]); >>> 638 } else { >>> 639 names[UNBOX_RESULT] = new Name(ValueConversions.identity(), >>> 640 names[TRY_CATCH]); >>> 641 } >>> >>> >>> You could create the form without the identity transform for the non-primitive case? >>> >>> final int UNBOX_RESULT = type.returnType().isPrimitive() ? nameCursor++ : 0; >>> ... >>> >>> if (UNBOX_RESULT > 0) { ... >>> names[UNBOX_RESULT] = new Name(ValueConversions.unbox(type.returnType()), names[TRY_CATCH]); >>> } >> I can, but it complicates matching and compiling the pattern in InvokerBytecodeGenerator. I decided to keep the shape uniform for all cases. >> > > Ah, yes i see now, the code is simpler being kept uniform. > > Paul. > From daniel.fuchs at oracle.com Fri Feb 21 12:02:13 2014 From: daniel.fuchs at oracle.com (Daniel Fuchs) Date: Fri, 21 Feb 2014 13:02:13 +0100 Subject: RFR: 8032909: XSLT string-length returns incorrect length when string includes complementary chars In-Reply-To: <53064D0D.5030804@oracle.com> References: <5306173F.7050608@oracle.com> <53064D0D.5030804@oracle.com> Message-ID: <53074045.3060302@oracle.com> On 2/20/14 7:44 PM, huizhe wang wrote: > Hi Aleksej, > > The change looks good. It's good to consolidate test cases into a single > test for each encoding. Looks good! -- daniel > > Thanks, > Joe > > On 2/20/2014 6:54 AM, Aleksej Efimov wrote: >> Hi, >> >> There is a problem in XSLT string-length function: >> When string passed to this function contains complementary chars the >> current implementation of XPath treates this sequence as two >> characters (the string-length returns 2). But according to the w3 spec >> it should be treated as a single XPath character [1]. >> >> Please, review the proposed fix for this problem [2]. It replaces the >> 'length' call in xslt compiler with a call to 'codePointCount'. Also >> new regression test was added. >> >> To verify the fix following tests were executed: >> 1. JCK test sets: api/xsl api/xinclude api/javax_xml api/org_xml >> xml_schema >> 2. Regression tests from 'jdk_other' testset (includes added >> regression test) >> >> Thanks in advance, >> Aleksej >> >> [1] http://www.w3.org/TR/xpath/#strings >> [2] http://cr.openjdk.java.net/~aefimov/8032909/webrev.00 >> > From forax at univ-mlv.fr Fri Feb 21 12:33:38 2014 From: forax at univ-mlv.fr (Remi Forax) Date: Fri, 21 Feb 2014 13:33:38 +0100 Subject: [9] RFR (M): 8027827: Improve performance of catchException combinator In-Reply-To: <53073F8E.80707@oracle.com> References: <53052622.6010507@oracle.com> <0386E0D6-D515-4C2C-9D6E-DD3B53E8B3E2@oracle.com> <5306420B.9040200@oracle.com> <53073F8E.80707@oracle.com> Message-ID: <530747A2.8090508@univ-mlv.fr> Hi Valdimir, is there a reason to generate try { ... } catch(Throwable t) { if (Klass.isInstance(t)) { ... } throw t; } instead of: try { ... } catch(Klass t) { ... } maybe because the Klass can be resolved by the wrong classloader ? R?mi On 02/21/2014 12:59 PM, Vladimir Ivanov wrote: > Paul, thanks! > > Best regards, > Vladimir Ivanov > > On 2/21/14 2:11 PM, Paul Sandoz wrote: >> On Feb 20, 2014, at 6:57 PM, Vladimir Ivanov wrote: >> >>> Paul, >>> >>> Thanks for the feedback! See my answers inline. >>> >>> Updated webrev: >>> http://cr.openjdk.java.net/~vlivanov/8027827/final/webrev.01/ >>> >>> I finally figured out how to make caching work. This webrev contains these changes. >>> >> +1 >> >> >>> I changed LF representation a bit and added 2 auxiliary method handles - argument boxing and wrapping into Object[] and result unboxing. These operations depend on actual type and can't be shared among arbitrary combinators with the same basic type. They are used only during LF interpretation and are completely ignored in compiled LFs. >>> >> >>>> src/share/classes/java/lang/invoke/MethodHandleImpl.java >>>> >>>> 634 // t_{i+2}:L=ValueConversions.unbox(t_{i+1}) OR ValueConversions.identity(t_{i+1}) >>>> 635 if (type.returnType().isPrimitive()) { >>>> 636 names[UNBOX_RESULT] = new Name(ValueConversions.unbox(type.returnType()), >>>> 637 names[TRY_CATCH]); >>>> 638 } else { >>>> 639 names[UNBOX_RESULT] = new Name(ValueConversions.identity(), >>>> 640 names[TRY_CATCH]); >>>> 641 } >>>> >>>> >>>> You could create the form without the identity transform for the non-primitive case? >>>> >>>> final int UNBOX_RESULT = type.returnType().isPrimitive() ? nameCursor++ : 0; >>>> ... >>>> >>>> if (UNBOX_RESULT > 0) { ... >>>> names[UNBOX_RESULT] = new Name(ValueConversions.unbox(type.returnType()), names[TRY_CATCH]); >>>> } >>> I can, but it complicates matching and compiling the pattern in InvokerBytecodeGenerator. I decided to keep the shape uniform for all cases. >>> >> Ah, yes i see now, the code is simpler being kept uniform. >> >> Paul. >> > _______________________________________________ > mlvm-dev mailing list > mlvm-dev at openjdk.java.net > http://mail.openjdk.java.net/mailman/listinfo/mlvm-dev From vladimir.x.ivanov at oracle.com Fri Feb 21 12:54:26 2014 From: vladimir.x.ivanov at oracle.com (Vladimir Ivanov) Date: Fri, 21 Feb 2014 16:54:26 +0400 Subject: [9] RFR (M): 8027827: Improve performance of catchException combinator In-Reply-To: <530747A2.8090508@univ-mlv.fr> References: <53052622.6010507@oracle.com> <0386E0D6-D515-4C2C-9D6E-DD3B53E8B3E2@oracle.com> <5306420B.9040200@oracle.com> <53073F8E.80707@oracle.com> <530747A2.8090508@univ-mlv.fr> Message-ID: <53074C82.1050404@oracle.com> Remi, Yes, one reason is resolution peculiarities of exception class not located on boot classpath. The other reason is enabling extensive sharing of lambda forms. Best regards, Vladimir Ivanov On 2/21/14 4:33 PM, Remi Forax wrote: > Hi Valdimir, > is there a reason to generate > try { > ... > } catch(Throwable t) { > if (Klass.isInstance(t)) { > ... > } > throw t; > } > instead of: > try { > ... > } catch(Klass t) { > ... > } > > maybe because the Klass can be resolved by the wrong classloader ? > > R?mi > > On 02/21/2014 12:59 PM, Vladimir Ivanov wrote: >> Paul, thanks! >> >> Best regards, >> Vladimir Ivanov >> >> On 2/21/14 2:11 PM, Paul Sandoz wrote: >>> On Feb 20, 2014, at 6:57 PM, Vladimir Ivanov wrote: >>> >>>> Paul, >>>> >>>> Thanks for the feedback! See my answers inline. >>>> >>>> Updated webrev: >>>> http://cr.openjdk.java.net/~vlivanov/8027827/final/webrev.01/ >>>> >>>> I finally figured out how to make caching work. This webrev contains these changes. >>>> >>> +1 >>> >>> >>>> I changed LF representation a bit and added 2 auxiliary method handles - argument boxing and wrapping into Object[] and result unboxing. These operations depend on actual type and can't be shared among arbitrary combinators with the same basic type. They are used only during LF interpretation and are completely ignored in compiled LFs. >>>> >>> >>>>> src/share/classes/java/lang/invoke/MethodHandleImpl.java >>>>> >>>>> 634 // t_{i+2}:L=ValueConversions.unbox(t_{i+1}) OR ValueConversions.identity(t_{i+1}) >>>>> 635 if (type.returnType().isPrimitive()) { >>>>> 636 names[UNBOX_RESULT] = new Name(ValueConversions.unbox(type.returnType()), >>>>> 637 names[TRY_CATCH]); >>>>> 638 } else { >>>>> 639 names[UNBOX_RESULT] = new Name(ValueConversions.identity(), >>>>> 640 names[TRY_CATCH]); >>>>> 641 } >>>>> >>>>> >>>>> You could create the form without the identity transform for the non-primitive case? >>>>> >>>>> final int UNBOX_RESULT = type.returnType().isPrimitive() ? nameCursor++ : 0; >>>>> ... >>>>> >>>>> if (UNBOX_RESULT > 0) { ... >>>>> names[UNBOX_RESULT] = new Name(ValueConversions.unbox(type.returnType()), names[TRY_CATCH]); >>>>> } >>>> I can, but it complicates matching and compiling the pattern in InvokerBytecodeGenerator. I decided to keep the shape uniform for all cases. >>>> >>> Ah, yes i see now, the code is simpler being kept uniform. >>> >>> Paul. >>> >> _______________________________________________ >> mlvm-dev mailing list >> mlvm-dev at openjdk.java.net >> http://mail.openjdk.java.net/mailman/listinfo/mlvm-dev > > _______________________________________________ > mlvm-dev mailing list > mlvm-dev at openjdk.java.net > http://mail.openjdk.java.net/mailman/listinfo/mlvm-dev > From mark.sheppard at oracle.com Fri Feb 21 13:21:33 2014 From: mark.sheppard at oracle.com (Mark Sheppard) Date: Fri, 21 Feb 2014 13:21:33 +0000 Subject: RFR: JDK-8030780 - test/com/sun/corba/cachedSocket/7056731.sh leaves HelloServer behind Message-ID: <530752DD.3040703@oracle.com> Hi please oblige and review the follow change http://cr.openjdk.java.net/~msheppar/8030780/webrev/ to address the issue raised in https://bugs.openjdk.java.net/browse/JDK-8030780 summary: if the the java debugger failed to connect to the client, then the sequence of commands which include "kill -9" of the server process will not be executed. the server process is added to the post test kill In a successful execution this will result in a "no such process" message. regards Mark From brian.goetz at oracle.com Fri Feb 21 15:40:08 2014 From: brian.goetz at oracle.com (Brian Goetz) Date: Fri, 21 Feb 2014 07:40:08 -0800 Subject: Type of Class In-Reply-To: References: <53070B02.5020104@oracle.com> Message-ID: I understand why you want this, though I think you?ll find that there are still thousands of other things ?missing? from reflection. In the Java 1.0 days, the difference between the Java language and the class file was pretty small. So reflection served as both the class file (VM) reflection and language reflection mechanism. But, over time, the gap has grown wider. We?ve made the decision (though not always consistently applied) that reflection is about serving up the class file information to Java, not about answering questions about the Java language. So, for example, it can?t tell that one method is a bridge for another, or easily answer questions about inheritance or overriding. Similarly, the issues raised here are about gaps between the class file representation of a class and the language level model. Historically we have added some things to reflection to fill in these gaps. However, our current strategy is to expose this through javax.lang.model, which is designed to reflect the langauge-level view of the world, and this is what users really want anyway. Currently the only implementation of javax.lang.model that is available is in the compiler, exposed to annotation processors, but we have a plan to expose one backed by core reflection which is a more sensible way to express the information you are looking for. On Feb 21, 2014, at 2:27 AM, Stephen Colebourne wrote: > On 21 February 2014 08:14, David Holmes wrote: >>> Would it be reasonable to add the following methods: >>> - isNestedClass() >> This would be !isTopLevelClass() but otherwise >> isAnonymousClass() || isLocalClass() || isMemberClass() >>> - isInnerClass() >> isAnonymousClass() || isLocalClass() || (isMemberClass() && !static) >>> - isTopLevelClass() >> ! (isAnonymousClass() || isLocalClass() || isMemberClass()) >> and for completeness: >> - isStaticNestedClass() == isMemberClass() && static > > I think the next step is a JIRA unless anyone objects. > Stephen From joe.darcy at oracle.com Fri Feb 21 17:59:35 2014 From: joe.darcy at oracle.com (Joe Darcy) Date: Fri, 21 Feb 2014 09:59:35 -0800 Subject: Type of Class In-Reply-To: References: <53070B02.5020104@oracle.com> Message-ID: <53079407.7060807@oracle.com> On 02/21/2014 07:40 AM, Brian Goetz wrote: > I understand why you want this, though I think you?ll find that there are still thousands of other things ?missing? from reflection. > > In the Java 1.0 days, the difference between the Java language and the class file was pretty small. So reflection served as both the class file (VM) reflection and language reflection mechanism. But, over time, the gap has grown wider. We?ve made the decision (though not always consistently applied) that reflection is about serving up the class file information to Java, not about answering questions about the Java language. So, for example, it can?t tell that one method is a bridge for another, or easily answer questions about inheritance or overriding. Similarly, the issues raised here are about gaps between the class file representation of a class and the language level model. > > Historically we have added some things to reflection to fill in these gaps. However, our current strategy is to expose this through javax.lang.model, which is designed to reflect the langauge-level view of the world, and this is what users really want anyway. Currently the only implementation of javax.lang.model that is available is in the compiler, exposed to annotation processors, but we have a plan to expose one backed by core reflection which is a more sensible way to express the information you are looking for. > > > The nested class situation is so tricky, I wrote a blog entry about it complete with a diagram: https://blogs.oracle.com/darcy/entry/nested_inner_member_and_top The javax.lang.model API uses a NESTING_KIND enum to model these distinctions: http://download.java.net/jdk8/docs/api/javax/lang/model/element/NestingKind.html As of JDK 8, we've published a proof of concept implementation of javax.lang.model backed by core reflection as sample code: http://hg.openjdk.java.net/jdk8/tl/langtools/file/tip/src/share/sample/language/model/CoreReflectionFactory.java Cheers, -Joe From christian.thalinger at oracle.com Fri Feb 21 18:40:48 2014 From: christian.thalinger at oracle.com (Christian Thalinger) Date: Fri, 21 Feb 2014 10:40:48 -0800 Subject: [9] RFR (M): 8027827: Improve performance of catchException combinator In-Reply-To: <5306420B.9040200@oracle.com> References: <53052622.6010507@oracle.com> <0386E0D6-D515-4C2C-9D6E-DD3B53E8B3E2@oracle.com> <5306420B.9040200@oracle.com> Message-ID: <96ED4FA7-80A4-4C79-AC8C-89DD2D3CA157@oracle.com> On Feb 20, 2014, at 9:57 AM, Vladimir Ivanov wrote: > Paul, > > Thanks for the feedback! See my answers inline. > > Updated webrev: > http://cr.openjdk.java.net/~vlivanov/8027827/final/webrev.01/ > > I finally figured out how to make caching work. This webrev contains > these changes. > > I changed LF representation a bit and added 2 auxiliary method handles > - argument boxing and wrapping into Object[] and result unboxing. These > operations depend on actual type and can't be shared among arbitrary > combinators with the same basic type. They are used only during LF > interpretation and are completely ignored in compiled LFs. > > On 2/20/14 7:51 PM, Paul Sandoz wrote: >> Hi Vladimir, >> >> I know just enough about this area to be dangerous.... >> >> >> src/share/classes/java/lang/invoke/BoundMethodHandle.java >> >> 865 private static final SpeciesData[] SPECIES_DATA_CACHE = new SpeciesData[4]; >> >> >> Only 3 elements are stored in the above array? > Fixed. > >> >> >> >> src/share/classes/java/lang/invoke/MethodHandleImpl.java >> >> 634 // t_{i+2}:L=ValueConversions.unbox(t_{i+1}) OR ValueConversions.identity(t_{i+1}) >> 635 if (type.returnType().isPrimitive()) { >> 636 names[UNBOX_RESULT] = new Name(ValueConversions.unbox(type.returnType()), >> 637 names[TRY_CATCH]); >> 638 } else { >> 639 names[UNBOX_RESULT] = new Name(ValueConversions.identity(), >> 640 names[TRY_CATCH]); >> 641 } >> >> >> You could create the form without the identity transform for the non-primitive case? >> >> final int UNBOX_RESULT = type.returnType().isPrimitive() ? nameCursor++ : 0; >> ... >> >> if (UNBOX_RESULT > 0) { ... >> names[UNBOX_RESULT] = new Name(ValueConversions.unbox(type.returnType()), names[TRY_CATCH]); >> } > I can, but it complicates matching and compiling the pattern in > InvokerBytecodeGenerator. I decided to keep the shape uniform for all cases. > >> >> >> 699 try { >> 700 GUARD_WITH_CATCH >> 701 = IMPL_LOOKUP.findStatic(MethodHandleImpl.class, "guardWithCatch", >> 702 MethodType.methodType(Object.class, MethodHandle.class, Class.class, MethodHandle.class, Object[].class)); >> 703 } catch (ReflectiveOperationException ex) { >> 704 throw new RuntimeException(ex); >> 705 } >> 706 return GUARD_WITH_CATCH; >> >> >> Should #704 be: >> >> throw newInternalError(ex); >> >> ? > Fixed. Also updated other places where RuntimeException was thrown. A tiny bit of history here on newInternalError: in JDK 8 we added new constructors that take a cause as argument; JDK 7 doesn?t have them. In order to make java.lang.invoke backporting from 8 to 7 easier we used a wrapper method to instantiate InternalErrors. > >> >> >> test/java/lang/invoke/MethodHandles/TestCatchException.java >> >> 100 Object o = new Object(); >> 101 Object[] obj1 = new Object[] { "str" }; >> 102 >> 103 Object r1 = target.invokeExact(o, o, o, o, o, o, o, o, obj1); >> 104 Object r2 = gwc.invokeExact(o, o, o, o, o, o, o, o, obj1); >> 105 assertTrue(r1 != null); >> 106 assertTrue(r1.equals(r2)); >> >> To be on the safe side should probably assert as follows: >> >> assertEquals(r1, obj); >> assertEquals(r2, obj); > Fixed. > > Best regards, > Vladimir Ivanov > >> Paul. >> >> >> On Feb 19, 2014, at 10:46 PM, Vladimir Ivanov wrote: >> >>> http://cr.openjdk.java.net/~vlivanov/8027827/final/webrev.00 >>> https://jbs.oracle.com/bugs/browse/JDK-8027827 >>> 354 lines changed: 193 ins; 91 del; 70 mod >>> >>> OVERVIEW >>> >>> MethodHandles.catchException combinator implementation is based on generic invokers (MethodHandleImpl$GuardWithCatch.invoke_*). It is significantly slower than a Java equivalent (try-catch). >>> >>> Future Nashorn performance improvements require catchException combinator speed to be on par with try-catch in Java. >>> >>> So, it should be represented in a more efficient form. >>> >>> I chose the following lambda form representation: >>> >>> t_{i}:L=ValueConversions.array(a1:L,...,a_{k}:L); >>> t_{i+1}:L=MethodHandleImpl.guardWithCatch(t_{p1}, t_{p2}, t_{p3}, t_{i}:L); >>> t_{i+2}:I=ValueConversions.unbox(t7:L); >>> OR :L=ValueConversions.identity(t_{n+1}) >>> OR :V=ValueConversions.ignore(t_{n+1}) >>> >>> where: >>> a1, ..., a_{k} - arguments >>> t_{p1}, t_{p2}, t_{p3} - target method handle, exception class, catcher method handle respectively; passed as bounded parameters; >>> >>> During lambda form compilation it is converted into bytecode equivalent of the following Java code: >>> try { >>> return target.invokeBasic(...); >>> } catch(Throwable e) { >>> if (!exClass.isInstance(e)) throw e; >>> return catcher.invokeBasic(e, ...); >>> } >>> >>> There's a set of microbenchmarks (attached to the bug) I wrote to verify performance characteristics of new implementation. >>> >>> FURTHER WORK >>> >>> What is missing is lambda form caching. The plan is to have a single lambda form per basic type, but it needs more work - current representation is suitable for sharing on bytecode level, but lambda form interpretation doesn't work well (arguments boxing + result unboxing are problematic). >>> >>> TESTING >>> >>> Tests: microbenchmarks, jdk/java/lang/invoke/, nashorn with optimistic types (unit tests, octane). >>> >>> Tested in 2 modes: >>> * COMPILE_THRESHOLD=30 >>> * COMPILE_THRESHOLD=0 -Xverify:all >>> >>> OTHER >>> >>> 1) Update of cached name and member in LF compilation loop (see InvokerBytecodeGenerator.generateCustomizedCodeBytes) fixes a bug during compilation of selectAlternative when running with COMPILE_THRESHOLD=0. >>> >>> 2) As part of this change, I fix existing bug [1], so I add regression test for it. >>> >>> Thanks! >>> >>> Best regards, >>> Vladimir Ivanov >>> >>> [1] https://bugs.openjdk.java.net/browse/JDK-8034120 >> > _______________________________________________ > mlvm-dev mailing list > mlvm-dev at openjdk.java.net > http://mail.openjdk.java.net/mailman/listinfo/mlvm-dev From scolebourne at joda.org Fri Feb 21 21:27:01 2014 From: scolebourne at joda.org (Stephen Colebourne) Date: Fri, 21 Feb 2014 21:27:01 +0000 Subject: Type of Class In-Reply-To: References: <53070B02.5020104@oracle.com> Message-ID: But is javax.lang.model a plan for JDK 9? To give some idea of the pain in this area, here is the code I ended up with: if (type.isInterface() || type.isAnnotation() || type.isPrimitive() || type.isArray() || type.isEnum() || type.isSynthetic() || Modifier.isAbstract(type.getModifiers()) || type.isAnonymousClass() || type.isLocalClass() || (type.isMemberClass() && Modifier.isStatic(type.getModifiers()) == false)) { // error } Its pretty laughable really. I just wanted to say "is it a normal class". FWIW, the KindType does not distinguish between static and inner member classes either, which was the part that took most of my time to figure out. Stephen On 21 February 2014 15:40, Brian Goetz wrote: > I understand why you want this, though I think you'll find that there are still thousands of other things "missing" from reflection. > > In the Java 1.0 days, the difference between the Java language and the class file was pretty small. So reflection served as both the class file (VM) reflection and language reflection mechanism. But, over time, the gap has grown wider. We've made the decision (though not always consistently applied) that reflection is about serving up the class file information to Java, not about answering questions about the Java language. So, for example, it can't tell that one method is a bridge for another, or easily answer questions about inheritance or overriding. Similarly, the issues raised here are about gaps between the class file representation of a class and the language level model. > > Historically we have added some things to reflection to fill in these gaps. However, our current strategy is to expose this through javax.lang.model, which is designed to reflect the langauge-level view of the world, and this is what users really want anyway. Currently the only implementation of javax.lang.model that is available is in the compiler, exposed to annotation processors, but we have a plan to expose one backed by core reflection which is a more sensible way to express the information you are looking for. > > > On Feb 21, 2014, at 2:27 AM, Stephen Colebourne wrote: > >> On 21 February 2014 08:14, David Holmes wrote: >>>> Would it be reasonable to add the following methods: >>>> - isNestedClass() >>> This would be !isTopLevelClass() but otherwise >>> isAnonymousClass() || isLocalClass() || isMemberClass() >>>> - isInnerClass() >>> isAnonymousClass() || isLocalClass() || (isMemberClass() && !static) >>>> - isTopLevelClass() >>> ! (isAnonymousClass() || isLocalClass() || isMemberClass()) >>> and for completeness: >>> - isStaticNestedClass() == isMemberClass() && static >> >> I think the next step is a JIRA unless anyone objects. >> Stephen > From mike.duigou at oracle.com Fri Feb 21 22:41:47 2014 From: mike.duigou at oracle.com (Mike Duigou) Date: Fri, 21 Feb 2014 14:41:47 -0800 Subject: RFR: 8035584: (s) ArrayList(c) should avoid inflation if c is empty Message-ID: <0541A3EA-A2A5-4977-8054-4EDB28366216@oracle.com> Hello all; This changeset consists of two small performance improvements for ArrayList. Both are related to the lazy initialization introduced in JDK-8011200. The first change is in the ArrayList(int capacity) constructor and forces lazy initialization if the requested capacity is zero. It's been observed that in cases where zero capacity is requested that it is very likely that the list never receives any elements. For these cases we permanently avoid the allocation of an element array. The second change, noticed by Gustav ?kesson, involves the ArrayList(Collection c) constructor. If c is an empty collection then there is no reason to inflate the backing array for the ArrayList. http://cr.openjdk.java.net/~mduigou/JDK-8035584/0/webrev/ I also took the opportunity to the -> {@code } conversion for the javadoc. Enjoy! Mike From martinrb at google.com Fri Feb 21 22:56:43 2014 From: martinrb at google.com (Martin Buchholz) Date: Fri, 21 Feb 2014 14:56:43 -0800 Subject: RFR: 8035584: (s) ArrayList(c) should avoid inflation if c is empty In-Reply-To: <0541A3EA-A2A5-4977-8054-4EDB28366216@oracle.com> References: <0541A3EA-A2A5-4977-8054-4EDB28366216@oracle.com> Message-ID: You should do -> code conversion separately, and do it pervasively across the entire JDK. This is not right. + * {@code (o==null ? get(i)==null : o.equals(get(i)))} ObElisp, avoiding the above bug: (defun tt-code () (interactive) (query-replace-regexp "<\\(tt\\|code\\)>\\([^<>{}&@]+\\)" "{@code \\2}")) On Fri, Feb 21, 2014 at 2:41 PM, Mike Duigou wrote: > Hello all; > > This changeset consists of two small performance improvements for > ArrayList. Both are related to the lazy initialization introduced in > JDK-8011200. > > The first change is in the ArrayList(int capacity) constructor and forces > lazy initialization if the requested capacity is zero. It's been observed > that in cases where zero capacity is requested that it is very likely that > the list never receives any elements. For these cases we permanently avoid > the allocation of an element array. > > The second change, noticed by Gustav ?kesson, involves the > ArrayList(Collection c) constructor. If c is an empty collection then there > is no reason to inflate the backing array for the ArrayList. > > http://cr.openjdk.java.net/~mduigou/JDK-8035584/0/webrev/ > > I also took the opportunity to the -> {@code } conversion for > the javadoc. > > Enjoy! > > Mike From martinrb at google.com Fri Feb 21 22:59:26 2014 From: martinrb at google.com (Martin Buchholz) Date: Fri, 21 Feb 2014 14:59:26 -0800 Subject: RFR: 8035584: (s) ArrayList(c) should avoid inflation if c is empty In-Reply-To: <0541A3EA-A2A5-4977-8054-4EDB28366216@oracle.com> References: <0541A3EA-A2A5-4977-8054-4EDB28366216@oracle.com> Message-ID: You accidentally deleted a stray space here? - this.elementData = EMPTY_ELEMENTDATA; + this.elementData = EMPTY_ELEMENTDATA; On Fri, Feb 21, 2014 at 2:41 PM, Mike Duigou wrote: > Hello all; > > This changeset consists of two small performance improvements for > ArrayList. Both are related to the lazy initialization introduced in > JDK-8011200. > > The first change is in the ArrayList(int capacity) constructor and forces > lazy initialization if the requested capacity is zero. It's been observed > that in cases where zero capacity is requested that it is very likely that > the list never receives any elements. For these cases we permanently avoid > the allocation of an element array. > > The second change, noticed by Gustav ?kesson, involves the > ArrayList(Collection c) constructor. If c is an empty collection then there > is no reason to inflate the backing array for the ArrayList. > > http://cr.openjdk.java.net/~mduigou/JDK-8035584/0/webrev/ > > I also took the opportunity to the -> {@code } conversion for > the javadoc. > > Enjoy! > > Mike From martinrb at google.com Fri Feb 21 23:03:42 2014 From: martinrb at google.com (Martin Buchholz) Date: Fri, 21 Feb 2014 15:03:42 -0800 Subject: RFR: 8035584: (s) ArrayList(c) should avoid inflation if c is empty In-Reply-To: <0541A3EA-A2A5-4977-8054-4EDB28366216@oracle.com> References: <0541A3EA-A2A5-4977-8054-4EDB28366216@oracle.com> Message-ID: public ArrayList(int initialCapacity) { - super(); if (initialCapacity < 0) throw new IllegalArgumentException("Illegal Capacity: "+ initialCapacity); - this.elementData = new Object[initialCapacity]; + this.elementData = (initialCapacity > 0) + ? new Object[initialCapacity] + : EMPTY_ELEMENTDATA; } When optimizing for special cases, we should try very hard minimize overhead in the common case. In the above, we now have two branches in the common case. Instead, if (initialCapacity > 0) this.elementData = new Object[initialCapacity]; else if (initialCapacity == 0) this.elementData = EMPTY_ELEMENTDATA; else barf On Fri, Feb 21, 2014 at 2:41 PM, Mike Duigou wrote: > Hello all; > > This changeset consists of two small performance improvements for > ArrayList. Both are related to the lazy initialization introduced in > JDK-8011200. > > The first change is in the ArrayList(int capacity) constructor and forces > lazy initialization if the requested capacity is zero. It's been observed > that in cases where zero capacity is requested that it is very likely that > the list never receives any elements. For these cases we permanently avoid > the allocation of an element array. > > The second change, noticed by Gustav ?kesson, involves the > ArrayList(Collection c) constructor. If c is an empty collection then there > is no reason to inflate the backing array for the ArrayList. > > http://cr.openjdk.java.net/~mduigou/JDK-8035584/0/webrev/ > > I also took the opportunity to the -> {@code } conversion for > the javadoc. > > Enjoy! > > Mike From dmitry.degrave at oracle.com Sat Feb 22 12:22:44 2014 From: dmitry.degrave at oracle.com (dmeetry degrave) Date: Sat, 22 Feb 2014 16:22:44 +0400 Subject: [7u backport] RFR: 7122142: (ann) Race condition between isAnnotationPresent and getAnnotations Message-ID: <53089694.4070308@oracle.com> Hi all, I would like to ask for a review of combined back port for 7u-dev/7u80. The main goal is to have a fix for 7122142 in jdk7, it also integrates the changes from 8005232, 7185456, 8022721 https://bugs.openjdk.java.net/browse/JDK-7122142 https://bugs.openjdk.java.net/browse/JDK-8005232 https://bugs.openjdk.java.net/browse/JDK-7185456 https://bugs.openjdk.java.net/browse/JDK-8022721 Original jdk8 changes: 7122142: http://hg.openjdk.java.net/jdk8/tl/jdk/rev/e4ce6502eac0 8005232: http://hg.openjdk.java.net/jdk8/tl/jdk/rev/1109bfff4e92 7185456: http://hg.openjdk.java.net/jdk8/tl/jdk/rev/ae03282ba501 8022721: http://hg.openjdk.java.net/jdk8/tl/jdk/rev/2281a7f79738 back port: http://cr.openjdk.java.net/~dmeetry/7122142.8005232.7185456.8022721/webrev.0/ Patches can't be applied cleanly, hence it was a manual back port, though the final result is equivalent to applying the patches in chronological order (8005232, 7185456, 7122142, 8022721) and applying all the relevant rejected parts (with one exception, AnnotationTypeRuntimeAssumptionTest.java test was not included due to jdk8 API). All tests in test/java/lang/annotation passed. thanks, dmeetry From Alan.Bateman at oracle.com Sun Feb 23 08:09:07 2014 From: Alan.Bateman at oracle.com (Alan Bateman) Date: Sun, 23 Feb 2014 08:09:07 +0000 Subject: RFR - 6857566 (bf) DirectByteBuffer garbage creation can outpace reclamation In-Reply-To: <5306345E.3060302@gmail.com> References: <52F1F174.8070105@gmail.com> <52F21094.3090400@oracle.com> <5306345E.3060302@gmail.com> Message-ID: <5309ACA3.7090500@oracle.com> On 20/02/2014 16:59, Peter Levart wrote: > : > > Hi Alan, > > I made another webrev with minor changes based on your feedback... > > http://cr.openjdk.java.net/~plevart/jdk9-dev/DirectBufferAlloc/webrev.03/ Thanks for the updates, this is good work and I think this is good to go. > > No System.exit in this webrev any more. The test should still run in > othervm though, since it uses special VM option > "-XX:MaxDirectMemorySize=128m". Without it there might be to much > direct memory available by default on some machine (like mine with > 16GB of RAM) to provoke an error in 5 seconds time... I agree, this has to run in its own VM. -Alan. From tristan.yan at oracle.com Sun Feb 23 09:40:24 2014 From: tristan.yan at oracle.com (Tristan Yan) Date: Sun, 23 Feb 2014 17:40:24 +0800 Subject: RFR for JDK-8031374: java/util/concurrent/ConcurrentQueues/OfferRemoveLoops.java fails Intermittently Message-ID: <6B2B9D6A-F0DA-4F00-BE95-68AC7A03B6D0@oracle.com> Hi All Could you please help to review fix for JDK-803174. http://cr.openjdk.java.net/~tyan/JDK-8031374/webrev.00/ Description: There are a couple of code change for this code. 1. Original code has used known problematic Thread.stop. Which could cause ThreadDeath as all we know. I change it to a customized stopThread method. 2. Test failed in windows, I analyze the failure by changing Thread.yield() in remover thread to Thread.sleep(50). This is a simulation for slow machine. Which shows exact same failures as we can see in bug description. By adding more debug info, we can see although we tried to give up CPU by using Thread.yield().(I have tried to use Thread.sleep(1L) as well, the result is same), there is no guarantee that os will pick up a new thread to execute. In Windows, this is more obvious. Because the execution is slow. even when the timeout happens, offer thread and remove thread won?t have chance to get the point that i % 1024 == 0. This will cause the failure like we see in the log. My fix here is when the timeout happens, but i is still less than 1024. Stop offer thread and remover thread right away instead letting them continuously wait the point to i == 1024. 3. I replace Thread.yield to Thread.sleep(0L). I saw a couple of discussion that Thread.yield is not required to give up CPU. Thank you Tristan From mark.sheppard at oracle.com Sun Feb 23 17:04:07 2014 From: mark.sheppard at oracle.com (Mark Sheppard) Date: Sun, 23 Feb 2014 17:04:07 +0000 Subject: RFR: JDK-8033075 - Broken Link in API docs Message-ID: <530A2A07.8060009@oracle.com> Hi please oblige and review the following changes http://cr.openjdk.java.net/~msheppar/8033075/webrev/ which address the issue raised in https://bugs.openjdk.java.net/browse/JDK-8033075 summary: a number of HREFs to CORBA specification exist in the CORBA api docs these now have dead URLs. The URLs contain cgi.omg.org in the path. This path element needs to be changed to www.omg.org. regards Mark From martinrb at google.com Sun Feb 23 19:46:30 2014 From: martinrb at google.com (Martin Buchholz) Date: Sun, 23 Feb 2014 11:46:30 -0800 Subject: RFR for JDK-8031374: java/util/concurrent/ConcurrentQueues/OfferRemoveLoops.java fails Intermittently In-Reply-To: <6B2B9D6A-F0DA-4F00-BE95-68AC7A03B6D0@oracle.com> References: <6B2B9D6A-F0DA-4F00-BE95-68AC7A03B6D0@oracle.com> Message-ID: Hi Tristan, (thanks for working on my old crappy tests, and apologies for always giving you a hard time) I couldn't find the bug report. Can you provide a URL? Thread.stop is only called in case the test has already failed (presumed "hung"), as a last resort. Perhaps the timeout used in the test (300 ms) is too small on some systems? + protected void giveupCPU(){ + try { + Thread.sleep(0L); + } catch (InterruptedException ignore) {} } I know of no reason why Thread.sleep(0) should be any more effective than Thread.yield. If it was more effective, then why wouldn't Thread.yield on that platform be fixed to be implemented exactly the same way? IOW if this is a problem, fix the JDK! /** Polls occasionally for quitting time. */ protected boolean quittingTime(long i) { - return (i % 1024) == 0 && quittingTime(); + return stopRequest || quittingTime() && (i % 1024 == 0 || i < 1024); + } You've completely broken the intent of this code, which was to poll ***occasionally*** for quitting time. On Sun, Feb 23, 2014 at 1:40 AM, Tristan Yan wrote: > Hi All > Could you please help to review fix for JDK-803174. > > http://cr.openjdk.java.net/~tyan/JDK-8031374/webrev.00/ > > Description: > There are a couple of code change for this code. > 1. Original code has used known problematic Thread.stop. Which could cause > ThreadDeath as all we know. I change it to a customized stopThread method. > 2. Test failed in windows, I analyze the failure by changing > Thread.yield() in remover thread to Thread.sleep(50). This is a simulation > for slow machine. Which shows exact same failures as we can see in bug > description. By adding more debug info, we can see although we tried to > give up CPU by using Thread.yield().(I have tried to use Thread.sleep(1L) > as well, the result is same), there is no guarantee that os will pick up a > new thread to execute. In Windows, this is more obvious. Because the > execution is slow. even when the timeout happens, offer thread and remove > thread won?t have chance to get the point that i % 1024 == 0. This will > cause the failure like we see in the log. My fix here is when the timeout > happens, but i is still less than 1024. Stop offer thread and remover > thread right away instead letting them continuously wait the point to i == > 1024. > 3. I replace Thread.yield to Thread.sleep(0L). I saw a couple of > discussion that Thread.yield is not required to give up CPU. > > Thank you > Tristan From Alan.Bateman at oracle.com Sun Feb 23 20:40:18 2014 From: Alan.Bateman at oracle.com (Alan Bateman) Date: Sun, 23 Feb 2014 20:40:18 +0000 Subject: RFR: JDK-8033075 - Broken Link in API docs In-Reply-To: <530A2A07.8060009@oracle.com> References: <530A2A07.8060009@oracle.com> Message-ID: <530A5CB2.60900@oracle.com> On 23/02/2014 17:04, Mark Sheppard wrote: > Hi > > please oblige and review the following changes > > http://cr.openjdk.java.net/~msheppar/8033075/webrev/ > > which address the issue raised in > > https://bugs.openjdk.java.net/browse/JDK-8033075 > > summary: > a number of HREFs to CORBA specification exist in the CORBA api docs > these now have dead URLs. The URLs contain cgi.omg.org in the path. > This path element needs to be changed to www.omg.org. This looks okay to me and look to see that links are being checked. -Alan. From david.holmes at oracle.com Sun Feb 23 22:59:21 2014 From: david.holmes at oracle.com (David Holmes) Date: Mon, 24 Feb 2014 08:59:21 +1000 Subject: RFR for JDK-8031374: java/util/concurrent/ConcurrentQueues/OfferRemoveLoops.java fails Intermittently In-Reply-To: <6B2B9D6A-F0DA-4F00-BE95-68AC7A03B6D0@oracle.com> References: <6B2B9D6A-F0DA-4F00-BE95-68AC7A03B6D0@oracle.com> Message-ID: <530A7D49.6000404@oracle.com> Hi Tristan, I don't have time to analyse the original failure mode nor your proposed changes but ... On 23/02/2014 7:40 PM, Tristan Yan wrote: > Hi All > Could you please help to review fix for JDK-803174. > > http://cr.openjdk.java.net/~tyan/JDK-8031374/webrev.00/ > > Description: > There are a couple of code change for this code. > 1. Original code has used known problematic Thread.stop. Which could cause ThreadDeath as all we know. I change it to a customized stopThread method. > 2. Test failed in windows, I analyze the failure by changing Thread.yield() in remover thread to Thread.sleep(50). This is a simulation for slow machine. Which shows exact same failures as we can see in bug description. By adding more debug info, we can see although we tried to give up CPU by using Thread.yield().(I have tried to use Thread.sleep(1L) as well, the result is same), there is no guarantee that os will pick up a new thread to execute. In Windows, this is more obvious. Because the execution is slow. even when the timeout happens, offer thread and remove thread won?t have chance to get the point that i % 1024 == 0. This will cause the failure like we see in the log. My fix here is when the timeout happens, but i is still less than 1024. Stop offer thread and remover thread right away instead letting them continuously wait the point to i == 1024. > 3. I replace Thread.yield to Thread.sleep(0L). I saw a couple of discussion that Thread.yield is not required to give up CPU. Thread.sleep(0) has no meaningful semantics - it says "sleep for at least zero time". That can be a no-op, or a yield-like operation or ... An actual sleep of at least 10ms should be used, preferably a bit longer as you don't know how the OS will behave if the sleep time requested is less than the natural resolution of the sleep mechanism. For Martin's benefit, while Thread.yield was always assumed to do the right thing and act as reasonably expected, that got broken by the Linux folks with CFS at one point. So loops involving yield are out of favour. David > Thank you > Tristan > From tristan.yan at oracle.com Mon Feb 24 00:23:04 2014 From: tristan.yan at oracle.com (Tristan Yan) Date: Sun, 23 Feb 2014 16:23:04 -0800 (PST) Subject: =?utf-8?B?562U5aSNOiBSRlIgZm9yIEpESy04MDMxMzc0OiBq?= =?utf-8?B?YXZhL3V0aWwvY29uY3VycmVudC9Db25jdXI=?= =?utf-8?B?cmVudFF1ZXVlcy9PZmZlclJlbW92ZUxvb3A=?= =?utf-8?B?cy5qYXZhIGZhaWxzIEludGVybWl0dGVudGx5?= In-Reply-To: References: <6B2B9D6A-F0DA-4F00-BE95-68AC7A03B6D0@oracle.com> Message-ID: <65a677bc-304d-4a46-8465-7aa36c007212@default> Thank you for reviewing this. Martin ? The original bug report is here: https://bugs.openjdk.java.net/browse/JDK-8031374. ? I haven?t reproduced this bug but I can simulate to reproduce this failure by changing ?yield() in remover thread to Thread.sleep(20). You have commented ?You've completely broken the intent of this code, which was to poll ***occasionally*** for quitting time?. I tried to not changed the logic for the test. This failure comes when only 1st rounds(1024 loop) wasn?t finished in given quitting time(before timeout). Which was 300ms. One solution is increasing default timeout as you suggested. But question is how big should we increase. When the test is really slow and could not finish 1st round(1024 loop) before timeout, I couldn?t figure out what?s the reason timeout value. Also this definitely will slow down the tests when it run in fast machine. Which is the case most of time. So I took other step that skip wait if test doesn't finish 1st round(1024 loop) before timeout. I won?t say I completely broke the intent of the code here because it?s rare case.(Only if the machine is slow and slow enough that the test doesn't finish 1st round before timeout). ? The reason that replace Thread.yield to Thread.sleep(0L) because David Holmes point out in the bug ?sleep will certainly give up the CPU, whereas yield is not required to.? Also in other mail, David pointed I should use Thread.sleep(10L) instead of Thread.sleep(0L) preferably a bit longer as we don't know how the OS will behave if the sleep time requested is less than the natural resolution of the sleep mechanism. For the experiment I?ve done in Windows. Thread.yeild() or Thread.sleep(10L) won?t guarantee current thread give up the CPU. This is a hint to OS. This makes in windows remover and offer thread could wait to each other more more than other other operating system. This is also the one of the reason that can explain why we've seen this in windows only. ? Regards Tristan ? ? ? ???: Martin Buchholz [mailto:martinrb at google.com] ????: Monday, February 24, 2014 3:47 AM ???: Tristan Yan ??: core-libs-dev ??: Re: RFR for JDK-8031374: java/util/concurrent/ConcurrentQueues/OfferRemoveLoops.java fails Intermittently ? Hi Tristan, ? (thanks for working on my old crappy tests, and apologies for always giving you a hard time) ? I couldn't find the bug report. ?Can you provide a URL? ? Thread.stop is only called in case the test has already failed (presumed "hung"), as a last resort. ? Perhaps the timeout used in the test (300 ms) is too small on some systems? ? +??????????? protected void giveupCPU(){ +????????????? ??try { +??????????????????? Thread.sleep(0L); +??????????????? } catch (InterruptedException ignore) {} ???????????? } ? I know of no reason why Thread.sleep(0) should be any more effective than Thread.yield. ?If it was more effective, then why wouldn't Thread.yield on that platform be fixed to be implemented exactly the same way? ?IOW if this is a problem, fix the JDK! ? ???????????? /** Polls occasionally for quitting time. */ ???????????? protected boolean quittingTime(long i) { -??????????????? return (i % 1024) == 0 && quittingTime(); +??????????????? return stopRequest || quittingTime() && (i % 1024 == 0 || i < 1024); +??????????? } ? You've completely broken the intent of this code, which was to poll ***occasionally*** for quitting time. ? ? On Sun, Feb 23, 2014 at 1:40 AM, Tristan Yan wrote: Hi All Could you please help to review fix for JDK-803174. http://cr.openjdk.java.net/~tyan/JDK-8031374/webrev.00/ Description: There are a couple of code change for this code. 1. Original code has used known problematic Thread.stop. Which could cause ThreadDeath as all we know. I change it to a customized stopThread method. 2. Test failed in windows, I analyze the failure by changing Thread.yield() in remover thread to Thread.sleep(50). This is a simulation for slow machine. Which shows exact same failures as we can see in bug description. By adding more debug info, we can see although we tried to give up CPU by using Thread.yield().(I have tried to use Thread.sleep(1L) as well, the result is same), there is no guarantee that os will pick up a new thread to execute. In Windows, this is more obvious. Because the execution is slow. even when the timeout happens, offer thread and remove thread won?t have chance to get the point that i % 1024 == 0. This will cause the failure like we see in the log. My fix here is when the timeout happens, but i is still less than 1024. Stop offer thread and remover thread right away instead letting them continuously wait the point to i == 1024. 3. I replace Thread.yield to Thread.sleep(0L). I saw a couple of discussion that Thread.yield is not required to give up CPU. Thank you Tristan ? From martinrb at google.com Mon Feb 24 02:03:36 2014 From: martinrb at google.com (Martin Buchholz) Date: Sun, 23 Feb 2014 18:03:36 -0800 Subject: =?UTF-8?Q?Re=3A_=E7=AD=94=E5=A4=8D=3A_RFR_for_JDK=2D8031374=3A_java=2Futil=2Fconcurren?= =?UTF-8?Q?t=2FConcurrentQueues=2FOfferRemoveLoops=2Ejava_fails_Intermittentl?= =?UTF-8?Q?y?= In-Reply-To: <65a677bc-304d-4a46-8465-7aa36c007212@default> References: <6B2B9D6A-F0DA-4F00-BE95-68AC7A03B6D0@oracle.com> <65a677bc-304d-4a46-8465-7aa36c007212@default> Message-ID: I'm doing some independent cleanup and checking into jsr166 CVS: This gets rid of the Thread.stop and introduces a Semaphore that can be waited on instead of spinning. Try to work from this new version. If you can never reproduce the failure to verify a fix, I suggest leaving the bug unfixed until we can. Presumably you have a Windows machine in a lab somewhere that produced this originally? http://gee.cs.oswego.edu/cgi-bin/viewcvs.cgi/jsr166/src/test/jtreg/util/concurrent/ConcurrentQueues/OfferRemoveLoops.java?view=co Index: OfferRemoveLoops.java =================================================================== RCS file: /export/home/jsr166/jsr166/jsr166/src/test/jtreg/util/concurrent/ConcurrentQueues/OfferRemoveLoops.java,v retrieving revision 1.7 diff -u -r1.7 OfferRemoveLoops.java --- OfferRemoveLoops.java 18 Jan 2013 04:23:28 -0000 1.7 +++ OfferRemoveLoops.java 24 Feb 2014 01:57:03 -0000 @@ -32,6 +32,7 @@ import java.util.*; import java.util.concurrent.*; import java.util.concurrent.atomic.*; +import static java.util.concurrent.TimeUnit.*; @SuppressWarnings({"unchecked", "rawtypes", "deprecation"}) public class OfferRemoveLoops { @@ -71,9 +72,10 @@ final long timeoutMillis = 10L * 1000L; final int maxChunkSize = 1042; final int maxQueueSize = 10 * maxChunkSize; + final CountDownLatch done = new CountDownLatch(3); - /** Poor man's bounded buffer. */ - final AtomicLong approximateCount = new AtomicLong(0L); + /** Poor man's bounded buffer; prevents unbounded queue expansion. */ + final Semaphore offers = new Semaphore(maxQueueSize); abstract class CheckedThread extends Thread { CheckedThread(String name) { @@ -89,42 +91,44 @@ protected boolean quittingTime(long i) { return (i % 1024) == 0 && quittingTime(); } - protected abstract void realRun(); + protected abstract void realRun() throws Exception; public void run() { try { realRun(); } catch (Throwable t) { unexpected(t); } } } Thread offerer = new CheckedThread("offerer") { - protected void realRun() { - final long chunkSize = getRandom().nextInt(maxChunkSize) + 2; + protected void realRun() throws InterruptedException { + final int chunkSize = getRandom().nextInt(maxChunkSize) + 20; long c = 0; - for (long i = 0; ! quittingTime(i); i++) { + while (! quittingTime()) { if (q.offer(Long.valueOf(c))) { if ((++c % chunkSize) == 0) { - approximateCount.getAndAdd(chunkSize); - while (approximateCount.get() > maxQueueSize) - Thread.yield(); + offers.acquire(chunkSize); } } else { Thread.yield(); - }}}}; + } + } + done.countDown(); + }}; Thread remover = new CheckedThread("remover") { protected void realRun() { - final long chunkSize = getRandom().nextInt(maxChunkSize) + 2; + final int chunkSize = getRandom().nextInt(maxChunkSize) + 20; long c = 0; - for (long i = 0; ! quittingTime(i); i++) { + while (! quittingTime()) { if (q.remove(Long.valueOf(c))) { if ((++c % chunkSize) == 0) { - approximateCount.getAndAdd(-chunkSize); + offers.release(chunkSize); } } else { Thread.yield(); } } q.clear(); - approximateCount.set(0); // Releases waiting offerer thread + offers.release(1<<30); // Releases waiting offerer thread + done.countDown(); }}; Thread scanner = new CheckedThread("scanner") { @@ -139,18 +143,19 @@ break; } Thread.yield(); - }}}; + } + done.countDown(); + }}; - for (Thread thread : new Thread[] { offerer, remover, scanner }) { - thread.join(timeoutMillis + testDurationMillis); - if (thread.isAlive()) { - System.err.printf("Hung thread: %s%n", thread.getName()); - failed++; - for (StackTraceElement e : thread.getStackTrace()) - System.err.println(e); - // Kludge alert - thread.stop(); - thread.join(timeoutMillis); + if (! done.await(timeoutMillis + testDurationMillis, MILLISECONDS)) { + for (Thread thread : new Thread[] { offerer, remover, scanner }) { + if (thread.isAlive()) { + System.err.printf("Hung thread: %s%n", thread.getName()); + failed++; + for (StackTraceElement e : thread.getStackTrace()) + System.err.println(e); + thread.interrupt(); + } } } } On Sun, Feb 23, 2014 at 4:23 PM, Tristan Yan wrote: > Thank you for reviewing this. Martin > > > > The original bug report is here: > https://bugs.openjdk.java.net/browse/JDK-8031374. > > > > I haven?t reproduced this bug but I can simulate to reproduce this failure > by changing yield() in remover thread to Thread.sleep(20). > > You have commented ?You've completely broken the intent of this code, > which was to poll ***occasionally*** for quitting time?. I tried to not > changed the logic for the test. This failure comes when only 1strounds(1024 loop) wasn?t finished in given quitting time(before timeout). > Which was 300ms. One solution is increasing default timeout as you > suggested. But question is how big should we increase. When the test is > really slow and could not finish 1st round(1024 loop) before timeout, I > couldn?t figure out what?s the reason timeout value. Also this definitely > will slow down the tests when it run in fast machine. Which is the case > most of time. So I took other step that skip wait if test doesn't finish 1 > st round(1024 loop) before timeout. I won?t say I completely broke the > intent of the code here because it?s rare case.(Only if the machine is slow > and slow enough that the test doesn't finish 1st round before timeout). > > > > The reason that replace Thread.yield to Thread.sleep(0L) because David > Holmes point out in the bug ?sleep will certainly give up the CPU, > whereas yield is not required to.? Also in other mail, David pointed I > should use Thread.sleep(10L) instead of Thread.sleep(0L) preferably a bit > longer as we don't know how the OS will behave if the sleep time requested > is less than the natural resolution of the sleep mechanism. For the > experiment I?ve done in Windows. Thread.yeild() or Thread.sleep(10L) won?t > guarantee current thread give up the CPU. This is a hint to OS. This makes > in windows remover and offer thread could wait to each other more more than > other other operating system. This is also the one of the reason that can > explain why we've seen this in windows only. > > > > Regards > > Tristan > > > > > > > > *???**:* Martin Buchholz [mailto:martinrb at google.com] > *????:* Monday, February 24, 2014 3:47 AM > *???:* Tristan Yan > *??:* core-libs-dev > *??:* Re: RFR for JDK-8031374: > java/util/concurrent/ConcurrentQueues/OfferRemoveLoops.java fails > Intermittently > > > > Hi Tristan, > > > > (thanks for working on my old crappy tests, and apologies for always > giving you a hard time) > > > > I couldn't find the bug report. Can you provide a URL? > > > > Thread.stop is only called in case the test has already failed (presumed > "hung"), as a last resort. > > > > Perhaps the timeout used in the test (300 ms) is too small on some systems? > > > > + protected void giveupCPU(){ > > + try { > > + Thread.sleep(0L); > > + } catch (InterruptedException ignore) {} > > } > > > > I know of no reason why Thread.sleep(0) should be any more effective than > Thread.yield. If it was more effective, then why wouldn't Thread.yield on > that platform be fixed to be implemented exactly the same way? IOW if this > is a problem, fix the JDK! > > > > /** Polls occasionally for quitting time. */ > > protected boolean quittingTime(long i) { > > - return (i % 1024) == 0 && quittingTime(); > > + return stopRequest || quittingTime() && (i % 1024 == 0 || i < 1024); > > + } > > > > You've completely broken the intent of this code, which was to poll > ***occasionally*** for quitting time. > > > > > > On Sun, Feb 23, 2014 at 1:40 AM, Tristan Yan > wrote: > > Hi All > Could you please help to review fix for JDK-803174. > > http://cr.openjdk.java.net/~tyan/JDK-8031374/webrev.00/ > > Description: > There are a couple of code change for this code. > 1. Original code has used known problematic Thread.stop. Which could cause > ThreadDeath as all we know. I change it to a customized stopThread method. > 2. Test failed in windows, I analyze the failure by changing > Thread.yield() in remover thread to Thread.sleep(50). This is a simulation > for slow machine. Which shows exact same failures as we can see in bug > description. By adding more debug info, we can see although we tried to > give up CPU by using Thread.yield().(I have tried to use Thread.sleep(1L) > as well, the result is same), there is no guarantee that os will pick up a > new thread to execute. In Windows, this is more obvious. Because the > execution is slow. even when the timeout happens, offer thread and remove > thread won?t have chance to get the point that i % 1024 == 0. This will > cause the failure like we see in the log. My fix here is when the timeout > happens, but i is still less than 1024. Stop offer thread and remover > thread right away instead letting them continuously wait the point to i == > 1024. > 3. I replace Thread.yield to Thread.sleep(0L). I saw a couple of > discussion that Thread.yield is not required to give up CPU. > > Thank you > Tristan > > > From martinrb at google.com Mon Feb 24 02:11:19 2014 From: martinrb at google.com (Martin Buchholz) Date: Sun, 23 Feb 2014 18:11:19 -0800 Subject: RFR for JDK-8031374: java/util/concurrent/ConcurrentQueues/OfferRemoveLoops.java fails Intermittently In-Reply-To: <530A7D49.6000404@oracle.com> References: <6B2B9D6A-F0DA-4F00-BE95-68AC7A03B6D0@oracle.com> <530A7D49.6000404@oracle.com> Message-ID: Thanks, David - I didn't know about CFS - Completely Fair Scheduler - and yield, but: It seems it's mostly about a thread not starving a lower-priority process. This is only a test, and yield is only here to help the other thread at the same priority gets some cycles, so it should be Mostly Harmless. That said, I eliminated one set of yields. From martinrb at google.com Mon Feb 24 02:16:01 2014 From: martinrb at google.com (Martin Buchholz) Date: Sun, 23 Feb 2014 18:16:01 -0800 Subject: =?UTF-8?Q?Re=3A_=E7=AD=94=E5=A4=8D=3A_RFR_for_JDK=2D8031374=3A_java=2Futil=2Fconcurren?= =?UTF-8?Q?t=2FConcurrentQueues=2FOfferRemoveLoops=2Ejava_fails_Intermittentl?= =?UTF-8?Q?y?= In-Reply-To: <65a677bc-304d-4a46-8465-7aa36c007212@default> References: <6B2B9D6A-F0DA-4F00-BE95-68AC7A03B6D0@oracle.com> <65a677bc-304d-4a46-8465-7aa36c007212@default> Message-ID: I may very well be missing something, but the actual extra timeout for threads to finish is 10 whole seconds, which should be long enough to process a single chunk, even on Windows. If you can repro this consistently, try to find out which queue implementation is affected. We can also shrink max queue size and chunk size to reduce time to traverse the queue elements. On Sun, Feb 23, 2014 at 4:23 PM, Tristan Yan wrote: > Thank you for reviewing this. Martin > > > > The original bug report is here: > https://bugs.openjdk.java.net/browse/JDK-8031374. > > > > I haven?t reproduced this bug but I can simulate to reproduce this failure > by changing yield() in remover thread to Thread.sleep(20). > > You have commented ?You've completely broken the intent of this code, > which was to poll ***occasionally*** for quitting time?. I tried to not > changed the logic for the test. This failure comes when only 1strounds(1024 loop) wasn?t finished in given quitting time(before timeout). > Which was 300ms. One solution is increasing default timeout as you > suggested. But question is how big should we increase. When the test is > really slow and could not finish 1st round(1024 loop) before timeout, I > couldn?t figure out what?s the reason timeout value. Also this definitely > will slow down the tests when it run in fast machine. Which is the case > most of time. So I took other step that skip wait if test doesn't finish 1 > st round(1024 loop) before timeout. I won?t say I completely broke the > intent of the code here because it?s rare case.(Only if the machine is slow > and slow enough that the test doesn't finish 1st round before timeout). > > > > The reason that replace Thread.yield to Thread.sleep(0L) because David > Holmes point out in the bug ?sleep will certainly give up the CPU, > whereas yield is not required to.? Also in other mail, David pointed I > should use Thread.sleep(10L) instead of Thread.sleep(0L) preferably a bit > longer as we don't know how the OS will behave if the sleep time requested > is less than the natural resolution of the sleep mechanism. For the > experiment I?ve done in Windows. Thread.yeild() or Thread.sleep(10L) won?t > guarantee current thread give up the CPU. This is a hint to OS. This makes > in windows remover and offer thread could wait to each other more more than > other other operating system. This is also the one of the reason that can > explain why we've seen this in windows only. > > > > Regards > > Tristan > > > > > > > > *???**:* Martin Buchholz [mailto:martinrb at google.com] > *????:* Monday, February 24, 2014 3:47 AM > *???:* Tristan Yan > *??:* core-libs-dev > *??:* Re: RFR for JDK-8031374: > java/util/concurrent/ConcurrentQueues/OfferRemoveLoops.java fails > Intermittently > > > > Hi Tristan, > > > > (thanks for working on my old crappy tests, and apologies for always > giving you a hard time) > > > > I couldn't find the bug report. Can you provide a URL? > > > > Thread.stop is only called in case the test has already failed (presumed > "hung"), as a last resort. > > > > Perhaps the timeout used in the test (300 ms) is too small on some systems? > > > > + protected void giveupCPU(){ > > + try { > > + Thread.sleep(0L); > > + } catch (InterruptedException ignore) {} > > } > > > > I know of no reason why Thread.sleep(0) should be any more effective than > Thread.yield. If it was more effective, then why wouldn't Thread.yield on > that platform be fixed to be implemented exactly the same way? IOW if this > is a problem, fix the JDK! > > > > /** Polls occasionally for quitting time. */ > > protected boolean quittingTime(long i) { > > - return (i % 1024) == 0 && quittingTime(); > > + return stopRequest || quittingTime() && (i % 1024 == 0 || i < 1024); > > + } > > > > You've completely broken the intent of this code, which was to poll > ***occasionally*** for quitting time. > > > > > > On Sun, Feb 23, 2014 at 1:40 AM, Tristan Yan > wrote: > > Hi All > Could you please help to review fix for JDK-803174. > > http://cr.openjdk.java.net/~tyan/JDK-8031374/webrev.00/ > > Description: > There are a couple of code change for this code. > 1. Original code has used known problematic Thread.stop. Which could cause > ThreadDeath as all we know. I change it to a customized stopThread method. > 2. Test failed in windows, I analyze the failure by changing > Thread.yield() in remover thread to Thread.sleep(50). This is a simulation > for slow machine. Which shows exact same failures as we can see in bug > description. By adding more debug info, we can see although we tried to > give up CPU by using Thread.yield().(I have tried to use Thread.sleep(1L) > as well, the result is same), there is no guarantee that os will pick up a > new thread to execute. In Windows, this is more obvious. Because the > execution is slow. even when the timeout happens, offer thread and remove > thread won?t have chance to get the point that i % 1024 == 0. This will > cause the failure like we see in the log. My fix here is when the timeout > happens, but i is still less than 1024. Stop offer thread and remover > thread right away instead letting them continuously wait the point to i == > 1024. > 3. I replace Thread.yield to Thread.sleep(0L). I saw a couple of > discussion that Thread.yield is not required to give up CPU. > > Thank you > Tristan > > > From tristan.yan at oracle.com Mon Feb 24 02:41:13 2014 From: tristan.yan at oracle.com (Tristan Yan) Date: Sun, 23 Feb 2014 18:41:13 -0800 (PST) Subject: =?utf-8?B?562U5aSNOiDnrZTlpI06IFJGUiBmb3IgSkRLLTgwMzEzNzQ=?= =?utf-8?B?OiBqYXZhL3V0aWwvY29uY3VycmVudC9Db25j?= =?utf-8?B?dXJyZW50UXVldWVzL09mZmVyUmVtb3ZlTG9v?= =?utf-8?B?cHMuamF2YSBmYWlscyBJbnRlcm1pdHRlbnRseQ==?= In-Reply-To: References: <6B2B9D6A-F0DA-4F00-BE95-68AC7A03B6D0@oracle.com> <65a677bc-304d-4a46-8465-7aa36c007212@default> Message-ID: <8e9d40cf-aad5-4b93-a003-ff326a479ade@default> Hi Martin This test failed once in our nightly test. So this is a real case failure. But unfortunately we couldn?t know more except the log we had. 10 extra seconds may need serve 1024 loop totally, also in windows Thread.yield() doesn?t give up CPU most of times, then we can may have the situation that remover is trying to remove object from a empty queue but it doesn?t find anything a couple of times(it doesn?t give up cpu time) then offer thread get cpu time. And offer thread insert a couple of elements and queue is full. ?Offer thread tries to give up CPU but Thread.yield() doesn?t really give up. Let?s assume it takes 1024 loop again. And offer thread got timeout. ?Then remover thread try to remove elements, and it takes another 1024 loop to get to timeout as well. So 10 seconds may need distribute to 2048 loop at most. Every loop has 5 ms foreach. That?s why I simulate this case with a Thread.sleep(20). I admit I can?t reproduce it without changing code, this is all theoretical analysis. But this is closest possible reason that I can deduce. Thank you. Tristan ? ???: Martin Buchholz [mailto:martinrb at google.com] ????: Monday, February 24, 2014 10:16 AM ???: Tristan Yan ??: core-libs-dev ??: Re: ??: RFR for JDK-8031374: java/util/concurrent/ConcurrentQueues/OfferRemoveLoops.java fails Intermittently ? I may very well be missing something, but the actual extra timeout for threads to finish is 10 whole seconds, which should be long enough to process a single chunk, even on Windows. ? ? If you can repro this consistently, try to find out which queue implementation is affected. ? ? We can also shrink max queue size and chunk size to reduce time to traverse the queue elements. ? On Sun, Feb 23, 2014 at 4:23 PM, Tristan Yan wrote: Thank you for reviewing this. Martin ? The original bug report is here: https://bugs.openjdk.java.net/browse/JDK-8031374. ? I haven?t reproduced this bug but I can simulate to reproduce this failure by changing ?yield() in remover thread to Thread.sleep(20). You have commented ?You've completely broken the intent of this code, which was to poll ***occasionally*** for quitting time?. I tried to not changed the logic for the test. This failure comes when only 1st rounds(1024 loop) wasn?t finished in given quitting time(before timeout). Which was 300ms. One solution is increasing default timeout as you suggested. But question is how big should we increase. When the test is really slow and could not finish 1st round(1024 loop) before timeout, I couldn?t figure out what?s the reason timeout value. Also this definitely will slow down the tests when it run in fast machine. Which is the case most of time. So I took other step that skip wait if test doesn't finish 1st round(1024 loop) before timeout. I won?t say I completely broke the intent of the code here because it?s rare case.(Only if the machine is slow and slow enough that the test doesn't finish 1st round before timeout). ? The reason that replace Thread.yield to Thread.sleep(0L) because David Holmes point out in the bug ?sleep will certainly give up the CPU, whereas yield is not required to.? Also in other mail, David pointed I should use Thread.sleep(10L) instead of Thread.sleep(0L) preferably a bit longer as we don't know how the OS will behave if the sleep time requested is less than the natural resolution of the sleep mechanism. For the experiment I?ve done in Windows. Thread.yeild() or Thread.sleep(10L) won?t guarantee current thread give up the CPU. This is a hint to OS. This makes in windows remover and offer thread could wait to each other more more than other other operating system. This is also the one of the reason that can explain why we've seen this in windows only. ? Regards Tristan ? ? ? ???: Martin Buchholz [mailto:HYPERLINK "mailto:martinrb at google.com"martinrb at google.com] ????: Monday, February 24, 2014 3:47 AM ???: Tristan Yan ??: core-libs-dev ??: Re: RFR for JDK-8031374: java/util/concurrent/ConcurrentQueues/OfferRemoveLoops.java fails Intermittently ? Hi Tristan, ? (thanks for working on my old crappy tests, and apologies for always giving you a hard time) ? I couldn't find the bug report. ?Can you provide a URL? ? Thread.stop is only called in case the test has already failed (presumed "hung"), as a last resort. ? Perhaps the timeout used in the test (300 ms) is too small on some systems? ? +??????????? protected void giveupCPU(){ +????????????? ??try { +??????????????????? Thread.sleep(0L); +??????????????? } catch (InterruptedException ignore) {} ???????????? } ? I know of no reason why Thread.sleep(0) should be any more effective than Thread.yield. ?If it was more effective, then why wouldn't Thread.yield on that platform be fixed to be implemented exactly the same way? ?IOW if this is a problem, fix the JDK! ? ???????????? /** Polls occasionally for quitting time. */ ???????????? protected boolean quittingTime(long i) { -??????????????? return (i % 1024) == 0 && quittingTime(); +??????????????? return stopRequest || quittingTime() && (i % 1024 == 0 || i < 1024); +??????????? } ? You've completely broken the intent of this code, which was to poll ***occasionally*** for quitting time. ? ? On Sun, Feb 23, 2014 at 1:40 AM, Tristan Yan wrote: Hi All Could you please help to review fix for JDK-803174. http://cr.openjdk.java.net/~tyan/JDK-8031374/webrev.00/ Description: There are a couple of code change for this code. 1. Original code has used known problematic Thread.stop. Which could cause ThreadDeath as all we know. I change it to a customized stopThread method. 2. Test failed in windows, I analyze the failure by changing Thread.yield() in remover thread to Thread.sleep(50). This is a simulation for slow machine. Which shows exact same failures as we can see in bug description. By adding more debug info, we can see although we tried to give up CPU by using Thread.yield().(I have tried to use Thread.sleep(1L) as well, the result is same), there is no guarantee that os will pick up a new thread to execute. In Windows, this is more obvious. Because the execution is slow. even when the timeout happens, offer thread and remove thread won?t have chance to get the point that i % 1024 == 0. This will cause the failure like we see in the log. My fix here is when the timeout happens, but i is still less than 1024. Stop offer thread and remover thread right away instead letting them continuously wait the point to i == 1024. 3. I replace Thread.yield to Thread.sleep(0L). I saw a couple of discussion that Thread.yield is not required to give up CPU. Thank you Tristan ? ? From martinrb at google.com Mon Feb 24 02:59:01 2014 From: martinrb at google.com (Martin Buchholz) Date: Sun, 23 Feb 2014 18:59:01 -0800 Subject: =?UTF-8?B?UmU6IOetlOWkjTog562U5aSNOiBSRlIgZm9yIEpESy04MDMxMzc0OiBqYXZhL3V0aWwvYw==?= =?UTF-8?B?b25jdXJyZW50L0NvbmN1cnJlbnRRdWV1ZXMvT2ZmZXJSZW1vdmVMb29wcy5qYXZhIGZhaWxzIEludGVy?= =?UTF-8?B?bWl0dGVudGx5?= In-Reply-To: <8e9d40cf-aad5-4b93-a003-ff326a479ade@default> References: <6B2B9D6A-F0DA-4F00-BE95-68AC7A03B6D0@oracle.com> <65a677bc-304d-4a46-8465-7aa36c007212@default> <8e9d40cf-aad5-4b93-a003-ff326a479ade@default> Message-ID: Hi Tristan, I don't think we should change the test without a sufficiently solid reason, which usually means reproducing the failure, even if e.g. only once in 1000 times. Here we don't know which queue implementation is causing the failure, and we don't know what chunk size was being used, which might cause the failure to be reproduced more reliably. A single test failure might be due to cosmic rays! Let's leave unfixed this while it cannot be reproduced. You can check in my general purpose improvements to the test from jsr166 CVS. On Sun, Feb 23, 2014 at 6:41 PM, Tristan Yan wrote: > Hi Martin > > This test failed once in our nightly test. So this is a real case failure. > But unfortunately we couldn?t know more except the log we had. > > 10 extra seconds may need serve 1024 loop totally, also in windows > Thread.yield() doesn?t give up CPU most of times, then we can may have the > situation that remover is trying to remove object from a empty queue but it > doesn?t find anything a couple of times(it doesn?t give up cpu time) then > offer thread get cpu time. And offer thread insert a couple of elements and > queue is full. Offer thread tries to give up CPU but Thread.yield() > doesn?t really give up. Let?s assume it takes 1024 loop again. And offer > thread got timeout. Then remover thread try to remove elements, and it > takes another 1024 loop to get to timeout as well. So 10 seconds may need > distribute to 2048 loop at most. Every loop has 5 ms foreach. That?s why I > simulate this case with a Thread.sleep(20). I admit I can?t reproduce it > without changing code, this is all theoretical analysis. But this is > closest possible reason that I can deduce. > > Thank you. > > Tristan > > > > *???**:* Martin Buchholz [mailto:martinrb at google.com] > *????:* Monday, February 24, 2014 10:16 AM > > *???:* Tristan Yan > *??:* core-libs-dev > *??:* Re: ??: RFR for JDK-8031374: > java/util/concurrent/ConcurrentQueues/OfferRemoveLoops.java fails > Intermittently > > > > I may very well be missing something, but the actual extra timeout for > threads to finish is 10 whole seconds, which should be long enough to > process a single chunk, even on Windows. > > > > If you can repro this consistently, try to find out which queue > implementation is affected. > > > > We can also shrink max queue size and chunk size to reduce time to > traverse the queue elements. > > > > On Sun, Feb 23, 2014 at 4:23 PM, Tristan Yan > wrote: > > Thank you for reviewing this. Martin > > > > The original bug report is here: > https://bugs.openjdk.java.net/browse/JDK-8031374. > > > > I haven?t reproduced this bug but I can simulate to reproduce this failure > by changing yield() in remover thread to Thread.sleep(20). > > You have commented ?You've completely broken the intent of this code, > which was to poll ***occasionally*** for quitting time?. I tried to not > changed the logic for the test. This failure comes when only 1strounds(1024 loop) wasn?t finished in given quitting time(before timeout). > Which was 300ms. One solution is increasing default timeout as you > suggested. But question is how big should we increase. When the test is > really slow and could not finish 1st round(1024 loop) before timeout, I > couldn?t figure out what?s the reason timeout value. Also this definitely > will slow down the tests when it run in fast machine. Which is the case > most of time. So I took other step that skip wait if test doesn't finish 1 > st round(1024 loop) before timeout. I won?t say I completely broke the > intent of the code here because it?s rare case.(Only if the machine is slow > and slow enough that the test doesn't finish 1st round before timeout). > > > > The reason that replace Thread.yield to Thread.sleep(0L) because David > Holmes point out in the bug ?sleep will certainly give up the CPU, > whereas yield is not required to.? Also in other mail, David pointed I > should use Thread.sleep(10L) instead of Thread.sleep(0L) preferably a bit > longer as we don't know how the OS will behave if the sleep time requested > is less than the natural resolution of the sleep mechanism. For the > experiment I?ve done in Windows. Thread.yeild() or Thread.sleep(10L) won?t > guarantee current thread give up the CPU. This is a hint to OS. This makes > in windows remover and offer thread could wait to each other more more than > other other operating system. This is also the one of the reason that can > explain why we've seen this in windows only. > > > > Regards > > Tristan > > > > > > > > *???**:* Martin Buchholz [mailto:martinrb at google.com] > *????:* Monday, February 24, 2014 3:47 AM > *???:* Tristan Yan > *??:* core-libs-dev > *??:* Re: RFR for JDK-8031374: > java/util/concurrent/ConcurrentQueues/OfferRemoveLoops.java fails > Intermittently > > > > Hi Tristan, > > > > (thanks for working on my old crappy tests, and apologies for always > giving you a hard time) > > > > I couldn't find the bug report. Can you provide a URL? > > > > Thread.stop is only called in case the test has already failed (presumed > "hung"), as a last resort. > > > > Perhaps the timeout used in the test (300 ms) is too small on some systems? > > > > + protected void giveupCPU(){ > > + try { > > + Thread.sleep(0L); > > + } catch (InterruptedException ignore) {} > > } > > > > I know of no reason why Thread.sleep(0) should be any more effective than > Thread.yield. If it was more effective, then why wouldn't Thread.yield on > that platform be fixed to be implemented exactly the same way? IOW if this > is a problem, fix the JDK! > > > > /** Polls occasionally for quitting time. */ > > protected boolean quittingTime(long i) { > > - return (i % 1024) == 0 && quittingTime(); > > + return stopRequest || quittingTime() && (i % 1024 == 0 || i < 1024); > > + } > > > > You've completely broken the intent of this code, which was to poll > ***occasionally*** for quitting time. > > > > > > On Sun, Feb 23, 2014 at 1:40 AM, Tristan Yan > wrote: > > Hi All > Could you please help to review fix for JDK-803174. > > http://cr.openjdk.java.net/~tyan/JDK-8031374/webrev.00/ > > Description: > There are a couple of code change for this code. > 1. Original code has used known problematic Thread.stop. Which could cause > ThreadDeath as all we know. I change it to a customized stopThread method. > 2. Test failed in windows, I analyze the failure by changing > Thread.yield() in remover thread to Thread.sleep(50). This is a simulation > for slow machine. Which shows exact same failures as we can see in bug > description. By adding more debug info, we can see although we tried to > give up CPU by using Thread.yield().(I have tried to use Thread.sleep(1L) > as well, the result is same), there is no guarantee that os will pick up a > new thread to execute. In Windows, this is more obvious. Because the > execution is slow. even when the timeout happens, offer thread and remove > thread won?t have chance to get the point that i % 1024 == 0. This will > cause the failure like we see in the log. My fix here is when the timeout > happens, but i is still less than 1024. Stop offer thread and remover > thread right away instead letting them continuously wait the point to i == > 1024. > 3. I replace Thread.yield to Thread.sleep(0L). I saw a couple of > discussion that Thread.yield is not required to give up CPU. > > Thank you > Tristan > > > > > From tristan.yan at oracle.com Mon Feb 24 03:03:09 2014 From: tristan.yan at oracle.com (Tristan Yan) Date: Sun, 23 Feb 2014 19:03:09 -0800 (PST) Subject: =?utf-8?B?562U5aSNOiDnrZTlpI06IOetlOWkjTogUkZSIGZvciBKREstODAzMQ==?= =?utf-8?B?Mzc0OiBqYXZhL3V0aWwvY29uY3VycmVudC9Dbw==?= =?utf-8?B?bmN1cnJlbnRRdWV1ZXMvT2ZmZXJSZW1vdmVMbw==?= =?utf-8?B?b3BzLmphdmEgZmFpbHMgSW50ZXJtaXR0ZW50bHk=?= In-Reply-To: References: <6B2B9D6A-F0DA-4F00-BE95-68AC7A03B6D0@oracle.com> <65a677bc-304d-4a46-8465-7aa36c007212@default> <8e9d40cf-aad5-4b93-a003-ff326a479ade@default> Message-ID: <1d660f66-d999-4e0b-886e-38476a57945c@default> I am ok with this unfix for now. Martin. And thank you for bringing the improvement to jsr166 CVS. Regards Tristan ? ???: Martin Buchholz [mailto:martinrb at google.com] ????: Monday, February 24, 2014 10:59 AM ???: Tristan Yan ??: core-libs-dev ??: Re: ??: ??: RFR for JDK-8031374: java/util/concurrent/ConcurrentQueues/OfferRemoveLoops.java fails Intermittently ? Hi Tristan, ? I don't think we should change the test without a sufficiently solid reason, which usually means reproducing the failure, even if e.g. only once in 1000 times. ?Here we don't know which queue implementation is causing the failure, and we don't know what chunk size was being used, which might cause the failure to be reproduced more reliably. ?A single test failure might be due to cosmic rays! ?Let's leave unfixed this while it cannot be reproduced. ?You can check in my general purpose improvements to the test from jsr166 CVS. ? On Sun, Feb 23, 2014 at 6:41 PM, Tristan Yan wrote: Hi Martin This test failed once in our nightly test. So this is a real case failure. But unfortunately we couldn?t know more except the log we had. 10 extra seconds may need serve 1024 loop totally, also in windows Thread.yield() doesn?t give up CPU most of times, then we can may have the situation that remover is trying to remove object from a empty queue but it doesn?t find anything a couple of times(it doesn?t give up cpu time) then offer thread get cpu time. And offer thread insert a couple of elements and queue is full. ?Offer thread tries to give up CPU but Thread.yield() doesn?t really give up. Let?s assume it takes 1024 loop again. And offer thread got timeout. ?Then remover thread try to remove elements, and it takes another 1024 loop to get to timeout as well. So 10 seconds may need distribute to 2048 loop at most. Every loop has 5 ms foreach. That?s why I simulate this case with a Thread.sleep(20). I admit I can?t reproduce it without changing code, this is all theoretical analysis. But this is closest possible reason that I can deduce. Thank you. Tristan ? ???: Martin Buchholz [mailto:HYPERLINK "mailto:martinrb at google.com"martinrb at google.com] ????: Monday, February 24, 2014 10:16 AM ???: Tristan Yan ??: core-libs-dev ??: Re: ??: RFR for JDK-8031374: java/util/concurrent/ConcurrentQueues/OfferRemoveLoops.java fails Intermittently ? I may very well be missing something, but the actual extra timeout for threads to finish is 10 whole seconds, which should be long enough to process a single chunk, even on Windows. ? ? If you can repro this consistently, try to find out which queue implementation is affected. ? ? We can also shrink max queue size and chunk size to reduce time to traverse the queue elements. ? On Sun, Feb 23, 2014 at 4:23 PM, Tristan Yan wrote: Thank you for reviewing this. Martin ? The original bug report is here: https://bugs.openjdk.java.net/browse/JDK-8031374. ? I haven?t reproduced this bug but I can simulate to reproduce this failure by changing ?yield() in remover thread to Thread.sleep(20). You have commented ?You've completely broken the intent of this code, which was to poll ***occasionally*** for quitting time?. I tried to not changed the logic for the test. This failure comes when only 1st rounds(1024 loop) wasn?t finished in given quitting time(before timeout). Which was 300ms. One solution is increasing default timeout as you suggested. But question is how big should we increase. When the test is really slow and could not finish 1st round(1024 loop) before timeout, I couldn?t figure out what?s the reason timeout value. Also this definitely will slow down the tests when it run in fast machine. Which is the case most of time. So I took other step that skip wait if test doesn't finish 1st round(1024 loop) before timeout. I won?t say I completely broke the intent of the code here because it?s rare case.(Only if the machine is slow and slow enough that the test doesn't finish 1st round before timeout). ? The reason that replace Thread.yield to Thread.sleep(0L) because David Holmes point out in the bug ?sleep will certainly give up the CPU, whereas yield is not required to.? Also in other mail, David pointed I should use Thread.sleep(10L) instead of Thread.sleep(0L) preferably a bit longer as we don't know how the OS will behave if the sleep time requested is less than the natural resolution of the sleep mechanism. For the experiment I?ve done in Windows. Thread.yeild() or Thread.sleep(10L) won?t guarantee current thread give up the CPU. This is a hint to OS. This makes in windows remover and offer thread could wait to each other more more than other other operating system. This is also the one of the reason that can explain why we've seen this in windows only. ? Regards Tristan ? ? ? ???: Martin Buchholz [mailto:HYPERLINK "mailto:martinrb at google.com"martinrb at google.com] ????: Monday, February 24, 2014 3:47 AM ???: Tristan Yan ??: core-libs-dev ??: Re: RFR for JDK-8031374: java/util/concurrent/ConcurrentQueues/OfferRemoveLoops.java fails Intermittently ? Hi Tristan, ? (thanks for working on my old crappy tests, and apologies for always giving you a hard time) ? I couldn't find the bug report. ?Can you provide a URL? ? Thread.stop is only called in case the test has already failed (presumed "hung"), as a last resort. ? Perhaps the timeout used in the test (300 ms) is too small on some systems? ? +??????????? protected void giveupCPU(){ +????????????? ??try { +??????????????????? Thread.sleep(0L); +??????????????? } catch (InterruptedException ignore) {} ???????????? } ? I know of no reason why Thread.sleep(0) should be any more effective than Thread.yield. ?If it was more effective, then why wouldn't Thread.yield on that platform be fixed to be implemented exactly the same way? ?IOW if this is a problem, fix the JDK! ? ???????????? /** Polls occasionally for quitting time. */ ???????????? protected boolean quittingTime(long i) { -??????????????? return (i % 1024) == 0 && quittingTime(); +??????????????? return stopRequest || quittingTime() && (i % 1024 == 0 || i < 1024); +??????????? } ? You've completely broken the intent of this code, which was to poll ***occasionally*** for quitting time. ? ? On Sun, Feb 23, 2014 at 1:40 AM, Tristan Yan wrote: Hi All Could you please help to review fix for JDK-803174. http://cr.openjdk.java.net/~tyan/JDK-8031374/webrev.00/ Description: There are a couple of code change for this code. 1. Original code has used known problematic Thread.stop. Which could cause ThreadDeath as all we know. I change it to a customized stopThread method. 2. Test failed in windows, I analyze the failure by changing Thread.yield() in remover thread to Thread.sleep(50). This is a simulation for slow machine. Which shows exact same failures as we can see in bug description. By adding more debug info, we can see although we tried to give up CPU by using Thread.yield().(I have tried to use Thread.sleep(1L) as well, the result is same), there is no guarantee that os will pick up a new thread to execute. In Windows, this is more obvious. Because the execution is slow. even when the timeout happens, offer thread and remove thread won?t have chance to get the point that i % 1024 == 0. This will cause the failure like we see in the log. My fix here is when the timeout happens, but i is still less than 1024. Stop offer thread and remover thread right away instead letting them continuously wait the point to i == 1024. 3. I replace Thread.yield to Thread.sleep(0L). I saw a couple of discussion that Thread.yield is not required to give up CPU. Thank you Tristan ? ? ? From ivan.gerasimov at oracle.com Mon Feb 24 05:59:14 2014 From: ivan.gerasimov at oracle.com (Ivan Gerasimov) Date: Mon, 24 Feb 2014 09:59:14 +0400 Subject: RFR [6853696] (ref) ReferenceQueue.remove(timeout) may return null even if timeout has not expired Message-ID: <530ADFB2.8000100@oracle.com> Hello! ReferenceQueue.remove(timeout) may return too early, i.e. before the specified timeout has elapsed. Would you please review the fix? The change also includes a regression test, which can be used to demonstrate the issue. BUGURL: https://bugs.openjdk.java.net/browse/JDK-6853696 WEBREV: http://cr.openjdk.java.net/~igerasim/6853696/0/webrev/ Sincerely yours, Ivan From petr.pchelko at oracle.com Mon Feb 24 09:02:13 2014 From: petr.pchelko at oracle.com (Petr Pchelko) Date: Mon, 24 Feb 2014 13:02:13 +0400 Subject: [9] Review request: JDK-8035640 JNU_CHECK_EXCEPTION should support c++ JNI syntax Message-ID: Hello, Please review the fix for the issue: https://bugs.openjdk.java.net/browse/JDK-8035640 The fix is available at: http://cr.openjdk.java.net/~pchelko/9/8035640/webrev.00/ In AWT code we have quite a lot of C++ sources, but JNU_CHECK_EXCEPTION macros could not be used there, because the JNI syntax is different in C++. If approved I'll integrate this fix into the client forest, because we need this in client to fix parfait issues. Thank you, With best regards. Petr. From chris.hegarty at oracle.com Mon Feb 24 09:05:05 2014 From: chris.hegarty at oracle.com (Chris Hegarty) Date: Mon, 24 Feb 2014 09:05:05 +0000 Subject: RFR for JDK-8031374: java/util/concurrent/ConcurrentQueues/OfferRemoveLoops.java fails Intermittently In-Reply-To: <1d660f66-d999-4e0b-886e-38476a57945c@default> References: <6B2B9D6A-F0DA-4F00-BE95-68AC7A03B6D0@oracle.com> <65a677bc-304d-4a46-8465-7aa36c007212@default> <8e9d40cf-aad5-4b93-a003-ff326a479ade@default> <1d660f66-d999-4e0b-886e-38476a57945c@default> Message-ID: <3BDEFCE4-28B5-40E4-AB59-ABD63D84C3CD@oracle.com> Tristan, Can you please file a new bug and bring in the changes that Martin pushed to the 166 CVS. http://gee.cs.oswego.edu/cgi-bin/viewcvs.cgi/jsr166/src/test/jtreg/util/concurrent/ConcurrentQueues/OfferRemoveLoops.java?view=co Create a changeset, and I can do the OpenJDK review. -Chris. On 24 Feb 2014, at 03:03, Tristan Yan wrote: > I am ok with this unfix for now. Martin. And thank you for bringing the improvement to jsr166 CVS. > > Regards > > Tristan > > > > ???: Martin Buchholz [mailto:martinrb at google.com] > ????: Monday, February 24, 2014 10:59 AM > ???: Tristan Yan > ??: core-libs-dev > ??: Re: ??: ??: RFR for JDK-8031374: java/util/concurrent/ConcurrentQueues/OfferRemoveLoops.java fails Intermittently > > > > Hi Tristan, > > > > I don't think we should change the test without a sufficiently solid reason, which usually means reproducing the failure, even if e.g. only once in 1000 times. Here we don't know which queue implementation is causing the failure, and we don't know what chunk size was being used, which might cause the failure to be reproduced more reliably. A single test failure might be due to cosmic rays! Let's leave unfixed this while it cannot be reproduced. You can check in my general purpose improvements to the test from jsr166 CVS. > > > > On Sun, Feb 23, 2014 at 6:41 PM, Tristan Yan wrote: > > Hi Martin > > This test failed once in our nightly test. So this is a real case failure. But unfortunately we couldn?t know more except the log we had. > > 10 extra seconds may need serve 1024 loop totally, also in windows Thread.yield() doesn?t give up CPU most of times, then we can may have the situation that remover is trying to remove object from a empty queue but it doesn?t find anything a couple of times(it doesn?t give up cpu time) then offer thread get cpu time. And offer thread insert a couple of elements and queue is full. Offer thread tries to give up CPU but Thread.yield() doesn?t really give up. Let?s assume it takes 1024 loop again. And offer thread got timeout. Then remover thread try to remove elements, and it takes another 1024 loop to get to timeout as well. So 10 seconds may need distribute to 2048 loop at most. Every loop has 5 ms foreach. That?s why I simulate this case with a Thread.sleep(20). I admit I can?t reproduce it without changing code, this is all theoretical analysis. But this is closest possible reason that I can deduce. > > Thank you. > > Tristan > > > > ???: Martin Buchholz [mailto:HYPERLINK "mailto:martinrb at google.com"martinrb at google.com] > ????: Monday, February 24, 2014 10:16 AM > > > ???: Tristan Yan > ??: core-libs-dev > > ??: Re: ??: RFR for JDK-8031374: java/util/concurrent/ConcurrentQueues/OfferRemoveLoops.java fails Intermittently > > > > I may very well be missing something, but the actual extra timeout for threads to finish is 10 whole seconds, which should be long enough to process a single chunk, even on Windows. > > > > If you can repro this consistently, try to find out which queue implementation is affected. > > > > We can also shrink max queue size and chunk size to reduce time to traverse the queue elements. > > > > On Sun, Feb 23, 2014 at 4:23 PM, Tristan Yan wrote: > > Thank you for reviewing this. Martin > > > > The original bug report is here: https://bugs.openjdk.java.net/browse/JDK-8031374. > > > > I haven?t reproduced this bug but I can simulate to reproduce this failure by changing yield() in remover thread to Thread.sleep(20). > > You have commented ?You've completely broken the intent of this code, which was to poll ***occasionally*** for quitting time?. I tried to not changed the logic for the test. This failure comes when only 1st rounds(1024 loop) wasn?t finished in given quitting time(before timeout). Which was 300ms. One solution is increasing default timeout as you suggested. But question is how big should we increase. When the test is really slow and could not finish 1st round(1024 loop) before timeout, I couldn?t figure out what?s the reason timeout value. Also this definitely will slow down the tests when it run in fast machine. Which is the case most of time. So I took other step that skip wait if test doesn't finish 1st round(1024 loop) before timeout. I won?t say I completely broke the intent of the code here because it?s rare case.(Only if the machine is slow and slow enough that the test doesn't finish 1st round before timeout). > > > > The reason that replace Thread.yield to Thread.sleep(0L) because David Holmes point out in the bug ?sleep will certainly give up the CPU, whereas yield is not required to.? Also in other mail, David pointed I should use Thread.sleep(10L) instead of Thread.sleep(0L) preferably a bit longer as we don't know how the OS will behave if the sleep time requested is less than the natural resolution of the sleep mechanism. For the experiment I?ve done in Windows. Thread.yeild() or Thread.sleep(10L) won?t guarantee current thread give up the CPU. This is a hint to OS. This makes in windows remover and offer thread could wait to each other more more than other other operating system. This is also the one of the reason that can explain why we've seen this in windows only. > > > > Regards > > Tristan > > > > > > > > ???: Martin Buchholz [mailto:HYPERLINK "mailto:martinrb at google.com"martinrb at google.com] > ????: Monday, February 24, 2014 3:47 AM > ???: Tristan Yan > ??: core-libs-dev > ??: Re: RFR for JDK-8031374: java/util/concurrent/ConcurrentQueues/OfferRemoveLoops.java fails Intermittently > > > > Hi Tristan, > > > > (thanks for working on my old crappy tests, and apologies for always giving you a hard time) > > > > I couldn't find the bug report. Can you provide a URL? > > > > Thread.stop is only called in case the test has already failed (presumed "hung"), as a last resort. > > > > Perhaps the timeout used in the test (300 ms) is too small on some systems? > > > > + protected void giveupCPU(){ > + try { > + Thread.sleep(0L); > + } catch (InterruptedException ignore) {} > } > > > > I know of no reason why Thread.sleep(0) should be any more effective than Thread.yield. If it was more effective, then why wouldn't Thread.yield on that platform be fixed to be implemented exactly the same way? IOW if this is a problem, fix the JDK! > > > > /** Polls occasionally for quitting time. */ > protected boolean quittingTime(long i) { > - return (i % 1024) == 0 && quittingTime(); > + return stopRequest || quittingTime() && (i % 1024 == 0 || i < 1024); > + } > > > > You've completely broken the intent of this code, which was to poll ***occasionally*** for quitting time. > > > > > > On Sun, Feb 23, 2014 at 1:40 AM, Tristan Yan wrote: > > Hi All > Could you please help to review fix for JDK-803174. > > http://cr.openjdk.java.net/~tyan/JDK-8031374/webrev.00/ > > Description: > There are a couple of code change for this code. > 1. Original code has used known problematic Thread.stop. Which could cause ThreadDeath as all we know. I change it to a customized stopThread method. > 2. Test failed in windows, I analyze the failure by changing Thread.yield() in remover thread to Thread.sleep(50). This is a simulation for slow machine. Which shows exact same failures as we can see in bug description. By adding more debug info, we can see although we tried to give up CPU by using Thread.yield().(I have tried to use Thread.sleep(1L) as well, the result is same), there is no guarantee that os will pick up a new thread to execute. In Windows, this is more obvious. Because the execution is slow. even when the timeout happens, offer thread and remove thread won?t have chance to get the point that i % 1024 == 0. This will cause the failure like we see in the log. My fix here is when the timeout happens, but i is still less than 1024. Stop offer thread and remover thread right away instead letting them continuously wait the point to i == 1024. > 3. I replace Thread.yield to Thread.sleep(0L). I saw a couple of discussion that Thread.yield is not required to give up CPU. > > Thank you > Tristan > > > > > > From paul.sandoz at oracle.com Mon Feb 24 11:23:20 2014 From: paul.sandoz at oracle.com (Paul Sandoz) Date: Mon, 24 Feb 2014 12:23:20 +0100 Subject: JDK 9 RFR of JDK-8035453: Fix serial lint warnings in com.sun.tools and elsewhere In-Reply-To: <53067053.2060501@oracle.com> References: <53067053.2060501@oracle.com> Message-ID: Hi Joe, On Feb 20, 2014, at 10:14 PM, Joe Darcy wrote: > Hello, > > Please review my proposed changes for: > > JDK-8035453: Fix serial lint warnings in com.sun.tools and elsewhere > http://cr.openjdk.java.net/~darcy/8035453.0/webrev/ > Looks good. I agree with Paul B. that it seems a little odd to have to suppress serialization on an abstract class, especially one that implements Serializable (plus Connector.Argument also extends Serializable): > --- old/src/share/classes/com/sun/tools/jdi/ConnectorImpl.java 2014-02-20 13:03:08.000000000 -0800 > +++ new/src/share/classes/com/sun/tools/jdi/ConnectorImpl.java 2014-02-20 13:03:08.000000000 -0800 > @@ -144,6 +144,7 @@ > return string; > } > > + @SuppressWarnings("serial") // JDK implementation class > abstract class ArgumentImpl implements Connector.Argument, Cloneable, Serializable { > private String name; > private String label; Paul. From Alan.Bateman at oracle.com Mon Feb 24 12:10:03 2014 From: Alan.Bateman at oracle.com (Alan Bateman) Date: Mon, 24 Feb 2014 12:10:03 +0000 Subject: [9] Review request: JDK-8035640 JNU_CHECK_EXCEPTION should support c++ JNI syntax In-Reply-To: References: Message-ID: <530B369B.3050707@oracle.com> On 24/02/2014 09:02, Petr Pchelko wrote: > Hello, > > Please review the fix for the issue: > https://bugs.openjdk.java.net/browse/JDK-8035640 > The fix is available at: > http://cr.openjdk.java.net/~pchelko/9/8035640/webrev.00/ > > In AWT code we have quite a lot of C++ sources, but JNU_CHECK_EXCEPTION macros could not be used there, because the JNI syntax is different in C++. > If approved I'll integrate this fix into the client forest, because we need this in client to fix parfait issues. > > Thank you, > With best regards. Petr. This looks okay to me. One suggestion is to use #endif /* __cplusplus */ so that it's consistent with the other usages (also makes it a bit easier when there are nested ifdefs). As regards logistics then jdk9/dev might be the more suitable forest to push this to. I suggest this because it looks to me that jdk9/client is pulling down changes from jdk9/dev very regularly (which is good). On the other hand there doesn't appear to be regular integrations from jdk9/client to jdk9/dev yet. I see changes in jdk9/client from mid-December that has still not been pushed to jdk9/dev. It's just a suggestion to ensure that the changes get to both forests in timely manner. -Alan. From petr.pchelko at oracle.com Mon Feb 24 12:21:46 2014 From: petr.pchelko at oracle.com (Petr Pchelko) Date: Mon, 24 Feb 2014 16:21:46 +0400 Subject: [9] Review request: JDK-8035640 JNU_CHECK_EXCEPTION should support c++ JNI syntax In-Reply-To: <530B369B.3050707@oracle.com> References: <530B369B.3050707@oracle.com> Message-ID: Hello, Alan. Thank you for the review. > This looks okay to me. One suggestion is to use #endif /* __cplusplus */ so that it's consistent with the other usages (also makes it a bit easier when there are nested ifdefs). Updated the fix: http://cr.openjdk.java.net/~pchelko/9/8035640/webrev.01/ > As regards logistics then jdk9/dev might be the more suitable forest to push this to. I suggest this because it looks to me that jdk9/client is pulling down changes from jdk9/dev very regularly (which is good). On the other hand there doesn't appear to be regular integrations from jdk9/client to jdk9/dev yet. I see changes in jdk9/client from mid-December that has still not been pushed to jdk9/dev. It's just a suggestion to ensure that the changes get to both forests in timely manner. No problem. I think we could easily wait until the next integration while dependent fixes are being reviewed. I'll push this into dev forest. With best regards. Petr. On 24.02.2014, at 16:10, Alan Bateman wrote: > On 24/02/2014 09:02, Petr Pchelko wrote: >> Hello, >> >> Please review the fix for the issue: >> https://bugs.openjdk.java.net/browse/JDK-8035640 >> The fix is available at: >> http://cr.openjdk.java.net/~pchelko/9/8035640/webrev.00/ >> >> In AWT code we have quite a lot of C++ sources, but JNU_CHECK_EXCEPTION macros could not be used there, because the JNI syntax is different in C++. >> If approved I'll integrate this fix into the client forest, because we need this in client to fix parfait issues. >> >> Thank you, >> With best regards. Petr. > This looks okay to me. One suggestion is to use #endif /* __cplusplus */ so that it's consistent with the other usages (also makes it a bit easier when there are nested ifdefs). > > As regards logistics then jdk9/dev might be the more suitable forest to push this to. I suggest this because it looks to me that jdk9/client is pulling down changes from jdk9/dev very regularly (which is good). On the other hand there doesn't appear to be regular integrations from jdk9/client to jdk9/dev yet. I see changes in jdk9/client from mid-December that has still not been pushed to jdk9/dev. It's just a suggestion to ensure that the changes get to both forests in timely manner. > > -Alan. From peter.levart at gmail.com Mon Feb 24 12:42:26 2014 From: peter.levart at gmail.com (Peter Levart) Date: Mon, 24 Feb 2014 13:42:26 +0100 Subject: [7u backport] RFR: 7122142: (ann) Race condition between isAnnotationPresent and getAnnotations In-Reply-To: <53089694.4070308@oracle.com> References: <53089694.4070308@oracle.com> Message-ID: <530B3E32.7020303@gmail.com> Hi Dmeetry, On 02/22/2014 01:22 PM, dmeetry degrave wrote: > Hi all, > > I would like to ask for a review of combined back port for > 7u-dev/7u80. The main goal is to have a fix for 7122142 in jdk7, it > also integrates the changes from 8005232, 7185456, 8022721 > > https://bugs.openjdk.java.net/browse/JDK-7122142 > https://bugs.openjdk.java.net/browse/JDK-8005232 > https://bugs.openjdk.java.net/browse/JDK-7185456 > https://bugs.openjdk.java.net/browse/JDK-8022721 > > Original jdk8 changes: > > 7122142: http://hg.openjdk.java.net/jdk8/tl/jdk/rev/e4ce6502eac0 > 8005232: http://hg.openjdk.java.net/jdk8/tl/jdk/rev/1109bfff4e92 > 7185456: http://hg.openjdk.java.net/jdk8/tl/jdk/rev/ae03282ba501 > 8022721: http://hg.openjdk.java.net/jdk8/tl/jdk/rev/2281a7f79738 > > back port: > > http://cr.openjdk.java.net/~dmeetry/7122142.8005232.7185456.8022721/webrev.0/ > > > Patches can't be applied cleanly, hence it was a manual back port, > though the final result is equivalent to applying the patches in > chronological order (8005232, 7185456, 7122142, 8022721) and applying > all the relevant rejected parts It's good to see those patches being back-ported to 7u. By browsing the webrev, I don't see any obvious difference between the original patches and the backport. Do you happen to remember in what part of code there were rejects so that you had to manually apply the changes? > (with one exception, AnnotationTypeRuntimeAssumptionTest.java test was > not included due to jdk8 API). Ah, It's the Class.getDeclaredAnnotation(Class) that's new in JDK8. Here's the changed test that only uses the JDK7 API so you can include this test too: http://cr.openjdk.java.net/~plevart/jdk7u/7122142/AnnotationTypeRuntimeAssumptionTest.java > > All tests in test/java/lang/annotation passed. > > thanks, > dmeetry Regards, Peter From peter.levart at gmail.com Mon Feb 24 13:46:23 2014 From: peter.levart at gmail.com (Peter Levart) Date: Mon, 24 Feb 2014 14:46:23 +0100 Subject: RFR - 6857566 (bf) DirectByteBuffer garbage creation can outpace reclamation In-Reply-To: <5309ACA3.7090500@oracle.com> References: <52F1F174.8070105@gmail.com> <52F21094.3090400@oracle.com> <5306345E.3060302@gmail.com> <5309ACA3.7090500@oracle.com> Message-ID: <530B4D2F.6070807@gmail.com> On 02/23/2014 09:09 AM, Alan Bateman wrote: >> http://cr.openjdk.java.net/~plevart/jdk9-dev/DirectBufferAlloc/webrev.03/ > Thanks for the updates, this is good work and I think this is good to go. Do I need another official approval? Regards, Peter From Alan.Bateman at oracle.com Mon Feb 24 13:49:01 2014 From: Alan.Bateman at oracle.com (Alan Bateman) Date: Mon, 24 Feb 2014 13:49:01 +0000 Subject: RFR - 6857566 (bf) DirectByteBuffer garbage creation can outpace reclamation In-Reply-To: <530B4D2F.6070807@gmail.com> References: <52F1F174.8070105@gmail.com> <52F21094.3090400@oracle.com> <5306345E.3060302@gmail.com> <5309ACA3.7090500@oracle.com> <530B4D2F.6070807@gmail.com> Message-ID: <530B4DCD.9050306@oracle.com> On 24/02/2014 13:46, Peter Levart wrote: > On 02/23/2014 09:09 AM, Alan Bateman wrote: >>> http://cr.openjdk.java.net/~plevart/jdk9-dev/DirectBufferAlloc/webrev.03/ >>> >> Thanks for the updates, this is good work and I think this is good to >> go. > > Do I need another official approval? Changes to the jdk repository only require one Reviewer (I think it's just the hotspot repository where it requires two). -Alan From tristan.yan at oracle.com Mon Feb 24 13:52:36 2014 From: tristan.yan at oracle.com (Tristan Yan) Date: Mon, 24 Feb 2014 05:52:36 -0800 (PST) Subject: =?utf-8?B?562U5aSNOiBSRlIgZm9yIEpESy04MDMxMzc0OiBq?= =?utf-8?B?YXZhL3V0aWwvY29uY3VycmVudC9Db25jdXI=?= =?utf-8?B?cmVudFF1ZXVlcy9PZmZlclJlbW92ZUxvb3A=?= =?utf-8?B?cy5qYXZhIGZhaWxzIEludGVybWl0dGVudGx5?= In-Reply-To: <3BDEFCE4-28B5-40E4-AB59-ABD63D84C3CD@oracle.com> References: <6B2B9D6A-F0DA-4F00-BE95-68AC7A03B6D0@oracle.com> <65a677bc-304d-4a46-8465-7aa36c007212@default> <8e9d40cf-aad5-4b93-a003-ff326a479ade@default> <1d660f66-d999-4e0b-886e-38476a57945c@default> <3BDEFCE4-28B5-40E4-AB59-ABD63D84C3CD@oracle.com> Message-ID: <7b1e2f13-8732-40ae-aaf8-f722d4e5bfd0@default> Thank you for taking care of this. Chris Filed a bug for this https://bugs.openjdk.java.net/browse/JDK-8035661 Tristan -----????----- ???: Chris Hegarty ????: Monday, February 24, 2014 5:05 PM ???: Tristan Yan ??: Martin Buchholz; core-libs-dev ??: Re: RFR for JDK-8031374: java/util/concurrent/ConcurrentQueues/OfferRemoveLoops.java fails Intermittently Tristan, Can you please file a new bug and bring in the changes that Martin pushed to the 166 CVS. http://gee.cs.oswego.edu/cgi-bin/viewcvs.cgi/jsr166/src/test/jtreg/util/concurrent/ConcurrentQueues/OfferRemoveLoops.java?view=co Create a changeset, and I can do the OpenJDK review. -Chris. On 24 Feb 2014, at 03:03, Tristan Yan wrote: > I am ok with this unfix for now. Martin. And thank you for bringing the improvement to jsr166 CVS. > > Regards > > Tristan > > > > ???: Martin Buchholz [mailto:martinrb at google.com] > ????: Monday, February 24, 2014 10:59 AM > ???: Tristan Yan > ??: core-libs-dev > ??: Re: ??: ??: RFR for JDK-8031374: java/util/concurrent/ConcurrentQueues/OfferRemoveLoops.java fails Intermittently > > > > Hi Tristan, > > > > I don't think we should change the test without a sufficiently solid reason, which usually means reproducing the failure, even if e.g. only once in 1000 times. Here we don't know which queue implementation is causing the failure, and we don't know what chunk size was being used, which might cause the failure to be reproduced more reliably. A single test failure might be due to cosmic rays! Let's leave unfixed this while it cannot be reproduced. You can check in my general purpose improvements to the test from jsr166 CVS. > > > > On Sun, Feb 23, 2014 at 6:41 PM, Tristan Yan wrote: > > Hi Martin > > This test failed once in our nightly test. So this is a real case failure. But unfortunately we couldn?t know more except the log we had. > > 10 extra seconds may need serve 1024 loop totally, also in windows Thread.yield() doesn?t give up CPU most of times, then we can may have the situation that remover is trying to remove object from a empty queue but it doesn?t find anything a couple of times(it doesn?t give up cpu time) then offer thread get cpu time. And offer thread insert a couple of elements and queue is full. Offer thread tries to give up CPU but Thread.yield() doesn?t really give up. Let?s assume it takes 1024 loop again. And offer thread got timeout. Then remover thread try to remove elements, and it takes another 1024 loop to get to timeout as well. So 10 seconds may need distribute to 2048 loop at most. Every loop has 5 ms foreach. That?s why I simulate this case with a Thread.sleep(20). I admit I can?t reproduce it without changing code, this is all theoretical analysis. But this is closest possible reason that I can deduce. > > Thank you. > > Tristan > > > > ???: Martin Buchholz [mailto:HYPERLINK "mailto:martinrb at google.com"martinrb at google.com] > ????: Monday, February 24, 2014 10:16 AM > > > ???: Tristan Yan > ??: core-libs-dev > > ??: Re: ??: RFR for JDK-8031374: java/util/concurrent/ConcurrentQueues/OfferRemoveLoops.java fails Intermittently > > > > I may very well be missing something, but the actual extra timeout for threads to finish is 10 whole seconds, which should be long enough to process a single chunk, even on Windows. > > > > If you can repro this consistently, try to find out which queue implementation is affected. > > > > We can also shrink max queue size and chunk size to reduce time to traverse the queue elements. > > > > On Sun, Feb 23, 2014 at 4:23 PM, Tristan Yan wrote: > > Thank you for reviewing this. Martin > > > > The original bug report is here: https://bugs.openjdk.java.net/browse/JDK-8031374. > > > > I haven?t reproduced this bug but I can simulate to reproduce this failure by changing yield() in remover thread to Thread.sleep(20). > > You have commented ?You've completely broken the intent of this code, which was to poll ***occasionally*** for quitting time?. I tried to not changed the logic for the test. This failure comes when only 1st rounds(1024 loop) wasn?t finished in given quitting time(before timeout). Which was 300ms. One solution is increasing default timeout as you suggested. But question is how big should we increase. When the test is really slow and could not finish 1st round(1024 loop) before timeout, I couldn?t figure out what?s the reason timeout value. Also this definitely will slow down the tests when it run in fast machine. Which is the case most of time. So I took other step that skip wait if test doesn't finish 1st round(1024 loop) before timeout. I won?t say I completely broke the intent of the code here because it?s rare case.(Only if the machine is slow and slow enough that the test doesn't finish 1st round before timeout). > > > > The reason that replace Thread.yield to Thread.sleep(0L) because David Holmes point out in the bug ?sleep will certainly give up the CPU, whereas yield is not required to.? Also in other mail, David pointed I should use Thread.sleep(10L) instead of Thread.sleep(0L) preferably a bit longer as we don't know how the OS will behave if the sleep time requested is less than the natural resolution of the sleep mechanism. For the experiment I?ve done in Windows. Thread.yeild() or Thread.sleep(10L) won?t guarantee current thread give up the CPU. This is a hint to OS. This makes in windows remover and offer thread could wait to each other more more than other other operating system. This is also the one of the reason that can explain why we've seen this in windows only. > > > > Regards > > Tristan > > > > > > > > ???: Martin Buchholz [mailto:HYPERLINK "mailto:martinrb at google.com"martinrb at google.com] > ????: Monday, February 24, 2014 3:47 AM > ???: Tristan Yan > ??: core-libs-dev > ??: Re: RFR for JDK-8031374: java/util/concurrent/ConcurrentQueues/OfferRemoveLoops.java fails Intermittently > > > > Hi Tristan, > > > > (thanks for working on my old crappy tests, and apologies for always giving you a hard time) > > > > I couldn't find the bug report. Can you provide a URL? > > > > Thread.stop is only called in case the test has already failed (presumed "hung"), as a last resort. > > > > Perhaps the timeout used in the test (300 ms) is too small on some systems? > > > > + protected void giveupCPU(){ > + try { > + Thread.sleep(0L); > + } catch (InterruptedException ignore) {} > } > > > > I know of no reason why Thread.sleep(0) should be any more effective than Thread.yield. If it was more effective, then why wouldn't Thread.yield on that platform be fixed to be implemented exactly the same way? IOW if this is a problem, fix the JDK! > > > > /** Polls occasionally for quitting time. */ > protected boolean quittingTime(long i) { > - return (i % 1024) == 0 && quittingTime(); > + return stopRequest || quittingTime() && (i % 1024 == 0 || i < 1024); > + } > > > > You've completely broken the intent of this code, which was to poll ***occasionally*** for quitting time. > > > > > > On Sun, Feb 23, 2014 at 1:40 AM, Tristan Yan wrote: > > Hi All > Could you please help to review fix for JDK-803174. > > http://cr.openjdk.java.net/~tyan/JDK-8031374/webrev.00/ > > Description: > There are a couple of code change for this code. > 1. Original code has used known problematic Thread.stop. Which could cause ThreadDeath as all we know. I change it to a customized stopThread method. > 2. Test failed in windows, I analyze the failure by changing Thread.yield() in remover thread to Thread.sleep(50). This is a simulation for slow machine. Which shows exact same failures as we can see in bug description. By adding more debug info, we can see although we tried to give up CPU by using Thread.yield().(I have tried to use Thread.sleep(1L) as well, the result is same), there is no guarantee that os will pick up a new thread to execute. In Windows, this is more obvious. Because the execution is slow. even when the timeout happens, offer thread and remove thread won?t have chance to get the point that i % 1024 == 0. This will cause the failure like we see in the log. My fix here is when the timeout happens, but i is still less than 1024. Stop offer thread and remover thread right away instead letting them continuously wait the point to i == 1024. > 3. I replace Thread.yield to Thread.sleep(0L). I saw a couple of discussion that Thread.yield is not required to give up CPU. > > Thank you > Tristan > > > > > > From roger.riggs at oracle.com Mon Feb 24 14:37:29 2014 From: roger.riggs at oracle.com (roger riggs) Date: Mon, 24 Feb 2014 09:37:29 -0500 Subject: RFR [6853696] (ref) ReferenceQueue.remove(timeout) may return null even if timeout has not expired In-Reply-To: <530ADFB2.8000100@oracle.com> References: <530ADFB2.8000100@oracle.com> Message-ID: <530B5929.7040907@oracle.com> Hi Ivan, The code is correct as written but there might be some creep in the end time due to the sampling of System.nanoTime. I would be inclined to calculate the final time of the timeout once and then compare simply with the current nanotime. long end = (timeout == 0) ? Long.MAX_VALUE : (System.nanoTime() + timeout * 1000000); Then the test in the loop can be: if (System.nanoTime() > end) { return null; } Roger (Not a Reviewer) On 2/24/2014 12:59 AM, Ivan Gerasimov wrote: > Hello! > > ReferenceQueue.remove(timeout) may return too early, i.e. before the > specified timeout has elapsed. > > Would you please review the fix? > The change also includes a regression test, which can be used to > demonstrate the issue. > > BUGURL: https://bugs.openjdk.java.net/browse/JDK-6853696 > WEBREV: http://cr.openjdk.java.net/~igerasim/6853696/0/webrev/ > > Sincerely yours, > Ivan From martinrb at google.com Mon Feb 24 15:25:43 2014 From: martinrb at google.com (Martin Buchholz) Date: Mon, 24 Feb 2014 07:25:43 -0800 Subject: RFR [6853696] (ref) ReferenceQueue.remove(timeout) may return null even if timeout has not expired In-Reply-To: <530B5929.7040907@oracle.com> References: <530ADFB2.8000100@oracle.com> <530B5929.7040907@oracle.com> Message-ID: On Mon, Feb 24, 2014 at 6:37 AM, roger riggs wrote: > > if (System.nanoTime() > end) { > As I wrote many years ago, To compare two nanoTime values long t0 = System.nanoTime(); ... long t1 = System.nanoTime(); one should use t1 - t0 < 0, not t1 < t0, because of the possibility of numerical overflow. http://docs.oracle.com/javase/7/docs/api/java/lang/System.html#nanoTime() From ivan.gerasimov at oracle.com Mon Feb 24 15:34:01 2014 From: ivan.gerasimov at oracle.com (Ivan Gerasimov) Date: Mon, 24 Feb 2014 19:34:01 +0400 Subject: RFR [6853696] (ref) ReferenceQueue.remove(timeout) may return null even if timeout has not expired In-Reply-To: <530B5929.7040907@oracle.com> References: <530ADFB2.8000100@oracle.com> <530B5929.7040907@oracle.com> Message-ID: <530B6669.1030409@oracle.com> Thank you Roger for looking at the fix! On 24.02.2014 18:37, roger riggs wrote: > Hi Ivan, > > The code is correct as written but there might be some creep in the > end time due to the sampling of System.nanoTime. > > I would be inclined to calculate the final time of the timeout once > and then compare simply with the current nanotime. > > long end = (timeout == 0) ? Long.MAX_VALUE : (System.nanoTime() + > timeout * 1000000); > I don't think it is always correct. According to the documentation, "The value returned represents nanoseconds since some fixed but *arbitrary* origin time". http://download.java.net/jdk8/docs/api/java/lang/System.html#nanoTime-- Thus, Long.MAX_VALUE may just happen to represent some valid time in the near future, and we'll achieve wrong result. Sincerely yours, Ivan > Then the test in the loop can be: > > if (System.nanoTime() > end) { > return null; > } > > Roger (Not a Reviewer) > > On 2/24/2014 12:59 AM, Ivan Gerasimov wrote: >> Hello! >> >> ReferenceQueue.remove(timeout) may return too early, i.e. before the >> specified timeout has elapsed. >> >> Would you please review the fix? >> The change also includes a regression test, which can be used to >> demonstrate the issue. >> >> BUGURL: https://bugs.openjdk.java.net/browse/JDK-6853696 >> WEBREV: http://cr.openjdk.java.net/~igerasim/6853696/0/webrev/ >> >> Sincerely yours, >> Ivan > > From anthony.petrov at oracle.com Mon Feb 24 12:21:00 2014 From: anthony.petrov at oracle.com (Anthony Petrov) Date: Mon, 24 Feb 2014 16:21:00 +0400 Subject: [9] Review request: JDK-8035640 JNU_CHECK_EXCEPTION should support c++ JNI syntax In-Reply-To: References: <530B369B.3050707@oracle.com> Message-ID: <530B392C.8060106@oracle.com> Hi Petr, The fix looks fine to me. -- best regards, Anthony On 2/24/2014 4:21 PM, Petr Pchelko wrote: > Hello, Alan. > > Thank you for the review. > >> This looks okay to me. One suggestion is to use #endif /* __cplusplus */ so that it's consistent with the other usages (also makes it a bit easier when there are nested ifdefs). > Updated the fix: http://cr.openjdk.java.net/~pchelko/9/8035640/webrev.01/ > >> As regards logistics then jdk9/dev might be the more suitable forest to push this to. I suggest this because it looks to me that jdk9/client is pulling down changes from jdk9/dev very regularly (which is good). On the other hand there doesn't appear to be regular integrations from jdk9/client to jdk9/dev yet. I see changes in jdk9/client from mid-December that has still not been pushed to jdk9/dev. It's just a suggestion to ensure that the changes get to both forests in timely manner. > No problem. I think we could easily wait until the next integration while dependent fixes are being reviewed. I'll push this into dev forest. > > With best regards. Petr. > > On 24.02.2014, at 16:10, Alan Bateman wrote: > >> On 24/02/2014 09:02, Petr Pchelko wrote: >>> Hello, >>> >>> Please review the fix for the issue: >>> https://bugs.openjdk.java.net/browse/JDK-8035640 >>> The fix is available at: >>> http://cr.openjdk.java.net/~pchelko/9/8035640/webrev.00/ >>> >>> In AWT code we have quite a lot of C++ sources, but JNU_CHECK_EXCEPTION macros could not be used there, because the JNI syntax is different in C++. >>> If approved I'll integrate this fix into the client forest, because we need this in client to fix parfait issues. >>> >>> Thank you, >>> With best regards. Petr. >> This looks okay to me. One suggestion is to use #endif /* __cplusplus */ so that it's consistent with the other usages (also makes it a bit easier when there are nested ifdefs). >> >> As regards logistics then jdk9/dev might be the more suitable forest to push this to. I suggest this because it looks to me that jdk9/client is pulling down changes from jdk9/dev very regularly (which is good). On the other hand there doesn't appear to be regular integrations from jdk9/client to jdk9/dev yet. I see changes in jdk9/client from mid-December that has still not been pushed to jdk9/dev. It's just a suggestion to ensure that the changes get to both forests in timely manner. >> >> -Alan. > From martinrb at google.com Mon Feb 24 16:54:29 2014 From: martinrb at google.com (Martin Buchholz) Date: Mon, 24 Feb 2014 08:54:29 -0800 Subject: RFR for JDK-8031374: java/util/concurrent/ConcurrentQueues/OfferRemoveLoops.java fails Intermittently In-Reply-To: <530A7D49.6000404@oracle.com> References: <6B2B9D6A-F0DA-4F00-BE95-68AC7A03B6D0@oracle.com> <530A7D49.6000404@oracle.com> Message-ID: On Sun, Feb 23, 2014 at 2:59 PM, David Holmes wrote: > > > For Martin's benefit, while Thread.yield was always assumed to do the > right thing and act as reasonably expected, that got broken by the Linux > folks with CFS at one point. So loops involving yield are out of favour. > Following up a bit ... when I call Thread.yield, I am generally fine with continuing to run if there is no other process available to claim the CPU, so the new Linux CFS behavior is what I was always coding for. hotspot engineers could implement Thread.yield by calling nanosleep(0), as suggested by the code snippet below, but I think that would be bad - I'm happy with the Linux CFS behavior. // Short sleep, direct OS call. // // Note: certain versions of Linux CFS scheduler (since 2.6.23) do not guarantee // sched_yield(2) will actually give up the CPU: // // * Alone on this pariticular CPU, keeps running. // * Before the introduction of "skip_buddy" with "compat_yield" disabled // (pre 2.6.39). // // So calling this with 0 is an alternative. From chris.hegarty at oracle.com Mon Feb 24 17:23:12 2014 From: chris.hegarty at oracle.com (Chris Hegarty) Date: Mon, 24 Feb 2014 17:23:12 +0000 Subject: RFR: JDK-8030780 - test/com/sun/corba/cachedSocket/7056731.sh leaves HelloServer behind In-Reply-To: <530752DD.3040703@oracle.com> References: <530752DD.3040703@oracle.com> Message-ID: <530B8000.5010904@oracle.com> Looks ok to me Mark. -Chris. On 21/02/14 13:21, Mark Sheppard wrote: > Hi > > please oblige and review the follow change > > http://cr.openjdk.java.net/~msheppar/8030780/webrev/ > > to address the issue raised in > > https://bugs.openjdk.java.net/browse/JDK-8030780 > > summary: > if the the java debugger failed to connect to the client, then the > sequence of commands which include "kill -9" of the server process will > not be executed. > > the server process is added to the post test kill > In a successful execution this will result in a "no such process" message. > > regards > Mark From ivan.gerasimov at oracle.com Mon Feb 24 17:26:39 2014 From: ivan.gerasimov at oracle.com (Ivan Gerasimov) Date: Mon, 24 Feb 2014 21:26:39 +0400 Subject: RFR [6943190] TEST_BUG: java/lang/Runtime/exec/ExecWithInput.java hardcodes path to cat Message-ID: <530B80CF.9090403@oracle.com> Hello! We've got one quite old report about been unable to run the test under Android. https://bugs.openjdk.java.net/browse/JDK-6943190 That was due to hard-coded path to the cat utility as /bin/cat. When investigating, I found two other tests under the same directory that use /usr/bin/cat and /usr/bin/echo. These two test seem to (almost) never run because of the unusual path. Here's the first version of the webrev, with the fix to only three tests mentioned above: http://cr.openjdk.java.net/~igerasim/6943190/0/webrev/ java/lang/Runtime/exec/ directory has also got several other tests, which run shell commands. Some of them have absolute path and some of them don't. So I have a general question: Why would we ever need the absolute path for the commands? Why wouldn't we rely on the PATH env variable? Sincerely yours, Ivan From john.r.rose at oracle.com Mon Feb 24 17:46:42 2014 From: john.r.rose at oracle.com (John Rose) Date: Mon, 24 Feb 2014 09:46:42 -0800 Subject: [9] RFR (M): 8027827: Improve performance of catchException combinator In-Reply-To: <5306420B.9040200@oracle.com> References: <53052622.6010507@oracle.com> <0386E0D6-D515-4C2C-9D6E-DD3B53E8B3E2@oracle.com> <5306420B.9040200@oracle.com> Message-ID: <0516A100-6FAF-4718-8A9F-A82D7C7E7061@oracle.com> On Feb 20, 2014, at 9:57 AM, Vladimir Ivanov wrote: > Updated webrev: > http://cr.openjdk.java.net/~vlivanov/8027827/final/webrev.01/ > > I finally figured out how to make caching work. This webrev contains these changes. > > I changed LF representation a bit and added 2 auxiliary method handles - argument boxing and wrapping into Object[] and result unboxing. These operations depend on actual type and can't be shared among arbitrary combinators with the same basic type. They are used only during LF interpretation and are completely ignored in compiled LFs. This is a good step forward, thanks! Some comments: I prefer the bounds check expression pos+3 < lambdaForm.names.length. (One integer constant, limit to right.) The predicate isGuardWithCatch must test all three subforms. Or else there must be assertions to ensure that names[pos+2] is of the expected form. The problem is that LF's can sometimes be edited (e.g., by binding operations) and there is no insurance that your pattern of three expressions will be preserved in all cases. I see you are trying to do unboxing elimination here; this is not a safe or effective way to do it, IMO. Put in a "FIXME" comment and file a bug to deal better with unboxing ops in LFs. I have some WIP code toward this end which we can talk about. You've probably seen of that business, about internally LF marking expressions as intrinsics to guide bytecode generation. Why is the logic about cachedLambdaForm commented out? It looks correct, but is there a bug? Consider replacing GUARD_WITH_CATCH with Lazy.NF_guardWithCatch, and using the NF instead of MH for the intrinsic. ? John From mark.reinhold at oracle.com Mon Feb 24 18:20:50 2014 From: mark.reinhold at oracle.com (mark.reinhold at oracle.com) Date: Mon, 24 Feb 2014 10:20:50 -0800 (PST) Subject: JEP 191: Foreign Function Interface Message-ID: <20140224182050.96F77138AF@eggemoggin.niobe.net> Posted: http://openjdk.java.net/jeps/191 - Mark From mike.duigou at oracle.com Mon Feb 24 18:26:22 2014 From: mike.duigou at oracle.com (Mike Duigou) Date: Mon, 24 Feb 2014 10:26:22 -0800 Subject: RFR [6853696] (ref) ReferenceQueue.remove(timeout) may return null even if timeout has not expired In-Reply-To: <530B5929.7040907@oracle.com> References: <530ADFB2.8000100@oracle.com> <530B5929.7040907@oracle.com> Message-ID: <41CC73D0-1754-4A62-A90B-03AD2F372CBC@oracle.com> On Feb 24 2014, at 06:37 , roger riggs wrote: > Hi Ivan, > > The code is correct as written but there might be some creep in the end time due to the sampling of System.nanoTime. > > I would be inclined to calculate the final time of the timeout once > and then compare simply with the current nanotime. > > long end = (timeout == 0) ? Long.MAX_VALUE : (System.nanoTime() + timeout * 1000000); I hate seeing numerical constants TimeUnit.MILLISECONDS.toNanos(timeout) > > Then the test in the loop can be: > > if (System.nanoTime() > end) { > return null; > } This compare should be re-written in the overflow compensating style Martin mentions. Mike > > Roger (Not a Reviewer) > > On 2/24/2014 12:59 AM, Ivan Gerasimov wrote: >> Hello! >> >> ReferenceQueue.remove(timeout) may return too early, i.e. before the specified timeout has elapsed. >> >> Would you please review the fix? >> The change also includes a regression test, which can be used to demonstrate the issue. >> >> BUGURL: https://bugs.openjdk.java.net/browse/JDK-6853696 >> WEBREV: http://cr.openjdk.java.net/~igerasim/6853696/0/webrev/ >> >> Sincerely yours, >> Ivan > From martinrb at google.com Mon Feb 24 19:38:22 2014 From: martinrb at google.com (Martin Buchholz) Date: Mon, 24 Feb 2014 11:38:22 -0800 Subject: RFR [6943190] TEST_BUG: java/lang/Runtime/exec/ExecWithInput.java hardcodes path to cat In-Reply-To: <530B80CF.9090403@oracle.com> References: <530B80CF.9090403@oracle.com> Message-ID: Thanks for working on these ancient Process tests. I would prefer to see them using "feature tests". You wanna do "cat /dev/zero"? Then look for cat in /bin and /usr/bin and check for /dev/zero as well, e.g. using File.isExecutable Doing OS-specific things like: + static final boolean isLinux = + System.getProperty("os.name", "unknown").startsWith("Linux"); should be a last resort. On Mon, Feb 24, 2014 at 9:26 AM, Ivan Gerasimov wrote: > Hello! > > We've got one quite old report about been unable to run the test under > Android. > https://bugs.openjdk.java.net/browse/JDK-6943190 > > That was due to hard-coded path to the cat utility as /bin/cat. > When investigating, I found two other tests under the same directory that > use /usr/bin/cat and /usr/bin/echo. > These two test seem to (almost) never run because of the unusual path. > > Here's the first version of the webrev, with the fix to only three tests > mentioned above: > http://cr.openjdk.java.net/~igerasim/6943190/0/webrev/ > > java/lang/Runtime/exec/ directory has also got several other tests, which > run shell commands. > Some of them have absolute path and some of them don't. > > So I have a general question: Why would we ever need the absolute path for > the commands? > Why wouldn't we rely on the PATH env variable? > > Sincerely yours, > Ivan > > > From martinrb at google.com Mon Feb 24 19:39:41 2014 From: martinrb at google.com (Martin Buchholz) Date: Mon, 24 Feb 2014 11:39:41 -0800 Subject: RFR [6943190] TEST_BUG: java/lang/Runtime/exec/ExecWithInput.java hardcodes path to cat In-Reply-To: <530B80CF.9090403@oracle.com> References: <530B80CF.9090403@oracle.com> Message-ID: On Mon, Feb 24, 2014 at 9:26 AM, Ivan Gerasimov wrote: > Why wouldn't we rely on the PATH env variable? > It makes the test dependent on the user's PATH, introducing a new source of flakiness. From dmitry.degrave at oracle.com Mon Feb 24 21:00:38 2014 From: dmitry.degrave at oracle.com (dmeetry degrave) Date: Tue, 25 Feb 2014 01:00:38 +0400 Subject: [7u backport] RFR: 7122142: (ann) Race condition between isAnnotationPresent and getAnnotations In-Reply-To: <530B3E32.7020303@gmail.com> References: <53089694.4070308@oracle.com> <530B3E32.7020303@gmail.com> Message-ID: <530BB2F6.2000507@oracle.com> Thanks for looking at this, Peter! On 02/24/2014 04:42 PM, Peter Levart wrote: > Hi Dmeetry, > > On 02/22/2014 01:22 PM, dmeetry degrave wrote: >> Hi all, >> >> I would like to ask for a review of combined back port for >> 7u-dev/7u80. The main goal is to have a fix for 7122142 in jdk7, it >> also integrates the changes from 8005232, 7185456, 8022721 >> >> https://bugs.openjdk.java.net/browse/JDK-7122142 >> https://bugs.openjdk.java.net/browse/JDK-8005232 >> https://bugs.openjdk.java.net/browse/JDK-7185456 >> https://bugs.openjdk.java.net/browse/JDK-8022721 >> >> Original jdk8 changes: >> >> 7122142: http://hg.openjdk.java.net/jdk8/tl/jdk/rev/e4ce6502eac0 >> 8005232: http://hg.openjdk.java.net/jdk8/tl/jdk/rev/1109bfff4e92 >> 7185456: http://hg.openjdk.java.net/jdk8/tl/jdk/rev/ae03282ba501 >> 8022721: http://hg.openjdk.java.net/jdk8/tl/jdk/rev/2281a7f79738 >> >> back port: >> >> http://cr.openjdk.java.net/~dmeetry/7122142.8005232.7185456.8022721/webrev.0/ >> >> >> Patches can't be applied cleanly, hence it was a manual back port, >> though the final result is equivalent to applying the patches in >> chronological order (8005232, 7185456, 7122142, 8022721) and applying >> all the relevant rejected parts > > It's good to see those patches being back-ported to 7u. By browsing the > webrev, I don't see any obvious difference between the original patches > and the backport. there shouldn't be any! > Do you happen to remember in what part of code there > were rejects so that you had to manually apply the changes? there were conflicts due to small difference between 7 and 8 (copyrights, white spaces, @SuppressWarnings, Class,...). I copied all rejected parts and original patches here: http://cr.openjdk.java.net/~dmeetry/7122142.8005232.7185456.8022721/webrev.1/rej/ >> (with one exception, AnnotationTypeRuntimeAssumptionTest.java test was >> not included due to jdk8 API). > > Ah, It's the Class.getDeclaredAnnotation(Class) that's new in JDK8. > Here's the changed test that only uses the JDK7 API so you can include > this test too: > > http://cr.openjdk.java.net/~plevart/jdk7u/7122142/AnnotationTypeRuntimeAssumptionTest.java Thanks! http://cr.openjdk.java.net/~dmeetry/7122142.8005232.7185456.8022721/webrev.1/ (just with the new test added). thanks, dmeetry >> >> All tests in test/java/lang/annotation passed. >> >> thanks, >> dmeetry > > Regards, Peter > From mandy.chung at oracle.com Mon Feb 24 22:11:11 2014 From: mandy.chung at oracle.com (Mandy Chung) Date: Mon, 24 Feb 2014 14:11:11 -0800 Subject: RFR [6853696] (ref) ReferenceQueue.remove(timeout) may return null even if timeout has not expired In-Reply-To: <530ADFB2.8000100@oracle.com> References: <530ADFB2.8000100@oracle.com> Message-ID: <530BC37F.2050908@oracle.com> On 2/23/14 9:59 PM, Ivan Gerasimov wrote: > Hello! > > ReferenceQueue.remove(timeout) may return too early, i.e. before the > specified timeout has elapsed. > > Would you please review the fix? > The change also includes a regression test, which can be used to > demonstrate the issue. > > BUGURL: https://bugs.openjdk.java.net/browse/JDK-6853696 > WEBREV: http://cr.openjdk.java.net/~igerasim/6853696/0/webrev/ > I'll review the patch once you address the numerical overflow issue Martin points out. One minor comment - I suggest to use "start" and "end" instead of "before" and "after". One comment on the test, line 61: I think you want to check thread.suspect expects to be true. It may be simply keeping the returned value of ReferenceQueue.remove call and check that in line 61 instead of having thread.suspect variable. line 76: why do you want to catch InterruptedException? If interrupted, should the test fail? Mandy From brent.christian at oracle.com Mon Feb 24 23:52:23 2014 From: brent.christian at oracle.com (Brent Christian) Date: Mon, 24 Feb 2014 15:52:23 -0800 Subject: RFR 8027640 : String.indexOf(String,int) for the empty string case not specified Message-ID: <530BDB37.40603@oracle.com> Please review my changes for: https://bugs.openjdk.java.net/browse/JDK-8027640 The webrev is here: http://cr.openjdk.java.net/~bchristi/8027640/webrev.00/ This is a small cleanup to reconcile the String[last]indexOf() spec with longstanding behavior. There are no changes to executable code. In String, the "Math.min(fromIndex, this.length())" clause is brought over from String[Buffer|Builder] for indexOf(String,int) and lastIndexOf(String,int). This resolves the issue for which the bug was filed. To further sync up the [last]indexOf() docs, I ported the 6940381 wording improvements from String to String[Buffer|Builder]. I also made a few doclint updates to the code in question. Thanks, -Brent From brian.burkhalter at oracle.com Tue Feb 25 00:32:35 2014 From: brian.burkhalter at oracle.com (Brian Burkhalter) Date: Mon, 24 Feb 2014 16:32:35 -0800 Subject: JDK 9 RFR of JDK-8035279: Clean up internal deprecations in BigInteger In-Reply-To: <5306BC4B.4090909@oracle.com> References: <5305D6EB.30806@oracle.com> <53064179.2060703@gmail.com> <530660FA.1060407@oracle.com> <96CF3236-4E65-4D64-BCF3-1E9660D29BAA@oracle.com> <5306BC4B.4090909@oracle.com> Message-ID: On Feb 20, 2014, at 6:39 PM, David Holmes wrote: > Not clear what you mean by this. This is more or less my reaction to this entire thread, so to speak. ;-) Anyway, thanks for all the comments. Note that I am ignoring the powerCache field comments for the moment. To distill the discussion down to just the proposed changes I posted, my reading is that there is more or less consensus on two points: 1) the instance fields in question *should* be volatile for this proposed change set 2) non-zero initial values should be avoided in case of instance leaking to non-constructing threads Is this accurate? On second thought it occurred to me that instead of any complicated or contentious changes, as it were, the ugliness I was trying to remove from the code could just as well be addressed by simply changing the names of the affected instance variables to indicate what their respective values really represent, e.g., "bitCount" becomes "bitCountPlusOne" and we remove the @Deprecated and @deprecated. Yeah this is still ugly ... Thanks, Brian From joe.darcy at oracle.com Tue Feb 25 02:22:11 2014 From: joe.darcy at oracle.com (Joe Darcy) Date: Mon, 24 Feb 2014 18:22:11 -0800 Subject: JDK 9 RFR of JDK-8035453: Fix serial lint warnings in com.sun.tools and elsewhere In-Reply-To: References: <53067053.2060501@oracle.com> Message-ID: <530BFE53.3040603@oracle.com> Hi Paul, On 02/24/2014 03:23 AM, Paul Sandoz wrote: > Hi Joe, > > On Feb 20, 2014, at 10:14 PM, Joe Darcy wrote: > >> bHello, >> >> Please review my proposed changes for: >> >> JDK-8035453: Fix serial lint warnings in com.sun.tools and elsewhere >> http://cr.openjdk.java.net/~darcy/8035453.0/webrev/ >> > Looks good. > > I agree with Paul B. that it seems a little odd to have to suppress serialization on an abstract class, especially one that implements Serializable (plus Connector.Argument also extends Serializable): > Despite cleaning up many serial warnings, I've tried hard to avoid learning too much about the serialization wire protocol ;-) From what I was able to discern by reading the serialization specification [1], If a class does *not* declare a serialVersionUID, the serialVersionUID of its superclass is *not* included in the serialver hash computation of the child class. However, my understanding is that changes to the fields stored in superclass and changes to the semantics of its readObject / writeObjects methods could affect the serialization of the child class. Therefore, if that understanding is correct, it seems reasonable for javac to warn about serializable abstract classes even though one cannot have bare instances of the class. Thanks for the review, -Joe [1] http://docs.oracle.com/javase/7/docs/platform/serialization/spec/class.html#4100 From david.holmes at oracle.com Tue Feb 25 02:52:02 2014 From: david.holmes at oracle.com (David Holmes) Date: Tue, 25 Feb 2014 12:52:02 +1000 Subject: RFR [6853696] (ref) ReferenceQueue.remove(timeout) may return null even if timeout has not expired In-Reply-To: <530B6669.1030409@oracle.com> References: <530ADFB2.8000100@oracle.com> <530B5929.7040907@oracle.com> <530B6669.1030409@oracle.com> Message-ID: <530C0552.5040503@oracle.com> On 25/02/2014 1:34 AM, Ivan Gerasimov wrote: > Thank you Roger for looking at the fix! > > On 24.02.2014 18:37, roger riggs wrote: >> Hi Ivan, >> >> The code is correct as written but there might be some creep in the >> end time due to the sampling of System.nanoTime. >> >> I would be inclined to calculate the final time of the timeout once >> and then compare simply with the current nanotime. >> >> long end = (timeout == 0) ? Long.MAX_VALUE : (System.nanoTime() + >> timeout * 1000000); >> > I don't think it is always correct. > According to the documentation, "The value returned represents > nanoseconds since some fixed but *arbitrary* origin time". > http://download.java.net/jdk8/docs/api/java/lang/System.html#nanoTime-- > > Thus, Long.MAX_VALUE may just happen to represent some valid time in the > near future, and we'll achieve wrong result. Correct. Simplest thing is to calculate end ignoring whether timeout is zero, and only check "now < end" if timeout != 0. David > Sincerely yours, > Ivan > >> Then the test in the loop can be: >> >> if (System.nanoTime() > end) { >> return null; >> } >> >> Roger (Not a Reviewer) >> >> On 2/24/2014 12:59 AM, Ivan Gerasimov wrote: >>> Hello! >>> >>> ReferenceQueue.remove(timeout) may return too early, i.e. before the >>> specified timeout has elapsed. >>> >>> Would you please review the fix? >>> The change also includes a regression test, which can be used to >>> demonstrate the issue. >>> >>> BUGURL: https://bugs.openjdk.java.net/browse/JDK-6853696 >>> WEBREV: http://cr.openjdk.java.net/~igerasim/6853696/0/webrev/ >>> >>> Sincerely yours, >>> Ivan >> >> > From ivan.gerasimov at oracle.com Tue Feb 25 03:49:40 2014 From: ivan.gerasimov at oracle.com (Ivan Gerasimov) Date: Tue, 25 Feb 2014 07:49:40 +0400 Subject: RFR [6853696] (ref) ReferenceQueue.remove(timeout) may return null even if timeout has not expired In-Reply-To: <530C0552.5040503@oracle.com> References: <530ADFB2.8000100@oracle.com> <530B5929.7040907@oracle.com> <530B6669.1030409@oracle.com> <530C0552.5040503@oracle.com> Message-ID: <530C12D4.8030009@oracle.com> On 25.02.2014 6:52, David Holmes wrote: > On 25/02/2014 1:34 AM, Ivan Gerasimov wrote: >> Thank you Roger for looking at the fix! >> >> On 24.02.2014 18:37, roger riggs wrote: >>> Hi Ivan, >>> >>> The code is correct as written but there might be some creep in the >>> end time due to the sampling of System.nanoTime. >>> >>> I would be inclined to calculate the final time of the timeout once >>> and then compare simply with the current nanotime. >>> >>> long end = (timeout == 0) ? Long.MAX_VALUE : (System.nanoTime() + >>> timeout * 1000000); >>> >> I don't think it is always correct. >> According to the documentation, "The value returned represents >> nanoseconds since some fixed but *arbitrary* origin time". >> http://download.java.net/jdk8/docs/api/java/lang/System.html#nanoTime-- >> >> Thus, Long.MAX_VALUE may just happen to represent some valid time in the >> near future, and we'll achieve wrong result. > > Correct. Simplest thing is to calculate end ignoring whether timeout > is zero, and only check "now < end" if timeout != 0. David, as Martin said before, 'now < end' may hold true even if now is after end (now is negative). In my version of the code, I operate on the difference (after - before), thus avoiding the possible overflow. Sincerely yours, Ivan > > David > >> Sincerely yours, >> Ivan >> >>> Then the test in the loop can be: >>> >>> if (System.nanoTime() > end) { >>> return null; >>> } >>> >>> Roger (Not a Reviewer) >>> >>> On 2/24/2014 12:59 AM, Ivan Gerasimov wrote: >>>> Hello! >>>> >>>> ReferenceQueue.remove(timeout) may return too early, i.e. before the >>>> specified timeout has elapsed. >>>> >>>> Would you please review the fix? >>>> The change also includes a regression test, which can be used to >>>> demonstrate the issue. >>>> >>>> BUGURL: https://bugs.openjdk.java.net/browse/JDK-6853696 >>>> WEBREV: http://cr.openjdk.java.net/~igerasim/6853696/0/webrev/ >>>> >>>> Sincerely yours, >>>> Ivan >>> >>> >> > > From joe.darcy at oracle.com Tue Feb 25 04:22:31 2014 From: joe.darcy at oracle.com (Joe Darcy) Date: Mon, 24 Feb 2014 20:22:31 -0800 Subject: JDK 9 RFR of JDK-8035452: Fix serial lint warnings in core libs In-Reply-To: References: <530666A5.8050607@oracle.com> Message-ID: <530C1A87.2070608@oracle.com> Hi Paul, On 02/20/2014 12:49 PM, Paul Benedict wrote: > Joe, I find it interesting that you suppressed the serial warning on > an abstract class. I'd like to know more about that. Is this a > universal rule? Are serial uids not important for abstract classes? I wouldn't generalize that way from this example. The serial hash of NavigableSubMap has differed in different JDK releases, but its subclasses do define serialVersionUID fields. I assume the set of non-transient fields in NavigableSubMap has stayed unchanged, but I haven't verified that. If I hadn't found the change in the hash value, I would have added the serialVersionUID to NavigableSubMap too. HTH, -Joe > > > On Thu, Feb 20, 2014 at 2:33 PM, Joe Darcy > wrote: > > Hello, > > Please review the patch below which will address the handful of > remaining serial lint warning in the core libraries. > > Thanks, > > -Joe > > diff -r e5a09bc70308 src/share/classes/java/util/EnumSet.java > --- a/src/share/classes/java/util/EnumSet.java Thu Feb 20 > 13:03:36 2014 +0000 > +++ b/src/share/classes/java/util/EnumSet.java Thu Feb 20 > 12:32:52 2014 -0800 > @@ -80,6 +80,8 @@ > public abstract class EnumSet> extends > AbstractSet > implements Cloneable, java.io.Serializable > { > + private static final long serialVersionUID = > 1009687484059888093L; > + > /** > * The class of all the elements of this set. > */ > diff -r e5a09bc70308 src/share/classes/java/util/TreeMap.java > --- a/src/share/classes/java/util/TreeMap.java Thu Feb 20 > 13:03:36 2014 +0000 > +++ b/src/share/classes/java/util/TreeMap.java Thu Feb 20 > 12:32:52 2014 -0800 > @@ -1333,6 +1333,7 @@ > /** > * @serial include > */ > + @SuppressWarnings("serial") // Abstract class > abstract static class NavigableSubMap extends > AbstractMap > implements NavigableMap, java.io.Serializable { > /** > diff -r e5a09bc70308 > src/share/classes/sun/reflect/annotation/ExceptionProxy.java > --- a/src/share/classes/sun/reflect/annotation/ExceptionProxy.java > Thu Feb 20 13:03:36 2014 +0000 > +++ b/src/share/classes/sun/reflect/annotation/ExceptionProxy.java > Thu Feb 20 12:33:21 2014 -0800 > @@ -37,5 +37,6 @@ > * @since 1.5 > */ > public abstract class ExceptionProxy implements > java.io.Serializable { > + private static final long serialVersionUID = > 7241930048386631401L; > protected abstract RuntimeException generateException(); > } > > > > > -- > Cheers, > Paul From david.holmes at oracle.com Tue Feb 25 05:10:02 2014 From: david.holmes at oracle.com (David Holmes) Date: Tue, 25 Feb 2014 15:10:02 +1000 Subject: RFR [6853696] (ref) ReferenceQueue.remove(timeout) may return null even if timeout has not expired In-Reply-To: <530C12D4.8030009@oracle.com> References: <530ADFB2.8000100@oracle.com> <530B5929.7040907@oracle.com> <530B6669.1030409@oracle.com> <530C0552.5040503@oracle.com> <530C12D4.8030009@oracle.com> Message-ID: <530C25AA.9000400@oracle.com> On 25/02/2014 1:49 PM, Ivan Gerasimov wrote: > > On 25.02.2014 6:52, David Holmes wrote: >> On 25/02/2014 1:34 AM, Ivan Gerasimov wrote: >>> Thank you Roger for looking at the fix! >>> >>> On 24.02.2014 18:37, roger riggs wrote: >>>> Hi Ivan, >>>> >>>> The code is correct as written but there might be some creep in the >>>> end time due to the sampling of System.nanoTime. >>>> >>>> I would be inclined to calculate the final time of the timeout once >>>> and then compare simply with the current nanotime. >>>> >>>> long end = (timeout == 0) ? Long.MAX_VALUE : (System.nanoTime() + >>>> timeout * 1000000); >>>> >>> I don't think it is always correct. >>> According to the documentation, "The value returned represents >>> nanoseconds since some fixed but *arbitrary* origin time". >>> http://download.java.net/jdk8/docs/api/java/lang/System.html#nanoTime-- >>> >>> Thus, Long.MAX_VALUE may just happen to represent some valid time in the >>> near future, and we'll achieve wrong result. >> >> Correct. Simplest thing is to calculate end ignoring whether timeout >> is zero, and only check "now < end" if timeout != 0. > > David, as Martin said before, 'now < end' may hold true even if now is > after end (now is negative). > In my version of the code, I operate on the difference (after - before), > thus avoiding the possible overflow. Sorry, I put "now < end" in quotes to mean the effect of now Sincerely yours, > Ivan > >> >> David >> >>> Sincerely yours, >>> Ivan >>> >>>> Then the test in the loop can be: >>>> >>>> if (System.nanoTime() > end) { >>>> return null; >>>> } >>>> >>>> Roger (Not a Reviewer) >>>> >>>> On 2/24/2014 12:59 AM, Ivan Gerasimov wrote: >>>>> Hello! >>>>> >>>>> ReferenceQueue.remove(timeout) may return too early, i.e. before the >>>>> specified timeout has elapsed. >>>>> >>>>> Would you please review the fix? >>>>> The change also includes a regression test, which can be used to >>>>> demonstrate the issue. >>>>> >>>>> BUGURL: https://bugs.openjdk.java.net/browse/JDK-6853696 >>>>> WEBREV: http://cr.openjdk.java.net/~igerasim/6853696/0/webrev/ >>>>> >>>>> Sincerely yours, >>>>> Ivan >>>> >>>> >>> >> >> > From stuart.marks at oracle.com Tue Feb 25 06:53:41 2014 From: stuart.marks at oracle.com (Stuart Marks) Date: Mon, 24 Feb 2014 22:53:41 -0800 Subject: JDK 9 RFR of JDK-8035452: Fix serial lint warnings in core libs In-Reply-To: <530C1A87.2070608@oracle.com> References: <530666A5.8050607@oracle.com> <530C1A87.2070608@oracle.com> Message-ID: <530C3DF5.3040707@oracle.com> On 2/24/14 8:22 PM, Joe Darcy wrote: > On 02/20/2014 12:49 PM, Paul Benedict wrote: >> Joe, I find it interesting that you suppressed the serial warning on an >> abstract class. I'd like to know more about that. Is this a universal rule? >> Are serial uids not important for abstract classes? > > I wouldn't generalize that way from this example. > > The serial hash of NavigableSubMap has differed in different JDK releases, > but its subclasses do define serialVersionUID fields. I assume the set of > non-transient fields in NavigableSubMap has stayed unchanged, but I haven't > verified that. > > If I hadn't found the change in the hash value, I would have added the > serialVersionUID to NavigableSubMap too. And in his reply to Paul, Joe said: > From what I was able to discern by reading the serialization specification > [1], If a class does *not* declare a serialVersionUID, the serialVersionUID > of its superclass is *not* included in the serialver hash computation of the > child class. However, my understanding is that changes to the fields stored > in superclass and changes to the semantics of its readObject / writeObjects > methods could affect the serialization of the child class. I think we need to take a closer look at these issues. I believe that abstract, serializable superclasses *do* need to have a serialVersionUID defined. The reason is that when a subclass is serialized, its superclass descriptor (an ObjectStreamClass) is also serialized. Upon deserialization, the descriptor's svuid is matched against the svuid of the class loaded at runtime, and if there is a mismatch, InvalidClassException ensues. While the svuid of an abstract superclass isn't included in the subclass' svuid hash, the svuid of the superclass does affect serial compatibility of subclasses as described above. Thus, an apparently innocuous change to the superclass might prevent serial compatibility of its subclasses, no matter how carefully the subclasses are programmed. If the NavigableSubMap class has changed svuid values over several releases, well, unfortunately we may have a compatibility problem already in the field. We'd need to choose which release to be compatible with. Since 8 isn't quite out yet, we might be able to change an early 8-update and 9 to be compatibile with the latest 7-update. Note that the svuid of a class does not relate solely to the fields that are serialized. It's an attempt at a version hash of the *implementation* of a class, not a version of the serial protocol. Even changes to a class that don't affect the serialized output stream can affect the svuid. For example, renaming a package-private method will affect the svuid. See section 4.6 of the serialization spec. While we're at it (sorry...) in the diffs for your other serial warnings patch JDK-8035453, there are several lines where the serial warning is suppressed like so: + at SuppressWarnings("serial") // JDK implementation class As you know, serialization can expose the private fields of a class, making them public in a sense. Serialization can also expose what are internal, implementation classes, if these classes are part of a serializable object graph that is exposed to applications. I don't know about the specific situation with the DOM classes, but even if a serializable class is internal, we might need to be concerned about serialization compatibility. Finally, EnumSet doesn't need a serial version UID. It's serialized using a proxy class, so EnumSet never appears in a serialized byte stream. (Note, its readObject throws an exception unconditionally.) So it's probably safe to suppress its serialization warning. s'marks From tristan.yan at oracle.com Tue Feb 25 08:34:05 2014 From: tristan.yan at oracle.com (Tristan Yan) Date: Tue, 25 Feb 2014 16:34:05 +0800 Subject: RFR for JDK-8035388: TEST_BUG: java/rmi/activation/Activatable/checkActivateRef/CheckActivateRef.java fails Message-ID: <530C557D.9040003@oracle.com> Hi All Could you please help to review code fix for JDK-8035388. http://cr.openjdk.java.net/~tyan/JDK-8035388/webrev.00/ Description: method inactiveObject in ActivationGroupImpl.java, release lock happen before checkInactiveGroup. This makes groupInactive reset even before next inactiveObject started. Regards Tristan From yiming.wang at oracle.com Tue Feb 25 08:49:44 2014 From: yiming.wang at oracle.com (Eric Wang) Date: Tue, 25 Feb 2014 16:49:44 +0800 Subject: RFR for bug JDK-8035633 TEST_BUG: java/net/NetworkInterface/Equals.java and some tests failed on windows intermittently Message-ID: <530C5928.5070108@oracle.com> Hi Everyone, I'm working on the test bug https://bugs.openjdk.java.net/browse/JDK-8035633, There are 4 tests (one is in a closed test) failed due to NullPointerException. All tests failed at similar places "if (isWindows && ni.getDisplayName().contains("Teredo"))", the root cause is the NetworkInterface may return null if no display name is available. so the fix is to make sure the display name is not null before calling the method "contains("Teredo")", something like "if (isWindows && displayName != null && displayName.contains("Teredo"))" Please let me know if you have any comment. The webrev will be sent after internal review. Thanks, Eric From Alan.Bateman at oracle.com Tue Feb 25 08:59:23 2014 From: Alan.Bateman at oracle.com (Alan Bateman) Date: Tue, 25 Feb 2014 08:59:23 +0000 Subject: RFR for bug JDK-8035633 TEST_BUG: java/net/NetworkInterface/Equals.java and some tests failed on windows intermittently In-Reply-To: <530C5928.5070108@oracle.com> References: <530C5928.5070108@oracle.com> Message-ID: <530C5B6B.8090206@oracle.com> On 25/02/2014 08:49, Eric Wang wrote: > Hi Everyone, > > I'm working on the test bug > https://bugs.openjdk.java.net/browse/JDK-8035633, There are 4 tests > (one is in a closed test) failed due to NullPointerException. > All tests failed at similar places "if (isWindows && > ni.getDisplayName().contains("Teredo"))", the root cause is the > NetworkInterface may return null if no display name is available. > so the fix is to make sure the display name is not null before calling > the method "contains("Teredo")", something like "if (isWindows && > displayName != null && displayName.contains("Teredo"))" > > Please let me know if you have any comment. The webrev will be sent > after internal review. Yes, NetworkInterface's getDisplyName is allowed to return null and it looks like this wasn't taken into account when these tests were changed via JDK-8022963 to skip tunnels. Once you have a patch then net-dev would be the best list to bring it to. -Alan. From nhahtdh at gmail.com Tue Feb 25 09:30:28 2014 From: nhahtdh at gmail.com (Hong Dai Thanh) Date: Tue, 25 Feb 2014 17:30:28 +0800 Subject: RFR: JDK-6609854: Regex does not match correctly for negative nested character classes Message-ID: *Pretext* The reference implementation has strange behavior when there are nested character classes involved. This is described briefly in the bug report: https://bugs.openjdk.java.net/browse/JDK-6609854 And also brought up again with more details on stackoverflow.com http://stackoverflow.com/questions/21934168/double-negation-of-regex-character-classes *Proposed patch* I have looked at the current implementation and I'd like to propose the following patch. The patch is *not yet tested against the test cases in test package*. However, *the structure has been verified via reflection*. *(Commented lines are lines to be removed. Lines with trailing comment are those to be added.)* In method private CharProperty clazz(boolean consume) case ']': firstInClass = false; if (prev != null) { if (consume) next(); if (!include) { // prev = prev.complement(); // } // return prev; } break; default: firstInClass = false; break; } node = range(bits); // if (include) { if (prev == null) { prev = node; } else { if (prev != node) prev = union(prev, node); } // } else { // if (prev == null) { // prev = node.complement(); // } else { // if (prev != node) // prev = setDifference(prev, node); // } // } ch = peek(); *Fix Explanation* In the current code, a nested character class that generates a single Node (for example, [^[^5-6]], [^[^c]], [^[^456]]) will never reach the check if (include) outside the switch statement. As a result, the negation is not applied to the nested character class. It also causes redundancy with patterns such as [^45[^67]] or [^4589[^67]], since the first character 4 triggers complement() on the BitClass bits and every subsequence character (5, 8, 9) triggers setDifference() on the BitClass bits and the complement of the same BitClass bits. *Current structure* [^45[^67]] Start. Start unanchored match (minLength=1) Pattern.union (character class union). Match any character matched by either character classes below: Pattern.setDifference (character class subtraction). Match any character matched by the 1st character class, but NOT the 2nd character class: CharProperty.complement (character class negation). Match any character NOT matched by the following character class: BitClass. Optimized character class with boolean[] to match characters in Latin-1 (code point <= 255). Match the following 2 character(s): [U+0034][U+0035] 45 BitClass. Optimized character class with boolean[] to match characters in Latin-1 (code point <= 255). Match the following 2 character(s): [U+0034][U+0035] 45 Pattern.setDifference (character class subtraction). Match any character matched by the 1st character class, but NOT the 2nd character class: CharProperty.complement (character class negation). Match any character NOT matched by the following character class: BitClass. Optimized character class with boolean[] to match characters in Latin-1 (code point <= 255). Match the following 2 character(s): [U+0036][U+0037] 67 BitClass. Optimized character class with boolean[] to match characters in Latin-1 (code point <= 255). Match the following 2 character(s): [U+0036][U+0037] 67 LastNode Node. Accept match *Structure after patch* [^45[^67]] Start. Start unanchored match (minLength=1) CharProperty.complement (character class negation). Match any character NOT matched by the following character class: Pattern.union (character class union). Match any character matched by either character classes below: BitClass. Optimized character class with boolean[] to match characters in Latin-1 (code point <= 255). Match the following 2 character(s): [U+0034][U+0035] 45 CharProperty.complement (character class negation). Match any character NOT matched by the following character class: BitClass. Optimized character class with boolean[] to match characters in Latin-1 (code point <= 255). Match the following 2 character(s): [U+0036][U+0037] 67 LastNode Node. Accept match From vladimir.x.ivanov at oracle.com Tue Feb 25 10:15:02 2014 From: vladimir.x.ivanov at oracle.com (Vladimir Ivanov) Date: Tue, 25 Feb 2014 14:15:02 +0400 Subject: [9] RFR (M): 8027827: Improve performance of catchException combinator In-Reply-To: <0516A100-6FAF-4718-8A9F-A82D7C7E7061@oracle.com> References: <53052622.6010507@oracle.com> <0386E0D6-D515-4C2C-9D6E-DD3B53E8B3E2@oracle.com> <5306420B.9040200@oracle.com> <0516A100-6FAF-4718-8A9F-A82D7C7E7061@oracle.com> Message-ID: <530C6D26.4090000@oracle.com> John, Thanks for review! Updated webrev: http://cr.openjdk.java.net/~vlivanov/8027827/final/webrev.02/ See my comments inline. Also, integrated some pending cleanups (e.g. improved selectAlternative detection). Best regards, Vladimir Ivanov On 2/24/14 9:46 PM, John Rose wrote: > On Feb 20, 2014, at 9:57 AM, Vladimir Ivanov > > wrote: > >> Updated webrev: >> http://cr.openjdk.java.net/~vlivanov/8027827/final/webrev.01/ >> >> I finally figured out how to make caching work. This webrev contains >> these changes. >> >> I changed LF representation a bit and added 2 auxiliary method handles >> - argument boxing and wrapping into Object[] and result unboxing. >> These operations depend on actual type and can't be shared among >> arbitrary combinators with the same basic type. They are used only >> during LF interpretation and are completely ignored in compiled LFs. > > This is a good step forward, thanks! > > Some comments: > > I prefer the bounds check expression pos+3 < lambdaForm.names.length. > (One integer constant, limit to right.) Fixed. > The predicate isGuardWithCatch must test all three subforms. Or else > there must be assertions to ensure that names[pos+2] is of the expected > form. The problem is that LF's can sometimes be edited (e.g., by > binding operations) and there is no insurance that your pattern of three > expressions will be preserved in all cases. Fixed. > I see you are trying to do unboxing elimination here; this is not a safe > or effective way to do it, IMO. Put in a "FIXME" comment and file a bug > to deal better with unboxing ops in LFs. I have some WIP code toward > this end which we can talk about. You've probably seen of that > business, about internally LF marking expressions as intrinsics to guide > bytecode generation. Added a comment. Will file a bug once this change is in. > Why is the logic about cachedLambdaForm commented out? It looks > correct, but is there a bug? I think you looked at a stale version. Latest webrev have caching enabled. I had some problems with sharing before because boxing/unboxing is specific for different types and couldn't be shared among combinators with the same basic type. > Consider replacing GUARD_WITH_CATCH with Lazy.NF_guardWithCatch, and > using the NF instead of MH for the intrinsic. Done. Also updated other similar places in MethodHandleInfo. > > ? John From petr.pchelko at oracle.com Tue Feb 25 10:31:56 2014 From: petr.pchelko at oracle.com (Petr Pchelko) Date: Tue, 25 Feb 2014 14:31:56 +0400 Subject: Proposal: Make JNU_Throw* clear a pending exception Message-ID: Hello, Core and AWT teams. In AWT we have a lot of pending exception warnings which are now being fixed. A big fraction of this warnings is about a pending JNI exception at a call to JNU_Throw*. Why don?t we add an env->ExceptionClear() call in the beginning of each JNU_Throw function? It is absolutely safe because: 1. We are rethrowing an exception an loosing the original one anyway. 2. In case there?s no pending exception the ExceptionClear is a no-op. If we do this the code would become much cleaner, because currently we have to manually clear a pending exception before every call to JNU_Throw*. What do you think about this proposal? Thank you. With best regards. Petr. From Alan.Bateman at oracle.com Tue Feb 25 10:42:33 2014 From: Alan.Bateman at oracle.com (Alan Bateman) Date: Tue, 25 Feb 2014 10:42:33 +0000 Subject: Proposal: Make JNU_Throw* clear a pending exception In-Reply-To: References: Message-ID: <530C7399.7050706@oracle.com> On 25/02/2014 10:31, Petr Pchelko wrote: > Hello, Core and AWT teams. > > In AWT we have a lot of pending exception warnings which are now being fixed. A big fraction of this warnings is about a pending JNI exception at a call to JNU_Throw*. > Why don?t we add an env->ExceptionClear() call in the beginning of each JNU_Throw function? It is absolutely safe because: > 1. We are rethrowing an exception an loosing the original one anyway. > 2. In case there?s no pending exception the ExceptionClear is a no-op. > > If we do this the code would become much cleaner, because currently we have to manually clear a pending exception before every call to JNU_Throw*. > > What do you think about this proposal? > I can see how this might be attractive but doesn't it mean you are suppressing an important exception? We've fixed many areas in the last few weeks and I think a common case was just that whoever wrote the original code didn't realize that some JNI functions having a pending exception when they fail. In those cases then often it is a simple matter of just returning from the JNI function (maybe after some clean-up). Are the examples that you are looking at along these lines? I guess I'm mostly interested to know whether the JNU_Throw usages are needed or not. -Alan. From ali.ebrahimi1781 at gmail.com Tue Feb 25 11:13:33 2014 From: ali.ebrahimi1781 at gmail.com (Ali Ebrahimi) Date: Tue, 25 Feb 2014 14:43:33 +0330 Subject: Signature of MethodHandleInfo.reflectAs is not specific enough In-Reply-To: <528008C9.7000905@univ-mlv.fr> References: <528008C9.7000905@univ-mlv.fr> Message-ID: Hi, I know, this is too late, but I want to share my suggestion: public T reflectAs(Class expected, MethodHandles.Lookup lookup) Member mr = reflectAs(Member.class, MethodHandles.lookup()); AnnotatedElement ae = reflectAs(AnnotatedElement.class, MethodHandles.lookup()); AnnotatedElement am = reflectAs(Member.class, MethodHandles.lookup()); Field fd= reflectAs(Field.class, MethodHandles.lookup()); Constructor cr = reflectAs(Constructor.class, MethodHandles.lookup()); Method md = reflectAs(Method.class, MethodHandles.lookup()); Field fm= reflectAs(Member.class, MethodHandles.lookup()); Constructor cm = reflectAs(Member.class, MethodHandles.lookup()); Method mm = reflectAs(Member.class, MethodHandles.lookup()); Field fa= reflectAs(AnnotatedElement.class, MethodHandles.lookup()); Constructor ca = reflectAs(AnnotatedElement.class, MethodHandles.lookup()); Method ma = reflectAs(AnnotatedElement.class, MethodHandles.lookup()); Member mf= reflectAs(Field.class, MethodHandles.lookup()); Member mc = reflectAs(Constructor.class, MethodHandles.lookup()); Member mrm = reflectAs(Method.class, MethodHandles.lookup()); AnnotatedElement af= reflectAs(Field.class, MethodHandles.lookup()); AnnotatedElement ac = reflectAs(Constructor.class, MethodHandles.lookup()); AnnotatedElement aem = reflectAs(Method.class, MethodHandles.lookup()); Method mdc = reflectAs(Constructor.class, MethodHandles.lookup());//fails Constructor crm = reflectAs(Method.class, MethodHandles.lookup());//fails Constructor cf = reflectAs(Field.class, MethodHandles.lookup());//fails Field fc = reflectAs(Constructor.class, MethodHandles.lookup());//fails AnnotatedElement as = reflectAs(String.class, MethodHandles.lookup());//fails String ss = reflectAs(String.class, MethodHandles.lookup());//fails String sm = reflectAs(Method.class, MethodHandles.lookup());//fails Regards, Ali Ebrahimi On Mon, Nov 11, 2013 at 1:59 AM, Remi Forax wrote: > The is a stupid issue with the signature of MethodHandleInfo.reflectAs, > j.l.r.Field, Method or Constructor implement two interfaces Member and > AnnotatedElement, with the current signature, the code > info.reflectAs(Member.class, lookup) > works but the code > info.reflectAs(AnnotatedElement.class, lookup) > doesn't work. > > Because there is no way to do an 'or' between several bounds of > a type variable, I think that the signature of reflectAs should be > changed from : > public T reflectAs(Class expected, Lookup lookup); > to > public T reflectAs(Class expected, Lookup lookup); > > and the javadoc should be modified to explain that a Member or > AnnotatedElement are > valid bounds of T. > > As a side effect, the signature of MethodHandles.reflectAs(Class, > MethodHandle) > should be updated accordingly. > > There is a workaround, one can write: > (AnnotatedElement)info.reflectAs(Member.class, lookup) > but it's at best weird. > > cheers, > R?mi > > _______________________________________________ > mlvm-dev mailing list > mlvm-dev at openjdk.java.net > http://mail.openjdk.java.net/mailman/listinfo/mlvm-dev > From petr.pchelko at oracle.com Tue Feb 25 11:26:09 2014 From: petr.pchelko at oracle.com (Petr Pchelko) Date: Tue, 25 Feb 2014 15:26:09 +0400 Subject: Proposal: Make JNU_Throw* clear a pending exception In-Reply-To: <530C7399.7050706@oracle.com> References: <530C7399.7050706@oracle.com> Message-ID: <18CF8FC4-5524-4980-96BF-5A9E361637C0@oracle.com> Hello, Alan. > I can see how this might be attractive but doesn't it mean you are suppressing an important exception? In case we?ve already got into the JNU_Throw we will throw a new exception and override the original one anyway. However I agree that this might suppress the warning for the code analysis tools. > Are the examples that you are looking at along these lines? There are a number of examples when JNU_Throw is used to: 1. Replace the message of an original exception: src/share/native/sun/awt/image/awt_ImageRep.c:192 There are a few such places. 2. Rethrow some meaningful exception if the call to some function failed: src/windows/native/sun/windows/awt_Window.cpp:2861 This is a much more common use case. In this case we have a return code from some method and we do not care if it was failed because of JNI exception or for some other reason. This is the main case where we need to add env->ExceptionClear() everywhere. 3. Quite common is to use it in the initIDs method to rethrow NoSuchFieldError: src/share/native/sun/awt/image/BufImgSurfaceData.c:77 This one is questionable, I think that rethrowing is not needed here at all, the original exception is much more informative. 4. Where currently are throwing an exception with pure JNI, but it could be replaces with JNU: src/windows/native/sun/windows/awt_PrintJob.cpp:1365 Similar to #2. With best regards. Petr. 25 ????. 2014 ?., ? 2:42 ????? ???????, Alan Bateman ???????(?): > On 25/02/2014 10:31, Petr Pchelko wrote: >> Hello, Core and AWT teams. >> >> In AWT we have a lot of pending exception warnings which are now being fixed. A big fraction of this warnings is about a pending JNI exception at a call to JNU_Throw*. >> Why don?t we add an env->ExceptionClear() call in the beginning of each JNU_Throw function? It is absolutely safe because: >> 1. We are rethrowing an exception an loosing the original one anyway. >> 2. In case there?s no pending exception the ExceptionClear is a no-op. >> >> If we do this the code would become much cleaner, because currently we have to manually clear a pending exception before every call to JNU_Throw*. >> >> What do you think about this proposal? >> > I can see how this might be attractive but doesn't it mean you are suppressing an important exception? We've fixed many areas in the last few weeks and I think a common case was just that whoever wrote the original code didn't realize that some JNI functions having a pending exception when they fail. In those cases then often it is a simple matter of just returning from the JNI function (maybe after some clean-up). Are the examples that you are looking at along these lines? I guess I'm mostly interested to know whether the JNU_Throw usages are needed or not. > > -Alan. From chris.hegarty at oracle.com Tue Feb 25 11:35:53 2014 From: chris.hegarty at oracle.com (Chris Hegarty) Date: Tue, 25 Feb 2014 11:35:53 +0000 Subject: Proposal: Make JNU_Throw* clear a pending exception In-Reply-To: <18CF8FC4-5524-4980-96BF-5A9E361637C0@oracle.com> References: <530C7399.7050706@oracle.com> <18CF8FC4-5524-4980-96BF-5A9E361637C0@oracle.com> Message-ID: <530C8019.40309@oracle.com> On 25/02/14 11:26, Petr Pchelko wrote: > Hello, Alan. > >> I can see how this might be attractive but doesn't it mean you are suppressing an important exception? > In case we?ve already got into the JNU_Throw we will throw a new exception and override the original one anyway. However I agree that this might suppress the warning for the code analysis tools. > >> Are the examples that you are looking at along these lines? > There are a number of examples when JNU_Throw is used to: > 1. Replace the message of an original exception: src/share/native/sun/awt/image/awt_ImageRep.c:192 > There are a few such places. > > 2. Rethrow some meaningful exception if the call to some function failed: src/windows/native/sun/windows/awt_Window.cpp:2861 > This is a much more common use case. In this case we have a return code from some method and we do not care if it was failed because of JNI exception or for some other reason. This is the main case where we need to add env->ExceptionClear() everywhere. > > 3. Quite common is to use it in the initIDs method to rethrow NoSuchFieldError: src/share/native/sun/awt/image/BufImgSurfaceData.c:77 > This one is questionable, I think that rethrowing is not needed here at all, the original exception is much more informative. Agreed. Similar to NetworkInterface.c Line:172 http://hg.openjdk.java.net/jdk9/dev/jdk/file/6ee5c47bdba7/src/solaris/native/java/net/NetworkInterface.c#172 -Chris. > 4. Where currently are throwing an exception with pure JNI, but it could be replaces with JNU: src/windows/native/sun/windows/awt_PrintJob.cpp:1365 > Similar to #2. > > With best regards. Petr. > > 25 ????. 2014 ?., ? 2:42 ????? ???????, Alan Bateman ???????(?): > >> On 25/02/2014 10:31, Petr Pchelko wrote: >>> Hello, Core and AWT teams. >>> >>> In AWT we have a lot of pending exception warnings which are now being fixed. A big fraction of this warnings is about a pending JNI exception at a call to JNU_Throw*. >>> Why don?t we add an env->ExceptionClear() call in the beginning of each JNU_Throw function? It is absolutely safe because: >>> 1. We are rethrowing an exception an loosing the original one anyway. >>> 2. In case there?s no pending exception the ExceptionClear is a no-op. >>> >>> If we do this the code would become much cleaner, because currently we have to manually clear a pending exception before every call to JNU_Throw*. >>> >>> What do you think about this proposal? >>> >> I can see how this might be attractive but doesn't it mean you are suppressing an important exception? We've fixed many areas in the last few weeks and I think a common case was just that whoever wrote the original code didn't realize that some JNI functions having a pending exception when they fail. In those cases then often it is a simple matter of just returning from the JNI function (maybe after some clean-up). Are the examples that you are looking at along these lines? I guess I'm mostly interested to know whether the JNU_Throw usages are needed or not. >> >> -Alan. > From paul.sandoz at oracle.com Tue Feb 25 12:26:21 2014 From: paul.sandoz at oracle.com (Paul Sandoz) Date: Tue, 25 Feb 2014 13:26:21 +0100 Subject: JDK 9 RFR of JDK-8035279: Clean up internal deprecations in BigInteger In-Reply-To: References: <5305D6EB.30806@oracle.com> <53064179.2060703@gmail.com> <530660FA.1060407@oracle.com> <96CF3236-4E65-4D64-BCF3-1E9660D29BAA@oracle.com> <5306BC4B.4090909@oracle.com> Message-ID: <8229AE11-1ABF-474D-961F-BBD5ACA3DD82@oracle.com> On Feb 25, 2014, at 1:32 AM, Brian Burkhalter wrote: > > On Feb 20, 2014, at 6:39 PM, David Holmes wrote: > >> Not clear what you mean by this. > > This is more or less my reaction to this entire thread, so to speak. ;-) Anyway, thanks for all the comments. > :-) > Note that I am ignoring the powerCache field comments for the moment. > > To distill the discussion down to just the proposed changes I posted, my reading is that there is more or less consensus on two points: > > 1) the instance fields in question *should* be volatile for this proposed change set Yes. Sorry, i mislead you earlier, i got confused about what hotspot does (and realistically most other platforms) compared to what is currently specified (and hopefully rectified in the JMM update). > 2) non-zero initial values should be avoided in case of instance leaking to non-constructing threads > If say, in your patch, the bitCount field was not volatile then there is a possibility on some platforms that a non-constructing thread may observe default values: class X { static BigInteger x; static void threadOne() { x = new BigInteger("11111111", 2); // A compiler is allowed to move code such that the put to x can occur before the put to bitCount // If bitCount is volatile then a memory barrier or fence is placed around the put of bitCount // such that (in general) code above cannot be moved below and code below cannot be moved above // (in addition to other things going with the caches on the processors to ensure visibility) } static void threadTwo() { if (x != null) { int bc = x.bitCount(); assert bc = 8; // may fail, could return 0 } } } I tweaked the example from http://www.cs.umd.edu/~pugh/java/memoryModel/jsr-133-faq.html (although 10 years old that article is one of the best around at explaining things). If you leak the "this" reference to another thread during construction then all bets are off w.r.t. the visibility of any final fields. Note the use of putVolatile on the final fields when deserializing, this ensures fences are in place so that the visibility of the final fields is preserved [*]. > Is this accurate? > > On second thought it occurred to me that instead of any complicated or contentious changes, as it were, the ugliness I was trying to remove from the code could just as well be addressed by simply changing the names of the affected instance variables to indicate what their respective values really represent, e.g., "bitCount" becomes "bitCountPlusOne" and we remove the @Deprecated and @deprecated. Yeah this is still ugly ... > Might as well just remove the @Deprecated stuff. I think it is fine under the circumstances to have offsets and getter methods that return the correct values. Paul. [*] Strictly speaking it should only be necessary to do a put*Volatile on the last final field set, others can use put*. And, unfortunately this is one use-case where developers need to use Unsafe. Even if we have fence method on Objects the final field still needs to be set and reflection is currently slower, although MethodHandles could be an alternative when using a trusted Lookup instance. From paul.sandoz at oracle.com Tue Feb 25 12:41:04 2014 From: paul.sandoz at oracle.com (Paul Sandoz) Date: Tue, 25 Feb 2014 13:41:04 +0100 Subject: RFR 8027640 : String.indexOf(String, int) for the empty string case not specified In-Reply-To: <530BDB37.40603@oracle.com> References: <530BDB37.40603@oracle.com> Message-ID: On Feb 25, 2014, at 12:52 AM, Brent Christian wrote: > Please review my changes for: > https://bugs.openjdk.java.net/browse/JDK-8027640 > > The webrev is here: > http://cr.openjdk.java.net/~bchristi/8027640/webrev.00/ > In AbstractStringBuilder there is a small typo, "k<": 1348 *

The returned index is the largest value {@code k<} for which: Otherwise looks ok. Paul. > > This is a small cleanup to reconcile the String[last]indexOf() spec with longstanding behavior. There are no changes to executable code. > > In String, the "Math.min(fromIndex, this.length())" clause is brought over from String[Buffer|Builder] for indexOf(String,int) and lastIndexOf(String,int). This resolves the issue for which the bug was filed. > > To further sync up the [last]indexOf() docs, I ported the 6940381 wording improvements from String to String[Buffer|Builder]. I also made a few doclint updates to the code in question. > > Thanks, > -Brent > From roger.riggs at oracle.com Tue Feb 25 14:14:08 2014 From: roger.riggs at oracle.com (roger riggs) Date: Tue, 25 Feb 2014 09:14:08 -0500 Subject: Proposal: Make JNU_Throw* clear a pending exception In-Reply-To: <530C8019.40309@oracle.com> References: <530C7399.7050706@oracle.com> <18CF8FC4-5524-4980-96BF-5A9E361637C0@oracle.com> <530C8019.40309@oracle.com> Message-ID: <530CA530.5070904@oracle.com> In some cases, I would expect that the exception being overridden would/should become the 'cause' of the new exception so it is not cleared but chained. Does JNI support that? On the original issue, discarding of exceptions should be explicit not implicit. Keep (or insert) the exceptionClear(). Roger On 2/25/2014 6:35 AM, Chris Hegarty wrote: > > > On 25/02/14 11:26, Petr Pchelko wrote: >> Hello, Alan. >> >>> I can see how this might be attractive but doesn't it mean you are >>> suppressing an important exception? >> In case we?ve already got into the JNU_Throw we will throw a new >> exception and override the original one anyway. However I agree that >> this might suppress the warning for the code analysis tools. >> >>> Are the examples that you are looking at along these lines? >> There are a number of examples when JNU_Throw is used to: >> 1. Replace the message of an original exception: >> src/share/native/sun/awt/image/awt_ImageRep.c:192 >> There are a few such places. >> >> 2. Rethrow some meaningful exception if the call to some function >> failed: src/windows/native/sun/windows/awt_Window.cpp:2861 >> This is a much more common use case. In this case we have a return >> code from some method and we do not care if it was failed because of >> JNI exception or for some other reason. This is the main case where >> we need to add env->ExceptionClear() everywhere. >> >> 3. Quite common is to use it in the initIDs method to rethrow >> NoSuchFieldError: src/share/native/sun/awt/image/BufImgSurfaceData.c:77 >> This one is questionable, I think that rethrowing is not needed here >> at all, the original exception is much more informative. > > Agreed. Similar to NetworkInterface.c Line:172 > > http://hg.openjdk.java.net/jdk9/dev/jdk/file/6ee5c47bdba7/src/solaris/native/java/net/NetworkInterface.c#172 > > > -Chris. > >> 4. Where currently are throwing an exception with pure JNI, but it >> could be replaces with JNU: >> src/windows/native/sun/windows/awt_PrintJob.cpp:1365 >> Similar to #2. >> >> With best regards. Petr. >> >> 25 ????. 2014 ?., ? 2:42 ????? ???????, Alan Bateman >> ???????(?): >> >>> On 25/02/2014 10:31, Petr Pchelko wrote: >>>> Hello, Core and AWT teams. >>>> >>>> In AWT we have a lot of pending exception warnings which are now >>>> being fixed. A big fraction of this warnings is about a pending JNI >>>> exception at a call to JNU_Throw*. >>>> Why don?t we add an env->ExceptionClear() call in the beginning of >>>> each JNU_Throw function? It is absolutely safe because: >>>> 1. We are rethrowing an exception an loosing the original one anyway. >>>> 2. In case there?s no pending exception the ExceptionClear is a no-op. >>>> >>>> If we do this the code would become much cleaner, because currently >>>> we have to manually clear a pending exception before every call to >>>> JNU_Throw*. >>>> >>>> What do you think about this proposal? >>>> >>> I can see how this might be attractive but doesn't it mean you are >>> suppressing an important exception? We've fixed many areas in the >>> last few weeks and I think a common case was just that whoever wrote >>> the original code didn't realize that some JNI functions having a >>> pending exception when they fail. In those cases then often it is a >>> simple matter of just returning from the JNI function (maybe after >>> some clean-up). Are the examples that you are looking at along these >>> lines? I guess I'm mostly interested to know whether the JNU_Throw >>> usages are needed or not. >>> >>> -Alan. >> From ivan.gerasimov at oracle.com Tue Feb 25 14:22:27 2014 From: ivan.gerasimov at oracle.com (Ivan Gerasimov) Date: Tue, 25 Feb 2014 18:22:27 +0400 Subject: RFR [6853696] (ref) ReferenceQueue.remove(timeout) may return null even if timeout has not expired In-Reply-To: <41CC73D0-1754-4A62-A90B-03AD2F372CBC@oracle.com> References: <530ADFB2.8000100@oracle.com> <530B5929.7040907@oracle.com> <41CC73D0-1754-4A62-A90B-03AD2F372CBC@oracle.com> Message-ID: <530CA723.5040706@oracle.com> Thank you Mike! On 24.02.2014 22:26, Mike Duigou wrote: > On Feb 24 2014, at 06:37 , roger riggs wrote: > >> Hi Ivan, >> >> The code is correct as written but there might be some creep in the end time due to the sampling of System.nanoTime. >> >> I would be inclined to calculate the final time of the timeout once >> and then compare simply with the current nanotime. >> >> long end = (timeout == 0) ? Long.MAX_VALUE : (System.nanoTime() + timeout * 1000000); > I hate seeing numerical constants > > TimeUnit.MILLISECONDS.toNanos(timeout) Yes, this is much clearer. Though I used the opposite conversion: NANOSECONDS.toMillis(end - start); Would you please take a look at the updated webrev: http://cr.openjdk.java.net/~igerasim/6853696/1/webrev/ Sincerely yours, Ivan > >> Then the test in the loop can be: >> >> if (System.nanoTime() > end) { >> return null; >> } > This compare should be re-written in the overflow compensating style Martin mentions. > > Mike > >> Roger (Not a Reviewer) >> >> On 2/24/2014 12:59 AM, Ivan Gerasimov wrote: >>> Hello! >>> >>> ReferenceQueue.remove(timeout) may return too early, i.e. before the specified timeout has elapsed. >>> >>> Would you please review the fix? >>> The change also includes a regression test, which can be used to demonstrate the issue. >>> >>> BUGURL: https://bugs.openjdk.java.net/browse/JDK-6853696 >>> WEBREV: http://cr.openjdk.java.net/~igerasim/6853696/0/webrev/ >>> >>> Sincerely yours, >>> Ivan > > From ivan.gerasimov at oracle.com Tue Feb 25 14:32:38 2014 From: ivan.gerasimov at oracle.com (Ivan Gerasimov) Date: Tue, 25 Feb 2014 18:32:38 +0400 Subject: RFR [6853696] (ref) ReferenceQueue.remove(timeout) may return null even if timeout has not expired In-Reply-To: <530BC37F.2050908@oracle.com> References: <530ADFB2.8000100@oracle.com> <530BC37F.2050908@oracle.com> Message-ID: <530CA986.9090605@oracle.com> Thank you Mandy! On 25.02.2014 2:11, Mandy Chung wrote: > On 2/23/14 9:59 PM, Ivan Gerasimov wrote: >> Hello! >> >> ReferenceQueue.remove(timeout) may return too early, i.e. before the >> specified timeout has elapsed. >> >> Would you please review the fix? >> The change also includes a regression test, which can be used to >> demonstrate the issue. >> >> BUGURL: https://bugs.openjdk.java.net/browse/JDK-6853696 >> WEBREV: http://cr.openjdk.java.net/~igerasim/6853696/0/webrev/ >> > > I'll review the patch once you address the numerical overflow issue > Martin points out. One minor comment - I suggest to use "start" and > "end" instead of "before" and "after". > I already operate on the difference (end-start), so the overflow should not happen. Changed the names to start and end as you suggested. > One comment on the test, line 61: I think you want to check > thread.suspect expects to be true. It may be simply keeping the > returned value of ReferenceQueue.remove call and check that in line 61 > instead of having thread.suspect variable. Done. > line 76: why do you want to catch InterruptedException? If > interrupted, should the test fail? ReferenceQueue#remove() can throw InterruptedException, so I had to handle it. In the new webrev I set the initial value of actual to TIMEOUT, so if the thread is interrupted the test will pass. Please take a look at the updated webrev: http://cr.openjdk.java.net/~igerasim/6853696/1/webrev/ Sincerely yours, Ivan > > Mandy > From xueming.shen at oracle.com Tue Feb 25 16:07:02 2014 From: xueming.shen at oracle.com (Xueming Shen) Date: Tue, 25 Feb 2014 08:07:02 -0800 Subject: RFR: JDK-6609854: Regex does not match correctly for negative nested character classes In-Reply-To: References: Message-ID: <530CBFA6.9010301@oracle.com> Couple rounds of emails had been exchanged internally regarding this issue years ago and a proposal had been briefly discussed at this list back to 2011 [1]. The only reason that hold us to just fix it is the "compatibility" concern, since the described behavior has been there for decade. It appears we can try it again, as if we are getting more support:-) Thanks! -Sherman [1] http://mail.openjdk.java.net/pipermail/core-libs-dev/2011-June/006957.html On 2/25/14 1:30 AM, Hong Dai Thanh wrote: > *Pretext* > > The reference implementation has strange behavior when there are > nested character classes involved. > > This is described briefly in the bug report: > https://bugs.openjdk.java.net/browse/JDK-6609854 > > And also brought up again with more details on stackoverflow.com > > > http://stackoverflow.com/questions/21934168/double-negation-of-regex-character-classes > > *Proposed patch* > > I have looked at the current implementation and I'd like to propose > the following patch. > > The patch is *not yet tested against the test cases in test package*. > > However, *the structure has been verified via reflection*. > > /(Commented lines are lines to be removed. Lines with trailing comment > are those to be added.)/ > > In method private CharProperty clazz(boolean consume) > > case ']': > firstInClass = false; > if (prev != null) { > if (consume) > next(); > if (!include) { // > prev = prev.complement(); // > } // > return prev; > } > break; > default: > firstInClass = false; > break; > } > node = range(bits); > // if (include) { > if (prev == null) { > prev = node; > } else { > if (prev != node) > prev = union(prev, node); > } > // } else { > // if (prev == null) { > // prev = node.complement(); > // } else { > // if (prev != node) > // prev = setDifference(prev, node); > // } > // } > ch = peek(); > > *Fix Explanation* > > In the current code, a nested character class that generates a single > Node (for example, [^[^5-6]], [^[^c]], [^[^456]]) will never reach the > check if (include) outside the switch statement. As a result, the > negation is not applied to the nested character class. > > It also causes redundancy with patterns such as [^45[^67]] or > [^4589[^67]], since the first character 4 triggers complement() on the > BitClass bits and every subsequence character (5, 8, 9) triggers > setDifference() on the BitClass bits and the complement of the same > BitClass bits. > > *Current structure* > > [^45[^67]] > Start. Start unanchored match (minLength=1) > Pattern.union (character class union). Match any character matched by > either character classes below: > Pattern.setDifference (character class subtraction). Match any > character matched by the 1st character class, but NOT the 2nd > character class: > CharProperty.complement (character class negation). Match any > character NOT matched by the following character class: > BitClass. Optimized character class with boolean[] to match > characters in Latin-1 (code point <= 255). Match the following 2 > character(s): > [U+0034][U+0035] > 45 > BitClass. Optimized character class with boolean[] to match > characters in Latin-1 (code point <= 255). Match the following 2 > character(s): > [U+0034][U+0035] > 45 > Pattern.setDifference (character class subtraction). Match any > character matched by the 1st character class, but NOT the 2nd > character class: > CharProperty.complement (character class negation). Match any > character NOT matched by the following character class: > BitClass. Optimized character class with boolean[] to match > characters in Latin-1 (code point <= 255). Match the following 2 > character(s): > [U+0036][U+0037] > 67 > BitClass. Optimized character class with boolean[] to match > characters in Latin-1 (code point <= 255). Match the following 2 > character(s): > [U+0036][U+0037] > 67 > LastNode > Node. Accept match > > *Structure after patch* > > [^45[^67]] > Start. Start unanchored match (minLength=1) > CharProperty.complement (character class negation). Match any > character NOT matched by the following character class: > Pattern.union (character class union). Match any character matched > by either character classes below: > BitClass. Optimized character class with boolean[] to match > characters in Latin-1 (code point <= 255). Match the following 2 > character(s): > [U+0034][U+0035] > 45 > CharProperty.complement (character class negation). Match any > character NOT matched by the following character class: > BitClass. Optimized character class with boolean[] to match > characters in Latin-1 (code point <= 255). Match the following 2 > character(s): > [U+0036][U+0037] > 67 > LastNode > Node. Accept match From mandy.chung at oracle.com Tue Feb 25 17:53:57 2014 From: mandy.chung at oracle.com (Mandy Chung) Date: Tue, 25 Feb 2014 09:53:57 -0800 Subject: RFR [6853696] (ref) ReferenceQueue.remove(timeout) may return null even if timeout has not expired In-Reply-To: <530CA986.9090605@oracle.com> References: <530ADFB2.8000100@oracle.com> <530BC37F.2050908@oracle.com> <530CA986.9090605@oracle.com> Message-ID: <530CD8B5.90000@oracle.com> On 2/25/14 6:32 AM, Ivan Gerasimov wrote: >> line 76: why do you want to catch InterruptedException? If >> interrupted, should the test fail? > ReferenceQueue#remove() can throw InterruptedException, so I had to > handle it. > In the new webrev I set the initial value of actual to TIMEOUT, so if > the thread is interrupted the test will pass. > Catching the InterruptedException would hide any unexpected issue as the test doesn't really expect to be interrupted. For example if the test is interrupted (kill -9), we should let the test to fail which is fine. I suggest not to catch it. line 61: I think the test should be: if (thread.reference != null || thread.actual < TIMEOUT) you may want to update the exception. You could simply throw RuntimeException (just a minor point). > Please take a look at the updated webrev: > http://cr.openjdk.java.net/~igerasim/6853696/1/webrev/ Looks okay. I have a slight preference to keep the simple division to convert to millis rather than loading the enum TimeUnit class for this purpose. Mandy From martinrb at google.com Tue Feb 25 18:33:16 2014 From: martinrb at google.com (Martin Buchholz) Date: Tue, 25 Feb 2014 10:33:16 -0800 Subject: JDK 9 RFR of JDK-8035452: Fix serial lint warnings in core libs In-Reply-To: <530C3DF5.3040707@oracle.com> References: <530666A5.8050607@oracle.com> <530C1A87.2070608@oracle.com> <530C3DF5.3040707@oracle.com> Message-ID: (As I've said before ...) Serialization is too hard to get right. You need a full-time engineer (Peter Jones used to be that) who specializes in serialization and builds tools to ensure that serialization is implemented correctly, e.g. that internal implementation details don't leak out and that cross-version and cross-implementation tests are written so that the promise of serialization is not broken by careless JDK engineers (no one is careful enough!). Our testing culture and tools tend to assume only a single JDK at a time. The default serialization tries to be helpful, but too often exposes implementation details. It would have been better to force everyone to write those writeObject methods. It would have been better if most Collection implementations serialized the same way, an int size followed by all the elements in iterator order. There should have been a utility method to help do that. Compare and contrast with the reliable "serialization" provided by Collection.toString. On Mon, Feb 24, 2014 at 10:53 PM, Stuart Marks wrote: > On 2/24/14 8:22 PM, Joe Darcy wrote: > >> On 02/20/2014 12:49 PM, Paul Benedict wrote: >> >>> Joe, I find it interesting that you suppressed the serial warning on an >>> abstract class. I'd like to know more about that. Is this a universal >>> rule? >>> Are serial uids not important for abstract classes? >>> >> >> I wouldn't generalize that way from this example. >> >> The serial hash of NavigableSubMap has differed in different JDK releases, >> but its subclasses do define serialVersionUID fields. I assume the set of >> non-transient fields in NavigableSubMap has stayed unchanged, but I >> haven't >> verified that. >> >> If I hadn't found the change in the hash value, I would have added the >> serialVersionUID to NavigableSubMap too. >> > > And in his reply to Paul, Joe said: > >> From what I was able to discern by reading the serialization specification >> [1], If a class does *not* declare a serialVersionUID, the >> serialVersionUID >> of its superclass is *not* included in the serialver hash computation of >> the >> child class. However, my understanding is that changes to the fields >> stored >> in superclass and changes to the semantics of its readObject / >> writeObjects >> methods could affect the serialization of the child class. >> > > I think we need to take a closer look at these issues. > > I believe that abstract, serializable superclasses *do* need to have a > serialVersionUID defined. The reason is that when a subclass is serialized, > its superclass descriptor (an ObjectStreamClass) is also serialized. Upon > deserialization, the descriptor's svuid is matched against the svuid of the > class loaded at runtime, and if there is a mismatch, InvalidClassException > ensues. > > While the svuid of an abstract superclass isn't included in the subclass' > svuid hash, the svuid of the superclass does affect serial compatibility of > subclasses as described above. Thus, an apparently innocuous change to the > superclass might prevent serial compatibility of its subclasses, no matter > how carefully the subclasses are programmed. > > If the NavigableSubMap class has changed svuid values over several > releases, well, unfortunately we may have a compatibility problem already > in the field. We'd need to choose which release to be compatible with. > Since 8 isn't quite out yet, we might be able to change an early 8-update > and 9 to be compatibile with the latest 7-update. > > Note that the svuid of a class does not relate solely to the fields that > are serialized. It's an attempt at a version hash of the *implementation* > of a class, not a version of the serial protocol. Even changes to a class > that don't affect the serialized output stream can affect the svuid. For > example, renaming a package-private method will affect the svuid. See > section 4.6 of the serialization spec. > > While we're at it (sorry...) in the diffs for your other serial warnings > patch JDK-8035453, there are several lines where the serial warning is > suppressed like so: > > + at SuppressWarnings("serial") // JDK implementation class > > As you know, serialization can expose the private fields of a class, > making them public in a sense. Serialization can also expose what are > internal, implementation classes, if these classes are part of a > serializable object graph that is exposed to applications. I don't know > about the specific situation with the DOM classes, but even if a > serializable class is internal, we might need to be concerned about > serialization compatibility. > > Finally, EnumSet doesn't need a serial version UID. It's serialized using > a proxy class, so EnumSet never appears in a serialized byte stream. (Note, > its readObject throws an exception unconditionally.) So it's probably safe > to suppress its serialization warning. > > s'marks > > > From mike.duigou at oracle.com Tue Feb 25 18:47:11 2014 From: mike.duigou at oracle.com (Mike Duigou) Date: Tue, 25 Feb 2014 10:47:11 -0800 Subject: RFR [6853696] (ref) ReferenceQueue.remove(timeout) may return null even if timeout has not expired In-Reply-To: <530CA723.5040706@oracle.com> References: <530ADFB2.8000100@oracle.com> <530B5929.7040907@oracle.com> <41CC73D0-1754-4A62-A90B-03AD2F372CBC@oracle.com> <530CA723.5040706@oracle.com> Message-ID: <5DE1FEC2-4EC4-4A5C-8214-DB235639E9EB@oracle.com> Looks OK. Some minor comments: - Some of the static fields in the test could be final (queue, weakReference, startedSignal). - After the join() loop in the test you could check that the weakReference is cleared and enqueued. - Why is the check thread.actual < TIMEOUT only done if thread.reference is null? This would seem to be appropriate for both cases. (I like Mandy's suggestion of using || on the conditions). - I agree with Mandy about throwing RuntimeException instead of Exception. Mike On Feb 25 2014, at 06:22 , Ivan Gerasimov wrote: > Thank you Mike! > > On 24.02.2014 22:26, Mike Duigou wrote: >> On Feb 24 2014, at 06:37 , roger riggs wrote: >> >>> Hi Ivan, >>> >>> The code is correct as written but there might be some creep in the end time due to the sampling of System.nanoTime. >>> >>> I would be inclined to calculate the final time of the timeout once >>> and then compare simply with the current nanotime. >>> >>> long end = (timeout == 0) ? Long.MAX_VALUE : (System.nanoTime() + timeout * 1000000); >> I hate seeing numerical constants >> >> TimeUnit.MILLISECONDS.toNanos(timeout) > Yes, this is much clearer. > Though I used the opposite conversion: NANOSECONDS.toMillis(end - start); > > Would you please take a look at the updated webrev: > http://cr.openjdk.java.net/~igerasim/6853696/1/webrev/ > > Sincerely yours, > Ivan > >> >>> Then the test in the loop can be: >>> >>> if (System.nanoTime() > end) { >>> return null; >>> } >> This compare should be re-written in the overflow compensating style Martin mentions. >> >> Mike >> >>> Roger (Not a Reviewer) >>> >>> On 2/24/2014 12:59 AM, Ivan Gerasimov wrote: >>>> Hello! >>>> >>>> ReferenceQueue.remove(timeout) may return too early, i.e. before the specified timeout has elapsed. >>>> >>>> Would you please review the fix? >>>> The change also includes a regression test, which can be used to demonstrate the issue. >>>> >>>> BUGURL: https://bugs.openjdk.java.net/browse/JDK-6853696 >>>> WEBREV: http://cr.openjdk.java.net/~igerasim/6853696/0/webrev/ >>>> >>>> Sincerely yours, >>>> Ivan >> >> > From brian.burkhalter at oracle.com Tue Feb 25 20:36:10 2014 From: brian.burkhalter at oracle.com (Brian Burkhalter) Date: Tue, 25 Feb 2014 12:36:10 -0800 Subject: [REFRESH] Re: JDK 9 RFR of JDK-8035279: Clean up internal deprecations in BigInteger In-Reply-To: <8229AE11-1ABF-474D-961F-BBD5ACA3DD82@oracle.com> References: <5305D6EB.30806@oracle.com> <53064179.2060703@gmail.com> <530660FA.1060407@oracle.com> <96CF3236-4E65-4D64-BCF3-1E9660D29BAA@oracle.com> <5306BC4B.4090909@oracle.com> <8229AE11-1ABF-474D-961F-BBD5ACA3DD82@oracle.com> Message-ID: <61708919-691D-4EBF-8685-0CC5FC309701@oracle.com> On Feb 25, 2014, at 4:26 AM, Paul Sandoz wrote: > Might as well just remove the @Deprecated stuff. I think it is fine under the circumstances to have offsets and getter methods that return the correct values. I have posted a new webrev taking this approach: http://cr.openjdk.java.net/~bpb/8035279/webrev.01/ A review would be appreciated. Thanks, Brian From brian.burkhalter at oracle.com Tue Feb 25 20:38:16 2014 From: brian.burkhalter at oracle.com (Brian Burkhalter) Date: Tue, 25 Feb 2014 12:38:16 -0800 Subject: JDK 9 RFR of JDK-8035279: Clean up internal deprecations in BigInteger In-Reply-To: <264E7F8B-E421-4D91-88D9-A22EF0366F2A@oracle.com> References: <5305B914.7000901@oracle.com> <264E7F8B-E421-4D91-88D9-A22EF0366F2A@oracle.com> Message-ID: <5C039E72-638F-494D-8A78-96ADEFFC267B@oracle.com> On Feb 20, 2014, at 1:42 AM, Paul Sandoz wrote: > Not sure the static powerCache field, in the original code, needs to be volatile either: > > 1137 private static volatile BigInteger[][] powerCache; Is there consensus on whether "volatile" is necessary here? Thanks, Brian From stuart.marks at oracle.com Tue Feb 25 21:29:41 2014 From: stuart.marks at oracle.com (Stuart Marks) Date: Tue, 25 Feb 2014 13:29:41 -0800 Subject: JDK 9 RFR of JDK-8035452: Fix serial lint warnings in core libs In-Reply-To: References: <530666A5.8050607@oracle.com> <530C1A87.2070608@oracle.com> <530C3DF5.3040707@oracle.com> Message-ID: <530D0B45.3080609@oracle.com> Hi Martin, Yes, um, I agree. :-) Perhaps theere is some hope here: http://openjdk.java.net/jeps/187 Which of course will avoid all the mistakes in the first version of serialization. That will create room for bigger and better mistakes. :-) And then of course there is this: http://openjdk.java.net/jeps/154 s'marks On 2/25/14 10:33 AM, Martin Buchholz wrote: > (As I've said before ...) > > Serialization is too hard to get right. You need a full-time engineer (Peter > Jones used to be that) who specializes in serialization and builds tools to > ensure that serialization is implemented correctly, e.g. that internal > implementation details don't leak out and that cross-version and > cross-implementation tests are written so that the promise of serialization is > not broken by careless JDK engineers (no one is careful enough!). Our testing > culture and tools tend to assume only a single JDK at a time. > > The default serialization tries to be helpful, but too often exposes > implementation details. It would have been better to force everyone to write > those writeObject methods. It would have been better if most Collection > implementations serialized the same way, an int size followed by all the > elements in iterator order. There should have been a utility method to help > do that. Compare and contrast with the reliable "serialization" provided by > Collection.toString. > > > On Mon, Feb 24, 2014 at 10:53 PM, Stuart Marks > wrote: > > On 2/24/14 8:22 PM, Joe Darcy wrote: > > On 02/20/2014 12:49 PM, Paul Benedict wrote: > > Joe, I find it interesting that you suppressed the serial warning > on an > abstract class. I'd like to know more about that. Is this a > universal rule? > Are serial uids not important for abstract classes? > > > I wouldn't generalize that way from this example. > > The serial hash of NavigableSubMap has differed in different JDK releases, > but its subclasses do define serialVersionUID fields. I assume the set of > non-transient fields in NavigableSubMap has stayed unchanged, but I > haven't > verified that. > > If I hadn't found the change in the hash value, I would have added the > serialVersionUID to NavigableSubMap too. > > > And in his reply to Paul, Joe said: > > From what I was able to discern by reading the serialization specification > [1], If a class does *not* declare a serialVersionUID, the > serialVersionUID > of its superclass is *not* included in the serialver hash computation > of the > child class. However, my understanding is that changes to the fields > stored > in superclass and changes to the semantics of its readObject / > writeObjects > methods could affect the serialization of the child class. > > > I think we need to take a closer look at these issues. > > I believe that abstract, serializable superclasses *do* need to have a > serialVersionUID defined. The reason is that when a subclass is > serialized, its superclass descriptor (an ObjectStreamClass) is also > serialized. Upon deserialization, the descriptor's svuid is matched > against the svuid of the class loaded at runtime, and if there is a > mismatch, InvalidClassException ensues. > > While the svuid of an abstract superclass isn't included in the subclass' > svuid hash, the svuid of the superclass does affect serial compatibility > of subclasses as described above. Thus, an apparently innocuous change to > the superclass might prevent serial compatibility of its subclasses, no > matter how carefully the subclasses are programmed. > > If the NavigableSubMap class has changed svuid values over several > releases, well, unfortunately we may have a compatibility problem already > in the field. We'd need to choose which release to be compatible with. > Since 8 isn't quite out yet, we might be able to change an early 8-update > and 9 to be compatibile with the latest 7-update. > > Note that the svuid of a class does not relate solely to the fields that > are serialized. It's an attempt at a version hash of the *implementation* > of a class, not a version of the serial protocol. Even changes to a class > that don't affect the serialized output stream can affect the svuid. For > example, renaming a package-private method will affect the svuid. See > section 4.6 of the serialization spec. > > While we're at it (sorry...) in the diffs for your other serial warnings > patch JDK-8035453, there are several lines where the serial warning is > suppressed like so: > > + at SuppressWarnings("serial") // JDK implementation class > > As you know, serialization can expose the private fields of a class, > making them public in a sense. Serialization can also expose what are > internal, implementation classes, if these classes are part of a > serializable object graph that is exposed to applications. I don't know > about the specific situation with the DOM classes, but even if a > serializable class is internal, we might need to be concerned about > serialization compatibility. > > Finally, EnumSet doesn't need a serial version UID. It's serialized using > a proxy class, so EnumSet never appears in a serialized byte stream. > (Note, its readObject throws an exception unconditionally.) So it's > probably safe to suppress its serialization warning. > > s'marks > > > From stuart.marks at oracle.com Tue Feb 25 21:47:15 2014 From: stuart.marks at oracle.com (Stuart Marks) Date: Tue, 25 Feb 2014 13:47:15 -0800 Subject: RFR for JDK-8035388: TEST_BUG: java/rmi/activation/Activatable/checkActivateRef/CheckActivateRef.java fails In-Reply-To: <530C557D.9040003@oracle.com> References: <530C557D.9040003@oracle.com> Message-ID: <530D0F63.90401@oracle.com> On 2/25/14 12:34 AM, Tristan Yan wrote: > Could you please help to review code fix for JDK-8035388. > > http://cr.openjdk.java.net/~tyan/JDK-8035388/webrev.00/ > > Description: > method inactiveObject in ActivationGroupImpl.java, release lock happen before > checkInactiveGroup. This makes groupInactive reset even before next > inactiveObject started. Hi Tristan, Unfortunately the changes to src/share/classes/sun/rmi/server/ActivationGroupImpl.java of the form, } finally { - releaseLock(id); checkInactiveGroup(); + releaseLock(id); } are incorrect. These changes expand the scope of the acquireLock/releaseLock pair to include the call to checkInactiveGroup. The acquireLock/releaseLock methods in this class maintain a list of locked activation IDs in the "lockedIDs" list. The presence of an ID in this list indicates that it's been locked. The problem with having checkInactiveGroup() within the scope of the acquireLock/releaseLock is that checkInactiveGroup has this code: if (active.size() == 0 && lockedIDs.size() == 0 && groupInactive == false) { groupInactive = true; groupMarkedInactive = true; } Thus, if checkInactiveGroup() is called before releaseLock(), the lockedIDs list will always have at least one member, and the condition will always be false. Therefore, this activation group will **never** be made inactive, even if all activated objects within the group become inactive. That's a bug. The test passes because it successfully reactivates an object within the same activation group. However, the intent of the test is to inactivate the group, then reactivate the object, forcing rmid to spawn a new activation group. The changes in this patch don't fulfil this intent. s'marks From ivan.gerasimov at oracle.com Tue Feb 25 21:46:02 2014 From: ivan.gerasimov at oracle.com (Ivan Gerasimov) Date: Wed, 26 Feb 2014 01:46:02 +0400 Subject: RFR [6853696] (ref) ReferenceQueue.remove(timeout) may return null even if timeout has not expired In-Reply-To: <530CD8B5.90000@oracle.com> References: <530ADFB2.8000100@oracle.com> <530BC37F.2050908@oracle.com> <530CA986.9090605@oracle.com> <530CD8B5.90000@oracle.com> Message-ID: <530D0F1A.2050503@oracle.com> Thank you Mandy! On 25.02.2014 21:53, Mandy Chung wrote: > On 2/25/14 6:32 AM, Ivan Gerasimov wrote: >>> line 76: why do you want to catch InterruptedException? If >>> interrupted, should the test fail? >> ReferenceQueue#remove() can throw InterruptedException, so I had to >> handle it. >> In the new webrev I set the initial value of actual to TIMEOUT, so if >> the thread is interrupted the test will pass. >> > > Catching the InterruptedException would hide any unexpected issue as > the test doesn't really expect to be interrupted. For example if the > test is interrupted (kill -9), we should let the test to fail which is > fine. I suggest not to catch it. > Ok. I changed the code to throw RuntimeException. > line 61: I think the test should be: > if (thread.reference != null || thread.actual < TIMEOUT) > Sorry, I'm not clear why. We have two threads: 1) The lucky one gets non-null reference when it calls remove(). For this thread the actual time it had spent on waiting may be much less than 1 sec timeout. 2) The one which receives null from remove(). The amount of time it should have waited before returning from remove() should not be less than timeout. That's what we should check here: if the thread is not the lucky one (reference == null), we make sure it spent whole second waiting in remove(). Please find the slightly updated version of webrev here: http://cr.openjdk.java.net/~igerasim/6853696/2/webrev/ I didn't change the if () clause, as I'm not yet sure why it should be done. Sincerely yours, Ivan > you may want to update the exception. You could simply throw > RuntimeException (just a minor point). > >> Please take a look at the updated webrev: >> http://cr.openjdk.java.net/~igerasim/6853696/1/webrev/ > > Looks okay. I have a slight preference to keep the simple division to > convert to millis rather than loading the enum TimeUnit class for this > purpose. > > Mandy > > From ivan.gerasimov at oracle.com Tue Feb 25 21:46:45 2014 From: ivan.gerasimov at oracle.com (Ivan Gerasimov) Date: Wed, 26 Feb 2014 01:46:45 +0400 Subject: RFR [6853696] (ref) ReferenceQueue.remove(timeout) may return null even if timeout has not expired In-Reply-To: <5DE1FEC2-4EC4-4A5C-8214-DB235639E9EB@oracle.com> References: <530ADFB2.8000100@oracle.com> <530B5929.7040907@oracle.com> <41CC73D0-1754-4A62-A90B-03AD2F372CBC@oracle.com> <530CA723.5040706@oracle.com> <5DE1FEC2-4EC4-4A5C-8214-DB235639E9EB@oracle.com> Message-ID: <530D0F45.8000106@oracle.com> Thanks you Mike! On 25.02.2014 22:47, Mike Duigou wrote: > Looks OK. Some minor comments: > > - Some of the static fields in the test could be final (queue, weakReference, startedSignal). Done. > - After the join() loop in the test you could check that the weakReference is cleared and enqueued. I added the check the weakReference is cleared, but it's not enqueued. When the reference is removed from the queue, its queue member is set to NULL. And checking that the reference is enqueued is actually making sure that (this.queue == ReferenceQueue.ENQUEUED), which doesn't hold. > - Why is the check thread.actual < TIMEOUT only done if thread.reference is null? This would seem to be appropriate for both cases. (I like Mandy's suggestion of using || on the conditions). Sorry, I don't understand why. As I wrote in the reply to Mandy: We have two threads: 1) The lucky one gets non-null reference when it calls remove(). For this thread the actual time it had spent on waiting may be much less than 1 sec timeout. 2) The one which receives null from remove(). The amount of time it should have waited before returning from remove() should not be less than timeout. That's what we should check here: if the thread is not the lucky one (reference == null), we make sure it spent whole second waiting in remove(). > - I agree with Mandy about throwing RuntimeException instead of Exception. Done. BTW. My IDE suggests that a private field ReferenceQueue.queueLength is initialized, but never used. Wouldn't it be appropriate to remove it with this change? Sincerely yours, Ivan > Mike > > On Feb 25 2014, at 06:22 , Ivan Gerasimov wrote: > >> Thank you Mike! >> >> On 24.02.2014 22:26, Mike Duigou wrote: >>> On Feb 24 2014, at 06:37 , roger riggs wrote: >>> >>>> Hi Ivan, >>>> >>>> The code is correct as written but there might be some creep in the end time due to the sampling of System.nanoTime. >>>> >>>> I would be inclined to calculate the final time of the timeout once >>>> and then compare simply with the current nanotime. >>>> >>>> long end = (timeout == 0) ? Long.MAX_VALUE : (System.nanoTime() + timeout * 1000000); >>> I hate seeing numerical constants >>> >>> TimeUnit.MILLISECONDS.toNanos(timeout) >> Yes, this is much clearer. >> Though I used the opposite conversion: NANOSECONDS.toMillis(end - start); >> >> Would you please take a look at the updated webrev: >> http://cr.openjdk.java.net/~igerasim/6853696/1/webrev/ >> >> Sincerely yours, >> Ivan >> >>>> Then the test in the loop can be: >>>> >>>> if (System.nanoTime() > end) { >>>> return null; >>>> } >>> This compare should be re-written in the overflow compensating style Martin mentions. >>> >>> Mike >>> >>>> Roger (Not a Reviewer) >>>> >>>> On 2/24/2014 12:59 AM, Ivan Gerasimov wrote: >>>>> Hello! >>>>> >>>>> ReferenceQueue.remove(timeout) may return too early, i.e. before the specified timeout has elapsed. >>>>> >>>>> Would you please review the fix? >>>>> The change also includes a regression test, which can be used to demonstrate the issue. >>>>> >>>>> BUGURL: https://bugs.openjdk.java.net/browse/JDK-6853696 >>>>> WEBREV: http://cr.openjdk.java.net/~igerasim/6853696/0/webrev/ >>>>> >>>>> Sincerely yours, >>>>> Ivan >>> > > From brent.christian at oracle.com Tue Feb 25 22:15:28 2014 From: brent.christian at oracle.com (Brent Christian) Date: Tue, 25 Feb 2014 14:15:28 -0800 Subject: RFR 8027640 : String.indexOf(String, int) for the empty string case not specified In-Reply-To: References: <530BDB37.40603@oracle.com> Message-ID: <530D1600.5070405@oracle.com> On 2/25/14 4:41 AM, Paul Sandoz wrote: > On Feb 25, 2014, at 12:52 AM, Brent Christian wrote: > >> Please review my changes for: >> https://bugs.openjdk.java.net/browse/JDK-8027640 >> >> The webrev is here: >> http://cr.openjdk.java.net/~bchristi/8027640/webrev.00/ >> > > In AbstractStringBuilder there is a small typo, "k<": > > 1348 *

The returned index is the largest value {@code k<} for which: > Great catch, Paul. Thanks! -Brent From vladimir.x.ivanov at oracle.com Wed Feb 26 00:16:04 2014 From: vladimir.x.ivanov at oracle.com (Vladimir Ivanov) Date: Wed, 26 Feb 2014 04:16:04 +0400 Subject: RFR (S): 8033666: Make sure @ForceInline is everywhere it needs to be in sun.misc and java.lang.invoke Message-ID: <530D3244.6080702@oracle.com> http://cr.openjdk.java.net/~vlivanov/8033666/webrev.01/ https://bugs.openjdk.java.net/browse/JDK-8033666 ValueConversions::castReference was introduced to workaround a problem with unreliable inlining of Class::cast method. Unfortunately, since ValueConversions class is located in sun.invoke.util, @java.lang.invoke.ForceInline annotation isn't visible to it. A proper fix would be to teach Hotspot to treat Class::cast specifically and always inline it, but it requires extensive exploration of performance implications. Filed 8035809 [1] to track that. As an interim fix, I moved castReference method into java.lang.invoke.MethodHandleImpl class and added new entry point ValueConversions::cast which accepts a method handle to a method which should be used for casting. Testing: manual (looked through -XX:+PrintInlining output to ensure MHI::castReference is inlined), jdk/java/lang/invoke, octane. Thanks! Best regards, Vladimir Ivanov [1] 8035809: Improve inlining of Class::cast method https://bugs.openjdk.java.net/browse/JDK-8035809 From christian.thalinger at oracle.com Wed Feb 26 01:17:30 2014 From: christian.thalinger at oracle.com (Christian Thalinger) Date: Tue, 25 Feb 2014 17:17:30 -0800 Subject: RFR (S): 8033666: Make sure @ForceInline is everywhere it needs to be in sun.misc and java.lang.invoke In-Reply-To: <530D3244.6080702@oracle.com> References: <530D3244.6080702@oracle.com> Message-ID: <46FA2D40-37EB-4C7E-A6A6-78ECDEDB8145@oracle.com> Looks good. While touching the code some time ago John and I were playing around with the idea to call Class.cast in the failing case: + static T castReference(Class t, U x) { + // inlined Class.cast because we can't ForceInline it + if (x != null && !t.isInstance(x)) + t.cast(x); + return (T) x; + } + Then we don?t have to duplicate the throwing logic. On Feb 25, 2014, at 4:16 PM, Vladimir Ivanov wrote: > http://cr.openjdk.java.net/~vlivanov/8033666/webrev.01/ > https://bugs.openjdk.java.net/browse/JDK-8033666 > > ValueConversions::castReference was introduced to workaround a problem > with unreliable inlining of Class::cast method. Unfortunately, since > ValueConversions class is located in sun.invoke.util, > @java.lang.invoke.ForceInline annotation isn't visible to it. > > A proper fix would be to teach Hotspot to treat Class::cast specifically > and always inline it, but it requires extensive exploration of > performance implications. Filed 8035809 [1] to track that. > > As an interim fix, I moved castReference method into > java.lang.invoke.MethodHandleImpl class and added new entry point > ValueConversions::cast which accepts a method handle to a method which > should be used for casting. > > Testing: manual (looked through -XX:+PrintInlining output to ensure > MHI::castReference is inlined), jdk/java/lang/invoke, octane. > > Thanks! > > Best regards, > Vladimir Ivanov > > [1] 8035809: Improve inlining of Class::cast method > https://bugs.openjdk.java.net/browse/JDK-8035809 > > _______________________________________________ > mlvm-dev mailing list > mlvm-dev at openjdk.java.net > http://mail.openjdk.java.net/mailman/listinfo/mlvm-dev From john.r.rose at oracle.com Wed Feb 26 01:32:56 2014 From: john.r.rose at oracle.com (John Rose) Date: Tue, 25 Feb 2014 17:32:56 -0800 Subject: RFR (S): 8033666: Make sure @ForceInline is everywhere it needs to be in sun.misc and java.lang.invoke In-Reply-To: <46FA2D40-37EB-4C7E-A6A6-78ECDEDB8145@oracle.com> References: <530D3244.6080702@oracle.com> <46FA2D40-37EB-4C7E-A6A6-78ECDEDB8145@oracle.com> Message-ID: On Feb 25, 2014, at 5:17 PM, Christian Thalinger wrote: > While touching the code some time ago John and I were playing around with the idea to call Class.cast in the failing case: > > + static T castReference(Class t, U x) { > + // inlined Class.cast because we can't ForceInline it > + if (x != null && !t.isInstance(x)) > + t.cast(x); > + return (T) x; > + } > + > > Then we don?t have to duplicate the throwing logic. Here's the compromise version I have: static Error castReferenceFail(Object x, Class t) { t.cast(x); throw new AssertionError(); } (a) It doesn't duplicate throw code from Class.cast. (b) It has a real throw in the force-inlined method (not shown), so the optimizer can't fail to cut the path. If the inlining of t.cast _does_ fail, or if the optimizer inlines t.cast but fails to prove that the throw test duplicates the dominating test, then the optimizer will not be able to push type information into x, because it won't be able to prove that the t.cast branch diverges from the main branch: The duplicate throw (in the shown code) is just an extra touch; it is harmless and gives one more hint to the system. This would be overkill which would pass back the seemingly promised exception: static ClassCastException castReferenceFail(Object x, Class t) { try { t.cast(x); } catch (ClassCastException ex) { return ex; } throw new AssertionError(); } As with a method or two in MethodHandleStatics, this helper method would seem to create an exception for the client to immediately throw, but in fact would throw a suitable exception immediately. ? John From john.r.rose at oracle.com Wed Feb 26 01:34:40 2014 From: john.r.rose at oracle.com (John Rose) Date: Tue, 25 Feb 2014 17:34:40 -0800 Subject: RFR (S): 8033666: Make sure @ForceInline is everywhere it needs to be in sun.misc and java.lang.invoke In-Reply-To: <530D3244.6080702@oracle.com> References: <530D3244.6080702@oracle.com> Message-ID: <2820A518-F2FD-437D-BA14-72645EC23D52@oracle.com> On Feb 25, 2014, at 4:16 PM, Vladimir Ivanov wrote: > As an interim fix, I moved castReference method into > java.lang.invoke.MethodHandleImpl class and added new entry point > ValueConversions::cast which accepts a method handle to a method which > should be used for casting. P.S. Reviewed! Ship it. From david.holmes at oracle.com Wed Feb 26 04:44:17 2014 From: david.holmes at oracle.com (David Holmes) Date: Wed, 26 Feb 2014 14:44:17 +1000 Subject: RFR [6853696] (ref) ReferenceQueue.remove(timeout) may return null even if timeout has not expired In-Reply-To: <530D0F1A.2050503@oracle.com> References: <530ADFB2.8000100@oracle.com> <530BC37F.2050908@oracle.com> <530CA986.9090605@oracle.com> <530CD8B5.90000@oracle.com> <530D0F1A.2050503@oracle.com> Message-ID: <530D7121.4050700@oracle.com> Hi Ivan, 143 long start = (timeout == 0) ? 0 : System.nanoTime(); This can simply be: 143 long start = System.nanoTime(); If timeout==0 then you will never execute the code that even looks at start. Cheers, David On 26/02/2014 7:46 AM, Ivan Gerasimov wrote: > Thank you Mandy! > > On 25.02.2014 21:53, Mandy Chung wrote: >> On 2/25/14 6:32 AM, Ivan Gerasimov wrote: >>>> line 76: why do you want to catch InterruptedException? If >>>> interrupted, should the test fail? >>> ReferenceQueue#remove() can throw InterruptedException, so I had to >>> handle it. >>> In the new webrev I set the initial value of actual to TIMEOUT, so if >>> the thread is interrupted the test will pass. >>> >> >> Catching the InterruptedException would hide any unexpected issue as >> the test doesn't really expect to be interrupted. For example if the >> test is interrupted (kill -9), we should let the test to fail which is >> fine. I suggest not to catch it. >> > Ok. I changed the code to throw RuntimeException. > >> line 61: I think the test should be: >> if (thread.reference != null || thread.actual < TIMEOUT) >> > Sorry, I'm not clear why. > We have two threads: > 1) The lucky one gets non-null reference when it calls remove(). For > this thread the actual time it had spent on waiting may be much less > than 1 sec timeout. > 2) The one which receives null from remove(). The amount of time it > should have waited before returning from remove() should not be less > than timeout. > > That's what we should check here: > if the thread is not the lucky one (reference == null), we make sure it > spent whole second waiting in remove(). > > Please find the slightly updated version of webrev here: > http://cr.openjdk.java.net/~igerasim/6853696/2/webrev/ > I didn't change the if () clause, as I'm not yet sure why it should be > done. > > Sincerely yours, > Ivan > >> you may want to update the exception. You could simply throw >> RuntimeException (just a minor point). >> >>> Please take a look at the updated webrev: >>> http://cr.openjdk.java.net/~igerasim/6853696/1/webrev/ >> >> Looks okay. I have a slight preference to keep the simple division to >> convert to millis rather than loading the enum TimeUnit class for this >> purpose. >> >> Mandy >> >> > From ivan.gerasimov at oracle.com Wed Feb 26 05:47:59 2014 From: ivan.gerasimov at oracle.com (Ivan Gerasimov) Date: Wed, 26 Feb 2014 09:47:59 +0400 Subject: RFR [6853696] (ref) ReferenceQueue.remove(timeout) may return null even if timeout has not expired In-Reply-To: <530D7121.4050700@oracle.com> References: <530ADFB2.8000100@oracle.com> <530BC37F.2050908@oracle.com> <530CA986.9090605@oracle.com> <530CD8B5.90000@oracle.com> <530D0F1A.2050503@oracle.com> <530D7121.4050700@oracle.com> Message-ID: <530D800F.5040302@oracle.com> Thanks David! On 26.02.2014 8:44, David Holmes wrote: > Hi Ivan, > > 143 long start = (timeout == 0) ? 0 : System.nanoTime(); > > This can simply be: > > 143 long start = System.nanoTime(); > > If timeout==0 then you will never execute the code that even looks at > start. > That's right. System.nanoTime() isn't too expensive, but it's still a native function which may end up with a syscall, so I tried to avoid calling it unless necessary. Sincerely yours, Ivan > Cheers, > David > > On 26/02/2014 7:46 AM, Ivan Gerasimov wrote: >> Thank you Mandy! >> >> On 25.02.2014 21:53, Mandy Chung wrote: >>> On 2/25/14 6:32 AM, Ivan Gerasimov wrote: >>>>> line 76: why do you want to catch InterruptedException? If >>>>> interrupted, should the test fail? >>>> ReferenceQueue#remove() can throw InterruptedException, so I had to >>>> handle it. >>>> In the new webrev I set the initial value of actual to TIMEOUT, so if >>>> the thread is interrupted the test will pass. >>>> >>> >>> Catching the InterruptedException would hide any unexpected issue as >>> the test doesn't really expect to be interrupted. For example if the >>> test is interrupted (kill -9), we should let the test to fail which is >>> fine. I suggest not to catch it. >>> >> Ok. I changed the code to throw RuntimeException. >> >>> line 61: I think the test should be: >>> if (thread.reference != null || thread.actual < TIMEOUT) >>> >> Sorry, I'm not clear why. >> We have two threads: >> 1) The lucky one gets non-null reference when it calls remove(). For >> this thread the actual time it had spent on waiting may be much less >> than 1 sec timeout. >> 2) The one which receives null from remove(). The amount of time it >> should have waited before returning from remove() should not be less >> than timeout. >> >> That's what we should check here: >> if the thread is not the lucky one (reference == null), we make sure it >> spent whole second waiting in remove(). >> >> Please find the slightly updated version of webrev here: >> http://cr.openjdk.java.net/~igerasim/6853696/2/webrev/ >> I didn't change the if () clause, as I'm not yet sure why it should be >> done. >> >> Sincerely yours, >> Ivan >> >>> you may want to update the exception. You could simply throw >>> RuntimeException (just a minor point). >>> >>>> Please take a look at the updated webrev: >>>> http://cr.openjdk.java.net/~igerasim/6853696/1/webrev/ >>> >>> Looks okay. I have a slight preference to keep the simple division to >>> convert to millis rather than loading the enum TimeUnit class for this >>> purpose. >>> >>> Mandy >>> >>> >> > From mandy.chung at oracle.com Wed Feb 26 06:19:29 2014 From: mandy.chung at oracle.com (Mandy Chung) Date: Tue, 25 Feb 2014 22:19:29 -0800 Subject: RFR [6853696] (ref) ReferenceQueue.remove(timeout) may return null even if timeout has not expired In-Reply-To: <530D0F1A.2050503@oracle.com> References: <530ADFB2.8000100@oracle.com> <530BC37F.2050908@oracle.com> <530CA986.9090605@oracle.com> <530CD8B5.90000@oracle.com> <530D0F1A.2050503@oracle.com> Message-ID: <530D8771.1020407@oracle.com> On 2/25/2014 1:46 PM, Ivan Gerasimov wrote: > >> line 61: I think the test should be: >> if (thread.reference != null || thread.actual < TIMEOUT) >> > Sorry, I'm not clear why. > We have two threads: > 1) The lucky one gets non-null reference when it calls remove(). For > this thread the actual time it had spent on waiting may be much less > than 1 sec timeout. > 2) The one which receives null from remove(). The amount of time it > should have waited before returning from remove() should not be less > than timeout. > I missed that you remove the strong reference (line 57). I think it's good to hold the strong reference so that ReferenceQueue.remove(timeout) will timeout and the test can verify reliably. Perhaps once you check the EarlyTimeout threads (both should timeout with null reference), you can clear the referent and call System.gc() and then verify if queue.remove(longTimeout) should not block as the reference should be enqueued for removal. Mandy > That's what we should check here: > if the thread is not the lucky one (reference == null), we make sure > it spent whole second waiting in remove(). > > Please find the slightly updated version of webrev here: > http://cr.openjdk.java.net/~igerasim/6853696/2/webrev/ > I didn't change the if () clause, as I'm not yet sure why it should be > done. From ivan.gerasimov at oracle.com Wed Feb 26 07:08:07 2014 From: ivan.gerasimov at oracle.com (Ivan Gerasimov) Date: Wed, 26 Feb 2014 11:08:07 +0400 Subject: RFR [6853696] (ref) ReferenceQueue.remove(timeout) may return null even if timeout has not expired In-Reply-To: <530D8771.1020407@oracle.com> References: <530ADFB2.8000100@oracle.com> <530BC37F.2050908@oracle.com> <530CA986.9090605@oracle.com> <530CD8B5.90000@oracle.com> <530D0F1A.2050503@oracle.com> <530D8771.1020407@oracle.com> Message-ID: <530D92D7.2060304@oracle.com> On 26.02.2014 10:19, Mandy Chung wrote: > > On 2/25/2014 1:46 PM, Ivan Gerasimov wrote: >> >>> line 61: I think the test should be: >>> if (thread.reference != null || thread.actual < TIMEOUT) >>> >> Sorry, I'm not clear why. >> We have two threads: >> 1) The lucky one gets non-null reference when it calls remove(). For >> this thread the actual time it had spent on waiting may be much less >> than 1 sec timeout. >> 2) The one which receives null from remove(). The amount of time it >> should have waited before returning from remove() should not be less >> than timeout. >> > > I missed that you remove the strong reference (line 57). I think > it's good to hold the strong reference so that > ReferenceQueue.remove(timeout) will timeout and the test can verify > reliably. > This is an important part. If we didn't remove the strong reference then both threads would wait for the specified period of time even without the fix. The point is to make both threads wake up from lock.wait(timeout), and we do it by removing the strong reference and forcing it be enqueued. Sincerely yours, Ivan > Perhaps once you check the EarlyTimeout threads (both should timeout > with null reference), you can clear the referent and call System.gc() > and then verify if queue.remove(longTimeout) should not block as the > reference should be enqueued for removal. > > Mandy > >> That's what we should check here: >> if the thread is not the lucky one (reference == null), we make sure >> it spent whole second waiting in remove(). >> >> Please find the slightly updated version of webrev here: >> http://cr.openjdk.java.net/~igerasim/6853696/2/webrev/ >> I didn't change the if () clause, as I'm not yet sure why it should >> be done. > > > From vladimir.x.ivanov at oracle.com Wed Feb 26 08:06:56 2014 From: vladimir.x.ivanov at oracle.com (Vladimir Ivanov) Date: Wed, 26 Feb 2014 12:06:56 +0400 Subject: RFR (S): 8033666: Make sure @ForceInline is everywhere it needs to be in sun.misc and java.lang.invoke In-Reply-To: <2820A518-F2FD-437D-BA14-72645EC23D52@oracle.com> References: <530D3244.6080702@oracle.com> <2820A518-F2FD-437D-BA14-72645EC23D52@oracle.com> Message-ID: <530DA0A0.4050005@oracle.com> John, Chris, thanks for review! Best regards, Vladimir Ivanov On 2/26/14 5:34 AM, John Rose wrote: > On Feb 25, 2014, at 4:16 PM, Vladimir Ivanov wrote: > >> As an interim fix, I moved castReference method into >> java.lang.invoke.MethodHandleImpl class and added new entry point >> ValueConversions::cast which accepts a method handle to a method which >> should be used for casting. > > P.S. Reviewed! Ship it. > From paul.sandoz at oracle.com Wed Feb 26 10:15:51 2014 From: paul.sandoz at oracle.com (Paul Sandoz) Date: Wed, 26 Feb 2014 11:15:51 +0100 Subject: [REFRESH] JDK 9 RFR of JDK-8035279: Clean up internal deprecations in BigInteger In-Reply-To: <61708919-691D-4EBF-8685-0CC5FC309701@oracle.com> References: <5305D6EB.30806@oracle.com> <53064179.2060703@gmail.com> <530660FA.1060407@oracle.com> <96CF3236-4E65-4D64-BCF3-1E9660D29BAA@oracle.com> <5306BC4B.4090909@oracle.com> <8229AE11-1ABF-474D-961F-BBD5ACA3DD82@oracle.com> <61708919-691D-4EBF-8685-0CC5FC309701@oracle.com> Message-ID: On Feb 25, 2014, at 9:36 PM, Brian Burkhalter wrote: > > On Feb 25, 2014, at 4:26 AM, Paul Sandoz wrote: > >> Might as well just remove the @Deprecated stuff. I think it is fine under the circumstances to have offsets and getter methods that return the correct values. > > I have posted a new webrev taking this approach: > > http://cr.openjdk.java.net/~bpb/8035279/webrev.01/ > > A review would be appreciated. > It might be worth refining the comments, replacing: 145 // These "redundant fields" are initialized with recognizable nonsense 146 // values, and cached the first time they are needed (or never, if they 147 // aren't needed). with: // The following fields are stable variables. A stable variable's value changes // at most once from the default zero value to a non-zero stable value. // A stable value is calculated lazily on demand. // /** * One plus the bitCount of this BigInteger. This is a stable variable. ... /** * One plus the bitLength of this BigInteger. This is a stable variable. ... /** * Two plus the lowest set bit of this BigInteger. This is a stable variable. Also, is @serial is relevant any more? Plus to be pedantic for firstNonzeroIntNumPlusTwo with zero magnitudes: /** * Two plus the index of the lowest-order int in the magnitude of this * BigInteger that contains a nonzero int. This is a stable variable. * The least significant int has int-number 0, the next int in order of * increasing significance has int-number 1, and so forth. * *

Note: never used for a BigInteger with a magnitude of zero. * @see #firstNonzeroIntNum() /** * Returns the index of the int that contains the first nonzero int in the * little-endian binary representation of the magnitude (int 0 is the * least significant). If the magnitude is zero, return value is undefined. * *

Note: never used for a BigInteger with a magnitude of zero. * @see #getInt. */ private int firstNonzeroIntNum() { int fn = firstNonzeroIntNumPlusTwo - 2; if (fn == -3) { // firstNonzeroIntNum not initialized yet // Search for the first nonzero int int i; int mlen = mag.length; for (i = mlen - 1; i >= 0 && mag[i] == 0; i--) ; fn = mlen - i - 1; firstNonzeroIntNumPlusTwo = fn + 2; // offset by two to initialize } return fn; } An unnecessary assignment is removed from the above method. -- We can revisit and clean things up if/when the JMM is updated. Paul. From peter.levart at gmail.com Wed Feb 26 12:56:18 2014 From: peter.levart at gmail.com (Peter Levart) Date: Wed, 26 Feb 2014 13:56:18 +0100 Subject: JDK 9 RFR of JDK-8035279: Clean up internal deprecations in BigInteger In-Reply-To: <5C039E72-638F-494D-8A78-96ADEFFC267B@oracle.com> References: <5305B914.7000901@oracle.com> <264E7F8B-E421-4D91-88D9-A22EF0366F2A@oracle.com> <5C039E72-638F-494D-8A78-96ADEFFC267B@oracle.com> Message-ID: <530DE472.80405@gmail.com> On 02/25/2014 09:38 PM, Brian Burkhalter wrote: > On Feb 20, 2014, at 1:42 AM, Paul Sandoz wrote: > >> Not sure the static powerCache field, in the original code, needs to be volatile either: >> >> 1137 private static volatile BigInteger[][] powerCache; > Is there consensus on whether "volatile" is necessary here? I think it has to be volatile. The powerCache implementation was added in the following changeset: http://hg.openjdk.java.net/jdk9/dev/jdk/rev/7546 ...and improved later in the following: http://hg.openjdk.java.net/jdk9/dev/jdk/rev/7586 It uses a copy-on-write technique to extend the cache with new values when needed. volatile is mandatory here to safely publish the newly constructed array-of-arrays and the newly constructed sub-array to other threads. Without volatile, other threads could see null slots where BigInteger[] and/or BigInteger objects should be... Regards, Peter > > Thanks, > > Brian From paul.sandoz at oracle.com Wed Feb 26 13:38:18 2014 From: paul.sandoz at oracle.com (Paul Sandoz) Date: Wed, 26 Feb 2014 14:38:18 +0100 Subject: JDK 9 RFR of JDK-8035279: Clean up internal deprecations in BigInteger In-Reply-To: <5C039E72-638F-494D-8A78-96ADEFFC267B@oracle.com> References: <5305B914.7000901@oracle.com> <264E7F8B-E421-4D91-88D9-A22EF0366F2A@oracle.com> <5C039E72-638F-494D-8A78-96ADEFFC267B@oracle.com> Message-ID: <0D061461-C44C-4EBB-8848-E0647C374A34@oracle.com> On Feb 25, 2014, at 9:38 PM, Brian Burkhalter wrote: > > On Feb 20, 2014, at 1:42 AM, Paul Sandoz wrote: > >> Not sure the static powerCache field, in the original code, needs to be volatile either: >> >> 1137 private static volatile BigInteger[][] powerCache; > > Is there consensus on whether "volatile" is necessary here? > Looking back at the discussions i believe it was made volatile to ensure threads don't observe a partially updated and published cache lines. Since we are already using Unsafe for deserialization I think it might be possible to do the following instead (warning: not tested!): /** * The cache of powers of each radix. This allows us to not have to * recalculate powers of radix^(2^n) more than once. This speeds * Schoenhage recursive base conversion significantly. */ private static final BigInteger[][] powerCache; ... private static BigInteger getRadixConversionCache(int radix, int exponent) { // Relaxed read of cache line from power cache BigInteger[] cacheLine = powerCache[radix]; if (exponent < cacheLine.length) { return cacheLine[exponent]; } // Copy and expand the cache line up to and including the exponent int oldLength = cacheLine.length; cacheLine = Arrays.copyOf(cacheLine, exponent + 1); for (int i = oldLength; i <= exponent; i++) { cacheLine[i] = cacheLine[i - 1].pow(2); } // Lazy write of new cache line to power cache // Ensure all writes to the new cache line are ordered before // it's publication in the power cache UnsafeHolder.lazySetCacheLine(powerCache, radix, cacheLine); return cacheLine[exponent]; } private static class UnsafeHolder { private static final sun.misc.Unsafe unsafe; private static final long signumOffset; private static final long magOffset; private static final int biaOffset; private static final int biaShift; static { try { unsafe = sun.misc.Unsafe.getUnsafe(); signumOffset = unsafe.objectFieldOffset (BigInteger.class.getDeclaredField("signum")); magOffset = unsafe.objectFieldOffset (BigInteger.class.getDeclaredField("mag")); biaOffset = unsafe.arrayBaseOffset(BigInteger[][].class); int scale = unsafe.arrayIndexScale(BigInteger[][].class); if ((scale & (scale - 1)) != 0) throw new Error("data type scale not a power of two"); biaShift = 31 - Integer.numberOfLeadingZeros(scale); } catch (Exception ex) { throw new ExceptionInInitializerError(ex); } } static void putSign(BigInteger bi, int sign) { unsafe.putIntVolatile(bi, signumOffset, sign); } static void putMag(BigInteger bi, int[] magnitude) { unsafe.putObjectVolatile(bi, magOffset, magnitude); } static void lazySetCacheLine(BigInteger[][] cache, int radix, BigInteger[] cacheLine) { unsafe.putOrderedObject(cache, biaOffset + ((long) radix << biaShift), cacheLine); } } Paul. From paul.sandoz at oracle.com Wed Feb 26 15:12:21 2014 From: paul.sandoz at oracle.com (Paul Sandoz) Date: Wed, 26 Feb 2014 16:12:21 +0100 Subject: Unsafe: removing the monitorEnter/monitorExit/tryMonitorEnter methods Message-ID: Hi, Out of all the methods on Unsafe i think the monitorEnter/monitorExit/tryMonitorEnter are the least used and are very strong candidates for removal. 99% of use-cases are supported by classes in the java.util.concurrent.locks package. Within the JDK itself it is only used in one JDK test file test/java/lang/ProcessBuilder/Basic.java: while (unsafe.tryMonitorEnter(s)) { unsafe.monitorExit(s); Thread.sleep(1); } for a test verifying an EOF is received on pending reads and it is polling to check when the process builder acquires the lock before destroying the process, presumably to avoid some sort of race condition that occasionally causes the test to fail. I believe this test as been through a number of rounds, i stared at things for a bit, but cannot quickly work out a replacement; i lack the knowledge on this area. Outside of the JDK i can only find one usage of monitorExit/Enter (according to grep code) in JBoss modules, and i believe this is only used on Java 1.6 to work around some limitations in class loading that were fixed in 1.7. Given such very limited use i propose to remove these methods after having worked out a fix for ProcessBuilder/Basic.java test. Paul. From paul.sandoz at oracle.com Wed Feb 26 15:42:48 2014 From: paul.sandoz at oracle.com (Paul Sandoz) Date: Wed, 26 Feb 2014 16:42:48 +0100 Subject: Unsafe: efficiently comparing two byte arrays Message-ID: Hi, A common reason why Unsafe is used is to more efficiently compare two unsigned byte arrays, viewing those byte arrays as long arrays. See Guava [1] and a number of apache frameworks for similar code. One solution is to provide such functionality in Arrays for all primitives and probably refs [2]: int Arrays.compare(byte[], byte[]); Then it is easy to create a comparator using a method reference: Comparator c = Arrays::compare; There could, initially, be Java implementations for those methods, including using Unsafe for byte[]. I gather those methods could be intrinsified to implementations using SIMD instructions on supported platforms. I don't know if that is possible today with Hotspot, but regardless i think a good start would be to have Java-based implementations in place. Thoughts? Paul. [1] http://docs.guava-libraries.googlecode.com/git/javadoc/com/google/common/primitives/UnsignedBytes.html#lexicographicalComparator() [2] https://bugs.openjdk.java.net/browse/JDK-8033148 From xueming.shen at oracle.com Wed Feb 26 16:22:04 2014 From: xueming.shen at oracle.com (Xueming Shen) Date: Wed, 26 Feb 2014 08:22:04 -0800 Subject: RFR JDK-8035814: Broken link in java.nio.StandardCharsets Message-ID: <530E14AC.7000002@oracle.com> Hi, please help review the trivial doc typo fix. https://bugs.openjdk.java.net/browse/JDK-8035814 http://cr.openjdk.java.net/~sherman/8035814/webrev Thanks! -Sherman From sean.coffey at oracle.com Wed Feb 26 16:50:17 2014 From: sean.coffey at oracle.com (=?ISO-8859-1?Q?Se=E1n_Coffey?=) Date: Wed, 26 Feb 2014 16:50:17 +0000 Subject: RFR: 8035618 : Four api/org_omg/CORBA TCK tests fail under plugin only Message-ID: <530E1B49.8080607@oracle.com> Looking to push a fix to JDK 8. A CORBA issue was discovered during TCK-Plugin testing. The NPE seen should be handled and the defaultPresentationManager should be returned where necessary. Bug ID : https://bugs.openjdk.java.net/browse/JDK-8035618 webrev : http://cr.openjdk.java.net/~coffeys/webrev.8035618/webrev/ regards, Sean. From martinrb at google.com Wed Feb 26 17:21:40 2014 From: martinrb at google.com (Martin Buchholz) Date: Wed, 26 Feb 2014 09:21:40 -0800 Subject: Unsafe: efficiently comparing two byte arrays In-Reply-To: References: Message-ID: Every once in a while I see an attempt to introduce a "general purpose" unsafe array class, but efforts stall due to: - it's not obvious whether unaligned access to e.g. 8 bytes via long is even possible or more efficient than just reading 8 bytes - endianness is an issue - for best performance, you also want to elide those pesky array bound checks (but hotspot can do a better job of that) I think it's worth doing, but it's harder than it looks, and probably needs help from the VM via intrinsics. --- lexicographicalComparator saves a lot of cycles. On Wed, Feb 26, 2014 at 7:42 AM, Paul Sandoz wrote: > Hi, > > A common reason why Unsafe is used is to more efficiently compare two > unsigned byte arrays, viewing those byte arrays as long arrays. See Guava > [1] and a number of apache frameworks for similar code. > > One solution is to provide such functionality in Arrays for all primitives > and probably refs [2]: > > int Arrays.compare(byte[], byte[]); > > Then it is easy to create a comparator using a method reference: > > Comparator c = Arrays::compare; > > There could, initially, be Java implementations for those methods, > including using Unsafe for byte[]. I gather those methods could be > intrinsified to implementations using SIMD instructions on supported > platforms. I don't know if that is possible today with Hotspot, but > regardless i think a good start would be to have Java-based implementations > in place. > > Thoughts? > > Paul. > > [1] > http://docs.guava-libraries.googlecode.com/git/javadoc/com/google/common/primitives/UnsignedBytes.html#lexicographicalComparator() > > [2] https://bugs.openjdk.java.net/browse/JDK-8033148 > > From roger.riggs at oracle.com Wed Feb 26 18:07:27 2014 From: roger.riggs at oracle.com (roger riggs) Date: Wed, 26 Feb 2014 13:07:27 -0500 Subject: RFR JDK-8035814: Broken link in java.nio.StandardCharsets In-Reply-To: <530E14AC.7000002@oracle.com> References: <530E14AC.7000002@oracle.com> Message-ID: <530E2D5F.40004@oracle.com> Looks fine, (not a Reviewer) On 2/26/2014 11:22 AM, Xueming Shen wrote: > Hi, please help review the trivial doc typo fix. > > https://bugs.openjdk.java.net/browse/JDK-8035814 > http://cr.openjdk.java.net/~sherman/8035814/webrev > > Thanks! > -Sherman From brian.goetz at oracle.com Wed Feb 26 18:10:43 2014 From: brian.goetz at oracle.com (Brian Goetz) Date: Wed, 26 Feb 2014 13:10:43 -0500 Subject: Unsafe: removing the monitorEnter/monitorExit/tryMonitorEnter methods In-Reply-To: References: Message-ID: <530E2E23.30506@oracle.com> Seems like a symbolic victory, at least :) It's less unsafe than some unsafe methods (you can use it to create a DoS but not to violate safety constraints like bounds checking or pointer casting) but its a start! On 2/26/2014 10:12 AM, Paul Sandoz wrote: > Hi, > > Out of all the methods on Unsafe i think the monitorEnter/monitorExit/tryMonitorEnter are the least used and are very strong candidates for removal. > > 99% of use-cases are supported by classes in the java.util.concurrent.locks package. > > > Within the JDK itself it is only used in one JDK test file test/java/lang/ProcessBuilder/Basic.java: > > while (unsafe.tryMonitorEnter(s)) { > unsafe.monitorExit(s); > Thread.sleep(1); > } > > for a test verifying an EOF is received on pending reads and it is polling to check when the process builder acquires the lock before destroying the process, presumably to avoid some sort of race condition that occasionally causes the test to fail. > > I believe this test as been through a number of rounds, i stared at things for a bit, but cannot quickly work out a replacement; i lack the knowledge on this area. > > > Outside of the JDK i can only find one usage of monitorExit/Enter (according to grep code) in JBoss modules, and i believe this is only used on Java 1.6 to work around some limitations in class loading that were fixed in 1.7. > > > Given such very limited use i propose to remove these methods after having worked out a fix for ProcessBuilder/Basic.java test. > > Paul. > From Alan.Bateman at oracle.com Wed Feb 26 18:12:39 2014 From: Alan.Bateman at oracle.com (Alan Bateman) Date: Wed, 26 Feb 2014 18:12:39 +0000 Subject: RFR JDK-8035814: Broken link in java.nio.StandardCharsets In-Reply-To: <530E14AC.7000002@oracle.com> References: <530E14AC.7000002@oracle.com> Message-ID: <530E2E97.6030406@oracle.com> On 26/02/2014 16:22, Xueming Shen wrote: > Hi, please help review the trivial doc typo fix. > > https://bugs.openjdk.java.net/browse/JDK-8035814 > http://cr.openjdk.java.net/~sherman/8035814/webrev Looks fine. You might want to correct the package name in the issue synopsis before you create the change-set comment. -Alan. From mandy.chung at oracle.com Wed Feb 26 18:43:54 2014 From: mandy.chung at oracle.com (Mandy Chung) Date: Wed, 26 Feb 2014 10:43:54 -0800 Subject: RFR [6853696] (ref) ReferenceQueue.remove(timeout) may return null even if timeout has not expired In-Reply-To: <530D92D7.2060304@oracle.com> References: <530ADFB2.8000100@oracle.com> <530BC37F.2050908@oracle.com> <530CA986.9090605@oracle.com> <530CD8B5.90000@oracle.com> <530D0F1A.2050503@oracle.com> <530D8771.1020407@oracle.com> <530D92D7.2060304@oracle.com> Message-ID: <530E35EA.2030108@oracle.com> On 2/25/2014 11:08 PM, Ivan Gerasimov wrote: >> I missed that you remove the strong reference (line 57). I think >> it's good to hold the strong reference so that >> ReferenceQueue.remove(timeout) will timeout and the test can verify >> reliably. >> > This is an important part. > If we didn't remove the strong reference then both threads would wait > for the specified period of time even without the fix. > The point is to make both threads wake up from lock.wait(timeout), and > we do it by removing the strong reference and forcing it be enqueued Now it becomes clear what the test does (thanks). It'd be helpful to add some comments. The test should also verify that only one thread gets a non-null reference == weakReference and the other should get a null reference in addition to the wait time > TIMEOUT. Mandy From volker.simonis at gmail.com Wed Feb 26 18:51:29 2014 From: volker.simonis at gmail.com (Volker Simonis) Date: Wed, 26 Feb 2014 19:51:29 +0100 Subject: RFR(S): 8035881: PPC64: Fix AIX build in ppc-aix-port/stage after syncing 7133499 and 8028293 from jdk8u Message-ID: Hi, please review the following small AIX-only change for ppc-aix-port/stage (and eventually jdk8u) which became necessary after we have synced the following two changes from jdk8u: 8028293: Check local configuration for actual ephemeral port range 7133499: (fc) FileChannel.read not preempted by asynchronous close on OS X This change also updates src/share/lib/security/java.security-aix to incorporate the latest changes from the other platforms security files (i.e. java.security-aix is now equal to java.security-linux). http://cr.openjdk.java.net/~simonis/webrevs/8035881/ https://bugs.openjdk.java.net/browse/JDK-8035881 This change is similar to the already reviewed and pushed "8033154: PPC64: Fix AIX build after integration into jdk9/dev" for jdk9/dev except that jdk8u doesn?t have "8032451: (dc) DatagramChannel.join should support include-mode filtering on OS X" until now so the corresponding fixes aren't needed. See the thread http://mail.openjdk.java.net/pipermail/ppc-aix-port-dev/2014-January/001653.html for the previous review. make/lib/NioLibraries.gmk src/aix/native/sun/nio/ch/AixNativeThread.c 7133499 did some changes to NativeThread.c. Instead of adding another platform specific section to that file I've moved the AIX implementation to it's own file which will only be used during the AIX port. Notice that the changes in the makefile are in a AIX-specific section and won't affect any other platform. src/solaris/classes/sun/net/PortConfig.java This is another file which wasn't present in our stage/jdk repository but in the new jdk8u/jdk. Just added the required, AIX-specific section to the file. src/share/lib/security/java.security-aix Recent changes have updated the various src/share/lib/security/java.security* files. Without these changes, the JDK failes to pass the java/lang/SecurityManager/CheckPackageAccess.java jtreg regression test. I've just updated java.security-aix to be the same like the corresponding Linux version (as this was done - and agreed upon that it is OK - in the initial AIX change). Thanks, Volker From Alan.Bateman at oracle.com Wed Feb 26 18:54:39 2014 From: Alan.Bateman at oracle.com (Alan Bateman) Date: Wed, 26 Feb 2014 18:54:39 +0000 Subject: RFR: 8035618 : Four api/org_omg/CORBA TCK tests fail under plugin only In-Reply-To: <530E1B49.8080607@oracle.com> References: <530E1B49.8080607@oracle.com> Message-ID: <530E386F.6090002@oracle.com> On 26/02/2014 16:50, Se?n Coffey wrote: > Looking to push a fix to JDK 8. A CORBA issue was discovered during > TCK-Plugin testing. The NPE seen should be handled and the > defaultPresentationManager should be returned where necessary. > > Bug ID : https://bugs.openjdk.java.net/browse/JDK-8035618 > webrev : http://cr.openjdk.java.net/~coffeys/webrev.8035618/webrev/ The AppContext usage looks okay but I worry that changing this method to synchronized will lead to contention. At least for the non security manager then can this be changed so that the defaultPresentationManager is initialized via a holder class and changed to volatile and initialized via the usual double checked locking idiom. There are other options but I think this one is the important case and should not be synchronized. -Alan. From Alan.Bateman at oracle.com Wed Feb 26 19:00:14 2014 From: Alan.Bateman at oracle.com (Alan Bateman) Date: Wed, 26 Feb 2014 19:00:14 +0000 Subject: RFR(S): 8035881: PPC64: Fix AIX build in ppc-aix-port/stage after syncing 7133499 and 8028293 from jdk8u In-Reply-To: References: Message-ID: <530E39BE.9040904@oracle.com> On 26/02/2014 18:51, Volker Simonis wrote: > Hi, > > please review the following small AIX-only change for > ppc-aix-port/stage (and eventually jdk8u) which became necessary after > we have synced the following two changes from jdk8u: > > 8028293: Check local configuration for actual ephemeral port range > 7133499: (fc) FileChannel.read not preempted by asynchronous close on OS X > > This change also updates src/share/lib/security/java.security-aix to > incorporate the latest changes from the other platforms security files > (i.e. java.security-aix is now equal to java.security-linux). > > http://cr.openjdk.java.net/~simonis/webrevs/8035881/ > https://bugs.openjdk.java.net/browse/JDK-8035881 > This looks okay to me although I have no way to check the default ephemeral port rate on AIX. -Alan. From vladimir.kozlov at oracle.com Wed Feb 26 19:24:14 2014 From: vladimir.kozlov at oracle.com (Vladimir Kozlov) Date: Wed, 26 Feb 2014 11:24:14 -0800 Subject: RFR(S): 8035881: PPC64: Fix AIX build in ppc-aix-port/stage after syncing 7133499 and 8028293 from jdk8u In-Reply-To: <530E39BE.9040904@oracle.com> References: <530E39BE.9040904@oracle.com> Message-ID: <530E3F5E.7000501@oracle.com> I will do JPRT control build and push this fix into ppc-aix-port/stage Thanks, Vladimir On 2/26/14 11:00 AM, Alan Bateman wrote: > On 26/02/2014 18:51, Volker Simonis wrote: >> Hi, >> >> please review the following small AIX-only change for >> ppc-aix-port/stage (and eventually jdk8u) which became necessary after >> we have synced the following two changes from jdk8u: >> >> 8028293: Check local configuration for actual ephemeral port range >> 7133499: (fc) FileChannel.read not preempted by asynchronous close on >> OS X >> >> This change also updates src/share/lib/security/java.security-aix to >> incorporate the latest changes from the other platforms security files >> (i.e. java.security-aix is now equal to java.security-linux). >> >> http://cr.openjdk.java.net/~simonis/webrevs/8035881/ >> https://bugs.openjdk.java.net/browse/JDK-8035881 >> > This looks okay to me although I have no way to check the default > ephemeral port rate on AIX. > > -Alan. > From brent.christian at oracle.com Wed Feb 26 19:34:51 2014 From: brent.christian at oracle.com (Brent Christian) Date: Wed, 26 Feb 2014 11:34:51 -0800 Subject: RFR 6835233 : Fedora 9 jdk regression test failed: java/lang/instrument/ParallelTransformerLoader.sh Message-ID: <530E41DB.20801@oracle.com> File under "chipping away at test stabilization issues." https://bugs.openjdk.java.net/browse/JDK-6835233 I've done some repeated runs of this test on my Linux machine. The test fails every time with 6u3. It fails intermittently on 7 (after 145 iterations for 7u45, and 62 iterations for 7u60b07). I have not been able to reproduce the failure on 8 or 9, running 1000 iterations each on 8b115, 8b129, and 9b02. I would like to resolve this bug by removing the "@ignore" tag for JDK 9, and bring the test back into rotation. If the failure comes back, I'll submit a new issue for further investigation. The change is: # @bug 5088398 -# @ignore until bug 6835233 dealt with # @summary Test parallel class loading by parallel transformers. Thanks, -Brent From brian.burkhalter at oracle.com Wed Feb 26 19:35:04 2014 From: brian.burkhalter at oracle.com (Brian Burkhalter) Date: Wed, 26 Feb 2014 11:35:04 -0800 Subject: [REFRESH] JDK 9 RFR of JDK-8035279: Clean up internal deprecations in BigInteger In-Reply-To: References: <5305D6EB.30806@oracle.com> <53064179.2060703@gmail.com> <530660FA.1060407@oracle.com> <96CF3236-4E65-4D64-BCF3-1E9660D29BAA@oracle.com> <5306BC4B.4090909@oracle.com> <8229AE11-1ABF-474D-961F-BBD5ACA3DD82@oracle.com> <61708919-691D-4EBF-8685-0CC5FC309701@oracle.com> Message-ID: On Feb 26, 2014, at 2:15 AM, Paul Sandoz wrote: >> I have posted a new webrev taking this approach: >> >> http://cr.openjdk.java.net/~bpb/8035279/webrev.01/ >> >> A review would be appreciated. >> Thanks for the suggestions ? > [?] > private int firstNonzeroIntNum() { > int fn = firstNonzeroIntNumPlusTwo - 2; I made all suggested changes except the third line below. Why do we test for equality with -3? If the primitive int default value of zero is used, for firstNonzeroIntNumPlusTwo, as it is, then we should still test whether fn equals -2, n'est-ce pas? > private int firstNonzeroIntNum() { > int fn = firstNonzeroIntNumPlusTwo - 2; > if (fn == -3) { // firstNonzeroIntNum not initialized yet > // Search for the first nonzero int > int i; > int mlen = mag.length; > for (i = mlen - 1; i >= 0 && mag[i] == 0; i--) > ; > fn = mlen - i - 1; > firstNonzeroIntNumPlusTwo = fn + 2; // offset by two to initialize > } > return fn; > } > > An unnecessary assignment is removed from the above method. Yes I noticed that. > -- > > We can revisit and clean things up if/when the JMM is updated. Is there an issue for JMM update to which a comment with a link to this thread could be added? Thanks, Brian From brian.burkhalter at oracle.com Wed Feb 26 19:51:17 2014 From: brian.burkhalter at oracle.com (Brian Burkhalter) Date: Wed, 26 Feb 2014 11:51:17 -0800 Subject: JDK 9 RFR of JDK-8035279: Clean up internal deprecations in BigInteger In-Reply-To: <530DE472.80405@gmail.com> References: <5305B914.7000901@oracle.com> <264E7F8B-E421-4D91-88D9-A22EF0366F2A@oracle.com> <5C039E72-638F-494D-8A78-96ADEFFC267B@oracle.com> <530DE472.80405@gmail.com> Message-ID: On Feb 26, 2014, at 4:56 AM, Peter Levart wrote: >>> Not sure the static powerCache field, in the original code, needs to be volatile either: >>> >>> 1137 private static volatile BigInteger[][] powerCache; >> Is there consensus on whether "volatile" is necessary here? > > I think it has to be volatile. The powerCache implementation was added in the following changeset: > > http://hg.openjdk.java.net/jdk9/dev/jdk/rev/7546 > > ...and improved later in the following: > > http://hg.openjdk.java.net/jdk9/dev/jdk/rev/7586 > > It uses a copy-on-write technique to extend the cache with new values when needed. volatile is mandatory here to safely publish the newly constructed array-of-arrays and the newly constructed sub-array to other threads. Without volatile, other threads could see null slots where BigInteger[] and/or BigInteger objects should be... Thanks for the clarification, Peter. Brian From brian.burkhalter at oracle.com Wed Feb 26 19:55:16 2014 From: brian.burkhalter at oracle.com (Brian Burkhalter) Date: Wed, 26 Feb 2014 11:55:16 -0800 Subject: JDK 9 RFR of JDK-8035279: Clean up internal deprecations in BigInteger In-Reply-To: <0D061461-C44C-4EBB-8848-E0647C374A34@oracle.com> References: <5305B914.7000901@oracle.com> <264E7F8B-E421-4D91-88D9-A22EF0366F2A@oracle.com> <5C039E72-638F-494D-8A78-96ADEFFC267B@oracle.com> <0D061461-C44C-4EBB-8848-E0647C374A34@oracle.com> Message-ID: On Feb 26, 2014, at 5:38 AM, Paul Sandoz wrote: >>> Not sure the static powerCache field, in the original code, needs to be volatile either: >>> >>> 1137 private static volatile BigInteger[][] powerCache; >> >> Is there consensus on whether "volatile" is necessary here? >> > > Looking back at the discussions i believe it was made volatile to ensure threads don't observe a partially updated and published cache lines. > > Since we are already using Unsafe for deserialization I think it might be possible to do the following instead (warning: not tested!): Thanks for the suggestion, Paul. Assuming it is correct, in what way would this be a better approach? (I apologize for being obtuse.) If it looks worth doing, I'll file another issue to track the idea. I already have it on my list anyway to follow up on Alan Eliasen's comment in the BigInteger code: * This could be changed to a more complicated caching method using * {@code Future}. */ private static BigInteger getRadixConversionCache(int radix, int exponent) { Thanks, Brian From roger.riggs at oracle.com Wed Feb 26 20:34:32 2014 From: roger.riggs at oracle.com (roger riggs) Date: Wed, 26 Feb 2014 15:34:32 -0500 Subject: RFR 9: 8035889: jdk testlibrary - add printing of values of failed assertions Message-ID: <530E4FD8.9040605@oracle.com> The testlibrary for the jdk should be printing the values in the failed assertions to make debugging easier and quicker. The webrev adds the printing of the failed assertions and added methods for formatting and unconditional fail methods. Webrev: http://cr.openjdk.java.net/~rriggs/webrev-testlibrary-asserts-8035889/ Bug: 8035889: jdk testlibrary - add printing of values of failed assertions Thanks, Roger [1] https://bugs.openjdk.java.net/browse/JDK-8035889 From martinrb at google.com Wed Feb 26 20:54:24 2014 From: martinrb at google.com (Martin Buchholz) Date: Wed, 26 Feb 2014 12:54:24 -0800 Subject: Unsafe: removing the monitorEnter/monitorExit/tryMonitorEnter methods In-Reply-To: References: Message-ID: I don't recall having added this particular wart to test/java/lang/ProcessBuilder/Basic.java, and history suggests that others did that. It does seem that being able to tell whether a java object monitor is currently locked is useful for debugging and monitoring - there should be a way to do that. On Wed, Feb 26, 2014 at 7:12 AM, Paul Sandoz wrote: > Hi, > > Out of all the methods on Unsafe i think the > monitorEnter/monitorExit/tryMonitorEnter are the least used and are very > strong candidates for removal. > > 99% of use-cases are supported by classes in the > java.util.concurrent.locks package. > > > Within the JDK itself it is only used in one JDK test file > test/java/lang/ProcessBuilder/Basic.java: > > while (unsafe.tryMonitorEnter(s)) { > unsafe.monitorExit(s); > Thread.sleep(1); > } > > for a test verifying an EOF is received on pending reads and it is polling > to check when the process builder acquires the lock before destroying the > process, presumably to avoid some sort of race condition that occasionally > causes the test to fail. > > I believe this test as been through a number of rounds, i stared at things > for a bit, but cannot quickly work out a replacement; i lack the knowledge > on this area. > > > Outside of the JDK i can only find one usage of monitorExit/Enter > (according to grep code) in JBoss modules, and i believe this is only used > on Java 1.6 to work around some limitations in class loading that were > fixed in 1.7. > > > Given such very limited use i propose to remove these methods after having > worked out a fix for ProcessBuilder/Basic.java test. > > Paul. > From paul.sandoz at oracle.com Wed Feb 26 21:29:38 2014 From: paul.sandoz at oracle.com (Paul Sandoz) Date: Wed, 26 Feb 2014 22:29:38 +0100 Subject: [REFRESH] JDK 9 RFR of JDK-8035279: Clean up internal deprecations in BigInteger In-Reply-To: References: <5305D6EB.30806@oracle.com> <53064179.2060703@gmail.com> <530660FA.1060407@oracle.com> <96CF3236-4E65-4D64-BCF3-1E9660D29BAA@oracle.com> <5306BC4B.4090909@oracle.com> <8229AE11-1ABF-474D-961F-BBD5ACA3DD82@oracle.com> <61708919-691D-4EBF-8685-0CC5FC309701@oracle.com> Message-ID: On Feb 26, 2014, at 8:35 PM, Brian Burkhalter wrote: >> [?] >> private int firstNonzeroIntNum() { >> int fn = firstNonzeroIntNumPlusTwo - 2; > > I made all suggested changes except the third line below. Why do we test for equality with -3? If the primitive int default value of zero is used, for firstNonzeroIntNumPlusTwo, as it is, then we should still test whether fn equals -2, n'est-ce pas? > Oops, Oui. You can probably tell i was experimenting with PlusThree to take into account a 0 magnitude. > >> -- >> >> We can revisit and clean things up if/when the JMM is updated. > > Is there an issue for JMM update to which a comment with a link to this thread could be added? > No, there are no issues yet, the discussions are still in flux and nothing concrete has been decided yet. Probably the best way to track as a reminder is to add a new issue for BigInteger to revisit stable fields (JMM) and Unsafe use (Enhanced Volatiles). Paul. From ivan.gerasimov at oracle.com Wed Feb 26 21:30:55 2014 From: ivan.gerasimov at oracle.com (Ivan Gerasimov) Date: Thu, 27 Feb 2014 01:30:55 +0400 Subject: RFR [6943190] TEST_BUG: java/lang/Runtime/exec/ExecWithInput.java hardcodes path to cat In-Reply-To: References: <530B80CF.9090403@oracle.com> Message-ID: <530E5D0F.90409@oracle.com> On 24.02.2014 23:38, Martin Buchholz wrote: > Thanks for working on these ancient Process tests. > I would prefer to see them using "feature tests". > > You wanna do "cat /dev/zero"? Then look for cat in /bin and /usr/bin > and check for /dev/zero as well, e.g. using File.isExecutable > OK. Here's another iteration. I introduced a utility class UnixCommands, which encapsulates searching for the command under several fixed directories. I didn't touch /dev/null though. I think we can assume it exists in every Unix system. Many other tests depend on it anyways. Would you please have a chance to review the updated wevrev? http://cr.openjdk.java.net/~igerasim/6943190/1/webrev/ Sincerely yours, Ivan > Doing OS-specific things like: > > + static final boolean isLinux = > + System.getProperty("os.name ", "unknown").startsWith("Linux"); > > should be a last resort. > > > On Mon, Feb 24, 2014 at 9:26 AM, Ivan Gerasimov > > wrote: > > Hello! > > We've got one quite old report about been unable to run the test > under Android. > https://bugs.openjdk.java.net/browse/JDK-6943190 > > That was due to hard-coded path to the cat utility as /bin/cat. > When investigating, I found two other tests under the same > directory that use /usr/bin/cat and /usr/bin/echo. > These two test seem to (almost) never run because of the unusual path. > > Here's the first version of the webrev, with the fix to only three > tests mentioned above: > http://cr.openjdk.java.net/~igerasim/6943190/0/webrev/ > > > java/lang/Runtime/exec/ directory has also got several other > tests, which run shell commands. > Some of them have absolute path and some of them don't. > > So I have a general question: Why would we ever need the absolute > path for the commands? > Why wouldn't we rely on the PATH env variable? > > Sincerely yours, > Ivan > > > From sean.coffey at oracle.com Wed Feb 26 21:47:14 2014 From: sean.coffey at oracle.com (=?ISO-8859-1?Q?Se=E1n_Coffey?=) Date: Wed, 26 Feb 2014 21:47:14 +0000 Subject: RFR: 8035618 : Four api/org_omg/CORBA TCK tests fail under plugin only In-Reply-To: <530E386F.6090002@oracle.com> References: <530E1B49.8080607@oracle.com> <530E386F.6090002@oracle.com> Message-ID: <530E60E2.3020407@oracle.com> Good points Alan. Changes uploaded here : http://cr.openjdk.java.net/~coffeys/webrev.8035618.v2/webrev/ regards, Sean. On 26/02/14 18:54, Alan Bateman wrote: > On 26/02/2014 16:50, Se?n Coffey wrote: >> Looking to push a fix to JDK 8. A CORBA issue was discovered during >> TCK-Plugin testing. The NPE seen should be handled and the >> defaultPresentationManager should be returned where necessary. >> >> Bug ID : https://bugs.openjdk.java.net/browse/JDK-8035618 >> webrev : http://cr.openjdk.java.net/~coffeys/webrev.8035618/webrev/ > The AppContext usage looks okay but I worry that changing this method > to synchronized will lead to contention. At least for the non security > manager then can this be changed so that the > defaultPresentationManager is initialized via a holder class and > changed to volatile and initialized via the usual double checked > locking idiom. There are other options but I think this one is the > important case and should not be synchronized. > > -Alan. > From paul.sandoz at oracle.com Wed Feb 26 21:53:57 2014 From: paul.sandoz at oracle.com (Paul Sandoz) Date: Wed, 26 Feb 2014 22:53:57 +0100 Subject: JDK 9 RFR of JDK-8035279: Clean up internal deprecations in BigInteger In-Reply-To: References: <5305B914.7000901@oracle.com> <264E7F8B-E421-4D91-88D9-A22EF0366F2A@oracle.com> <5C039E72-638F-494D-8A78-96ADEFFC267B@oracle.com> <0D061461-C44C-4EBB-8848-E0647C374A34@oracle.com> Message-ID: <0A4793AC-509F-4183-94B8-1DD2ABD7D2DC@oracle.com> On Feb 26, 2014, at 8:55 PM, Brian Burkhalter wrote: > > On Feb 26, 2014, at 5:38 AM, Paul Sandoz wrote: > >>>> Not sure the static powerCache field, in the original code, needs to be volatile either: >>>> >>>> 1137 private static volatile BigInteger[][] powerCache; >>> >>> Is there consensus on whether "volatile" is necessary here? >>> >> >> Looking back at the discussions i believe it was made volatile to ensure threads don't observe a partially updated and published cache lines. >> >> Since we are already using Unsafe for deserialization I think it might be possible to do the following instead (warning: not tested!): > > Thanks for the suggestion, Paul. Assuming it is correct, in what way would this be a better approach? (I apologize for being obtuse.) > IMHO it is a simpler solution. It's a cache line that requires updating not the cache line*s*, as the current approach can potentially knock out cache lines for other radixes. > If it looks worth doing, I'll file another issue to track the idea. I already have it on my list anyway to follow up on Alan Eliasen's comment in the BigInteger code: > > * This could be changed to a more complicated caching method using > * {@code Future}. > */ > private static BigInteger getRadixConversionCache(int radix, int exponent) { > Not quite sure what that would entail. Paul. From ivan.gerasimov at oracle.com Wed Feb 26 22:12:39 2014 From: ivan.gerasimov at oracle.com (Ivan Gerasimov) Date: Thu, 27 Feb 2014 02:12:39 +0400 Subject: RFR [6853696] (ref) ReferenceQueue.remove(timeout) may return null even if timeout has not expired In-Reply-To: <530E35EA.2030108@oracle.com> References: <530ADFB2.8000100@oracle.com> <530BC37F.2050908@oracle.com> <530CA986.9090605@oracle.com> <530CD8B5.90000@oracle.com> <530D0F1A.2050503@oracle.com> <530D8771.1020407@oracle.com> <530D92D7.2060304@oracle.com> <530E35EA.2030108@oracle.com> Message-ID: <530E66D7.3070101@oracle.com> On 26.02.2014 22:43, Mandy Chung wrote: > On 2/25/2014 11:08 PM, Ivan Gerasimov wrote: >>> I missed that you remove the strong reference (line 57). I think >>> it's good to hold the strong reference so that >>> ReferenceQueue.remove(timeout) will timeout and the test can verify >>> reliably. >>> >> This is an important part. >> If we didn't remove the strong reference then both threads would wait >> for the specified period of time even without the fix. >> The point is to make both threads wake up from lock.wait(timeout), >> and we do it by removing the strong reference and forcing it be enqueued > > Now it becomes clear what the test does (thanks). It'd be helpful to > add some comments. The test should also verify that only one thread > gets a non-null reference == weakReference and the other should get a > null reference in addition to the wait time > TIMEOUT. > > Good point, thank you Mandy. I should have added comments at the very beginning. Would you take a look at the last updated webrev, with the suggested changes? Sincerely yours, Ivan > Mandy > > From mandy.chung at oracle.com Wed Feb 26 22:15:41 2014 From: mandy.chung at oracle.com (Mandy Chung) Date: Wed, 26 Feb 2014 14:15:41 -0800 Subject: RFR: 8035618 : Four api/org_omg/CORBA TCK tests fail under plugin only In-Reply-To: <530E60E2.3020407@oracle.com> References: <530E1B49.8080607@oracle.com> <530E386F.6090002@oracle.com> <530E60E2.3020407@oracle.com> Message-ID: <530E678D.1020900@oracle.com> Looks good to me. Mandy On 2/26/2014 1:47 PM, Se?n Coffey wrote: > Good points Alan. Changes uploaded here : > http://cr.openjdk.java.net/~coffeys/webrev.8035618.v2/webrev/ > > regards, > Sean. > > On 26/02/14 18:54, Alan Bateman wrote: >> On 26/02/2014 16:50, Se?n Coffey wrote: >>> Looking to push a fix to JDK 8. A CORBA issue was discovered during >>> TCK-Plugin testing. The NPE seen should be handled and the >>> defaultPresentationManager should be returned where necessary. >>> >>> Bug ID : https://bugs.openjdk.java.net/browse/JDK-8035618 >>> webrev : http://cr.openjdk.java.net/~coffeys/webrev.8035618/webrev/ >> The AppContext usage looks okay but I worry that changing this method >> to synchronized will lead to contention. At least for the non >> security manager then can this be changed so that the >> defaultPresentationManager is initialized via a holder class and >> changed to volatile and initialized via the usual double checked >> locking idiom. There are other options but I think this one is the >> important case and should not be synchronized. >> >> -Alan. >> > From roger.riggs at oracle.com Wed Feb 26 22:19:35 2014 From: roger.riggs at oracle.com (roger riggs) Date: Wed, 26 Feb 2014 17:19:35 -0500 Subject: RFR [6943190] TEST_BUG: java/lang/Runtime/exec/ExecWithInput.java hardcodes path to cat In-Reply-To: <530E5D0F.90409@oracle.com> References: <530B80CF.9090403@oracle.com> <530E5D0F.90409@oracle.com> Message-ID: <530E6877.5050504@oracle.com> Hi Ivan, A few comments on UnixCommands: - The trailing "_" on the method names looks very odd, they could all use some suffix "Pgm" "X", etc. - I'm not sure the specific command methods do you any good (over a method with a string argument). since they immediately revert to the string command name. - The final "0" in findCommand0 is using the convention for a native method but it is not native. - Use the File methods for concatenation rather than string concatenation. i.e. new File (parent, child). Roger On 2/26/2014 4:30 PM, Ivan Gerasimov wrote: > > On 24.02.2014 23:38, Martin Buchholz wrote: >> Thanks for working on these ancient Process tests. >> I would prefer to see them using "feature tests". >> >> You wanna do "cat /dev/zero"? Then look for cat in /bin and /usr/bin >> and check for /dev/zero as well, e.g. using File.isExecutable >> > OK. Here's another iteration. > I introduced a utility class UnixCommands, which encapsulates > searching for the command under several fixed directories. > > I didn't touch /dev/null though. I think we can assume it exists in > every Unix system. Many other tests depend on it anyways. > > Would you please have a chance to review the updated wevrev? > http://cr.openjdk.java.net/~igerasim/6943190/1/webrev/ > > Sincerely yours, > Ivan > >> Doing OS-specific things like: >> >> + static final boolean isLinux = >> + System.getProperty("os.name ", >> "unknown").startsWith("Linux"); >> >> should be a last resort. >> >> >> On Mon, Feb 24, 2014 at 9:26 AM, Ivan Gerasimov >> > wrote: >> >> Hello! >> >> We've got one quite old report about been unable to run the test >> under Android. >> https://bugs.openjdk.java.net/browse/JDK-6943190 >> >> That was due to hard-coded path to the cat utility as /bin/cat. >> When investigating, I found two other tests under the same >> directory that use /usr/bin/cat and /usr/bin/echo. >> These two test seem to (almost) never run because of the unusual >> path. >> >> Here's the first version of the webrev, with the fix to only three >> tests mentioned above: >> http://cr.openjdk.java.net/~igerasim/6943190/0/webrev/ >> >> >> java/lang/Runtime/exec/ directory has also got several other >> tests, which run shell commands. >> Some of them have absolute path and some of them don't. >> >> So I have a general question: Why would we ever need the absolute >> path for the commands? >> Why wouldn't we rely on the PATH env variable? >> >> Sincerely yours, >> Ivan >> >> >> > From mandy.chung at oracle.com Wed Feb 26 22:23:00 2014 From: mandy.chung at oracle.com (Mandy Chung) Date: Wed, 26 Feb 2014 14:23:00 -0800 Subject: RFR [6853696] (ref) ReferenceQueue.remove(timeout) may return null even if timeout has not expired In-Reply-To: <530E66D7.3070101@oracle.com> References: <530ADFB2.8000100@oracle.com> <530BC37F.2050908@oracle.com> <530CA986.9090605@oracle.com> <530CD8B5.90000@oracle.com> <530D0F1A.2050503@oracle.com> <530D8771.1020407@oracle.com> <530D92D7.2060304@oracle.com> <530E35EA.2030108@oracle.com> <530E66D7.3070101@oracle.com> Message-ID: <530E6944.7000107@oracle.com> On 2/26/2014 2:12 PM, Ivan Gerasimov wrote: > Good point, thank you Mandy. I should have added comments at the very > beginning. > Would you take a look at the last updated webrev, with the suggested > changes? > http://cr.openjdk.java.net/~igerasim/6853696/3/webrev/ thanks for making the change. It looks okay. Line 81 - it could check thread.reference != null && thread.reference == weakReference A typo in ReferenceQueue.java line 148. I can sponsor this patch for you. No need to generate the webrev and you can simply send me the patch with commit message. thanks Mandy From chris.hegarty at oracle.com Wed Feb 26 22:56:35 2014 From: chris.hegarty at oracle.com (Chris Hegarty) Date: Wed, 26 Feb 2014 22:56:35 +0000 Subject: RFR: 8035618 : Four api/org_omg/CORBA TCK tests fail under plugin only In-Reply-To: <530E60E2.3020407@oracle.com> References: <530E1B49.8080607@oracle.com> <530E386F.6090002@oracle.com> <530E60E2.3020407@oracle.com> Message-ID: <530E7123.8000707@oracle.com> On 26/02/2014 21:47, Se?n Coffey wrote: > Good points Alan. Changes uploaded here : > http://cr.openjdk.java.net/~coffeys/webrev.8035618.v2/webrev/ Looks ok to me Sean. -Chris. > > regards, > Sean. > > On 26/02/14 18:54, Alan Bateman wrote: >> On 26/02/2014 16:50, Se?n Coffey wrote: >>> Looking to push a fix to JDK 8. A CORBA issue was discovered during >>> TCK-Plugin testing. The NPE seen should be handled and the >>> defaultPresentationManager should be returned where necessary. >>> >>> Bug ID : https://bugs.openjdk.java.net/browse/JDK-8035618 >>> webrev : http://cr.openjdk.java.net/~coffeys/webrev.8035618/webrev/ >> The AppContext usage looks okay but I worry that changing this method >> to synchronized will lead to contention. At least for the non security >> manager then can this be changed so that the >> defaultPresentationManager is initialized via a holder class and >> changed to volatile and initialized via the usual double checked >> locking idiom. There are other options but I think this one is the >> important case and should not be synchronized. >> >> -Alan. >> > From sean.coffey at oracle.com Wed Feb 26 23:07:08 2014 From: sean.coffey at oracle.com (sean.coffey at oracle.com) Date: Wed, 26 Feb 2014 23:07:08 +0000 Subject: hg: jdk8/tl/corba: 8035618: Four api/org_omg/CORBA TCK tests fail under plugin only Message-ID: <20140226230711.5750362F74@hg.openjdk.java.net> Changeset: 0683ee308085 Author: coffeys Date: 2014-02-26 23:04 +0000 URL: http://hg.openjdk.java.net/jdk8/tl/corba/rev/0683ee308085 8035618: Four api/org_omg/CORBA TCK tests fail under plugin only Reviewed-by: mchung, chegar ! src/share/classes/com/sun/corba/se/spi/orb/ORB.java From brian.burkhalter at oracle.com Wed Feb 26 23:32:14 2014 From: brian.burkhalter at oracle.com (Brian Burkhalter) Date: Wed, 26 Feb 2014 15:32:14 -0800 Subject: [REFRESH] JDK 9 RFR of JDK-8035279: Clean up internal deprecations in BigInteger In-Reply-To: References: <5305D6EB.30806@oracle.com> <53064179.2060703@gmail.com> <530660FA.1060407@oracle.com> <96CF3236-4E65-4D64-BCF3-1E9660D29BAA@oracle.com> <5306BC4B.4090909@oracle.com> <8229AE11-1ABF-474D-961F-BBD5ACA3DD82@oracle.com> <61708919-691D-4EBF-8685-0CC5FC309701@oracle.com> Message-ID: <10AE28AF-8765-430C-9D83-D0A88062B096@oracle.com> On Feb 26, 2014, at 1:29 PM, Paul Sandoz wrote: >> I made all suggested changes except the third line below. Why do we test for equality with -3? If the primitive int default value of zero is used, for firstNonzeroIntNumPlusTwo, as it is, then we should still test whether fn equals -2, n'est-ce pas? >> > > Oops, Oui. You can probably tell i was experimenting with PlusThree to take into account a 0 magnitude. So may I obtain a +1 from a JDK 9 Reviewer now? >> >>> -- >>> >>> We can revisit and clean things up if/when the JMM is updated. >> >> Is there an issue for JMM update to which a comment with a link to this thread could be added? >> > > No, there are no issues yet, the discussions are still in flux and nothing concrete has been decided yet. Probably the best way to track as a reminder is to add a new issue for BigInteger to revisit stable fields (JMM) and Unsafe use (Enhanced Volatiles). I added a task https://bugs.openjdk.java.net/browse/JDK-8035906 for this which is linked to another pre-existing one which I thought related. Thanks, Brian From mandy.chung at oracle.com Wed Feb 26 23:33:11 2014 From: mandy.chung at oracle.com (Mandy Chung) Date: Wed, 26 Feb 2014 15:33:11 -0800 Subject: Unsafe: removing the monitorEnter/monitorExit/tryMonitorEnter methods In-Reply-To: References: Message-ID: <530E79B7.3020505@oracle.com> On 2/26/2014 12:54 PM, Martin Buchholz wrote: > I don't recall having added this particular wart > to test/java/lang/ProcessBuilder/Basic.java, and history suggests that > others did that. > > It does seem that being able to tell whether a java object monitor is > currently locked is useful for debugging and monitoring - there should be a > way to do that. The closest one I can think of is java.lang.management.ThreadMXBean that can find what object monitors are locked and you can compare the LockInfo with the classname and identity hash of the object you are interested in. However, it means that the test would have a dependency on the management module. ThreadMXBean tmxbean = ManagementFactory.getThreadMXBean(); ThreadInfo[] tinfos = tmxbean.dumpAllThreads(true, false); ThreadInfo.getLockedMonitors() returns the locked object monitors. Mandy > > On Wed, Feb 26, 2014 at 7:12 AM, Paul Sandoz wrote: > >> Hi, >> >> Out of all the methods on Unsafe i think the >> monitorEnter/monitorExit/tryMonitorEnter are the least used and are very >> strong candidates for removal. >> >> 99% of use-cases are supported by classes in the >> java.util.concurrent.locks package. >> >> >> Within the JDK itself it is only used in one JDK test file >> test/java/lang/ProcessBuilder/Basic.java: >> >> while (unsafe.tryMonitorEnter(s)) { >> unsafe.monitorExit(s); >> Thread.sleep(1); >> } >> >> for a test verifying an EOF is received on pending reads and it is polling >> to check when the process builder acquires the lock before destroying the >> process, presumably to avoid some sort of race condition that occasionally >> causes the test to fail. >> >> I believe this test as been through a number of rounds, i stared at things >> for a bit, but cannot quickly work out a replacement; i lack the knowledge >> on this area. >> >> >> Outside of the JDK i can only find one usage of monitorExit/Enter >> (according to grep code) in JBoss modules, and i believe this is only used >> on Java 1.6 to work around some limitations in class loading that were >> fixed in 1.7. >> >> >> Given such very limited use i propose to remove these methods after having >> worked out a fix for ProcessBuilder/Basic.java test. >> >> Paul. >> From brian.burkhalter at oracle.com Wed Feb 26 23:51:06 2014 From: brian.burkhalter at oracle.com (Brian Burkhalter) Date: Wed, 26 Feb 2014 15:51:06 -0800 Subject: JDK 9 RFR of JDK-8035279: Clean up internal deprecations in BigInteger In-Reply-To: <0A4793AC-509F-4183-94B8-1DD2ABD7D2DC@oracle.com> References: <5305B914.7000901@oracle.com> <264E7F8B-E421-4D91-88D9-A22EF0366F2A@oracle.com> <5C039E72-638F-494D-8A78-96ADEFFC267B@oracle.com> <0D061461-C44C-4EBB-8848-E0647C374A34@oracle.com> <0A4793AC-509F-4183-94B8-1DD2ABD7D2DC@oracle.com> Message-ID: <41862830-7AC0-4B37-B947-542532AE63B9@oracle.com> On Feb 26, 2014, at 1:53 PM, Paul Sandoz wrote: >> Thanks for the suggestion, Paul. Assuming it is correct, in what way would this be a better approach? (I apologize for being obtuse.) >> > > IMHO it is a simpler solution. It's a cache line that requires updating not the cache line*s*, as the current approach can potentially knock out cache lines for other radixes. > > >> If it looks worth doing, I'll file another issue to track the idea. I already have it on my list anyway to follow up on Alan Eliasen's comment in the BigInteger code: >> >> * This could be changed to a more complicated caching method using >> * {@code Future}. >> */ >> private static BigInteger getRadixConversionCache(int radix, int exponent) { >> > > Not quite sure what that would entail. I filed Yet Another Issue to keep track of this. Thanks, Brian From brian.burkhalter at oracle.com Wed Feb 26 23:58:55 2014 From: brian.burkhalter at oracle.com (Brian Burkhalter) Date: Wed, 26 Feb 2014 15:58:55 -0800 Subject: JDK 9 RFR of JDK-8035279: Clean up internal deprecations in BigInteger In-Reply-To: <41862830-7AC0-4B37-B947-542532AE63B9@oracle.com> References: <5305B914.7000901@oracle.com> <264E7F8B-E421-4D91-88D9-A22EF0366F2A@oracle.com> <5C039E72-638F-494D-8A78-96ADEFFC267B@oracle.com> <0D061461-C44C-4EBB-8848-E0647C374A34@oracle.com> <0A4793AC-509F-4183-94B8-1DD2ABD7D2DC@oracle.com> <41862830-7AC0-4B37-B947-542532AE63B9@oracle.com> Message-ID: <5F2F5F9E-72F5-41B3-8A21-C7041DA55FF9@oracle.com> On Feb 26, 2014, at 3:51 PM, Brian Burkhalter wrote: >>> * This could be changed to a more complicated caching method using >>> * {@code Future}. >>> */ >>> private static BigInteger getRadixConversionCache(int radix, int exponent) { >>> >> >> Not quite sure what that would entail. > > I filed Yet Another Issue to keep track of this. Forgot the link: https://bugs.openjdk.java.net/browse/JDK-8035908 Brian From stuart.marks at oracle.com Thu Feb 27 00:44:36 2014 From: stuart.marks at oracle.com (Stuart Marks) Date: Wed, 26 Feb 2014 16:44:36 -0800 Subject: RFR(s): 8034999 change rmidRunning to a simple lookup (RMI test library) In-Reply-To: <53041B55.20904@oracle.com> References: <53041B55.20904@oracle.com> Message-ID: <530E8A74.2000805@oracle.com> Hi all, Any takers for this review? s'marks On 2/18/14 6:47 PM, Stuart Marks wrote: > Hi all, > > Please review this change to remove a redundant timing-retry loop from the > rmidRunning() routine of the RMI test library. It is replaced with a simple rmid > registry lookup that returns status immediately, without retries. > > Bug: > > https://bugs.openjdk.java.net/browse/JDK-8034999 > > Webrev: > > http://cr.openjdk.java.net/~smarks/reviews/8034999/webrev.0/ > > Thanks, > > s'marks From joe.darcy at oracle.com Thu Feb 27 00:50:45 2014 From: joe.darcy at oracle.com (Joseph Darcy) Date: Wed, 26 Feb 2014 16:50:45 -0800 Subject: RFR(s): 8034999 change rmidRunning to a simple lookup (RMI test library) In-Reply-To: <530E8A74.2000805@oracle.com> References: <53041B55.20904@oracle.com> <530E8A74.2000805@oracle.com> Message-ID: <530E8BE5.7000003@oracle.com> Hi Stuart, Looks good to go back; thanks, -Joe On 2/26/2014 4:44 PM, Stuart Marks wrote: > Hi all, > > Any takers for this review? > > s'marks > > On 2/18/14 6:47 PM, Stuart Marks wrote: >> Hi all, >> >> Please review this change to remove a redundant timing-retry loop >> from the >> rmidRunning() routine of the RMI test library. It is replaced with a >> simple rmid >> registry lookup that returns status immediately, without retries. >> >> Bug: >> >> https://bugs.openjdk.java.net/browse/JDK-8034999 >> >> Webrev: >> >> http://cr.openjdk.java.net/~smarks/reviews/8034999/webrev.0/ >> >> Thanks, >> >> s'marks From mandy.chung at oracle.com Thu Feb 27 02:01:16 2014 From: mandy.chung at oracle.com (Mandy Chung) Date: Wed, 26 Feb 2014 18:01:16 -0800 Subject: RFR 9: 8035889: jdk testlibrary - add printing of values of failed assertions In-Reply-To: <530E4FD8.9040605@oracle.com> References: <530E4FD8.9040605@oracle.com> Message-ID: <530E9C6C.8080409@oracle.com> Hi Roger, On 2/26/2014 12:34 PM, roger riggs wrote: > The testlibrary for the jdk should be printing the values in the failed > assertions to make debugging easier and quicker. > > The webrev adds the printing of the failed assertions and added methods > for formatting and unconditional fail methods. > > Webrev: > http://cr.openjdk.java.net/~rriggs/webrev-testlibrary-asserts-8035889/ > AssertsTest.java: line 28: @library doesn't look like it's needed. There is no jdk/test/testlibrary directory and I think jdk.testlibrary.* are found as relative to $test.src. Otherwise, the change looks okay. Now that jtreg supports TestNG and I wonder if this class should retire some day (there are only about 10 existing tests using this class). Are you writing new tests using this Asserts class? Mandy > Bug: > 8035889: jdk testlibrary - add printing of values of failed assertions > > Thanks, Roger > > [1] https://bugs.openjdk.java.net/browse/JDK-8035889 > > From Roger.Riggs at Oracle.com Thu Feb 27 03:09:44 2014 From: Roger.Riggs at Oracle.com (Roger Riggs) Date: Wed, 26 Feb 2014 22:09:44 -0500 Subject: RFR 9: 8035889: jdk testlibrary - add printing of values of failed assertions In-Reply-To: <530E9C6C.8080409@oracle.com> References: <530E4FD8.9040605@oracle.com> <530E9C6C.8080409@oracle.com> Message-ID: <530EAC78.4000301@Oracle.com> Hi Mandy, Yes, it might be more productive to switch the tests to TestNG. But it did provide support in cases where TestNG could not be used, for example in a directory of existing tests that had custom reporting. But I remember there is a problem with TestNG having a dependency for XML which is not supported in Profile1 and a number of tests had to be disabled in that configuration. Will XML always be available. Do we need to solve or work around that problem with TestNG? Thanks, Roger On 2/26/14 9:01 PM, Mandy Chung wrote: > Hi Roger, > > On 2/26/2014 12:34 PM, roger riggs wrote: >> The testlibrary for the jdk should be printing the values in the failed >> assertions to make debugging easier and quicker. >> >> The webrev adds the printing of the failed assertions and added methods >> for formatting and unconditional fail methods. >> >> Webrev: >> http://cr.openjdk.java.net/~rriggs/webrev-testlibrary-asserts-8035889/ >> > > AssertsTest.java: line 28: @library doesn't look like it's needed. > There is no jdk/test/testlibrary directory and I think > jdk.testlibrary.* are found as relative to $test.src. > > Otherwise, the change looks okay. > > Now that jtreg supports TestNG and I wonder if this class should > retire some day (there are only about 10 existing tests using this > class). Are you writing new tests using this Asserts class? > > Mandy > >> Bug: >> 8035889: jdk testlibrary - add printing of values of failed >> assertions >> >> Thanks, Roger >> >> [1] https://bugs.openjdk.java.net/browse/JDK-8035889 >> >> > From mandy.chung at oracle.com Thu Feb 27 03:17:51 2014 From: mandy.chung at oracle.com (Mandy Chung) Date: Wed, 26 Feb 2014 19:17:51 -0800 Subject: RFR 9: 8035889: jdk testlibrary - add printing of values of failed assertions In-Reply-To: <530EAC78.4000301@Oracle.com> References: <530E4FD8.9040605@oracle.com> <530E9C6C.8080409@oracle.com> <530EAC78.4000301@Oracle.com> Message-ID: <530EAE5F.3050308@oracle.com> On 2/26/2014 7:09 PM, Roger Riggs wrote: > Hi Mandy, > > Yes, it might be more productive to switch the tests to TestNG. > But it did provide support in cases where TestNG could not be used, > for example in a directory of existing tests that had custom reporting. > > But I remember there is a problem with TestNG having a dependency for XML > which is not supported in Profile1 and a number of tests had to be > disabled > in that configuration. Will XML always be available. Do we need to solve > or work around that problem with TestNG? > This is a good point. When we want to test just the base module for example, how can we run TestNG tests? We need to address that certainly. My comment on TestNG is a question for new tests using this Asserts class. Your patch is fine to go (after taking out @library tag if I got it correct). Mandy > Thanks, Roger > > On 2/26/14 9:01 PM, Mandy Chung wrote: >> Hi Roger, >> >> On 2/26/2014 12:34 PM, roger riggs wrote: >>> The testlibrary for the jdk should be printing the values in the failed >>> assertions to make debugging easier and quicker. >>> >>> The webrev adds the printing of the failed assertions and added methods >>> for formatting and unconditional fail methods. >>> >>> Webrev: >>> http://cr.openjdk.java.net/~rriggs/webrev-testlibrary-asserts-8035889/ >>> >> >> AssertsTest.java: line 28: @library doesn't look like it's needed. >> There is no jdk/test/testlibrary directory and I think >> jdk.testlibrary.* are found as relative to $test.src. >> >> Otherwise, the change looks okay. >> >> Now that jtreg supports TestNG and I wonder if this class should >> retire some day (there are only about 10 existing tests using this >> class). Are you writing new tests using this Asserts class? >> >> Mandy >> >>> Bug: >>> 8035889: jdk testlibrary - add printing of values of failed >>> assertions >>> >>> Thanks, Roger >>> >>> [1] https://bugs.openjdk.java.net/browse/JDK-8035889 >>> >>> >> > From staffan.larsen at oracle.com Thu Feb 27 06:58:48 2014 From: staffan.larsen at oracle.com (Staffan Larsen) Date: Thu, 27 Feb 2014 07:58:48 +0100 Subject: RFR 6835233 : Fedora 9 jdk regression test failed: java/lang/instrument/ParallelTransformerLoader.sh In-Reply-To: <530E41DB.20801@oracle.com> References: <530E41DB.20801@oracle.com> Message-ID: <5A25AA54-BD79-4590-8230-0C875CAF181D@oracle.com> Looks good! Thanks, /Staffan On 26 feb 2014, at 20:34, Brent Christian wrote: > File under "chipping away at test stabilization issues." > > https://bugs.openjdk.java.net/browse/JDK-6835233 > > I've done some repeated runs of this test on my Linux machine. The test fails every time with 6u3. It fails intermittently on 7 (after 145 iterations for 7u45, and 62 iterations for 7u60b07). I have not been able to reproduce the failure on 8 or 9, running 1000 iterations each on 8b115, 8b129, and 9b02. > > I would like to resolve this bug by removing the "@ignore" tag for JDK 9, and bring the test back into rotation. If the failure comes back, I'll submit a new issue for further investigation. > > The change is: > > # @bug 5088398 > -# @ignore until bug 6835233 dealt with > # @summary Test parallel class loading by parallel transformers. > > Thanks, > -Brent From peter.levart at gmail.com Thu Feb 27 07:29:01 2014 From: peter.levart at gmail.com (Peter Levart) Date: Thu, 27 Feb 2014 08:29:01 +0100 Subject: Unsafe: removing the monitorEnter/monitorExit/tryMonitorEnter methods In-Reply-To: References: Message-ID: <530EE93D.7090406@gmail.com> On 02/26/2014 09:54 PM, Martin Buchholz wrote: > I don't recall having added this particular wart > to test/java/lang/ProcessBuilder/Basic.java, and history suggests that > others did that. > > It does seem that being able to tell whether a java object monitor is > currently locked is useful for debugging and monitoring - there should be a > way to do that. Thread.holdsLock(Object) ? Regards, Peter > > > On Wed, Feb 26, 2014 at 7:12 AM, Paul Sandoz wrote: > >> Hi, >> >> Out of all the methods on Unsafe i think the >> monitorEnter/monitorExit/tryMonitorEnter are the least used and are very >> strong candidates for removal. >> >> 99% of use-cases are supported by classes in the >> java.util.concurrent.locks package. >> >> >> Within the JDK itself it is only used in one JDK test file >> test/java/lang/ProcessBuilder/Basic.java: >> >> while (unsafe.tryMonitorEnter(s)) { >> unsafe.monitorExit(s); >> Thread.sleep(1); >> } >> >> for a test verifying an EOF is received on pending reads and it is polling >> to check when the process builder acquires the lock before destroying the >> process, presumably to avoid some sort of race condition that occasionally >> causes the test to fail. >> >> I believe this test as been through a number of rounds, i stared at things >> for a bit, but cannot quickly work out a replacement; i lack the knowledge >> on this area. >> >> >> Outside of the JDK i can only find one usage of monitorExit/Enter >> (according to grep code) in JBoss modules, and i believe this is only used >> on Java 1.6 to work around some limitations in class loading that were >> fixed in 1.7. >> >> >> Given such very limited use i propose to remove these methods after having >> worked out a fix for ProcessBuilder/Basic.java test. >> >> Paul. >> From peter.levart at gmail.com Thu Feb 27 07:30:43 2014 From: peter.levart at gmail.com (Peter Levart) Date: Thu, 27 Feb 2014 08:30:43 +0100 Subject: Unsafe: removing the monitorEnter/monitorExit/tryMonitorEnter methods In-Reply-To: <530EE93D.7090406@gmail.com> References: <530EE93D.7090406@gmail.com> Message-ID: <530EE9A3.4080101@gmail.com> On 02/27/2014 08:29 AM, Peter Levart wrote: > On 02/26/2014 09:54 PM, Martin Buchholz wrote: >> I don't recall having added this particular wart >> to test/java/lang/ProcessBuilder/Basic.java, and history suggests that >> others did that. >> >> It does seem that being able to tell whether a java object monitor is >> currently locked is useful for debugging and monitoring - there >> should be a >> way to do that. > > Thread.holdsLock(Object) ? Ah, you meant to query from some other thread, right? Peter > > Regards, Peter > >> >> >> On Wed, Feb 26, 2014 at 7:12 AM, Paul Sandoz >> wrote: >> >>> Hi, >>> >>> Out of all the methods on Unsafe i think the >>> monitorEnter/monitorExit/tryMonitorEnter are the least used and are >>> very >>> strong candidates for removal. >>> >>> 99% of use-cases are supported by classes in the >>> java.util.concurrent.locks package. >>> >>> >>> Within the JDK itself it is only used in one JDK test file >>> test/java/lang/ProcessBuilder/Basic.java: >>> >>> while (unsafe.tryMonitorEnter(s)) { >>> unsafe.monitorExit(s); >>> Thread.sleep(1); >>> } >>> >>> for a test verifying an EOF is received on pending reads and it is >>> polling >>> to check when the process builder acquires the lock before >>> destroying the >>> process, presumably to avoid some sort of race condition that >>> occasionally >>> causes the test to fail. >>> >>> I believe this test as been through a number of rounds, i stared at >>> things >>> for a bit, but cannot quickly work out a replacement; i lack the >>> knowledge >>> on this area. >>> >>> >>> Outside of the JDK i can only find one usage of monitorExit/Enter >>> (according to grep code) in JBoss modules, and i believe this is >>> only used >>> on Java 1.6 to work around some limitations in class loading that were >>> fixed in 1.7. >>> >>> >>> Given such very limited use i propose to remove these methods after >>> having >>> worked out a fix for ProcessBuilder/Basic.java test. >>> >>> Paul. >>> > From david.holmes at oracle.com Thu Feb 27 07:41:58 2014 From: david.holmes at oracle.com (David Holmes) Date: Thu, 27 Feb 2014 17:41:58 +1000 Subject: Unsafe: removing the monitorEnter/monitorExit/tryMonitorEnter methods In-Reply-To: <530EE9A3.4080101@gmail.com> References: <530EE93D.7090406@gmail.com> <530EE9A3.4080101@gmail.com> Message-ID: <530EEC46.40605@oracle.com> On 27/02/2014 5:30 PM, Peter Levart wrote: > On 02/27/2014 08:29 AM, Peter Levart wrote: >> On 02/26/2014 09:54 PM, Martin Buchholz wrote: >>> I don't recall having added this particular wart >>> to test/java/lang/ProcessBuilder/Basic.java, and history suggests that >>> others did that. >>> >>> It does seem that being able to tell whether a java object monitor is >>> currently locked is useful for debugging and monitoring - there >>> should be a >>> way to do that. >> >> Thread.holdsLock(Object) ? > > Ah, you meant to query from some other thread, right? Right - that is the usage in Basic.java but I'm still scratching my head trying to understand what finding the BufferedInputStream locked is supposed to signify. David ----- > Peter > >> >> Regards, Peter >> >>> >>> >>> On Wed, Feb 26, 2014 at 7:12 AM, Paul Sandoz >>> wrote: >>> >>>> Hi, >>>> >>>> Out of all the methods on Unsafe i think the >>>> monitorEnter/monitorExit/tryMonitorEnter are the least used and are >>>> very >>>> strong candidates for removal. >>>> >>>> 99% of use-cases are supported by classes in the >>>> java.util.concurrent.locks package. >>>> >>>> >>>> Within the JDK itself it is only used in one JDK test file >>>> test/java/lang/ProcessBuilder/Basic.java: >>>> >>>> while (unsafe.tryMonitorEnter(s)) { >>>> unsafe.monitorExit(s); >>>> Thread.sleep(1); >>>> } >>>> >>>> for a test verifying an EOF is received on pending reads and it is >>>> polling >>>> to check when the process builder acquires the lock before >>>> destroying the >>>> process, presumably to avoid some sort of race condition that >>>> occasionally >>>> causes the test to fail. >>>> >>>> I believe this test as been through a number of rounds, i stared at >>>> things >>>> for a bit, but cannot quickly work out a replacement; i lack the >>>> knowledge >>>> on this area. >>>> >>>> >>>> Outside of the JDK i can only find one usage of monitorExit/Enter >>>> (according to grep code) in JBoss modules, and i believe this is >>>> only used >>>> on Java 1.6 to work around some limitations in class loading that were >>>> fixed in 1.7. >>>> >>>> >>>> Given such very limited use i propose to remove these methods after >>>> having >>>> worked out a fix for ProcessBuilder/Basic.java test. >>>> >>>> Paul. >>>> >> > From eliasen at mindspring.com Thu Feb 27 07:43:20 2014 From: eliasen at mindspring.com (Alan Eliasen) Date: Thu, 27 Feb 2014 00:43:20 -0700 Subject: JDK 9 RFR of JDK-8035279: Clean up internal deprecations in BigInteger In-Reply-To: <0A4793AC-509F-4183-94B8-1DD2ABD7D2DC@oracle.com> References: <5305B914.7000901@oracle.com> <264E7F8B-E421-4D91-88D9-A22EF0366F2A@oracle.com> <5C039E72-638F-494D-8A78-96ADEFFC267B@oracle.com> <0D061461-C44C-4EBB-8848-E0647C374A34@oracle.com> <0A4793AC-509F-4183-94B8-1DD2ABD7D2DC@oracle.com> Message-ID: <530EEC98.2070903@mindspring.com> > On Feb 26, 2014, at 8:55 PM, Brian Burkhalter > wrote: >> If it looks worth doing, I'll file another issue to track the idea. >> I already have it on my list anyway to follow up on Alan Eliasen's >> comment in the BigInteger code: >> >> * This could be changed to a more complicated caching method using >> * {@code Future}. */ >> private static BigInteger getRadixConversionCache(int radix, int exponent) { On 02/26/2014 02:53 PM, Paul Sandoz wrote: > Not quite sure what that would entail. > > Paul. See, for example, p. 108 of "Java Concurrency in Practice," if one of the guys there has a copy. ;) I started writing a version of that cache using Future but it got looking a bit hairy. I submitted the very-simple version I did (which is not the one that was in the final release) because I thought anything more complex would hold up review too much. -- Alan Eliasen eliasen at mindspring.com http://futureboy.us/ From Alan.Bateman at oracle.com Thu Feb 27 07:45:01 2014 From: Alan.Bateman at oracle.com (Alan Bateman) Date: Thu, 27 Feb 2014 07:45:01 +0000 Subject: RFR: 8035618 : Four api/org_omg/CORBA TCK tests fail under plugin only In-Reply-To: <530E60E2.3020407@oracle.com> References: <530E1B49.8080607@oracle.com> <530E386F.6090002@oracle.com> <530E60E2.3020407@oracle.com> Message-ID: <530EECFD.8060406@oracle.com> On 26/02/2014 21:47, Se?n Coffey wrote: > Good points Alan. Changes uploaded here : > http://cr.openjdk.java.net/~coffeys/webrev.8035618.v2/webrev/ This looks much better. Are you sure that pmLock is needed? Is there any reason not to synchronize on the AppContext when adding the PresentationManager key? Are you planning to propose a fix for jdk9/dev too? It would be good to eliminate the direct dependency on sun.awt.AppContext while doing that. -Alan From Alan.Bateman at oracle.com Thu Feb 27 07:47:57 2014 From: Alan.Bateman at oracle.com (Alan Bateman) Date: Thu, 27 Feb 2014 07:47:57 +0000 Subject: RFR 6835233 : Fedora 9 jdk regression test failed: java/lang/instrument/ParallelTransformerLoader.sh In-Reply-To: <530E41DB.20801@oracle.com> References: <530E41DB.20801@oracle.com> Message-ID: <530EEDAD.9050400@oracle.com> On 26/02/2014 19:34, Brent Christian wrote: > : > > The change is: > > # @bug 5088398 > -# @ignore until bug 6835233 dealt with > # @summary Test parallel class loading by parallel transformers. This looks okay to me too, I assume that if there is any residual issue that it will show up quickly once the test is running again. -Alan From paul.sandoz at oracle.com Thu Feb 27 08:17:42 2014 From: paul.sandoz at oracle.com (Paul Sandoz) Date: Thu, 27 Feb 2014 09:17:42 +0100 Subject: [REFRESH] JDK 9 RFR of JDK-8035279: Clean up internal deprecations in BigInteger In-Reply-To: <10AE28AF-8765-430C-9D83-D0A88062B096@oracle.com> References: <5305D6EB.30806@oracle.com> <53064179.2060703@gmail.com> <530660FA.1060407@oracle.com> <96CF3236-4E65-4D64-BCF3-1E9660D29BAA@oracle.com> <5306BC4B.4090909@oracle.com> <8229AE11-1ABF-474D-961F-BBD5ACA3DD82@oracle.com> <61708919-691D-4EBF-8685-0CC5FC309701@oracle.com> <10AE28AF-8765-430C-9D83-D0A88062B096@oracle.com> Message-ID: <247D03A1-0D88-437D-B2F1-9CD5DAE8492D@oracle.com> On Feb 27, 2014, at 12:32 AM, Brian Burkhalter wrote: > > On Feb 26, 2014, at 1:29 PM, Paul Sandoz wrote: > >>> I made all suggested changes except the third line below. Why do we test for equality with -3? If the primitive int default value of zero is used, for firstNonzeroIntNumPlusTwo, as it is, then we should still test whether fn equals -2, n'est-ce pas? >>> >> >> Oops, Oui. You can probably tell i was experimenting with PlusThree to take into account a 0 magnitude. > > So may I obtain a +1 from a JDK 9 Reviewer now? > Indeed you may. >>> >>>> -- >>>> >>>> We can revisit and clean things up if/when the JMM is updated. >>> >>> Is there an issue for JMM update to which a comment with a link to this thread could be added? >>> >> >> No, there are no issues yet, the discussions are still in flux and nothing concrete has been decided yet. Probably the best way to track as a reminder is to add a new issue for BigInteger to revisit stable fields (JMM) and Unsafe use (Enhanced Volatiles). > > I added a task https://bugs.openjdk.java.net/browse/JDK-8035906 for this which is linked to another pre-existing one which I thought related. > OK. Paul. From paul.sandoz at oracle.com Thu Feb 27 08:22:13 2014 From: paul.sandoz at oracle.com (Paul Sandoz) Date: Thu, 27 Feb 2014 09:22:13 +0100 Subject: JDK 9 RFR of JDK-8035279: Clean up internal deprecations in BigInteger In-Reply-To: <41862830-7AC0-4B37-B947-542532AE63B9@oracle.com> References: <5305B914.7000901@oracle.com> <264E7F8B-E421-4D91-88D9-A22EF0366F2A@oracle.com> <5C039E72-638F-494D-8A78-96ADEFFC267B@oracle.com> <0D061461-C44C-4EBB-8848-E0647C374A34@oracle.com> <0A4793AC-509F-4183-94B8-1DD2ABD7D2DC@oracle.com> <41862830-7AC0-4B37-B947-542532AE63B9@oracle.com> Message-ID: On Feb 27, 2014, at 12:51 AM, Brian Burkhalter wrote: > > On Feb 26, 2014, at 1:53 PM, Paul Sandoz wrote: > >>> Thanks for the suggestion, Paul. Assuming it is correct, in what way would this be a better approach? (I apologize for being obtuse.) >>> >> >> IMHO it is a simpler solution. It's a cache line that requires updating not the cache line*s*, as the current approach can potentially knock out cache lines for other radixes. >> I forgot to mention a slightly more complex variation is to CAS the cache line instead of a lazy set, which can ensure a line does not knock another containing a higher exponent. Paul. From peter.levart at gmail.com Thu Feb 27 08:38:59 2014 From: peter.levart at gmail.com (Peter Levart) Date: Thu, 27 Feb 2014 09:38:59 +0100 Subject: Unsafe: removing the monitorEnter/monitorExit/tryMonitorEnter methods In-Reply-To: <530EEC46.40605@oracle.com> References: <530EE93D.7090406@gmail.com> <530EE9A3.4080101@gmail.com> <530EEC46.40605@oracle.com> Message-ID: <530EF9A3.1000108@gmail.com> On 02/27/2014 08:41 AM, David Holmes wrote: > On 27/02/2014 5:30 PM, Peter Levart wrote: >> On 02/27/2014 08:29 AM, Peter Levart wrote: >>> On 02/26/2014 09:54 PM, Martin Buchholz wrote: >>>> I don't recall having added this particular wart >>>> to test/java/lang/ProcessBuilder/Basic.java, and history suggests that >>>> others did that. >>>> >>>> It does seem that being able to tell whether a java object monitor is >>>> currently locked is useful for debugging and monitoring - there >>>> should be a >>>> way to do that. >>> >>> Thread.holdsLock(Object) ? >> >> Ah, you meant to query from some other thread, right? > > Right - that is the usage in Basic.java but I'm still scratching my > head trying to understand what finding the BufferedInputStream locked > is supposed to signify. Here's the relevant code block: while (unsafe.tryMonitorEnter(s)) { unsafe.monitorExit(s); Thread.sleep(1); } This code block waits until it finds that some thread has acquired the lock on 's' for at least 1 ms (or else it can miss it and will wait further) ... Since this block is executed for s instanceof BufferedInputStream, what this block does, is wait until one of synchronized methods on BufferedInputStream is called. The thread that does that, calls read() or read(byte[]) and blocks while holding a lock, since the child process that is spawned and whose stdout the thread is trying to read in this situation does a 10 minute sleep. I think the following could be used instead for the same purpose: while (!isLocked(s, 100L)) { Thread.sleep(1); } // using the following utility method: static boolean isLocked(final Object monitor, long millis) throws InterruptedException { Thread t = new Thread() { @Override public void run() { synchronized (monitor) { } } }; t.start(); t.join(millis); return t.isAlive(); } Regards, Peter > > David > ----- > >> Peter >> >>> >>> Regards, Peter >>> >>>> >>>> >>>> On Wed, Feb 26, 2014 at 7:12 AM, Paul Sandoz >>>> wrote: >>>> >>>>> Hi, >>>>> >>>>> Out of all the methods on Unsafe i think the >>>>> monitorEnter/monitorExit/tryMonitorEnter are the least used and are >>>>> very >>>>> strong candidates for removal. >>>>> >>>>> 99% of use-cases are supported by classes in the >>>>> java.util.concurrent.locks package. >>>>> >>>>> >>>>> Within the JDK itself it is only used in one JDK test file >>>>> test/java/lang/ProcessBuilder/Basic.java: >>>>> >>>>> while (unsafe.tryMonitorEnter(s)) { >>>>> unsafe.monitorExit(s); >>>>> Thread.sleep(1); >>>>> } >>>>> >>>>> for a test verifying an EOF is received on pending reads and it is >>>>> polling >>>>> to check when the process builder acquires the lock before >>>>> destroying the >>>>> process, presumably to avoid some sort of race condition that >>>>> occasionally >>>>> causes the test to fail. >>>>> >>>>> I believe this test as been through a number of rounds, i stared at >>>>> things >>>>> for a bit, but cannot quickly work out a replacement; i lack the >>>>> knowledge >>>>> on this area. >>>>> >>>>> >>>>> Outside of the JDK i can only find one usage of monitorExit/Enter >>>>> (according to grep code) in JBoss modules, and i believe this is >>>>> only used >>>>> on Java 1.6 to work around some limitations in class loading that >>>>> were >>>>> fixed in 1.7. >>>>> >>>>> >>>>> Given such very limited use i propose to remove these methods after >>>>> having >>>>> worked out a fix for ProcessBuilder/Basic.java test. >>>>> >>>>> Paul. >>>>> >>> >> From ivan.gerasimov at oracle.com Thu Feb 27 09:34:27 2014 From: ivan.gerasimov at oracle.com (Ivan Gerasimov) Date: Thu, 27 Feb 2014 13:34:27 +0400 Subject: RFR [6943190] TEST_BUG: java/lang/Runtime/exec/ExecWithInput.java hardcodes path to cat In-Reply-To: <530E6877.5050504@oracle.com> References: <530B80CF.9090403@oracle.com> <530E5D0F.90409@oracle.com> <530E6877.5050504@oracle.com> Message-ID: <530F06A3.1030107@oracle.com> Thank you Roger for looking at this. On 27.02.2014 2:19, roger riggs wrote: > Hi Ivan, > > A few comments on UnixCommands: > > - The trailing "_" on the method names looks very odd, they could all > use some suffix "Pgm" "X", etc. > I replaced true_() and false_() with explicit calls to findCommand(). > - I'm not sure the specific command methods do you any good > (over a method with a string argument). since they immediately > revert to the string command name. > They are shorter :-) > - The final "0" in findCommand0 is using the convention for a native > method but it is not native. > Yes. But I think the convention is more general -- the f0() is usually a function of a lower level than f(). Grepping shows some examples: ./jdk/share/classes/java/security/KeyStore.java: run0() is called from run() ./jdk/share/classes/java/util/regex/Pattern.java: match0() is called from match() and In the tests, main0() sometimes is called from main(), test0() from test(), etc. No problem to rename it, but I don't think it should confuse too much. > - Use the File methods for concatenation rather than string > concatenation. > i.e. new File (parent, child). Good point, thanks! Please find the updated webrev here: http://cr.openjdk.java.net/~igerasim/6943190/2/webrev/ Sincerely yours, Ivan > > Roger > > > > > On 2/26/2014 4:30 PM, Ivan Gerasimov wrote: >> >> On 24.02.2014 23:38, Martin Buchholz wrote: >>> Thanks for working on these ancient Process tests. >>> I would prefer to see them using "feature tests". >>> >>> You wanna do "cat /dev/zero"? Then look for cat in /bin and >>> /usr/bin and check for /dev/zero as well, e.g. using File.isExecutable >>> >> OK. Here's another iteration. >> I introduced a utility class UnixCommands, which encapsulates >> searching for the command under several fixed directories. >> >> I didn't touch /dev/null though. I think we can assume it exists in >> every Unix system. Many other tests depend on it anyways. >> >> Would you please have a chance to review the updated wevrev? >> http://cr.openjdk.java.net/~igerasim/6943190/1/webrev/ >> >> Sincerely yours, >> Ivan >> >>> Doing OS-specific things like: >>> >>> + static final boolean isLinux = >>> + System.getProperty("os.name ", >>> "unknown").startsWith("Linux"); >>> >>> should be a last resort. >>> >>> >>> On Mon, Feb 24, 2014 at 9:26 AM, Ivan Gerasimov >>> > wrote: >>> >>> Hello! >>> >>> We've got one quite old report about been unable to run the test >>> under Android. >>> https://bugs.openjdk.java.net/browse/JDK-6943190 >>> >>> That was due to hard-coded path to the cat utility as /bin/cat. >>> When investigating, I found two other tests under the same >>> directory that use /usr/bin/cat and /usr/bin/echo. >>> These two test seem to (almost) never run because of the unusual >>> path. >>> >>> Here's the first version of the webrev, with the fix to only three >>> tests mentioned above: >>> http://cr.openjdk.java.net/~igerasim/6943190/0/webrev/ >>> >>> >>> java/lang/Runtime/exec/ directory has also got several other >>> tests, which run shell commands. >>> Some of them have absolute path and some of them don't. >>> >>> So I have a general question: Why would we ever need the absolute >>> path for the commands? >>> Why wouldn't we rely on the PATH env variable? >>> >>> Sincerely yours, >>> Ivan >>> >>> >>> >> > > > From sean.coffey at oracle.com Thu Feb 27 09:38:58 2014 From: sean.coffey at oracle.com (=?ISO-8859-1?Q?Se=E1n_Coffey?=) Date: Thu, 27 Feb 2014 09:38:58 +0000 Subject: RFR: 8035618 : Four api/org_omg/CORBA TCK tests fail under plugin only In-Reply-To: <530EECFD.8060406@oracle.com> References: <530E1B49.8080607@oracle.com> <530E386F.6090002@oracle.com> <530E60E2.3020407@oracle.com> <530EECFD.8060406@oracle.com> Message-ID: <530F07B2.9000300@oracle.com> On 27/02/2014 07:45, Alan Bateman wrote: > On 26/02/2014 21:47, Se?n Coffey wrote: >> Good points Alan. Changes uploaded here : >> http://cr.openjdk.java.net/~coffeys/webrev.8035618.v2/webrev/ > This looks much better. Are you sure that pmLock is needed? Is there > any reason not to synchronize on the AppContext when adding the > PresentationManager key? I was conscious of possible contention coming on the AppContext object if syncing on it. I figured a static lock should work (given that we had zero synchronization in the old code) - Apologies for not including your name in final push. It's been pointed out to me that I should include all reviewers. I thought I could only include reviewers once I had an 'ok'. > > Are you planning to propose a fix for jdk9/dev too? It would be good > to eliminate the direct dependency on sun.awt.AppContext while doing > that. Yes - I'll get the direct AppContext reference removed for JDK 9 fix via JavaAWTAccess but will need to re-examine the code to see if we can avoid sun.awt code. regards, Sean. > > -Alan From Alan.Bateman at oracle.com Thu Feb 27 09:49:38 2014 From: Alan.Bateman at oracle.com (Alan Bateman) Date: Thu, 27 Feb 2014 09:49:38 +0000 Subject: RFR: 8035618 : Four api/org_omg/CORBA TCK tests fail under plugin only In-Reply-To: <530F07B2.9000300@oracle.com> References: <530E1B49.8080607@oracle.com> <530E386F.6090002@oracle.com> <530E60E2.3020407@oracle.com> <530EECFD.8060406@oracle.com> <530F07B2.9000300@oracle.com> Message-ID: <530F0A32.6050602@oracle.com> On 27/02/2014 09:38, Se?n Coffey wrote: > > On 27/02/2014 07:45, Alan Bateman wrote: >> On 26/02/2014 21:47, Se?n Coffey wrote: >>> Good points Alan. Changes uploaded here : >>> http://cr.openjdk.java.net/~coffeys/webrev.8035618.v2/webrev/ >> This looks much better. Are you sure that pmLock is needed? Is there >> any reason not to synchronize on the AppContext when adding the >> PresentationManager key? > I was conscious of possible contention coming on the AppContext object > if syncing on it. I figured a static lock should work (given that we > had zero synchronization in the old code) Assuming AppContext is mostly for applets and JNLP applications then I wouldn't expect it should be a problem. I also wouldn't expect there are many cases with multiple applets in the same VM all using CORBA at the same time. So what you have it okay but equally okay to drop pmLock and use ac instead. > Yes - I'll get the direct AppContext reference removed for JDK 9 fix > via JavaAWTAccess but will need to re-examine the code to see if we > can avoid sun.awt code. That would be great. -Alan. From david.holmes at oracle.com Thu Feb 27 09:58:24 2014 From: david.holmes at oracle.com (David Holmes) Date: Thu, 27 Feb 2014 19:58:24 +1000 Subject: Unsafe: removing the monitorEnter/monitorExit/tryMonitorEnter methods In-Reply-To: <530EF9A3.1000108@gmail.com> References: <530EE93D.7090406@gmail.com> <530EE9A3.4080101@gmail.com> <530EEC46.40605@oracle.com> <530EF9A3.1000108@gmail.com> Message-ID: <530F0C40.7070303@oracle.com> On 27/02/2014 6:38 PM, Peter Levart wrote: > On 02/27/2014 08:41 AM, David Holmes wrote: >> On 27/02/2014 5:30 PM, Peter Levart wrote: >>> On 02/27/2014 08:29 AM, Peter Levart wrote: >>>> On 02/26/2014 09:54 PM, Martin Buchholz wrote: >>>>> I don't recall having added this particular wart >>>>> to test/java/lang/ProcessBuilder/Basic.java, and history suggests that >>>>> others did that. >>>>> >>>>> It does seem that being able to tell whether a java object monitor is >>>>> currently locked is useful for debugging and monitoring - there >>>>> should be a >>>>> way to do that. >>>> >>>> Thread.holdsLock(Object) ? >>> >>> Ah, you meant to query from some other thread, right? >> >> Right - that is the usage in Basic.java but I'm still scratching my >> head trying to understand what finding the BufferedInputStream locked >> is supposed to signify. > > Here's the relevant code block: > > while (unsafe.tryMonitorEnter(s)) { > unsafe.monitorExit(s); > Thread.sleep(1); > } > > This code block waits until it finds that some thread has acquired the > lock on 's' for at least 1 ms (or else it can miss it and will wait > further) ... > > Since this block is executed for s instanceof BufferedInputStream, what > this block does, is wait until one of synchronized methods on > BufferedInputStream is called. The thread that does that, calls read() > or read(byte[]) and blocks while holding a lock, since the child process > that is spawned and whose stdout the thread is trying to read in this > situation does a 10 minute sleep. Ah I see - thanks for the analysis. So this is a classic 'barrier' waiting for the other thread to have reached the synchronized method. > I think the following could be used instead for the same purpose: > > while (!isLocked(s, 100L)) { > Thread.sleep(1); > } > > // using the following utility method: > > static boolean isLocked(final Object monitor, long millis) throws > InterruptedException { > Thread t = new Thread() { > @Override > public void run() { > synchronized (monitor) { } > } > }; > t.start(); > t.join(millis); > return t.isAlive(); > } Functionally yes this will only return true if the object is locked for >100ms. But it is potentially going to generate a lot of temporary threads. And in theory at least the JIT can optimize the run method into nothing. Still it is a cute trick :) David > > Regards, Peter > >> >> David >> ----- >> >>> Peter >>> >>>> >>>> Regards, Peter >>>> >>>>> >>>>> >>>>> On Wed, Feb 26, 2014 at 7:12 AM, Paul Sandoz >>>>> wrote: >>>>> >>>>>> Hi, >>>>>> >>>>>> Out of all the methods on Unsafe i think the >>>>>> monitorEnter/monitorExit/tryMonitorEnter are the least used and are >>>>>> very >>>>>> strong candidates for removal. >>>>>> >>>>>> 99% of use-cases are supported by classes in the >>>>>> java.util.concurrent.locks package. >>>>>> >>>>>> >>>>>> Within the JDK itself it is only used in one JDK test file >>>>>> test/java/lang/ProcessBuilder/Basic.java: >>>>>> >>>>>> while (unsafe.tryMonitorEnter(s)) { >>>>>> unsafe.monitorExit(s); >>>>>> Thread.sleep(1); >>>>>> } >>>>>> >>>>>> for a test verifying an EOF is received on pending reads and it is >>>>>> polling >>>>>> to check when the process builder acquires the lock before >>>>>> destroying the >>>>>> process, presumably to avoid some sort of race condition that >>>>>> occasionally >>>>>> causes the test to fail. >>>>>> >>>>>> I believe this test as been through a number of rounds, i stared at >>>>>> things >>>>>> for a bit, but cannot quickly work out a replacement; i lack the >>>>>> knowledge >>>>>> on this area. >>>>>> >>>>>> >>>>>> Outside of the JDK i can only find one usage of monitorExit/Enter >>>>>> (according to grep code) in JBoss modules, and i believe this is >>>>>> only used >>>>>> on Java 1.6 to work around some limitations in class loading that >>>>>> were >>>>>> fixed in 1.7. >>>>>> >>>>>> >>>>>> Given such very limited use i propose to remove these methods after >>>>>> having >>>>>> worked out a fix for ProcessBuilder/Basic.java test. >>>>>> >>>>>> Paul. >>>>>> >>>> >>> > From david.holmes at oracle.com Thu Feb 27 10:07:29 2014 From: david.holmes at oracle.com (David Holmes) Date: Thu, 27 Feb 2014 20:07:29 +1000 Subject: JDK 9 RFR of JDK-8035279: Clean up internal deprecations in BigInteger In-Reply-To: <530DE472.80405@gmail.com> References: <5305B914.7000901@oracle.com> <264E7F8B-E421-4D91-88D9-A22EF0366F2A@oracle.com> <5C039E72-638F-494D-8A78-96ADEFFC267B@oracle.com> <530DE472.80405@gmail.com> Message-ID: <530F0E61.9050507@oracle.com> On 26/02/2014 10:56 PM, Peter Levart wrote: > On 02/25/2014 09:38 PM, Brian Burkhalter wrote: >> On Feb 20, 2014, at 1:42 AM, Paul Sandoz wrote: >> >>> Not sure the static powerCache field, in the original code, needs to >>> be volatile either: >>> >>> 1137 private static volatile BigInteger[][] powerCache; >> Is there consensus on whether "volatile" is necessary here? > > I think it has to be volatile. The powerCache implementation was added > in the following changeset: Definitely has to be volatile for current code. I don't know what I was looking at when I said it didn't need to be - I didn't see the code that grows the cache. David ----- > http://hg.openjdk.java.net/jdk9/dev/jdk/rev/7546 > > ...and improved later in the following: > > http://hg.openjdk.java.net/jdk9/dev/jdk/rev/7586 > > It uses a copy-on-write technique to extend the cache with new values > when needed. volatile is mandatory here to safely publish the newly > constructed array-of-arrays and the newly constructed sub-array to other > threads. Without volatile, other threads could see null slots where > BigInteger[] and/or BigInteger objects should be... > > Regards, Peter > >> >> Thanks, >> >> Brian > From paul.sandoz at oracle.com Thu Feb 27 10:37:31 2014 From: paul.sandoz at oracle.com (Paul Sandoz) Date: Thu, 27 Feb 2014 11:37:31 +0100 Subject: Unsafe: efficiently comparing two byte arrays In-Reply-To: References: Message-ID: On Feb 26, 2014, at 6:21 PM, Martin Buchholz wrote: > Every once in a while I see an attempt to introduce a "general purpose" unsafe array class, but efforts stall due to: Yes, in general this is "arrays 2.0" stuff, which is tricky especially since on and off-heap seem to have conflicting requirements (the latter might require some form of explicit layout where as for the former it would be better to let hotspot work out what to do with better GC mechanisms). Mostly in the context of just comparing byte arrays: > - it's not obvious whether unaligned access to e.g. 8 bytes via long is even possible or more efficient than just reading 8 bytes > - endianness is an issue In Guava's implementation there is an optimization for big endian natural byte order and a clever trick otherwise: for (int i = 0; i < minWords * Longs.BYTES; i += Longs.BYTES) { long lw = theUnsafe.getLong(left, BYTE_ARRAY_BASE_OFFSET + (long) i); long rw = theUnsafe.getLong(right, BYTE_ARRAY_BASE_OFFSET + (long) i); if (lw != rw) { if (BIG_ENDIAN) { return UnsignedLongs.compare(lw, rw); } /* * We want to compare only the first index where left[index] != right[index]. * This corresponds to the least significant nonzero byte in lw ^ rw, since lw * and rw are little-endian. Long.numberOfTrailingZeros(diff) tells us the least * significant nonzero bit, and zeroing out the first three bits of L.nTZ gives us the * shift to get that least significant nonzero byte. */ int n = Long.numberOfTrailingZeros(lw ^ rw) & ~0x7; return (int) (((lw >>> n) & UNSIGNED_MASK) - ((rw >>> n) & UNSIGNED_MASK)); } } > - for best performance, you also want to elide those pesky array bound checks (but hotspot can do a better job of that) > No such checks are provided by Unsafe :-) For loops a check can often be hoisted out (i think hotspot already can do that). Recently there have been some optimisations implemented for "(i & (a.length - 1))" patterns (e.g. see HashMap). But there are always gonna be cases where it cannot fully remove such checks and people reach for Unsafe to save some cycles. > I think it's worth doing, but it's harder than it looks, and probably needs help from the VM via intrinsics. > > --- > > lexicographicalComparator saves a lot of cycles. > Yes, in this context we can provide such methods and over time they could be modified to benefit from any future improvements. Just trying to whittle down small use-cases as to why Unsafe is currently used today. Paul. From paul.sandoz at oracle.com Thu Feb 27 10:40:03 2014 From: paul.sandoz at oracle.com (Paul Sandoz) Date: Thu, 27 Feb 2014 11:40:03 +0100 Subject: Unsafe: removing the monitorEnter/monitorExit/tryMonitorEnter methods In-Reply-To: <530E2E23.30506@oracle.com> References: <530E2E23.30506@oracle.com> Message-ID: <9530088F-AE3D-4D7A-A158-D0545A6E147E@oracle.com> On Feb 26, 2014, at 7:10 PM, Brian Goetz wrote: > Seems like a symbolic victory, at least :) I'll take what ever small victory i can with this beast of a class :-) > It's less unsafe than some unsafe methods (you can use it to create a DoS but not to violate safety constraints like bounds checking or pointer casting) but its a start! > Indeed! Paul. From scolebourne at joda.org Thu Feb 27 11:12:48 2014 From: scolebourne at joda.org (Stephen Colebourne) Date: Thu, 27 Feb 2014 11:12:48 +0000 Subject: Unsafe: removing the monitorEnter/monitorExit/tryMonitorEnter methods In-Reply-To: References: Message-ID: On 26 February 2014 20:54, Martin Buchholz wrote: > It does seem that being able to tell whether a java object monitor is > currently locked is useful for debugging and monitoring - there should be a > way to do that. Perhaps it would be useful to be able to expose a java object monitor as an instance of Lock? Lock lk = Lock.ofMonitor(object) if (lk.tryLock()) { ... } Such a method feels like it would be a useful missing link between synchronized and locks. Stephen From peter.levart at gmail.com Thu Feb 27 11:52:16 2014 From: peter.levart at gmail.com (Peter Levart) Date: Thu, 27 Feb 2014 12:52:16 +0100 Subject: Unsafe: removing the monitorEnter/monitorExit/tryMonitorEnter methods In-Reply-To: <530F0C40.7070303@oracle.com> References: <530EE93D.7090406@gmail.com> <530EE9A3.4080101@gmail.com> <530EEC46.40605@oracle.com> <530EF9A3.1000108@gmail.com> <530F0C40.7070303@oracle.com> Message-ID: <530F26F0.3090204@gmail.com> On 02/27/2014 10:58 AM, David Holmes wrote: > On 27/02/2014 6:38 PM, Peter Levart wrote: >> On 02/27/2014 08:41 AM, David Holmes wrote: >>> On 27/02/2014 5:30 PM, Peter Levart wrote: >>>> On 02/27/2014 08:29 AM, Peter Levart wrote: >>>>> On 02/26/2014 09:54 PM, Martin Buchholz wrote: >>>>>> I don't recall having added this particular wart >>>>>> to test/java/lang/ProcessBuilder/Basic.java, and history suggests >>>>>> that >>>>>> others did that. >>>>>> >>>>>> It does seem that being able to tell whether a java object >>>>>> monitor is >>>>>> currently locked is useful for debugging and monitoring - there >>>>>> should be a >>>>>> way to do that. >>>>> >>>>> Thread.holdsLock(Object) ? >>>> >>>> Ah, you meant to query from some other thread, right? >>> >>> Right - that is the usage in Basic.java but I'm still scratching my >>> head trying to understand what finding the BufferedInputStream locked >>> is supposed to signify. >> >> Here's the relevant code block: >> >> while (unsafe.tryMonitorEnter(s)) { >> unsafe.monitorExit(s); >> Thread.sleep(1); >> } >> >> This code block waits until it finds that some thread has acquired the >> lock on 's' for at least 1 ms (or else it can miss it and will wait >> further) ... >> >> Since this block is executed for s instanceof BufferedInputStream, what >> this block does, is wait until one of synchronized methods on >> BufferedInputStream is called. The thread that does that, calls read() >> or read(byte[]) and blocks while holding a lock, since the child process >> that is spawned and whose stdout the thread is trying to read in this >> situation does a 10 minute sleep. > > Ah I see - thanks for the analysis. So this is a classic 'barrier' > waiting for the other thread to have reached the synchronized method. > >> I think the following could be used instead for the same purpose: >> >> while (!isLocked(s, 100L)) { >> Thread.sleep(1); >> } >> >> // using the following utility method: >> >> static boolean isLocked(final Object monitor, long millis) throws >> InterruptedException { >> Thread t = new Thread() { >> @Override >> public void run() { >> synchronized (monitor) { } >> } >> }; >> t.start(); >> t.join(millis); >> return t.isAlive(); >> } > > Functionally yes this will only return true if the object is locked > for >100ms. But it is potentially going to generate a lot of temporary > threads. And in theory at least the JIT can optimize the run method > into nothing. Ok, then what about the following: static boolean isLocked(final Object monitor, final long millis) throws InterruptedException { return new Thread() { volatile boolean unlocked; @Override public void run() { synchronized (monitor) { unlocked = true; } } boolean isLocked() throws InterruptedException { start(); join(millis); return !unlocked; } }.isLocked(); } // and usage it like: while (!isLocked(s, 100L)) { Thread.sleep(100L); } This will allocate about 10 Thread objects per second until the other thread finally reaches the synchronized read() method in BufferedInputStream... Regards, Peter > > Still it is a cute trick :) > > David > >> >> Regards, Peter >> >>> >>> David >>> ----- >>> >>>> Peter >>>> >>>>> >>>>> Regards, Peter >>>>> >>>>>> >>>>>> >>>>>> On Wed, Feb 26, 2014 at 7:12 AM, Paul Sandoz >>>>>> >>>>>> wrote: >>>>>> >>>>>>> Hi, >>>>>>> >>>>>>> Out of all the methods on Unsafe i think the >>>>>>> monitorEnter/monitorExit/tryMonitorEnter are the least used and are >>>>>>> very >>>>>>> strong candidates for removal. >>>>>>> >>>>>>> 99% of use-cases are supported by classes in the >>>>>>> java.util.concurrent.locks package. >>>>>>> >>>>>>> >>>>>>> Within the JDK itself it is only used in one JDK test file >>>>>>> test/java/lang/ProcessBuilder/Basic.java: >>>>>>> >>>>>>> while (unsafe.tryMonitorEnter(s)) { >>>>>>> unsafe.monitorExit(s); >>>>>>> Thread.sleep(1); >>>>>>> } >>>>>>> >>>>>>> for a test verifying an EOF is received on pending reads and it is >>>>>>> polling >>>>>>> to check when the process builder acquires the lock before >>>>>>> destroying the >>>>>>> process, presumably to avoid some sort of race condition that >>>>>>> occasionally >>>>>>> causes the test to fail. >>>>>>> >>>>>>> I believe this test as been through a number of rounds, i stared >>>>>>> at public static boolean isLocked(final Object monitor, >>>>>>> final long millis) throws InterruptedException { >>>>>>> return new Thread() { >>>>>>> volatile boolean unlocked; >>>>>>> >>>>>>> @Override >>>>>>> public void run() { >>>>>>> synchronized (monitor) { unlocked = true; } >>>>>>> } >>>>>>> >>>>>>> boolean isLocked() throws InterruptedException { >>>>>>> start(); >>>>>>> join(millis); >>>>>>> return !unlocked; >>>>>>> } >>>>>>> }.isLocked(); >>>>>>> } >>>>>>> >>>>>>> things >>>>>>> for a bit, but cannot quickly work out a replacement; i lack the >>>>>>> knowledge >>>>>>> on this area. >>>>>>> >>>>>>> >>>>>>> Outside of the JDK i can only find one usage of monitorExit/Enter >>>>>>> (according to grep code) in JBoss modules, and i believe this is >>>>>>> only used >>>>>>> on Java 1.6 to work around some limitations in class loading that >>>>>>> were >>>>>>> fixed in 1.7. >>>>>>> >>>>>>> >>>>>>> Given such very limited use i propose to remove these methods after >>>>>>> having >>>>>>> worked out a fix for ProcessBuilder/Basic.java test. >>>>>>> >>>>>>> Paul. >>>>>>> >>>>> >>>> >> From paul.sandoz at oracle.com Thu Feb 27 12:29:34 2014 From: paul.sandoz at oracle.com (Paul Sandoz) Date: Thu, 27 Feb 2014 13:29:34 +0100 Subject: Unsafe: removing the monitorEnter/monitorExit/tryMonitorEnter methods In-Reply-To: <530F26F0.3090204@gmail.com> References: <530EE93D.7090406@gmail.com> <530EE9A3.4080101@gmail.com> <530EEC46.40605@oracle.com> <530EF9A3.1000108@gmail.com> <530F0C40.7070303@oracle.com> <530F26F0.3090204@gmail.com> Message-ID: On Feb 27, 2014, at 12:52 PM, Peter Levart wrote: > > Ok, then what about the following: > > static boolean isLocked(final Object monitor, final long millis) throws InterruptedException { > return new Thread() { > volatile boolean unlocked; > > @Override > public void run() { > synchronized (monitor) { unlocked = true; } > } > > boolean isLocked() throws InterruptedException { > start(); > join(millis); > return !unlocked; > } > }.isLocked(); > } > > > // and usage it like: > > while (!isLocked(s, 100L)) { > Thread.sleep(100L); > } > > > This will allocate about 10 Thread objects per second until the other thread finally reaches the synchronized read() method in BufferedInputStream... > I like it. I do have a slight preference for Mandy's solution, see below, but i am not sure we can guarantee it is supported on all test platforms. Paul. try { while (!isMonitorLockHeld(monitor)) { Thread.sleep(1); } } catch (UnsupportedOperationException e) { // Back up plan? Thread.sleep(TimeUnit.SECONDS.toMillis(1)); } /** * Returns {@code true} if and only if the a thread holds the * monitor lock on the specified object. * * @throws java.lang.UnsupportedOperationException if monitoring of * object monitor use is not supported */ public static boolean isMonitorLockHeld(Object o) { int oihc = System.identityHashCode(o); ThreadMXBean tmxbean = ManagementFactory.getThreadMXBean(); ThreadInfo[] tinfos = tmxbean.dumpAllThreads(true, false); for (ThreadInfo ti : tinfos) { for (MonitorInfo mi : ti.getLockedMonitors()) { if (mi.getIdentityHashCode() == oihc) return true; } } return false; } From david.holmes at oracle.com Thu Feb 27 12:38:25 2014 From: david.holmes at oracle.com (David Holmes) Date: Thu, 27 Feb 2014 22:38:25 +1000 Subject: Unsafe: removing the monitorEnter/monitorExit/tryMonitorEnter methods In-Reply-To: References: Message-ID: <530F31C1.8000604@oracle.com> On 27/02/2014 9:12 PM, Stephen Colebourne wrote: > On 26 February 2014 20:54, Martin Buchholz wrote: >> It does seem that being able to tell whether a java object monitor is >> currently locked is useful for debugging and monitoring - there should be a >> way to do that. > > Perhaps it would be useful to be able to expose a java object monitor > as an instance of Lock? > > Lock lk = Lock.ofMonitor(object) > if (lk.tryLock()) { > ... > } > > Such a method feels like it would be a useful missing link between > synchronized and locks. I don't think this is workable. Exposing a monitor as Lock would allow you to break the guarantees/requirements involving balanced-locking for monitors. David > Stephen > From jeroen at sumatra.nl Thu Feb 27 13:39:08 2014 From: jeroen at sumatra.nl (Jeroen Frijters) Date: Thu, 27 Feb 2014 13:39:08 +0000 Subject: Unsafe: removing the monitorEnter/monitorExit/tryMonitorEnter methods In-Reply-To: <530F31C1.8000604@oracle.com> References: <530F31C1.8000604@oracle.com> Message-ID: David Holmes wrote: > I don't think this is workable. Exposing a monitor as Lock would allow > you to break the guarantees/requirements involving balanced-locking for > monitors. Where are these guarantees/requirements documented? Regards, Jeroen From peter.levart at gmail.com Thu Feb 27 13:50:22 2014 From: peter.levart at gmail.com (Peter Levart) Date: Thu, 27 Feb 2014 14:50:22 +0100 Subject: JDK 9 RFR of JDK-8035279: Clean up internal deprecations in BigInteger In-Reply-To: References: <5305B914.7000901@oracle.com> <264E7F8B-E421-4D91-88D9-A22EF0366F2A@oracle.com> <5C039E72-638F-494D-8A78-96ADEFFC267B@oracle.com> <0D061461-C44C-4EBB-8848-E0647C374A34@oracle.com> <0A4793AC-509F-4183-94B8-1DD2ABD7D2DC@oracle.com> <41862830-7AC0-4B37-B947-542532AE63B9@oracle.com> Message-ID: <530F429E.6040204@gmail.com> On 02/27/2014 09:22 AM, Paul Sandoz wrote: > On Feb 27, 2014, at 12:51 AM, Brian Burkhalter wrote: > >> On Feb 26, 2014, at 1:53 PM, Paul Sandoz wrote: >> >>>> Thanks for the suggestion, Paul. Assuming it is correct, in what way would this be a better approach? (I apologize for being obtuse.) >>>> >>> IMHO it is a simpler solution. It's a cache line that requires updating not the cache line*s*, as the current approach can potentially knock out cache lines for other radixes. Potentially yes, but not very likely, since the whole powerCache reference is re-read again *after* the computation of cacheLine and then replaced with a modified clone immediately after that. It should be noted what Alan Eliasen already pointed-out in a discussion back in June 2013: http://mail.openjdk.java.net/pipermail/core-libs-dev/2013-June/018375.html ...so I think what could be improved in current implementation is the potential and more likely useless computation of the same values for the same radix by multiple concurrent threads. Back then I presented an alternative solution which solves that: http://mail.openjdk.java.net/pipermail/core-libs-dev/2013-June/018457.html ...and as can be seen, does not use volatile read/write to publish BigIntegers - that's why it is important that internal implementation of such immutable classes like BigInteger is tolerant to unsafe publication... Regards, Peter > > I forgot to mention a slightly more complex variation is to CAS the cache line instead of a lazy set, which can ensure a line does not knock another containing a higher exponent. > > Paul. From joel.franck at oracle.com Thu Feb 27 13:44:11 2014 From: joel.franck at oracle.com (Joel Borggren-Franck) Date: Thu, 27 Feb 2014 14:44:11 +0100 Subject: [7u backport] RFR: 7122142: (ann) Race condition between isAnnotationPresent and getAnnotations In-Reply-To: <530BB2F6.2000507@oracle.com> References: <53089694.4070308@oracle.com> <530B3E32.7020303@gmail.com> <530BB2F6.2000507@oracle.com> Message-ID: <20140227134411.GH4321@oracle.com> Hi, I looked at webrev.1. Looks good. cheers /Joel On 2014-02-25, dmeetry degrave wrote: > Thanks for looking at this, Peter! > > On 02/24/2014 04:42 PM, Peter Levart wrote: > >Hi Dmeetry, > > > >On 02/22/2014 01:22 PM, dmeetry degrave wrote: > >>Hi all, > >> > >>I would like to ask for a review of combined back port for > >>7u-dev/7u80. The main goal is to have a fix for 7122142 in jdk7, it > >>also integrates the changes from 8005232, 7185456, 8022721 > >> > >>https://bugs.openjdk.java.net/browse/JDK-7122142 > >>https://bugs.openjdk.java.net/browse/JDK-8005232 > >>https://bugs.openjdk.java.net/browse/JDK-7185456 > >>https://bugs.openjdk.java.net/browse/JDK-8022721 > >> > >>Original jdk8 changes: > >> > >>7122142: http://hg.openjdk.java.net/jdk8/tl/jdk/rev/e4ce6502eac0 > >>8005232: http://hg.openjdk.java.net/jdk8/tl/jdk/rev/1109bfff4e92 > >>7185456: http://hg.openjdk.java.net/jdk8/tl/jdk/rev/ae03282ba501 > >>8022721: http://hg.openjdk.java.net/jdk8/tl/jdk/rev/2281a7f79738 > >> > >>back port: > >> > >>http://cr.openjdk.java.net/~dmeetry/7122142.8005232.7185456.8022721/webrev.0/ > >> > >> > >>Patches can't be applied cleanly, hence it was a manual back port, > >>though the final result is equivalent to applying the patches in > >>chronological order (8005232, 7185456, 7122142, 8022721) and applying > >>all the relevant rejected parts > > > >It's good to see those patches being back-ported to 7u. By browsing the > >webrev, I don't see any obvious difference between the original patches > >and the backport. > > there shouldn't be any! > > >Do you happen to remember in what part of code there > >were rejects so that you had to manually apply the changes? > > there were conflicts due to small difference between 7 and 8 > (copyrights, white spaces, @SuppressWarnings, Class,...). > > I copied all rejected parts and original patches here: > > http://cr.openjdk.java.net/~dmeetry/7122142.8005232.7185456.8022721/webrev.1/rej/ > > >>(with one exception, AnnotationTypeRuntimeAssumptionTest.java test was > >>not included due to jdk8 API). > > > >Ah, It's the Class.getDeclaredAnnotation(Class) that's new in JDK8. > >Here's the changed test that only uses the JDK7 API so you can include > >this test too: > > > >http://cr.openjdk.java.net/~plevart/jdk7u/7122142/AnnotationTypeRuntimeAssumptionTest.java > > Thanks! > > http://cr.openjdk.java.net/~dmeetry/7122142.8005232.7185456.8022721/webrev.1/ > > (just with the new test added). > > thanks, > dmeetry > >> > >>All tests in test/java/lang/annotation passed. > >> > >>thanks, > >>dmeetry > > > >Regards, Peter > > From vitalyd at gmail.com Thu Feb 27 13:56:32 2014 From: vitalyd at gmail.com (Vitaly Davidovich) Date: Thu, 27 Feb 2014 08:56:32 -0500 Subject: Unsafe: removing the monitorEnter/monitorExit/tryMonitorEnter methods In-Reply-To: References: <530F31C1.8000604@oracle.com> Message-ID: Right, there's no such guarantee/requirement. JVMS mentions that implementations are permitted to enforce balance (called structured locking there) but not required: http://docs.oracle.com/javase/specs/jvms/se7/html/jvms-2.html#jvms-2.11.10 Thanks Sent from my phone On Feb 27, 2014 8:40 AM, "Jeroen Frijters" wrote: > David Holmes wrote: > > I don't think this is workable. Exposing a monitor as Lock would allow > > you to break the guarantees/requirements involving balanced-locking for > > monitors. > > Where are these guarantees/requirements documented? > > Regards, > Jeroen > > From fweimer at redhat.com Thu Feb 27 14:09:27 2014 From: fweimer at redhat.com (Florian Weimer) Date: Thu, 27 Feb 2014 15:09:27 +0100 Subject: Unsafe: efficiently comparing two byte arrays In-Reply-To: References: Message-ID: <530F4717.10001@redhat.com> On 02/27/2014 11:37 AM, Paul Sandoz wrote: > /* > * We want to compare only the first index where left[index] != right[index]. > * This corresponds to the least significant nonzero byte in lw ^ rw, since lw > * and rw are little-endian. Long.numberOfTrailingZeros(diff) tells us the least > * significant nonzero bit, and zeroing out the first three bits of L.nTZ gives us the > * shift to get that least significant nonzero byte. > */ > int n = Long.numberOfTrailingZeros(lw ^ rw) & ~0x7; > return (int) (((lw >>> n) & UNSIGNED_MASK) - ((rw >>> n) & UNSIGNED_MASK)); This really depends on the microarchitecture, but for many current CPUs, conversion to big endian with Long::reverseBytes(long) will be faster (assuming that it's intrinsified). This is not just a performance optimization, it can also be used to plug an alleged timing oracle: -- Florian Weimer / Red Hat Product Security Team From david.lloyd at redhat.com Thu Feb 27 14:24:36 2014 From: david.lloyd at redhat.com (David M. Lloyd) Date: Thu, 27 Feb 2014 08:24:36 -0600 Subject: Unsafe: removing the monitorEnter/monitorExit/tryMonitorEnter methods In-Reply-To: References: <530F31C1.8000604@oracle.com> Message-ID: <530F4AA4.4070507@redhat.com> This is actually a somewhat unfortunate "feature". Had this not been in place, I could have just used custom bytecodes to perform the lock magic that I ended up using Unsafe for (I'm the guy that wrote JBoss Modules, i.e. apparently the one usage "in the wild" of these methods). The restriction is kind of pointless anyway as it's not really "unsafe", it just can lead to buggy code. And given that the only way to utilize those two particular codes (monitorenter/monitorexit) from Java is via "synchronized", which is necessarily balanced, the restriction seems to impose needless limitation. On 02/27/2014 07:56 AM, Vitaly Davidovich wrote: > Right, there's no such guarantee/requirement. JVMS mentions that > implementations are permitted to enforce balance (called structured locking > there) but not required: > http://docs.oracle.com/javase/specs/jvms/se7/html/jvms-2.html#jvms-2.11.10 > > Thanks > > Sent from my phone > On Feb 27, 2014 8:40 AM, "Jeroen Frijters" wrote: > >> David Holmes wrote: >>> I don't think this is workable. Exposing a monitor as Lock would allow >>> you to break the guarantees/requirements involving balanced-locking for >>> monitors. >> >> Where are these guarantees/requirements documented? >> >> Regards, >> Jeroen >> >> -- - DML From heinz at javaspecialists.eu Thu Feb 27 14:51:06 2014 From: heinz at javaspecialists.eu (Dr Heinz M. Kabutz) Date: Thu, 27 Feb 2014 16:51:06 +0200 Subject: Unsafe: removing the monitorEnter/monitorExit/tryMonitorEnter methods In-Reply-To: References: Message-ID: <530F50DA.3010308@javaspecialists.eu> I have never used this "in the wild", but rather have moved over to the ReentrantLock when I needed that particular functionality. However, I do see a place for this ability. As I wrote in here: http://www.javaspecialists.eu/archive/Issue194.html - sometimes you need to modify code that is already using a particular locking mechanism. To then redo everything with ReentrantLock can introduce errors. If I had a say, I would vote to either keep that method in Unsafe (which is where I think it belongs) or provide an alternative way to make the tryMonitorEnter() mechanism available to those that might end up needing it. Since I bought my Suzuki Jimny 7 years ago, I have not needed my spare wheel even once. So yeah, I could take it off and drive around without it. But chances are, the day after I take it off, I will need it. tryMonitorEnter() does no harm and it is out of reach of most programmers. Regards Heinz -- Dr Heinz M. Kabutz (PhD CompSci) Author of "The Java(tm) Specialists' Newsletter" Oracle Java Champion 2005-2013 JavaOne Rock Star Speaker 2012 http://www.javaspecialists.eu Tel: +30 69 75 595 262 Skype: kabutz Paul Sandoz wrote: > Hi, > > Out of all the methods on Unsafe i think the monitorEnter/monitorExit/tryMonitorEnter are the least used and are very strong candidates for removal. > > 99% of use-cases are supported by classes in the java.util.concurrent.locks package. > > > Within the JDK itself it is only used in one JDK test file test/java/lang/ProcessBuilder/Basic.java: > > while (unsafe.tryMonitorEnter(s)) { > unsafe.monitorExit(s); > Thread.sleep(1); > } > > for a test verifying an EOF is received on pending reads and it is polling to check when the process builder acquires the lock before destroying the process, presumably to avoid some sort of race condition that occasionally causes the test to fail. > > I believe this test as been through a number of rounds, i stared at things for a bit, but cannot quickly work out a replacement; i lack the knowledge on this area. > > > Outside of the JDK i can only find one usage of monitorExit/Enter (according to grep code) in JBoss modules, and i believe this is only used on Java 1.6 to work around some limitations in class loading that were fixed in 1.7. > > > Given such very limited use i propose to remove these methods after having worked out a fix for ProcessBuilder/Basic.java test. > > Paul. > From aph at redhat.com Thu Feb 27 14:52:38 2014 From: aph at redhat.com (Andrew Haley) Date: Thu, 27 Feb 2014 14:52:38 +0000 Subject: Unsafe: efficiently comparing two byte arrays In-Reply-To: References: Message-ID: <530F5136.20405@redhat.com> Hi, On 02/26/2014 03:42 PM, Paul Sandoz wrote: > A common reason why Unsafe is used is to more efficiently compare two unsigned byte arrays, viewing those byte arrays as long arrays. Why not simply provide ByteBuffer.forArray(byte[]) ? Then we'd have all the ByteBuffer intrinsics for put() and get() operations. Andrew. From paul.sandoz at oracle.com Thu Feb 27 15:05:06 2014 From: paul.sandoz at oracle.com (Paul Sandoz) Date: Thu, 27 Feb 2014 16:05:06 +0100 Subject: Unsafe: efficiently comparing two byte arrays In-Reply-To: <530F5136.20405@redhat.com> References: <530F5136.20405@redhat.com> Message-ID: <8D3E2037-E5D1-4D2E-B779-E7EDD2AF4F58@oracle.com> On Feb 27, 2014, at 3:52 PM, Andrew Haley wrote: > Hi, > > On 02/26/2014 03:42 PM, Paul Sandoz wrote: > >> A common reason why Unsafe is used is to more efficiently compare two unsigned byte arrays, viewing those byte arrays as long arrays. > > Why not simply provide ByteBuffer.forArray(byte[]) ? Then we'd have all the > ByteBuffer intrinsics for put() and get() operations. > Good point, even if it does require the creation of two ByteBuffer instances. An implementation of Arrays.compare could do that if things were suitably intrinsified, which i am not sure is the case as of today. I will do some measurements and report back. Paul. From paul.sandoz at oracle.com Thu Feb 27 15:06:14 2014 From: paul.sandoz at oracle.com (Paul Sandoz) Date: Thu, 27 Feb 2014 16:06:14 +0100 Subject: JDK 9 RFR of JDK-8035279: Clean up internal deprecations in BigInteger In-Reply-To: <530F429E.6040204@gmail.com> References: <5305B914.7000901@oracle.com> <264E7F8B-E421-4D91-88D9-A22EF0366F2A@oracle.com> <5C039E72-638F-494D-8A78-96ADEFFC267B@oracle.com> <0D061461-C44C-4EBB-8848-E0647C374A34@oracle.com> <0A4793AC-509F-4183-94B8-1DD2ABD7D2DC@oracle.com> <41862830-7AC0-4B37-B947-542532AE63B9@oracle.com> <530F429E.6040204@gmail.com> Message-ID: <3DCE5A13-1121-4A71-93ED-1F11611E94E5@oracle.com> On Feb 27, 2014, at 2:50 PM, Peter Levart wrote: > On 02/27/2014 09:22 AM, Paul Sandoz wrote: >> On Feb 27, 2014, at 12:51 AM, Brian Burkhalter wrote: >> >>> On Feb 26, 2014, at 1:53 PM, Paul Sandoz wrote: >>> >>>>> Thanks for the suggestion, Paul. Assuming it is correct, in what way would this be a better approach? (I apologize for being obtuse.) >>>>> >>>> IMHO it is a simpler solution. It's a cache line that requires updating not the cache line*s*, as the current approach can potentially knock out cache lines for other radixes. > > Potentially yes, but not very likely, since the whole powerCache reference is re-read again *after* the computation of cacheLine and then replaced with a modified clone immediately after that. It should be noted what Alan Eliasen already pointed-out in a discussion back in June 2013: > > http://mail.openjdk.java.net/pipermail/core-libs-dev/2013-June/018375.html > Yes, i was suggesting a CAS could be used to avoid replacing with a smaller cache line: while (!UnsafeHolder.casCacheLine(powerCache, radix, cacheLine, newCacheLine)) { cacheLine = powerCache[radix]; if (exponent < cacheLine.length) { break; } } but it can still result in two or more threads duplicating work as you say below, dunno how critical that is. I suppose if i was super paranoid about this i write some initialization code to warm the cache up :-) also being super paranoid perhaps there should be a limit of exponents that are cached? > ...so I think what could be improved in current implementation is the potential and more likely useless computation of the same values for the same radix by multiple concurrent threads. > Back then I presented an alternative solution which solves that: > > http://mail.openjdk.java.net/pipermail/core-libs-dev/2013-June/018457.html > > ...and as can be seen, does not use volatile read/write to publish BigIntegers - that's why it is important that internal implementation of such immutable classes like BigInteger is tolerant to unsafe publication... > Interesting, nice use of a linked list. Not sure that is safe publication of cacheLine into powerCache, the last element in a line could be observed as null, plus although it is unlikely, the compiler could shuffle up the array element store? Paul. From david.lloyd at redhat.com Thu Feb 27 15:11:44 2014 From: david.lloyd at redhat.com (David M. Lloyd) Date: Thu, 27 Feb 2014 09:11:44 -0600 Subject: Unsafe: removing the monitorEnter/monitorExit/tryMonitorEnter methods In-Reply-To: <530F50DA.3010308@javaspecialists.eu> References: <530F50DA.3010308@javaspecialists.eu> Message-ID: <530F55B0.1080907@redhat.com> A "Monitors" class could possibly be introduced which simply has three (intrinsic) static methods: void lock(Object obj), void unlock(Object obj), boolean tryLock(Object obj). Could even add a void lockInterruptibly(Object obj) throws InterruptedException. Why not? Monitors are not going anywhere, and they're considerably lighter than ReentrantLock in terms of memory usage and allocations in the (pretty common) case where you only need zero or one conditions. Avoiding something like this out of loyalty to new APIs is probably pretty misguided. On 02/27/2014 08:51 AM, Dr Heinz M. Kabutz wrote: > I have never used this "in the wild", but rather have moved over to the > ReentrantLock when I needed that particular functionality. > > However, I do see a place for this ability. As I wrote in here: > http://www.javaspecialists.eu/archive/Issue194.html - sometimes you need > to modify code that is already using a particular locking mechanism. To > then redo everything with ReentrantLock can introduce errors. > > If I had a say, I would vote to either keep that method in Unsafe (which > is where I think it belongs) or provide an alternative way to make the > tryMonitorEnter() mechanism available to those that might end up needing > it. > > Since I bought my Suzuki Jimny 7 years ago, I have not needed my spare > wheel even once. So yeah, I could take it off and drive around without > it. But chances are, the day after I take it off, I will need it. > > tryMonitorEnter() does no harm and it is out of reach of most programmers. > > Regards > > Heinz -- - DML From aph at redhat.com Thu Feb 27 15:12:10 2014 From: aph at redhat.com (Andrew Haley) Date: Thu, 27 Feb 2014 15:12:10 +0000 Subject: Unsafe: efficiently comparing two byte arrays In-Reply-To: <8D3E2037-E5D1-4D2E-B779-E7EDD2AF4F58@oracle.com> References: <530F5136.20405@redhat.com> <8D3E2037-E5D1-4D2E-B779-E7EDD2AF4F58@oracle.com> Message-ID: <530F55CA.3060302@redhat.com> On 02/27/2014 03:05 PM, Paul Sandoz wrote: > > On Feb 27, 2014, at 3:52 PM, Andrew Haley wrote: > >> Hi, >> >> On 02/26/2014 03:42 PM, Paul Sandoz wrote: >> >>> A common reason why Unsafe is used is to more efficiently compare two unsigned byte arrays, viewing those byte arrays as long arrays. >> >> Why not simply provide ByteBuffer.forArray(byte[]) ? Then we'd have all the ByteBuffer intrinsics for put() and get() operations. > > Good point, even if it does require the creation of two ByteBuffer instances. > > An implementation of Arrays.compare could do that if things were suitably intrinsified, which i am not sure is the case as of today. > > I will do some measurements and report back. Unless something has been done in HotSpot pretty recently, the ByteBuffer intrinsics are not as performat as they might be, so the results might be a little underwhelming. Andrew. From roger.riggs at oracle.com Thu Feb 27 15:34:01 2014 From: roger.riggs at oracle.com (roger riggs) Date: Thu, 27 Feb 2014 10:34:01 -0500 Subject: RFR 9: 8035889: jdk testlibrary - add printing of values of failed assertions In-Reply-To: <530EAE5F.3050308@oracle.com> References: <530E4FD8.9040605@oracle.com> <530E9C6C.8080409@oracle.com> <530EAC78.4000301@Oracle.com> <530EAE5F.3050308@oracle.com> Message-ID: <530F5AE9.8000405@oracle.com> Hi Mandy, I updated the webrev: http://cr.openjdk.java.net/~rriggs/webrev-testlibrary-asserts-8035889/ Alan suggested copying serviceability-dev so they have a chance to review if desired. I want to investigate if it is possible to use the TestNG Assert classes without the TestNG execution framework. It would be necessary to compile/run against TestNG.jar but it might not need the entire mechanism. Thanks, Roger On 2/26/2014 10:17 PM, Mandy Chung wrote: > On 2/26/2014 7:09 PM, Roger Riggs wrote: >> Hi Mandy, >> >> Yes, it might be more productive to switch the tests to TestNG. >> But it did provide support in cases where TestNG could not be used, >> for example in a directory of existing tests that had custom reporting. >> >> But I remember there is a problem with TestNG having a dependency for XML >> which is not supported in Profile1 and a number of tests had to be >> disabled >> in that configuration. Will XML always be available. Do we need to >> solve >> or work around that problem with TestNG? >> > > This is a good point. When we want to test just the base module for > example, how can we run TestNG tests? We need to address that certainly. > > My comment on TestNG is a question for new tests using this Asserts > class. Your patch is fine to go (after taking out @library tag if I > got it correct). > > Mandy > >> Thanks, Roger >> >> On 2/26/14 9:01 PM, Mandy Chung wrote: >>> Hi Roger, >>> >>> On 2/26/2014 12:34 PM, roger riggs wrote: >>>> The testlibrary for the jdk should be printing the values in the >>>> failed >>>> assertions to make debugging easier and quicker. >>>> >>>> The webrev adds the printing of the failed assertions and added >>>> methods >>>> for formatting and unconditional fail methods. >>>> >>>> Webrev: >>>> http://cr.openjdk.java.net/~rriggs/webrev-testlibrary-asserts-8035889/ >>>> >>> >>> AssertsTest.java: line 28: @library doesn't look like it's needed. >>> There is no jdk/test/testlibrary directory and I think >>> jdk.testlibrary.* are found as relative to $test.src. >>> >>> Otherwise, the change looks okay. >>> >>> Now that jtreg supports TestNG and I wonder if this class should >>> retire some day (there are only about 10 existing tests using this >>> class). Are you writing new tests using this Asserts class? >>> >>> Mandy >>> >>>> Bug: >>>> 8035889: jdk testlibrary - add printing of values of failed >>>> assertions >>>> >>>> Thanks, Roger >>>> >>>> [1] https://bugs.openjdk.java.net/browse/JDK-8035889 >>>> >>>> >>> >> > From roger.riggs at oracle.com Thu Feb 27 15:38:44 2014 From: roger.riggs at oracle.com (roger riggs) Date: Thu, 27 Feb 2014 10:38:44 -0500 Subject: RFR [6943190] TEST_BUG: java/lang/Runtime/exec/ExecWithInput.java hardcodes path to cat In-Reply-To: <530F06A3.1030107@oracle.com> References: <530B80CF.9090403@oracle.com> <530E5D0F.90409@oracle.com> <530E6877.5050504@oracle.com> <530F06A3.1030107@oracle.com> Message-ID: <530F5C04.5030504@oracle.com> Hi Ivan, On 2/27/2014 4:34 AM, Ivan Gerasimov wrote: >> - I'm not sure the specific command methods do you any good >> (over a method with a string argument). since they immediately >> revert to the string command name. >> > They are shorter :-) yes, but it creates a level of indirection that someone reading the code has to chase through. To me it would create less code and make for easier reading to call findCommand in all cases. $.02, Roger ... > > Please find the updated webrev here: > http://cr.openjdk.java.net/~igerasim/6943190/2/webrev/ > > Sincerely yours, > Ivan > From paul.sandoz at oracle.com Thu Feb 27 17:23:28 2014 From: paul.sandoz at oracle.com (Paul Sandoz) Date: Thu, 27 Feb 2014 18:23:28 +0100 Subject: Unsafe: efficiently comparing two byte arrays In-Reply-To: <530F55CA.3060302@redhat.com> References: <530F5136.20405@redhat.com> <8D3E2037-E5D1-4D2E-B779-E7EDD2AF4F58@oracle.com> <530F55CA.3060302@redhat.com> Message-ID: On Feb 27, 2014, at 4:12 PM, Andrew Haley wrote: > On 02/27/2014 03:05 PM, Paul Sandoz wrote: >> >> On Feb 27, 2014, at 3:52 PM, Andrew Haley wrote: >> >>> Hi, >>> >>> On 02/26/2014 03:42 PM, Paul Sandoz wrote: >>> >>>> A common reason why Unsafe is used is to more efficiently compare two unsigned byte arrays, viewing those byte arrays as long arrays. >>> >>> Why not simply provide ByteBuffer.forArray(byte[]) ? Then we'd have all the ByteBuffer intrinsics for put() and get() operations. >> >> Good point, even if it does require the creation of two ByteBuffer instances. >> >> An implementation of Arrays.compare could do that if things were suitably intrinsified, which i am not sure is the case as of today. >> >> I will do some measurements and report back. > > Unless something has been done in HotSpot pretty recently, the > ByteBuffer intrinsics are not as performat as they might be, so the > results might be a little underwhelming. > Yes. For this benchmark: http://cr.openjdk.java.net/~psandoz/unsafe/ByteArrayCompareTest.java Here are results for comparing two byte arrays of size N containing the same content, on my mac book: http://cr.openjdk.java.net/~psandoz/unsafe/byte-array-compare-results.txt The unsafe test is about ~3/4 x faster than the safe test. And the buffer test is slower than the safe test. I also included a variant of the unsafe test that uses Long.reverseBytes; the results are similar to the unsafe test. I believe reverseBytes is intrinsified as is numberOfTrailingZeros (i tried to disable it with -XX:DisableIntrinsic=_reverseBytes_l but it did not make any great different to the results, which makes me think i configured the option incorrectly). Paul. From paul.sandoz at oracle.com Thu Feb 27 17:36:49 2014 From: paul.sandoz at oracle.com (Paul Sandoz) Date: Thu, 27 Feb 2014 18:36:49 +0100 Subject: Unsafe: removing the monitorEnter/monitorExit/tryMonitorEnter methods In-Reply-To: <530F4AA4.4070507@redhat.com> References: <530F31C1.8000604@oracle.com> <530F4AA4.4070507@redhat.com> Message-ID: Hi David, On Feb 27, 2014, at 3:24 PM, David M. Lloyd wrote: > (I'm the guy that wrote JBoss Modules, i.e. apparently the one usage "in the wild" of these methods). > Yes :-) the only one i have found so far. IIUC JBoss Modules only needs to use those methods on Java 1.6, is that correct? Thanks, Paul. From heinz at javaspecialists.eu Thu Feb 27 18:10:48 2014 From: heinz at javaspecialists.eu (Dr Heinz M. Kabutz) Date: Thu, 27 Feb 2014 20:10:48 +0200 Subject: Unsafe: removing the monitorEnter/monitorExit/tryMonitorEnter methods In-Reply-To: <530F55B0.1080907@redhat.com> References: <530F50DA.3010308@javaspecialists.eu> <530F55B0.1080907@redhat.com> Message-ID: <530F7FA8.5000103@javaspecialists.eu> Plus, monitors can get optimized beautifully when there is no contention, although that falls away for the code that uses monitorEnter/monitorExit calls directly as far as I could tell. Regards Heinz -- Dr Heinz M. Kabutz (PhD CompSci) Author of "The Java(tm) Specialists' Newsletter" Oracle Java Champion 2005-2013 JavaOne Rock Star Speaker 2012 http://www.javaspecialists.eu Tel: +30 69 75 595 262 Skype: kabutz David M. Lloyd wrote: > A "Monitors" class could possibly be introduced which simply has three > (intrinsic) static methods: void lock(Object obj), void unlock(Object > obj), boolean tryLock(Object obj). > > Could even add a void lockInterruptibly(Object obj) throws > InterruptedException. > > Why not? Monitors are not going anywhere, and they're considerably > lighter than ReentrantLock in terms of memory usage and allocations in > the (pretty common) case where you only need zero or one conditions. > Avoiding something like this out of loyalty to new APIs is probably > pretty misguided. > > On 02/27/2014 08:51 AM, Dr Heinz M. Kabutz wrote: >> I have never used this "in the wild", but rather have moved over to the >> ReentrantLock when I needed that particular functionality. >> >> However, I do see a place for this ability. As I wrote in here: >> http://www.javaspecialists.eu/archive/Issue194.html - sometimes you need >> to modify code that is already using a particular locking mechanism. To >> then redo everything with ReentrantLock can introduce errors. >> >> If I had a say, I would vote to either keep that method in Unsafe (which >> is where I think it belongs) or provide an alternative way to make the >> tryMonitorEnter() mechanism available to those that might end up needing >> it. >> >> Since I bought my Suzuki Jimny 7 years ago, I have not needed my spare >> wheel even once. So yeah, I could take it off and drive around without >> it. But chances are, the day after I take it off, I will need it. >> >> tryMonitorEnter() does no harm and it is out of reach of most >> programmers. >> >> Regards >> >> Heinz > > From peter.levart at gmail.com Thu Feb 27 19:02:45 2014 From: peter.levart at gmail.com (Peter Levart) Date: Thu, 27 Feb 2014 20:02:45 +0100 Subject: JDK 9 RFR of JDK-8035279: Clean up internal deprecations in BigInteger In-Reply-To: <3DCE5A13-1121-4A71-93ED-1F11611E94E5@oracle.com> References: <5305B914.7000901@oracle.com> <264E7F8B-E421-4D91-88D9-A22EF0366F2A@oracle.com> <5C039E72-638F-494D-8A78-96ADEFFC267B@oracle.com> <0D061461-C44C-4EBB-8848-E0647C374A34@oracle.com> <0A4793AC-509F-4183-94B8-1DD2ABD7D2DC@oracle.com> <41862830-7AC0-4B37-B947-542532AE63B9@oracle.com> <530F429E.6040204@gmail.com> <3DCE5A13-1121-4A71-93ED-1F11611E94E5@oracle.com> Message-ID: <530F8BD5.8050103@gmail.com> On 02/27/2014 04:06 PM, Paul Sandoz wrote: >> >Back then I presented an alternative solution which solves that: >> > >> >http://mail.openjdk.java.net/pipermail/core-libs-dev/2013-June/018457.html >> > >> >...and as can be seen, does not use volatile read/write to publish BigIntegers - that's why it is important that internal implementation of such immutable classes like BigInteger is tolerant to unsafe publication... >> > > Interesting, nice use of a linked list. Not sure that is safe publication of cacheLine into powerCache, the last element in a line could be observed as null, plus although it is unlikely, the compiler could shuffle up the array element store? Hi Paul, Any element of powerCacheIndex array of arrays and/or any element of it's BigInteger[] sub-arrays can be observed as null and the code accounts for that. You can view the scheme as two-level cache. The 1st level is 'powerCacheIndex' where BigInteger[]s and BigIntegers are published unsafely. If non-null values are observed, they are correct, if null values are observed, the code looks through into the 2nd level 'powerCache' array of linked-lists which is a properly synchronized lazily initialized cache of values where each is calculated at most once. Regards, Peter From david.lloyd at redhat.com Thu Feb 27 19:06:55 2014 From: david.lloyd at redhat.com (David M. Lloyd) Date: Thu, 27 Feb 2014 13:06:55 -0600 Subject: Unsafe: removing the monitorEnter/monitorExit/tryMonitorEnter methods In-Reply-To: References: <530F31C1.8000604@oracle.com> <530F4AA4.4070507@redhat.com> Message-ID: <530F8CCF.5010201@redhat.com> On 02/27/2014 11:36 AM, Paul Sandoz wrote: > Hi David, > > On Feb 27, 2014, at 3:24 PM, David M. Lloyd wrote: >> (I'm the guy that wrote JBoss Modules, i.e. apparently the one usage "in the wild" of these methods). >> > > Yes :-) the only one i have found so far. > > IIUC JBoss Modules only needs to use those methods on Java 1.6, is that correct? Right, to recursively release the lock (which, btw, in general, is a very dicey thing to do for reasons obvious and non-obvious). -- - DML From joe.darcy at oracle.com Thu Feb 27 20:11:46 2014 From: joe.darcy at oracle.com (Joe Darcy) Date: Thu, 27 Feb 2014 12:11:46 -0800 Subject: JDK 9 RFR of JDK-8035452: Fix serial lint warnings in core libs In-Reply-To: <530C3DF5.3040707@oracle.com> References: <530666A5.8050607@oracle.com> <530C1A87.2070608@oracle.com> <530C3DF5.3040707@oracle.com> Message-ID: <530F9C02.5090500@oracle.com> On 02/24/2014 10:53 PM, Stuart Marks wrote: > On 2/24/14 8:22 PM, Joe Darcy wrote: >> On 02/20/2014 12:49 PM, Paul Benedict wrote: >>> Joe, I find it interesting that you suppressed the serial warning on an >>> abstract class. I'd like to know more about that. Is this a >>> universal rule? >>> Are serial uids not important for abstract classes? >> >> I wouldn't generalize that way from this example. >> >> The serial hash of NavigableSubMap has differed in different JDK >> releases, >> but its subclasses do define serialVersionUID fields. I assume the >> set of >> non-transient fields in NavigableSubMap has stayed unchanged, but I >> haven't >> verified that. >> >> If I hadn't found the change in the hash value, I would have added the >> serialVersionUID to NavigableSubMap too. > > And in his reply to Paul, Joe said: >> From what I was able to discern by reading the serialization >> specification >> [1], If a class does *not* declare a serialVersionUID, the >> serialVersionUID >> of its superclass is *not* included in the serialver hash computation >> of the >> child class. However, my understanding is that changes to the fields >> stored >> in superclass and changes to the semantics of its readObject / >> writeObjects >> methods could affect the serialization of the child class. > > I think we need to take a closer look at these issues. > > I believe that abstract, serializable superclasses *do* need to have a > serialVersionUID defined. The reason is that when a subclass is > serialized, its superclass descriptor (an ObjectStreamClass) is also > serialized. Upon deserialization, the descriptor's svuid is matched > against the svuid of the class loaded at runtime, and if there is a > mismatch, InvalidClassException ensues. > > While the svuid of an abstract superclass isn't included in the > subclass' svuid hash, the svuid of the superclass does affect serial > compatibility of subclasses as described above. Thus, an apparently > innocuous change to the superclass might prevent serial compatibility > of its subclasses, no matter how carefully the subclasses are programmed. > > If the NavigableSubMap class has changed svuid values over several > releases, well, unfortunately we may have a compatibility problem > already in the field. We'd need to choose which release to be > compatible with. Since 8 isn't quite out yet, we might be able to > change an early 8-update and 9 to be compatibile with the latest > 7-update. > > Note that the svuid of a class does not relate solely to the fields > that are serialized. It's an attempt at a version hash of the > *implementation* of a class, not a version of the serial protocol. > Even changes to a class that don't affect the serialized output stream > can affect the svuid. For example, renaming a package-private method > will affect the svuid. See section 4.6 of the serialization spec. I am trying hard to remain blissfully ignorant of any more low-level details of the serialization format; however, I might not be successful on that goal much longer ;-) My preference in a case like this is to add the svuid if for no other reason that is is simple to explain and understand, even if it is not strictly required. > While we're at it (sorry...) in the diffs for your other serial > warnings patch JDK-8035453, there are several lines where the serial > warning is suppressed like so: > > + at SuppressWarnings("serial") // JDK implementation class > > As you know, serialization can expose the private fields of a class, > making them public in a sense. Serialization can also expose what are > internal, implementation classes, if these classes are part of a > serializable object graph that is exposed to applications. I don't > know about the specific situation with the DOM classes, but even if a > serializable class is internal, we might need to be concerned about > serialization compatibility. There is a difference in character between a serializable class in Java SE (java.* and javax.*) and the jdk.Exported(true) types in the JDK and a serializable class that lives in sun.* or some other jdk.Exported(false) area. For that latter, the serialization contract has to be different, with fewer guarantees, just as the general usage contract for those types has fewer guarantees. I think this is analogous to putting non-serializable classes into collections; the collection itself is serializable, but it won't be anymore if you put non-serializable objects into it. If a user happens to have a direct or indirect reference to an object of a JDK implementation type, the compatibility contract is weaker than if an object with a public Java SE type were being dealt with. > > Finally, EnumSet doesn't need a serial version UID. It's serialized > using a proxy class, so EnumSet never appears in a serialized byte > stream. (Note, its readObject throws an exception unconditionally.) So > it's probably safe to suppress its serialization warning. > Yes, EnumSet was a bit tricky, it is serializable itself, but uses a proxy internally. ("Effective Java, 2nd edition" both recommends the proxy pattern and recommends adding a svuid to all serializable classes, but doesn't explicitly give guidance to this combination of features.) To avoid adding a long comment explaining the proxy pattern and why a svuid on EnumSet isn't really required, my preference would just be to add the svuid if it doesn't cause any harm. Thanks, -Joe From brian.burkhalter at oracle.com Thu Feb 27 20:14:35 2014 From: brian.burkhalter at oracle.com (Brian Burkhalter) Date: Thu, 27 Feb 2014 12:14:35 -0800 Subject: [REFRESH] JDK 9 RFR of JDK-8035279: Clean up internal deprecations in BigInteger In-Reply-To: <247D03A1-0D88-437D-B2F1-9CD5DAE8492D@oracle.com> References: <5305D6EB.30806@oracle.com> <53064179.2060703@gmail.com> <530660FA.1060407@oracle.com> <96CF3236-4E65-4D64-BCF3-1E9660D29BAA@oracle.com> <5306BC4B.4090909@oracle.com> <8229AE11-1ABF-474D-961F-BBD5ACA3DD82@oracle.com> <61708919-691D-4EBF-8685-0CC5FC309701@oracle.com> <10AE28AF-8765-430C-9D83-D0A88062B096@oracle.com> <247D03A1-0D88-437D-B2F1-9CD5DAE8492D@oracle.com> Message-ID: <8520F61D-B0CE-4E83-8336-938C745823D1@oracle.com> On Feb 27, 2014, at 12:17 AM, Paul Sandoz wrote: >> So may I obtain a +1 from a JDK 9 Reviewer now? >> > > Indeed you may. Thanks, Paul. I refreshed the webrev http://cr.openjdk.java.net/~bpb/8035279/webrev.01/ with the agreed upon version. Brian From mandy.chung at oracle.com Thu Feb 27 20:32:58 2014 From: mandy.chung at oracle.com (Mandy Chung) Date: Thu, 27 Feb 2014 12:32:58 -0800 Subject: RFR 9: 8035889: jdk testlibrary - add printing of values of failed assertions In-Reply-To: <530F5AE9.8000405@oracle.com> References: <530E4FD8.9040605@oracle.com> <530E9C6C.8080409@oracle.com> <530EAC78.4000301@Oracle.com> <530EAE5F.3050308@oracle.com> <530F5AE9.8000405@oracle.com> Message-ID: <530FA0F9.5000106@oracle.com> Looks okay to me. Mandy On 2/27/14 7:34 AM, roger riggs wrote: > Hi Mandy, > > I updated the webrev: > http://cr.openjdk.java.net/~rriggs/webrev-testlibrary-asserts-8035889/ > > Alan suggested copying serviceability-dev so they have a chance to > review if desired. > > I want to investigate if it is possible to use the TestNG Assert > classes without > the TestNG execution framework. > It would be necessary to compile/run against TestNG.jar but it might not > need the entire mechanism. > > Thanks, Roger > > On 2/26/2014 10:17 PM, Mandy Chung wrote: >> On 2/26/2014 7:09 PM, Roger Riggs wrote: >>> Hi Mandy, >>> >>> Yes, it might be more productive to switch the tests to TestNG. >>> But it did provide support in cases where TestNG could not be used, >>> for example in a directory of existing tests that had custom reporting. >>> >>> But I remember there is a problem with TestNG having a dependency >>> for XML >>> which is not supported in Profile1 and a number of tests had to be >>> disabled >>> in that configuration. Will XML always be available. Do we need to >>> solve >>> or work around that problem with TestNG? >>> >> >> This is a good point. When we want to test just the base module for >> example, how can we run TestNG tests? We need to address that certainly. >> >> My comment on TestNG is a question for new tests using this Asserts >> class. Your patch is fine to go (after taking out @library tag if I >> got it correct). >> >> Mandy >> >>> Thanks, Roger >>> >>> On 2/26/14 9:01 PM, Mandy Chung wrote: >>>> Hi Roger, >>>> >>>> On 2/26/2014 12:34 PM, roger riggs wrote: >>>>> The testlibrary for the jdk should be printing the values in the >>>>> failed >>>>> assertions to make debugging easier and quicker. >>>>> >>>>> The webrev adds the printing of the failed assertions and added >>>>> methods >>>>> for formatting and unconditional fail methods. >>>>> >>>>> Webrev: >>>>> http://cr.openjdk.java.net/~rriggs/webrev-testlibrary-asserts-8035889/ >>>>> >>>>> >>>> >>>> AssertsTest.java: line 28: @library doesn't look like it's needed. >>>> There is no jdk/test/testlibrary directory and I think >>>> jdk.testlibrary.* are found as relative to $test.src. >>>> >>>> Otherwise, the change looks okay. >>>> >>>> Now that jtreg supports TestNG and I wonder if this class should >>>> retire some day (there are only about 10 existing tests using this >>>> class). Are you writing new tests using this Asserts class? >>>> >>>> Mandy >>>> >>>>> Bug: >>>>> 8035889: jdk testlibrary - add printing of values of failed >>>>> assertions >>>>> >>>>> Thanks, Roger >>>>> >>>>> [1] https://bugs.openjdk.java.net/browse/JDK-8035889 >>>>> >>>>> >>>> >>> >> > From paul.sandoz at oracle.com Thu Feb 27 20:31:51 2014 From: paul.sandoz at oracle.com (Paul Sandoz) Date: Thu, 27 Feb 2014 21:31:51 +0100 Subject: JDK 9 RFR of JDK-8035279: Clean up internal deprecations in BigInteger In-Reply-To: <530F8BD5.8050103@gmail.com> References: <5305B914.7000901@oracle.com> <264E7F8B-E421-4D91-88D9-A22EF0366F2A@oracle.com> <5C039E72-638F-494D-8A78-96ADEFFC267B@oracle.com> <0D061461-C44C-4EBB-8848-E0647C374A34@oracle.com> <0A4793AC-509F-4183-94B8-1DD2ABD7D2DC@oracle.com> <41862830-7AC0-4B37-B947-542532AE63B9@oracle.com> <530F429E.6040204@gmail.com> <3DCE5A13-1121-4A71-93ED-1F11611E94E5@oracle.com> <530F8BD5.8050103@gmail.com> Message-ID: <965D62C9-47B1-46D6-9121-8915575D75F6@oracle.com> On Feb 27, 2014, at 8:02 PM, Peter Levart wrote: > > On 02/27/2014 04:06 PM, Paul Sandoz wrote: >>> > Back then I presented an alternative solution which solves that: >>> > >>> > http://mail.openjdk.java.net/pipermail/core-libs-dev/2013-June/018457.html >>> > >>> > ...and as can be seen, does not use volatile read/write to publish BigIntegers - that's why it is important that internal implementation of such immutable classes like BigInteger is tolerant to unsafe publication... >>> > >> Interesting, nice use of a linked list. Not sure that is safe publication of cacheLine into powerCache, the last element in a line could be observed as null, plus although it is unlikely, the compiler could shuffle up the array element store? > > Hi Paul, > > Any element of powerCacheIndex array of arrays and/or any element of it's BigInteger[] sub-arrays can be observed as null and the code accounts for that. You can view the scheme as two-level cache. The 1st level is 'powerCacheIndex' where BigInteger[]s and BigIntegers are published unsafely. If non-null values are observed, they are correct, if null values are observed, the code looks through into the 2nd level 'powerCache' array of linked-lists which is a properly synchronized lazily initialized cache of values where each is calculated at most once. > Doh! i somehow missed that null check, thanks, i get it now. Paul. From john.r.rose at oracle.com Thu Feb 27 20:39:20 2014 From: john.r.rose at oracle.com (John Rose) Date: Thu, 27 Feb 2014 12:39:20 -0800 Subject: [9] RFR (M): 8027827: Improve performance of catchException combinator In-Reply-To: <530C6D26.4090000@oracle.com> References: <53052622.6010507@oracle.com> <0386E0D6-D515-4C2C-9D6E-DD3B53E8B3E2@oracle.com> <5306420B.9040200@oracle.com> <0516A100-6FAF-4718-8A9F-A82D7C7E7061@oracle.com> <530C6D26.4090000@oracle.com> Message-ID: <059F4A27-AA9E-486E-82E6-1077B76AE524@oracle.com> On Feb 26, 2014, at 3:44 AM, Vladimir Ivanov wrote: >> >> Maybe use invokeWithArguments with target and catcher? That at least is >> a one-liner, and probably more efficient. > > Yes, that's a good idea! At least, it considerably simplifies the code. > > Updated webrev: > http://cr.openjdk.java.net/~vlivanov/8027827/final/webrev.03/ Thumbs up. Your use of invokeWithArguments in the unspecialized code is a good design pattern. The semantics are clear in the original method. This in turn gives a clear basis for specializing for each combination of argument arities and types. Specialization should be done using low-level, high-leverage mechanisms like bytecode spinning or even JIT optimizations. Put another way, if we have reasonable bytecode-generation intrinsics, feeding to good JIT optimizations, we don't need top-level specializations in the source code. The need for those has always been a mark of weakness in the HotSpot implementation of MHs. (Fredrik's JRockit implementation did it all in the JIT!) We will continue to push down specializations to lower layers. ? John From mandy.chung at oracle.com Thu Feb 27 21:15:45 2014 From: mandy.chung at oracle.com (Mandy Chung) Date: Thu, 27 Feb 2014 13:15:45 -0800 Subject: Unsafe: removing the monitorEnter/monitorExit/tryMonitorEnter methods In-Reply-To: References: <530EE93D.7090406@gmail.com> <530EE9A3.4080101@gmail.com> <530EEC46.40605@oracle.com> <530EF9A3.1000108@gmail.com> <530F0C40.7070303@oracle.com> <530F26F0.3090204@gmail.com> Message-ID: <530FAB01.8090803@oracle.com> On 2/27/14 4:29 AM, Paul Sandoz wrote: >> This will allocate about 10 Thread objects per second until the other thread finally reaches the synchronized read() method in BufferedInputStream... >> > > I like it. I like it too. > I do have a slight preference for Mandy's solution, see below, but i am not sure we can guarantee it is supported on all test platforms. The hotspot VM implementation supports it on all platforms. Note that this cost is not small - it stops the world and up call to JVM. Mandy > > Paul. > > try { > while (!isMonitorLockHeld(monitor)) { > Thread.sleep(1); > } > } catch (UnsupportedOperationException e) { > // Back up plan? > Thread.sleep(TimeUnit.SECONDS.toMillis(1)); > } > > /** > * Returns {@code true} if and only if the a thread holds the > * monitor lock on the specified object. > * > * @throws java.lang.UnsupportedOperationException if monitoring of > * object monitor use is not supported > */ > public static boolean isMonitorLockHeld(Object o) { > int oihc = System.identityHashCode(o); > ThreadMXBean tmxbean = ManagementFactory.getThreadMXBean(); > ThreadInfo[] tinfos = tmxbean.dumpAllThreads(true, false); > > for (ThreadInfo ti : tinfos) { > for (MonitorInfo mi : ti.getLockedMonitors()) { > if (mi.getIdentityHashCode() == oihc) > return true; > } > } > return false; > } > From mike.duigou at oracle.com Fri Feb 28 05:10:24 2014 From: mike.duigou at oracle.com (Mike Duigou) Date: Thu, 27 Feb 2014 21:10:24 -0800 Subject: Proposal: Make JNU_Throw* clear a pending exception In-Reply-To: <530CA530.5070904@oracle.com> References: <530C7399.7050706@oracle.com> <18CF8FC4-5524-4980-96BF-5A9E361637C0@oracle.com> <530C8019.40309@oracle.com> <530CA530.5070904@oracle.com> Message-ID: <96B8D6F9-33A8-4B46-9E28-F53DC10DD127@oracle.com> Or a suppressed exception. Mike On Feb 25 2014, at 06:14 , Roger Riggs wrote: > In some cases, I would expect that the exception being overridden > would/should become the 'cause' of the new exception so it is not cleared > but chained. Does JNI support that? > > On the original issue, discarding of exceptions should be explicit not implicit. > Keep (or insert) the exceptionClear(). > > Roger > > On 2/25/2014 6:35 AM, Chris Hegarty wrote: >> >> >> On 25/02/14 11:26, Petr Pchelko wrote: >>> Hello, Alan. >>> >>>> I can see how this might be attractive but doesn't it mean you are suppressing an important exception? >>> In case we?ve already got into the JNU_Throw we will throw a new exception and override the original one anyway. However I agree that this might suppress the warning for the code analysis tools. >>> >>>> Are the examples that you are looking at along these lines? >>> There are a number of examples when JNU_Throw is used to: >>> 1. Replace the message of an original exception: src/share/native/sun/awt/image/awt_ImageRep.c:192 >>> There are a few such places. >>> >>> 2. Rethrow some meaningful exception if the call to some function failed: src/windows/native/sun/windows/awt_Window.cpp:2861 >>> This is a much more common use case. In this case we have a return code from some method and we do not care if it was failed because of JNI exception or for some other reason. This is the main case where we need to add env->ExceptionClear() everywhere. >>> >>> 3. Quite common is to use it in the initIDs method to rethrow NoSuchFieldError: src/share/native/sun/awt/image/BufImgSurfaceData.c:77 >>> This one is questionable, I think that rethrowing is not needed here at all, the original exception is much more informative. >> >> Agreed. Similar to NetworkInterface.c Line:172 >> >> http://hg.openjdk.java.net/jdk9/dev/jdk/file/6ee5c47bdba7/src/solaris/native/java/net/NetworkInterface.c#172 >> >> -Chris. >> >>> 4. Where currently are throwing an exception with pure JNI, but it could be replaces with JNU: src/windows/native/sun/windows/awt_PrintJob.cpp:1365 >>> Similar to #2. >>> >>> With best regards. Petr. >>> >>> 25 ????. 2014 ?., ? 2:42 ????? ???????, Alan Bateman ???????(?): >>> >>>> On 25/02/2014 10:31, Petr Pchelko wrote: >>>>> Hello, Core and AWT teams. >>>>> >>>>> In AWT we have a lot of pending exception warnings which are now being fixed. A big fraction of this warnings is about a pending JNI exception at a call to JNU_Throw*. >>>>> Why don?t we add an env->ExceptionClear() call in the beginning of each JNU_Throw function? It is absolutely safe because: >>>>> 1. We are rethrowing an exception an loosing the original one anyway. >>>>> 2. In case there?s no pending exception the ExceptionClear is a no-op. >>>>> >>>>> If we do this the code would become much cleaner, because currently we have to manually clear a pending exception before every call to JNU_Throw*. >>>>> >>>>> What do you think about this proposal? >>>>> >>>> I can see how this might be attractive but doesn't it mean you are suppressing an important exception? We've fixed many areas in the last few weeks and I think a common case was just that whoever wrote the original code didn't realize that some JNI functions having a pending exception when they fail. In those cases then often it is a simple matter of just returning from the JNI function (maybe after some clean-up). Are the examples that you are looking at along these lines? I guess I'm mostly interested to know whether the JNU_Throw usages are needed or not. >>>> >>>> -Alan. >>> > From huizhe.wang at oracle.com Fri Feb 28 07:51:56 2014 From: huizhe.wang at oracle.com (huizhe wang) Date: Thu, 27 Feb 2014 23:51:56 -0800 Subject: RFR (JAXP): 8036004 : Incorrect license header for a test class Message-ID: <5310401C.4020801@oracle.com> Hi, This is a fix to the license header for class jdk/test/javax/xml/jaxp/transform/8004476/TestBase.java. Bug: https://bugs.openjdk.java.net/browse/JDK-8036004 Webrev: http://cr.openjdk.java.net/~joehw/jdk9/8036004/webrev/ Thanks, Joe From Alan.Bateman at oracle.com Fri Feb 28 08:01:32 2014 From: Alan.Bateman at oracle.com (Alan Bateman) Date: Fri, 28 Feb 2014 08:01:32 +0000 Subject: RFR (JAXP): 8036004 : Incorrect license header for a test class In-Reply-To: <5310401C.4020801@oracle.com> References: <5310401C.4020801@oracle.com> Message-ID: <5310425C.7050405@oracle.com> On 28/02/2014 07:51, huizhe wang wrote: > Hi, > > This is a fix to the license header for class > jdk/test/javax/xml/jaxp/transform/8004476/TestBase.java. > > Bug: > https://bugs.openjdk.java.net/browse/JDK-8036004 > > Webrev: > http://cr.openjdk.java.net/~joehw/jdk9/8036004/webrev/ Looks okay, just check the date from the hg logs as you want to put 2013 in the new header. -Alan From yasu at ysfactory.dip.jp Fri Feb 28 09:18:10 2014 From: yasu at ysfactory.dip.jp (Yasumasa Suenaga) Date: Fri, 28 Feb 2014 18:18:10 +0900 Subject: JDK-8036003: Add variable not to separate debug information. Message-ID: <20140228091835.58EB8C21F24@msgw007-05.ocn.ad.jp> Hi all, Currently, configure script can accept --disable-debug-symbols and --disable-zip-debug-info as controlling to generate debug information. However, current makefiles cannot build ELF binaries which is contained debug information with "images" target. Some Linux distros use RPM as package manager. RPM is built by rpmbuild command, it strips symbols and debug information during to generate rpm packages. https://fedoraproject.org/wiki/Packaging:Debuginfo For example, OpenJDK8 in Fedora20 ships libjvm.so and libjvm.debuginfo . libjvm.debuginfo is generated in OpenJDK's makefiles, however it does not contain debug information. Actual debug information is shipped by OpenJDK debuginfo package. This packaging is important when we have to debug JVM/JNI libraries. If we want to use debugging tools (GDB, SystemTap, etc...), they may requires debuginfo package. Debuggee (e.g. libjvm.so) points libjvm.so.debug which is located at sub directory in /usr/lib/debug . It is defined in ELF section (.note.gnu.build-id) of libjvm.so . However libjvm.so.debug does not contain valid debug information. We need to access libjvm.debuginfo.debug at same location. When we think to build OpenJDK rpm packages, we have to build ELF binaries which contain all debug information. we should not to separate debug information. Thus I want to add a variable "SEPARATED_DEBUGINFO_FILES" . If we pass "SEPARATED_DEBUGINFO_FILES=false" to make command, "make" does not execute objcopy command for generating debuginfo binaries. If we build rpm packages, we also have to add "STRIP_POLICY=no_strip" to arguments of make command. Or ELF binaries will be stripped. I've uploaded webrev for this enhancement. This is separated 3-part: top of forest, hotspot, jdk http://cr.openjdk.java.net/~ysuenaga/JDK-8036003/ Please review it and sponsoring! Thanks, Yasumasa P.S. I tried to add SEPARATED_DEBUGINFO_FILES as a configure option like "--disable-separated-debug-info" . I ran configure which is located at jdk9/dev directory after I ran autoconf and common/autoconf/autogen.sh. However it failed. I guess this is caused by changeset as below. JDK-8035495: Improvements in autoconf integration http://hg.openjdk.java.net/jdk9/dev/rev/6e29cd9ac2b4 This changes add "CHECKME" option to configure script. However, this changes affects "configure" only. It should change "configure.ac" . From paul.sandoz at oracle.com Fri Feb 28 09:38:01 2014 From: paul.sandoz at oracle.com (Paul Sandoz) Date: Fri, 28 Feb 2014 10:38:01 +0100 Subject: Unsafe: removing the monitorEnter/monitorExit/tryMonitorEnter methods In-Reply-To: <530F50DA.3010308@javaspecialists.eu> References: <530F50DA.3010308@javaspecialists.eu> Message-ID: <144A3A2D-85FF-4F3B-A4DA-1780B809F36B@oracle.com> On Feb 27, 2014, at 3:51 PM, Dr Heinz M. Kabutz wrote: > I have never used this "in the wild", but rather have moved over to the ReentrantLock when I needed that particular functionality. > > However, I do see a place for this ability. As I wrote in here: http://www.javaspecialists.eu/archive/Issue194.html - sometimes you need to modify code that is already using a particular locking mechanism. To then redo everything with ReentrantLock can introduce errors. > > If I had a say, I would vote to either keep that method in Unsafe (which is where I think it belongs) or provide an alternative way to make the tryMonitorEnter() mechanism available to those that might end up needing it. > > Since I bought my Suzuki Jimny 7 years ago, I have not needed my spare wheel even once. So yeah, I could take it off and drive around without it. But chances are, the day after I take it off, I will need it. > > tryMonitorEnter() does no harm and it is out of reach of most programmers. > I think your analogy to a spare wheel is misleading. Continuing with the car theme i liken this feature to a pair of fluffy dice dangling from the rear view mirror [1]: distracting; mostly useless; and mostly harmless. All the data so far indicates this is legacy code [2]. My suggestion, if you consider it important enough, is to write and circulate a draft JEP for such a public API [3]. Paul. [1] http://en.wikipedia.org/wiki/Fuzzy_dice [2] The only dependency on "tryMonitorEnter" i have found was very recently introduced in a test for which there are two known alternatives. [3] It would be interesting to know if some such public API was considered when j.u.c.locks were introduced, which was almost 10 years ago, and yet in all that time this feature is now virtually unused (not just by you but by many other Java developers too). From staffan.larsen at oracle.com Fri Feb 28 09:43:20 2014 From: staffan.larsen at oracle.com (Staffan Larsen) Date: Fri, 28 Feb 2014 10:43:20 +0100 Subject: RFR 9: 8035889: jdk testlibrary - add printing of values of failed assertions In-Reply-To: <530F5AE9.8000405@oracle.com> References: <530E4FD8.9040605@oracle.com> <530E9C6C.8080409@oracle.com> <530EAC78.4000301@Oracle.com> <530EAE5F.3050308@oracle.com> <530F5AE9.8000405@oracle.com> Message-ID: <6A397A15-56DD-41FA-885F-4A3BF2415CB6@oracle.com> Looks good to me! Thanks, /Staffan On 27 feb 2014, at 16:34, roger riggs wrote: > Hi Mandy, > > I updated the webrev: > http://cr.openjdk.java.net/~rriggs/webrev-testlibrary-asserts-8035889/ > > Alan suggested copying serviceability-dev so they have a chance to review if desired. > > I want to investigate if it is possible to use the TestNG Assert classes without > the TestNG execution framework. > It would be necessary to compile/run against TestNG.jar but it might not > need the entire mechanism. > > Thanks, Roger > > On 2/26/2014 10:17 PM, Mandy Chung wrote: >> On 2/26/2014 7:09 PM, Roger Riggs wrote: >>> Hi Mandy, >>> >>> Yes, it might be more productive to switch the tests to TestNG. >>> But it did provide support in cases where TestNG could not be used, >>> for example in a directory of existing tests that had custom reporting. >>> >>> But I remember there is a problem with TestNG having a dependency for XML >>> which is not supported in Profile1 and a number of tests had to be disabled >>> in that configuration. Will XML always be available. Do we need to solve >>> or work around that problem with TestNG? >>> >> >> This is a good point. When we want to test just the base module for example, how can we run TestNG tests? We need to address that certainly. >> >> My comment on TestNG is a question for new tests using this Asserts class. Your patch is fine to go (after taking out @library tag if I got it correct). >> >> Mandy >> >>> Thanks, Roger >>> >>> On 2/26/14 9:01 PM, Mandy Chung wrote: >>>> Hi Roger, >>>> >>>> On 2/26/2014 12:34 PM, roger riggs wrote: >>>>> The testlibrary for the jdk should be printing the values in the failed >>>>> assertions to make debugging easier and quicker. >>>>> >>>>> The webrev adds the printing of the failed assertions and added methods >>>>> for formatting and unconditional fail methods. >>>>> >>>>> Webrev: >>>>> http://cr.openjdk.java.net/~rriggs/webrev-testlibrary-asserts-8035889/ >>>>> >>>> >>>> AssertsTest.java: line 28: @library doesn't look like it's needed. There is no jdk/test/testlibrary directory and I think jdk.testlibrary.* are found as relative to $test.src. >>>> >>>> Otherwise, the change looks okay. >>>> >>>> Now that jtreg supports TestNG and I wonder if this class should retire some day (there are only about 10 existing tests using this class). Are you writing new tests using this Asserts class? >>>> >>>> Mandy >>>> >>>>> Bug: >>>>> 8035889: jdk testlibrary - add printing of values of failed assertions >>>>> >>>>> Thanks, Roger >>>>> >>>>> [1] https://bugs.openjdk.java.net/browse/JDK-8035889 >>>>> >>>>> >>>> >>> >> > From david.holmes at oracle.com Fri Feb 28 11:13:14 2014 From: david.holmes at oracle.com (David Holmes) Date: Fri, 28 Feb 2014 21:13:14 +1000 Subject: JDK-8036003: Add variable not to separate debug information. In-Reply-To: <20140228091835.58EB8C21F24@msgw007-05.ocn.ad.jp> References: <20140228091835.58EB8C21F24@msgw007-05.ocn.ad.jp> Message-ID: <53106F4A.8030905@oracle.com> Hi, As I put in the bug report this seems way too complicated. Seems to me all you need to do to get what you want is not use FDS and not strip the symbols from the binary. The former is trivial. The latter is more awkward as the strip policy stuff does not work as I would want it to work, but still doable. David On 28/02/2014 7:18 PM, Yasumasa Suenaga wrote: > Hi all, > > > Currently, configure script can accept --disable-debug-symbols and > --disable-zip-debug-info as controlling to generate debug information. > However, current makefiles cannot build ELF binaries which is contained > debug information with "images" target. > > Some Linux distros use RPM as package manager. > RPM is built by rpmbuild command, it strips symbols and debug information > during to generate rpm packages. > https://fedoraproject.org/wiki/Packaging:Debuginfo > > For example, OpenJDK8 in Fedora20 ships libjvm.so and libjvm.debuginfo . > libjvm.debuginfo is generated in OpenJDK's makefiles, however it does not > contain debug information. Actual debug information is shipped by OpenJDK > debuginfo package. > This packaging is important when we have to debug JVM/JNI libraries. > If we want to use debugging tools (GDB, SystemTap, etc...), they may requires > debuginfo package. Debuggee (e.g. libjvm.so) points libjvm.so.debug which is > located at sub directory in /usr/lib/debug . It is defined in ELF section > (.note.gnu.build-id) of libjvm.so . However libjvm.so.debug does not contain > valid debug information. We need to access libjvm.debuginfo.debug at same location. > > > When we think to build OpenJDK rpm packages, we have to build ELF binaries > which contain all debug information. we should not to separate debug information. > > > Thus I want to add a variable "SEPARATED_DEBUGINFO_FILES" . > If we pass "SEPARATED_DEBUGINFO_FILES=false" to make command, "make" does not > execute objcopy command for generating debuginfo binaries. > > If we build rpm packages, we also have to add "STRIP_POLICY=no_strip" to arguments > of make command. Or ELF binaries will be stripped. > > > I've uploaded webrev for this enhancement. > This is separated 3-part: top of forest, hotspot, jdk > http://cr.openjdk.java.net/~ysuenaga/JDK-8036003/ > > Please review it and sponsoring! > > > Thanks, > > Yasumasa > > > P.S. > I tried to add SEPARATED_DEBUGINFO_FILES as a configure option like > "--disable-separated-debug-info" . > I ran configure which is located at jdk9/dev directory after I ran autoconf > and common/autoconf/autogen.sh. However it failed. > > I guess this is caused by changeset as below. > JDK-8035495: Improvements in autoconf integration > http://hg.openjdk.java.net/jdk9/dev/rev/6e29cd9ac2b4 > > This changes add "CHECKME" option to configure script. However, this changes > affects "configure" only. It should change "configure.ac" . > From daniel.daugherty at oracle.com Fri Feb 28 14:04:34 2014 From: daniel.daugherty at oracle.com (Daniel D. Daugherty) Date: Fri, 28 Feb 2014 07:04:34 -0700 Subject: JDK-8036003: Add variable not to separate debug information. In-Reply-To: <53106F4A.8030905@oracle.com> References: <20140228091835.58EB8C21F24@msgw007-05.ocn.ad.jp> <53106F4A.8030905@oracle.com> Message-ID: <53109772.9070808@oracle.com> The proper way to fix this is to disable FDS. We should not need yet another option to control debug info. Dan On 2/28/14 4:13 AM, David Holmes wrote: > Hi, > > As I put in the bug report this seems way too complicated. Seems to me > all you need to do to get what you want is not use FDS and not strip the > symbols from the binary. The former is trivial. The latter is more > awkward as the strip policy stuff does not work as I would want it to > work, but still doable. > > David > > On 28/02/2014 7:18 PM, Yasumasa Suenaga wrote: >> Hi all, >> >> >> Currently, configure script can accept --disable-debug-symbols and >> --disable-zip-debug-info as controlling to generate debug information. >> However, current makefiles cannot build ELF binaries which is contained >> debug information with "images" target. >> >> Some Linux distros use RPM as package manager. >> RPM is built by rpmbuild command, it strips symbols and debug information >> during to generate rpm packages. >> https://fedoraproject.org/wiki/Packaging:Debuginfo >> >> For example, OpenJDK8 in Fedora20 ships libjvm.so and libjvm.debuginfo . >> libjvm.debuginfo is generated in OpenJDK's makefiles, however it does not >> contain debug information. Actual debug information is shipped by OpenJDK >> debuginfo package. >> This packaging is important when we have to debug JVM/JNI libraries. >> If we want to use debugging tools (GDB, SystemTap, etc...), they may requires >> debuginfo package. Debuggee (e.g. libjvm.so) points libjvm.so.debug which is >> located at sub directory in /usr/lib/debug . It is defined in ELF section >> (.note.gnu.build-id) of libjvm.so . However libjvm.so.debug does not contain >> valid debug information. We need to access libjvm.debuginfo.debug at same location. >> >> >> When we think to build OpenJDK rpm packages, we have to build ELF binaries >> which contain all debug information. we should not to separate debug information. >> >> >> Thus I want to add a variable "SEPARATED_DEBUGINFO_FILES" . >> If we pass "SEPARATED_DEBUGINFO_FILES=false" to make command, "make" does not >> execute objcopy command for generating debuginfo binaries. >> >> If we build rpm packages, we also have to add "STRIP_POLICY=no_strip" to arguments >> of make command. Or ELF binaries will be stripped. >> >> >> I've uploaded webrev for this enhancement. >> This is separated 3-part: top of forest, hotspot, jdk >> http://cr.openjdk.java.net/~ysuenaga/JDK-8036003/ >> >> Please review it and sponsoring! >> >> >> Thanks, >> >> Yasumasa >> >> >> P.S. >> I tried to add SEPARATED_DEBUGINFO_FILES as a configure option like >> "--disable-separated-debug-info" . >> I ran configure which is located at jdk9/dev directory after I ran autoconf >> and common/autoconf/autogen.sh. However it failed. >> >> I guess this is caused by changeset as below. >> JDK-8035495: Improvements in autoconf integration >> http://hg.openjdk.java.net/jdk9/dev/rev/6e29cd9ac2b4 >> >> This changes add "CHECKME" option to configure script. However, this changes >> affects "configure" only. It should change "configure.ac" . >> From vladimir.x.ivanov at oracle.com Fri Feb 28 14:48:35 2014 From: vladimir.x.ivanov at oracle.com (Vladimir Ivanov) Date: Fri, 28 Feb 2014 18:48:35 +0400 Subject: [9] RFR (M): 8027827: Improve performance of catchException combinator In-Reply-To: <059F4A27-AA9E-486E-82E6-1077B76AE524@oracle.com> References: <53052622.6010507@oracle.com> <0386E0D6-D515-4C2C-9D6E-DD3B53E8B3E2@oracle.com> <5306420B.9040200@oracle.com> <0516A100-6FAF-4718-8A9F-A82D7C7E7061@oracle.com> <530C6D26.4090000@oracle.com> <059F4A27-AA9E-486E-82E6-1077B76AE524@oracle.com> Message-ID: <5310A1C3.4080109@oracle.com> Thanks for review, John! Best regards, Vladimir Ivanov On 2/28/14 12:39 AM, John Rose wrote: > On Feb 26, 2014, at 3:44 AM, Vladimir Ivanov > > wrote: > >>> >>> Maybe use invokeWithArguments with target and catcher? That at least is >>> a one-liner, and probably more efficient. >> >> Yes, that's a good idea! At least, it considerably simplifies the code. >> >> Updated webrev: >> http://cr.openjdk.java.net/~vlivanov/8027827/final/webrev.03/ > > Thumbs up. > > Your use of invokeWithArguments in the unspecialized code is a good > design pattern. The semantics are clear in the original method. This > in turn gives a clear basis for specializing for each combination of > argument arities and types. Specialization should be done using > low-level, high-leverage mechanisms like bytecode spinning or even JIT > optimizations. > > Put another way, if we have reasonable bytecode-generation intrinsics, > feeding to good JIT optimizations, we don't need top-level > specializations in the source code. The need for those has always been > a mark of weakness in the HotSpot implementation of MHs. (Fredrik's > JRockit implementation did it all in the JIT!) We will continue to push > down specializations to lower layers. > > ? John From yasu at ysfactory.dip.jp Fri Feb 28 15:38:24 2014 From: yasu at ysfactory.dip.jp (Yasumasa Suenaga) Date: Sat, 01 Mar 2014 00:38:24 +0900 Subject: JDK-8036003: Add variable not to separate debug information. In-Reply-To: <53109772.9070808@oracle.com> References: <20140228091835.58EB8C21F24@msgw007-05.ocn.ad.jp> <53106F4A.8030905@oracle.com> <53109772.9070808@oracle.com> Message-ID: <5310AD70.2070908@ysfactory.dip.jp> > The proper way to fix this is to disable FDS. Does this mean I have to pass --disable-debug-symbols to configure ? I've added comment to JDK-8036003, I think if we pass --disable-debug-symbols to configure, gcc/g++ is not passed -g option. "-g" is needed to generate debuginfo. Thanks, Yasumasa On 2014/02/28 23:04, Daniel D. Daugherty wrote: > The proper way to fix this is to disable FDS. We should not need > yet another option to control debug info. > > Dan > > > On 2/28/14 4:13 AM, David Holmes wrote: >> Hi, >> >> As I put in the bug report this seems way too complicated. Seems to me >> all you need to do to get what you want is not use FDS and not strip the >> symbols from the binary. The former is trivial. The latter is more >> awkward as the strip policy stuff does not work as I would want it to >> work, but still doable. >> >> David >> >> On 28/02/2014 7:18 PM, Yasumasa Suenaga wrote: >>> Hi all, >>> >>> >>> Currently, configure script can accept --disable-debug-symbols and >>> --disable-zip-debug-info as controlling to generate debug information. >>> However, current makefiles cannot build ELF binaries which is contained >>> debug information with "images" target. >>> >>> Some Linux distros use RPM as package manager. >>> RPM is built by rpmbuild command, it strips symbols and debug information >>> during to generate rpm packages. >>> https://fedoraproject.org/wiki/Packaging:Debuginfo >>> >>> For example, OpenJDK8 in Fedora20 ships libjvm.so and libjvm.debuginfo . >>> libjvm.debuginfo is generated in OpenJDK's makefiles, however it does not >>> contain debug information. Actual debug information is shipped by OpenJDK >>> debuginfo package. >>> This packaging is important when we have to debug JVM/JNI libraries. >>> If we want to use debugging tools (GDB, SystemTap, etc...), they may requires >>> debuginfo package. Debuggee (e.g. libjvm.so) points libjvm.so.debug which is >>> located at sub directory in /usr/lib/debug . It is defined in ELF section >>> (.note.gnu.build-id) of libjvm.so . However libjvm.so.debug does not contain >>> valid debug information. We need to access libjvm.debuginfo.debug at same location. >>> >>> >>> When we think to build OpenJDK rpm packages, we have to build ELF binaries >>> which contain all debug information. we should not to separate debug information. >>> >>> >>> Thus I want to add a variable "SEPARATED_DEBUGINFO_FILES" . >>> If we pass "SEPARATED_DEBUGINFO_FILES=false" to make command, "make" does not >>> execute objcopy command for generating debuginfo binaries. >>> >>> If we build rpm packages, we also have to add "STRIP_POLICY=no_strip" to arguments >>> of make command. Or ELF binaries will be stripped. >>> >>> >>> I've uploaded webrev for this enhancement. >>> This is separated 3-part: top of forest, hotspot, jdk >>> http://cr.openjdk.java.net/~ysuenaga/JDK-8036003/ >>> >>> Please review it and sponsoring! >>> >>> >>> Thanks, >>> >>> Yasumasa >>> >>> >>> P.S. >>> I tried to add SEPARATED_DEBUGINFO_FILES as a configure option like >>> "--disable-separated-debug-info" . >>> I ran configure which is located at jdk9/dev directory after I ran autoconf >>> and common/autoconf/autogen.sh. However it failed. >>> >>> I guess this is caused by changeset as below. >>> JDK-8035495: Improvements in autoconf integration >>> http://hg.openjdk.java.net/jdk9/dev/rev/6e29cd9ac2b4 >>> >>> This changes add "CHECKME" option to configure script. However, this changes >>> affects "configure" only. It should change "configure.ac" . >>> > From blackdrag at gmx.org Fri Feb 28 15:56:07 2014 From: blackdrag at gmx.org (Jochen Theodorou) Date: Fri, 28 Feb 2014 16:56:07 +0100 Subject: rationale for swallowing exceptions in Class#getEnumConstantsShared Message-ID: <5310B197.1090707@gmx.org> Hi, we stumbled here about a strange error involving enums generated from our compiler and that did make me see getEnumConstantsShared() in java.lang.Class. Now the implementation more or less says, that if calling values() fails in any way, return null and ignore the exception. But why is the exception ignored? It could contain valuable information about what did go wrong.... In my code for example, something obviously goes wrong... sometimes... depending on moon-phases or something. and I have not the slightest idea what. bye Jochen -- Jochen "blackdrag" Theodorou - Groovy Project Tech Lead blog: http://blackdragsview.blogspot.com/ german groovy discussion newsgroup: de.comp.lang.misc For Groovy programming sources visit http://groovy-lang.org From peter.levart at gmail.com Fri Feb 28 16:36:51 2014 From: peter.levart at gmail.com (Peter Levart) Date: Fri, 28 Feb 2014 17:36:51 +0100 Subject: rationale for swallowing exceptions in Class#getEnumConstantsShared In-Reply-To: <5310B197.1090707@gmx.org> References: <5310B197.1090707@gmx.org> Message-ID: <5310BB23.6030809@gmail.com> On 02/28/2014 04:56 PM, Jochen Theodorou wrote: > Hi, > > we stumbled here about a strange error involving enums generated from > our compiler and that did make me see getEnumConstantsShared() in > java.lang.Class. > > Now the implementation more or less says, that if calling values() > fails in any way, return null and ignore the exception. But why is the > exception ignored? It could contain valuable information about what > did go wrong.... Hi Jochen, I agree that the exception should be thrown (possibly an unchecked IllegalArgumentException or something like that with the cause from reflective operation). If Class.isEnum() returns true then the class should be considered an enum and if it is not conformant to specification (which may cause one of those exceptions) then this should be reported as exception and not as null return. The getEnumConstantsShared() is also used as implementation for public method: /** * Returns the elements of this enum class or *null if this** ** * Class object does not represent an enum type*. * * @return an array containing the values comprising the enum class * represented by this Class object in the order they're * declared, or null if this Class object does not * represent an enum type * @since 1.5 */ public T[] getEnumConstants() { T[] values = getEnumConstantsShared(); return (values != null) ? values.clone() : null; } The behaviour of above method is not consistent with: /** * Returns *true if and only if this class was declared as an enum* in the * source code. * * @return true if and only if this class was declared as an enum in the * source code * @since 1.5 */ public boolean isEnum() { // An enum must both directly extend java.lang.Enum and have // the ENUM bit set; classes for specialized enum constants // don't do the former. return (this.getModifiers() & ENUM) != 0 && this.getSuperclass() == java.lang.Enum.class; } ...which can return true at the same time as getEnumConstants() returns null for the same class... I don't believe there's any code out there that relies on this faulty behaviour of getEnumConstants() public method. The other use of getEnumConstantsShared() is to support the Enum.valueOf(Class, String) method which already throws IllegalArgumentException if the 1st parameter does not represent an enum class, so there's no compatibility concerns about that use. What do experts say? Regards, Peter > > In my code for example, something obviously goes wrong... sometimes... > depending on moon-phases or something. and I have not the slightest > idea what. > > bye Jochen > > From huizhe.wang at oracle.com Fri Feb 28 16:46:40 2014 From: huizhe.wang at oracle.com (huizhe wang) Date: Fri, 28 Feb 2014 08:46:40 -0800 Subject: RFR (JAXP): 8036004 : Incorrect license header for a test class In-Reply-To: <5310425C.7050405@oracle.com> References: <5310401C.4020801@oracle.com> <5310425C.7050405@oracle.com> Message-ID: <5310BD70.8000404@oracle.com> On 2/28/2014 12:01 AM, Alan Bateman wrote: > On 28/02/2014 07:51, huizhe wang wrote: >> Hi, >> >> This is a fix to the license header for class >> jdk/test/javax/xml/jaxp/transform/8004476/TestBase.java. >> >> Bug: >> https://bugs.openjdk.java.net/browse/JDK-8036004 >> >> Webrev: >> http://cr.openjdk.java.net/~joehw/jdk9/8036004/webrev/ > Looks okay, just check the date from the hg logs as you want to put > 2013 in the new header. Thanks, fixed (2013). -Joe > > -Alan From blackdrag at gmx.org Fri Feb 28 16:48:52 2014 From: blackdrag at gmx.org (Jochen Theodorou) Date: Fri, 28 Feb 2014 17:48:52 +0100 Subject: rationale for swallowing exceptions in Class#getEnumConstantsShared In-Reply-To: <5310BB23.6030809@gmail.com> References: <5310B197.1090707@gmx.org> <5310BB23.6030809@gmail.com> Message-ID: <5310BDF4.6040808@gmx.org> Am 28.02.2014 17:36, schrieb Peter Levart: [...] > The other use of > getEnumConstantsShared() is to support the Enum.valueOf(Class, String) > method which already throws IllegalArgumentException if the 1st > parameter does not represent an enum class, so there's no compatibility > concerns about that use. Enum.valueOf is the method through which I discovered this. I have exactly this case, an enum, which seems not to fit getEnumConstantsShared for an unknown reason. My additional problem though is, that sometimes the call works and sometimes not. bye Jochen -- Jochen "blackdrag" Theodorou - Groovy Project Tech Lead blog: http://blackdragsview.blogspot.com/ german groovy discussion newsgroup: de.comp.lang.misc For Groovy programming sources visit http://groovy-lang.org From stuart.marks at oracle.com Fri Feb 28 16:58:44 2014 From: stuart.marks at oracle.com (Stuart Marks) Date: Fri, 28 Feb 2014 08:58:44 -0800 Subject: JDK 9 RFR of JDK-8035452: Fix serial lint warnings in core libs In-Reply-To: <530F9C02.5090500@oracle.com> References: <530666A5.8050607@oracle.com> <530C1A87.2070608@oracle.com> <530C3DF5.3040707@oracle.com> <530F9C02.5090500@oracle.com> Message-ID: <5310C044.9090409@oracle.com> On 2/27/14 12:11 PM, Joe Darcy wrote: > I am trying hard to remain blissfully ignorant of any more low-level details of > the serialization format; however, I might not be successful on that goal much > longer ;-) I believe your latter statement is correct. :-) > My preference in a case like this is to add the svuid if for no other reason > that is is simple to explain and understand, even if it is not strictly required. In general, it does seem reasonable to add a svuid in cases where it's difficult or impractical to prove that the lack of a svuid causes no compatibility issues. > There is a difference in character between a serializable class in Java SE > (java.* and javax.*) and the jdk.Exported(true) types in the JDK and a > serializable class that lives in sun.* or some other jdk.Exported(false) area. > > For that latter, the serialization contract has to be different, with fewer > guarantees, just as the general usage contract for those types has fewer > guarantees. I think this is analogous to putting non-serializable classes into > collections; the collection itself is serializable, but it won't be anymore if > you put non-serializable objects into it. > > If a user happens to have a direct or indirect reference to an object of a JDK > implementation type, the compatibility contract is weaker than if an object with > a public Java SE type were being dealt with. I'm not sure I buy this. Unfortunately, serialization differs from the general usage contract in that serialization exposes internals. Just like it can expose private (non-transient) fields, serialization can also can expose what might otherwise look like purely internal implementation types. The canonical example of how an application can get a reference to an apparently internal class is java.util.TimeZone. If an app calls TimeZone.getDefault(), it gets back an instance of sun.util.calendar.ZoneInfo without ever mentioning this class by name. Furthermore, TimeZone and ZoneInfo are serializable, so they can be serialized directly or as part of another serializable object graph, and an instance of s.u.c.ZoneInfo does appear in the serialized byte stream. If s.u.c.ZoneInfo were not to have a svuid, an apparently innocuous change to it would clearly cause application incompatibilities. As it happens, s.u.c.ZoneInfo does have a svuid. I also note in passing that TimeZone, an abstract class, also has a svuid. >> Finally, EnumSet doesn't need a serial version UID. It's serialized using a >> proxy class, so EnumSet never appears in a serialized byte stream. (Note, its >> readObject throws an exception unconditionally.) So it's probably safe to >> suppress its serialization warning. > > Yes, EnumSet was a bit tricky, it is serializable itself, but uses a proxy > internally. ("Effective Java, 2nd edition" both recommends the proxy pattern and > recommends adding a svuid to all serializable classes, but doesn't explicitly > give guidance to this combination of features.) > > To avoid adding a long comment explaining the proxy pattern and why a svuid on > EnumSet isn't really required, my preference would just be to add the svuid if > it doesn't cause any harm. In this case I think it's possible by local reasoning (looking elsewhere in the EnumSet.java source code) to determine that EnumSet instances themselves are never actually serialized, because of the use of the serialization proxy pattern. If nothing else, adding a svuid here sets a bad example, so suppressing the warning is reasonable. I don't think a long comment is necessary. Probably something like the following is sufficient: @SuppressWarnings("serial") // no serialVersionUID because of serialization proxy s'marks From stuart.marks at oracle.com Fri Feb 28 17:27:26 2014 From: stuart.marks at oracle.com (Stuart Marks) Date: Fri, 28 Feb 2014 09:27:26 -0800 Subject: RFR 6835233 : Fedora 9 jdk regression test failed: java/lang/instrument/ParallelTransformerLoader.sh In-Reply-To: <530E41DB.20801@oracle.com> References: <530E41DB.20801@oracle.com> Message-ID: <5310C6FE.6040406@oracle.com> On 2/26/14 11:34 AM, Brent Christian wrote: > File under "chipping away at test stabilization issues." > > https://bugs.openjdk.java.net/browse/JDK-6835233 > > I would like to resolve this bug by removing the "@ignore" tag for JDK 9, and > bring the test back into rotation. If the failure comes back, I'll submit a new > issue for further investigation. +1 from me on "chipping away at test stabilization issues." :-) I guess there is some risk of adding new intermittent failures, but tackling @ignore'd tests is important too. Thanks for keeping an eye this. s'marks From robert.field at oracle.com Fri Feb 28 18:45:50 2014 From: robert.field at oracle.com (robert.field at oracle.com) Date: Fri, 28 Feb 2014 18:45:50 +0000 Subject: hg: jdk8/tl/jdk: 8035777: Consistent Lambda construction Message-ID: <20140228184805.22B6D6239C@hg.openjdk.java.net> Changeset: 183a8c520b4a Author: rfield Date: 2014-02-28 10:43 -0800 URL: http://hg.openjdk.java.net/jdk8/tl/jdk/rev/183a8c520b4a 8035777: Consistent Lambda construction Reviewed-by: ahgross, briangoetz, dlsmith ! src/share/classes/java/lang/invoke/AbstractValidatingLambdaMetafactory.java ! src/share/classes/java/lang/invoke/TypeConvertingMethodAdapter.java + test/java/lang/invoke/lambda/LambdaReceiver.java + test/java/lang/invoke/lambda/LambdaReceiverBridge.java + test/java/lang/invoke/lambda/LambdaReceiver_anotherpkg/LambdaReceiver_A.java + test/java/lang/invoke/lambda/LambdaReturn.java + test/java/lang/invoke/lambda/MetafactoryArityTest.java + test/java/lang/invoke/lambda/MetafactoryParameterCastTest.java + test/java/lang/invoke/lambda/MetafactorySamReturnTest.java From brent.christian at oracle.com Fri Feb 28 19:01:55 2014 From: brent.christian at oracle.com (Brent Christian) Date: Fri, 28 Feb 2014 11:01:55 -0800 Subject: RFR 6835233 : Fedora 9 jdk regression test failed: java/lang/instrument/ParallelTransformerLoader.sh In-Reply-To: <5310C6FE.6040406@oracle.com> References: <530E41DB.20801@oracle.com> <5310C6FE.6040406@oracle.com> Message-ID: <5310DD23.1040706@oracle.com> On 2/28/14 9:27 AM, Stuart Marks wrote: > > I guess there is some risk of adding new intermittent failures, but > tackling @ignore'd tests is important too. Right - the main risk is that we will see this test fail again at some point in the future. I'll be keeping an eye out for that. Thanks for the review, guys. -Brent From martinrb at google.com Fri Feb 28 20:24:57 2014 From: martinrb at google.com (Martin Buchholz) Date: Fri, 28 Feb 2014 12:24:57 -0800 Subject: Unsafe: removing the monitorEnter/monitorExit/tryMonitorEnter methods In-Reply-To: <530F55B0.1080907@redhat.com> References: <530F50DA.3010308@javaspecialists.eu> <530F55B0.1080907@redhat.com> Message-ID: I find myself in agreement with Heinz and David Loyd. j.u.c. locks have one primary advantage over builtin locks: flexibility. There used to be a performance advantage, but today it seems like builtin locks use fewer resources and can be preferred if the added flexibility is not required. So why not add back in some of the flexibility when it doesn't impose too much of a burden on the VM? Check out the public methods of ReentrantLock and see which are easy to support. For the purposes of the original usage of Unsafe in this thread. we want an isLocked method on builtin monitors. On Thu, Feb 27, 2014 at 7:11 AM, David M. Lloyd wrote: > A "Monitors" class could possibly be introduced which simply has three > (intrinsic) static methods: void lock(Object obj), void unlock(Object obj), > boolean tryLock(Object obj). > > Could even add a void lockInterruptibly(Object obj) throws > InterruptedException. > > Why not? Monitors are not going anywhere, and they're considerably > lighter than ReentrantLock in terms of memory usage and allocations in the > (pretty common) case where you only need zero or one conditions. Avoiding > something like this out of loyalty to new APIs is probably pretty misguided. > > > On 02/27/2014 08:51 AM, Dr Heinz M. Kabutz wrote: > >> I have never used this "in the wild", but rather have moved over to the >> ReentrantLock when I needed that particular functionality. >> >> However, I do see a place for this ability. As I wrote in here: >> http://www.javaspecialists.eu/archive/Issue194.html - sometimes you need >> to modify code that is already using a particular locking mechanism. To >> then redo everything with ReentrantLock can introduce errors. >> >> If I had a say, I would vote to either keep that method in Unsafe (which >> is where I think it belongs) or provide an alternative way to make the >> tryMonitorEnter() mechanism available to those that might end up needing >> it. >> >> Since I bought my Suzuki Jimny 7 years ago, I have not needed my spare >> wheel even once. So yeah, I could take it off and drive around without >> it. But chances are, the day after I take it off, I will need it. >> >> tryMonitorEnter() does no harm and it is out of reach of most programmers. >> >> Regards >> >> Heinz >> > > > -- > - DML > From john.r.rose at oracle.com Fri Feb 28 20:46:58 2014 From: john.r.rose at oracle.com (John Rose) Date: Fri, 28 Feb 2014 12:46:58 -0800 Subject: Signature of MethodHandleInfo.reflectAs is not specific enough In-Reply-To: References: <528008C9.7000905@univ-mlv.fr> Message-ID: <11925E47-E9C3-4234-ADE5-0599CC71254B@oracle.com> On Feb 25, 2014, at 3:13 AM, Ali Ebrahimi wrote: > I know, this is too late, but I want to share my suggestion: > > public T reflectAs(Class expected, MethodHandles.Lookup lookup) Isn't this the same as public T reflectAs... ? I think we considered AccessibleObject but rejected it as not buying anything significant compared with Member which is an interface. Perhaps public T reflectAs... with both interfaces, would have been slightly better. As the API is written (and yes it is too late to change) I don't think there are any use cases (at least with ground types) which require an extra cast. Thank you for looking at it. ? John From john.r.rose at oracle.com Fri Feb 28 21:11:00 2014 From: john.r.rose at oracle.com (John Rose) Date: Fri, 28 Feb 2014 13:11:00 -0800 Subject: Unsafe: removing the monitorEnter/monitorExit/tryMonitorEnter methods In-Reply-To: <144A3A2D-85FF-4F3B-A4DA-1780B809F36B@oracle.com> References: <530F50DA.3010308@javaspecialists.eu> <144A3A2D-85FF-4F3B-A4DA-1780B809F36B@oracle.com> Message-ID: On Feb 28, 2014, at 1:38 AM, Paul Sandoz wrote: >> But chances are, the day after I take it off, I will need it. >> >> tryMonitorEnter() does no harm and it is out of reach of most programmers. >> > > I think your analogy to a spare wheel is misleading. Continuing with the car theme i liken this feature to a pair of fluffy dice dangling from the rear view mirror [1]: distracting; mostly useless; and mostly harmless. The dice in your analogy are *too* useless; nobody can prove tryLock or isLocked useless, just in bad taste. It's more like a spare tire, maybe just for snow, which is really hard to access, as in "break glass in case of emergency". I threw it in, way back when, knowing the operation is portable and natural, and expected that folks would break the glass if they needed it. Despite the barrier to entry, a couple folks did. > All the data so far indicates this is legacy code [2]. I think the j.u.c library removed most needs to break the glass (connect to Unsafe). The users went to the standard API, and JVM native object monitors went out of fashion. But we are not ready to deprecate them, maybe never, so we should consider whether they need additional modernizations like tryLock or isLocked. > My suggestion, if you consider it important enough, is to write and circulate a draft JEP for such a public API [3]. I second Paul's suggestion here. If you like it, buy the unlocked version! The cost includes working the JEP process, and persuading someone to engineer it. Since JMM++ is proposing some.field.volatile.readVolatile(), can someObject.synchronized.isLocked() be far behind? (OK, ugly bikeshed; System.getObjectMonitor(someObject).isLocked().) Part of the cost of working the JEP conversation will be convincing people that the proposed APIs cannot be subverted somehow or will somehow lead the unwary to write bad code. The Unsafe API sidesteps such conversations. Maybe that is its permanent niche: A locker to store those power tools which we can't figure out how to make safely usable by everybody. (Yes, I used "permanent" and "sun.misc.Unsafe" in a single thought, along with "maybe". But let's try to wean ourselves away from it, at least bit by bit.) ? John From roger.riggs at oracle.com Fri Feb 28 21:45:44 2014 From: roger.riggs at oracle.com (roger riggs) Date: Fri, 28 Feb 2014 16:45:44 -0500 Subject: RFR 9: 8035106 Typo in java.time.format.Parsed error message Message-ID: <53110388.80104@oracle.com> Please review a typo correction in java.time.format.Parsed. The exception message is corrected. Webrev: http://cr.openjdk.java.net/~rriggs/webrev-parsed-typo-8035106/ Thanks, Roger From roger.riggs at oracle.com Fri Feb 28 21:48:13 2014 From: roger.riggs at oracle.com (roger riggs) Date: Fri, 28 Feb 2014 16:48:13 -0500 Subject: RFR 9: 8032491: DateTimeFormatter fixed width adjacent value parsing does not match spec Message-ID: <5311041D.8070105@oracle.com> Please review this bug fix from Steve Colebourne for java.time parsing of fixed width adjacent values. The patch includes new tests. Webrev: http://cr.openjdk.java.net/~rriggs/webrev-parse-adjacent-8032491/ Thanks, Roger From Lance.Andersen at oracle.com Fri Feb 28 21:51:16 2014 From: Lance.Andersen at oracle.com (Lance Andersen - Oracle) Date: Fri, 28 Feb 2014 16:51:16 -0500 Subject: RFR 9: 8035106 Typo in java.time.format.Parsed error message In-Reply-To: <53110388.80104@oracle.com> References: <53110388.80104@oracle.com> Message-ID: <074575A1-9877-40A1-B72E-3B90765C93C8@oracle.com> looks fine. Took me a moment to figure out the change but makes sense :-) On Feb 28, 2014, at 4:45 PM, roger riggs wrote: > Please review a typo correction in java.time.format.Parsed. > The exception message is corrected. > > Webrev: > http://cr.openjdk.java.net/~rriggs/webrev-parsed-typo-8035106/ > > Thanks, Roger > > -------------- next part -------------- Lance Andersen| Principal Member of Technical Staff | +1.781.442.2037 Oracle Java Engineering 1 Network Drive Burlington, MA 01803 Lance.Andersen at oracle.com From Lance.Andersen at oracle.com Fri Feb 28 22:05:58 2014 From: Lance.Andersen at oracle.com (Lance Andersen - Oracle) Date: Fri, 28 Feb 2014 17:05:58 -0500 Subject: RFR 9: 8032491: DateTimeFormatter fixed width adjacent value parsing does not match spec In-Reply-To: <5311041D.8070105@oracle.com> References: <5311041D.8070105@oracle.com> Message-ID: <07BE5097-4641-4085-9A44-01F99174F53F@oracle.com> Looks Ok. Kind of surprised the tck tests have no assertion details in the tests. Minor nit would have been nice to have even a minor comment for the new method DateTimeFormatterBuilder though that seems to be the norm in some scenarios for the smaller methods. On Feb 28, 2014, at 4:48 PM, roger riggs wrote: > Please review this bug fix from Steve Colebourne for java.time parsing of > fixed width adjacent values. The patch includes new tests. > > Webrev: > http://cr.openjdk.java.net/~rriggs/webrev-parse-adjacent-8032491/ > > Thanks, Roger > -------------- next part -------------- Lance Andersen| Principal Member of Technical Staff | +1.781.442.2037 Oracle Java Engineering 1 Network Drive Burlington, MA 01803 Lance.Andersen at oracle.com From david.x.li at oracle.com Fri Feb 28 22:11:58 2014 From: david.x.li at oracle.com (David Li) Date: Fri, 28 Feb 2014 14:11:58 -0800 Subject: RFR (JAXP): 8035469 : Xerces Update: EncodingMap does not recognise Java-style encodings Cp1141-Cp1149 Message-ID: <531109AE.2060203@oracle.com> Hi, This is an update from Xerces for a fixed encoding map entry in file EncodingMap.java. For details, please refer to: https://bugs.openjdk.java.net/browse/JDK-8035469 Webrevs: http://cr.openjdk.java.net/~joehw/jdk9/8035469/webrev/ (I don't have a openjdk username yet, so Joe Wang uploaded it) No new tests since the change is minor. There were no tests from Apache fixes. Thanks, David From Lance.Andersen at oracle.com Fri Feb 28 22:16:46 2014 From: Lance.Andersen at oracle.com (Lance Andersen - Oracle) Date: Fri, 28 Feb 2014 17:16:46 -0500 Subject: RFR (JAXP): 8035469 : Xerces Update: EncodingMap does not recognise Java-style encodings Cp1141-Cp1149 In-Reply-To: <531109AE.2060203@oracle.com> References: <531109AE.2060203@oracle.com> Message-ID: <19452860-5542-4EF5-90C7-F182C3791A02@oracle.com> Looks fine. On Feb 28, 2014, at 5:11 PM, David Li wrote: > Hi, > > This is an update from Xerces for a fixed encoding map entry in file EncodingMap.java. For details, please refer to: https://bugs.openjdk.java.net/browse/JDK-8035469 > > Webrevs: http://cr.openjdk.java.net/~joehw/jdk9/8035469/webrev/ > (I don't have a openjdk username yet, so Joe Wang uploaded it) > > No new tests since the change is minor. There were no tests from Apache fixes. > > Thanks, > David -------------- next part -------------- Lance Andersen| Principal Member of Technical Staff | +1.781.442.2037 Oracle Java Engineering 1 Network Drive Burlington, MA 01803 Lance.Andersen at oracle.com From brian.goetz at oracle.com Fri Feb 28 22:17:38 2014 From: brian.goetz at oracle.com (Brian Goetz) Date: Fri, 28 Feb 2014 17:17:38 -0500 Subject: Unsafe: removing the monitorEnter/monitorExit/tryMonitorEnter methods In-Reply-To: References: Message-ID: <53110B02.7070602@oracle.com> Except that Lock has features that are not supported by intrinsic locks (timed wait, interruptible wait.) So the Lock returned would not conform to Lock's contract, and attempting to call these methods would probably throw UOE. On 2/27/2014 6:12 AM, Stephen Colebourne wrote: > On 26 February 2014 20:54, Martin Buchholz wrote: >> It does seem that being able to tell whether a java object monitor is >> currently locked is useful for debugging and monitoring - there should be a >> way to do that. > > Perhaps it would be useful to be able to expose a java object monitor > as an instance of Lock? > > Lock lk = Lock.ofMonitor(object) > if (lk.tryLock()) { > ... > } > > Such a method feels like it would be a useful missing link between > synchronized and locks. > > Stephen > From roger.riggs at oracle.com Fri Feb 28 22:23:31 2014 From: roger.riggs at oracle.com (roger riggs) Date: Fri, 28 Feb 2014 17:23:31 -0500 Subject: RFR 9: 8032491: DateTimeFormatter fixed width adjacent value parsing does not match spec In-Reply-To: <07BE5097-4641-4085-9A44-01F99174F53F@oracle.com> References: <5311041D.8070105@oracle.com> <07BE5097-4641-4085-9A44-01F99174F53F@oracle.com> Message-ID: <53110C63.6000007@oracle.com> Hi Lance, I'll see if I can add some comments. The style for tests in java.time is less focused on statements in the spec due to the way the API developed. Perhaps the tests should have been just regular unit tests. Thanks, Roger On 2/28/2014 5:05 PM, Lance Andersen - Oracle wrote: > Looks Ok. Kind of surprised the tck tests have no assertion details in the tests. Minor nit would have been nice to have even a minor comment for the new method DateTimeFormatterBuilder though that seems to be the norm in some scenarios for the smaller methods. > > > On Feb 28, 2014, at 4:48 PM, roger riggs wrote: > >> Please review this bug fix from Steve Colebourne for java.time parsing of >> fixed width adjacent values. The patch includes new tests. >> >> Webrev: >> http://cr.openjdk.java.net/~rriggs/webrev-parse-adjacent-8032491/ >> >> Thanks, Roger >> > From roger.riggs at oracle.com Fri Feb 28 22:25:07 2014 From: roger.riggs at oracle.com (roger riggs) Date: Fri, 28 Feb 2014 17:25:07 -0500 Subject: RFR 9: 8035813: Broken link in java.lang.Iterable Message-ID: <53110CC3.8050000@oracle.com> Please review this fix for a broken link in the javadoc. Webrev: http://cr.openjdk.java.net/~rriggs/webrev-link-broken-8035813/ Thanks, Roger From martinrb at google.com Fri Feb 28 22:29:03 2014 From: martinrb at google.com (Martin Buchholz) Date: Fri, 28 Feb 2014 14:29:03 -0800 Subject: Unsafe: efficiently comparing two byte arrays In-Reply-To: References: <530F5136.20405@redhat.com> <8D3E2037-E5D1-4D2E-B779-E7EDD2AF4F58@oracle.com> <530F55CA.3060302@redhat.com> Message-ID: Are word-size reads in ByteBuffer actually intrinsified? I could find no evidence for that. Perhaps this is an important missing optimization for hotspot to make? I see DirectByteBuffer, but not garden-variety ByteBuffer, using Unsafe. share/classes/java/nio/Direct-X-Buffer.java.template Probably should be investigated using disassembly of jitted code, which I have never done. --- I've been on both sides of the Unsafe-using fence. On the one hand I helped implement LexicographicalComparator, and on the other dissuade others from adding even more usages of Unsafe. On Thu, Feb 27, 2014 at 9:23 AM, Paul Sandoz wrote: > > On Feb 27, 2014, at 4:12 PM, Andrew Haley wrote: > > > On 02/27/2014 03:05 PM, Paul Sandoz wrote: > >> > >> On Feb 27, 2014, at 3:52 PM, Andrew Haley wrote: > >> > >>> Hi, > >>> > >>> On 02/26/2014 03:42 PM, Paul Sandoz wrote: > >>> > >>>> A common reason why Unsafe is used is to more efficiently compare two > unsigned byte arrays, viewing those byte arrays as long arrays. > >>> > >>> Why not simply provide ByteBuffer.forArray(byte[]) ? Then we'd have > all the ByteBuffer intrinsics for put() and get() operations. > >> > >> Good point, even if it does require the creation of two ByteBuffer > instances. > >> > >> An implementation of Arrays.compare could do that if things were > suitably intrinsified, which i am not sure is the case as of today. > >> > >> I will do some measurements and report back. > > > > Unless something has been done in HotSpot pretty recently, the > > ByteBuffer intrinsics are not as performat as they might be, so the > > results might be a little underwhelming. > > > > Yes. > > For this benchmark: > > http://cr.openjdk.java.net/~psandoz/unsafe/ByteArrayCompareTest.java > > Here are results for comparing two byte arrays of size N containing the > same content, on my mac book: > > > http://cr.openjdk.java.net/~psandoz/unsafe/byte-array-compare-results.txt > > The unsafe test is about ~3/4 x faster than the safe test. And the buffer > test is slower than the safe test. > > I also included a variant of the unsafe test that uses Long.reverseBytes; > the results are similar to the unsafe test. I believe reverseBytes is > intrinsified as is numberOfTrailingZeros (i tried to disable it with > -XX:DisableIntrinsic=_reverseBytes_l but it did not make any great > different to the results, which makes me think i configured the option > incorrectly). > > Paul. > From Lance.Andersen at oracle.com Fri Feb 28 22:42:17 2014 From: Lance.Andersen at oracle.com (Lance Andersen - Oracle) Date: Fri, 28 Feb 2014 17:42:17 -0500 Subject: RFR 9: 8035813: Broken link in java.lang.Iterable In-Reply-To: <53110CC3.8050000@oracle.com> References: <53110CC3.8050000@oracle.com> Message-ID: <3E310F99-B680-486E-88C0-2E412DA122D3@oracle.com> +1 On Feb 28, 2014, at 5:25 PM, roger riggs wrote: > Please review this fix for a broken link in the javadoc. > > Webrev: > http://cr.openjdk.java.net/~rriggs/webrev-link-broken-8035813/ > > Thanks, Roger > -------------- next part -------------- Lance Andersen| Principal Member of Technical Staff | +1.781.442.2037 Oracle Java Engineering 1 Network Drive Burlington, MA 01803 Lance.Andersen at oracle.com From huizhe.wang at oracle.com Fri Feb 28 22:43:05 2014 From: huizhe.wang at oracle.com (huizhe wang) Date: Fri, 28 Feb 2014 14:43:05 -0800 Subject: RFR (JAXP): 8035469 : Xerces Update: EncodingMap does not recognise Java-style encodings Cp1141-Cp1149 In-Reply-To: <19452860-5542-4EF5-90C7-F182C3791A02@oracle.com> References: <531109AE.2060203@oracle.com> <19452860-5542-4EF5-90C7-F182C3791A02@oracle.com> Message-ID: <531110F9.2000303@oracle.com> Looks good. I'll sponsor the change. Thanks, Joe On 2/28/2014 2:16 PM, Lance Andersen - Oracle wrote: > Looks fine. > > On Feb 28, 2014, at 5:11 PM, David Li wrote: > >> Hi, >> >> This is an update from Xerces for a fixed encoding map entry in file EncodingMap.java. For details, please refer to: https://bugs.openjdk.java.net/browse/JDK-8035469 >> >> Webrevs: http://cr.openjdk.java.net/~joehw/jdk9/8035469/webrev/ >> (I don't have a openjdk username yet, so Joe Wang uploaded it) >> >> No new tests since the change is minor. There were no tests from Apache fixes. >> >> Thanks, >> David > > > Lance Andersen| Principal Member of Technical Staff | +1.781.442.2037 > Oracle Java Engineering > 1 Network Drive > Burlington, MA 01803 > Lance.Andersen at oracle.com > From aleksey.shipilev at oracle.com Fri Feb 28 22:48:01 2014 From: aleksey.shipilev at oracle.com (Aleksey Shipilev) Date: Sat, 01 Mar 2014 02:48:01 +0400 Subject: Unsafe: efficiently comparing two byte arrays In-Reply-To: References: <530F5136.20405@redhat.com> <8D3E2037-E5D1-4D2E-B779-E7EDD2AF4F58@oracle.com> <530F55CA.3060302@redhat.com> Message-ID: <53111221.2080707@oracle.com> On 03/01/2014 02:29 AM, Martin Buchholz wrote: > Are word-size reads in ByteBuffer actually intrinsified? I could find no > evidence for that. Perhaps this is an important missing optimization for > hotspot to make? It is a missing optimization, and we will get to that: https://bugs.openjdk.java.net/browse/JDK-8026049 -Aleksey. From aph at redhat.com Fri Feb 28 22:48:51 2014 From: aph at redhat.com (Andrew Haley) Date: Fri, 28 Feb 2014 22:48:51 +0000 Subject: Unsafe: efficiently comparing two byte arrays In-Reply-To: References: <530F5136.20405@redhat.com> <8D3E2037-E5D1-4D2E-B779-E7EDD2AF4F58@oracle.com> <530F55CA.3060302@redhat.com> Message-ID: <53111253.3090907@redhat.com> On 02/28/2014 10:29 PM, Martin Buchholz wrote: > Are word-size reads in ByteBuffer actually intrinsified? No. Andrew. From martinrb at google.com Fri Feb 28 22:58:06 2014 From: martinrb at google.com (Martin Buchholz) Date: Fri, 28 Feb 2014 14:58:06 -0800 Subject: Unsafe: efficiently comparing two byte arrays In-Reply-To: <53111221.2080707@oracle.com> References: <530F5136.20405@redhat.com> <8D3E2037-E5D1-4D2E-B779-E7EDD2AF4F58@oracle.com> <530F55CA.3060302@redhat.com> <53111221.2080707@oracle.com> Message-ID: Thank you! On Fri, Feb 28, 2014 at 2:48 PM, Aleksey Shipilev < aleksey.shipilev at oracle.com> wrote: > On 03/01/2014 02:29 AM, Martin Buchholz wrote: > > Are word-size reads in ByteBuffer actually intrinsified? I could find no > > evidence for that. Perhaps this is an important missing optimization for > > hotspot to make? > > It is a missing optimization, and we will get to that: > https://bugs.openjdk.java.net/browse/JDK-8026049 > > -Aleksey. > > From david.holmes at oracle.com Fri Feb 28 23:47:37 2014 From: david.holmes at oracle.com (David Holmes) Date: Sat, 01 Mar 2014 09:47:37 +1000 Subject: JDK-8036003: Add variable not to separate debug information. In-Reply-To: <5310AD70.2070908@ysfactory.dip.jp> References: <20140228091835.58EB8C21F24@msgw007-05.ocn.ad.jp> <53106F4A.8030905@oracle.com> <53109772.9070808@oracle.com> <5310AD70.2070908@ysfactory.dip.jp> Message-ID: <53112019.9050504@oracle.com> On 1/03/2014 1:38 AM, Yasumasa Suenaga wrote: >> The proper way to fix this is to disable FDS. > > Does this mean I have to pass --disable-debug-symbols to configure ? > I've added comment to JDK-8036003, I think if we pass --disable-debug-symbols > to configure, gcc/g++ is not passed -g option. "-g" is needed to generate > debuginfo. There are three pieces to all of this: 1. Generating debug symbols in the binaries (via gcc -g or whatever) 2. Generating debuginfo files (zipped or not) (FDS) 3. Stripping debug symbols from the binaries (strip-policy) It may be that we don't have clean separation between them, and if so that should be fixed, but I don't see the current proposal as the way forward. Also there may well be differences between how things are handled on the JDK side and hotspot side. I will try to look closer if I get a chance but my time is limited at the moment. David > > Thanks, > > Yasumasa > > > On 2014/02/28 23:04, Daniel D. Daugherty wrote: >> The proper way to fix this is to disable FDS. We should not need >> yet another option to control debug info. >> >> Dan >> >> >> On 2/28/14 4:13 AM, David Holmes wrote: >>> Hi, >>> >>> As I put in the bug report this seems way too complicated. Seems to me >>> all you need to do to get what you want is not use FDS and not strip the >>> symbols from the binary. The former is trivial. The latter is more >>> awkward as the strip policy stuff does not work as I would want it to >>> work, but still doable. >>> >>> David >>> >>> On 28/02/2014 7:18 PM, Yasumasa Suenaga wrote: >>>> Hi all, >>>> >>>> >>>> Currently, configure script can accept --disable-debug-symbols and >>>> --disable-zip-debug-info as controlling to generate debug information. >>>> However, current makefiles cannot build ELF binaries which is contained >>>> debug information with "images" target. >>>> >>>> Some Linux distros use RPM as package manager. >>>> RPM is built by rpmbuild command, it strips symbols and debug information >>>> during to generate rpm packages. >>>> https://fedoraproject.org/wiki/Packaging:Debuginfo >>>> >>>> For example, OpenJDK8 in Fedora20 ships libjvm.so and libjvm.debuginfo . >>>> libjvm.debuginfo is generated in OpenJDK's makefiles, however it does not >>>> contain debug information. Actual debug information is shipped by OpenJDK >>>> debuginfo package. >>>> This packaging is important when we have to debug JVM/JNI libraries. >>>> If we want to use debugging tools (GDB, SystemTap, etc...), they may requires >>>> debuginfo package. Debuggee (e.g. libjvm.so) points libjvm.so.debug which is >>>> located at sub directory in /usr/lib/debug . It is defined in ELF section >>>> (.note.gnu.build-id) of libjvm.so . However libjvm.so.debug does not contain >>>> valid debug information. We need to access libjvm.debuginfo.debug at same location. >>>> >>>> >>>> When we think to build OpenJDK rpm packages, we have to build ELF binaries >>>> which contain all debug information. we should not to separate debug information. >>>> >>>> >>>> Thus I want to add a variable "SEPARATED_DEBUGINFO_FILES" . >>>> If we pass "SEPARATED_DEBUGINFO_FILES=false" to make command, "make" does not >>>> execute objcopy command for generating debuginfo binaries. >>>> >>>> If we build rpm packages, we also have to add "STRIP_POLICY=no_strip" to arguments >>>> of make command. Or ELF binaries will be stripped. >>>> >>>> >>>> I've uploaded webrev for this enhancement. >>>> This is separated 3-part: top of forest, hotspot, jdk >>>> http://cr.openjdk.java.net/~ysuenaga/JDK-8036003/ >>>> >>>> Please review it and sponsoring! >>>> >>>> >>>> Thanks, >>>> >>>> Yasumasa >>>> >>>> >>>> P.S. >>>> I tried to add SEPARATED_DEBUGINFO_FILES as a configure option like >>>> "--disable-separated-debug-info" . >>>> I ran configure which is located at jdk9/dev directory after I ran autoconf >>>> and common/autoconf/autogen.sh. However it failed. >>>> >>>> I guess this is caused by changeset as below. >>>> JDK-8035495: Improvements in autoconf integration >>>> http://hg.openjdk.java.net/jdk9/dev/rev/6e29cd9ac2b4 >>>> >>>> This changes add "CHECKME" option to configure script. However, this changes >>>> affects "configure" only. It should change "configure.ac" . >>>> >> >