From jonathan.gibbons at oracle.com Thu Apr 2 23:52:04 2015 From: jonathan.gibbons at oracle.com (Jonathan Gibbons) Date: Thu, 02 Apr 2015 16:52:04 -0700 Subject: RFR: JDK-8076583: move jdk.Exported from langtools to jdk Message-ID: <551DD624.4060701@oracle.com> Sorry for the relatively wide distribution. JDK-8076583 is a conceptually simple cleanup, to move the source file for the jdk.Exported class from the langtools repo (where it is a singleton outlier) to the jdk repo (alongside most of the rest of the classes in the jdk package hierarchy). The class was originally placed in the langtools repo for bootstrapping reasons that no longer apply. As a result of moving the source file, references to java.base in a number of places in the langtools repo can be cleaned up. @Build folk: There is a trivial change to a makefile in the langtools repo. @Core-libs folk: The source file gets moved into the jdk/ repo. @Compiler-dev folk: We can remove references to the java.base folder from the private Ant build, and from IDE support files. JBS: https://bugs.openjdk.java.net/browse/JDK-8076583 Webrev: http://cr.openjdk.java.net/~jjg/8076583/webrev.00/ -- Jon From joe.darcy at oracle.com Fri Apr 3 01:27:58 2015 From: joe.darcy at oracle.com (Joseph D. Darcy) Date: Thu, 02 Apr 2015 18:27:58 -0700 Subject: RFR: JDK-8076583: move jdk.Exported from langtools to jdk In-Reply-To: <551DD624.4060701@oracle.com> References: <551DD624.4060701@oracle.com> Message-ID: <551DEC9E.3050906@oracle.com> Jon, The actual file move looks fine to me. Thanks, -Joe On 4/2/2015 4:52 PM, Jonathan Gibbons wrote: > Sorry for the relatively wide distribution. > > JDK-8076583 is a conceptually simple cleanup, to move the source file > for the jdk.Exported class from the langtools repo (where it is a > singleton outlier) to the jdk repo (alongside most of the rest of the > classes in the jdk package hierarchy). The class was originally placed > in the langtools repo for bootstrapping reasons that no longer apply. > > As a result of moving the source file, references to java.base in a > number of places in the langtools repo can be cleaned up. > > @Build folk: > There is a trivial change to a makefile in the langtools repo. > > @Core-libs folk: > The source file gets moved into the jdk/ repo. > > @Compiler-dev folk: > We can remove references to the java.base folder from the private Ant > build, and from IDE support files. > > JBS: https://bugs.openjdk.java.net/browse/JDK-8076583 > Webrev: http://cr.openjdk.java.net/~jjg/8076583/webrev.00/ > > -- Jon From Alan.Bateman at oracle.com Fri Apr 3 06:47:34 2015 From: Alan.Bateman at oracle.com (Alan Bateman) Date: Fri, 03 Apr 2015 07:47:34 +0100 Subject: RFR: JDK-8076583: move jdk.Exported from langtools to jdk In-Reply-To: <551DD624.4060701@oracle.com> References: <551DD624.4060701@oracle.com> Message-ID: <551E3786.9090005@oracle.com> On 03/04/2015 00:52, Jonathan Gibbons wrote: > Sorry for the relatively wide distribution. > > JDK-8076583 is a conceptually simple cleanup, to move the source file > for the jdk.Exported class from the langtools repo (where it is a > singleton outlier) to the jdk repo (alongside most of the rest of the > classes in the jdk package hierarchy). The class was originally placed > in the langtools repo for bootstrapping reasons that no longer apply. > > As a result of moving the source file, references to java.base in a > number of places in the langtools repo can be cleaned up. > > @Build folk: > There is a trivial change to a makefile in the langtools repo. > > @Core-libs folk: > The source file gets moved into the jdk/ repo. The move looks okay although the new home might be temporary given the design principles in JEP 200 (meaning java.base should not export the jdk API package). -Alan From mandy.chung at oracle.com Fri Apr 3 14:47:49 2015 From: mandy.chung at oracle.com (Mandy Chung) Date: Fri, 03 Apr 2015 07:47:49 -0700 Subject: RFR: JDK-8076583: move jdk.Exported from langtools to jdk In-Reply-To: <551DD624.4060701@oracle.com> References: <551DD624.4060701@oracle.com> Message-ID: <551EA815.4010408@oracle.com> On 4/2/2015 4:52 PM, Jonathan Gibbons wrote: > Sorry for the relatively wide distribution. > > JDK-8076583 is a conceptually simple cleanup, to move the source file > for the jdk.Exported class from the langtools repo (where it is a > singleton outlier) to the jdk repo (alongside most of the rest of the > classes in the jdk package hierarchy). The class was originally placed > in the langtools repo for bootstrapping reasons that no longer apply. > > As a result of moving the source file, references to java.base in a > number of places in the langtools repo can be cleaned up. > > @Build folk: > There is a trivial change to a makefile in the langtools repo. > > @Core-libs folk: > The source file gets moved into the jdk/ repo. > > @Compiler-dev folk: > We can remove references to the java.base folder from the private Ant > build, and from IDE support files. > > JBS: https://bugs.openjdk.java.net/browse/JDK-8076583 > Webrev: http://cr.openjdk.java.net/~jjg/8076583/webrev.00/ Moving jdk.Exported to jdk repo is fine. Mandy From jan.lahoda at oracle.com Fri Apr 3 17:34:17 2015 From: jan.lahoda at oracle.com (Jan Lahoda) Date: Fri, 03 Apr 2015 19:34:17 +0200 Subject: RFR: JDK-8076583: move jdk.Exported from langtools to jdk In-Reply-To: <551DD624.4060701@oracle.com> References: <551DD624.4060701@oracle.com> Message-ID: <551ECF19.70305@oracle.com> On 3.4.2015 01:52, Jonathan Gibbons wrote: > Sorry for the relatively wide distribution. > > JDK-8076583 is a conceptually simple cleanup, to move the source file > for the jdk.Exported class from the langtools repo (where it is a > singleton outlier) to the jdk repo (alongside most of the rest of the > classes in the jdk package hierarchy). The class was originally placed > in the langtools repo for bootstrapping reasons that no longer apply. > > As a result of moving the source file, references to java.base in a > number of places in the langtools repo can be cleaned up. > > @Build folk: > There is a trivial change to a makefile in the langtools repo. > > @Core-libs folk: > The source file gets moved into the jdk/ repo. > > @Compiler-dev folk: > We can remove references to the java.base folder from the private Ant > build, and from IDE support files. These changes seem fine to me. I am not that big expert on the IntelliJ support, though. Jan > > JBS: https://bugs.openjdk.java.net/browse/JDK-8076583 > Webrev: http://cr.openjdk.java.net/~jjg/8076583/webrev.00/ > > -- Jon From tim.bell at oracle.com Fri Apr 3 20:14:43 2015 From: tim.bell at oracle.com (Tim Bell) Date: Fri, 03 Apr 2015 13:14:43 -0700 Subject: RFR: JDK-8076583: move jdk.Exported from langtools to jdk In-Reply-To: <551DD624.4060701@oracle.com> References: <551DD624.4060701@oracle.com> Message-ID: <551EF4B3.6080904@oracle.com> > @Build folk: > There is a trivial change to a makefile in the langtools repo. Looks good to me. Tim On 04/02/15 16:52, Jonathan Gibbons wrote: > Sorry for the relatively wide distribution. > > JDK-8076583 is a conceptually simple cleanup, to move the source file > for the jdk.Exported class from the langtools repo (where it is a > singleton outlier) to the jdk repo (alongside most of the rest of the > classes in the jdk package hierarchy). The class was originally placed > in the langtools repo for bootstrapping reasons that no longer apply. > > As a result of moving the source file, references to java.base in a > number of places in the langtools repo can be cleaned up. > > @Build folk: > There is a trivial change to a makefile in the langtools repo. > > @Core-libs folk: > The source file gets moved into the jdk/ repo. > > @Compiler-dev folk: > We can remove references to the java.base folder from the private Ant > build, and from IDE support files. > > JBS: https://bugs.openjdk.java.net/browse/JDK-8076583 > Webrev: http://cr.openjdk.java.net/~jjg/8076583/webrev.00/ > > -- Jon From lowasser at google.com Tue Apr 7 21:10:52 2015 From: lowasser at google.com (Louis Wasserman) Date: Tue, 07 Apr 2015 21:10:52 +0000 Subject: String concatenation tweaks In-Reply-To: <55244704.8080709@gmail.com> References: <5500E84F.3080800@oracle.com> <55022114.8040904@oracle.com> <55034189.4000900@oracle.com> <55244704.8080709@gmail.com> Message-ID: Could that actually be provided immutability-safely? I suppose an append-only, fixed-length builder would be potentially safe. Part of the trickiness there is with primitive parameters, where presizing and doing the actual append both require calculating the size of the primitive when converted to a string. The current approach just uses an upper bound for the primitive type as a whole, which is fine and allocates a small constant excess in most cases, but I'm not sure how we could avoid duplicating that computation in a public API. On Tue, Apr 7, 2015 at 2:07 PM Peter Levart wrote: > Hi Louis, > > This is nice. Amd could be even nicer. In case the estimated initial > StringBuilder capacity is exactly the final String length, then > constructing the String could skip the char[] copying step as the > StringBuilder instance does not escape. But in order to be safe, this would > have to be a special kind of StringBuilder. Like the following: > > > http://cr.openjdk.java.net/~plevart/misc/ThreadLocalStringBuilder/webrev.01/ > > Such class would be useful for direct API use too. > > > Regards, Peter > > > On 03/13/2015 10:40 PM, Louis Wasserman wrote: > > Got it. I think the only cases we have to worry about, then, are buffer > size overflows resulting in NegativeArraySizeException, or possibly an > explicitly thrown OutOfMemoryError (which is StringBuilder's response when > the buffer size tries to exceed Integer.MAX_VALUE). I think we might > conceivably deal with this by rewriting the bytecode to -- I think we can > improve on this with jump hackery to avoid repetition, but essentially -- > > int length = 3; // sum of the constant strings; we can check that this > won't overflow at compile time but I think it couldn't no matter what > because of method length constraints > String mStr = m().toString(); > length += mStr.length(); > if (length < 0) { > throw new OutOfMemoryError(); > } > String nStr = n().toString(); > length += nStr.length(); > if (length < 0) { > throw new OutOfMemoryError(); > } > > This continues to expand the bytecode, but probably manageably -- we > don't actually need a local for length; if (int < 0) is easy in bytecode, > and we can have only one OOME site that all the ifs jump to? > > On Fri, Mar 13, 2015 at 12:59 PM Alex Buckley > wrote: > >> I do recognize that the proposed implementation doesn't reorder the >> evaluation of subexpressions. >> >> When discussing the proposed implementation of '+' -- whose key element >> is calling append(String) with a pre-computed value -- I was careful to >> set aside asynchronous OOMEs, but I see that even synchronous OOMEs are >> sidetracking us. My concern is not heap pressure; my concern is >> arbitrary unchecked exceptions arising from the (int) and >> append(String) calls. >> >> For sake of argument, I'll simplify "unchecked exceptions" to just >> RuntimeExceptions, not Errors. If you can guarantee that no >> RuntimeExceptions are thrown synchronously during the execution of those >> method bodies on the JVM, then '+' cannot fail and the timing of >> subexpression evaluation is unobservable (ordering is still observable, >> as required). I think this guarantee is just a matter of reviewing the >> method bodies. >> >> Alex >> >> On 3/12/2015 6:01 PM, Louis Wasserman wrote: >> > I confess I'm not sure how that "quality" goal would be achievable in >> > bytecode without deliberately allocating arrays we then discard. >> > >> > For what it's worth, delaying or avoiding OOMEs seems a desirable goal >> > in general, and up to a constant multiplicative factor, this >> > implementation seems to allocate the same amount in the same order. >> > That is, we're still computing m().toString() before n().toString(), and >> > up to a constant multiplicative factor, m().toString() allocates the >> > same number of bytes as the StringBuilder the status quo generates. So >> > if m() does something like allocate a char[Integer.MAX_VALUE], we still >> > OOM at the appropriate time. >> > >> > Other notes: this implementation would tend to decrease maximum >> > allocation, so it'd reduce OOMEs. Also, since the StringBuilder will >> > never need further expansion and we're only using the String and >> > primitive overloads of append, the only way for append to OOME would be >> > in append(float) and append(double), which allocate a FloatingDecimal >> > (which may, in turn, allocate a new thread-local char[26] if one isn't >> > already there). >> > >> > On Thu, Mar 12, 2015 at 4:28 PM Alex Buckley > > > wrote: >> > >> > More abstract presentation. Given the expression: >> > >> > "foo" + m() + n() >> > >> > you must not evaluate n() if evaluation of "foo" + m() completes >> > abruptly. The proposed implementation evaluates n() regardless. >> > >> > All is not lost. In the proposed implementation, the abrupt >> completion >> > of "foo" + m() could occur because an append call fails or (thanks >> to >> > Jon for pointing this out) the StringBuilder ctor fails. The >> > quality-of-implementation issue is thus: if the proposed >> implementation >> > is of sufficiently high quality to guarantee that the ctor and the >> first >> > append both succeed, then the evaluation of "foo" + m() will always >> > complete normally, and it would be an unobservable (thus acceptable) >> > implementation detail to evaluate n() early. >> > >> > Alex >> > >> > On 3/11/2015 10:26 PM, Jeremy Manson wrote: >> > > Isn't Louis's proposed behavior equivalent to saying "the >> rightmost >> > > concatenation threw an OOME" instead of "some concatenation in >> the >> > > middle threw an OOME"? >> > > >> > > It's true that the intermediate String concatenations haven't >> > occurred >> > > at that point, but in the JDK's current implementation, that's >> true, >> > > too: the concatenations that have occurred at that point are >> > > StringBuilder ones, not String ones. If any of the append >> operations >> > > throws an OOME, no Strings have been created at all, either in >> > Louis's >> > > implementation or in the JDK's. >> > > >> > > Ultimately, isn't this a quality of implementation issue? And >> if so, >> > > isn't it a quality of implementation issue that doesn't provide >> any >> > > additional quality? I can't imagine code whose semantics relies >> on >> > > this, and if they do, they are relying on something >> > > implementation-dependent. >> > > >> > > Jeremy >> > > >> > > On Wed, Mar 11, 2015 at 6:13 PM, Alex Buckley >> > >> > > > > >> wrote: >> > > >> > > On 3/11/2015 2:01 PM, Louis Wasserman wrote: >> > > >> > > So for example, "foo" + myInt + myString + "bar" + myObj >> > would be >> > > compiled to the equivalent of >> > > >> > > int myIntTmp = myInt; >> > > String myStringTmp = String.valueOf(myString); // defend >> > against >> > > null >> > > String myObjTmp = >> > String.valueOf(String.valueOf(____myObj)); // defend >> > > against evil toString implementations returning null >> > > >> > > return new StringBuilder( >> > > 17 // length of "foo" (3) + max length of myInt >> (11) + >> > > length of >> > > "bar" (3) >> > > + myStringTmp.length() >> > > + myObjTmp.length()) >> > > .append("foo") >> > > .append(myIntTmp) >> > > .append(myStringTmp) >> > > .append("bar") >> > > .append(myObjTmp) >> > > .toString(); >> > > >> > > As far as language constraints go, the JLS is (apparently >> > > deliberately) >> > > vague about how string concatenation is implemented. "An >> > > implementation >> > > may choose to perform conversion and concatenation in one >> > step >> > > to avoid >> > > creating and then discarding an intermediate String >> > object. To >> > > increase >> > > the performance of repeated string concatenation, a Java >> > > compiler may >> > > use the StringBuffer class or a similar technique to >> > reduce the >> > > number >> > > of intermediate String objects that are created by >> > evaluation of an >> > > expression." We see no reason this approach would not >> > qualify as a >> > > "similar technique." >> > > >> > > >> > > The really key property of the string concatenation operator >> is >> > > left-associativity. Later subexpressions must not be >> > evaluated until >> > > earlier subexpressions have been successfully evaluated AND >> > > concatenated. Consider this expression: >> > > >> > > "foo" + m() + n() >> > > >> > > which JLS8 15.8 specifies to mean: >> > > >> > > ("foo" + m()) + n() >> > > >> > > We know from JLS8 15.6 that if m() throws, then foo+m() >> > throws, and >> > > n() will never be evaluated. >> > > >> > > Happily, your translation doesn't appear to catch and swallow >> > > exceptions when eagerly evaluating each subexpression in >> > turn, so I >> > > believe you won't evaluate n() if m() already threw. >> > > >> > > Unhappily, a call to append(..) can in general fail with >> > > OutOfMemoryError. (I'm not talking about asynchronous >> > exceptions in >> > > general, but rather the sense that append(..) manipulates the >> > heap >> > > so an OOME is at least plausible.) In the OpenJDK >> > implementation, if >> > > blah.append(m()) fails with OOME, then n() hasn't been >> > evaluated yet >> > > -- that's "real" left-associativity. In the proposed >> > implementation, >> > > it's possible that more memory is available when evaluating >> > m() and >> > > n() upfront than at the time of an append call, so n() is >> > evaluated >> > > even if append(<>) fails -- that's not >> > > left-associative. >> > > >> > > Perhaps you can set my mind at ease that append(..) can't >> > fail with >> > > OOME? >> > > >> > > Alex >> > > >> > > >> > >> > > -------------- next part -------------- An HTML attachment was scrubbed... URL: From peter.levart at gmail.com Tue Apr 7 21:07:16 2015 From: peter.levart at gmail.com (Peter Levart) Date: Tue, 07 Apr 2015 23:07:16 +0200 Subject: String concatenation tweaks In-Reply-To: References: <5500E84F.3080800@oracle.com> <55022114.8040904@oracle.com> <55034189.4000900@oracle.com> Message-ID: <55244704.8080709@gmail.com> Hi Louis, This is nice. Amd could be even nicer. In case the estimated initial StringBuilder capacity is exactly the final String length, then constructing the String could skip the char[] copying step as the StringBuilder instance does not escape. But in order to be safe, this would have to be a special kind of StringBuilder. Like the following: http://cr.openjdk.java.net/~plevart/misc/ThreadLocalStringBuilder/webrev.01/ Such class would be useful for direct API use too. Regards, Peter On 03/13/2015 10:40 PM, Louis Wasserman wrote: > Got it. I think the only cases we have to worry about, then, are > buffer size overflows resulting in NegativeArraySizeException, or > possibly an explicitly thrown OutOfMemoryError (which is > StringBuilder's response when the buffer size tries to exceed > Integer.MAX_VALUE). I think we might conceivably deal with this by > rewriting the bytecode to -- I think we can improve on this with jump > hackery to avoid repetition, but essentially -- > > int length = 3; // sum of the constant strings; we can check that this > won't overflow at compile time but I think it couldn't no matter what > because of method length constraints > String mStr = m().toString(); > length += mStr.length(); > if (length < 0) { > throw new OutOfMemoryError(); > } > String nStr = n().toString(); > length += nStr.length(); > if (length < 0) { > throw new OutOfMemoryError(); > } > > This continues to expand the bytecode, but probably manageably -- we > don't actually need a local for length; if (int < 0) is easy in > bytecode, and we can have only one OOME site that all the ifs jump to? > > On Fri, Mar 13, 2015 at 12:59 PM Alex Buckley > wrote: > > I do recognize that the proposed implementation doesn't reorder the > evaluation of subexpressions. > > When discussing the proposed implementation of '+' -- whose key > element > is calling append(String) with a pre-computed value -- I was > careful to > set aside asynchronous OOMEs, but I see that even synchronous > OOMEs are > sidetracking us. My concern is not heap pressure; my concern is > arbitrary unchecked exceptions arising from the (int) and > append(String) calls. > > For sake of argument, I'll simplify "unchecked exceptions" to just > RuntimeExceptions, not Errors. If you can guarantee that no > RuntimeExceptions are thrown synchronously during the execution of > those > method bodies on the JVM, then '+' cannot fail and the timing of > subexpression evaluation is unobservable (ordering is still > observable, > as required). I think this guarantee is just a matter of reviewing the > method bodies. > > Alex > > On 3/12/2015 6:01 PM, Louis Wasserman wrote: > > I confess I'm not sure how that "quality" goal would be > achievable in > > bytecode without deliberately allocating arrays we then discard. > > > > For what it's worth, delaying or avoiding OOMEs seems a > desirable goal > > in general, and up to a constant multiplicative factor, this > > implementation seems to allocate the same amount in the same order. > > That is, we're still computing m().toString() before > n().toString(), and > > up to a constant multiplicative factor, m().toString() allocates the > > same number of bytes as the StringBuilder the status quo > generates. So > > if m() does something like allocate a char[Integer.MAX_VALUE], > we still > > OOM at the appropriate time. > > > > Other notes: this implementation would tend to decrease maximum > > allocation, so it'd reduce OOMEs. Also, since the StringBuilder > will > > never need further expansion and we're only using the String and > > primitive overloads of append, the only way for append to OOME > would be > > in append(float) and append(double), which allocate a > FloatingDecimal > > (which may, in turn, allocate a new thread-local char[26] if one > isn't > > already there). > > > > On Thu, Mar 12, 2015 at 4:28 PM Alex Buckley > > > >> wrote: > > > > More abstract presentation. Given the expression: > > > > "foo" + m() + n() > > > > you must not evaluate n() if evaluation of "foo" + m() completes > > abruptly. The proposed implementation evaluates n() regardless. > > > > All is not lost. In the proposed implementation, the abrupt > completion > > of "foo" + m() could occur because an append call fails or > (thanks to > > Jon for pointing this out) the StringBuilder ctor fails. The > > quality-of-implementation issue is thus: if the proposed > implementation > > is of sufficiently high quality to guarantee that the ctor > and the first > > append both succeed, then the evaluation of "foo" + m() will > always > > complete normally, and it would be an unobservable (thus > acceptable) > > implementation detail to evaluate n() early. > > > > Alex > > > > On 3/11/2015 10:26 PM, Jeremy Manson wrote: > > > Isn't Louis's proposed behavior equivalent to saying "the > rightmost > > > concatenation threw an OOME" instead of "some > concatenation in the > > > middle threw an OOME"? > > > > > > It's true that the intermediate String concatenations haven't > > occurred > > > at that point, but in the JDK's current implementation, > that's true, > > > too: the concatenations that have occurred at that point are > > > StringBuilder ones, not String ones. If any of the > append operations > > > throws an OOME, no Strings have been created at all, > either in > > Louis's > > > implementation or in the JDK's. > > > > > > Ultimately, isn't this a quality of implementation > issue? And if so, > > > isn't it a quality of implementation issue that doesn't > provide any > > > additional quality? I can't imagine code whose semantics > relies on > > > this, and if they do, they are relying on something > > > implementation-dependent. > > > > > > Jeremy > > > > > > On Wed, Mar 11, 2015 at 6:13 PM, Alex Buckley > > > > > > > __com > > >>> wrote: > > > > > > On 3/11/2015 2:01 PM, Louis Wasserman wrote: > > > > > > So for example, "foo" + myInt + myString + "bar" > + myObj > > would be > > > compiled to the equivalent of > > > > > > int myIntTmp = myInt; > > > String myStringTmp = String.valueOf(myString); // > defend > > against > > > null > > > String myObjTmp = > > String.valueOf(String.valueOf(____myObj)); // defend > > > against evil toString implementations returning null > > > > > > return new StringBuilder( > > > 17 // length of "foo" (3) + max length of > myInt (11) + > > > length of > > > "bar" (3) > > > + myStringTmp.length() > > > + myObjTmp.length()) > > > .append("foo") > > > .append(myIntTmp) > > > .append(myStringTmp) > > > .append("bar") > > > .append(myObjTmp) > > > .toString(); > > > > > > As far as language constraints go, the JLS is > (apparently > > > deliberately) > > > vague about how string concatenation is > implemented. "An > > > implementation > > > may choose to perform conversion and > concatenation in one > > step > > > to avoid > > > creating and then discarding an intermediate String > > object. To > > > increase > > > the performance of repeated string concatenation, > a Java > > > compiler may > > > use the StringBuffer class or a similar technique to > > reduce the > > > number > > > of intermediate String objects that are created by > > evaluation of an > > > expression." We see no reason this approach > would not > > qualify as a > > > "similar technique." > > > > > > > > > The really key property of the string concatenation > operator is > > > left-associativity. Later subexpressions must not be > > evaluated until > > > earlier subexpressions have been successfully > evaluated AND > > > concatenated. Consider this expression: > > > > > > "foo" + m() + n() > > > > > > which JLS8 15.8 specifies to mean: > > > > > > ("foo" + m()) + n() > > > > > > We know from JLS8 15.6 that if m() throws, then foo+m() > > throws, and > > > n() will never be evaluated. > > > > > > Happily, your translation doesn't appear to catch and > swallow > > > exceptions when eagerly evaluating each subexpression in > > turn, so I > > > believe you won't evaluate n() if m() already threw. > > > > > > Unhappily, a call to append(..) can in general fail with > > > OutOfMemoryError. (I'm not talking about asynchronous > > exceptions in > > > general, but rather the sense that append(..) > manipulates the > > heap > > > so an OOME is at least plausible.) In the OpenJDK > > implementation, if > > > blah.append(m()) fails with OOME, then n() hasn't been > > evaluated yet > > > -- that's "real" left-associativity. In the proposed > > implementation, > > > it's possible that more memory is available when > evaluating > > m() and > > > n() upfront than at the time of an append call, so n() is > > evaluated > > > even if append(<>) fails -- that's not > > > left-associative. > > > > > > Perhaps you can set my mind at ease that append(..) can't > > fail with > > > OOME? > > > > > > Alex > > > > > > > > > -------------- next part -------------- An HTML attachment was scrubbed... URL: From peter.levart at gmail.com Tue Apr 7 21:37:02 2015 From: peter.levart at gmail.com (Peter Levart) Date: Tue, 07 Apr 2015 23:37:02 +0200 Subject: String concatenation tweaks In-Reply-To: References: <5500E84F.3080800@oracle.com> <55022114.8040904@oracle.com> <55034189.4000900@oracle.com> <55244704.8080709@gmail.com> Message-ID: <55244DFE.2050200@gmail.com> On 04/07/2015 11:10 PM, Louis Wasserman wrote: > Could that actually be provided immutability-safely? I suppose an > append-only, fixed-length builder would be potentially safe. I think the presented implementation *is* thread and immutability safe. It only allows mutations from the constructing thread and disables them as soon as toString() is called. > > Part of the trickiness there is with primitive parameters, where > presizing and doing the actual append both require calculating the > size of the primitive when converted to a string. The current > approach just uses an upper bound for the primitive type as a whole, > which is fine and allocates a small constant excess in most cases, but > I'm not sure how we could avoid duplicating that computation in a > public API. I've been thinking of static methods parallel to: String.valueOf(int) - String.lengthOf(int) String.valueOf(long) - String.lengthOf(long) ... ... The integral variants could be quick to compute (the implementations are already there - they are just not public), the floating point maybe not so. But the floating point conversion is already taking much more overhead and estimating an upper bound would be fine in case floating point is involved. If one wants to optimize the case of very long concatenation of exactly sized Strings, integrals and other Objects with just one or few floating point values, then he can always do it explicitly: "....long string...." + String.valueOf(Math.PI); Regards, Peter > > On Tue, Apr 7, 2015 at 2:07 PM Peter Levart > wrote: > > Hi Louis, > > This is nice. Amd could be even nicer. In case the estimated > initial StringBuilder capacity is exactly the final String length, > then constructing the String could skip the char[] copying step as > the StringBuilder instance does not escape. But in order to be > safe, this would have to be a special kind of StringBuilder. Like > the following: > > http://cr.openjdk.java.net/~plevart/misc/ThreadLocalStringBuilder/webrev.01/ > > > Such class would be useful for direct API use too. > > > Regards, Peter > > > On 03/13/2015 10:40 PM, Louis Wasserman wrote: >> Got it. I think the only cases we have to worry about, then, are >> buffer size overflows resulting in NegativeArraySizeException, or >> possibly an explicitly thrown OutOfMemoryError (which is >> StringBuilder's response when the buffer size tries to exceed >> Integer.MAX_VALUE). I think we might conceivably deal with this >> by rewriting the bytecode to -- I think we can improve on this >> with jump hackery to avoid repetition, but essentially -- >> >> int length = 3; // sum of the constant strings; we can check that >> this won't overflow at compile time but I think it couldn't no >> matter what because of method length constraints >> String mStr = m().toString(); >> length += mStr.length(); >> if (length < 0) { >> throw new OutOfMemoryError(); >> } >> String nStr = n().toString(); >> length += nStr.length(); >> if (length < 0) { >> throw new OutOfMemoryError(); >> } >> >> This continues to expand the bytecode, but probably manageably -- >> we don't actually need a local for length; if (int < 0) is easy >> in bytecode, and we can have only one OOME site that all the ifs >> jump to? >> >> On Fri, Mar 13, 2015 at 12:59 PM Alex Buckley >> > wrote: >> >> I do recognize that the proposed implementation doesn't >> reorder the >> evaluation of subexpressions. >> >> When discussing the proposed implementation of '+' -- whose >> key element >> is calling append(String) with a pre-computed value -- I was >> careful to >> set aside asynchronous OOMEs, but I see that even synchronous >> OOMEs are >> sidetracking us. My concern is not heap pressure; my concern is >> arbitrary unchecked exceptions arising from the (int) and >> append(String) calls. >> >> For sake of argument, I'll simplify "unchecked exceptions" to >> just >> RuntimeExceptions, not Errors. If you can guarantee that no >> RuntimeExceptions are thrown synchronously during the >> execution of those >> method bodies on the JVM, then '+' cannot fail and the timing of >> subexpression evaluation is unobservable (ordering is still >> observable, >> as required). I think this guarantee is just a matter of >> reviewing the >> method bodies. >> >> Alex >> >> On 3/12/2015 6:01 PM, Louis Wasserman wrote: >> > I confess I'm not sure how that "quality" goal would be >> achievable in >> > bytecode without deliberately allocating arrays we then >> discard. >> > >> > For what it's worth, delaying or avoiding OOMEs seems a >> desirable goal >> > in general, and up to a constant multiplicative factor, this >> > implementation seems to allocate the same amount in the >> same order. >> > That is, we're still computing m().toString() before >> n().toString(), and >> > up to a constant multiplicative factor, m().toString() >> allocates the >> > same number of bytes as the StringBuilder the status quo >> generates. So >> > if m() does something like allocate a >> char[Integer.MAX_VALUE], we still >> > OOM at the appropriate time. >> > >> > Other notes: this implementation would tend to decrease maximum >> > allocation, so it'd reduce OOMEs. Also, since the >> StringBuilder will >> > never need further expansion and we're only using the >> String and >> > primitive overloads of append, the only way for append to >> OOME would be >> > in append(float) and append(double), which allocate a >> FloatingDecimal >> > (which may, in turn, allocate a new thread-local char[26] >> if one isn't >> > already there). >> > >> > On Thu, Mar 12, 2015 at 4:28 PM Alex Buckley >> >> > > >> wrote: >> > >> > More abstract presentation. Given the expression: >> > >> > "foo" + m() + n() >> > >> > you must not evaluate n() if evaluation of "foo" + m() >> completes >> > abruptly. The proposed implementation evaluates n() >> regardless. >> > >> > All is not lost. In the proposed implementation, the >> abrupt completion >> > of "foo" + m() could occur because an append call fails >> or (thanks to >> > Jon for pointing this out) the StringBuilder ctor >> fails. The >> > quality-of-implementation issue is thus: if the >> proposed implementation >> > is of sufficiently high quality to guarantee that the >> ctor and the first >> > append both succeed, then the evaluation of "foo" + m() >> will always >> > complete normally, and it would be an unobservable >> (thus acceptable) >> > implementation detail to evaluate n() early. >> > >> > Alex >> > >> > On 3/11/2015 10:26 PM, Jeremy Manson wrote: >> > > Isn't Louis's proposed behavior equivalent to saying >> "the rightmost >> > > concatenation threw an OOME" instead of "some >> concatenation in the >> > > middle threw an OOME"? >> > > >> > > It's true that the intermediate String >> concatenations haven't >> > occurred >> > > at that point, but in the JDK's current >> implementation, that's true, >> > > too: the concatenations that have occurred at that >> point are >> > > StringBuilder ones, not String ones. If any of the >> append operations >> > > throws an OOME, no Strings have been created at all, >> either in >> > Louis's >> > > implementation or in the JDK's. >> > > >> > > Ultimately, isn't this a quality of implementation >> issue? And if so, >> > > isn't it a quality of implementation issue that >> doesn't provide any >> > > additional quality? I can't imagine code whose >> semantics relies on >> > > this, and if they do, they are relying on something >> > > implementation-dependent. >> > > >> > > Jeremy >> > > >> > > On Wed, Mar 11, 2015 at 6:13 PM, Alex Buckley >> > > >> > >> > > > __com >> > > >>> wrote: >> > > >> > > On 3/11/2015 2:01 PM, Louis Wasserman wrote: >> > > >> > > So for example, "foo" + myInt + myString + >> "bar" + myObj >> > would be >> > > compiled to the equivalent of >> > > >> > > int myIntTmp = myInt; >> > > String myStringTmp = >> String.valueOf(myString); // defend >> > against >> > > null >> > > String myObjTmp = >> > String.valueOf(String.valueOf(____myObj)); // defend >> > > against evil toString implementations >> returning null >> > > >> > > return new StringBuilder( >> > > 17 // length of "foo" (3) + max >> length of myInt (11) + >> > > length of >> > > "bar" (3) >> > > + myStringTmp.length() >> > > + myObjTmp.length()) >> > > .append("foo") >> > > .append(myIntTmp) >> > > .append(myStringTmp) >> > > .append("bar") >> > > .append(myObjTmp) >> > > .toString(); >> > > >> > > As far as language constraints go, the JLS >> is (apparently >> > > deliberately) >> > > vague about how string concatenation is >> implemented. "An >> > > implementation >> > > may choose to perform conversion and >> concatenation in one >> > step >> > > to avoid >> > > creating and then discarding an intermediate >> String >> > object. To >> > > increase >> > > the performance of repeated string >> concatenation, a Java >> > > compiler may >> > > use the StringBuffer class or a similar >> technique to >> > reduce the >> > > number >> > > of intermediate String objects that are >> created by >> > evaluation of an >> > > expression." We see no reason this approach >> would not >> > qualify as a >> > > "similar technique." >> > > >> > > >> > > The really key property of the string >> concatenation operator is >> > > left-associativity. Later subexpressions must not be >> > evaluated until >> > > earlier subexpressions have been successfully >> evaluated AND >> > > concatenated. Consider this expression: >> > > >> > > "foo" + m() + n() >> > > >> > > which JLS8 15.8 specifies to mean: >> > > >> > > ("foo" + m()) + n() >> > > >> > > We know from JLS8 15.6 that if m() throws, then >> foo+m() >> > throws, and >> > > n() will never be evaluated. >> > > >> > > Happily, your translation doesn't appear to >> catch and swallow >> > > exceptions when eagerly evaluating each >> subexpression in >> > turn, so I >> > > believe you won't evaluate n() if m() already threw. >> > > >> > > Unhappily, a call to append(..) can in general >> fail with >> > > OutOfMemoryError. (I'm not talking about >> asynchronous >> > exceptions in >> > > general, but rather the sense that append(..) >> manipulates the >> > heap >> > > so an OOME is at least plausible.) In the OpenJDK >> > implementation, if >> > > blah.append(m()) fails with OOME, then n() >> hasn't been >> > evaluated yet >> > > -- that's "real" left-associativity. In the proposed >> > implementation, >> > > it's possible that more memory is available when >> evaluating >> > m() and >> > > n() upfront than at the time of an append call, >> so n() is >> > evaluated >> > > even if append(<>) fails -- >> that's not >> > > left-associative. >> > > >> > > Perhaps you can set my mind at ease that >> append(..) can't >> > fail with >> > > OOME? >> > > >> > > Alex >> > > >> > > >> > >> > -------------- next part -------------- An HTML attachment was scrubbed... URL: From lowasser at google.com Tue Apr 7 21:42:02 2015 From: lowasser at google.com (Louis Wasserman) Date: Tue, 07 Apr 2015 21:42:02 +0000 Subject: String concatenation tweaks In-Reply-To: <55244DFE.2050200@gmail.com> References: <5500E84F.3080800@oracle.com> <55022114.8040904@oracle.com> <55034189.4000900@oracle.com> <55244704.8080709@gmail.com> <55244DFE.2050200@gmail.com> Message-ID: In some ways it might be more sensible to provide those as optimized logarithms rather than making them part of String? The current private stringSize implementation is calibrated for short inputs, though, as opposed to a fast general logarithm -- compare Guava's IntMath.log10(int, RoundingMode) at http://docs.guava-libraries.googlecode.com/git-history/release/javadoc/com/google/common/math/IntMath.html#log10(int, java.math.RoundingMode) ). In any event, the vanilla StringBuilder proposal above represents a respectable win. If a FixedStringBuilder type or something that avoided the copy entirely were under consideration, I'd very much like to be involved in the API design, but I'd tend to prefer moving this change forward first as an incremental improvement? On Tue, Apr 7, 2015 at 2:37 PM Peter Levart wrote: > > > On 04/07/2015 11:10 PM, Louis Wasserman wrote: > > Could that actually be provided immutability-safely? I suppose an > append-only, fixed-length builder would be potentially safe. > > > I think the presented implementation *is* thread and immutability safe. It > only allows mutations from the constructing thread and disables them as > soon as toString() is called. > > > > Part of the trickiness there is with primitive parameters, where presizing > and doing the actual append both require calculating the size of the > primitive when converted to a string. The current approach just uses an > upper bound for the primitive type as a whole, which is fine and allocates > a small constant excess in most cases, but I'm not sure how we could avoid > duplicating that computation in a public API. > > > I've been thinking of static methods parallel to: > > String.valueOf(int) - String.lengthOf(int) > String.valueOf(long) - String.lengthOf(long) > ... > ... > > > The integral variants could be quick to compute (the implementations are > already there - they are just not public), the floating point maybe not so. > But the floating point conversion is already taking much more overhead and > estimating an upper bound would be fine in case floating point is involved. > If one wants to optimize the case of very long concatenation of exactly > sized Strings, integrals and other Objects with just one or few floating > point values, then he can always do it explicitly: > > "....long string...." + String.valueOf(Math.PI); > > Regards, Peter > > > > > On Tue, Apr 7, 2015 at 2:07 PM Peter Levart > wrote: > >> Hi Louis, >> >> This is nice. Amd could be even nicer. In case the estimated initial >> StringBuilder capacity is exactly the final String length, then >> constructing the String could skip the char[] copying step as the >> StringBuilder instance does not escape. But in order to be safe, this would >> have to be a special kind of StringBuilder. Like the following: >> >> >> http://cr.openjdk.java.net/~plevart/misc/ThreadLocalStringBuilder/webrev.01/ >> >> Such class would be useful for direct API use too. >> >> >> Regards, Peter >> >> >> On 03/13/2015 10:40 PM, Louis Wasserman wrote: >> >> Got it. I think the only cases we have to worry about, then, are buffer >> size overflows resulting in NegativeArraySizeException, or possibly an >> explicitly thrown OutOfMemoryError (which is StringBuilder's response when >> the buffer size tries to exceed Integer.MAX_VALUE). I think we might >> conceivably deal with this by rewriting the bytecode to -- I think we can >> improve on this with jump hackery to avoid repetition, but essentially -- >> >> int length = 3; // sum of the constant strings; we can check that this >> won't overflow at compile time but I think it couldn't no matter what >> because of method length constraints >> String mStr = m().toString(); >> length += mStr.length(); >> if (length < 0) { >> throw new OutOfMemoryError(); >> } >> String nStr = n().toString(); >> length += nStr.length(); >> if (length < 0) { >> throw new OutOfMemoryError(); >> } >> >> This continues to expand the bytecode, but probably manageably -- we >> don't actually need a local for length; if (int < 0) is easy in bytecode, >> and we can have only one OOME site that all the ifs jump to? >> >> On Fri, Mar 13, 2015 at 12:59 PM Alex Buckley >> wrote: >> >>> I do recognize that the proposed implementation doesn't reorder the >>> evaluation of subexpressions. >>> >>> When discussing the proposed implementation of '+' -- whose key element >>> is calling append(String) with a pre-computed value -- I was careful to >>> set aside asynchronous OOMEs, but I see that even synchronous OOMEs are >>> sidetracking us. My concern is not heap pressure; my concern is >>> arbitrary unchecked exceptions arising from the (int) and >>> append(String) calls. >>> >>> For sake of argument, I'll simplify "unchecked exceptions" to just >>> RuntimeExceptions, not Errors. If you can guarantee that no >>> RuntimeExceptions are thrown synchronously during the execution of those >>> method bodies on the JVM, then '+' cannot fail and the timing of >>> subexpression evaluation is unobservable (ordering is still observable, >>> as required). I think this guarantee is just a matter of reviewing the >>> method bodies. >>> >>> Alex >>> >>> On 3/12/2015 6:01 PM, Louis Wasserman wrote: >>> > I confess I'm not sure how that "quality" goal would be achievable in >>> > bytecode without deliberately allocating arrays we then discard. >>> > >>> > For what it's worth, delaying or avoiding OOMEs seems a desirable goal >>> > in general, and up to a constant multiplicative factor, this >>> > implementation seems to allocate the same amount in the same order. >>> > That is, we're still computing m().toString() before n().toString(), >>> and >>> > up to a constant multiplicative factor, m().toString() allocates the >>> > same number of bytes as the StringBuilder the status quo generates. So >>> > if m() does something like allocate a char[Integer.MAX_VALUE], we still >>> > OOM at the appropriate time. >>> > >>> > Other notes: this implementation would tend to decrease maximum >>> > allocation, so it'd reduce OOMEs. Also, since the StringBuilder will >>> > never need further expansion and we're only using the String and >>> > primitive overloads of append, the only way for append to OOME would be >>> > in append(float) and append(double), which allocate a FloatingDecimal >>> > (which may, in turn, allocate a new thread-local char[26] if one isn't >>> > already there). >>> > >>> > On Thu, Mar 12, 2015 at 4:28 PM Alex Buckley >> > > wrote: >>> > >>> > More abstract presentation. Given the expression: >>> > >>> > "foo" + m() + n() >>> > >>> > you must not evaluate n() if evaluation of "foo" + m() completes >>> > abruptly. The proposed implementation evaluates n() regardless. >>> > >>> > All is not lost. In the proposed implementation, the abrupt >>> completion >>> > of "foo" + m() could occur because an append call fails or (thanks >>> to >>> > Jon for pointing this out) the StringBuilder ctor fails. The >>> > quality-of-implementation issue is thus: if the proposed >>> implementation >>> > is of sufficiently high quality to guarantee that the ctor and the >>> first >>> > append both succeed, then the evaluation of "foo" + m() will always >>> > complete normally, and it would be an unobservable (thus >>> acceptable) >>> > implementation detail to evaluate n() early. >>> > >>> > Alex >>> > >>> > On 3/11/2015 10:26 PM, Jeremy Manson wrote: >>> > > Isn't Louis's proposed behavior equivalent to saying "the >>> rightmost >>> > > concatenation threw an OOME" instead of "some concatenation in >>> the >>> > > middle threw an OOME"? >>> > > >>> > > It's true that the intermediate String concatenations haven't >>> > occurred >>> > > at that point, but in the JDK's current implementation, that's >>> true, >>> > > too: the concatenations that have occurred at that point are >>> > > StringBuilder ones, not String ones. If any of the append >>> operations >>> > > throws an OOME, no Strings have been created at all, either in >>> > Louis's >>> > > implementation or in the JDK's. >>> > > >>> > > Ultimately, isn't this a quality of implementation issue? And >>> if so, >>> > > isn't it a quality of implementation issue that doesn't provide >>> any >>> > > additional quality? I can't imagine code whose semantics >>> relies on >>> > > this, and if they do, they are relying on something >>> > > implementation-dependent. >>> > > >>> > > Jeremy >>> > > >>> > > On Wed, Mar 11, 2015 at 6:13 PM, Alex Buckley >>> > >>> > > >> > >> wrote: >>> > > >>> > > On 3/11/2015 2:01 PM, Louis Wasserman wrote: >>> > > >>> > > So for example, "foo" + myInt + myString + "bar" + myObj >>> > would be >>> > > compiled to the equivalent of >>> > > >>> > > int myIntTmp = myInt; >>> > > String myStringTmp = String.valueOf(myString); // defend >>> > against >>> > > null >>> > > String myObjTmp = >>> > String.valueOf(String.valueOf(____myObj)); // defend >>> > > against evil toString implementations returning null >>> > > >>> > > return new StringBuilder( >>> > > 17 // length of "foo" (3) + max length of myInt >>> (11) + >>> > > length of >>> > > "bar" (3) >>> > > + myStringTmp.length() >>> > > + myObjTmp.length()) >>> > > .append("foo") >>> > > .append(myIntTmp) >>> > > .append(myStringTmp) >>> > > .append("bar") >>> > > .append(myObjTmp) >>> > > .toString(); >>> > > >>> > > As far as language constraints go, the JLS is >>> (apparently >>> > > deliberately) >>> > > vague about how string concatenation is implemented. >>> "An >>> > > implementation >>> > > may choose to perform conversion and concatenation in >>> one >>> > step >>> > > to avoid >>> > > creating and then discarding an intermediate String >>> > object. To >>> > > increase >>> > > the performance of repeated string concatenation, a Java >>> > > compiler may >>> > > use the StringBuffer class or a similar technique to >>> > reduce the >>> > > number >>> > > of intermediate String objects that are created by >>> > evaluation of an >>> > > expression." We see no reason this approach would not >>> > qualify as a >>> > > "similar technique." >>> > > >>> > > >>> > > The really key property of the string concatenation >>> operator is >>> > > left-associativity. Later subexpressions must not be >>> > evaluated until >>> > > earlier subexpressions have been successfully evaluated AND >>> > > concatenated. Consider this expression: >>> > > >>> > > "foo" + m() + n() >>> > > >>> > > which JLS8 15.8 specifies to mean: >>> > > >>> > > ("foo" + m()) + n() >>> > > >>> > > We know from JLS8 15.6 that if m() throws, then foo+m() >>> > throws, and >>> > > n() will never be evaluated. >>> > > >>> > > Happily, your translation doesn't appear to catch and >>> swallow >>> > > exceptions when eagerly evaluating each subexpression in >>> > turn, so I >>> > > believe you won't evaluate n() if m() already threw. >>> > > >>> > > Unhappily, a call to append(..) can in general fail with >>> > > OutOfMemoryError. (I'm not talking about asynchronous >>> > exceptions in >>> > > general, but rather the sense that append(..) manipulates >>> the >>> > heap >>> > > so an OOME is at least plausible.) In the OpenJDK >>> > implementation, if >>> > > blah.append(m()) fails with OOME, then n() hasn't been >>> > evaluated yet >>> > > -- that's "real" left-associativity. In the proposed >>> > implementation, >>> > > it's possible that more memory is available when evaluating >>> > m() and >>> > > n() upfront than at the time of an append call, so n() is >>> > evaluated >>> > > even if append(<>) fails -- that's not >>> > > left-associative. >>> > > >>> > > Perhaps you can set my mind at ease that append(..) can't >>> > fail with >>> > > OOME? >>> > > >>> > > Alex >>> > > >>> > > >>> > >>> >> >> > -------------- next part -------------- An HTML attachment was scrubbed... URL: From joel.franck at oracle.com Wed Apr 8 18:43:31 2015 From: joel.franck at oracle.com (=?utf-8?Q?Joel_Borggr=C3=A9n-Franck?=) Date: Wed, 8 Apr 2015 20:43:31 +0200 Subject: [PATCH] fix self-assignment in TypeVar In-Reply-To: References: Message-ID: <7F5DDC8E-A1BA-46AD-9192-56790C6785CC@oracle.com> Hi Liam, I noticed this last year and changed it in the anno-pipeline/dev forest that has now been merged. Thanks for bringing this up though. cheers /Joel > On 27 mar 2015, at 00:05, Liam Miller-Cushon wrote: > > I think this is harmless, but it looks suspicious and is probably > worth cleaning up. > > (If it's actually a bug then this is not the right patch.) > > # HG changeset patch > # User Liam Miller-Cushon > # Date 1427408687 25200 > # Thu Mar 26 15:24:47 2015 -0700 > # Node ID 99c3cd409d0a12538c6f7e96c45e87b81d9db8bf > # Parent 1a0808932668a1fa688d3c0ed1f794adc9b3ce18 > Remove self-assignment in TypeVar > > diff -r 1a0808932668 -r 99c3cd409d0a > src/jdk.compiler/share/classes/com/sun/tools/javac/code/Type.java > --- a/src/jdk.compiler/share/classes/com/sun/tools/javac/code/Type.java > Thu Mar 26 16:17:36 2015 +0100 > +++ b/src/jdk.compiler/share/classes/com/sun/tools/javac/code/Type.java > Thu Mar 26 15:24:47 2015 -0700 > @@ -1466,7 +1466,6 @@ > public TypeVar(Name name, Symbol owner, Type lower) { > super(null, TypeMetadata.empty); > tsym = new TypeVariableSymbol(0, name, this, owner); > - this.bound = bound; > this.lower = lower; > } From cushon at google.com Wed Apr 8 20:29:28 2015 From: cushon at google.com (Liam Miller-Cushon) Date: Wed, 8 Apr 2015 13:29:28 -0700 Subject: [PATCH] fix self-assignment in TypeVar In-Reply-To: <7F5DDC8E-A1BA-46AD-9192-56790C6785CC@oracle.com> References: <7F5DDC8E-A1BA-46AD-9192-56790C6785CC@oracle.com> Message-ID: Great, thanks Joel. On Wed, Apr 8, 2015 at 11:43 AM, Joel Borggr?n-Franck < joel.franck at oracle.com> wrote: > Hi Liam, > > I noticed this last year and changed it in the anno-pipeline/dev forest > that has now been merged. Thanks for bringing this up though. > > cheers > /Joel > > > On 27 mar 2015, at 00:05, Liam Miller-Cushon wrote: > > > > I think this is harmless, but it looks suspicious and is probably > > worth cleaning up. > > > > (If it's actually a bug then this is not the right patch.) > > > > # HG changeset patch > > # User Liam Miller-Cushon > > # Date 1427408687 25200 > > # Thu Mar 26 15:24:47 2015 -0700 > > # Node ID 99c3cd409d0a12538c6f7e96c45e87b81d9db8bf > > # Parent 1a0808932668a1fa688d3c0ed1f794adc9b3ce18 > > Remove self-assignment in TypeVar > > > > diff -r 1a0808932668 -r 99c3cd409d0a > > src/jdk.compiler/share/classes/com/sun/tools/javac/code/Type.java > > --- a/src/jdk.compiler/share/classes/com/sun/tools/javac/code/Type.java > > Thu Mar 26 16:17:36 2015 +0100 > > +++ b/src/jdk.compiler/share/classes/com/sun/tools/javac/code/Type.java > > Thu Mar 26 15:24:47 2015 -0700 > > @@ -1466,7 +1466,6 @@ > > public TypeVar(Name name, Symbol owner, Type lower) { > > super(null, TypeMetadata.empty); > > tsym = new TypeVariableSymbol(0, name, this, owner); > > - this.bound = bound; > > this.lower = lower; > > } > > -------------- next part -------------- An HTML attachment was scrubbed... URL: From jonathan.gibbons at oracle.com Fri Apr 10 23:19:27 2015 From: jonathan.gibbons at oracle.com (Jonathan Gibbons) Date: Fri, 10 Apr 2015 16:19:27 -0700 Subject: Re; hg: jdk9/dev/langtools: 8031744: Annotations on many Language Model elements are not returned Message-ID: <55285A7F.4000904@oracle.com> Earlier this week, I pushed a changeset into JDK 9 for work that was developed as part of the anno-pipeline project. See http://mail.openjdk.java.net/pipermail/jdk9-dev-changes/2015-April/003124.html Changeset: 62e285806e83 Author: jjg Date: 2015-04-07 11:04 -0700 URL:http://hg.openjdk.java.net/jdk9/dev/langtools/rev/62e285806e83 8031744: Annotations on many Language Model elements are not returned Reviewed-by: jfranck, mcimadamore, emc, jlahoda, jjg Contributed-by:joel.franck at oracle.com ,maurizio.cimadamore at oracle.com I regret that I omitted to mention Werner Dietl (wmdietl) for his part in testing this work in conjunction with the Checker Framework. His work is duly noted here. Thank you Werner, -- Jon -------------- next part -------------- An HTML attachment was scrubbed... URL: From indigowarrior9 at yahoo.com Tue Apr 14 06:40:15 2015 From: indigowarrior9 at yahoo.com (A B) Date: Tue, 14 Apr 2015 07:40:15 +0100 Subject: Questions about OpenJDK mailing list / Personal Introduction Message-ID: <1428993615.51395.YahooMailBasic@web172901.mail.ir2.yahoo.com> Good morning Jonathan, Member of the OpenJDK Developer group. 1. Today I joined the OpenJDK Compiler-dev mailing list and want to make sure: - I behave well on that list, - That I blend in with the rest of the crowd, - do not start to ask the wrong questions to the wrong people here, - do not want to waste peoples valuable time, by asking stuff that I can find and learn myself. That is why I sent you this email first, so I can fully understand the true nature of this mailing list. ================================================================================================== 2. INTRODUCTION: (Allow me to introduce myself briefly) Ronald, 50 years young IT veteran, started my journey 35 years behind a the keyboard of a: - Tandy TRS-80 of my friend's father (http://www.old-computers.com/museum/computer.asp?c=409) - Later I bought my own Texas TI99/4a homecomputer (http://www.old-computers.com/museum/computer.asp?c=409)? After that I switch to PC computing (http://www.old-computers.com/museum/computer.asp?st=1&c=1044) and continued walking that PC-path ever since. 3. I am an auto-didactic/ self-learner, so I learn everything myself (given good documentation and crystal-clear-and-simple-examples). I graduated as a "Industrial Automation" Engineer in the Netherland (kind of a "Bachelor of Computer Science" in the Netherlands) I have 18 years of "business experience" in a wide variety of IT firms and almost all IT-positions, a jack-of-all-trade so to speak. I have programmed in many language over the years (professionally and hobby-wise): DOS: - Texas Instrument Basic, - Turbo Pascal, C, - Microsoft C, - Dbase III, IV, - Clipper, - Assembler (Motorola, intel), - Prolog (a little bit in highschool) - C++,(a little bit in highschool) - Java (started in its very beginning, back in 1996, but never followed-through with it, in that time) WINDOWS: - Visual Basic 6 - Visual C In short, I do understand computer-programming and have developer a programmers-mindset over all the years :-) ================================================================================================== 3. CORE OF MY QUESTION: Since a few weeks, I have picked "my Java Trail" again and am currently up studying the Java Language again. On that trail, i encounters some javac generated compiler-errors and went looking for its documentation, error-meaning, and how to solve them. Makes sense right? Learning by doing, trail and error. This quickly expanded my thirst of knowledge into "wanting-to-know-how-a-java-compiler-does-its-thing" under the hood. I landed on the OpenJDK page, and started studying that material along with many internet compiler-builder courses, _ http://www.diku.dk/~torbenm/Basics/ _ http://www.slideshare.net/leopk01/compiler-chapter-1 _ https://www.cs.princeton.edu/~appel/modern/basic/java/ _ http://www.slideshare.net/GonzaloSantiago/design-and-implementation-of-compiler _ http://www.slideshare.net/chammua/compilers-principles-techniques-and-tools-2nd-edition-34300992 since I want to be fully 100% be able the inner-workings of the OpenJDK Java compiler (like you girls and guys do) - http://openjdk.java.net/groups/compiler/ - http://openjdk.java.net/groups/compiler/doc/hhgtjavac/index.html - http://openjdk.java.net/groups/compiler/doc/package-overview/index.html ================================================================================================== 4. MY CHALLENGE: Currently i'm STUCK at this page:, Phase 2 - the ENTER phase of JAVAC: - http://openjdk.java.net/groups/compiler/doc/compilation-overview/index.html That is the very reason why I have now joined your mailing list, to be able to communicate with the creators of the OpenJDK compiler directly. since I could not find any better information source online on this topic. Just like my grandpa though me, "Always go back to the source of your challenges" ================================================================================================== QUESTIONS: - Is this allowed through the usage of this mailing list to ask question about it? - Is this the right place to ask question on the inner workings of the OpenJDK compiler? Thanks everybody for their reaction, hints, tips, contributions and shared knowledge. Regards, Ronald Vermeij From jonathan.gibbons at oracle.com Tue Apr 14 22:22:02 2015 From: jonathan.gibbons at oracle.com (Jonathan Gibbons) Date: Tue, 14 Apr 2015 15:22:02 -0700 Subject: Questions about OpenJDK mailing list / Personal Introduction In-Reply-To: <1428993615.51395.YahooMailBasic@web172901.mail.ir2.yahoo.com> References: <1428993615.51395.YahooMailBasic@web172901.mail.ir2.yahoo.com> Message-ID: <552D930A.9030008@oracle.com> Ronald, Thank you for your courteous and considerate introduction. This is certainly the place where the javac team and friends hang out, and you may be able to get answers to your questions. But if you're looking to come up to speed on Java, I'm not sure I would recommend starting with learning how the compiler works! -- Jon On 04/13/2015 11:40 PM, A B wrote: > Good morning Jonathan, Member of the OpenJDK Developer group. > > 1. Today I joined the OpenJDK Compiler-dev mailing list and want to make sure: > - I behave well on that list, > - That I blend in with the rest of the crowd, > - do not start to ask the wrong questions to the wrong people here, > - do not want to waste peoples valuable time, by asking stuff that I can find and learn myself. > That is why I sent you this email first, so I can fully understand the true nature of this mailing list. > ================================================================================================== > > 2. INTRODUCTION: (Allow me to introduce myself briefly) > Ronald, 50 years young IT veteran, started my journey 35 years behind a the keyboard of a: > - Tandy TRS-80 of my friend's father (http://www.old-computers.com/museum/computer.asp?c=409) > - Later I bought my own Texas TI99/4a homecomputer (http://www.old-computers.com/museum/computer.asp?c=409) > After that I switch to PC computing (http://www.old-computers.com/museum/computer.asp?st=1&c=1044) > and continued walking that PC-path ever since. > > 3. I am an auto-didactic/ self-learner, so I learn everything myself (given good documentation and crystal-clear-and-simple-examples). > I graduated as a "Industrial Automation" Engineer in the Netherland (kind of a "Bachelor of Computer Science" in the Netherlands) > I have 18 years of "business experience" in a wide variety of IT firms and almost all IT-positions, a jack-of-all-trade so to speak. > I have programmed in many language over the years (professionally and hobby-wise): > DOS: > - Texas Instrument Basic, > - Turbo Pascal, C, > - Microsoft C, > - Dbase III, IV, > - Clipper, > - Assembler (Motorola, intel), > - Prolog (a little bit in highschool) > - C++,(a little bit in highschool) > - Java (started in its very beginning, back in 1996, but never followed-through with it, in that time) > > WINDOWS: > - Visual Basic 6 > - Visual C > > In short, I do understand computer-programming and have developer a programmers-mindset over all the years :-) > > ================================================================================================== > 3. CORE OF MY QUESTION: > Since a few weeks, I have picked "my Java Trail" again and am currently up studying the Java Language again. On that trail, i encounters some > javac generated compiler-errors and went looking for its documentation, error-meaning, and how to solve them. > Makes sense right? Learning by doing, trail and error. > > This quickly expanded my thirst of knowledge into "wanting-to-know-how-a-java-compiler-does-its-thing" under the hood. > I landed on the OpenJDK page, and started studying that material along with many internet compiler-builder courses, > _ http://www.diku.dk/~torbenm/Basics/ > _ http://www.slideshare.net/leopk01/compiler-chapter-1 > _ https://www.cs.princeton.edu/~appel/modern/basic/java/ > _ http://www.slideshare.net/GonzaloSantiago/design-and-implementation-of-compiler > _ http://www.slideshare.net/chammua/compilers-principles-techniques-and-tools-2nd-edition-34300992 > > since I want to be fully 100% be able the inner-workings of the OpenJDK Java compiler (like you girls and guys do) > - http://openjdk.java.net/groups/compiler/ > - http://openjdk.java.net/groups/compiler/doc/hhgtjavac/index.html > - http://openjdk.java.net/groups/compiler/doc/package-overview/index.html > > ================================================================================================== > 4. MY CHALLENGE: > Currently i'm STUCK at this page:, Phase 2 - the ENTER phase of JAVAC: > - http://openjdk.java.net/groups/compiler/doc/compilation-overview/index.html > > That is the very reason why I have now joined your mailing list, to be able to communicate with the creators of the OpenJDK compiler directly. > since I could not find any better information source online on this topic. Just like my grandpa though me, "Always go back to the source of your challenges" > > ================================================================================================== > QUESTIONS: > - Is this allowed through the usage of this mailing list to ask question about it? > - Is this the right place to ask question on the inner workings of the OpenJDK compiler? > > Thanks everybody for their reaction, hints, tips, contributions and shared knowledge. > > Regards, > > Ronald Vermeij > From daniel.smith at oracle.com Wed Apr 15 17:02:04 2015 From: daniel.smith at oracle.com (Dan Smith) Date: Wed, 15 Apr 2015 11:02:04 -0600 Subject: Call for Speakers -- 2015 JVM Language Summit Message-ID: CALL FOR SPEAKERS -- JVM LANGUAGE SUMMIT, AUGUST 2015 We are pleased to announce the 2015 JVM Language Summit to be held at Oracle's Santa Clara campus on August 10-12, 2015. Registration is now open for speaker submissions (presentations and workshops) and will remain open until May 22, 2015. There is no registration fee for speakers. The JVM Language Summit is an open technical collaboration among language designers, compiler writers, tool builders, runtime engineers, and VM architects. We will share our experiences as creators of both the JVM and programming languages for the JVM. We also welcome non-JVM developers of similar technologies to attend or speak on their runtime, VM, or language of choice. Presentations will be recorded and made available to the public via the Oracle Technology Network. This event is being organized by language and JVM engineers; no marketers involved! So bring your slide rules and be prepared for some seriously geeky discussions. Format The summit is held in a single classroom-style room to support direct communication between participants. About 80-100 attendees are expected. As in previous years, we will divide the schedule between traditional presentations and "workshops." Workshops are informal, facilitated discussion groups among smaller, self-selected participants, and should enable deeper "dives" into the subject matter. If there is interest, there will also be impromptu "lightning talks." Traditional presentations (about 7 each day) will be given in a single track, while workshops (2?3 each day) will occur in parallel. Instructions for Speaker Registration If you'd like give a presentation or lead a workshop, please register as a Speaker and include a detailed abstract. There is no fee. You will be notified about whether your proposal has been accepted; if not, you will be able to register as a regular attendee. For a successful presentation or workshop submission, please note the following: - All talks should be deeply technical, given by designers and implementors to designers and implementors. We all speak Code here! - Each talk, we hope and expect, will inform the audience, in detail, about the state of the art of language design and implementation on the JVM, or will explore the present and future capabilities of the JVM itself. (Some will do so indirectly by discussing non-JVM technologies.) - Know your audience: attendees may not be likely to ever use your specific language or tool, but could learn something from your interactions with the JVM. A broad goal of the summit is to inspire us to work together on JVM-based technologies that enable a rich ecosystem at higher layers. We encourage speakers to submit both a presentation and a workshop; we will arrange to schedule the presentation before the workshop, so that the presentation can spark people's interest and the workshop will allow those who are really interested to go deeper into the subject area. Workshop facilitators may, but are not expected to, prepare presentation materials; in any case, they should come prepared to guide a deep technical discussion. To register: regonline.com/jvmls2015 For further information: jvmlangsummit.com Questions: inquire at jvmlangsummit.com We hope to see you in August! From martijnverburg at gmail.com Thu Apr 16 11:35:31 2015 From: martijnverburg at gmail.com (Martijn Verburg) Date: Thu, 16 Apr 2015 13:35:31 +0200 Subject: Javac compiler error discovered via project Kulla REPL? Message-ID: Hi all, >From the REPL hackday we had recently: "Following error message might be confusing, as the symbol might not be a variable: -> ClassName x = ClassName.create() | cannot find symbol | symbol: variable ClassName | ClassName x = ClassName.create(); | ^-------^ " Kulla team suggested this is a Java compiler (javac) error message and it should be submitted as a bug report against javac. Cheers, Martijn -------------- next part -------------- An HTML attachment was scrubbed... URL: From jan.lahoda at oracle.com Thu Apr 16 15:07:57 2015 From: jan.lahoda at oracle.com (Jan Lahoda) Date: Thu, 16 Apr 2015 17:07:57 +0200 Subject: Javac compiler error discovered via project Kulla REPL? In-Reply-To: References: Message-ID: <552FD04D.7070306@oracle.com> Hello Martijn, I've filled: https://bugs.openjdk.java.net/browse/JDK-8077970 Thanks, Jan On 16.4.2015 13:35, Martijn Verburg wrote: > Hi all, > > From the REPL hackday we had recently: > > "Following error message might be confusing, as the symbol might not be a > variable: > -> ClassName x = ClassName.create() > | cannot find symbol > | symbol: variable ClassName > | ClassName x = ClassName.create(); > | ^-------^ > " > > Kulla team suggested this is a Java compiler (javac) error message and it > should be submitted as a bug report against javac. > > Cheers, > Martijn > From alex.buckley at oracle.com Thu Apr 16 18:40:46 2015 From: alex.buckley at oracle.com (Alex Buckley) Date: Thu, 16 Apr 2015 11:40:46 -0700 Subject: Javac compiler error discovered via project Kulla REPL? In-Reply-To: <552FD04D.7070306@oracle.com> References: <552FD04D.7070306@oracle.com> Message-ID: <5530022E.9050609@oracle.com> The situation seems to be that the REPL gives no error for the identifier "ClassName" on the LHS of the assignment. Therefore, it must have been possible to classify the identifier as a simple name, in particular as a TypeName. Then, the use of the TypeName on the RHS of the assignment cannot be an error. I can imagine an alternative situation: the identifier "ClassName" is totally made up, and the REPL just happens to be reporting only one of the two "cannot find symbol" errors indicated by JDK-8077970. Which situation is it? Alex On 4/16/2015 8:07 AM, Jan Lahoda wrote: > Hello Martijn, > > I've filled: > https://bugs.openjdk.java.net/browse/JDK-8077970 > > Thanks, > Jan > > On 16.4.2015 13:35, Martijn Verburg wrote: >> Hi all, >> >> From the REPL hackday we had recently: >> >> "Following error message might be confusing, as the symbol might not be a >> variable: >> -> ClassName x = ClassName.create() >> | cannot find symbol >> | symbol: variable ClassName >> | ClassName x = ClassName.create(); >> | ^-------^ >> " >> >> Kulla team suggested this is a Java compiler (javac) error message and it >> should be submitted as a bug report against javac. >> >> Cheers, >> Martijn >> From robert.field at oracle.com Thu Apr 16 19:57:47 2015 From: robert.field at oracle.com (Robert Field) Date: Thu, 16 Apr 2015 12:57:47 -0700 Subject: Javac compiler error discovered via project Kulla REPL? In-Reply-To: <5530022E.9050609@oracle.com> References: <552FD04D.7070306@oracle.com> <5530022E.9050609@oracle.com> Message-ID: <5530143B.1050305@oracle.com> An HTML attachment was scrubbed... URL: From alex.buckley at oracle.com Thu Apr 16 20:43:17 2015 From: alex.buckley at oracle.com (Alex Buckley) Date: Thu, 16 Apr 2015 13:43:17 -0700 Subject: Javac compiler error discovered via project Kulla REPL? In-Reply-To: <5530143B.1050305@oracle.com> References: <552FD04D.7070306@oracle.com> <5530022E.9050609@oracle.com> <5530143B.1050305@oracle.com> Message-ID: <55301EE5.60802@oracle.com> OK, so in the method invocation expression (whether standalone or on the RHS of an assignment), javac should classify the identifier "Undefined" as a PackageName -- see JLS 6.5.2. Since no package with that name exists, an error is due - see JLS 6.5.3.1. It would be helpful for the "cannot find symbol" message to indicate that neither a variable nor a type nor a package called "Undefined" is in scope. I expect that when javac's second error message improves, so will the REPL's third error message. Alex On 4/16/2015 12:57 PM, Robert Field wrote: > Actually, none of the below. > > They both give the same errors: both a variable ClassName and a class > ClassName error. Which makes it even more potentially confusing. > > From Jan's bug report (showing the behavior of javac (no REPL)) (I've > added bolding) -- > > Consider this code: > --- > public class Test { > private void test() { > Undefined x = Undefined.create(); > } > } > --- > > When compiled, this code produces the following errors: > --- > Test.java:3: error: cannot find symbol > Undefined x = Undefined.create(); > ^ > symbol: *class Undefined* > location: class Test > Test.java:3: error: cannot find symbol > Undefined x = Undefined.create(); > ^ > symbol: *variable Undefined* > location: class Test > 2 errors > --- > > The REPL does -- > > -> Undefined x = Undefined.create() > | Error: > | cannot find symbol > | symbol: class Undefined > | Undefined x = Undefined.create(); > | ^-------^ > | Error: > | cannot find symbol > | symbol: class Undefined > | Undefined x = Undefined.create(); > | ^-------^ > | Error: > | cannot find symbol > | symbol: variable Undefined > | Undefined x = Undefined.create(); > | ^-------^ > > > -> > > The only differences are how the position is marked and the REPL gives > the first one twice (because it has been taken apart and put back > together with three occurrences of the name). > > Given just "Undefined.create()" both javac and REPL give just a > "variable Undefined" error > > -Robert > > > On 04/16/15 11:40, Alex Buckley wrote: >> The situation seems to be that the REPL gives no error for the >> identifier "ClassName" on the LHS of the assignment. Therefore, it >> must have been possible to classify the identifier as a simple name, >> in particular as a TypeName. Then, the use of the TypeName on the RHS >> of the assignment cannot be an error. >> >> I can imagine an alternative situation: the identifier "ClassName" is >> totally made up, and the REPL just happens to be reporting only one of >> the two "cannot find symbol" errors indicated by JDK-8077970. >> >> Which situation is it? >> >> Alex >> >> On 4/16/2015 8:07 AM, Jan Lahoda wrote: >>> Hello Martijn, >>> >>> I've filled: >>> https://bugs.openjdk.java.net/browse/JDK-8077970 >>> >>> Thanks, >>> Jan >>> >>> On 16.4.2015 13:35, Martijn Verburg wrote: >>>> Hi all, >>>> >>>> From the REPL hackday we had recently: >>>> >>>> "Following error message might be confusing, as the symbol might not >>>> be a >>>> variable: >>>> -> ClassName x = ClassName.create() >>>> | cannot find symbol >>>> | symbol: variable ClassName >>>> | ClassName x = ClassName.create(); >>>> | ^-------^ >>>> " >>>> >>>> Kulla team suggested this is a Java compiler (javac) error message >>>> and it >>>> should be submitted as a bug report against javac. >>>> >>>> Cheers, >>>> Martijn >>>> > From robert.field at oracle.com Fri Apr 17 00:35:53 2015 From: robert.field at oracle.com (Robert Field) Date: Thu, 16 Apr 2015 17:35:53 -0700 Subject: Javac compiler error discovered via project Kulla REPL? In-Reply-To: <55301EE5.60802@oracle.com> References: <552FD04D.7070306@oracle.com> <5530022E.9050609@oracle.com> <5530143B.1050305@oracle.com> <55301EE5.60802@oracle.com> Message-ID: <14cc4cd9a58.2767.4011f3a8741ca2aabce58b8b81f42d24@oracle.com> Right. And, yes, the REPL's will too. -Robert On April 16, 2015 1:43:34 PM Alex Buckley wrote: > OK, so in the method invocation expression (whether standalone or on the > RHS of an assignment), javac should classify the identifier "Undefined" > as a PackageName -- see JLS 6.5.2. Since no package with that name > exists, an error is due - see JLS 6.5.3.1. It would be helpful for the > "cannot find symbol" message to indicate that neither a variable nor a > type nor a package called "Undefined" is in scope. > > I expect that when javac's second error message improves, so will the > REPL's third error message. > > Alex > > On 4/16/2015 12:57 PM, Robert Field wrote: > > Actually, none of the below. > > > > They both give the same errors: both a variable ClassName and a class > > ClassName error. Which makes it even more potentially confusing. > > > > From Jan's bug report (showing the behavior of javac (no REPL)) (I've > > added bolding) -- > > > > Consider this code: > > --- > > public class Test { > > private void test() { > > Undefined x = Undefined.create(); > > } > > } > > --- > > > > When compiled, this code produces the following errors: > > --- > > Test.java:3: error: cannot find symbol > > Undefined x = Undefined.create(); > > ^ > > symbol: *class Undefined* > > location: class Test > > Test.java:3: error: cannot find symbol > > Undefined x = Undefined.create(); > > ^ > > symbol: *variable Undefined* > > location: class Test > > 2 errors > > --- > > > > The REPL does -- > > > > -> Undefined x = Undefined.create() > > | Error: > > | cannot find symbol > > | symbol: class Undefined > > | Undefined x = Undefined.create(); > > | ^-------^ > > | Error: > > | cannot find symbol > > | symbol: class Undefined > > | Undefined x = Undefined.create(); > > | ^-------^ > > | Error: > > | cannot find symbol > > | symbol: variable Undefined > > | Undefined x = Undefined.create(); > > | ^-------^ > > > > > > -> > > > > The only differences are how the position is marked and the REPL gives > > the first one twice (because it has been taken apart and put back > > together with three occurrences of the name). > > > > Given just "Undefined.create()" both javac and REPL give just a > > "variable Undefined" error > > > > -Robert > > > > > > On 04/16/15 11:40, Alex Buckley wrote: > >> The situation seems to be that the REPL gives no error for the > >> identifier "ClassName" on the LHS of the assignment. Therefore, it > >> must have been possible to classify the identifier as a simple name, > >> in particular as a TypeName. Then, the use of the TypeName on the RHS > >> of the assignment cannot be an error. > >> > >> I can imagine an alternative situation: the identifier "ClassName" is > >> totally made up, and the REPL just happens to be reporting only one of > >> the two "cannot find symbol" errors indicated by JDK-8077970. > >> > >> Which situation is it? > >> > >> Alex > >> > >> On 4/16/2015 8:07 AM, Jan Lahoda wrote: > >>> Hello Martijn, > >>> > >>> I've filled: > >>> https://bugs.openjdk.java.net/browse/JDK-8077970 > >>> > >>> Thanks, > >>> Jan > >>> > >>> On 16.4.2015 13:35, Martijn Verburg wrote: > >>>> Hi all, > >>>> > >>>> From the REPL hackday we had recently: > >>>> > >>>> "Following error message might be confusing, as the symbol might not > >>>> be a > >>>> variable: > >>>> -> ClassName x = ClassName.create() > >>>> | cannot find symbol > >>>> | symbol: variable ClassName > >>>> | ClassName x = ClassName.create(); > >>>> | ^-------^ > >>>> " > >>>> > >>>> Kulla team suggested this is a Java compiler (javac) error message > >>>> and it > >>>> should be submitted as a bug report against javac. > >>>> > >>>> Cheers, > >>>> Martijn > >>>> > > From aleksey.shipilev at oracle.com Fri Apr 17 16:51:06 2015 From: aleksey.shipilev at oracle.com (Aleksey Shipilev) Date: Fri, 17 Apr 2015 19:51:06 +0300 Subject: String concatenation tweaks In-Reply-To: <55014F7C.20201@oracle.com> References: <55014F7C.20201@oracle.com> Message-ID: <553139FA.20601@oracle.com> On 03/12/2015 11:34 AM, Aleksey Shipilev wrote: > I am not really fond of doing the optimizations on javac level: asking > users to recompile their programs for better performance and/or fixing > the (probable) javac bugs is arguably against what users expect. > But in this case, changing the bytecode shape before hitting the JIT > compiler seems to be the sanest route. JIT compilers have to maintain > more strong invariants than most users let on, see e.g.: > https://bugs.openjdk.java.net/browse/JDK-8043677 All right, let me propose a radical alternative here. Asking users to recompile their Java code for performance once is probably okay, if we describe the performance boosts. But asking for the second, third, (N+1)-th time would probably make our lives harder -- people would refuse to migrate, and we will be stuck supporting multiple possible code shapes in VM. Given we have arguments about the exact specifics how to generate string concat on javac side, chances are we would not get it right from the start; and we would need to make adjustments in future. This is especially scary if we want to introduce special string builders that would have to be leaked into bytecode ABIs. If only there was a way to declare the intent in Java code, and then delay the exact specifics how that intent is fulfilled (e.g. what code is generated) until the JVM runs... wait, that's invokedynamic! So, what if we make a radical ABI change once and for all: implement all concatenations via (for example) the signature-polymorphic java.lang.StringConcat.concat(Object... methods), and let JDK to figure out how to do this exactly at runtime? Thanks, -Aleksey. -------------- next part -------------- A non-text attachment was scrubbed... Name: signature.asc Type: application/pgp-signature Size: 819 bytes Desc: OpenPGP digital signature URL: From forax at univ-mlv.fr Fri Apr 17 17:04:58 2015 From: forax at univ-mlv.fr (Remi Forax) Date: Fri, 17 Apr 2015 19:04:58 +0200 Subject: String concatenation tweaks In-Reply-To: <553139FA.20601@oracle.com> References: <55014F7C.20201@oracle.com> <553139FA.20601@oracle.com> Message-ID: <55313D3A.10008@univ-mlv.fr> On 04/17/2015 06:51 PM, Aleksey Shipilev wrote: > On 03/12/2015 11:34 AM, Aleksey Shipilev wrote: >> I am not really fond of doing the optimizations on javac level: asking >> users to recompile their programs for better performance and/or fixing >> the (probable) javac bugs is arguably against what users expect. >> But in this case, changing the bytecode shape before hitting the JIT >> compiler seems to be the sanest route. JIT compilers have to maintain >> more strong invariants than most users let on, see e.g.: >> https://bugs.openjdk.java.net/browse/JDK-8043677 > All right, let me propose a radical alternative here. Asking users to > recompile their Java code for performance once is probably okay, if we > describe the performance boosts. But asking for the second, third, > (N+1)-th time would probably make our lives harder -- people would > refuse to migrate, and we will be stuck supporting multiple possible > code shapes in VM. > > Given we have arguments about the exact specifics how to generate string > concat on javac side, chances are we would not get it right from the > start; and we would need to make adjustments in future. This is > especially scary if we want to introduce special string builders that > would have to be leaked into bytecode ABIs. > > If only there was a way to declare the intent in Java code, and then > delay the exact specifics how that intent is fulfilled (e.g. what code > is generated) until the JVM runs... wait, that's invokedynamic! > > So, what if we make a radical ABI change once and for all: implement all > concatenations via (for example) the signature-polymorphic > java.lang.StringConcat.concat(Object... methods), and let JDK to figure > out how to do this exactly at runtime? yes :) you don't need a signature polymorphic method, indy is enough, Object a = ... int b = .. a + " foo " + b can be compiled to: aload 0 // a iload 1 // b invokedynamic (Ljava/lang/Object;I)Ljava/lang/String; java.lang.invoke.StringConcat [0, "foo", 1] note that in that case, Java can also support the interpolation syntax: "$a foo $b" because the translation is exactly the same :) > > Thanks, > -Aleksey. > cheers, R?mi From lowasser at google.com Fri Apr 17 17:11:41 2015 From: lowasser at google.com (Louis Wasserman) Date: Fri, 17 Apr 2015 17:11:41 +0000 Subject: String concatenation tweaks In-Reply-To: <55313D3A.10008@univ-mlv.fr> References: <55014F7C.20201@oracle.com> <553139FA.20601@oracle.com> <55313D3A.10008@univ-mlv.fr> Message-ID: I would be thrilled to see that level of magic, though that's probably beyond my personal abilities to implement or contribute as a patch. On Fri, Apr 17, 2015 at 10:05 AM Remi Forax wrote: > > On 04/17/2015 06:51 PM, Aleksey Shipilev wrote: > > On 03/12/2015 11:34 AM, Aleksey Shipilev wrote: > >> I am not really fond of doing the optimizations on javac level: asking > >> users to recompile their programs for better performance and/or fixing > >> the (probable) javac bugs is arguably against what users expect. > >> But in this case, changing the bytecode shape before hitting the JIT > >> compiler seems to be the sanest route. JIT compilers have to maintain > >> more strong invariants than most users let on, see e.g.: > >> https://bugs.openjdk.java.net/browse/JDK-8043677 > > All right, let me propose a radical alternative here. Asking users to > > recompile their Java code for performance once is probably okay, if we > > describe the performance boosts. But asking for the second, third, > > (N+1)-th time would probably make our lives harder -- people would > > refuse to migrate, and we will be stuck supporting multiple possible > > code shapes in VM. > > > > Given we have arguments about the exact specifics how to generate string > > concat on javac side, chances are we would not get it right from the > > start; and we would need to make adjustments in future. This is > > especially scary if we want to introduce special string builders that > > would have to be leaked into bytecode ABIs. > > > > If only there was a way to declare the intent in Java code, and then > > delay the exact specifics how that intent is fulfilled (e.g. what code > > is generated) until the JVM runs... wait, that's invokedynamic! > > > > So, what if we make a radical ABI change once and for all: implement all > > concatenations via (for example) the signature-polymorphic > > java.lang.StringConcat.concat(Object... methods), and let JDK to figure > > out how to do this exactly at runtime? > > yes :) > > you don't need a signature polymorphic method, indy is enough, > > Object a = ... > int b = .. > a + " foo " + b > > can be compiled to: > aload 0 // a > iload 1 // b > invokedynamic (Ljava/lang/Object;I)Ljava/lang/String; > java.lang.invoke.StringConcat [0, "foo", 1] > > note that in that case, Java can also support the interpolation syntax: > "$a foo $b" > because the translation is exactly the same :) > > > > > Thanks, > > -Aleksey. > > > > cheers, > R?mi > > -------------- next part -------------- An HTML attachment was scrubbed... URL: From rob.mckenna at oracle.com Mon Apr 20 16:03:46 2015 From: rob.mckenna at oracle.com (Rob McKenna) Date: Mon, 20 Apr 2015 17:03:46 +0100 Subject: Regression in the javac 1.8u40 In-Reply-To: <579b-55351c80-5b-4b5ec680@148209838> References: <579b-55351c80-5b-4b5ec680@148209838> Message-ID: <55352362.5060408@oracle.com> Probably better to discuss this on compiler-dev at openjdk.java.net -Rob On 20/04/15 16:33, Aaron Digulla wrote: > Hello, > > I've found a regression in generics handling of Java 8. The code can be compiled and works fine with Java 6u45 and 7u75. > > Basically, I have an interface with this method: > > > T foo(); > > where Bar is > > interface Bar> {} > > I can implement this method without problem > > @Override > public > T foo() { > return null; > } > > but I can't override such a method anymore in Java 8: > > @Override > public > T foo() { > T result = super.foo(); > return result; > } > > > I've a small project which demonstrates the issue. I tried to open a bug but couldn't create a user in the bug database. > > What should I do, now? > > Regards, > From maurizio.cimadamore at oracle.com Mon Apr 20 16:13:03 2015 From: maurizio.cimadamore at oracle.com (Maurizio Cimadamore) Date: Mon, 20 Apr 2015 17:13:03 +0100 Subject: Regression in the javac 1.8u40 In-Reply-To: <55352362.5060408@oracle.com> References: <579b-55351c80-5b-4b5ec680@148209838> <55352362.5060408@oracle.com> Message-ID: <5535258F.4040200@oracle.com> Hi Aaron, the following program seems to compile for me - what am I missing? interface Bar> { > T foo(); } class A implements Bar { @Override public > T foo() { return null; } } class B extends A { @Override public > T foo() { T result = super.foo(); return result; } } Maurizio On 20/04/15 17:03, Rob McKenna wrote: > > T foo(); > > where Bar is > > interface Bar> {} > > I can implement this method without problem > @Override > public > T foo() { > return null; > } > > but I can't override such a method anymore in Java 8: > > @Override > public > T foo() { > T result = super.foo(); > return result; > } From maurizio.cimadamore at oracle.com Tue Apr 21 14:22:25 2015 From: maurizio.cimadamore at oracle.com (Maurizio Cimadamore) Date: Tue, 21 Apr 2015 15:22:25 +0100 Subject: Regression in the javac 1.8u40 In-Reply-To: <5535258F.4040200@oracle.com> References: <579b-55351c80-5b-4b5ec680@148209838> <55352362.5060408@oracle.com> <5535258F.4040200@oracle.com> Message-ID: <55365D21.3020105@oracle.com> FYI, with the help of Aaron, we've been able to identify the problem - I've filed this: https://bugs.openjdk.java.net/browse/JDK-8078249 Maurizio On 20/04/15 17:13, Maurizio Cimadamore wrote: > Hi Aaron, > the following program seems to compile for me - what am I missing? > > > interface Bar> { > > T foo(); > } > > class A implements Bar { > @Override > public > T foo() { > return null; > } > } > > class B extends A { > @Override > public > T foo() { > T result = super.foo(); > return result; > } > } > > > Maurizio > > On 20/04/15 17:03, Rob McKenna wrote: >> > T foo(); >> >> where Bar is >> >> interface Bar> {} >> >> I can implement this method without problem >> @Override >> public > T foo() { >> return null; >> } >> >> but I can't override such a method anymore in Java 8: >> >> @Override >> public > T foo() { >> T result = super.foo(); >> return result; >> } > From georgiy.rakov at oracle.com Wed Apr 22 16:37:50 2015 From: georgiy.rakov at oracle.com (Georgiy Rakov) Date: Wed, 22 Apr 2015 19:37:50 +0300 Subject: Anonymous class creation and diamonds: bound 'E extends B' Message-ID: <5537CE5E.9040103@oracle.com> Hello, let's consider following example: class B {} class Foo> { public Foo complexMethod(E a) { return this; } } public class Test65 { public static void check() { Foo t4 = new Foo<>() { }; } } This code successfully compiles on JDK9b60. However according to my understanding the compilation should have failed as per new spec. Could you please tell if this understanding is correct. The reasons why I think that the compilation should have failed are presented below. E is inferred as B, where Y is a fresh type variable with the upper bound B. If this is correct the given code should cause compilation failure according to following new assertions presented in the JDK-8073593 issue comment : ***It is a compile-time error if the superclass or superinterface type of the anonymous class, T, or any subexpression of T, has one of the following forms: - A type variable (4.4) that was not declared as a type parameter (such as a type variable produced by capture conversion (5.1.10)) - An intersection type (4.9) - A class or interface type, where the class or interface declaration is not accessible from the class or interface in which the expression appears.*** The term "subexpression" includes type arguments of parameterized types (4.5), bounds of wildcards (4.5.1), and element types of array types (10.1). It excludes bounds of type variables.*** The reason for this is that anonymous class super type, which is parameterized, has a type argument (subexpression) being a /type variable which was not declared as a type parameter.// /// The fact that E is inferred as a type variable with the upper bound becomes more obvious if we decide to override complexMethod: Foo t4 = new Foo<>() { public Foo complexMethod(B a){ return this; } //public Foo complexMethod(B a){ return this; } ;//this causes compilation failure }; In this case providing return type as Foo causes compilation failure. Only specifying the return type as Foo gets compilation to succeed. In general the reasons why I think that E is inferred here as B are presented below (just meaningful steps are presented). Could you please tell if they are correct. 1. Initial bounds set created from type parameter bounds contains following items as per JLS 18.1.3: e :< B (e - is inference variable); e :< Object (this is added because e had no proper upper bounds); 2. Then these bounds are processed by resolution process (JLS 18.4). During resolution e :< Object causes instantiation e=Object according to following assertion from JLS 18.4: Otherwise, where a_i has /proper/ upper bounds U_1 , ..., U_k , T_i = glb(U_1 , ..., U_k ) 3. Incorporating e=Object causes following new constraint to be added Object :< B according to following assertion from JLS 18.3.1: a = U and S |<:| T imply ?S|[|a:=U|]| |<:| T|[|a:=U|]|? 5. Constraint Object :< B reduces to false causing the second resolution attempt to take effect according to following assertion from JLS 18.4: Otherwise, the result contains the bound /false/, so a second attempt is made to instantiate { a_1 , ..., a_n } by performing the step below. 4. Fresh type variable Y with upper bound B is introduced according to assertions from JLS 18.4 presented below (Y upper bound is glb(Object,B[e:=Y]) = B[e:=Y] = B): then let Y_1 , ..., Y_n be fresh type variables whose bounds are as follows: * For all /i/(1 ? /i/? /n/), where a_i has upper bounds U_1 , ..., U_k , let the upper bound of Y_i be glb(U_1 q, ..., U_k q), where qis the substitution |[|a_1 :=Y_1 , ..., a_n :=Y_n |]|. 5. Finally e is instantiated as a fresh type variableY with the upper boundB according to the following assertion from JLS 18.4: Otherwise, for all /i/ (1 ? /i/ ? /n/), all bounds of the form G|<|..., a_i , ...|>| = capture(G|<|...|>|) are removed from the current bound set, /_*and the bounds *_//_*a*_//_*_1 *_//_*= *_//_*Y_1 *_//_*, ..., *_//_*a*_//_*_n *_//_*= *_//_*Y_n *_//_*are incorporated. *_/ Thanks, Georgiy. -------------- next part -------------- An HTML attachment was scrubbed... URL: -------------- next part -------------- class B {} class F extends B {} class Foo> { public Foo complexMethod(E a) { return this; } } public class Test65 { public static void check() { Foo t4 = new Foo<>() { public Foo complexMethod(B a){ return this; } //public Foo complexMethod(B a){ return this; } ;//this causes compilation failure }; } } From maurizio.cimadamore at oracle.com Wed Apr 22 17:32:57 2015 From: maurizio.cimadamore at oracle.com (Maurizio Cimadamore) Date: Wed, 22 Apr 2015 18:32:57 +0100 Subject: Anonymous class creation and diamonds: bound 'E extends B' In-Reply-To: <5537CE5E.9040103@oracle.com> References: <5537CE5E.9040103@oracle.com> Message-ID: <5537DB49.9080606@oracle.com> I believe you are correct - and I believe the issue is that the fresh variable generated by javac is internally a TypeVar object, rather than a CapturedType object - meaning that it will fool the denotable check (which looks for CapturedTypes). @Srikanth - can you look into this? @Georgiy - good catch! Many thanks! Maurizio On 22/04/15 17:37, Georgiy Rakov wrote: > Hello, > > let's consider following example: > > class B {} > > class Foo> { > public Foo complexMethod(E a) { return this; } > } > > public class Test65 { > public static void check() { > Foo t4 = new Foo<>() { > }; > } > } > > This code successfully compiles on JDK9b60. However according to my > understanding the compilation should have failed as per new spec. > Could you please tell if this understanding is correct. > > The reasons why I think that the compilation should have failed are > presented below. > > E is inferred as B, where Y is a fresh type variable with the upper > bound B. If this is correct the given code should cause > compilation failure according to following new assertions presented in > the JDK-8073593 issue comment > : > > ***It is a compile-time error if the superclass or superinterface > type of the anonymous class, T, or any subexpression of T, has one > of the following forms: > - A type variable (4.4) that was not declared as a type parameter > (such as a type variable produced by capture conversion (5.1.10)) > - An intersection type (4.9) > - A class or interface type, where the class or interface > declaration is not accessible from the class or interface in which > the expression appears.*** > The term "subexpression" includes type arguments of parameterized > types (4.5), bounds of wildcards (4.5.1), and element types of > array types (10.1). It excludes bounds of type variables.*** > > The reason for this is that anonymous class super type, which is > parameterized, has a type argument (subexpression) being a /type > variable which was not declared as a type parameter.// > /// > The fact that E is inferred as a type variable with the upper bound > becomes more obvious if we decide to override complexMethod: > Foo t4 = new Foo<>() { > public Foo complexMethod(B a){ return this; } > //public Foo complexMethod(B a){ return this; } ;//this causes compilation failure > }; > > In this case providing return type as Foo causes compilation > failure. Only specifying the return type as Foo gets > compilation to succeed. > > In general the reasons why I think that E is inferred here as B are > presented below (just meaningful steps are presented). Could you > please tell if they are correct. > > 1. Initial bounds set created from type parameter bounds contains > following items as per JLS 18.1.3: > > e :< B (e - is inference variable); > e :< Object (this is added because e had no proper upper bounds); > > 2. Then these bounds are processed by resolution process (JLS 18.4). > During resolution e :< Object causes instantiation e=Object according > to following assertion from JLS 18.4: > > Otherwise, where a_i has /proper/ upper bounds U_1 , ..., U_k , > T_i = glb(U_1 , ..., U_k ) > > 3. Incorporating e=Object causes following new constraint to be added > Object :< B according to following assertion from JLS 18.3.1: > > a = U and S |<:| T imply ?S|[|a:=U|]| |<:| T|[|a:=U|]|? > > 5. Constraint Object :< B reduces to false causing the second > resolution attempt to take effect according to following assertion > from JLS 18.4: > > Otherwise, the result contains the bound /false/, so a second > attempt is made to instantiate { a_1 , ..., a_n } by performing > the step below. > > 4. Fresh type variable Y with upper bound B is introduced according > to assertions from JLS 18.4 presented below (Y upper bound is > glb(Object,B[e:=Y]) = B[e:=Y] = B): > > then let Y_1 , ..., Y_n be fresh type variables whose bounds are > as follows: > * For all /i/(1 ? /i/? /n/), where a_i has upper bounds U_1 , ..., > U_k , let the upper bound of Y_i be glb(U_1 q, ..., U_k q), where > qis the substitution |[|a_1 :=Y_1 , ..., a_n :=Y_n |]|. > > 5. Finally e is instantiated as a fresh type variableY with the upper > boundB according to the following assertion from JLS 18.4: > > Otherwise, for all /i/ (1 ? /i/ ? /n/), all bounds of the form > G|<|..., a_i , ...|>| = capture(G|<|...|>|) are removed from the > current bound set, /_*and the bounds *_//_*a*_//_*_1 *_//_*= > *_//_*Y_1 *_//_*, ..., *_//_*a*_//_*_n *_//_*= *_//_*Y_n *_//_*are > incorporated. > > *_/ > > Thanks, > Georgiy. -------------- next part -------------- An HTML attachment was scrubbed... URL: From wdietl at gmail.com Wed Apr 22 18:00:01 2015 From: wdietl at gmail.com (Werner Dietl) Date: Wed, 22 Apr 2015 14:00:01 -0400 Subject: Invalid Class-Path in compiler jar files Message-ID: I've made a clean jdk9/dev/langtools compiler build. I then try running the compiler using: java -classpath ./lib/java.compiler.jar:./lib/jdk.compiler.jar com.sun.tools.javac.Main -Xlint Min.java where Min.java is a trivial class. I get: warning: [path] bad path element "jdk9-dev/langtools/dist/./lib/@{jarclasspath}": no such file or directory 1 warning I think this is caused by the MANIFEST.MF files in these .jar files: Manifest-Version: 1.0 Ant-Version: Apache Ant 1.9.3 Created-By: 1.8.0_45-b14 (Oracle Corporation) Class-Path: @{jarclasspath} I use this way of invoking the compiler from an ant build.xml file. Does anyone else see the same behavior? Thanks, cu, WMD. -- http://www.google.com/profiles/wdietl From mark.reinhold at oracle.com Thu Apr 23 22:30:46 2015 From: mark.reinhold at oracle.com (mark.reinhold at oracle.com) Date: Thu, 23 Apr 2015 15:30:46 -0700 (PDT) Subject: JEP 247: Compile for Older Platform Versions Message-ID: <20150423223046.D40E557B97@eggemoggin.niobe.net> New JEP Candidate: http://openjdk.java.net/jeps/247 - Mark From georgiy.rakov at oracle.com Fri Apr 24 14:58:54 2015 From: georgiy.rakov at oracle.com (Georgiy Rakov) Date: Fri, 24 Apr 2015 17:58:54 +0300 Subject: Anonymous class creation and diamonds: bound 'E extends B' In-Reply-To: <5537DB49.9080606@oracle.com> References: <5537CE5E.9040103@oracle.com> <5537DB49.9080606@oracle.com> Message-ID: <553A5A2E.1060909@oracle.com> Created the issue for this: https://bugs.openjdk.java.net/browse/JDK-8078618 Thanks, Georgiy. On 22.04.2015 20:32, Maurizio Cimadamore wrote: > I believe you are correct - and I believe the issue is that the fresh > variable generated by javac is internally a TypeVar object, rather > than a CapturedType object - meaning that it will fool the denotable > check (which looks for CapturedTypes). > > @Srikanth - can you look into this? > > @Georgiy - good catch! Many thanks! > > Maurizio > > On 22/04/15 17:37, Georgiy Rakov wrote: >> Hello, >> >> let's consider following example: >> >> class B {} >> >> class Foo> { >> public Foo complexMethod(E a) { return this; } >> } >> >> public class Test65 { >> public static void check() { >> Foo t4 = new Foo<>() { >> }; >> } >> } >> >> This code successfully compiles on JDK9b60. However according to my >> understanding the compilation should have failed as per new spec. >> Could you please tell if this understanding is correct. >> >> The reasons why I think that the compilation should have failed are >> presented below. >> >> E is inferred as B, where Y is a fresh type variable with the >> upper bound B. If this is correct the given code should cause >> compilation failure according to following new assertions presented >> in the JDK-8073593 issue comment >> : >> >> ***It is a compile-time error if the superclass or superinterface >> type of the anonymous class, T, or any subexpression of T, has >> one of the following forms: >> - A type variable (4.4) that was not declared as a type parameter >> (such as a type variable produced by capture conversion (5.1.10)) >> - An intersection type (4.9) >> - A class or interface type, where the class or interface >> declaration is not accessible from the class or interface in >> which the expression appears.*** >> The term "subexpression" includes type arguments of parameterized >> types (4.5), bounds of wildcards (4.5.1), and element types of >> array types (10.1). It excludes bounds of type variables.*** >> >> The reason for this is that anonymous class super type, which is >> parameterized, has a type argument (subexpression) being a /type >> variable which was not declared as a type parameter.// >> /// >> The fact that E is inferred as a type variable with the upper bound >> becomes more obvious if we decide to override complexMethod: >> Foo t4 = new Foo<>() { >> public Foo complexMethod(B a){ return this; } >> //public Foo complexMethod(B a){ return this; } ;//this causes compilation failure >> }; >> >> In this case providing return type as Foo causes compilation >> failure. Only specifying the return type as Foo gets >> compilation to succeed. >> >> In general the reasons why I think that E is inferred here as B >> are presented below (just meaningful steps are presented). Could you >> please tell if they are correct. >> >> 1. Initial bounds set created from type parameter bounds contains >> following items as per JLS 18.1.3: >> >> e :< B (e - is inference variable); >> e :< Object (this is added because e had no proper upper bounds); >> >> 2. Then these bounds are processed by resolution process (JLS 18.4). >> During resolution e :< Object causes instantiation e=Object according >> to following assertion from JLS 18.4: >> >> Otherwise, where a_i has /proper/ upper bounds U_1 , ..., U_k , >> T_i = glb(U_1 , ..., U_k ) >> >> 3. Incorporating e=Object causes following new constraint to be added >> Object :< B according to following assertion from JLS 18.3.1: >> >> a = U and S |<:| T imply ?S|[|a:=U|]| |<:| T|[|a:=U|]|? >> >> 5. Constraint Object :< B reduces to false causing the second >> resolution attempt to take effect according to following assertion >> from JLS 18.4: >> >> Otherwise, the result contains the bound /false/, so a second >> attempt is made to instantiate { a_1 , ..., a_n } by performing >> the step below. >> >> 4. Fresh type variable Y with upper bound B is introduced >> according to assertions from JLS 18.4 presented below (Y upper bound >> is glb(Object,B[e:=Y]) = B[e:=Y] = B): >> >> then let Y_1 , ..., Y_n be fresh type variables whose bounds are >> as follows: >> * For all /i/(1 ? /i/? /n/), where a_i has upper bounds U_1 , >> ..., U_k , let the upper bound of Y_i be glb(U_1 q, ..., U_k q), >> where qis the substitution |[|a_1 :=Y_1 , ..., a_n :=Y_n |]|. >> >> 5. Finally e is instantiated as a fresh type variableY with the upper >> boundB according to the following assertion from JLS 18.4: >> >> Otherwise, for all /i/ (1 ? /i/ ? /n/), all bounds of the form >> G|<|..., a_i , ...|>| = capture(G|<|...|>|) are removed from the >> current bound set, /_*and the bounds *_//_*a*_//_*_1 *_//_*= >> *_//_*Y_1 *_//_*, ..., *_//_*a*_//_*_n *_//_*= *_//_*Y_n >> *_//_*are incorporated. >> >> *_/ >> >> Thanks, >> Georgiy. > -------------- next part -------------- An HTML attachment was scrubbed... URL: From srikanth.adayapalam at oracle.com Fri Apr 24 15:03:07 2015 From: srikanth.adayapalam at oracle.com (Srikanth) Date: Fri, 24 Apr 2015 20:33:07 +0530 Subject: Anonymous class creation and diamonds: bound 'E extends B' In-Reply-To: <553A5A2E.1060909@oracle.com> References: <5537CE5E.9040103@oracle.com> <5537DB49.9080606@oracle.com> <553A5A2E.1060909@oracle.com> Message-ID: <553A5B2B.2010405@oracle.com> On Friday 24 April 2015 08:28 PM, Georgiy Rakov wrote: > Created the issue for this: > https://bugs.openjdk.java.net/browse/JDK-8078618 Hi Georgiy, I am sorry, I already created a ticket for this, but didn't manage to notify here before you ended up creating a duplicate. I'll close the new one as the old one has a fix under consideration already. Thanks! Srikanth > > Thanks, > Georgiy. > > On 22.04.2015 20:32, Maurizio Cimadamore wrote: >> I believe you are correct - and I believe the issue is that the fresh >> variable generated by javac is internally a TypeVar object, rather >> than a CapturedType object - meaning that it will fool the denotable >> check (which looks for CapturedTypes). >> >> @Srikanth - can you look into this? >> >> @Georgiy - good catch! Many thanks! >> >> Maurizio >> >> On 22/04/15 17:37, Georgiy Rakov wrote: >>> Hello, >>> >>> let's consider following example: >>> >>> class B {} >>> >>> class Foo> { >>> public Foo complexMethod(E a) { return this; } >>> } >>> >>> public class Test65 { >>> public static void check() { >>> Foo t4 = new Foo<>() { >>> }; >>> } >>> } >>> >>> This code successfully compiles on JDK9b60. However according to my >>> understanding the compilation should have failed as per new spec. >>> Could you please tell if this understanding is correct. >>> >>> The reasons why I think that the compilation should have failed are >>> presented below. >>> >>> E is inferred as B, where Y is a fresh type variable with the >>> upper bound B. If this is correct the given code should cause >>> compilation failure according to following new assertions presented >>> in the JDK-8073593 issue comment >>> : >>> >>> ***It is a compile-time error if the superclass or >>> superinterface type of the anonymous class, T, or any >>> subexpression of T, has one of the following forms: >>> - A type variable (4.4) that was not declared as a type >>> parameter (such as a type variable produced by capture >>> conversion (5.1.10)) >>> - An intersection type (4.9) >>> - A class or interface type, where the class or interface >>> declaration is not accessible from the class or interface in >>> which the expression appears.*** >>> The term "subexpression" includes type arguments of >>> parameterized types (4.5), bounds of wildcards (4.5.1), and >>> element types of array types (10.1). It excludes bounds of type >>> variables.*** >>> >>> The reason for this is that anonymous class super type, which is >>> parameterized, has a type argument (subexpression) being a /type >>> variable which was not declared as a type parameter.// >>> /// >>> The fact that E is inferred as a type variable with the upper bound >>> becomes more obvious if we decide to override complexMethod: >>> Foo t4 = new Foo<>() { >>> public Foo complexMethod(B a){ return this; } >>> //public Foo complexMethod(B a){ return this; } ;//this causes compilation failure >>> }; >>> >>> In this case providing return type as Foo causes compilation >>> failure. Only specifying the return type as Foo gets >>> compilation to succeed. >>> >>> In general the reasons why I think that E is inferred here as B >>> are presented below (just meaningful steps are presented). Could you >>> please tell if they are correct. >>> >>> 1. Initial bounds set created from type parameter bounds contains >>> following items as per JLS 18.1.3: >>> >>> e :< B (e - is inference variable); >>> e :< Object (this is added because e had no proper upper bounds); >>> >>> 2. Then these bounds are processed by resolution process (JLS 18.4). >>> During resolution e :< Object causes instantiation e=Object >>> according to following assertion from JLS 18.4: >>> >>> Otherwise, where a_i has /proper/ upper bounds U_1 , ..., U_k , >>> T_i = glb(U_1 , ..., U_k ) >>> >>> 3. Incorporating e=Object causes following new constraint to be >>> added Object :< B according to following assertion from JLS >>> 18.3.1: >>> >>> a = U and S |<:| T imply ?S|[|a:=U|]| |<:| T|[|a:=U|]|? >>> >>> 5. Constraint Object :< B reduces to false causing the >>> second resolution attempt to take effect according to following >>> assertion from JLS 18.4: >>> >>> Otherwise, the result contains the bound /false/, so a second >>> attempt is made to instantiate { a_1 , ..., a_n } by performing >>> the step below. >>> >>> 4. Fresh type variable Y with upper bound B is introduced >>> according to assertions from JLS 18.4 presented below (Y upper bound >>> is glb(Object,B[e:=Y]) = B[e:=Y] = B): >>> >>> then let Y_1 , ..., Y_n be fresh type variables whose bounds are >>> as follows: >>> * For all /i/(1 ? /i/? /n/), where a_i has upper bounds U_1 , >>> ..., U_k , let the upper bound of Y_i be glb(U_1 q, ..., U_k q), >>> where qis the substitution |[|a_1 :=Y_1 , ..., a_n :=Y_n |]|. >>> >>> 5. Finally e is instantiated as a fresh type variableY with the >>> upper boundB according to the following assertion from JLS 18.4: >>> >>> Otherwise, for all /i/ (1 ? /i/ ? /n/), all bounds of the form >>> G|<|..., a_i , ...|>| = capture(G|<|...|>|) are removed from the >>> current bound set, /_*and the bounds *_//_*a*_//_*_1 *_//_*= >>> *_//_*Y_1 *_//_*, ..., *_//_*a*_//_*_n *_//_*= *_//_*Y_n >>> *_//_*are incorporated. >>> >>> *_/ >>> >>> Thanks, >>> Georgiy. >> > -------------- next part -------------- An HTML attachment was scrubbed... URL: From daniel.smith at oracle.com Fri Apr 24 19:34:18 2015 From: daniel.smith at oracle.com (Dan Smith) Date: Fri, 24 Apr 2015 13:34:18 -0600 Subject: Anonymous class creation and diamonds: bound 'E extends B' In-Reply-To: <5537DB49.9080606@oracle.com> References: <5537CE5E.9040103@oracle.com> <5537DB49.9080606@oracle.com> Message-ID: Georgiy asked me to take a look at this. FWIW, I agree: this is the kind of thing that the check is supposed to prevent (and, downstream, I wonder what Signature attribute we end up with, since we can't express 'B'. ?Dan > On Apr 22, 2015, at 11:32 AM, Maurizio Cimadamore wrote: > > I believe you are correct - and I believe the issue is that the fresh variable generated by javac is internally a TypeVar object, rather than a CapturedType object - meaning that it will fool the denotable check (which looks for CapturedTypes). > > @Srikanth - can you look into this? > > @Georgiy - good catch! Many thanks! > > Maurizio > > On 22/04/15 17:37, Georgiy Rakov wrote: >> Hello, >> >> let's consider following example: >> class B {} >> >> class Foo> { >> public Foo complexMethod(E a) { return this; } >> } >> >> public class Test65 { >> public static void check() { >> Foo t4 = new Foo<>() { >> }; >> } >> } >> This code successfully compiles on JDK9b60. However according to my understanding the compilation should have failed as per new spec. Could you please tell if this understanding is correct. >> >> The reasons why I think that the compilation should have failed are presented below. >> >> E is inferred as B, where Y is a fresh type variable with the upper bound B. If this is correct the given code should cause compilation failure according to following new assertions presented in the JDK-8073593 issue comment : >> ***It is a compile-time error if the superclass or superinterface type of the anonymous class, T, or any subexpression of T, has one of the following forms: >> - A type variable (4.4) that was not declared as a type parameter (such as a type variable produced by capture conversion (5.1.10)) >> - An intersection type (4.9) >> - A class or interface type, where the class or interface declaration is not accessible from the class or interface in which the expression appears.*** >> The term "subexpression" includes type arguments of parameterized types (4.5), bounds of wildcards (4.5.1), and element types of array types (10.1). It excludes bounds of type variables.*** >> The reason for this is that anonymous class super type, which is parameterized, has a type argument (subexpression) being a type variable which was not declared as a type parameter. >> >> The fact that E is inferred as a type variable with the upper bound becomes more obvious if we decide to override complexMethod: >> Foo t4 = new Foo<>() { >> public Foo complexMethod(B a){ return this; } >> //public Foo complexMethod(B a){ return this; } ;//this causes compilation failure >> }; >> >> In this case providing return type as Foo causes compilation failure. Only specifying the return type as Foo gets compilation to succeed. >> >> In general the reasons why I think that E is inferred here as B are presented below (just meaningful steps are presented). Could you please tell if they are correct. >> >> 1. Initial bounds set created from type parameter bounds contains following items as per JLS 18.1.3: >> e :< B (e - is inference variable); >> e :< Object (this is added because e had no proper upper bounds); >> 2. Then these bounds are processed by resolution process (JLS 18.4). During resolution e :< Object causes instantiation e=Object according to following assertion from JLS 18.4: >> Otherwise, where ai has proper upper bounds U1, ..., Uk, Ti = glb(U1, ..., Uk) >> 3. Incorporating e=Object causes following new constraint to be added Object :< B according to following assertion from JLS 18.3.1: >> a = U and S <: T imply ?S[a:=U] <: T[a:=U]? >> 5. Constraint Object :< B reduces to false causing the second resolution attempt to take effect according to following assertion from JLS 18.4: >> Otherwise, the result contains the bound false, so a second attempt is made to instantiate { a1, ..., an } by performing the step below. >> 4. Fresh type variable Y with upper bound B is introduced according to assertions from JLS 18.4 presented below (Y upper bound is glb(Object,B[e:=Y]) = B[e:=Y] = B): >> then let Y1, ..., Yn be fresh type variables whose bounds are as follows: >> * For all i (1 ? i ? n), where ai has upper bounds U1, ..., Uk, let the upper bound of Yi be glb(U1 q, ..., Uk q), where q is the substitution [a1:=Y1, ..., an:=Yn]. >> 5. Finally e is instantiated as a fresh type variable Y with the upper bound B according to the following assertion from JLS 18.4: >> Otherwise, for all i (1 ? i ? n), all bounds of the form G<..., ai, ...> = capture(G<...>) are removed from the current bound set, and the bounds a1 = Y1, ..., an = Yn are incorporated. >> >> Thanks, >> Georgiy. > -------------- next part -------------- An HTML attachment was scrubbed... URL: From wieland at testingtech.com Fri Apr 24 08:49:14 2015 From: wieland at testingtech.com (Jacob Wieland) Date: Fri, 24 Apr 2015 10:49:14 +0200 Subject: Java Performance Degradation in JDK7 and JDK8 Message-ID: Hello Folks, I still have the open problem https://bugs.openjdk.java.net/browse/JDK-8039262 that the javac performance has degraded significantly from 1.6 to 1.7 (and even worse to 1.8) in the 64bit versions. Since in our context, we are dealing with a lot of generated source1.4 Java input (either split into very large files with inner classes or big packages with lots of smaller classes), compiler performance is critical for our tool and this degradation forces us to continue recommending to our customers to use Java 1.6 for large projects (as is the norm) as 1.7 and 1.8 are pretty much unusable in this respect. Is anyone still working on this issue or is such significant performance degradation not a serious issue? My observations so far are: - it gets worse the more class files are being compiled/the more files reside in the source path - cpu usage goes through the roof over all available kernels - memory usage is much higher - garbage collection seems to be much more active Using -proc:none alleviates the problem slightly for the 1.7 version, but not for 1.8 (last we tested with jdk1.8.0_31) where the performance difference is a factor 5 or more! I can understand that a more advanced compiler has capabilities that a previous version does not have and thus sometimes has to do more work. But, it should still be possible (especially if given the -source 1.4 or -source 1.5 option as we do) to optimize it in such a way that unnecessary checks for generics, overriding methods, closures, annotations and other newer features can be turned off (if they are to blame, which I actually doubt from my observations). I would really appreciate your help in this regard and I think everyone would benefit from any bugfix you can offer for this. BR, Jacob Wieland -- -- ------------------------------ ------------------------------------------- Dr. Jacob Wieland Senior Software Engineer Testing Technologies IST GmbH Michaelkirchstra?e 17/18 10179 Berlin, Germany Phone +49 30 726 19 19 34 Email wieland at testingtech.com Fax +49 30 726 19 19 20 Internet www.testingtech.com --------------------------------------------------------------------------------------------------------------- ----------------------------------------------------------------------------------------------------------------- UPCOMING EVENTS SUBMIT YOUR TOPIC for the UCAAT 2015 Deadline: May 30, 2015ucaat.etsi.org/2015/CallForPresentations.html Apr 21-23, 2015 | SAE Conference & Exhibition Detroit, Michigan, USAwww.sae.org/congress/ Apr 28-30, 2015 | iqnite Dusseldorf, Germanywww.iqnite-conferences.com/de/index.aspx ----------------------------------------------------------------------------------------------------------------- Gesch?ftsf?hrung: Theofanis Vassiliou-Gioles, Stephan Pietsch, Pete Nicholson Handelsregister HRB 77805 B, Amtsgericht Charlottenburg Ust ID Nr.: DE 813 143 070 This email may contain confidential and privileged material for the sole use of the intended recipient. Any review, use, distribution or disclosure by others is strictly prohibited. If you are not the intended recipient (or authorized to receive for the recipient), please contact the sender by reply email and delete all copies of this message. -------------- next part -------------- An HTML attachment was scrubbed... URL: From wieland at testingtech.com Tue Apr 28 07:34:13 2015 From: wieland at testingtech.com (Jacob Wieland) Date: Tue, 28 Apr 2015 09:34:13 +0200 Subject: Java Performance Degradation in JDK7 and JDK8 In-Reply-To: References: Message-ID: Following up on my own post: maybe this has something to do with the bug as we are using inner classes a lot JDK-8000316 as we are using inner classes a lot. 2015-04-24 10:49 GMT+02:00 Jacob Wieland : > Hello Folks, > > > I still have the open problem > https://bugs.openjdk.java.net/browse/JDK-8039262 that the javac > performance has degraded significantly from 1.6 to 1.7 (and even worse to > 1.8) in the 64bit versions. Since in our context, we are dealing with a lot > of generated source1.4 Java input (either split into very large files with > inner classes or big packages with lots of smaller classes), compiler > performance is critical for our tool and this degradation forces us to > continue recommending to our customers to use Java 1.6 for large projects > (as is the norm) as 1.7 and 1.8 are pretty much unusable in this respect. > > Is anyone still working on this issue or is such significant performance > degradation not a serious issue? > > My observations so far are: > - it gets worse the more class files are being compiled/the more files > reside in the source path > - cpu usage goes through the roof over all available kernels > - memory usage is much higher > - garbage collection seems to be much more active > > Using -proc:none alleviates the problem slightly for the 1.7 version, but > not for 1.8 (last we tested with jdk1.8.0_31) where the performance > difference is a factor 5 or more! > > I can understand that a more advanced compiler has capabilities that a > previous version does not have and thus sometimes has > to do more work. But, it should still be possible (especially if given the > -source 1.4 or -source 1.5 option as we do) to optimize it in such a way > that unnecessary checks for generics, overriding methods, closures, > annotations and other newer features can be turned off (if they are to > blame, which I actually doubt from my observations). > > I would really appreciate your help in this regard and I think everyone > would benefit from any bugfix you can offer for this. > > BR, Jacob Wieland > > -- > -- > ------------------------------ > ------------------------------------------- > Dr. Jacob Wieland > Senior Software Engineer > > Testing Technologies IST GmbH > Michaelkirchstra?e 17/18 > 10179 Berlin, Germany > > Phone +49 30 726 19 19 34 Email wieland at testingtech.com > > Fax +49 30 726 19 19 20 Internet www.testingtech.com > > --------------------------------------------------------------------------------------------------------------- > > > ----------------------------------------------------------------------------------------------------------------- > > UPCOMING EVENTS > > SUBMIT YOUR TOPIC for the UCAAT 2015 > Deadline: May 30, 2015ucaat.etsi.org/2015/CallForPresentations.html > > Apr 21-23, 2015 | SAE Conference & Exhibition > Detroit, Michigan, USAwww.sae.org/congress/ > > Apr 28-30, 2015 | iqnite > Dusseldorf, Germanywww.iqnite-conferences.com/de/index.aspx > > > ----------------------------------------------------------------------------------------------------------------- > Gesch?ftsf?hrung: Theofanis Vassiliou-Gioles, Stephan Pietsch, Pete > Nicholson Handelsregister HRB 77805 B, Amtsgericht Charlottenburg Ust ID > Nr.: DE 813 143 070 This email may contain confidential and privileged > material for the sole use of the intended recipient. Any review, use, > distribution or disclosure by others is strictly prohibited. If you are not > the intended recipient (or authorized to receive for the recipient), please > contact the sender by reply email and delete all copies of this message. > -- -- ------------------------------ ------------------------------------------- Dr. Jacob Wieland Senior Software Engineer Testing Technologies IST GmbH Michaelkirchstra?e 17/18 10179 Berlin, Germany Phone +49 30 726 19 19 34 Email wieland at testingtech.com Fax +49 30 726 19 19 20 Internet www.testingtech.com --------------------------------------------------------------------------------------------------------------- ----------------------------------------------------------------------------------------------------------------- UPCOMING EVENTS SUBMIT YOUR TOPIC for the UCAAT 2015 Deadline: May 30, 2015ucaat.etsi.org/2015/CallForPresentations.html Apr 21-23, 2015 | SAE Conference & Exhibition Detroit, Michigan, USAwww.sae.org/congress/ Apr 28-30, 2015 | iqnite Dusseldorf, Germanywww.iqnite-conferences.com/de/index.aspx ----------------------------------------------------------------------------------------------------------------- Gesch?ftsf?hrung: Theofanis Vassiliou-Gioles, Stephan Pietsch, Pete Nicholson Handelsregister HRB 77805 B, Amtsgericht Charlottenburg Ust ID Nr.: DE 813 143 070 This email may contain confidential and privileged material for the sole use of the intended recipient. Any review, use, distribution or disclosure by others is strictly prohibited. If you are not the intended recipient (or authorized to receive for the recipient), please contact the sender by reply email and delete all copies of this message. -------------- next part -------------- An HTML attachment was scrubbed... URL: From fweimer at redhat.com Tue Apr 28 22:34:27 2015 From: fweimer at redhat.com (Florian Weimer) Date: Wed, 29 Apr 2015 00:34:27 +0200 Subject: Java Performance Degradation in JDK7 and JDK8 In-Reply-To: References: Message-ID: <55400AF3.9000606@redhat.com> On 04/24/2015 10:49 AM, Jacob Wieland wrote: > Is anyone still working on this issue or is such significant performance > degradation not a serious issue? Can you provide a script which generates an unencumbered, self-contained test case? -- Florian Weimer / Red Hat Product Security From wieland at testingtech.com Wed Apr 29 09:05:30 2015 From: wieland at testingtech.com (Jacob Wieland) Date: Wed, 29 Apr 2015 11:05:30 +0200 Subject: Java Performance Degradation in JDK7 and JDK8 In-Reply-To: <55400AF3.9000606@redhat.com> References: <55400AF3.9000606@redhat.com> Message-ID: Just copy the 3 jar files attached to the bug https://bugs.openjdk.java.net/browse/JDK-8039262 into a directory, add the attached TestCase.sh and run it while the directory is the current working dir. -------------- next part -------------- An HTML attachment was scrubbed... URL: -------------- next part -------------- A non-text attachment was scrubbed... Name: TestCase.sh Type: application/x-sh Size: 102 bytes Desc: not available URL: From maurizio.cimadamore at oracle.com Wed Apr 29 09:29:29 2015 From: maurizio.cimadamore at oracle.com (Maurizio Cimadamore) Date: Wed, 29 Apr 2015 10:29:29 +0100 Subject: Java Performance Degradation in JDK7 and JDK8 In-Reply-To: References: Message-ID: <5540A479.7010401@oracle.com> Hi Jacob, Stay assured - as we'll definitively look into this issue (I see it's already assigned to one of my colleagues). What I can say (w/o looking too much at the attached artifacts) is that in general, javac has no issue with compiling a lot of sources at once; at one point the build system was structured in such a way that all the JDK classes were compiled at once - and that (which is way more than your 187 sources - i.e. at least 10x that) took less than 20 seconds. SO there must some specific pattern triggering that issue. Given that you say you have 187 input sources and 48K output classes, I'd say you are using inner classes a lot. I wonder if you are hitting this: https://bugs.openjdk.java.net/browse/JDK-8000316 Maurizio On 24/04/15 09:49, Jacob Wieland wrote: > Hello Folks, > > > I still have the open problem > https://bugs.openjdk.java.net/browse/JDK-8039262 that the javac > performance has degraded significantly from 1.6 to 1.7 (and even worse > to 1.8) in the 64bit versions. Since in our context, we are dealing > with a lot of generated source1.4 Java input (either split into very > large files with inner classes or big packages with lots of smaller > classes), compiler performance is critical for our tool and this > degradation forces us to continue recommending to our customers to use > Java 1.6 for large projects (as is the norm) as 1.7 and 1.8 are pretty > much unusable in this respect. > > Is anyone still working on this issue or is such significant > performance degradation not a serious issue? > > My observations so far are: > - it gets worse the more class files are being compiled/the more files > reside in the source path > - cpu usage goes through the roof over all available kernels > - memory usage is much higher > - garbage collection seems to be much more active > > Using -proc:none alleviates the problem slightly for the 1.7 version, > but not for 1.8 (last we tested with jdk1.8.0_31) where the > performance difference is a factor 5 or more! > > I can understand that a more advanced compiler has capabilities that a > previous version does not have and thus sometimes has > to do more work. But, it should still be possible (especially if given > the -source 1.4 or -source 1.5 option as we do) to optimize it in such > a way that unnecessary checks for generics, overriding methods, > closures, annotations and other newer features can be turned off (if > they are to blame, which I actually doubt from my observations). > > I would really appreciate your help in this regard and I think > everyone would benefit from any bugfix you can offer for this. > > BR, Jacob Wieland > > -- > -- > ------------------------------ > ------------------------------------------- > Dr. Jacob Wieland > Senior Software Engineer > > Testing Technologies IST GmbH > Michaelkirchstra?e 17/18 > 10179 Berlin, Germany > > Phone +49 30 726 19 19 34 Email wieland at testingtech.com > > Fax +49 30 726 19 19 20 Internet www.testingtech.com > > --------------------------------------------------------------------------------------------------------------- > > ----------------------------------------------------------------------------------------------------------------- > > UPCOMING EVENTS > > SUBMIT YOUR TOPIC for the UCAAT 2015 > Deadline: May 30, 2015 > ucaat.etsi.org/2015/CallForPresentations.html > > Apr 21-23, 2015 | SAE Conference & Exhibition > Detroit, Michigan, USA > www.sae.org/congress/ > > Apr 28-30, 2015 | iqnite > Dusseldorf, Germany > www.iqnite-conferences.com/de/index.aspx > ----------------------------------------------------------------------------------------------------------------- > Gesch?ftsf?hrung: Theofanis Vassiliou-Gioles, Stephan Pietsch, Pete > Nicholson Handelsregister HRB 77805 B, Amtsgericht Charlottenburg Ust > ID Nr.: DE 813 143 070 This email may contain confidential and > privileged material for the sole use of the intended recipient. Any > review, use, distribution or disclosure by others is strictly > prohibited. If you are not the intended recipient (or authorized to > receive for the recipient), please contact the sender by reply email > and delete all copies of this message. -------------- next part -------------- An HTML attachment was scrubbed... URL: From maurizio.cimadamore at oracle.com Wed Apr 29 10:00:45 2015 From: maurizio.cimadamore at oracle.com (Maurizio Cimadamore) Date: Wed, 29 Apr 2015 11:00:45 +0100 Subject: Java Performance Degradation in JDK7 and JDK8 In-Reply-To: <5540A479.7010401@oracle.com> References: <5540A479.7010401@oracle.com> Message-ID: <5540ABCD.3080708@oracle.com> These are the numbers I'm getting: JDK 9 (b42) Note: generated_ttcn/TTCN3_CommonDefs.java uses or overrides a deprecated API. Note: Recompile with -Xlint:deprecation for details. real 0m46.306s user 2m17.489s sys 0m2.166s JDK 8 (GA) Note: generated_ttcn/TTCN3_CommonDefs.java uses or overrides a deprecated API. Note: Recompile with -Xlint:deprecation for details. real 6m58.748s user 8m43.546s sys 0m2.132s JDK 7 (1.7.0_79) Note: generated_ttcn/TTCN3_CommonDefs.java uses or overrides a deprecated API. Note: Recompile with -Xlint:deprecation for details. real 0m28.341s user 1m17.194s sys 0m1.886s As you can see there is a significant regression from JDK 7 to JDK 8 which was caused by https://bugs.openjdk.java.net/browse/JDK-8043253 (some stack trace analysis revealed the familiar pattern). This has also been fixed in JDK 8u20 (as stated in the bug evaluation). So, while JDK 8u20/9 is slower than JDK 7 (at least on my machine), the numbers are more or less in the same ballpark and the huge regression that was visible in earlier JDK 8 releases has now been fixed. If you are still experiencing the problem - can you please also submit the specific compiler versions you are using in your benchmark? Maurizio On 29/04/15 10:29, Maurizio Cimadamore wrote: > Hi Jacob, > Stay assured - as we'll definitively look into this issue (I see it's > already assigned to one of my colleagues). > > What I can say (w/o looking too much at the attached artifacts) is > that in general, javac has no issue with compiling a lot of sources at > once; at one point the build system was structured in such a way that > all the JDK classes were compiled at once - and that (which is way > more than your 187 sources - i.e. at least 10x that) took less than 20 > seconds. SO there must some specific pattern triggering that issue. > > Given that you say you have 187 input sources and 48K output classes, > I'd say you are using inner classes a lot. I wonder if you are hitting > this: > > https://bugs.openjdk.java.net/browse/JDK-8000316 > > Maurizio > > On 24/04/15 09:49, Jacob Wieland wrote: >> Hello Folks, >> >> >> I still have the open problem >> https://bugs.openjdk.java.net/browse/JDK-8039262 that the javac >> performance has degraded significantly from 1.6 to 1.7 (and even >> worse to 1.8) in the 64bit versions. Since in our context, we are >> dealing with a lot of generated source1.4 Java input (either split >> into very large files with inner classes or big packages with lots of >> smaller classes), compiler performance is critical for our tool and >> this degradation forces us to continue recommending to our customers >> to use Java 1.6 for large projects (as is the norm) as 1.7 and 1.8 >> are pretty much unusable in this respect. >> >> Is anyone still working on this issue or is such significant >> performance degradation not a serious issue? >> >> My observations so far are: >> - it gets worse the more class files are being compiled/the more >> files reside in the source path >> - cpu usage goes through the roof over all available kernels >> - memory usage is much higher >> - garbage collection seems to be much more active >> >> Using -proc:none alleviates the problem slightly for the 1.7 version, >> but not for 1.8 (last we tested with jdk1.8.0_31) where the >> performance difference is a factor 5 or more! >> >> I can understand that a more advanced compiler has capabilities that >> a previous version does not have and thus sometimes has >> to do more work. But, it should still be possible (especially if >> given the -source 1.4 or -source 1.5 option as we do) to optimize it >> in such a way that unnecessary checks for generics, overriding >> methods, closures, annotations and other newer features can be turned >> off (if they are to blame, which I actually doubt from my observations). >> >> I would really appreciate your help in this regard and I think >> everyone would benefit from any bugfix you can offer for this. >> >> BR, Jacob Wieland >> >> -- >> -- >> ------------------------------ >> ------------------------------------------- >> Dr. Jacob Wieland >> Senior Software Engineer >> >> Testing Technologies IST GmbH >> Michaelkirchstra?e 17/18 >> 10179 Berlin, Germany >> >> Phone +49 30 726 19 19 34 Email wieland at testingtech.com >> >> Fax +49 30 726 19 19 20 Internet www.testingtech.com >> >> --------------------------------------------------------------------------------------------------------------- >> >> ----------------------------------------------------------------------------------------------------------------- >> >> UPCOMING EVENTS >> >> SUBMIT YOUR TOPIC for the UCAAT 2015 >> Deadline: May 30, 2015 >> ucaat.etsi.org/2015/CallForPresentations.html >> >> Apr 21-23, 2015 | SAE Conference & Exhibition >> Detroit, Michigan, USA >> www.sae.org/congress/ >> >> Apr 28-30, 2015 | iqnite >> Dusseldorf, Germany >> www.iqnite-conferences.com/de/index.aspx >> ----------------------------------------------------------------------------------------------------------------- >> Gesch?ftsf?hrung: Theofanis Vassiliou-Gioles, Stephan Pietsch, Pete >> Nicholson Handelsregister HRB 77805 B, Amtsgericht Charlottenburg Ust >> ID Nr.: DE 813 143 070 This email may contain confidential and >> privileged material for the sole use of the intended recipient. Any >> review, use, distribution or disclosure by others is strictly >> prohibited. If you are not the intended recipient (or authorized to >> receive for the recipient), please contact the sender by reply email >> and delete all copies of this message. > -------------- next part -------------- An HTML attachment was scrubbed... URL: From maurizio.cimadamore at oracle.com Wed Apr 29 10:14:07 2015 From: maurizio.cimadamore at oracle.com (Maurizio Cimadamore) Date: Wed, 29 Apr 2015 11:14:07 +0100 Subject: Java Performance Degradation in JDK7 and JDK8 In-Reply-To: <5540ABCD.3080708@oracle.com> References: <5540A479.7010401@oracle.com> <5540ABCD.3080708@oracle.com> Message-ID: <5540AEEF.4080106@oracle.com> One last note/clarification - if you use JDK 9/JDK 8u20 with the following options: -source 6 -target 6 (which seem to be similar to the one you mention in the bug report), then the time goes back to this: warning: [options] bootstrap class path not set in conjunction with -source 1.6 warning: [options] source value 1.6 is obsolete and will be removed in a future release warning: [options] target value 1.6 is obsolete and will be removed in a future release warning: [options] To suppress warnings about obsolete options, use -Xlint:-options. Note: generated_ttcn/TTCN3_CommonDefs.java uses or overrides a deprecated API. Note: Recompile with -Xlint:deprecation for details. 4 warnings real 0m31.916s user 1m27.915s sys 0m1.639s Which is essentially the same as the one obtained in JDK 7. So, I see no regression whatsoever from 7 to 8 (if you compile with -source 7 - that is). Maurizio On 29/04/15 11:00, Maurizio Cimadamore wrote: > JDK 9 (b42) > > Note: generated_ttcn/TTCN3_CommonDefs.java uses or overrides a > deprecated API. > Note: Recompile with -Xlint:deprecation for details. > > real 0m46.306s > user 2m17.489s > sys 0m2.166s From wieland at testingtech.com Wed Apr 29 11:15:05 2015 From: wieland at testingtech.com (Jacob Wieland) Date: Wed, 29 Apr 2015 13:15:05 +0200 Subject: Java Performance Degradation in JDK7 and JDK8 In-Reply-To: <5540A479.7010401@oracle.com> References: <5540A479.7010401@oracle.com> Message-ID: Hello Maurizio, I was wondering the same thing. However, I have changed the code-pattern experimentally to use no inner classes (at least not as many) and instead use top-level-classes which all reference each other (and need explicit references where before there were convenient implicit references to the containing outer class objects) and the performance got worse instead of better, so I don't know if it has to do with inner classes after all. Of course, it could be inner classes in one case and lots of files in the other. But that's just guesswork. Is there a fix for the above issue that could be tried out. We are definitely not using class names like A$B in our context. 2015-04-29 11:29 GMT+02:00 Maurizio Cimadamore < maurizio.cimadamore at oracle.com>: > Hi Jacob, > Stay assured - as we'll definitively look into this issue (I see it's > already assigned to one of my colleagues). > > What I can say (w/o looking too much at the attached artifacts) is that in > general, javac has no issue with compiling a lot of sources at once; at one > point the build system was structured in such a way that all the JDK > classes were compiled at once - and that (which is way more than your 187 > sources - i.e. at least 10x that) took less than 20 seconds. SO there must > some specific pattern triggering that issue. > > Given that you say you have 187 input sources and 48K output classes, I'd > say you are using inner classes a lot. I wonder if you are hitting this: > > https://bugs.openjdk.java.net/browse/JDK-8000316 > > Maurizio > > > On 24/04/15 09:49, Jacob Wieland wrote: > > Hello Folks, > > > I still have the open problem > https://bugs.openjdk.java.net/browse/JDK-8039262 that the javac > performance has degraded significantly from 1.6 to 1.7 (and even worse to > 1.8) in the 64bit versions. Since in our context, we are dealing with a lot > of generated source1.4 Java input (either split into very large files with > inner classes or big packages with lots of smaller classes), compiler > performance is critical for our tool and this degradation forces us to > continue recommending to our customers to use Java 1.6 for large projects > (as is the norm) as 1.7 and 1.8 are pretty much unusable in this respect. > > Is anyone still working on this issue or is such significant performance > degradation not a serious issue? > > My observations so far are: > - it gets worse the more class files are being compiled/the more files > reside in the source path > - cpu usage goes through the roof over all available kernels > - memory usage is much higher > - garbage collection seems to be much more active > > Using -proc:none alleviates the problem slightly for the 1.7 version, > but not for 1.8 (last we tested with jdk1.8.0_31) where the performance > difference is a factor 5 or more! > > I can understand that a more advanced compiler has capabilities that a > previous version does not have and thus sometimes has > to do more work. But, it should still be possible (especially if given the > -source 1.4 or -source 1.5 option as we do) to optimize it in such a way > that unnecessary checks for generics, overriding methods, closures, > annotations and other newer features can be turned off (if they are to > blame, which I actually doubt from my observations). > > I would really appreciate your help in this regard and I think everyone > would benefit from any bugfix you can offer for this. > > BR, Jacob Wieland > > -- > -- > ------------------------------ > ------------------------------------------- > Dr. Jacob Wieland > Senior Software Engineer > > Testing Technologies IST GmbH > Michaelkirchstra?e 17/18 > 10179 Berlin, Germany > > Phone +49 30 726 19 19 34 Email wieland at testingtech.com > > Fax +49 30 726 19 19 20 Internet www.testingtech.com > > --------------------------------------------------------------------------------------------------------------- > > > ----------------------------------------------------------------------------------------------------------------- > > UPCOMING EVENTS > > SUBMIT YOUR TOPIC for the UCAAT 2015 > Deadline: May 30, 2015ucaat.etsi.org/2015/CallForPresentations.html > > Apr 21-23, 2015 | SAE Conference & Exhibition > Detroit, Michigan, USAwww.sae.org/congress/ > > Apr 28-30, 2015 | iqnite > Dusseldorf, Germanywww.iqnite-conferences.com/de/index.aspx > > ----------------------------------------------------------------------------------------------------------------- > Gesch?ftsf?hrung: Theofanis Vassiliou-Gioles, Stephan Pietsch, Pete > Nicholson Handelsregister HRB 77805 B, Amtsgericht Charlottenburg Ust ID > Nr.: DE 813 143 070 This email may contain confidential and privileged > material for the sole use of the intended recipient. Any review, use, > distribution or disclosure by others is strictly prohibited. If you are not > the intended recipient (or authorized to receive for the recipient), please > contact the sender by reply email and delete all copies of this message. > > > -- -- ------------------------------ ------------------------------------------- Dr. Jacob Wieland Senior Software Engineer Testing Technologies IST GmbH Michaelkirchstra?e 17/18 10179 Berlin, Germany Phone +49 30 726 19 19 34 Email wieland at testingtech.com Fax +49 30 726 19 19 20 Internet www.testingtech.com --------------------------------------------------------------------------------------------------------------- ----------------------------------------------------------------------------------------------------------------- UPCOMING EVENTS SUBMIT YOUR TOPIC for the UCAAT 2015 Deadline: May 30, 2015ucaat.etsi.org/2015/CallForPresentations.html Apr 21-23, 2015 | SAE Conference & Exhibition Detroit, Michigan, USAwww.sae.org/congress/ Apr 28-30, 2015 | iqnite Dusseldorf, Germanywww.iqnite-conferences.com/de/index.aspx ----------------------------------------------------------------------------------------------------------------- Gesch?ftsf?hrung: Theofanis Vassiliou-Gioles, Stephan Pietsch, Pete Nicholson Handelsregister HRB 77805 B, Amtsgericht Charlottenburg Ust ID Nr.: DE 813 143 070 This email may contain confidential and privileged material for the sole use of the intended recipient. Any review, use, distribution or disclosure by others is strictly prohibited. If you are not the intended recipient (or authorized to receive for the recipient), please contact the sender by reply email and delete all copies of this message. -------------- next part -------------- An HTML attachment was scrubbed... URL: From wieland at testingtech.com Wed Apr 29 11:44:45 2015 From: wieland at testingtech.com (Jacob Wieland) Date: Wed, 29 Apr 2015 13:44:45 +0200 Subject: Java Performance Degradation in JDK7 and JDK8 In-Reply-To: <5540ABCD.3080708@oracle.com> References: <5540A479.7010401@oracle.com> <5540ABCD.3080708@oracle.com> Message-ID: Hello Maurizio, are you sure that you used the 64bit versions of javac? I could only observe the behavior with these. Also, I just tried with jdk8u31-64b and it takes AGES (still running after 17 minutes where the jdk6 was done after 2), top shows 4GB VIRT memory use and 350 % load (on a 4core processor). So, I don't think it was that problem. 2015-04-29 12:00 GMT+02:00 Maurizio Cimadamore < maurizio.cimadamore at oracle.com>: > These are the numbers I'm getting: > > JDK 9 (b42) > > Note: generated_ttcn/TTCN3_CommonDefs.java uses or overrides a deprecated > API. > Note: Recompile with -Xlint:deprecation for details. > > real 0m46.306s > user 2m17.489s > sys 0m2.166s > > JDK 8 (GA) > > Note: generated_ttcn/TTCN3_CommonDefs.java uses or overrides a deprecated > API. > Note: Recompile with -Xlint:deprecation for details. > > real 6m58.748s > user 8m43.546s > sys 0m2.132s > > JDK 7 (1.7.0_79) > > Note: generated_ttcn/TTCN3_CommonDefs.java uses or overrides a deprecated > API. > Note: Recompile with -Xlint:deprecation for details. > > real 0m28.341s > user 1m17.194s > sys 0m1.886s > > > As you can see there is a significant regression from JDK 7 to JDK 8 which > was caused by > > https://bugs.openjdk.java.net/browse/JDK-8043253 > > (some stack trace analysis revealed the familiar pattern). This has also > been fixed in JDK 8u20 (as stated in the bug evaluation). > > So, while JDK 8u20/9 is slower than JDK 7 (at least on my machine), the > numbers are more or less in the same ballpark and the huge regression that > was visible in earlier JDK 8 releases has now been fixed. > > If you are still experiencing the problem - can you please also submit the > specific compiler versions you are using in your benchmark? > > Maurizio > > > > On 29/04/15 10:29, Maurizio Cimadamore wrote: > > Hi Jacob, > Stay assured - as we'll definitively look into this issue (I see it's > already assigned to one of my colleagues). > > What I can say (w/o looking too much at the attached artifacts) is that in > general, javac has no issue with compiling a lot of sources at once; at one > point the build system was structured in such a way that all the JDK > classes were compiled at once - and that (which is way more than your 187 > sources - i.e. at least 10x that) took less than 20 seconds. SO there must > some specific pattern triggering that issue. > > Given that you say you have 187 input sources and 48K output classes, I'd > say you are using inner classes a lot. I wonder if you are hitting this: > > https://bugs.openjdk.java.net/browse/JDK-8000316 > > Maurizio > > On 24/04/15 09:49, Jacob Wieland wrote: > > Hello Folks, > > > I still have the open problem > https://bugs.openjdk.java.net/browse/JDK-8039262 that the javac > performance has degraded significantly from 1.6 to 1.7 (and even worse to > 1.8) in the 64bit versions. Since in our context, we are dealing with a lot > of generated source1.4 Java input (either split into very large files with > inner classes or big packages with lots of smaller classes), compiler > performance is critical for our tool and this degradation forces us to > continue recommending to our customers to use Java 1.6 for large projects > (as is the norm) as 1.7 and 1.8 are pretty much unusable in this respect. > > Is anyone still working on this issue or is such significant performance > degradation not a serious issue? > > My observations so far are: > - it gets worse the more class files are being compiled/the more files > reside in the source path > - cpu usage goes through the roof over all available kernels > - memory usage is much higher > - garbage collection seems to be much more active > > Using -proc:none alleviates the problem slightly for the 1.7 version, > but not for 1.8 (last we tested with jdk1.8.0_31) where the performance > difference is a factor 5 or more! > > I can understand that a more advanced compiler has capabilities that a > previous version does not have and thus sometimes has > to do more work. But, it should still be possible (especially if given the > -source 1.4 or -source 1.5 option as we do) to optimize it in such a way > that unnecessary checks for generics, overriding methods, closures, > annotations and other newer features can be turned off (if they are to > blame, which I actually doubt from my observations). > > I would really appreciate your help in this regard and I think everyone > would benefit from any bugfix you can offer for this. > > BR, Jacob Wieland > > -- > -- > ------------------------------ > ------------------------------------------- > Dr. Jacob Wieland > Senior Software Engineer > > Testing Technologies IST GmbH > Michaelkirchstra?e 17/18 > 10179 Berlin, Germany > > Phone +49 30 726 19 19 34 Email wieland at testingtech.com > > Fax +49 30 726 19 19 20 Internet www.testingtech.com > > --------------------------------------------------------------------------------------------------------------- > > > ----------------------------------------------------------------------------------------------------------------- > > UPCOMING EVENTS > > SUBMIT YOUR TOPIC for the UCAAT 2015 > Deadline: May 30, 2015ucaat.etsi.org/2015/CallForPresentations.html > > Apr 21-23, 2015 | SAE Conference & Exhibition > Detroit, Michigan, USAwww.sae.org/congress/ > > Apr 28-30, 2015 | iqnite > Dusseldorf, Germanywww.iqnite-conferences.com/de/index.aspx > > ----------------------------------------------------------------------------------------------------------------- > Gesch?ftsf?hrung: Theofanis Vassiliou-Gioles, Stephan Pietsch, Pete > Nicholson Handelsregister HRB 77805 B, Amtsgericht Charlottenburg Ust ID > Nr.: DE 813 143 070 This email may contain confidential and privileged > material for the sole use of the intended recipient. Any review, use, > distribution or disclosure by others is strictly prohibited. If you are not > the intended recipient (or authorized to receive for the recipient), please > contact the sender by reply email and delete all copies of this message. > > > > -- -- ------------------------------ ------------------------------------------- Dr. Jacob Wieland Senior Software Engineer Testing Technologies IST GmbH Michaelkirchstra?e 17/18 10179 Berlin, Germany Phone +49 30 726 19 19 34 Email wieland at testingtech.com Fax +49 30 726 19 19 20 Internet www.testingtech.com --------------------------------------------------------------------------------------------------------------- ----------------------------------------------------------------------------------------------------------------- UPCOMING EVENTS SUBMIT YOUR TOPIC for the UCAAT 2015 Deadline: May 30, 2015ucaat.etsi.org/2015/CallForPresentations.html Apr 21-23, 2015 | SAE Conference & Exhibition Detroit, Michigan, USAwww.sae.org/congress/ Apr 28-30, 2015 | iqnite Dusseldorf, Germanywww.iqnite-conferences.com/de/index.aspx ----------------------------------------------------------------------------------------------------------------- Gesch?ftsf?hrung: Theofanis Vassiliou-Gioles, Stephan Pietsch, Pete Nicholson Handelsregister HRB 77805 B, Amtsgericht Charlottenburg Ust ID Nr.: DE 813 143 070 This email may contain confidential and privileged material for the sole use of the intended recipient. Any review, use, distribution or disclosure by others is strictly prohibited. If you are not the intended recipient (or authorized to receive for the recipient), please contact the sender by reply email and delete all copies of this message. -------------- next part -------------- An HTML attachment was scrubbed... URL: From maurizio.cimadamore at oracle.com Wed Apr 29 13:12:16 2015 From: maurizio.cimadamore at oracle.com (Maurizio Cimadamore) Date: Wed, 29 Apr 2015 14:12:16 +0100 Subject: Java Performance Degradation in JDK7 and JDK8 In-Reply-To: References: <5540A479.7010401@oracle.com> <5540ABCD.3080708@oracle.com> Message-ID: <5540D8B0.3070604@oracle.com> On 29/04/15 12:44, Jacob Wieland wrote: > Hello Maurizio, > > are you sure that you used the 64bit versions of javac? I could only > observe the behavior with these. Yep I'm on a Ubuntu x64 machine. It's actually pretty standard hardware too - i.e. intel i5 (two cores, but OS sees 4 because of hyper-threading). > Also, I just tried with jdk8u31-64b and it takes AGES (still running > after 17 minutes where the jdk6 was done after 2), top shows 4GB VIRT > memory use and 350 % load (on a 4core processor). Maybe the reproducer you sent was incorrect? Maurizio > > So, I don't think it was that problem. > > 2015-04-29 12:00 GMT+02:00 Maurizio Cimadamore > >: > > These are the numbers I'm getting: > > JDK 9 (b42) > > Note: generated_ttcn/TTCN3_CommonDefs.java uses or overrides a > deprecated API. > Note: Recompile with -Xlint:deprecation for details. > > real 0m46.306s > user 2m17.489s > sys 0m2.166s > > JDK 8 (GA) > > Note: generated_ttcn/TTCN3_CommonDefs.java uses or overrides a > deprecated API. > Note: Recompile with -Xlint:deprecation for details. > > real 6m58.748s > user 8m43.546s > sys 0m2.132s > > JDK 7 (1.7.0_79) > > Note: generated_ttcn/TTCN3_CommonDefs.java uses or overrides a > deprecated API. > Note: Recompile with -Xlint:deprecation for details. > > real 0m28.341s > user 1m17.194s > sys 0m1.886s > > > As you can see there is a significant regression from JDK 7 to JDK > 8 which was caused by > > https://bugs.openjdk.java.net/browse/JDK-8043253 > > (some stack trace analysis revealed the familiar pattern). This > has also been fixed in JDK 8u20 (as stated in the bug evaluation). > > So, while JDK 8u20/9 is slower than JDK 7 (at least on my > machine), the numbers are more or less in the same ballpark and > the huge regression that was visible in earlier JDK 8 releases has > now been fixed. > > If you are still experiencing the problem - can you please also > submit the specific compiler versions you are using in your benchmark? > > Maurizio > > > > On 29/04/15 10:29, Maurizio Cimadamore wrote: >> Hi Jacob, >> Stay assured - as we'll definitively look into this issue (I see >> it's already assigned to one of my colleagues). >> >> What I can say (w/o looking too much at the attached artifacts) >> is that in general, javac has no issue with compiling a lot of >> sources at once; at one point the build system was structured in >> such a way that all the JDK classes were compiled at once - and >> that (which is way more than your 187 sources - i.e. at least 10x >> that) took less than 20 seconds. SO there must some specific >> pattern triggering that issue. >> >> Given that you say you have 187 input sources and 48K output >> classes, I'd say you are using inner classes a lot. I wonder if >> you are hitting this: >> >> https://bugs.openjdk.java.net/browse/JDK-8000316 >> >> Maurizio >> >> On 24/04/15 09:49, Jacob Wieland wrote: >>> Hello Folks, >>> >>> >>> I still have the open problem >>> https://bugs.openjdk.java.net/browse/JDK-8039262 that the javac >>> performance has degraded significantly from 1.6 to 1.7 (and even >>> worse to 1.8) in the 64bit versions. Since in our context, we >>> are dealing with a lot of generated source1.4 Java input (either >>> split into very large files with inner classes or big packages >>> with lots of smaller classes), compiler performance is critical >>> for our tool and this degradation forces us to continue >>> recommending to our customers to use Java 1.6 for large projects >>> (as is the norm) as 1.7 and 1.8 are pretty much unusable in this >>> respect. >>> >>> Is anyone still working on this issue or is such significant >>> performance degradation not a serious issue? >>> >>> My observations so far are: >>> - it gets worse the more class files are being compiled/the more >>> files reside in the source path >>> - cpu usage goes through the roof over all available kernels >>> - memory usage is much higher >>> - garbage collection seems to be much more active >>> >>> Using -proc:none alleviates the problem slightly for the 1.7 >>> version, but not for 1.8 (last we tested with jdk1.8.0_31) where >>> the performance difference is a factor 5 or more! >>> >>> I can understand that a more advanced compiler has capabilities >>> that a previous version does not have and thus sometimes has >>> to do more work. But, it should still be possible (especially if >>> given the -source 1.4 or -source 1.5 option as we do) to >>> optimize it in such a way that unnecessary checks for generics, >>> overriding methods, closures, annotations and other newer >>> features can be turned off (if they are to blame, which I >>> actually doubt from my observations). >>> >>> I would really appreciate your help in this regard and I think >>> everyone would benefit from any bugfix you can offer for this. >>> >>> BR, Jacob Wieland >>> >>> -- >>> -- >>> ------------------------------ >>> ------------------------------------------- >>> Dr. Jacob Wieland >>> Senior Software Engineer >>> >>> Testing Technologies IST GmbH >>> Michaelkirchstra?e 17/18 >>> 10179 Berlin, Germany >>> >>> Phone +49 30 726 19 19 34 Email wieland at testingtech.com >>> >>> Fax +49 30 726 19 19 20 Internet www.testingtech.com >>> >>> --------------------------------------------------------------------------------------------------------------- >>> >>> ----------------------------------------------------------------------------------------------------------------- >>> >>> UPCOMING EVENTS >>> >>> SUBMIT YOUR TOPIC for the UCAAT 2015 >>> Deadline: May 30, 2015 >>> ucaat.etsi.org/2015/CallForPresentations.html >>> >>> Apr 21-23, 2015 | SAE Conference & Exhibition >>> Detroit, Michigan, USA >>> www.sae.org/congress/ >>> >>> Apr 28-30, 2015 | iqnite >>> Dusseldorf, Germany >>> www.iqnite-conferences.com/de/index.aspx >>> ----------------------------------------------------------------------------------------------------------------- >>> Gesch?ftsf?hrung: Theofanis Vassiliou-Gioles, Stephan Pietsch, >>> Pete Nicholson Handelsregister HRB 77805 B, Amtsgericht >>> Charlottenburg Ust ID Nr.: DE 813 143 070 This email may contain >>> confidential and privileged material for the sole use of the >>> intended recipient. Any review, use, distribution or disclosure >>> by others is strictly prohibited. If you are not the intended >>> recipient (or authorized to receive for the recipient), please >>> contact the sender by reply email and delete all copies of this >>> message. >> > > > > > -- > -- > ------------------------------ > ------------------------------------------- > Dr. Jacob Wieland > Senior Software Engineer > > Testing Technologies IST GmbH > Michaelkirchstra?e 17/18 > 10179 Berlin, Germany > > Phone +49 30 726 19 19 34 Email wieland at testingtech.com > > Fax +49 30 726 19 19 20 Internet www.testingtech.com > > --------------------------------------------------------------------------------------------------------------- > > ----------------------------------------------------------------------------------------------------------------- > > UPCOMING EVENTS > > SUBMIT YOUR TOPIC for the UCAAT 2015 > Deadline: May 30, 2015 > ucaat.etsi.org/2015/CallForPresentations.html > > Apr 21-23, 2015 | SAE Conference & Exhibition > Detroit, Michigan, USA > www.sae.org/congress/ > > Apr 28-30, 2015 | iqnite > Dusseldorf, Germany > www.iqnite-conferences.com/de/index.aspx > ----------------------------------------------------------------------------------------------------------------- > Gesch?ftsf?hrung: Theofanis Vassiliou-Gioles, Stephan Pietsch, Pete > Nicholson Handelsregister HRB 77805 B, Amtsgericht Charlottenburg Ust > ID Nr.: DE 813 143 070 This email may contain confidential and > privileged material for the sole use of the intended recipient. Any > review, use, distribution or disclosure by others is strictly > prohibited. If you are not the intended recipient (or authorized to > receive for the recipient), please contact the sender by reply email > and delete all copies of this message. -------------- next part -------------- An HTML attachment was scrubbed... URL: From maurizio.cimadamore at oracle.com Wed Apr 29 13:55:56 2015 From: maurizio.cimadamore at oracle.com (Maurizio Cimadamore) Date: Wed, 29 Apr 2015 14:55:56 +0100 Subject: Java Performance Degradation in JDK7 and JDK8 In-Reply-To: <5540D8B0.3070604@oracle.com> References: <5540A479.7010401@oracle.com> <5540ABCD.3080708@oracle.com> <5540D8B0.3070604@oracle.com> Message-ID: <5540E2EC.9010106@oracle.com> For the records - this is what I get with JDK 8u45 (x64)- on Linux x64: warning: [options] bootstrap class path not set in conjunction with -source 1.6 Note: generated_ttcn/TTCN3_CommonDefs.java uses or overrides a deprecated API. Note: Recompile with -Xlint:deprecation for details. 1 warning real 0m35.347s user 1m43.281s sys 0m1.806s If you only experience the problem with x64 JDK, is it possible that we are staring more at a VM/runtime issue? Maurizio On 29/04/15 14:12, Maurizio Cimadamore wrote: > On 29/04/15 12:44, Jacob Wieland wrote: >> Hello Maurizio, >> >> are you sure that you used the 64bit versions of javac? I could only >> observe the behavior with these. > Yep I'm on a Ubuntu x64 machine. It's actually pretty standard > hardware too - i.e. intel i5 (two cores, but OS sees 4 because of > hyper-threading). >> Also, I just tried with jdk8u31-64b and it takes AGES (still running >> after 17 minutes where the jdk6 was done after 2), top shows 4GB VIRT >> memory use and 350 % load (on a 4core processor). > Maybe the reproducer you sent was incorrect? > > Maurizio >> >> So, I don't think it was that problem. >> >> 2015-04-29 12:00 GMT+02:00 Maurizio Cimadamore >> >: >> >> These are the numbers I'm getting: >> >> JDK 9 (b42) >> >> Note: generated_ttcn/TTCN3_CommonDefs.java uses or overrides a >> deprecated API. >> Note: Recompile with -Xlint:deprecation for details. >> >> real 0m46.306s >> user 2m17.489s >> sys 0m2.166s >> >> JDK 8 (GA) >> >> Note: generated_ttcn/TTCN3_CommonDefs.java uses or overrides a >> deprecated API. >> Note: Recompile with -Xlint:deprecation for details. >> >> real 6m58.748s >> user 8m43.546s >> sys 0m2.132s >> >> JDK 7 (1.7.0_79) >> >> Note: generated_ttcn/TTCN3_CommonDefs.java uses or overrides a >> deprecated API. >> Note: Recompile with -Xlint:deprecation for details. >> >> real 0m28.341s >> user 1m17.194s >> sys 0m1.886s >> >> >> As you can see there is a significant regression from JDK 7 to >> JDK 8 which was caused by >> >> https://bugs.openjdk.java.net/browse/JDK-8043253 >> >> (some stack trace analysis revealed the familiar pattern). This >> has also been fixed in JDK 8u20 (as stated in the bug evaluation). >> >> So, while JDK 8u20/9 is slower than JDK 7 (at least on my >> machine), the numbers are more or less in the same ballpark and >> the huge regression that was visible in earlier JDK 8 releases >> has now been fixed. >> >> If you are still experiencing the problem - can you please also >> submit the specific compiler versions you are using in your >> benchmark? >> >> Maurizio >> >> >> >> On 29/04/15 10:29, Maurizio Cimadamore wrote: >>> Hi Jacob, >>> Stay assured - as we'll definitively look into this issue (I see >>> it's already assigned to one of my colleagues). >>> >>> What I can say (w/o looking too much at the attached artifacts) >>> is that in general, javac has no issue with compiling a lot of >>> sources at once; at one point the build system was structured in >>> such a way that all the JDK classes were compiled at once - and >>> that (which is way more than your 187 sources - i.e. at least >>> 10x that) took less than 20 seconds. SO there must some specific >>> pattern triggering that issue. >>> >>> Given that you say you have 187 input sources and 48K output >>> classes, I'd say you are using inner classes a lot. I wonder if >>> you are hitting this: >>> >>> https://bugs.openjdk.java.net/browse/JDK-8000316 >>> >>> Maurizio >>> >>> On 24/04/15 09:49, Jacob Wieland wrote: >>>> Hello Folks, >>>> >>>> >>>> I still have the open problem >>>> https://bugs.openjdk.java.net/browse/JDK-8039262 that the javac >>>> performance has degraded significantly from 1.6 to 1.7 (and >>>> even worse to 1.8) in the 64bit versions. Since in our context, >>>> we are dealing with a lot of generated source1.4 Java input >>>> (either split into very large files with inner classes or big >>>> packages with lots of smaller classes), compiler performance is >>>> critical for our tool and this degradation forces us to >>>> continue recommending to our customers to use Java 1.6 for >>>> large projects (as is the norm) as 1.7 and 1.8 are pretty much >>>> unusable in this respect. >>>> >>>> Is anyone still working on this issue or is such significant >>>> performance degradation not a serious issue? >>>> >>>> My observations so far are: >>>> - it gets worse the more class files are being compiled/the >>>> more files reside in the source path >>>> - cpu usage goes through the roof over all available kernels >>>> - memory usage is much higher >>>> - garbage collection seems to be much more active >>>> >>>> Using -proc:none alleviates the problem slightly for the 1.7 >>>> version, but not for 1.8 (last we tested with >>>> jdk1.8.0_31) where the performance difference is a factor 5 or >>>> more! >>>> >>>> I can understand that a more advanced compiler has capabilities >>>> that a previous version does not have and thus sometimes has >>>> to do more work. But, it should still be possible (especially >>>> if given the -source 1.4 or -source 1.5 option as we do) to >>>> optimize it in such a way that unnecessary checks for generics, >>>> overriding methods, closures, annotations and other newer >>>> features can be turned off (if they are to blame, which I >>>> actually doubt from my observations). >>>> >>>> I would really appreciate your help in this regard and I think >>>> everyone would benefit from any bugfix you can offer for this. >>>> >>>> BR, Jacob Wieland >>>> >>>> -- >>>> -- >>>> ------------------------------ >>>> ------------------------------------------- >>>> Dr. Jacob Wieland >>>> Senior Software Engineer >>>> >>>> Testing Technologies IST GmbH >>>> Michaelkirchstra?e 17/18 >>>> 10179 Berlin, Germany >>>> >>>> Phone +49 30 726 19 19 34 Email wieland at testingtech.com >>>> >>>> Fax +49 30 726 19 19 20 Internet www.testingtech.com >>>> >>>> --------------------------------------------------------------------------------------------------------------- >>>> >>>> ----------------------------------------------------------------------------------------------------------------- >>>> >>>> UPCOMING EVENTS >>>> >>>> SUBMIT YOUR TOPIC for the UCAAT 2015 >>>> Deadline: May 30, 2015 >>>> ucaat.etsi.org/2015/CallForPresentations.html >>>> >>>> Apr 21-23, 2015 | SAE Conference & Exhibition >>>> Detroit, Michigan, USA >>>> www.sae.org/congress/ >>>> >>>> Apr 28-30, 2015 | iqnite >>>> Dusseldorf, Germany >>>> www.iqnite-conferences.com/de/index.aspx >>>> ----------------------------------------------------------------------------------------------------------------- >>>> Gesch?ftsf?hrung: Theofanis Vassiliou-Gioles, Stephan Pietsch, >>>> Pete Nicholson Handelsregister HRB 77805 B, Amtsgericht >>>> Charlottenburg Ust ID Nr.: DE 813 143 070 This email may >>>> contain confidential and privileged material for the sole use >>>> of the intended recipient. Any review, use, distribution or >>>> disclosure by others is strictly prohibited. If you are not the >>>> intended recipient (or authorized to receive for the >>>> recipient), please contact the sender by reply email and delete >>>> all copies of this message. >>> >> >> >> >> >> -- >> -- >> ------------------------------ >> ------------------------------------------- >> Dr. Jacob Wieland >> Senior Software Engineer >> >> Testing Technologies IST GmbH >> Michaelkirchstra?e 17/18 >> 10179 Berlin, Germany >> >> Phone +49 30 726 19 19 34 Email wieland at testingtech.com >> >> Fax +49 30 726 19 19 20 Internet www.testingtech.com >> >> --------------------------------------------------------------------------------------------------------------- >> >> ----------------------------------------------------------------------------------------------------------------- >> >> UPCOMING EVENTS >> >> SUBMIT YOUR TOPIC for the UCAAT 2015 >> Deadline: May 30, 2015 >> ucaat.etsi.org/2015/CallForPresentations.html >> >> Apr 21-23, 2015 | SAE Conference & Exhibition >> Detroit, Michigan, USA >> www.sae.org/congress/ >> >> Apr 28-30, 2015 | iqnite >> Dusseldorf, Germany >> www.iqnite-conferences.com/de/index.aspx >> ----------------------------------------------------------------------------------------------------------------- >> Gesch?ftsf?hrung: Theofanis Vassiliou-Gioles, Stephan Pietsch, Pete >> Nicholson Handelsregister HRB 77805 B, Amtsgericht Charlottenburg Ust >> ID Nr.: DE 813 143 070 This email may contain confidential and >> privileged material for the sole use of the intended recipient. Any >> review, use, distribution or disclosure by others is strictly >> prohibited. If you are not the intended recipient (or authorized to >> receive for the recipient), please contact the sender by reply email >> and delete all copies of this message. > -------------- next part -------------- An HTML attachment was scrubbed... URL: From wieland at testingtech.com Wed Apr 29 14:06:39 2015 From: wieland at testingtech.com (Jacob Wieland) Date: Wed, 29 Apr 2015 16:06:39 +0200 Subject: Java Performance Degradation in JDK7 and JDK8 In-Reply-To: <5540D8B0.3070604@oracle.com> References: <5540A479.7010401@oracle.com> <5540ABCD.3080708@oracle.com> <5540D8B0.3070604@oracle.com> Message-ID: I have to admit, the reproducer is only a small part of the actual generated code. In my preliminary tests, it already sufficed to show a difference (also, the smaller code still worked with 32 bit while the whole code runs out of memory in all 32 bit versions which makes comparison much harder ;-) and which is why I need the 64 bit versions). If I test with the complete code which is much bigger, the results are as follows: jdk1.6u45(64bit) 2GB MaxMem - 1:30 minutes jdk1.7u75(64bit) 2GB MaxMem - > 6 min jdk1.8u31(64bit) 1GB MaxMem - > 15 min jdk1.8u31(64bit) 2GB MaxMem - > 10 min jdk1.8u31(64bit) 4GB MaxMem - 2:20 min (-source/-target 6 does not seem to have any effect) So, if you throw insane (in comparison with 1.6) amounts of memory at 1.7/1.8, it is only about a third as slow, but this still is unacceptable. I actually think it has to do with parallellization and garbage collection. 2015-04-29 15:12 GMT+02:00 Maurizio Cimadamore < maurizio.cimadamore at oracle.com>: > On 29/04/15 12:44, Jacob Wieland wrote: > > Hello Maurizio, > > are you sure that you used the 64bit versions of javac? I could only > observe the behavior with these. > > Yep I'm on a Ubuntu x64 machine. It's actually pretty standard hardware > too - i.e. intel i5 (two cores, but OS sees 4 because of hyper-threading). > > Also, I just tried with jdk8u31-64b and it takes AGES (still running > after 17 minutes where the jdk6 was done after 2), top shows 4GB VIRT > memory use and 350 % load (on a 4core processor). > > Maybe the reproducer you sent was incorrect? > > Maurizio > > > So, I don't think it was that problem. > > 2015-04-29 12:00 GMT+02:00 Maurizio Cimadamore < > maurizio.cimadamore at oracle.com>: > >> These are the numbers I'm getting: >> >> JDK 9 (b42) >> >> Note: generated_ttcn/TTCN3_CommonDefs.java uses or overrides a deprecated >> API. >> Note: Recompile with -Xlint:deprecation for details. >> >> real 0m46.306s >> user 2m17.489s >> sys 0m2.166s >> >> JDK 8 (GA) >> >> Note: generated_ttcn/TTCN3_CommonDefs.java uses or overrides a deprecated >> API. >> Note: Recompile with -Xlint:deprecation for details. >> >> real 6m58.748s >> user 8m43.546s >> sys 0m2.132s >> >> JDK 7 (1.7.0_79) >> >> Note: generated_ttcn/TTCN3_CommonDefs.java uses or overrides a deprecated >> API. >> Note: Recompile with -Xlint:deprecation for details. >> >> real 0m28.341s >> user 1m17.194s >> sys 0m1.886s >> >> >> As you can see there is a significant regression from JDK 7 to JDK 8 >> which was caused by >> >> https://bugs.openjdk.java.net/browse/JDK-8043253 >> >> (some stack trace analysis revealed the familiar pattern). This has also >> been fixed in JDK 8u20 (as stated in the bug evaluation). >> >> So, while JDK 8u20/9 is slower than JDK 7 (at least on my machine), the >> numbers are more or less in the same ballpark and the huge regression that >> was visible in earlier JDK 8 releases has now been fixed. >> >> If you are still experiencing the problem - can you please also submit >> the specific compiler versions you are using in your benchmark? >> >> Maurizio >> >> >> >> On 29/04/15 10:29, Maurizio Cimadamore wrote: >> >> Hi Jacob, >> Stay assured - as we'll definitively look into this issue (I see it's >> already assigned to one of my colleagues). >> >> What I can say (w/o looking too much at the attached artifacts) is that >> in general, javac has no issue with compiling a lot of sources at once; at >> one point the build system was structured in such a way that all the JDK >> classes were compiled at once - and that (which is way more than your 187 >> sources - i.e. at least 10x that) took less than 20 seconds. SO there must >> some specific pattern triggering that issue. >> >> Given that you say you have 187 input sources and 48K output classes, I'd >> say you are using inner classes a lot. I wonder if you are hitting this: >> >> https://bugs.openjdk.java.net/browse/JDK-8000316 >> >> Maurizio >> >> On 24/04/15 09:49, Jacob Wieland wrote: >> >> Hello Folks, >> >> >> I still have the open problem >> https://bugs.openjdk.java.net/browse/JDK-8039262 that the javac >> performance has degraded significantly from 1.6 to 1.7 (and even worse to >> 1.8) in the 64bit versions. Since in our context, we are dealing with a lot >> of generated source1.4 Java input (either split into very large files with >> inner classes or big packages with lots of smaller classes), compiler >> performance is critical for our tool and this degradation forces us to >> continue recommending to our customers to use Java 1.6 for large projects >> (as is the norm) as 1.7 and 1.8 are pretty much unusable in this respect. >> >> Is anyone still working on this issue or is such significant >> performance degradation not a serious issue? >> >> My observations so far are: >> - it gets worse the more class files are being compiled/the more files >> reside in the source path >> - cpu usage goes through the roof over all available kernels >> - memory usage is much higher >> - garbage collection seems to be much more active >> >> Using -proc:none alleviates the problem slightly for the 1.7 version, >> but not for 1.8 (last we tested with jdk1.8.0_31) where the performance >> difference is a factor 5 or more! >> >> I can understand that a more advanced compiler has capabilities that a >> previous version does not have and thus sometimes has >> to do more work. But, it should still be possible (especially if given >> the -source 1.4 or -source 1.5 option as we do) to optimize it in such a >> way that unnecessary checks for generics, overriding methods, closures, >> annotations and other newer features can be turned off (if they are to >> blame, which I actually doubt from my observations). >> >> I would really appreciate your help in this regard and I think everyone >> would benefit from any bugfix you can offer for this. >> >> BR, Jacob Wieland >> >> -- >> -- >> ------------------------------ >> ------------------------------------------- >> Dr. Jacob Wieland >> Senior Software Engineer >> >> Testing Technologies IST GmbH >> Michaelkirchstra?e 17/18 >> 10179 Berlin, Germany >> >> Phone +49 30 726 19 19 34 Email wieland at testingtech.com >> >> Fax +49 30 726 19 19 20 Internet www.testingtech.com >> >> --------------------------------------------------------------------------------------------------------------- >> >> >> ----------------------------------------------------------------------------------------------------------------- >> >> UPCOMING EVENTS >> >> SUBMIT YOUR TOPIC for the UCAAT 2015 >> Deadline: May 30, 2015ucaat.etsi.org/2015/CallForPresentations.html >> >> Apr 21-23, 2015 | SAE Conference & Exhibition >> Detroit, Michigan, USAwww.sae.org/congress/ >> >> Apr 28-30, 2015 | iqnite >> Dusseldorf, Germanywww.iqnite-conferences.com/de/index.aspx >> >> ----------------------------------------------------------------------------------------------------------------- >> Gesch?ftsf?hrung: Theofanis Vassiliou-Gioles, Stephan Pietsch, Pete >> Nicholson Handelsregister HRB 77805 B, Amtsgericht Charlottenburg Ust ID >> Nr.: DE 813 143 070 This email may contain confidential and privileged >> material for the sole use of the intended recipient. Any review, use, >> distribution or disclosure by others is strictly prohibited. If you are not >> the intended recipient (or authorized to receive for the recipient), please >> contact the sender by reply email and delete all copies of this message. >> >> >> >> > > > -- > -- > ------------------------------ > ------------------------------------------- > Dr. Jacob Wieland > Senior Software Engineer > > Testing Technologies IST GmbH > Michaelkirchstra?e 17/18 > 10179 Berlin, Germany > > Phone +49 30 726 19 19 34 Email wieland at testingtech.com > > Fax +49 30 726 19 19 20 Internet www.testingtech.com > > --------------------------------------------------------------------------------------------------------------- > > > ----------------------------------------------------------------------------------------------------------------- > > UPCOMING EVENTS > > SUBMIT YOUR TOPIC for the UCAAT 2015 > Deadline: May 30, 2015ucaat.etsi.org/2015/CallForPresentations.html > > Apr 21-23, 2015 | SAE Conference & Exhibition > Detroit, Michigan, USAwww.sae.org/congress/ > > Apr 28-30, 2015 | iqnite > Dusseldorf, Germanywww.iqnite-conferences.com/de/index.aspx > > ----------------------------------------------------------------------------------------------------------------- > Gesch?ftsf?hrung: Theofanis Vassiliou-Gioles, Stephan Pietsch, Pete > Nicholson Handelsregister HRB 77805 B, Amtsgericht Charlottenburg Ust ID > Nr.: DE 813 143 070 This email may contain confidential and privileged > material for the sole use of the intended recipient. Any review, use, > distribution or disclosure by others is strictly prohibited. If you are not > the intended recipient (or authorized to receive for the recipient), please > contact the sender by reply email and delete all copies of this message. > > > -- -- ------------------------------ ------------------------------------------- Dr. Jacob Wieland Senior Software Engineer Testing Technologies IST GmbH Michaelkirchstra?e 17/18 10179 Berlin, Germany Phone +49 30 726 19 19 34 Email wieland at testingtech.com Fax +49 30 726 19 19 20 Internet www.testingtech.com --------------------------------------------------------------------------------------------------------------- ----------------------------------------------------------------------------------------------------------------- UPCOMING EVENTS SUBMIT YOUR TOPIC for the UCAAT 2015 Deadline: May 30, 2015ucaat.etsi.org/2015/CallForPresentations.html Apr 21-23, 2015 | SAE Conference & Exhibition Detroit, Michigan, USAwww.sae.org/congress/ Apr 28-30, 2015 | iqnite Dusseldorf, Germanywww.iqnite-conferences.com/de/index.aspx ----------------------------------------------------------------------------------------------------------------- Gesch?ftsf?hrung: Theofanis Vassiliou-Gioles, Stephan Pietsch, Pete Nicholson Handelsregister HRB 77805 B, Amtsgericht Charlottenburg Ust ID Nr.: DE 813 143 070 This email may contain confidential and privileged material for the sole use of the intended recipient. Any review, use, distribution or disclosure by others is strictly prohibited. If you are not the intended recipient (or authorized to receive for the recipient), please contact the sender by reply email and delete all copies of this message. -------------- next part -------------- An HTML attachment was scrubbed... URL: From maurizio.cimadamore at oracle.com Wed Apr 29 15:08:09 2015 From: maurizio.cimadamore at oracle.com (Maurizio Cimadamore) Date: Wed, 29 Apr 2015 16:08:09 +0100 Subject: Java Performance Degradation in JDK7 and JDK8 In-Reply-To: References: <5540A479.7010401@oracle.com> <5540ABCD.3080708@oracle.com> <5540D8B0.3070604@oracle.com> Message-ID: <5540F3D9.3000509@oracle.com> On 29/04/15 15:06, Jacob Wieland wrote: > I have to admit, the reproducer is only a small part of the actual > generated code. In my preliminary tests, it already sufficed to show a > difference (also, the smaller code still worked with 32 bit while the > whole code runs out of memory in all 32 bit versions which makes > comparison much harder ;-) and which is why I need the 64 bit > versions). If I test with the complete code which is much bigger, the > results are as follows: > > jdk1.6u45(64bit) 2GB MaxMem - 1:30 minutes > jdk1.7u75(64bit) 2GB MaxMem - > 6 min > jdk1.8u31(64bit) 1GB MaxMem - > 15 min > jdk1.8u31(64bit) 2GB MaxMem - > 10 min > jdk1.8u31(64bit) 4GB MaxMem - 2:20 min (-source/-target 6 does not > seem to have any effect) > > So, if you throw insane (in comparison with 1.6) amounts of memory at > 1.7/1.8, it is only about a third as slow, but this still is > unacceptable. I actually think it has to do with parallellization and > garbage collection. Note that the javac code base is single threaded - there's absolutely no parallelization going on; so what you see is the GC threads, which seems to point to an higher-then-before memory activity. I'll try your reproducer again by reducing the amount of memory and see if I get somewhere. Maurizio > > 2015-04-29 15:12 GMT+02:00 Maurizio Cimadamore > >: > > On 29/04/15 12:44, Jacob Wieland wrote: >> Hello Maurizio, >> >> are you sure that you used the 64bit versions of javac? I could >> only observe the behavior with these. > Yep I'm on a Ubuntu x64 machine. It's actually pretty standard > hardware too - i.e. intel i5 (two cores, but OS sees 4 because of > hyper-threading). >> Also, I just tried with jdk8u31-64b and it takes AGES (still >> running after 17 minutes where the jdk6 was done after 2), top >> shows 4GB VIRT memory use and 350 % load (on a 4core processor). > Maybe the reproducer you sent was incorrect? > > Maurizio > >> >> So, I don't think it was that problem. >> >> 2015-04-29 12:00 GMT+02:00 Maurizio Cimadamore >> > >: >> >> These are the numbers I'm getting: >> >> JDK 9 (b42) >> >> Note: generated_ttcn/TTCN3_CommonDefs.java uses or overrides >> a deprecated API. >> Note: Recompile with -Xlint:deprecation for details. >> >> real 0m46.306s >> user 2m17.489s >> sys 0m2.166s >> >> JDK 8 (GA) >> >> Note: generated_ttcn/TTCN3_CommonDefs.java uses or overrides >> a deprecated API. >> Note: Recompile with -Xlint:deprecation for details. >> >> real 6m58.748s >> user 8m43.546s >> sys 0m2.132s >> >> JDK 7 (1.7.0_79) >> >> Note: generated_ttcn/TTCN3_CommonDefs.java uses or overrides >> a deprecated API. >> Note: Recompile with -Xlint:deprecation for details. >> >> real 0m28.341s >> user 1m17.194s >> sys 0m1.886s >> >> >> As you can see there is a significant regression from JDK 7 >> to JDK 8 which was caused by >> >> https://bugs.openjdk.java.net/browse/JDK-8043253 >> >> (some stack trace analysis revealed the familiar pattern). >> This has also been fixed in JDK 8u20 (as stated in the bug >> evaluation). >> >> So, while JDK 8u20/9 is slower than JDK 7 (at least on my >> machine), the numbers are more or less in the same ballpark >> and the huge regression that was visible in earlier JDK 8 >> releases has now been fixed. >> >> If you are still experiencing the problem - can you please >> also submit the specific compiler versions you are using in >> your benchmark? >> >> Maurizio >> >> >> >> On 29/04/15 10:29, Maurizio Cimadamore wrote: >>> Hi Jacob, >>> Stay assured - as we'll definitively look into this issue (I >>> see it's already assigned to one of my colleagues). >>> >>> What I can say (w/o looking too much at the attached >>> artifacts) is that in general, javac has no issue with >>> compiling a lot of sources at once; at one point the build >>> system was structured in such a way that all the JDK classes >>> were compiled at once - and that (which is way more than >>> your 187 sources - i.e. at least 10x that) took less than 20 >>> seconds. SO there must some specific pattern triggering that >>> issue. >>> >>> Given that you say you have 187 input sources and 48K output >>> classes, I'd say you are using inner classes a lot. I wonder >>> if you are hitting this: >>> >>> https://bugs.openjdk.java.net/browse/JDK-8000316 >>> >>> Maurizio >>> >>> On 24/04/15 09:49, Jacob Wieland wrote: >>>> Hello Folks, >>>> >>>> >>>> I still have the open problem >>>> https://bugs.openjdk.java.net/browse/JDK-8039262 that the >>>> javac performance has degraded significantly from 1.6 to >>>> 1.7 (and even worse to 1.8) in the 64bit versions. Since in >>>> our context, we are dealing with a lot of generated >>>> source1.4 Java input (either split into very large files >>>> with inner classes or big packages with lots of smaller >>>> classes), compiler performance is critical for our tool and >>>> this degradation forces us to continue recommending to our >>>> customers to use Java 1.6 for large projects (as is the >>>> norm) as 1.7 and 1.8 are pretty much unusable in this respect. >>>> >>>> Is anyone still working on this issue or is such >>>> significant performance degradation not a serious issue? >>>> >>>> My observations so far are: >>>> - it gets worse the more class files are being compiled/the >>>> more files reside in the source path >>>> - cpu usage goes through the roof over all available kernels >>>> - memory usage is much higher >>>> - garbage collection seems to be much more active >>>> >>>> Using -proc:none alleviates the problem slightly for the >>>> 1.7 version, but not for 1.8 (last we tested with >>>> jdk1.8.0_31) where the performance difference is a factor 5 >>>> or more! >>>> >>>> I can understand that a more advanced compiler has >>>> capabilities that a previous version does not have and thus >>>> sometimes has >>>> to do more work. But, it should still be possible >>>> (especially if given the -source 1.4 or -source 1.5 option >>>> as we do) to optimize it in such a way that unnecessary >>>> checks for generics, overriding methods, closures, >>>> annotations and other newer features can be turned off (if >>>> they are to blame, which I actually doubt from my >>>> observations). >>>> >>>> I would really appreciate your help in this regard and I >>>> think everyone would benefit from any bugfix you can offer >>>> for this. >>>> >>>> BR, Jacob Wieland >>>> >>>> -- >>>> -- >>>> ------------------------------ >>>> ------------------------------------------- >>>> Dr. Jacob Wieland >>>> Senior Software Engineer >>>> >>>> Testing Technologies IST GmbH >>>> Michaelkirchstra?e 17/18 >>>> 10179 Berlin, Germany >>>> >>>> Phone +49 30 726 19 19 34 Email >>>> wieland at testingtech.com >>>> Fax +49 30 726 19 19 20 Internet www.testingtech.com >>>> >>>> --------------------------------------------------------------------------------------------------------------- >>>> >>>> ----------------------------------------------------------------------------------------------------------------- >>>> >>>> UPCOMING EVENTS >>>> >>>> SUBMIT YOUR TOPIC for the UCAAT 2015 >>>> Deadline: May 30, 2015 >>>> ucaat.etsi.org/2015/CallForPresentations.html >>>> >>>> Apr 21-23, 2015 | SAE Conference & Exhibition >>>> Detroit, Michigan, USA >>>> www.sae.org/congress/ >>>> >>>> Apr 28-30, 2015 | iqnite >>>> Dusseldorf, Germany >>>> www.iqnite-conferences.com/de/index.aspx >>>> ----------------------------------------------------------------------------------------------------------------- >>>> Gesch?ftsf?hrung: Theofanis Vassiliou-Gioles, Stephan >>>> Pietsch, Pete Nicholson Handelsregister HRB 77805 B, >>>> Amtsgericht Charlottenburg Ust ID Nr.: DE 813 143 070 This >>>> email may contain confidential and privileged material for >>>> the sole use of the intended recipient. Any review, use, >>>> distribution or disclosure by others is strictly >>>> prohibited. If you are not the intended recipient (or >>>> authorized to receive for the recipient), please contact >>>> the sender by reply email and delete all copies of this >>>> message. >>> >> >> >> >> >> -- >> -- >> ------------------------------ >> ------------------------------------------- >> Dr. Jacob Wieland >> Senior Software Engineer >> >> Testing Technologies IST GmbH >> Michaelkirchstra?e 17/18 >> 10179 Berlin, Germany >> >> Phone +49 30 726 19 19 34 Email wieland at testingtech.com >> >> Fax +49 30 726 19 19 20 Internet www.testingtech.com >> >> --------------------------------------------------------------------------------------------------------------- >> >> ----------------------------------------------------------------------------------------------------------------- >> >> UPCOMING EVENTS >> >> SUBMIT YOUR TOPIC for the UCAAT 2015 >> Deadline: May 30, 2015 >> ucaat.etsi.org/2015/CallForPresentations.html >> >> Apr 21-23, 2015 | SAE Conference & Exhibition >> Detroit, Michigan, USA >> www.sae.org/congress/ >> >> Apr 28-30, 2015 | iqnite >> Dusseldorf, Germany >> www.iqnite-conferences.com/de/index.aspx >> ----------------------------------------------------------------------------------------------------------------- >> Gesch?ftsf?hrung: Theofanis Vassiliou-Gioles, Stephan Pietsch, >> Pete Nicholson Handelsregister HRB 77805 B, Amtsgericht >> Charlottenburg Ust ID Nr.: DE 813 143 070 This email may contain >> confidential and privileged material for the sole use of the >> intended recipient. Any review, use, distribution or disclosure >> by others is strictly prohibited. If you are not the intended >> recipient (or authorized to receive for the recipient), please >> contact the sender by reply email and delete all copies of this >> message. > > > > > -- > -- > ------------------------------ > ------------------------------------------- > Dr. Jacob Wieland > Senior Software Engineer > > Testing Technologies IST GmbH > Michaelkirchstra?e 17/18 > 10179 Berlin, Germany > > Phone +49 30 726 19 19 34 Email wieland at testingtech.com > > Fax +49 30 726 19 19 20 Internet www.testingtech.com > > --------------------------------------------------------------------------------------------------------------- > > ----------------------------------------------------------------------------------------------------------------- > > UPCOMING EVENTS > > SUBMIT YOUR TOPIC for the UCAAT 2015 > Deadline: May 30, 2015 > ucaat.etsi.org/2015/CallForPresentations.html > > Apr 21-23, 2015 | SAE Conference & Exhibition > Detroit, Michigan, USA > www.sae.org/congress/ > > Apr 28-30, 2015 | iqnite > Dusseldorf, Germany > www.iqnite-conferences.com/de/index.aspx > ----------------------------------------------------------------------------------------------------------------- > Gesch?ftsf?hrung: Theofanis Vassiliou-Gioles, Stephan Pietsch, Pete > Nicholson Handelsregister HRB 77805 B, Amtsgericht Charlottenburg Ust > ID Nr.: DE 813 143 070 This email may contain confidential and > privileged material for the sole use of the intended recipient. Any > review, use, distribution or disclosure by others is strictly > prohibited. If you are not the intended recipient (or authorized to > receive for the recipient), please contact the sender by reply email > and delete all copies of this message. -------------- next part -------------- An HTML attachment was scrubbed... URL: From jan.lahoda at oracle.com Wed Apr 29 16:01:22 2015 From: jan.lahoda at oracle.com (Jan Lahoda) Date: Wed, 29 Apr 2015 18:01:22 +0200 Subject: Java Performance Degradation in JDK7 and JDK8 In-Reply-To: References: <5540A479.7010401@oracle.com> <5540ABCD.3080708@oracle.com> <5540D8B0.3070604@oracle.com> Message-ID: <55410052.5080205@oracle.com> On 29.4.2015 16:06, Jacob Wieland wrote: > I have to admit, the reproducer is only a small part of the actual > generated code. In my preliminary tests, it already sufficed to show a > difference (also, the smaller code still worked with 32 bit while the > whole code runs out of memory in all 32 bit versions which makes > comparison much harder ;-) and which is why I need the 64 bit versions). > If I test with the complete code which is much bigger, the results are > as follows: > > jdk1.6u45(64bit) 2GB MaxMem - 1:30 minutes > jdk1.7u75(64bit) 2GB MaxMem - > 6 min > jdk1.8u31(64bit) 1GB MaxMem - > 15 min > jdk1.8u31(64bit) 2GB MaxMem - > 10 min > jdk1.8u31(64bit) 4GB MaxMem - 2:20 min (-source/-target 6 does not seem > to have any effect) > > So, if you throw insane (in comparison with 1.6) amounts of memory at > 1.7/1.8, it is only about a third as slow, but this still is > unacceptable. I actually think it has to do with parallellization and When I was looking at JDK-8039262, a significant contributing factor to the slowdown (with enough memory and -source/-target 6) appeared to be Check.checkOverrideClashes - I believe this does checks that were not properly implemented in 6, contributing to the difference between 6 and 7 (which seems to be particularly visible for this testcase). I was looking at the checks a few times, trying to write them differently/faster while still performing the checks, but was not successful in that yet, unfortunately. Jan > garbage collection. > > 2015-04-29 15:12 GMT+02:00 Maurizio Cimadamore > >: > > On 29/04/15 12:44, Jacob Wieland wrote: >> Hello Maurizio, >> >> are you sure that you used the 64bit versions of javac? I could >> only observe the behavior with these. > Yep I'm on a Ubuntu x64 machine. It's actually pretty standard > hardware too - i.e. intel i5 (two cores, but OS sees 4 because of > hyper-threading). >> Also, I just tried with jdk8u31-64b and it takes AGES (still >> running after 17 minutes where the jdk6 was done after 2), top >> shows 4GB VIRT memory use and 350 % load (on a 4core processor). > Maybe the reproducer you sent was incorrect? > > Maurizio > >> >> So, I don't think it was that problem. >> >> 2015-04-29 12:00 GMT+02:00 Maurizio Cimadamore >> > >: >> >> These are the numbers I'm getting: >> >> JDK 9 (b42) >> >> Note: generated_ttcn/TTCN3_CommonDefs.java uses or overrides a >> deprecated API. >> Note: Recompile with -Xlint:deprecation for details. >> >> real 0m46.306s >> user 2m17.489s >> sys 0m2.166s >> >> JDK 8 (GA) >> >> Note: generated_ttcn/TTCN3_CommonDefs.java uses or overrides a >> deprecated API. >> Note: Recompile with -Xlint:deprecation for details. >> >> real 6m58.748s >> user 8m43.546s >> sys 0m2.132s >> >> JDK 7 (1.7.0_79) >> >> Note: generated_ttcn/TTCN3_CommonDefs.java uses or overrides a >> deprecated API. >> Note: Recompile with -Xlint:deprecation for details. >> >> real 0m28.341s >> user 1m17.194s >> sys 0m1.886s >> >> >> As you can see there is a significant regression from JDK 7 to >> JDK 8 which was caused by >> >> https://bugs.openjdk.java.net/browse/JDK-8043253 >> >> (some stack trace analysis revealed the familiar pattern). >> This has also been fixed in JDK 8u20 (as stated in the bug >> evaluation). >> >> So, while JDK 8u20/9 is slower than JDK 7 (at least on my >> machine), the numbers are more or less in the same ballpark >> and the huge regression that was visible in earlier JDK 8 >> releases has now been fixed. >> >> If you are still experiencing the problem - can you please >> also submit the specific compiler versions you are using in >> your benchmark? >> >> Maurizio >> >> >> >> On 29/04/15 10:29, Maurizio Cimadamore wrote: >>> Hi Jacob, >>> Stay assured - as we'll definitively look into this issue (I >>> see it's already assigned to one of my colleagues). >>> >>> What I can say (w/o looking too much at the attached >>> artifacts) is that in general, javac has no issue with >>> compiling a lot of sources at once; at one point the build >>> system was structured in such a way that all the JDK classes >>> were compiled at once - and that (which is way more than your >>> 187 sources - i.e. at least 10x that) took less than 20 >>> seconds. SO there must some specific pattern triggering that >>> issue. >>> >>> Given that you say you have 187 input sources and 48K output >>> classes, I'd say you are using inner classes a lot. I wonder >>> if you are hitting this: >>> >>> https://bugs.openjdk.java.net/browse/JDK-8000316 >>> >>> Maurizio >>> >>> On 24/04/15 09:49, Jacob Wieland wrote: >>>> Hello Folks, >>>> >>>> >>>> I still have the open problem >>>> https://bugs.openjdk.java.net/browse/JDK-8039262 that the >>>> javac performance has degraded significantly from 1.6 to 1.7 >>>> (and even worse to 1.8) in the 64bit versions. Since in our >>>> context, we are dealing with a lot of generated source1.4 >>>> Java input (either split into very large files with inner >>>> classes or big packages with lots of smaller classes), >>>> compiler performance is critical for our tool and this >>>> degradation forces us to continue recommending to our >>>> customers to use Java 1.6 for large projects (as is the >>>> norm) as 1.7 and 1.8 are pretty much unusable in this respect. >>>> >>>> Is anyone still working on this issue or is such significant >>>> performance degradation not a serious issue? >>>> >>>> My observations so far are: >>>> - it gets worse the more class files are being compiled/the >>>> more files reside in the source path >>>> - cpu usage goes through the roof over all available kernels >>>> - memory usage is much higher >>>> - garbage collection seems to be much more active >>>> >>>> Using -proc:none alleviates the problem slightly for the 1.7 >>>> version, but not for 1.8 (last we tested with >>>> jdk1.8.0_31) where the performance difference is a factor 5 >>>> or more! >>>> >>>> I can understand that a more advanced compiler has >>>> capabilities that a previous version does not have and thus >>>> sometimes has >>>> to do more work. But, it should still be possible >>>> (especially if given the -source 1.4 or -source 1.5 option >>>> as we do) to optimize it in such a way that unnecessary >>>> checks for generics, overriding methods, closures, >>>> annotations and other newer features can be turned off (if >>>> they are to blame, which I actually doubt from my observations). >>>> >>>> I would really appreciate your help in this regard and I >>>> think everyone would benefit from any bugfix you can offer >>>> for this. >>>> >>>> BR, Jacob Wieland >>>> >>>> -- >>>> -- >>>> ------------------------------ >>>> ------------------------------------------- >>>> Dr. Jacob Wieland >>>> Senior Software Engineer >>>> >>>> Testing Technologies IST GmbH >>>> Michaelkirchstra?e 17/18 >>>> 10179 Berlin, Germany >>>> >>>> Phone +49 30 726 19 19 34 Email >>>> wieland at testingtech.com >>>> Fax +49 30 726 19 19 20 Internet www.testingtech.com >>>> >>>> --------------------------------------------------------------------------------------------------------------- >>>> >>>> ----------------------------------------------------------------------------------------------------------------- >>>> >>>> UPCOMING EVENTS >>>> >>>> SUBMIT YOUR TOPIC for the UCAAT 2015 >>>> Deadline: May 30, 2015 >>>> ucaat.etsi.org/2015/CallForPresentations.html >>>> >>>> Apr 21-23, 2015 | SAE Conference & Exhibition >>>> Detroit, Michigan, USA >>>> www.sae.org/congress/ >>>> >>>> Apr 28-30, 2015 | iqnite >>>> Dusseldorf, Germany >>>> www.iqnite-conferences.com/de/index.aspx >>>> ----------------------------------------------------------------------------------------------------------------- >>>> Gesch?ftsf?hrung: Theofanis Vassiliou-Gioles, Stephan >>>> Pietsch, Pete Nicholson Handelsregister HRB 77805 B, >>>> Amtsgericht Charlottenburg Ust ID Nr.: DE 813 143 070 This >>>> email may contain confidential and privileged material for >>>> the sole use of the intended recipient. Any review, use, >>>> distribution or disclosure by others is strictly prohibited. >>>> If you are not the intended recipient (or authorized to >>>> receive for the recipient), please contact the sender by >>>> reply email and delete all copies of this message. >>> >> >> >> >> >> -- >> -- >> ------------------------------ >> ------------------------------------------- >> Dr. Jacob Wieland >> Senior Software Engineer >> >> Testing Technologies IST GmbH >> Michaelkirchstra?e 17/18 >> 10179 Berlin, Germany >> >> Phone +49 30 726 19 19 34 Email wieland at testingtech.com >> >> Fax +49 30 726 19 19 20 Internet www.testingtech.com >> >> --------------------------------------------------------------------------------------------------------------- >> >> ----------------------------------------------------------------------------------------------------------------- >> >> UPCOMING EVENTS >> >> SUBMIT YOUR TOPIC for the UCAAT 2015 >> Deadline: May 30, 2015 >> ucaat.etsi.org/2015/CallForPresentations.html >> >> Apr 21-23, 2015 | SAE Conference & Exhibition >> Detroit, Michigan, USA >> www.sae.org/congress/ >> >> Apr 28-30, 2015 | iqnite >> Dusseldorf, Germany >> www.iqnite-conferences.com/de/index.aspx >> ----------------------------------------------------------------------------------------------------------------- >> Gesch?ftsf?hrung: Theofanis Vassiliou-Gioles, Stephan Pietsch, >> Pete Nicholson Handelsregister HRB 77805 B, Amtsgericht >> Charlottenburg Ust ID Nr.: DE 813 143 070 This email may contain >> confidential and privileged material for the sole use of the >> intended recipient. Any review, use, distribution or disclosure by >> others is strictly prohibited. If you are not the intended >> recipient (or authorized to receive for the recipient), please >> contact the sender by reply email and delete all copies of this >> message. > > > > > -- > -- > ------------------------------ > ------------------------------------------- > Dr. Jacob Wieland > Senior Software Engineer > > Testing Technologies IST GmbH > Michaelkirchstra?e 17/18 > 10179 Berlin, Germany > > Phone +49 30 726 19 19 34 Email wieland at testingtech.com > > Fax +49 30 726 19 19 20 Internet www.testingtech.com > > --------------------------------------------------------------------------------------------------------------- > > ----------------------------------------------------------------------------------------------------------------- > > UPCOMING EVENTS > > SUBMIT YOUR TOPIC for the UCAAT 2015 > Deadline: May 30, 2015 > ucaat.etsi.org/2015/CallForPresentations.html > > Apr 21-23, 2015 | SAE Conference & Exhibition > Detroit, Michigan, USA > www.sae.org/congress/ > > Apr 28-30, 2015 | iqnite > Dusseldorf, Germany > www.iqnite-conferences.com/de/index.aspx > > > ----------------------------------------------------------------------------------------------------------------- > Gesch?ftsf?hrung: Theofanis Vassiliou-Gioles, Stephan Pietsch, Pete > Nicholson Handelsregister HRB 77805 B, Amtsgericht Charlottenburg Ust ID > Nr.: DE 813 143 070 This email may contain confidential and privileged > material for the sole use of the intended recipient. Any review, use, > distribution or disclosure by others is strictly prohibited. If you are > not the intended recipient (or authorized to receive for the recipient), > please contact the sender by reply email and delete all copies of this > message. From maurizio.cimadamore at oracle.com Wed Apr 29 16:43:06 2015 From: maurizio.cimadamore at oracle.com (Maurizio Cimadamore) Date: Wed, 29 Apr 2015 17:43:06 +0100 Subject: Java Performance Degradation in JDK7 and JDK8 In-Reply-To: <55410052.5080205@oracle.com> References: <5540A479.7010401@oracle.com> <5540ABCD.3080708@oracle.com> <5540D8B0.3070604@oracle.com> <55410052.5080205@oracle.com> Message-ID: <55410A1A.1090104@oracle.com> On 29/04/15 17:01, Jan Lahoda wrote: > On 29.4.2015 16:06, Jacob Wieland wrote: >> I have to admit, the reproducer is only a small part of the actual >> generated code. In my preliminary tests, it already sufficed to show a >> difference (also, the smaller code still worked with 32 bit while the >> whole code runs out of memory in all 32 bit versions which makes >> comparison much harder ;-) and which is why I need the 64 bit versions). >> If I test with the complete code which is much bigger, the results are >> as follows: >> >> jdk1.6u45(64bit) 2GB MaxMem - 1:30 minutes >> jdk1.7u75(64bit) 2GB MaxMem - > 6 min >> jdk1.8u31(64bit) 1GB MaxMem - > 15 min >> jdk1.8u31(64bit) 2GB MaxMem - > 10 min >> jdk1.8u31(64bit) 4GB MaxMem - 2:20 min (-source/-target 6 does not seem >> to have any effect) >> >> So, if you throw insane (in comparison with 1.6) amounts of memory at >> 1.7/1.8, it is only about a third as slow, but this still is >> unacceptable. I actually think it has to do with parallellization and > > When I was looking at JDK-8039262, a significant contributing factor > to the slowdown (with enough memory and -source/-target 6) appeared to > be Check.checkOverrideClashes - I believe this does checks that were > not properly implemented in 6, contributing to the difference between > 6 and 7 (which seems to be particularly visible for this testcase). I > was looking at the checks a few times, trying to write them > differently/faster while still performing the checks, but was not > successful in that yet, unfortunately. I see the issue now - it is reproducible with the following memory parameter (at least in my machine): -J-Xmx768M This give around 20sec in JDK 6 and 10+ minutes in JDK 8. All the time seems to be spent in desugaring, most specifically in TransTypes.addBridges - it seems like that method calls Types.implementation a lot - so my theory was that the fact that javac consumes more memory, forces the GC to get rid of the cached entries in the implementation/members closure caches (since such entries are deliberately backed up by SoftReferences), which in turn completely trashes performances. I instrumented the code a bit and this is what I found: *) With -Xmx768M Impl cache misses = 3346926 Members cache misses = 1042678 real 7m0.335s user 25m51.517s sys 0m4.947s *) W/o -Xmx768M Impl cache misses = 3346839 Members cache misses = 1042678 real 0m32.377s user 1m25.881s sys 0m2.232s Long story short - cache misses do not play a factor in here - there are some minor differences, but nothing out of the ordinary and defo nothing that would explain a multi-minute slowdown! Any ideas? Maurizio > > Jan > >> garbage collection. >> >> 2015-04-29 15:12 GMT+02:00 Maurizio Cimadamore >> > >: >> >> On 29/04/15 12:44, Jacob Wieland wrote: >>> Hello Maurizio, >>> >>> are you sure that you used the 64bit versions of javac? I could >>> only observe the behavior with these. >> Yep I'm on a Ubuntu x64 machine. It's actually pretty standard >> hardware too - i.e. intel i5 (two cores, but OS sees 4 because of >> hyper-threading). >>> Also, I just tried with jdk8u31-64b and it takes AGES (still >>> running after 17 minutes where the jdk6 was done after 2), top >>> shows 4GB VIRT memory use and 350 % load (on a 4core processor). >> Maybe the reproducer you sent was incorrect? >> >> Maurizio >> >>> >>> So, I don't think it was that problem. >>> >>> 2015-04-29 12:00 GMT+02:00 Maurizio Cimadamore >>> >> >: >>> >>> These are the numbers I'm getting: >>> >>> JDK 9 (b42) >>> >>> Note: generated_ttcn/TTCN3_CommonDefs.java uses or overrides a >>> deprecated API. >>> Note: Recompile with -Xlint:deprecation for details. >>> >>> real 0m46.306s >>> user 2m17.489s >>> sys 0m2.166s >>> >>> JDK 8 (GA) >>> >>> Note: generated_ttcn/TTCN3_CommonDefs.java uses or overrides a >>> deprecated API. >>> Note: Recompile with -Xlint:deprecation for details. >>> >>> real 6m58.748s >>> user 8m43.546s >>> sys 0m2.132s >>> >>> JDK 7 (1.7.0_79) >>> >>> Note: generated_ttcn/TTCN3_CommonDefs.java uses or overrides a >>> deprecated API. >>> Note: Recompile with -Xlint:deprecation for details. >>> >>> real 0m28.341s >>> user 1m17.194s >>> sys 0m1.886s >>> >>> >>> As you can see there is a significant regression from JDK 7 to >>> JDK 8 which was caused by >>> >>> https://bugs.openjdk.java.net/browse/JDK-8043253 >>> >>> (some stack trace analysis revealed the familiar pattern). >>> This has also been fixed in JDK 8u20 (as stated in the bug >>> evaluation). >>> >>> So, while JDK 8u20/9 is slower than JDK 7 (at least on my >>> machine), the numbers are more or less in the same ballpark >>> and the huge regression that was visible in earlier JDK 8 >>> releases has now been fixed. >>> >>> If you are still experiencing the problem - can you please >>> also submit the specific compiler versions you are using in >>> your benchmark? >>> >>> Maurizio >>> >>> >>> >>> On 29/04/15 10:29, Maurizio Cimadamore wrote: >>>> Hi Jacob, >>>> Stay assured - as we'll definitively look into this issue (I >>>> see it's already assigned to one of my colleagues). >>>> >>>> What I can say (w/o looking too much at the attached >>>> artifacts) is that in general, javac has no issue with >>>> compiling a lot of sources at once; at one point the build >>>> system was structured in such a way that all the JDK classes >>>> were compiled at once - and that (which is way more than your >>>> 187 sources - i.e. at least 10x that) took less than 20 >>>> seconds. SO there must some specific pattern triggering that >>>> issue. >>>> >>>> Given that you say you have 187 input sources and 48K output >>>> classes, I'd say you are using inner classes a lot. I wonder >>>> if you are hitting this: >>>> >>>> https://bugs.openjdk.java.net/browse/JDK-8000316 >>>> >>>> Maurizio >>>> >>>> On 24/04/15 09:49, Jacob Wieland wrote: >>>>> Hello Folks, >>>>> >>>>> >>>>> I still have the open problem >>>>> https://bugs.openjdk.java.net/browse/JDK-8039262 that the >>>>> javac performance has degraded significantly from 1.6 to 1.7 >>>>> (and even worse to 1.8) in the 64bit versions. Since in our >>>>> context, we are dealing with a lot of generated source1.4 >>>>> Java input (either split into very large files with inner >>>>> classes or big packages with lots of smaller classes), >>>>> compiler performance is critical for our tool and this >>>>> degradation forces us to continue recommending to our >>>>> customers to use Java 1.6 for large projects (as is the >>>>> norm) as 1.7 and 1.8 are pretty much unusable in this >>>>> respect. >>>>> >>>>> Is anyone still working on this issue or is such significant >>>>> performance degradation not a serious issue? >>>>> >>>>> My observations so far are: >>>>> - it gets worse the more class files are being compiled/the >>>>> more files reside in the source path >>>>> - cpu usage goes through the roof over all available kernels >>>>> - memory usage is much higher >>>>> - garbage collection seems to be much more active >>>>> >>>>> Using -proc:none alleviates the problem slightly for the 1.7 >>>>> version, but not for 1.8 (last we tested with >>>>> jdk1.8.0_31) where the performance difference is a factor 5 >>>>> or more! >>>>> >>>>> I can understand that a more advanced compiler has >>>>> capabilities that a previous version does not have and thus >>>>> sometimes has >>>>> to do more work. But, it should still be possible >>>>> (especially if given the -source 1.4 or -source 1.5 option >>>>> as we do) to optimize it in such a way that unnecessary >>>>> checks for generics, overriding methods, closures, >>>>> annotations and other newer features can be turned off (if >>>>> they are to blame, which I actually doubt from my >>>>> observations). >>>>> >>>>> I would really appreciate your help in this regard and I >>>>> think everyone would benefit from any bugfix you can offer >>>>> for this. >>>>> >>>>> BR, Jacob Wieland >>>>> >>>>> -- >>>>> -- >>>>> ------------------------------ >>>>> ------------------------------------------- >>>>> Dr. Jacob Wieland >>>>> Senior Software Engineer >>>>> >>>>> Testing Technologies IST GmbH >>>>> Michaelkirchstra?e 17/18 >>>>> 10179 Berlin, Germany >>>>> >>>>> Phone +49 30 726 19 19 34 Email >>>>> wieland at testingtech.com >>>>> Fax +49 30 726 19 19 20 Internet www.testingtech.com >>>>> >>>>> --------------------------------------------------------------------------------------------------------------- >>>>> >>>>> ----------------------------------------------------------------------------------------------------------------- >>>>> >>>>> UPCOMING EVENTS >>>>> >>>>> SUBMIT YOUR TOPIC for the UCAAT 2015 >>>>> Deadline: May 30, 2015 >>>>> ucaat.etsi.org/2015/CallForPresentations.html >>>>> >>>>> >>>>> Apr 21-23, 2015 | SAE Conference & Exhibition >>>>> Detroit, Michigan, USA >>>>> www.sae.org/congress/ >>>>> >>>>> Apr 28-30, 2015 | iqnite >>>>> Dusseldorf, Germany >>>>> www.iqnite-conferences.com/de/index.aspx >>>>> >>>>> ----------------------------------------------------------------------------------------------------------------- >>>>> Gesch?ftsf?hrung: Theofanis Vassiliou-Gioles, Stephan >>>>> Pietsch, Pete Nicholson Handelsregister HRB 77805 B, >>>>> Amtsgericht Charlottenburg Ust ID Nr.: DE 813 143 070 This >>>>> email may contain confidential and privileged material for >>>>> the sole use of the intended recipient. Any review, use, >>>>> distribution or disclosure by others is strictly prohibited. >>>>> If you are not the intended recipient (or authorized to >>>>> receive for the recipient), please contact the sender by >>>>> reply email and delete all copies of this message. >>>> >>> >>> >>> >>> >>> -- >>> -- >>> ------------------------------ >>> ------------------------------------------- >>> Dr. Jacob Wieland >>> Senior Software Engineer >>> >>> Testing Technologies IST GmbH >>> Michaelkirchstra?e 17/18 >>> 10179 Berlin, Germany >>> >>> Phone +49 30 726 19 19 34 Email wieland at testingtech.com >>> >>> Fax +49 30 726 19 19 20 Internet www.testingtech.com >>> >>> --------------------------------------------------------------------------------------------------------------- >>> >>> ----------------------------------------------------------------------------------------------------------------- >>> >>> UPCOMING EVENTS >>> >>> SUBMIT YOUR TOPIC for the UCAAT 2015 >>> Deadline: May 30, 2015 >>> ucaat.etsi.org/2015/CallForPresentations.html >>> >>> >>> Apr 21-23, 2015 | SAE Conference & Exhibition >>> Detroit, Michigan, USA >>> www.sae.org/congress/ >>> >>> Apr 28-30, 2015 | iqnite >>> Dusseldorf, Germany >>> www.iqnite-conferences.com/de/index.aspx >>> >>> ----------------------------------------------------------------------------------------------------------------- >>> Gesch?ftsf?hrung: Theofanis Vassiliou-Gioles, Stephan Pietsch, >>> Pete Nicholson Handelsregister HRB 77805 B, Amtsgericht >>> Charlottenburg Ust ID Nr.: DE 813 143 070 This email may contain >>> confidential and privileged material for the sole use of the >>> intended recipient. Any review, use, distribution or disclosure by >>> others is strictly prohibited. If you are not the intended >>> recipient (or authorized to receive for the recipient), please >>> contact the sender by reply email and delete all copies of this >>> message. >> >> >> >> >> -- >> -- >> ------------------------------ >> ------------------------------------------- >> Dr. Jacob Wieland >> Senior Software Engineer >> >> Testing Technologies IST GmbH >> Michaelkirchstra?e 17/18 >> 10179 Berlin, Germany >> >> Phone +49 30 726 19 19 34 Email wieland at testingtech.com >> >> Fax +49 30 726 19 19 20 Internet www.testingtech.com >> >> --------------------------------------------------------------------------------------------------------------- >> >> >> ----------------------------------------------------------------------------------------------------------------- >> >> >> UPCOMING EVENTS >> >> SUBMIT YOUR TOPIC for the UCAAT 2015 >> Deadline: May 30, 2015 >> ucaat.etsi.org/2015/CallForPresentations.html >> >> >> Apr 21-23, 2015 | SAE Conference & Exhibition >> Detroit, Michigan, USA >> www.sae.org/congress/ >> >> Apr 28-30, 2015 | iqnite >> Dusseldorf, Germany >> www.iqnite-conferences.com/de/index.aspx >> >> >> >> ----------------------------------------------------------------------------------------------------------------- >> >> Gesch?ftsf?hrung: Theofanis Vassiliou-Gioles, Stephan Pietsch, Pete >> Nicholson Handelsregister HRB 77805 B, Amtsgericht Charlottenburg Ust ID >> Nr.: DE 813 143 070 This email may contain confidential and privileged >> material for the sole use of the intended recipient. Any review, use, >> distribution or disclosure by others is strictly prohibited. If you are >> not the intended recipient (or authorized to receive for the recipient), >> please contact the sender by reply email and delete all copies of this >> message. From vicente.romero at oracle.com Wed Apr 29 16:51:28 2015 From: vicente.romero at oracle.com (Vicente-Arturo Romero-Zaldivar) Date: Wed, 29 Apr 2015 09:51:28 -0700 Subject: Java Performance Degradation in JDK7 and JDK8 In-Reply-To: <55410A1A.1090104@oracle.com> References: <5540A479.7010401@oracle.com> <5540ABCD.3080708@oracle.com> <5540D8B0.3070604@oracle.com> <55410052.5080205@oracle.com> <55410A1A.1090104@oracle.com> Message-ID: <55410C10.9010108@oracle.com> On 04/29/2015 09:43 AM, Maurizio Cimadamore wrote: > > > On 29/04/15 17:01, Jan Lahoda wrote: >> On 29.4.2015 16:06, Jacob Wieland wrote: >>> I have to admit, the reproducer is only a small part of the actual >>> generated code. In my preliminary tests, it already sufficed to show a >>> difference (also, the smaller code still worked with 32 bit while the >>> whole code runs out of memory in all 32 bit versions which makes >>> comparison much harder ;-) and which is why I need the 64 bit >>> versions). >>> If I test with the complete code which is much bigger, the results are >>> as follows: >>> >>> jdk1.6u45(64bit) 2GB MaxMem - 1:30 minutes >>> jdk1.7u75(64bit) 2GB MaxMem - > 6 min >>> jdk1.8u31(64bit) 1GB MaxMem - > 15 min >>> jdk1.8u31(64bit) 2GB MaxMem - > 10 min >>> jdk1.8u31(64bit) 4GB MaxMem - 2:20 min (-source/-target 6 does not seem >>> to have any effect) >>> >>> So, if you throw insane (in comparison with 1.6) amounts of memory at >>> 1.7/1.8, it is only about a third as slow, but this still is >>> unacceptable. I actually think it has to do with parallellization and >> >> When I was looking at JDK-8039262, a significant contributing factor >> to the slowdown (with enough memory and -source/-target 6) appeared >> to be Check.checkOverrideClashes - I believe this does checks that >> were not properly implemented in 6, contributing to the difference >> between 6 and 7 (which seems to be particularly visible for this >> testcase). I was looking at the checks a few times, trying to write >> them differently/faster while still performing the checks, but was >> not successful in that yet, unfortunately. > I see the issue now - it is reproducible with the following memory > parameter (at least in my machine): > > -J-Xmx768M > > This give around 20sec in JDK 6 and 10+ minutes in JDK 8. > > All the time seems to be spent in desugaring, most specifically in > TransTypes.addBridges - it seems like that method calls > Types.implementation a lot - so my theory was that the fact that javac > consumes more memory, forces the GC to get rid of the cached entries > in the implementation/members closure caches (since such entries are > deliberately backed up by SoftReferences), which in turn completely > trashes performances. I instrumented the code a bit and this is what I > found: > > *) With -Xmx768M > > Impl cache misses = 3346926 > Members cache misses = 1042678 > > real 7m0.335s > user 25m51.517s > sys 0m4.947s > > > *) W/o -Xmx768M > > Impl cache misses = 3346839 > Members cache misses = 1042678 > > real 0m32.377s > user 1m25.881s > sys 0m2.232s > > Long story short - cache misses do not play a factor in here - there > are some minor differences, but nothing out of the ordinary and defo > nothing that would explain a multi-minute slowdown! Any ideas? use flight recorder? Vicente > > Maurizio >> >> Jan >> >>> garbage collection. >>> >>> 2015-04-29 15:12 GMT+02:00 Maurizio Cimadamore >>> >> >: >>> >>> On 29/04/15 12:44, Jacob Wieland wrote: >>>> Hello Maurizio, >>>> >>>> are you sure that you used the 64bit versions of javac? I could >>>> only observe the behavior with these. >>> Yep I'm on a Ubuntu x64 machine. It's actually pretty standard >>> hardware too - i.e. intel i5 (two cores, but OS sees 4 because of >>> hyper-threading). >>>> Also, I just tried with jdk8u31-64b and it takes AGES (still >>>> running after 17 minutes where the jdk6 was done after 2), top >>>> shows 4GB VIRT memory use and 350 % load (on a 4core processor). >>> Maybe the reproducer you sent was incorrect? >>> >>> Maurizio >>> >>>> >>>> So, I don't think it was that problem. >>>> >>>> 2015-04-29 12:00 GMT+02:00 Maurizio Cimadamore >>>> >>> >: >>>> >>>> These are the numbers I'm getting: >>>> >>>> JDK 9 (b42) >>>> >>>> Note: generated_ttcn/TTCN3_CommonDefs.java uses or overrides a >>>> deprecated API. >>>> Note: Recompile with -Xlint:deprecation for details. >>>> >>>> real 0m46.306s >>>> user 2m17.489s >>>> sys 0m2.166s >>>> >>>> JDK 8 (GA) >>>> >>>> Note: generated_ttcn/TTCN3_CommonDefs.java uses or overrides a >>>> deprecated API. >>>> Note: Recompile with -Xlint:deprecation for details. >>>> >>>> real 6m58.748s >>>> user 8m43.546s >>>> sys 0m2.132s >>>> >>>> JDK 7 (1.7.0_79) >>>> >>>> Note: generated_ttcn/TTCN3_CommonDefs.java uses or overrides a >>>> deprecated API. >>>> Note: Recompile with -Xlint:deprecation for details. >>>> >>>> real 0m28.341s >>>> user 1m17.194s >>>> sys 0m1.886s >>>> >>>> >>>> As you can see there is a significant regression from JDK 7 to >>>> JDK 8 which was caused by >>>> >>>> https://bugs.openjdk.java.net/browse/JDK-8043253 >>>> >>>> (some stack trace analysis revealed the familiar pattern). >>>> This has also been fixed in JDK 8u20 (as stated in the bug >>>> evaluation). >>>> >>>> So, while JDK 8u20/9 is slower than JDK 7 (at least on my >>>> machine), the numbers are more or less in the same ballpark >>>> and the huge regression that was visible in earlier JDK 8 >>>> releases has now been fixed. >>>> >>>> If you are still experiencing the problem - can you please >>>> also submit the specific compiler versions you are using in >>>> your benchmark? >>>> >>>> Maurizio >>>> >>>> >>>> >>>> On 29/04/15 10:29, Maurizio Cimadamore wrote: >>>>> Hi Jacob, >>>>> Stay assured - as we'll definitively look into this issue (I >>>>> see it's already assigned to one of my colleagues). >>>>> >>>>> What I can say (w/o looking too much at the attached >>>>> artifacts) is that in general, javac has no issue with >>>>> compiling a lot of sources at once; at one point the build >>>>> system was structured in such a way that all the JDK classes >>>>> were compiled at once - and that (which is way more than your >>>>> 187 sources - i.e. at least 10x that) took less than 20 >>>>> seconds. SO there must some specific pattern triggering that >>>>> issue. >>>>> >>>>> Given that you say you have 187 input sources and 48K output >>>>> classes, I'd say you are using inner classes a lot. I wonder >>>>> if you are hitting this: >>>>> >>>>> https://bugs.openjdk.java.net/browse/JDK-8000316 >>>>> >>>>> Maurizio >>>>> >>>>> On 24/04/15 09:49, Jacob Wieland wrote: >>>>>> Hello Folks, >>>>>> >>>>>> >>>>>> I still have the open problem >>>>>> https://bugs.openjdk.java.net/browse/JDK-8039262 that the >>>>>> javac performance has degraded significantly from 1.6 to 1.7 >>>>>> (and even worse to 1.8) in the 64bit versions. Since in our >>>>>> context, we are dealing with a lot of generated source1.4 >>>>>> Java input (either split into very large files with inner >>>>>> classes or big packages with lots of smaller classes), >>>>>> compiler performance is critical for our tool and this >>>>>> degradation forces us to continue recommending to our >>>>>> customers to use Java 1.6 for large projects (as is the >>>>>> norm) as 1.7 and 1.8 are pretty much unusable in this >>>>>> respect. >>>>>> >>>>>> Is anyone still working on this issue or is such significant >>>>>> performance degradation not a serious issue? >>>>>> >>>>>> My observations so far are: >>>>>> - it gets worse the more class files are being compiled/the >>>>>> more files reside in the source path >>>>>> - cpu usage goes through the roof over all available kernels >>>>>> - memory usage is much higher >>>>>> - garbage collection seems to be much more active >>>>>> >>>>>> Using -proc:none alleviates the problem slightly for the 1.7 >>>>>> version, but not for 1.8 (last we tested with >>>>>> jdk1.8.0_31) where the performance difference is a factor 5 >>>>>> or more! >>>>>> >>>>>> I can understand that a more advanced compiler has >>>>>> capabilities that a previous version does not have and thus >>>>>> sometimes has >>>>>> to do more work. But, it should still be possible >>>>>> (especially if given the -source 1.4 or -source 1.5 option >>>>>> as we do) to optimize it in such a way that unnecessary >>>>>> checks for generics, overriding methods, closures, >>>>>> annotations and other newer features can be turned off (if >>>>>> they are to blame, which I actually doubt from my >>>>>> observations). >>>>>> >>>>>> I would really appreciate your help in this regard and I >>>>>> think everyone would benefit from any bugfix you can offer >>>>>> for this. >>>>>> >>>>>> BR, Jacob Wieland >>>>>> >>>>>> -- >>>>>> -- >>>>>> ------------------------------ >>>>>> ------------------------------------------- >>>>>> Dr. Jacob Wieland >>>>>> Senior Software Engineer >>>>>> >>>>>> Testing Technologies IST GmbH >>>>>> Michaelkirchstra?e 17/18 >>>>>> 10179 Berlin, Germany >>>>>> >>>>>> Phone +49 30 726 19 19 34 Email >>>>>> wieland at testingtech.com >>>>>> Fax +49 30 726 19 19 20 Internet www.testingtech.com >>>>>> >>>>>> --------------------------------------------------------------------------------------------------------------- >>>>>> >>>>>> >>>>>> ----------------------------------------------------------------------------------------------------------------- >>>>>> >>>>>> >>>>>> UPCOMING EVENTS >>>>>> >>>>>> SUBMIT YOUR TOPIC for the UCAAT 2015 >>>>>> Deadline: May 30, 2015 >>>>>> ucaat.etsi.org/2015/CallForPresentations.html >>>>>> >>>>>> >>>>>> Apr 21-23, 2015 | SAE Conference & Exhibition >>>>>> Detroit, Michigan, USA >>>>>> www.sae.org/congress/ >>>>>> >>>>>> Apr 28-30, 2015 | iqnite >>>>>> Dusseldorf, Germany >>>>>> www.iqnite-conferences.com/de/index.aspx >>>>>> >>>>>> ----------------------------------------------------------------------------------------------------------------- >>>>>> >>>>>> Gesch?ftsf?hrung: Theofanis Vassiliou-Gioles, Stephan >>>>>> Pietsch, Pete Nicholson Handelsregister HRB 77805 B, >>>>>> Amtsgericht Charlottenburg Ust ID Nr.: DE 813 143 070 This >>>>>> email may contain confidential and privileged material for >>>>>> the sole use of the intended recipient. Any review, use, >>>>>> distribution or disclosure by others is strictly prohibited. >>>>>> If you are not the intended recipient (or authorized to >>>>>> receive for the recipient), please contact the sender by >>>>>> reply email and delete all copies of this message. >>>>> >>>> >>>> >>>> >>>> >>>> -- >>>> -- >>>> ------------------------------ >>>> ------------------------------------------- >>>> Dr. Jacob Wieland >>>> Senior Software Engineer >>>> >>>> Testing Technologies IST GmbH >>>> Michaelkirchstra?e 17/18 >>>> 10179 Berlin, Germany >>>> >>>> Phone +49 30 726 19 19 34 Email wieland at testingtech.com >>>> >>>> Fax +49 30 726 19 19 20 Internet www.testingtech.com >>>> >>>> --------------------------------------------------------------------------------------------------------------- >>>> >>>> >>>> ----------------------------------------------------------------------------------------------------------------- >>>> >>>> >>>> UPCOMING EVENTS >>>> >>>> SUBMIT YOUR TOPIC for the UCAAT 2015 >>>> Deadline: May 30, 2015 >>>> ucaat.etsi.org/2015/CallForPresentations.html >>>> >>>> >>>> Apr 21-23, 2015 | SAE Conference & Exhibition >>>> Detroit, Michigan, USA >>>> www.sae.org/congress/ >>>> >>>> Apr 28-30, 2015 | iqnite >>>> Dusseldorf, Germany >>>> www.iqnite-conferences.com/de/index.aspx >>>> >>>> ----------------------------------------------------------------------------------------------------------------- >>>> >>>> Gesch?ftsf?hrung: Theofanis Vassiliou-Gioles, Stephan Pietsch, >>>> Pete Nicholson Handelsregister HRB 77805 B, Amtsgericht >>>> Charlottenburg Ust ID Nr.: DE 813 143 070 This email may contain >>>> confidential and privileged material for the sole use of the >>>> intended recipient. Any review, use, distribution or disclosure by >>>> others is strictly prohibited. If you are not the intended >>>> recipient (or authorized to receive for the recipient), please >>>> contact the sender by reply email and delete all copies of this >>>> message. >>> >>> >>> >>> >>> -- >>> -- >>> ------------------------------ >>> ------------------------------------------- >>> Dr. Jacob Wieland >>> Senior Software Engineer >>> >>> Testing Technologies IST GmbH >>> Michaelkirchstra?e 17/18 >>> 10179 Berlin, Germany >>> >>> Phone +49 30 726 19 19 34 Email wieland at testingtech.com >>> >>> Fax +49 30 726 19 19 20 Internet www.testingtech.com >>> >>> --------------------------------------------------------------------------------------------------------------- >>> >>> >>> ----------------------------------------------------------------------------------------------------------------- >>> >>> >>> UPCOMING EVENTS >>> >>> SUBMIT YOUR TOPIC for the UCAAT 2015 >>> Deadline: May 30, 2015 >>> ucaat.etsi.org/2015/CallForPresentations.html >>> >>> >>> Apr 21-23, 2015 | SAE Conference & Exhibition >>> Detroit, Michigan, USA >>> www.sae.org/congress/ >>> >>> Apr 28-30, 2015 | iqnite >>> Dusseldorf, Germany >>> www.iqnite-conferences.com/de/index.aspx >>> >>> >>> >>> ----------------------------------------------------------------------------------------------------------------- >>> >>> Gesch?ftsf?hrung: Theofanis Vassiliou-Gioles, Stephan Pietsch, Pete >>> Nicholson Handelsregister HRB 77805 B, Amtsgericht Charlottenburg >>> Ust ID >>> Nr.: DE 813 143 070 This email may contain confidential and privileged >>> material for the sole use of the intended recipient. Any review, use, >>> distribution or disclosure by others is strictly prohibited. If you are >>> not the intended recipient (or authorized to receive for the >>> recipient), >>> please contact the sender by reply email and delete all copies of this >>> message. > From maurizio.cimadamore at oracle.com Wed Apr 29 17:16:17 2015 From: maurizio.cimadamore at oracle.com (Maurizio Cimadamore) Date: Wed, 29 Apr 2015 18:16:17 +0100 Subject: Java Performance Degradation in JDK7 and JDK8 In-Reply-To: <55410C10.9010108@oracle.com> References: <5540A479.7010401@oracle.com> <5540ABCD.3080708@oracle.com> <5540D8B0.3070604@oracle.com> <55410052.5080205@oracle.com> <55410A1A.1090104@oracle.com> <55410C10.9010108@oracle.com> Message-ID: <554111E1.10305@oracle.com> Found it. The big offender is the listeners field in the CompoundScope (which is a list). The test ends up creating huge listeners lists - probably because of very deep inheritance hierarchies. If I avoid adding stuff to the listeners field in the compound scope, I get back to a sane scenario: real 0m32.540s user 1m15.298s sys 0m1.126s And the profiled memory usage is much more under control. So, looks like we need a way to prevent these listeners list to overwhelm javac ;-) Maurizio On 29/04/15 17:51, Vicente-Arturo Romero-Zaldivar wrote: > On 04/29/2015 09:43 AM, Maurizio Cimadamore wrote: >> >> >> On 29/04/15 17:01, Jan Lahoda wrote: >>> On 29.4.2015 16:06, Jacob Wieland wrote: >>>> I have to admit, the reproducer is only a small part of the actual >>>> generated code. In my preliminary tests, it already sufficed to show a >>>> difference (also, the smaller code still worked with 32 bit while the >>>> whole code runs out of memory in all 32 bit versions which makes >>>> comparison much harder ;-) and which is why I need the 64 bit >>>> versions). >>>> If I test with the complete code which is much bigger, the results are >>>> as follows: >>>> >>>> jdk1.6u45(64bit) 2GB MaxMem - 1:30 minutes >>>> jdk1.7u75(64bit) 2GB MaxMem - > 6 min >>>> jdk1.8u31(64bit) 1GB MaxMem - > 15 min >>>> jdk1.8u31(64bit) 2GB MaxMem - > 10 min >>>> jdk1.8u31(64bit) 4GB MaxMem - 2:20 min (-source/-target 6 does not >>>> seem >>>> to have any effect) >>>> >>>> So, if you throw insane (in comparison with 1.6) amounts of memory at >>>> 1.7/1.8, it is only about a third as slow, but this still is >>>> unacceptable. I actually think it has to do with parallellization and >>> >>> When I was looking at JDK-8039262, a significant contributing factor >>> to the slowdown (with enough memory and -source/-target 6) appeared >>> to be Check.checkOverrideClashes - I believe this does checks that >>> were not properly implemented in 6, contributing to the difference >>> between 6 and 7 (which seems to be particularly visible for this >>> testcase). I was looking at the checks a few times, trying to write >>> them differently/faster while still performing the checks, but was >>> not successful in that yet, unfortunately. >> I see the issue now - it is reproducible with the following memory >> parameter (at least in my machine): >> >> -J-Xmx768M >> >> This give around 20sec in JDK 6 and 10+ minutes in JDK 8. >> >> All the time seems to be spent in desugaring, most specifically in >> TransTypes.addBridges - it seems like that method calls >> Types.implementation a lot - so my theory was that the fact that >> javac consumes more memory, forces the GC to get rid of the cached >> entries in the implementation/members closure caches (since such >> entries are deliberately backed up by SoftReferences), which in turn >> completely trashes performances. I instrumented the code a bit and >> this is what I found: >> >> *) With -Xmx768M >> >> Impl cache misses = 3346926 >> Members cache misses = 1042678 >> >> real 7m0.335s >> user 25m51.517s >> sys 0m4.947s >> >> >> *) W/o -Xmx768M >> >> Impl cache misses = 3346839 >> Members cache misses = 1042678 >> >> real 0m32.377s >> user 1m25.881s >> sys 0m2.232s >> >> Long story short - cache misses do not play a factor in here - there >> are some minor differences, but nothing out of the ordinary and defo >> nothing that would explain a multi-minute slowdown! Any ideas? > > use flight recorder? > > Vicente > >> >> Maurizio >>> >>> Jan >>> >>>> garbage collection. >>>> >>>> 2015-04-29 15:12 GMT+02:00 Maurizio Cimadamore >>>> >>> >: >>>> >>>> On 29/04/15 12:44, Jacob Wieland wrote: >>>>> Hello Maurizio, >>>>> >>>>> are you sure that you used the 64bit versions of javac? I could >>>>> only observe the behavior with these. >>>> Yep I'm on a Ubuntu x64 machine. It's actually pretty standard >>>> hardware too - i.e. intel i5 (two cores, but OS sees 4 because of >>>> hyper-threading). >>>>> Also, I just tried with jdk8u31-64b and it takes AGES (still >>>>> running after 17 minutes where the jdk6 was done after 2), top >>>>> shows 4GB VIRT memory use and 350 % load (on a 4core processor). >>>> Maybe the reproducer you sent was incorrect? >>>> >>>> Maurizio >>>> >>>>> >>>>> So, I don't think it was that problem. >>>>> >>>>> 2015-04-29 12:00 GMT+02:00 Maurizio Cimadamore >>>>> >>>> >: >>>>> >>>>> These are the numbers I'm getting: >>>>> >>>>> JDK 9 (b42) >>>>> >>>>> Note: generated_ttcn/TTCN3_CommonDefs.java uses or >>>>> overrides a >>>>> deprecated API. >>>>> Note: Recompile with -Xlint:deprecation for details. >>>>> >>>>> real 0m46.306s >>>>> user 2m17.489s >>>>> sys 0m2.166s >>>>> >>>>> JDK 8 (GA) >>>>> >>>>> Note: generated_ttcn/TTCN3_CommonDefs.java uses or >>>>> overrides a >>>>> deprecated API. >>>>> Note: Recompile with -Xlint:deprecation for details. >>>>> >>>>> real 6m58.748s >>>>> user 8m43.546s >>>>> sys 0m2.132s >>>>> >>>>> JDK 7 (1.7.0_79) >>>>> >>>>> Note: generated_ttcn/TTCN3_CommonDefs.java uses or >>>>> overrides a >>>>> deprecated API. >>>>> Note: Recompile with -Xlint:deprecation for details. >>>>> >>>>> real 0m28.341s >>>>> user 1m17.194s >>>>> sys 0m1.886s >>>>> >>>>> >>>>> As you can see there is a significant regression from JDK >>>>> 7 to >>>>> JDK 8 which was caused by >>>>> >>>>> https://bugs.openjdk.java.net/browse/JDK-8043253 >>>>> >>>>> (some stack trace analysis revealed the familiar pattern). >>>>> This has also been fixed in JDK 8u20 (as stated in the bug >>>>> evaluation). >>>>> >>>>> So, while JDK 8u20/9 is slower than JDK 7 (at least on my >>>>> machine), the numbers are more or less in the same ballpark >>>>> and the huge regression that was visible in earlier JDK 8 >>>>> releases has now been fixed. >>>>> >>>>> If you are still experiencing the problem - can you please >>>>> also submit the specific compiler versions you are using in >>>>> your benchmark? >>>>> >>>>> Maurizio >>>>> >>>>> >>>>> >>>>> On 29/04/15 10:29, Maurizio Cimadamore wrote: >>>>>> Hi Jacob, >>>>>> Stay assured - as we'll definitively look into this issue (I >>>>>> see it's already assigned to one of my colleagues). >>>>>> >>>>>> What I can say (w/o looking too much at the attached >>>>>> artifacts) is that in general, javac has no issue with >>>>>> compiling a lot of sources at once; at one point the build >>>>>> system was structured in such a way that all the JDK classes >>>>>> were compiled at once - and that (which is way more than >>>>>> your >>>>>> 187 sources - i.e. at least 10x that) took less than 20 >>>>>> seconds. SO there must some specific pattern triggering that >>>>>> issue. >>>>>> >>>>>> Given that you say you have 187 input sources and 48K output >>>>>> classes, I'd say you are using inner classes a lot. I wonder >>>>>> if you are hitting this: >>>>>> >>>>>> https://bugs.openjdk.java.net/browse/JDK-8000316 >>>>>> >>>>>> Maurizio >>>>>> >>>>>> On 24/04/15 09:49, Jacob Wieland wrote: >>>>>>> Hello Folks, >>>>>>> >>>>>>> >>>>>>> I still have the open problem >>>>>>> https://bugs.openjdk.java.net/browse/JDK-8039262 that the >>>>>>> javac performance has degraded significantly from 1.6 to >>>>>>> 1.7 >>>>>>> (and even worse to 1.8) in the 64bit versions. Since in our >>>>>>> context, we are dealing with a lot of generated source1.4 >>>>>>> Java input (either split into very large files with inner >>>>>>> classes or big packages with lots of smaller classes), >>>>>>> compiler performance is critical for our tool and this >>>>>>> degradation forces us to continue recommending to our >>>>>>> customers to use Java 1.6 for large projects (as is the >>>>>>> norm) as 1.7 and 1.8 are pretty much unusable in this >>>>>>> respect. >>>>>>> >>>>>>> Is anyone still working on this issue or is such >>>>>>> significant >>>>>>> performance degradation not a serious issue? >>>>>>> >>>>>>> My observations so far are: >>>>>>> - it gets worse the more class files are being compiled/the >>>>>>> more files reside in the source path >>>>>>> - cpu usage goes through the roof over all available >>>>>>> kernels >>>>>>> - memory usage is much higher >>>>>>> - garbage collection seems to be much more active >>>>>>> >>>>>>> Using -proc:none alleviates the problem slightly for the >>>>>>> 1.7 >>>>>>> version, but not for 1.8 (last we tested with >>>>>>> jdk1.8.0_31) where the performance difference is a factor 5 >>>>>>> or more! >>>>>>> >>>>>>> I can understand that a more advanced compiler has >>>>>>> capabilities that a previous version does not have and thus >>>>>>> sometimes has >>>>>>> to do more work. But, it should still be possible >>>>>>> (especially if given the -source 1.4 or -source 1.5 option >>>>>>> as we do) to optimize it in such a way that unnecessary >>>>>>> checks for generics, overriding methods, closures, >>>>>>> annotations and other newer features can be turned off (if >>>>>>> they are to blame, which I actually doubt from my >>>>>>> observations). >>>>>>> >>>>>>> I would really appreciate your help in this regard and I >>>>>>> think everyone would benefit from any bugfix you can offer >>>>>>> for this. >>>>>>> >>>>>>> BR, Jacob Wieland >>>>>>> >>>>>>> -- >>>>>>> -- >>>>>>> ------------------------------ >>>>>>> ------------------------------------------- >>>>>>> Dr. Jacob Wieland >>>>>>> Senior Software Engineer >>>>>>> >>>>>>> Testing Technologies IST GmbH >>>>>>> Michaelkirchstra?e 17/18 >>>>>>> 10179 Berlin, Germany >>>>>>> >>>>>>> Phone +49 30 726 19 19 34 Email >>>>>>> wieland at testingtech.com >>>>>>> Fax +49 30 726 19 19 20 Internet www.testingtech.com >>>>>>> >>>>>>> --------------------------------------------------------------------------------------------------------------- >>>>>>> >>>>>>> >>>>>>> ----------------------------------------------------------------------------------------------------------------- >>>>>>> >>>>>>> >>>>>>> UPCOMING EVENTS >>>>>>> >>>>>>> SUBMIT YOUR TOPIC for the UCAAT 2015 >>>>>>> Deadline: May 30, 2015 >>>>>>> ucaat.etsi.org/2015/CallForPresentations.html >>>>>>> >>>>>>> >>>>>>> Apr 21-23, 2015 | SAE Conference & Exhibition >>>>>>> Detroit, Michigan, USA >>>>>>> www.sae.org/congress/ >>>>>>> >>>>>>> Apr 28-30, 2015 | iqnite >>>>>>> Dusseldorf, Germany >>>>>>> www.iqnite-conferences.com/de/index.aspx >>>>>>> >>>>>>> ----------------------------------------------------------------------------------------------------------------- >>>>>>> >>>>>>> Gesch?ftsf?hrung: Theofanis Vassiliou-Gioles, Stephan >>>>>>> Pietsch, Pete Nicholson Handelsregister HRB 77805 B, >>>>>>> Amtsgericht Charlottenburg Ust ID Nr.: DE 813 143 070 This >>>>>>> email may contain confidential and privileged material for >>>>>>> the sole use of the intended recipient. Any review, use, >>>>>>> distribution or disclosure by others is strictly >>>>>>> prohibited. >>>>>>> If you are not the intended recipient (or authorized to >>>>>>> receive for the recipient), please contact the sender by >>>>>>> reply email and delete all copies of this message. >>>>>> >>>>> >>>>> >>>>> >>>>> >>>>> -- >>>>> -- >>>>> ------------------------------ >>>>> ------------------------------------------- >>>>> Dr. Jacob Wieland >>>>> Senior Software Engineer >>>>> >>>>> Testing Technologies IST GmbH >>>>> Michaelkirchstra?e 17/18 >>>>> 10179 Berlin, Germany >>>>> >>>>> Phone +49 30 726 19 19 34 Email wieland at testingtech.com >>>>> >>>>> Fax +49 30 726 19 19 20 Internet www.testingtech.com >>>>> >>>>> --------------------------------------------------------------------------------------------------------------- >>>>> >>>>> >>>>> ----------------------------------------------------------------------------------------------------------------- >>>>> >>>>> >>>>> UPCOMING EVENTS >>>>> >>>>> SUBMIT YOUR TOPIC for the UCAAT 2015 >>>>> Deadline: May 30, 2015 >>>>> ucaat.etsi.org/2015/CallForPresentations.html >>>>> >>>>> >>>>> Apr 21-23, 2015 | SAE Conference & Exhibition >>>>> Detroit, Michigan, USA >>>>> www.sae.org/congress/ >>>>> >>>>> Apr 28-30, 2015 | iqnite >>>>> Dusseldorf, Germany >>>>> www.iqnite-conferences.com/de/index.aspx >>>>> >>>>> ----------------------------------------------------------------------------------------------------------------- >>>>> >>>>> Gesch?ftsf?hrung: Theofanis Vassiliou-Gioles, Stephan Pietsch, >>>>> Pete Nicholson Handelsregister HRB 77805 B, Amtsgericht >>>>> Charlottenburg Ust ID Nr.: DE 813 143 070 This email may contain >>>>> confidential and privileged material for the sole use of the >>>>> intended recipient. Any review, use, distribution or >>>>> disclosure by >>>>> others is strictly prohibited. If you are not the intended >>>>> recipient (or authorized to receive for the recipient), please >>>>> contact the sender by reply email and delete all copies of this >>>>> message. >>>> >>>> >>>> >>>> >>>> -- >>>> -- >>>> ------------------------------ >>>> ------------------------------------------- >>>> Dr. Jacob Wieland >>>> Senior Software Engineer >>>> >>>> Testing Technologies IST GmbH >>>> Michaelkirchstra?e 17/18 >>>> 10179 Berlin, Germany >>>> >>>> Phone +49 30 726 19 19 34 Email wieland at testingtech.com >>>> >>>> Fax +49 30 726 19 19 20 Internet www.testingtech.com >>>> >>>> --------------------------------------------------------------------------------------------------------------- >>>> >>>> >>>> ----------------------------------------------------------------------------------------------------------------- >>>> >>>> >>>> UPCOMING EVENTS >>>> >>>> SUBMIT YOUR TOPIC for the UCAAT 2015 >>>> Deadline: May 30, 2015 >>>> ucaat.etsi.org/2015/CallForPresentations.html >>>> >>>> >>>> Apr 21-23, 2015 | SAE Conference & Exhibition >>>> Detroit, Michigan, USA >>>> www.sae.org/congress/ >>>> >>>> Apr 28-30, 2015 | iqnite >>>> Dusseldorf, Germany >>>> www.iqnite-conferences.com/de/index.aspx >>>> >>>> >>>> >>>> ----------------------------------------------------------------------------------------------------------------- >>>> >>>> Gesch?ftsf?hrung: Theofanis Vassiliou-Gioles, Stephan Pietsch, Pete >>>> Nicholson Handelsregister HRB 77805 B, Amtsgericht Charlottenburg >>>> Ust ID >>>> Nr.: DE 813 143 070 This email may contain confidential and privileged >>>> material for the sole use of the intended recipient. Any review, use, >>>> distribution or disclosure by others is strictly prohibited. If you >>>> are >>>> not the intended recipient (or authorized to receive for the >>>> recipient), >>>> please contact the sender by reply email and delete all copies of this >>>> message. >> > From jan.lahoda at oracle.com Wed Apr 29 17:38:16 2015 From: jan.lahoda at oracle.com (Jan Lahoda) Date: Wed, 29 Apr 2015 19:38:16 +0200 Subject: Java Performance Degradation in JDK7 and JDK8 In-Reply-To: <554111E1.10305@oracle.com> References: <5540A479.7010401@oracle.com> <5540ABCD.3080708@oracle.com> <5540D8B0.3070604@oracle.com> <55410052.5080205@oracle.com> <55410A1A.1090104@oracle.com> <55410C10.9010108@oracle.com> <554111E1.10305@oracle.com> Message-ID: <55411708.3020109@oracle.com> In 9, the Scope listeners are (AFAIK) only needed to update the "mark" (so that the Types caches can detect obsolete entries). I think we may be able to avoid the listeners at the cost of slowing down getMark somewhat (getMark would ask the sub-scopes for their marks and sum the result). I'll try. Jan On 29.4.2015 19:16, Maurizio Cimadamore wrote: > Found it. > > The big offender is the listeners field in the CompoundScope (which is a > list). > > The test ends up creating huge listeners lists - probably because of > very deep inheritance hierarchies. > > If I avoid adding stuff to the listeners field in the compound scope, I > get back to a sane scenario: > > real 0m32.540s > user 1m15.298s > sys 0m1.126s > > > And the profiled memory usage is much more under control. > > So, looks like we need a way to prevent these listeners list to > overwhelm javac ;-) > > Maurizio > > On 29/04/15 17:51, Vicente-Arturo Romero-Zaldivar wrote: >> On 04/29/2015 09:43 AM, Maurizio Cimadamore wrote: >>> >>> >>> On 29/04/15 17:01, Jan Lahoda wrote: >>>> On 29.4.2015 16:06, Jacob Wieland wrote: >>>>> I have to admit, the reproducer is only a small part of the actual >>>>> generated code. In my preliminary tests, it already sufficed to show a >>>>> difference (also, the smaller code still worked with 32 bit while the >>>>> whole code runs out of memory in all 32 bit versions which makes >>>>> comparison much harder ;-) and which is why I need the 64 bit >>>>> versions). >>>>> If I test with the complete code which is much bigger, the results are >>>>> as follows: >>>>> >>>>> jdk1.6u45(64bit) 2GB MaxMem - 1:30 minutes >>>>> jdk1.7u75(64bit) 2GB MaxMem - > 6 min >>>>> jdk1.8u31(64bit) 1GB MaxMem - > 15 min >>>>> jdk1.8u31(64bit) 2GB MaxMem - > 10 min >>>>> jdk1.8u31(64bit) 4GB MaxMem - 2:20 min (-source/-target 6 does not >>>>> seem >>>>> to have any effect) >>>>> >>>>> So, if you throw insane (in comparison with 1.6) amounts of memory at >>>>> 1.7/1.8, it is only about a third as slow, but this still is >>>>> unacceptable. I actually think it has to do with parallellization and >>>> >>>> When I was looking at JDK-8039262, a significant contributing factor >>>> to the slowdown (with enough memory and -source/-target 6) appeared >>>> to be Check.checkOverrideClashes - I believe this does checks that >>>> were not properly implemented in 6, contributing to the difference >>>> between 6 and 7 (which seems to be particularly visible for this >>>> testcase). I was looking at the checks a few times, trying to write >>>> them differently/faster while still performing the checks, but was >>>> not successful in that yet, unfortunately. >>> I see the issue now - it is reproducible with the following memory >>> parameter (at least in my machine): >>> >>> -J-Xmx768M >>> >>> This give around 20sec in JDK 6 and 10+ minutes in JDK 8. >>> >>> All the time seems to be spent in desugaring, most specifically in >>> TransTypes.addBridges - it seems like that method calls >>> Types.implementation a lot - so my theory was that the fact that >>> javac consumes more memory, forces the GC to get rid of the cached >>> entries in the implementation/members closure caches (since such >>> entries are deliberately backed up by SoftReferences), which in turn >>> completely trashes performances. I instrumented the code a bit and >>> this is what I found: >>> >>> *) With -Xmx768M >>> >>> Impl cache misses = 3346926 >>> Members cache misses = 1042678 >>> >>> real 7m0.335s >>> user 25m51.517s >>> sys 0m4.947s >>> >>> >>> *) W/o -Xmx768M >>> >>> Impl cache misses = 3346839 >>> Members cache misses = 1042678 >>> >>> real 0m32.377s >>> user 1m25.881s >>> sys 0m2.232s >>> >>> Long story short - cache misses do not play a factor in here - there >>> are some minor differences, but nothing out of the ordinary and defo >>> nothing that would explain a multi-minute slowdown! Any ideas? >> >> use flight recorder? >> >> Vicente >> >>> >>> Maurizio >>>> >>>> Jan >>>> >>>>> garbage collection. >>>>> >>>>> 2015-04-29 15:12 GMT+02:00 Maurizio Cimadamore >>>>> >>>> >: >>>>> >>>>> On 29/04/15 12:44, Jacob Wieland wrote: >>>>>> Hello Maurizio, >>>>>> >>>>>> are you sure that you used the 64bit versions of javac? I could >>>>>> only observe the behavior with these. >>>>> Yep I'm on a Ubuntu x64 machine. It's actually pretty standard >>>>> hardware too - i.e. intel i5 (two cores, but OS sees 4 because of >>>>> hyper-threading). >>>>>> Also, I just tried with jdk8u31-64b and it takes AGES (still >>>>>> running after 17 minutes where the jdk6 was done after 2), top >>>>>> shows 4GB VIRT memory use and 350 % load (on a 4core processor). >>>>> Maybe the reproducer you sent was incorrect? >>>>> >>>>> Maurizio >>>>> >>>>>> >>>>>> So, I don't think it was that problem. >>>>>> >>>>>> 2015-04-29 12:00 GMT+02:00 Maurizio Cimadamore >>>>>> >>>>> >: >>>>>> >>>>>> These are the numbers I'm getting: >>>>>> >>>>>> JDK 9 (b42) >>>>>> >>>>>> Note: generated_ttcn/TTCN3_CommonDefs.java uses or >>>>>> overrides a >>>>>> deprecated API. >>>>>> Note: Recompile with -Xlint:deprecation for details. >>>>>> >>>>>> real 0m46.306s >>>>>> user 2m17.489s >>>>>> sys 0m2.166s >>>>>> >>>>>> JDK 8 (GA) >>>>>> >>>>>> Note: generated_ttcn/TTCN3_CommonDefs.java uses or >>>>>> overrides a >>>>>> deprecated API. >>>>>> Note: Recompile with -Xlint:deprecation for details. >>>>>> >>>>>> real 6m58.748s >>>>>> user 8m43.546s >>>>>> sys 0m2.132s >>>>>> >>>>>> JDK 7 (1.7.0_79) >>>>>> >>>>>> Note: generated_ttcn/TTCN3_CommonDefs.java uses or >>>>>> overrides a >>>>>> deprecated API. >>>>>> Note: Recompile with -Xlint:deprecation for details. >>>>>> >>>>>> real 0m28.341s >>>>>> user 1m17.194s >>>>>> sys 0m1.886s >>>>>> >>>>>> >>>>>> As you can see there is a significant regression from JDK >>>>>> 7 to >>>>>> JDK 8 which was caused by >>>>>> >>>>>> https://bugs.openjdk.java.net/browse/JDK-8043253 >>>>>> >>>>>> (some stack trace analysis revealed the familiar pattern). >>>>>> This has also been fixed in JDK 8u20 (as stated in the bug >>>>>> evaluation). >>>>>> >>>>>> So, while JDK 8u20/9 is slower than JDK 7 (at least on my >>>>>> machine), the numbers are more or less in the same ballpark >>>>>> and the huge regression that was visible in earlier JDK 8 >>>>>> releases has now been fixed. >>>>>> >>>>>> If you are still experiencing the problem - can you please >>>>>> also submit the specific compiler versions you are using in >>>>>> your benchmark? >>>>>> >>>>>> Maurizio >>>>>> >>>>>> >>>>>> >>>>>> On 29/04/15 10:29, Maurizio Cimadamore wrote: >>>>>>> Hi Jacob, >>>>>>> Stay assured - as we'll definitively look into this issue (I >>>>>>> see it's already assigned to one of my colleagues). >>>>>>> >>>>>>> What I can say (w/o looking too much at the attached >>>>>>> artifacts) is that in general, javac has no issue with >>>>>>> compiling a lot of sources at once; at one point the build >>>>>>> system was structured in such a way that all the JDK classes >>>>>>> were compiled at once - and that (which is way more than >>>>>>> your >>>>>>> 187 sources - i.e. at least 10x that) took less than 20 >>>>>>> seconds. SO there must some specific pattern triggering that >>>>>>> issue. >>>>>>> >>>>>>> Given that you say you have 187 input sources and 48K output >>>>>>> classes, I'd say you are using inner classes a lot. I wonder >>>>>>> if you are hitting this: >>>>>>> >>>>>>> https://bugs.openjdk.java.net/browse/JDK-8000316 >>>>>>> >>>>>>> Maurizio >>>>>>> >>>>>>> On 24/04/15 09:49, Jacob Wieland wrote: >>>>>>>> Hello Folks, >>>>>>>> >>>>>>>> >>>>>>>> I still have the open problem >>>>>>>> https://bugs.openjdk.java.net/browse/JDK-8039262 that the >>>>>>>> javac performance has degraded significantly from 1.6 to >>>>>>>> 1.7 >>>>>>>> (and even worse to 1.8) in the 64bit versions. Since in our >>>>>>>> context, we are dealing with a lot of generated source1.4 >>>>>>>> Java input (either split into very large files with inner >>>>>>>> classes or big packages with lots of smaller classes), >>>>>>>> compiler performance is critical for our tool and this >>>>>>>> degradation forces us to continue recommending to our >>>>>>>> customers to use Java 1.6 for large projects (as is the >>>>>>>> norm) as 1.7 and 1.8 are pretty much unusable in this >>>>>>>> respect. >>>>>>>> >>>>>>>> Is anyone still working on this issue or is such >>>>>>>> significant >>>>>>>> performance degradation not a serious issue? >>>>>>>> >>>>>>>> My observations so far are: >>>>>>>> - it gets worse the more class files are being compiled/the >>>>>>>> more files reside in the source path >>>>>>>> - cpu usage goes through the roof over all available >>>>>>>> kernels >>>>>>>> - memory usage is much higher >>>>>>>> - garbage collection seems to be much more active >>>>>>>> >>>>>>>> Using -proc:none alleviates the problem slightly for the >>>>>>>> 1.7 >>>>>>>> version, but not for 1.8 (last we tested with >>>>>>>> jdk1.8.0_31) where the performance difference is a factor 5 >>>>>>>> or more! >>>>>>>> >>>>>>>> I can understand that a more advanced compiler has >>>>>>>> capabilities that a previous version does not have and thus >>>>>>>> sometimes has >>>>>>>> to do more work. But, it should still be possible >>>>>>>> (especially if given the -source 1.4 or -source 1.5 option >>>>>>>> as we do) to optimize it in such a way that unnecessary >>>>>>>> checks for generics, overriding methods, closures, >>>>>>>> annotations and other newer features can be turned off (if >>>>>>>> they are to blame, which I actually doubt from my >>>>>>>> observations). >>>>>>>> >>>>>>>> I would really appreciate your help in this regard and I >>>>>>>> think everyone would benefit from any bugfix you can offer >>>>>>>> for this. >>>>>>>> >>>>>>>> BR, Jacob Wieland >>>>>>>> >>>>>>>> -- >>>>>>>> -- >>>>>>>> ------------------------------ >>>>>>>> ------------------------------------------- >>>>>>>> Dr. Jacob Wieland >>>>>>>> Senior Software Engineer >>>>>>>> >>>>>>>> Testing Technologies IST GmbH >>>>>>>> Michaelkirchstra?e 17/18 >>>>>>>> 10179 Berlin, Germany >>>>>>>> >>>>>>>> Phone +49 30 726 19 19 34 Email >>>>>>>> wieland at testingtech.com >>>>>>>> Fax +49 30 726 19 19 20 Internet www.testingtech.com >>>>>>>> >>>>>>>> --------------------------------------------------------------------------------------------------------------- >>>>>>>> >>>>>>>> >>>>>>>> ----------------------------------------------------------------------------------------------------------------- >>>>>>>> >>>>>>>> >>>>>>>> UPCOMING EVENTS >>>>>>>> >>>>>>>> SUBMIT YOUR TOPIC for the UCAAT 2015 >>>>>>>> Deadline: May 30, 2015 >>>>>>>> ucaat.etsi.org/2015/CallForPresentations.html >>>>>>>> >>>>>>>> >>>>>>>> Apr 21-23, 2015 | SAE Conference & Exhibition >>>>>>>> Detroit, Michigan, USA >>>>>>>> www.sae.org/congress/ >>>>>>>> >>>>>>>> Apr 28-30, 2015 | iqnite >>>>>>>> Dusseldorf, Germany >>>>>>>> www.iqnite-conferences.com/de/index.aspx >>>>>>>> >>>>>>>> ----------------------------------------------------------------------------------------------------------------- >>>>>>>> >>>>>>>> Gesch?ftsf?hrung: Theofanis Vassiliou-Gioles, Stephan >>>>>>>> Pietsch, Pete Nicholson Handelsregister HRB 77805 B, >>>>>>>> Amtsgericht Charlottenburg Ust ID Nr.: DE 813 143 070 This >>>>>>>> email may contain confidential and privileged material for >>>>>>>> the sole use of the intended recipient. Any review, use, >>>>>>>> distribution or disclosure by others is strictly >>>>>>>> prohibited. >>>>>>>> If you are not the intended recipient (or authorized to >>>>>>>> receive for the recipient), please contact the sender by >>>>>>>> reply email and delete all copies of this message. >>>>>>> >>>>>> >>>>>> >>>>>> >>>>>> >>>>>> -- >>>>>> -- >>>>>> ------------------------------ >>>>>> ------------------------------------------- >>>>>> Dr. Jacob Wieland >>>>>> Senior Software Engineer >>>>>> >>>>>> Testing Technologies IST GmbH >>>>>> Michaelkirchstra?e 17/18 >>>>>> 10179 Berlin, Germany >>>>>> >>>>>> Phone +49 30 726 19 19 34 Email wieland at testingtech.com >>>>>> >>>>>> Fax +49 30 726 19 19 20 Internet www.testingtech.com >>>>>> >>>>>> --------------------------------------------------------------------------------------------------------------- >>>>>> >>>>>> >>>>>> ----------------------------------------------------------------------------------------------------------------- >>>>>> >>>>>> >>>>>> UPCOMING EVENTS >>>>>> >>>>>> SUBMIT YOUR TOPIC for the UCAAT 2015 >>>>>> Deadline: May 30, 2015 >>>>>> ucaat.etsi.org/2015/CallForPresentations.html >>>>>> >>>>>> >>>>>> Apr 21-23, 2015 | SAE Conference & Exhibition >>>>>> Detroit, Michigan, USA >>>>>> www.sae.org/congress/ >>>>>> >>>>>> Apr 28-30, 2015 | iqnite >>>>>> Dusseldorf, Germany >>>>>> www.iqnite-conferences.com/de/index.aspx >>>>>> >>>>>> ----------------------------------------------------------------------------------------------------------------- >>>>>> >>>>>> Gesch?ftsf?hrung: Theofanis Vassiliou-Gioles, Stephan Pietsch, >>>>>> Pete Nicholson Handelsregister HRB 77805 B, Amtsgericht >>>>>> Charlottenburg Ust ID Nr.: DE 813 143 070 This email may contain >>>>>> confidential and privileged material for the sole use of the >>>>>> intended recipient. Any review, use, distribution or >>>>>> disclosure by >>>>>> others is strictly prohibited. If you are not the intended >>>>>> recipient (or authorized to receive for the recipient), please >>>>>> contact the sender by reply email and delete all copies of this >>>>>> message. >>>>> >>>>> >>>>> >>>>> >>>>> -- >>>>> -- >>>>> ------------------------------ >>>>> ------------------------------------------- >>>>> Dr. Jacob Wieland >>>>> Senior Software Engineer >>>>> >>>>> Testing Technologies IST GmbH >>>>> Michaelkirchstra?e 17/18 >>>>> 10179 Berlin, Germany >>>>> >>>>> Phone +49 30 726 19 19 34 Email wieland at testingtech.com >>>>> >>>>> Fax +49 30 726 19 19 20 Internet www.testingtech.com >>>>> >>>>> --------------------------------------------------------------------------------------------------------------- >>>>> >>>>> >>>>> ----------------------------------------------------------------------------------------------------------------- >>>>> >>>>> >>>>> UPCOMING EVENTS >>>>> >>>>> SUBMIT YOUR TOPIC for the UCAAT 2015 >>>>> Deadline: May 30, 2015 >>>>> ucaat.etsi.org/2015/CallForPresentations.html >>>>> >>>>> >>>>> Apr 21-23, 2015 | SAE Conference & Exhibition >>>>> Detroit, Michigan, USA >>>>> www.sae.org/congress/ >>>>> >>>>> Apr 28-30, 2015 | iqnite >>>>> Dusseldorf, Germany >>>>> www.iqnite-conferences.com/de/index.aspx >>>>> >>>>> >>>>> >>>>> ----------------------------------------------------------------------------------------------------------------- >>>>> >>>>> Gesch?ftsf?hrung: Theofanis Vassiliou-Gioles, Stephan Pietsch, Pete >>>>> Nicholson Handelsregister HRB 77805 B, Amtsgericht Charlottenburg >>>>> Ust ID >>>>> Nr.: DE 813 143 070 This email may contain confidential and privileged >>>>> material for the sole use of the intended recipient. Any review, use, >>>>> distribution or disclosure by others is strictly prohibited. If you >>>>> are >>>>> not the intended recipient (or authorized to receive for the >>>>> recipient), >>>>> please contact the sender by reply email and delete all copies of this >>>>> message. >>> >> > From jonathan.gibbons at oracle.com Wed Apr 29 21:26:46 2015 From: jonathan.gibbons at oracle.com (Jonathan Gibbons) Date: Wed, 29 Apr 2015 14:26:46 -0700 Subject: RFR: 8078054 [TESTBUG] tools/javac/Paths/wcMineField.sh failed with "operation not permitted" Message-ID: <55414C96.9080505@oracle.com> The test uses "cp -p" when copying some files to set up some jar files, and shortly afterwards, deletes those copies. Although generally acceptable, in some environments, the "cp -p" is not permitted. The copy can be avoided altogether, by updating the commands to create the jar files. http://cr.openjdk.java.net/~jjg/8078054-wcMineField/webrev.00/ -- Jon From joe.darcy at oracle.com Wed Apr 29 21:36:40 2015 From: joe.darcy at oracle.com (joe darcy) Date: Wed, 29 Apr 2015 14:36:40 -0700 Subject: RFR: 8078054 [TESTBUG] tools/javac/Paths/wcMineField.sh failed with "operation not permitted" In-Reply-To: <55414C96.9080505@oracle.com> References: <55414C96.9080505@oracle.com> Message-ID: <55414EE8.8010409@oracle.com> Looks fine Jon; cheers, -Joe On 4/29/2015 2:26 PM, Jonathan Gibbons wrote: > The test uses "cp -p" when copying some files to set up some jar > files, and shortly afterwards, deletes those copies. Although > generally acceptable, in some environments, the "cp -p" is not permitted. > > The copy can be avoided altogether, by updating the commands to create > the jar files. > > http://cr.openjdk.java.net/~jjg/8078054-wcMineField/webrev.00/ > > -- Jon From maurizio.cimadamore at oracle.com Thu Apr 30 15:03:26 2015 From: maurizio.cimadamore at oracle.com (Maurizio Cimadamore) Date: Thu, 30 Apr 2015 16:03:26 +0100 Subject: Java Performance Degradation in JDK7 and JDK8 In-Reply-To: <55411708.3020109@oracle.com> References: <5540A479.7010401@oracle.com> <5540ABCD.3080708@oracle.com> <5540D8B0.3070604@oracle.com> <55410052.5080205@oracle.com> <55410A1A.1090104@oracle.com> <55410C10.9010108@oracle.com> <554111E1.10305@oracle.com> <55411708.3020109@oracle.com> Message-ID: <5542443E.6030803@oracle.com> For the records - I've added a comment in https://bugs.openjdk.java.net/browse/JDK-8039262 Reporting in as much detail as possible the underlying cause of the performance regression. Maurizio On 29/04/15 18:38, Jan Lahoda wrote: > In 9, the Scope listeners are (AFAIK) only needed to update the "mark" > (so that the Types caches can detect obsolete entries). I think we may > be able to avoid the listeners at the cost of slowing down getMark > somewhat (getMark would ask the sub-scopes for their marks and sum the > result). I'll try. > > Jan > > On 29.4.2015 19:16, Maurizio Cimadamore wrote: >> Found it. >> >> The big offender is the listeners field in the CompoundScope (which is a >> list). >> >> The test ends up creating huge listeners lists - probably because of >> very deep inheritance hierarchies. >> >> If I avoid adding stuff to the listeners field in the compound scope, I >> get back to a sane scenario: >> >> real 0m32.540s >> user 1m15.298s >> sys 0m1.126s >> >> >> And the profiled memory usage is much more under control. >> >> So, looks like we need a way to prevent these listeners list to >> overwhelm javac ;-) >> >> Maurizio >> >> On 29/04/15 17:51, Vicente-Arturo Romero-Zaldivar wrote: >>> On 04/29/2015 09:43 AM, Maurizio Cimadamore wrote: >>>> >>>> >>>> On 29/04/15 17:01, Jan Lahoda wrote: >>>>> On 29.4.2015 16:06, Jacob Wieland wrote: >>>>>> I have to admit, the reproducer is only a small part of the actual >>>>>> generated code. In my preliminary tests, it already sufficed to >>>>>> show a >>>>>> difference (also, the smaller code still worked with 32 bit while >>>>>> the >>>>>> whole code runs out of memory in all 32 bit versions which makes >>>>>> comparison much harder ;-) and which is why I need the 64 bit >>>>>> versions). >>>>>> If I test with the complete code which is much bigger, the >>>>>> results are >>>>>> as follows: >>>>>> >>>>>> jdk1.6u45(64bit) 2GB MaxMem - 1:30 minutes >>>>>> jdk1.7u75(64bit) 2GB MaxMem - > 6 min >>>>>> jdk1.8u31(64bit) 1GB MaxMem - > 15 min >>>>>> jdk1.8u31(64bit) 2GB MaxMem - > 10 min >>>>>> jdk1.8u31(64bit) 4GB MaxMem - 2:20 min (-source/-target 6 does not >>>>>> seem >>>>>> to have any effect) >>>>>> >>>>>> So, if you throw insane (in comparison with 1.6) amounts of >>>>>> memory at >>>>>> 1.7/1.8, it is only about a third as slow, but this still is >>>>>> unacceptable. I actually think it has to do with parallellization >>>>>> and >>>>> >>>>> When I was looking at JDK-8039262, a significant contributing factor >>>>> to the slowdown (with enough memory and -source/-target 6) appeared >>>>> to be Check.checkOverrideClashes - I believe this does checks that >>>>> were not properly implemented in 6, contributing to the difference >>>>> between 6 and 7 (which seems to be particularly visible for this >>>>> testcase). I was looking at the checks a few times, trying to write >>>>> them differently/faster while still performing the checks, but was >>>>> not successful in that yet, unfortunately. >>>> I see the issue now - it is reproducible with the following memory >>>> parameter (at least in my machine): >>>> >>>> -J-Xmx768M >>>> >>>> This give around 20sec in JDK 6 and 10+ minutes in JDK 8. >>>> >>>> All the time seems to be spent in desugaring, most specifically in >>>> TransTypes.addBridges - it seems like that method calls >>>> Types.implementation a lot - so my theory was that the fact that >>>> javac consumes more memory, forces the GC to get rid of the cached >>>> entries in the implementation/members closure caches (since such >>>> entries are deliberately backed up by SoftReferences), which in turn >>>> completely trashes performances. I instrumented the code a bit and >>>> this is what I found: >>>> >>>> *) With -Xmx768M >>>> >>>> Impl cache misses = 3346926 >>>> Members cache misses = 1042678 >>>> >>>> real 7m0.335s >>>> user 25m51.517s >>>> sys 0m4.947s >>>> >>>> >>>> *) W/o -Xmx768M >>>> >>>> Impl cache misses = 3346839 >>>> Members cache misses = 1042678 >>>> >>>> real 0m32.377s >>>> user 1m25.881s >>>> sys 0m2.232s >>>> >>>> Long story short - cache misses do not play a factor in here - there >>>> are some minor differences, but nothing out of the ordinary and defo >>>> nothing that would explain a multi-minute slowdown! Any ideas? >>> >>> use flight recorder? >>> >>> Vicente >>> >>>> >>>> Maurizio >>>>> >>>>> Jan >>>>> >>>>>> garbage collection. >>>>>> >>>>>> 2015-04-29 15:12 GMT+02:00 Maurizio Cimadamore >>>>>> >>>>> >: >>>>>> >>>>>> On 29/04/15 12:44, Jacob Wieland wrote: >>>>>>> Hello Maurizio, >>>>>>> >>>>>>> are you sure that you used the 64bit versions of javac? I could >>>>>>> only observe the behavior with these. >>>>>> Yep I'm on a Ubuntu x64 machine. It's actually pretty standard >>>>>> hardware too - i.e. intel i5 (two cores, but OS sees 4 >>>>>> because of >>>>>> hyper-threading). >>>>>>> Also, I just tried with jdk8u31-64b and it takes AGES (still >>>>>>> running after 17 minutes where the jdk6 was done after 2), top >>>>>>> shows 4GB VIRT memory use and 350 % load (on a 4core >>>>>>> processor). >>>>>> Maybe the reproducer you sent was incorrect? >>>>>> >>>>>> Maurizio >>>>>> >>>>>>> >>>>>>> So, I don't think it was that problem. >>>>>>> >>>>>>> 2015-04-29 12:00 GMT+02:00 Maurizio Cimadamore >>>>>>> >>>>>> >: >>>>>>> >>>>>>> These are the numbers I'm getting: >>>>>>> >>>>>>> JDK 9 (b42) >>>>>>> >>>>>>> Note: generated_ttcn/TTCN3_CommonDefs.java uses or >>>>>>> overrides a >>>>>>> deprecated API. >>>>>>> Note: Recompile with -Xlint:deprecation for details. >>>>>>> >>>>>>> real 0m46.306s >>>>>>> user 2m17.489s >>>>>>> sys 0m2.166s >>>>>>> >>>>>>> JDK 8 (GA) >>>>>>> >>>>>>> Note: generated_ttcn/TTCN3_CommonDefs.java uses or >>>>>>> overrides a >>>>>>> deprecated API. >>>>>>> Note: Recompile with -Xlint:deprecation for details. >>>>>>> >>>>>>> real 6m58.748s >>>>>>> user 8m43.546s >>>>>>> sys 0m2.132s >>>>>>> >>>>>>> JDK 7 (1.7.0_79) >>>>>>> >>>>>>> Note: generated_ttcn/TTCN3_CommonDefs.java uses or >>>>>>> overrides a >>>>>>> deprecated API. >>>>>>> Note: Recompile with -Xlint:deprecation for details. >>>>>>> >>>>>>> real 0m28.341s >>>>>>> user 1m17.194s >>>>>>> sys 0m1.886s >>>>>>> >>>>>>> >>>>>>> As you can see there is a significant regression from JDK >>>>>>> 7 to >>>>>>> JDK 8 which was caused by >>>>>>> >>>>>>> https://bugs.openjdk.java.net/browse/JDK-8043253 >>>>>>> >>>>>>> (some stack trace analysis revealed the familiar pattern). >>>>>>> This has also been fixed in JDK 8u20 (as stated in the bug >>>>>>> evaluation). >>>>>>> >>>>>>> So, while JDK 8u20/9 is slower than JDK 7 (at least on my >>>>>>> machine), the numbers are more or less in the same ballpark >>>>>>> and the huge regression that was visible in earlier JDK 8 >>>>>>> releases has now been fixed. >>>>>>> >>>>>>> If you are still experiencing the problem - can you please >>>>>>> also submit the specific compiler versions you are using in >>>>>>> your benchmark? >>>>>>> >>>>>>> Maurizio >>>>>>> >>>>>>> >>>>>>> >>>>>>> On 29/04/15 10:29, Maurizio Cimadamore wrote: >>>>>>>> Hi Jacob, >>>>>>>> Stay assured - as we'll definitively look into this >>>>>>>> issue (I >>>>>>>> see it's already assigned to one of my colleagues). >>>>>>>> >>>>>>>> What I can say (w/o looking too much at the attached >>>>>>>> artifacts) is that in general, javac has no issue with >>>>>>>> compiling a lot of sources at once; at one point the build >>>>>>>> system was structured in such a way that all the JDK >>>>>>>> classes >>>>>>>> were compiled at once - and that (which is way more than >>>>>>>> your >>>>>>>> 187 sources - i.e. at least 10x that) took less than 20 >>>>>>>> seconds. SO there must some specific pattern triggering >>>>>>>> that >>>>>>>> issue. >>>>>>>> >>>>>>>> Given that you say you have 187 input sources and 48K >>>>>>>> output >>>>>>>> classes, I'd say you are using inner classes a lot. I >>>>>>>> wonder >>>>>>>> if you are hitting this: >>>>>>>> >>>>>>>> https://bugs.openjdk.java.net/browse/JDK-8000316 >>>>>>>> >>>>>>>> Maurizio >>>>>>>> >>>>>>>> On 24/04/15 09:49, Jacob Wieland wrote: >>>>>>>>> Hello Folks, >>>>>>>>> >>>>>>>>> >>>>>>>>> I still have the open problem >>>>>>>>> https://bugs.openjdk.java.net/browse/JDK-8039262 that the >>>>>>>>> javac performance has degraded significantly from 1.6 to >>>>>>>>> 1.7 >>>>>>>>> (and even worse to 1.8) in the 64bit versions. Since >>>>>>>>> in our >>>>>>>>> context, we are dealing with a lot of generated source1.4 >>>>>>>>> Java input (either split into very large files with inner >>>>>>>>> classes or big packages with lots of smaller classes), >>>>>>>>> compiler performance is critical for our tool and this >>>>>>>>> degradation forces us to continue recommending to our >>>>>>>>> customers to use Java 1.6 for large projects (as is the >>>>>>>>> norm) as 1.7 and 1.8 are pretty much unusable in this >>>>>>>>> respect. >>>>>>>>> >>>>>>>>> Is anyone still working on this issue or is such >>>>>>>>> significant >>>>>>>>> performance degradation not a serious issue? >>>>>>>>> >>>>>>>>> My observations so far are: >>>>>>>>> - it gets worse the more class files are being >>>>>>>>> compiled/the >>>>>>>>> more files reside in the source path >>>>>>>>> - cpu usage goes through the roof over all available >>>>>>>>> kernels >>>>>>>>> - memory usage is much higher >>>>>>>>> - garbage collection seems to be much more active >>>>>>>>> >>>>>>>>> Using -proc:none alleviates the problem slightly for the >>>>>>>>> 1.7 >>>>>>>>> version, but not for 1.8 (last we tested with >>>>>>>>> jdk1.8.0_31) where the performance difference is a >>>>>>>>> factor 5 >>>>>>>>> or more! >>>>>>>>> >>>>>>>>> I can understand that a more advanced compiler has >>>>>>>>> capabilities that a previous version does not have and >>>>>>>>> thus >>>>>>>>> sometimes has >>>>>>>>> to do more work. But, it should still be possible >>>>>>>>> (especially if given the -source 1.4 or -source 1.5 >>>>>>>>> option >>>>>>>>> as we do) to optimize it in such a way that unnecessary >>>>>>>>> checks for generics, overriding methods, closures, >>>>>>>>> annotations and other newer features can be turned off >>>>>>>>> (if >>>>>>>>> they are to blame, which I actually doubt from my >>>>>>>>> observations). >>>>>>>>> >>>>>>>>> I would really appreciate your help in this regard and I >>>>>>>>> think everyone would benefit from any bugfix you can >>>>>>>>> offer >>>>>>>>> for this. >>>>>>>>> >>>>>>>>> BR, Jacob Wieland >>>>>>>>> >>>>>>>>> -- >>>>>>>>> -- >>>>>>>>> ------------------------------ >>>>>>>>> ------------------------------------------- >>>>>>>>> Dr. Jacob Wieland >>>>>>>>> Senior Software Engineer >>>>>>>>> >>>>>>>>> Testing Technologies IST GmbH >>>>>>>>> Michaelkirchstra?e 17/18 >>>>>>>>> 10179 Berlin, Germany >>>>>>>>> >>>>>>>>> Phone +49 30 726 19 19 34 Email >>>>>>>>> wieland at testingtech.com >>>>>>>>> Fax +49 30 726 19 19 20 Internet www.testingtech.com >>>>>>>>> >>>>>>>>> --------------------------------------------------------------------------------------------------------------- >>>>>>>>> >>>>>>>>> >>>>>>>>> >>>>>>>>> ----------------------------------------------------------------------------------------------------------------- >>>>>>>>> >>>>>>>>> >>>>>>>>> >>>>>>>>> UPCOMING EVENTS >>>>>>>>> >>>>>>>>> SUBMIT YOUR TOPIC for the UCAAT 2015 >>>>>>>>> Deadline: May 30, 2015 >>>>>>>>> ucaat.etsi.org/2015/CallForPresentations.html >>>>>>>>> >>>>>>>>> >>>>>>>>> Apr 21-23, 2015 | SAE Conference & Exhibition >>>>>>>>> Detroit, Michigan, USA >>>>>>>>> www.sae.org/congress/ >>>>>>>>> >>>>>>>>> Apr 28-30, 2015 | iqnite >>>>>>>>> Dusseldorf, Germany >>>>>>>>> www.iqnite-conferences.com/de/index.aspx >>>>>>>>> >>>>>>>>> ----------------------------------------------------------------------------------------------------------------- >>>>>>>>> >>>>>>>>> >>>>>>>>> Gesch?ftsf?hrung: Theofanis Vassiliou-Gioles, Stephan >>>>>>>>> Pietsch, Pete Nicholson Handelsregister HRB 77805 B, >>>>>>>>> Amtsgericht Charlottenburg Ust ID Nr.: DE 813 143 070 >>>>>>>>> This >>>>>>>>> email may contain confidential and privileged material >>>>>>>>> for >>>>>>>>> the sole use of the intended recipient. Any review, use, >>>>>>>>> distribution or disclosure by others is strictly >>>>>>>>> prohibited. >>>>>>>>> If you are not the intended recipient (or authorized to >>>>>>>>> receive for the recipient), please contact the sender by >>>>>>>>> reply email and delete all copies of this message. >>>>>>>> >>>>>>> >>>>>>> >>>>>>> >>>>>>> >>>>>>> -- >>>>>>> -- >>>>>>> ------------------------------ >>>>>>> ------------------------------------------- >>>>>>> Dr. Jacob Wieland >>>>>>> Senior Software Engineer >>>>>>> >>>>>>> Testing Technologies IST GmbH >>>>>>> Michaelkirchstra?e 17/18 >>>>>>> 10179 Berlin, Germany >>>>>>> >>>>>>> Phone +49 30 726 19 19 34 Email wieland at testingtech.com >>>>>>> >>>>>>> Fax +49 30 726 19 19 20 Internet www.testingtech.com >>>>>>> >>>>>>> --------------------------------------------------------------------------------------------------------------- >>>>>>> >>>>>>> >>>>>>> >>>>>>> ----------------------------------------------------------------------------------------------------------------- >>>>>>> >>>>>>> >>>>>>> >>>>>>> UPCOMING EVENTS >>>>>>> >>>>>>> SUBMIT YOUR TOPIC for the UCAAT 2015 >>>>>>> Deadline: May 30, 2015 >>>>>>> ucaat.etsi.org/2015/CallForPresentations.html >>>>>>> >>>>>>> >>>>>>> Apr 21-23, 2015 | SAE Conference & Exhibition >>>>>>> Detroit, Michigan, USA >>>>>>> www.sae.org/congress/ >>>>>>> >>>>>>> Apr 28-30, 2015 | iqnite >>>>>>> Dusseldorf, Germany >>>>>>> www.iqnite-conferences.com/de/index.aspx >>>>>>> >>>>>>> ----------------------------------------------------------------------------------------------------------------- >>>>>>> >>>>>>> >>>>>>> Gesch?ftsf?hrung: Theofanis Vassiliou-Gioles, Stephan Pietsch, >>>>>>> Pete Nicholson Handelsregister HRB 77805 B, Amtsgericht >>>>>>> Charlottenburg Ust ID Nr.: DE 813 143 070 This email may >>>>>>> contain >>>>>>> confidential and privileged material for the sole use of the >>>>>>> intended recipient. Any review, use, distribution or >>>>>>> disclosure by >>>>>>> others is strictly prohibited. If you are not the intended >>>>>>> recipient (or authorized to receive for the recipient), please >>>>>>> contact the sender by reply email and delete all copies of this >>>>>>> message. >>>>>> >>>>>> >>>>>> >>>>>> >>>>>> -- >>>>>> -- >>>>>> ------------------------------ >>>>>> ------------------------------------------- >>>>>> Dr. Jacob Wieland >>>>>> Senior Software Engineer >>>>>> >>>>>> Testing Technologies IST GmbH >>>>>> Michaelkirchstra?e 17/18 >>>>>> 10179 Berlin, Germany >>>>>> >>>>>> Phone +49 30 726 19 19 34 Email wieland at testingtech.com >>>>>> >>>>>> Fax +49 30 726 19 19 20 Internet www.testingtech.com >>>>>> >>>>>> --------------------------------------------------------------------------------------------------------------- >>>>>> >>>>>> >>>>>> >>>>>> ----------------------------------------------------------------------------------------------------------------- >>>>>> >>>>>> >>>>>> >>>>>> UPCOMING EVENTS >>>>>> >>>>>> SUBMIT YOUR TOPIC for the UCAAT 2015 >>>>>> Deadline: May 30, 2015 >>>>>> ucaat.etsi.org/2015/CallForPresentations.html >>>>>> >>>>>> >>>>>> Apr 21-23, 2015 | SAE Conference & Exhibition >>>>>> Detroit, Michigan, USA >>>>>> www.sae.org/congress/ >>>>>> >>>>>> Apr 28-30, 2015 | iqnite >>>>>> Dusseldorf, Germany >>>>>> www.iqnite-conferences.com/de/index.aspx >>>>>> >>>>>> >>>>>> >>>>>> ----------------------------------------------------------------------------------------------------------------- >>>>>> >>>>>> >>>>>> Gesch?ftsf?hrung: Theofanis Vassiliou-Gioles, Stephan Pietsch, Pete >>>>>> Nicholson Handelsregister HRB 77805 B, Amtsgericht Charlottenburg >>>>>> Ust ID >>>>>> Nr.: DE 813 143 070 This email may contain confidential and >>>>>> privileged >>>>>> material for the sole use of the intended recipient. Any review, >>>>>> use, >>>>>> distribution or disclosure by others is strictly prohibited. If you >>>>>> are >>>>>> not the intended recipient (or authorized to receive for the >>>>>> recipient), >>>>>> please contact the sender by reply email and delete all copies of >>>>>> this >>>>>> message. >>>> >>> >> From wieland at testingtech.com Thu Apr 30 15:15:14 2015 From: wieland at testingtech.com (Jacob Wieland) Date: Thu, 30 Apr 2015 17:15:14 +0200 Subject: Java Performance Degradation in JDK7 and JDK8 In-Reply-To: <5542443E.6030803@oracle.com> References: <5540A479.7010401@oracle.com> <5540ABCD.3080708@oracle.com> <5540D8B0.3070604@oracle.com> <55410052.5080205@oracle.com> <55410A1A.1090104@oracle.com> <55410C10.9010108@oracle.com> <554111E1.10305@oracle.com> <55411708.3020109@oracle.com> <5542443E.6030803@oracle.com> Message-ID: thank you very much - very interesting. is there any estimate when a patch for 7/8 will be available? 2015-04-30 17:03 GMT+02:00 Maurizio Cimadamore < maurizio.cimadamore at oracle.com>: > For the records - I've added a comment in > > https://bugs.openjdk.java.net/browse/JDK-8039262 > > Reporting in as much detail as possible the underlying cause of the > performance regression. > > Maurizio > > > On 29/04/15 18:38, Jan Lahoda wrote: > >> In 9, the Scope listeners are (AFAIK) only needed to update the "mark" >> (so that the Types caches can detect obsolete entries). I think we may be >> able to avoid the listeners at the cost of slowing down getMark somewhat >> (getMark would ask the sub-scopes for their marks and sum the result). I'll >> try. >> >> Jan >> >> On 29.4.2015 19:16, Maurizio Cimadamore wrote: >> >>> Found it. >>> >>> The big offender is the listeners field in the CompoundScope (which is a >>> list). >>> >>> The test ends up creating huge listeners lists - probably because of >>> very deep inheritance hierarchies. >>> >>> If I avoid adding stuff to the listeners field in the compound scope, I >>> get back to a sane scenario: >>> >>> real 0m32.540s >>> user 1m15.298s >>> sys 0m1.126s >>> >>> >>> And the profiled memory usage is much more under control. >>> >>> So, looks like we need a way to prevent these listeners list to >>> overwhelm javac ;-) >>> >>> Maurizio >>> >>> On 29/04/15 17:51, Vicente-Arturo Romero-Zaldivar wrote: >>> >>>> On 04/29/2015 09:43 AM, Maurizio Cimadamore wrote: >>>> >>>>> >>>>> >>>>> On 29/04/15 17:01, Jan Lahoda wrote: >>>>> >>>>>> On 29.4.2015 16:06, Jacob Wieland wrote: >>>>>> >>>>>>> I have to admit, the reproducer is only a small part of the actual >>>>>>> generated code. In my preliminary tests, it already sufficed to show >>>>>>> a >>>>>>> difference (also, the smaller code still worked with 32 bit while the >>>>>>> whole code runs out of memory in all 32 bit versions which makes >>>>>>> comparison much harder ;-) and which is why I need the 64 bit >>>>>>> versions). >>>>>>> If I test with the complete code which is much bigger, the results >>>>>>> are >>>>>>> as follows: >>>>>>> >>>>>>> jdk1.6u45(64bit) 2GB MaxMem - 1:30 minutes >>>>>>> jdk1.7u75(64bit) 2GB MaxMem - > 6 min >>>>>>> jdk1.8u31(64bit) 1GB MaxMem - > 15 min >>>>>>> jdk1.8u31(64bit) 2GB MaxMem - > 10 min >>>>>>> jdk1.8u31(64bit) 4GB MaxMem - 2:20 min (-source/-target 6 does not >>>>>>> seem >>>>>>> to have any effect) >>>>>>> >>>>>>> So, if you throw insane (in comparison with 1.6) amounts of memory at >>>>>>> 1.7/1.8, it is only about a third as slow, but this still is >>>>>>> unacceptable. I actually think it has to do with parallellization and >>>>>>> >>>>>> >>>>>> When I was looking at JDK-8039262, a significant contributing factor >>>>>> to the slowdown (with enough memory and -source/-target 6) appeared >>>>>> to be Check.checkOverrideClashes - I believe this does checks that >>>>>> were not properly implemented in 6, contributing to the difference >>>>>> between 6 and 7 (which seems to be particularly visible for this >>>>>> testcase). I was looking at the checks a few times, trying to write >>>>>> them differently/faster while still performing the checks, but was >>>>>> not successful in that yet, unfortunately. >>>>>> >>>>> I see the issue now - it is reproducible with the following memory >>>>> parameter (at least in my machine): >>>>> >>>>> -J-Xmx768M >>>>> >>>>> This give around 20sec in JDK 6 and 10+ minutes in JDK 8. >>>>> >>>>> All the time seems to be spent in desugaring, most specifically in >>>>> TransTypes.addBridges - it seems like that method calls >>>>> Types.implementation a lot - so my theory was that the fact that >>>>> javac consumes more memory, forces the GC to get rid of the cached >>>>> entries in the implementation/members closure caches (since such >>>>> entries are deliberately backed up by SoftReferences), which in turn >>>>> completely trashes performances. I instrumented the code a bit and >>>>> this is what I found: >>>>> >>>>> *) With -Xmx768M >>>>> >>>>> Impl cache misses = 3346926 >>>>> Members cache misses = 1042678 >>>>> >>>>> real 7m0.335s >>>>> user 25m51.517s >>>>> sys 0m4.947s >>>>> >>>>> >>>>> *) W/o -Xmx768M >>>>> >>>>> Impl cache misses = 3346839 >>>>> Members cache misses = 1042678 >>>>> >>>>> real 0m32.377s >>>>> user 1m25.881s >>>>> sys 0m2.232s >>>>> >>>>> Long story short - cache misses do not play a factor in here - there >>>>> are some minor differences, but nothing out of the ordinary and defo >>>>> nothing that would explain a multi-minute slowdown! Any ideas? >>>>> >>>> >>>> use flight recorder? >>>> >>>> Vicente >>>> >>>> >>>>> Maurizio >>>>> >>>>>> >>>>>> Jan >>>>>> >>>>>> garbage collection. >>>>>>> >>>>>>> 2015-04-29 15:12 GMT+02:00 Maurizio Cimadamore >>>>>>> >>>>>> >: >>>>>>> >>>>>>> On 29/04/15 12:44, Jacob Wieland wrote: >>>>>>> >>>>>>>> Hello Maurizio, >>>>>>>> >>>>>>>> are you sure that you used the 64bit versions of javac? I could >>>>>>>> only observe the behavior with these. >>>>>>>> >>>>>>> Yep I'm on a Ubuntu x64 machine. It's actually pretty standard >>>>>>> hardware too - i.e. intel i5 (two cores, but OS sees 4 because of >>>>>>> hyper-threading). >>>>>>> >>>>>>>> Also, I just tried with jdk8u31-64b and it takes AGES (still >>>>>>>> running after 17 minutes where the jdk6 was done after 2), top >>>>>>>> shows 4GB VIRT memory use and 350 % load (on a 4core processor). >>>>>>>> >>>>>>> Maybe the reproducer you sent was incorrect? >>>>>>> >>>>>>> Maurizio >>>>>>> >>>>>>> >>>>>>>> So, I don't think it was that problem. >>>>>>>> >>>>>>>> 2015-04-29 12:00 GMT+02:00 Maurizio Cimadamore >>>>>>>> >>>>>>> >: >>>>>>>> >>>>>>>> These are the numbers I'm getting: >>>>>>>> >>>>>>>> JDK 9 (b42) >>>>>>>> >>>>>>>> Note: generated_ttcn/TTCN3_CommonDefs.java uses or >>>>>>>> overrides a >>>>>>>> deprecated API. >>>>>>>> Note: Recompile with -Xlint:deprecation for details. >>>>>>>> >>>>>>>> real 0m46.306s >>>>>>>> user 2m17.489s >>>>>>>> sys 0m2.166s >>>>>>>> >>>>>>>> JDK 8 (GA) >>>>>>>> >>>>>>>> Note: generated_ttcn/TTCN3_CommonDefs.java uses or >>>>>>>> overrides a >>>>>>>> deprecated API. >>>>>>>> Note: Recompile with -Xlint:deprecation for details. >>>>>>>> >>>>>>>> real 6m58.748s >>>>>>>> user 8m43.546s >>>>>>>> sys 0m2.132s >>>>>>>> >>>>>>>> JDK 7 (1.7.0_79) >>>>>>>> >>>>>>>> Note: generated_ttcn/TTCN3_CommonDefs.java uses or >>>>>>>> overrides a >>>>>>>> deprecated API. >>>>>>>> Note: Recompile with -Xlint:deprecation for details. >>>>>>>> >>>>>>>> real 0m28.341s >>>>>>>> user 1m17.194s >>>>>>>> sys 0m1.886s >>>>>>>> >>>>>>>> >>>>>>>> As you can see there is a significant regression from JDK >>>>>>>> 7 to >>>>>>>> JDK 8 which was caused by >>>>>>>> >>>>>>>> https://bugs.openjdk.java.net/browse/JDK-8043253 >>>>>>>> >>>>>>>> (some stack trace analysis revealed the familiar pattern). >>>>>>>> This has also been fixed in JDK 8u20 (as stated in the bug >>>>>>>> evaluation). >>>>>>>> >>>>>>>> So, while JDK 8u20/9 is slower than JDK 7 (at least on my >>>>>>>> machine), the numbers are more or less in the same ballpark >>>>>>>> and the huge regression that was visible in earlier JDK 8 >>>>>>>> releases has now been fixed. >>>>>>>> >>>>>>>> If you are still experiencing the problem - can you please >>>>>>>> also submit the specific compiler versions you are using in >>>>>>>> your benchmark? >>>>>>>> >>>>>>>> Maurizio >>>>>>>> >>>>>>>> >>>>>>>> >>>>>>>> On 29/04/15 10:29, Maurizio Cimadamore wrote: >>>>>>>> >>>>>>>>> Hi Jacob, >>>>>>>>> Stay assured - as we'll definitively look into this issue >>>>>>>>> (I >>>>>>>>> see it's already assigned to one of my colleagues). >>>>>>>>> >>>>>>>>> What I can say (w/o looking too much at the attached >>>>>>>>> artifacts) is that in general, javac has no issue with >>>>>>>>> compiling a lot of sources at once; at one point the build >>>>>>>>> system was structured in such a way that all the JDK >>>>>>>>> classes >>>>>>>>> were compiled at once - and that (which is way more than >>>>>>>>> your >>>>>>>>> 187 sources - i.e. at least 10x that) took less than 20 >>>>>>>>> seconds. SO there must some specific pattern triggering >>>>>>>>> that >>>>>>>>> issue. >>>>>>>>> >>>>>>>>> Given that you say you have 187 input sources and 48K >>>>>>>>> output >>>>>>>>> classes, I'd say you are using inner classes a lot. I >>>>>>>>> wonder >>>>>>>>> if you are hitting this: >>>>>>>>> >>>>>>>>> https://bugs.openjdk.java.net/browse/JDK-8000316 >>>>>>>>> >>>>>>>>> Maurizio >>>>>>>>> >>>>>>>>> On 24/04/15 09:49, Jacob Wieland wrote: >>>>>>>>> >>>>>>>>>> Hello Folks, >>>>>>>>>> >>>>>>>>>> >>>>>>>>>> I still have the open problem >>>>>>>>>> https://bugs.openjdk.java.net/browse/JDK-8039262 that the >>>>>>>>>> javac performance has degraded significantly from 1.6 to >>>>>>>>>> 1.7 >>>>>>>>>> (and even worse to 1.8) in the 64bit versions. Since in >>>>>>>>>> our >>>>>>>>>> context, we are dealing with a lot of generated source1.4 >>>>>>>>>> Java input (either split into very large files with inner >>>>>>>>>> classes or big packages with lots of smaller classes), >>>>>>>>>> compiler performance is critical for our tool and this >>>>>>>>>> degradation forces us to continue recommending to our >>>>>>>>>> customers to use Java 1.6 for large projects (as is the >>>>>>>>>> norm) as 1.7 and 1.8 are pretty much unusable in this >>>>>>>>>> respect. >>>>>>>>>> >>>>>>>>>> Is anyone still working on this issue or is such >>>>>>>>>> significant >>>>>>>>>> performance degradation not a serious issue? >>>>>>>>>> >>>>>>>>>> My observations so far are: >>>>>>>>>> - it gets worse the more class files are being >>>>>>>>>> compiled/the >>>>>>>>>> more files reside in the source path >>>>>>>>>> - cpu usage goes through the roof over all available >>>>>>>>>> kernels >>>>>>>>>> - memory usage is much higher >>>>>>>>>> - garbage collection seems to be much more active >>>>>>>>>> >>>>>>>>>> Using -proc:none alleviates the problem slightly for the >>>>>>>>>> 1.7 >>>>>>>>>> version, but not for 1.8 (last we tested with >>>>>>>>>> jdk1.8.0_31) where the performance difference is a factor >>>>>>>>>> 5 >>>>>>>>>> or more! >>>>>>>>>> >>>>>>>>>> I can understand that a more advanced compiler has >>>>>>>>>> capabilities that a previous version does not have and >>>>>>>>>> thus >>>>>>>>>> sometimes has >>>>>>>>>> to do more work. But, it should still be possible >>>>>>>>>> (especially if given the -source 1.4 or -source 1.5 option >>>>>>>>>> as we do) to optimize it in such a way that unnecessary >>>>>>>>>> checks for generics, overriding methods, closures, >>>>>>>>>> annotations and other newer features can be turned off (if >>>>>>>>>> they are to blame, which I actually doubt from my >>>>>>>>>> observations). >>>>>>>>>> >>>>>>>>>> I would really appreciate your help in this regard and I >>>>>>>>>> think everyone would benefit from any bugfix you can offer >>>>>>>>>> for this. >>>>>>>>>> >>>>>>>>>> BR, Jacob Wieland >>>>>>>>>> >>>>>>>>>> -- >>>>>>>>>> -- >>>>>>>>>> ------------------------------ >>>>>>>>>> ------------------------------------------- >>>>>>>>>> Dr. Jacob Wieland >>>>>>>>>> Senior Software Engineer >>>>>>>>>> >>>>>>>>>> Testing Technologies IST GmbH >>>>>>>>>> Michaelkirchstra?e 17/18 >>>>>>>>>> 10179 Berlin, Germany >>>>>>>>>> >>>>>>>>>> Phone +49 30 726 19 19 34 Email >>>>>>>>>> wieland at testingtech.com >>>>>>>>>> Fax +49 30 726 19 19 20 Internet www.testingtech.com >>>>>>>>>> >>>>>>>>>> --------------------------------------------------------------------------------------------------------------- >>>>>>>>>> >>>>>>>>>> >>>>>>>>>> >>>>>>>>>> ----------------------------------------------------------------------------------------------------------------- >>>>>>>>>> >>>>>>>>>> >>>>>>>>>> >>>>>>>>>> UPCOMING EVENTS >>>>>>>>>> >>>>>>>>>> SUBMIT YOUR TOPIC for the UCAAT 2015 >>>>>>>>>> Deadline: May 30, 2015 >>>>>>>>>> ucaat.etsi.org/2015/CallForPresentations.html >>>>>>>>>> >>>>>>>>>> >>>>>>>>>> Apr 21-23, 2015 | SAE Conference & Exhibition >>>>>>>>>> Detroit, Michigan, USA >>>>>>>>>> www.sae.org/congress/ >>>>>>>>>> >>>>>>>>>> Apr 28-30, 2015 | iqnite >>>>>>>>>> Dusseldorf, Germany >>>>>>>>>> www.iqnite-conferences.com/de/index.aspx >>>>>>>>>> >>>>>>>>>> ----------------------------------------------------------------------------------------------------------------- >>>>>>>>>> >>>>>>>>>> >>>>>>>>>> Gesch?ftsf?hrung: Theofanis Vassiliou-Gioles, Stephan >>>>>>>>>> Pietsch, Pete Nicholson Handelsregister HRB 77805 B, >>>>>>>>>> Amtsgericht Charlottenburg Ust ID Nr.: DE 813 143 070 This >>>>>>>>>> email may contain confidential and privileged material for >>>>>>>>>> the sole use of the intended recipient. Any review, use, >>>>>>>>>> distribution or disclosure by others is strictly >>>>>>>>>> prohibited. >>>>>>>>>> If you are not the intended recipient (or authorized to >>>>>>>>>> receive for the recipient), please contact the sender by >>>>>>>>>> reply email and delete all copies of this message. >>>>>>>>>> >>>>>>>>> >>>>>>>>> >>>>>>>> >>>>>>>> >>>>>>>> >>>>>>>> -- >>>>>>>> -- >>>>>>>> ------------------------------ >>>>>>>> ------------------------------------------- >>>>>>>> Dr. Jacob Wieland >>>>>>>> Senior Software Engineer >>>>>>>> >>>>>>>> Testing Technologies IST GmbH >>>>>>>> Michaelkirchstra?e 17/18 >>>>>>>> 10179 Berlin, Germany >>>>>>>> >>>>>>>> Phone +49 30 726 19 19 34 Email wieland at testingtech.com >>>>>>>> >>>>>>>> Fax +49 30 726 19 19 20 Internet www.testingtech.com >>>>>>>> >>>>>>>> --------------------------------------------------------------------------------------------------------------- >>>>>>>> >>>>>>>> >>>>>>>> >>>>>>>> ----------------------------------------------------------------------------------------------------------------- >>>>>>>> >>>>>>>> >>>>>>>> >>>>>>>> UPCOMING EVENTS >>>>>>>> >>>>>>>> SUBMIT YOUR TOPIC for the UCAAT 2015 >>>>>>>> Deadline: May 30, 2015 >>>>>>>> ucaat.etsi.org/2015/CallForPresentations.html >>>>>>>> >>>>>>>> >>>>>>>> Apr 21-23, 2015 | SAE Conference & Exhibition >>>>>>>> Detroit, Michigan, USA >>>>>>>> www.sae.org/congress/ >>>>>>>> >>>>>>>> Apr 28-30, 2015 | iqnite >>>>>>>> Dusseldorf, Germany >>>>>>>> www.iqnite-conferences.com/de/index.aspx >>>>>>>> >>>>>>>> ----------------------------------------------------------------------------------------------------------------- >>>>>>>> >>>>>>>> >>>>>>>> Gesch?ftsf?hrung: Theofanis Vassiliou-Gioles, Stephan Pietsch, >>>>>>>> Pete Nicholson Handelsregister HRB 77805 B, Amtsgericht >>>>>>>> Charlottenburg Ust ID Nr.: DE 813 143 070 This email may contain >>>>>>>> confidential and privileged material for the sole use of the >>>>>>>> intended recipient. Any review, use, distribution or >>>>>>>> disclosure by >>>>>>>> others is strictly prohibited. If you are not the intended >>>>>>>> recipient (or authorized to receive for the recipient), please >>>>>>>> contact the sender by reply email and delete all copies of this >>>>>>>> message. >>>>>>>> >>>>>>> >>>>>>> >>>>>>> >>>>>>> >>>>>>> -- >>>>>>> -- >>>>>>> ------------------------------ >>>>>>> ------------------------------------------- >>>>>>> Dr. Jacob Wieland >>>>>>> Senior Software Engineer >>>>>>> >>>>>>> Testing Technologies IST GmbH >>>>>>> Michaelkirchstra?e 17/18 >>>>>>> 10179 Berlin, Germany >>>>>>> >>>>>>> Phone +49 30 726 19 19 34 Email wieland at testingtech.com >>>>>>> >>>>>>> Fax +49 30 726 19 19 20 Internet www.testingtech.com >>>>>>> >>>>>>> --------------------------------------------------------------------------------------------------------------- >>>>>>> >>>>>>> >>>>>>> >>>>>>> ----------------------------------------------------------------------------------------------------------------- >>>>>>> >>>>>>> >>>>>>> >>>>>>> UPCOMING EVENTS >>>>>>> >>>>>>> SUBMIT YOUR TOPIC for the UCAAT 2015 >>>>>>> Deadline: May 30, 2015 >>>>>>> ucaat.etsi.org/2015/CallForPresentations.html >>>>>>> >>>>>>> >>>>>>> Apr 21-23, 2015 | SAE Conference & Exhibition >>>>>>> Detroit, Michigan, USA >>>>>>> www.sae.org/congress/ >>>>>>> >>>>>>> Apr 28-30, 2015 | iqnite >>>>>>> Dusseldorf, Germany >>>>>>> www.iqnite-conferences.com/de/index.aspx >>>>>>> >>>>>>> >>>>>>> >>>>>>> ----------------------------------------------------------------------------------------------------------------- >>>>>>> >>>>>>> >>>>>>> Gesch?ftsf?hrung: Theofanis Vassiliou-Gioles, Stephan Pietsch, Pete >>>>>>> Nicholson Handelsregister HRB 77805 B, Amtsgericht Charlottenburg >>>>>>> Ust ID >>>>>>> Nr.: DE 813 143 070 This email may contain confidential and >>>>>>> privileged >>>>>>> material for the sole use of the intended recipient. Any review, use, >>>>>>> distribution or disclosure by others is strictly prohibited. If you >>>>>>> are >>>>>>> not the intended recipient (or authorized to receive for the >>>>>>> recipient), >>>>>>> please contact the sender by reply email and delete all copies of >>>>>>> this >>>>>>> message. >>>>>>> >>>>>> >>>>> >>>> >>> > -- -- ------------------------------ ------------------------------------------- Dr. Jacob Wieland Senior Software Engineer Testing Technologies IST GmbH Michaelkirchstra?e 17/18 10179 Berlin, Germany Phone +49 30 726 19 19 34 Email wieland at testingtech.com Fax +49 30 726 19 19 20 Internet www.testingtech.com --------------------------------------------------------------------------------------------------------------- ----------------------------------------------------------------------------------------------------------------- UPCOMING EVENTS SUBMIT YOUR TOPIC for the UCAAT 2015 Deadline: May 30, 2015ucaat.etsi.org/2015/CallForPresentations.html Apr 21-23, 2015 | SAE Conference & Exhibition Detroit, Michigan, USAwww.sae.org/congress/ Apr 28-30, 2015 | iqnite Dusseldorf, Germanywww.iqnite-conferences.com/de/index.aspx ----------------------------------------------------------------------------------------------------------------- Gesch?ftsf?hrung: Theofanis Vassiliou-Gioles, Stephan Pietsch, Pete Nicholson Handelsregister HRB 77805 B, Amtsgericht Charlottenburg Ust ID Nr.: DE 813 143 070 This email may contain confidential and privileged material for the sole use of the intended recipient. Any review, use, distribution or disclosure by others is strictly prohibited. If you are not the intended recipient (or authorized to receive for the recipient), please contact the sender by reply email and delete all copies of this message. -------------- next part -------------- An HTML attachment was scrubbed... URL: From maurizio.cimadamore at oracle.com Thu Apr 30 15:51:50 2015 From: maurizio.cimadamore at oracle.com (Maurizio Cimadamore) Date: Thu, 30 Apr 2015 16:51:50 +0100 Subject: Java Performance Degradation in JDK7 and JDK8 In-Reply-To: References: <5540A479.7010401@oracle.com> <5540ABCD.3080708@oracle.com> <5540D8B0.3070604@oracle.com> <55410052.5080205@oracle.com> <55410A1A.1090104@oracle.com> <55410C10.9010108@oracle.com> <554111E1.10305@oracle.com> <55411708.3020109@oracle.com> <5542443E.6030803@oracle.com> Message-ID: <55424F96.8050705@oracle.com> On 30/04/15 16:15, Jacob Wieland wrote: > thank you very much - very interesting. > > is there any estimate when a patch for 7/8 will be available? We are currently playing with different patches - we will update this discussion when we know more about which steps will be taken. Regards Maurizio > > 2015-04-30 17:03 GMT+02:00 Maurizio Cimadamore > >: > > For the records - I've added a comment in > > https://bugs.openjdk.java.net/browse/JDK-8039262 > > Reporting in as much detail as possible the underlying cause of > the performance regression. > > Maurizio > > > On 29/04/15 18:38, Jan Lahoda wrote: > > In 9, the Scope listeners are (AFAIK) only needed to update > the "mark" (so that the Types caches can detect obsolete > entries). I think we may be able to avoid the listeners at the > cost of slowing down getMark somewhat (getMark would ask the > sub-scopes for their marks and sum the result). I'll try. > > Jan > > On 29.4.2015 19:16, Maurizio Cimadamore wrote: > > Found it. > > The big offender is the listeners field in the > CompoundScope (which is a > list). > > The test ends up creating huge listeners lists - probably > because of > very deep inheritance hierarchies. > > If I avoid adding stuff to the listeners field in the > compound scope, I > get back to a sane scenario: > > real 0m32.540s > user 1m15.298s > sys 0m1.126s > > > And the profiled memory usage is much more under control. > > So, looks like we need a way to prevent these listeners > list to > overwhelm javac ;-) > > Maurizio > > On 29/04/15 17:51, Vicente-Arturo Romero-Zaldivar wrote: > > On 04/29/2015 09:43 AM, Maurizio Cimadamore wrote: > > > > On 29/04/15 17:01, Jan Lahoda wrote: > > On 29.4.2015 16:06, Jacob Wieland wrote: > > I have to admit, the reproducer is only a > small part of the actual > generated code. In my preliminary tests, > it already sufficed to show a > difference (also, the smaller code still > worked with 32 bit while the > whole code runs out of memory in all 32 > bit versions which makes > comparison much harder ;-) and which is > why I need the 64 bit > versions). > If I test with the complete code which is > much bigger, the results are > as follows: > > jdk1.6u45(64bit) 2GB MaxMem - 1:30 minutes > jdk1.7u75(64bit) 2GB MaxMem - > 6 min > jdk1.8u31(64bit) 1GB MaxMem - > 15 min > jdk1.8u31(64bit) 2GB MaxMem - > 10 min > jdk1.8u31(64bit) 4GB MaxMem - 2:20 min > (-source/-target 6 does not > seem > to have any effect) > > So, if you throw insane (in comparison > with 1.6) amounts of memory at > 1.7/1.8, it is only about a third as slow, > but this still is > unacceptable. I actually think it has to > do with parallellization and > > > When I was looking at JDK-8039262, a > significant contributing factor > to the slowdown (with enough memory and > -source/-target 6) appeared > to be Check.checkOverrideClashes - I believe > this does checks that > were not properly implemented in 6, > contributing to the difference > between 6 and 7 (which seems to be > particularly visible for this > testcase). I was looking at the checks a few > times, trying to write > them differently/faster while still performing > the checks, but was > not successful in that yet, unfortunately. > > I see the issue now - it is reproducible with the > following memory > parameter (at least in my machine): > > -J-Xmx768M > > This give around 20sec in JDK 6 and 10+ minutes in > JDK 8. > > All the time seems to be spent in desugaring, most > specifically in > TransTypes.addBridges - it seems like that method > calls > Types.implementation a lot - so my theory was that > the fact that > javac consumes more memory, forces the GC to get > rid of the cached > entries in the implementation/members closure > caches (since such > entries are deliberately backed up by > SoftReferences), which in turn > completely trashes performances. I instrumented > the code a bit and > this is what I found: > > *) With -Xmx768M > > Impl cache misses = 3346926 > Members cache misses = 1042678 > > real 7m0.335s > user 25m51.517s > sys 0m4.947s > > > *) W/o -Xmx768M > > Impl cache misses = 3346839 > Members cache misses = 1042678 > > real 0m32.377s > user 1m25.881s > sys 0m2.232s > > Long story short - cache misses do not play a > factor in here - there > are some minor differences, but nothing out of the > ordinary and defo > nothing that would explain a multi-minute > slowdown! Any ideas? > > > use flight recorder? > > Vicente > > > Maurizio > > > Jan > > garbage collection. > > 2015-04-29 15:12 GMT+02:00 Maurizio Cimadamore > > >>: > > On 29/04/15 12:44, Jacob Wieland wrote: > > Hello Maurizio, > > are you sure that you used the > 64bit versions of javac? I could > only observe the behavior with these. > > Yep I'm on a Ubuntu x64 machine. It's > actually pretty standard > hardware too - i.e. intel i5 (two > cores, but OS sees 4 because of > hyper-threading). > > Also, I just tried with > jdk8u31-64b and it takes AGES (still > running after 17 minutes where the > jdk6 was done after 2), top > shows 4GB VIRT memory use and 350 > % load (on a 4core processor). > > Maybe the reproducer you sent was > incorrect? > > Maurizio > > > So, I don't think it was that problem. > > 2015-04-29 12:00 GMT+02:00 > Maurizio Cimadamore > > > >>: > > These are the numbers I'm getting: > > JDK 9 (b42) > > Note: > generated_ttcn/TTCN3_CommonDefs.java > uses or > overrides a > deprecated API. > Note: Recompile with > -Xlint:deprecation for details. > > real 0m46.306s > user 2m17.489s > sys 0m2.166s > > JDK 8 (GA) > > Note: > generated_ttcn/TTCN3_CommonDefs.java > uses or > overrides a > deprecated API. > Note: Recompile with > -Xlint:deprecation for details. > > real 6m58.748s > user 8m43.546s > sys 0m2.132s > > JDK 7 (1.7.0_79) > > Note: > generated_ttcn/TTCN3_CommonDefs.java > uses or > overrides a > deprecated API. > Note: Recompile with > -Xlint:deprecation for details. > > real 0m28.341s > user 1m17.194s > sys 0m1.886s > > > As you can see there is a > significant regression from JDK > 7 to > JDK 8 which was caused by > > https://bugs.openjdk.java.net/browse/JDK-8043253 > > (some stack trace analysis > revealed the familiar pattern). > This has also been fixed in > JDK 8u20 (as stated in the bug > evaluation). > > So, while JDK 8u20/9 is slower > than JDK 7 (at least on my > machine), the numbers are more > or less in the same ballpark > and the huge regression that > was visible in earlier JDK 8 > releases has now been fixed. > > If you are still experiencing > the problem - can you please > also submit the specific > compiler versions you are using in > your benchmark? > > Maurizio > > > > On 29/04/15 10:29, Maurizio > Cimadamore wrote: > > Hi Jacob, > Stay assured - as we'll > definitively look into this issue (I > see it's already assigned > to one of my colleagues). > > What I can say (w/o > looking too much at the attached > artifacts) is that in > general, javac has no issue with > compiling a lot of sources > at once; at one point the build > system was structured in > such a way that all the JDK classes > were compiled at once - > and that (which is way more than > your > 187 sources - i.e. at > least 10x that) took less than 20 > seconds. SO there must > some specific pattern triggering that > issue. > > Given that you say you > have 187 input sources and 48K output > classes, I'd say you are > using inner classes a lot. I wonder > if you are hitting this: > > https://bugs.openjdk.java.net/browse/JDK-8000316 > > Maurizio > > On 24/04/15 09:49, Jacob > Wieland wrote: > > Hello Folks, > > > I still have the open > problem > https://bugs.openjdk.java.net/browse/JDK-8039262 > that the > javac performance has > degraded significantly from 1.6 to > 1.7 > (and even worse to > 1.8) in the 64bit versions. > Since in our > context, we are > dealing with a lot of > generated source1.4 > Java input (either > split into very large files > with inner > classes or big > packages with lots of smaller > classes), > compiler performance > is critical for our tool and this > degradation forces us > to continue recommending to our > customers to use Java > 1.6 for large projects (as is the > norm) as 1.7 and 1.8 > are pretty much unusable in this > respect. > > Is anyone still > working on this issue or is such > significant > performance > degradation not a serious issue? > > My observations so far > are: > - it gets worse the > more class files are being > compiled/the > more files reside in > the source path > - cpu usage goes > through the roof over all > available > kernels > - memory usage is much > higher > - garbage collection > seems to be much more active > > Using -proc:none > alleviates the problem > slightly for the > 1.7 > version, but not for > 1.8 (last we tested with > jdk1.8.0_31) where the > performance difference is a > factor 5 > or more! > > I can understand that > a more advanced compiler has > capabilities that a > previous version does not have > and thus > sometimes has > to do more work. But, > it should still be possible > (especially if given > the -source 1.4 or -source 1.5 > option > as we do) to optimize > it in such a way that unnecessary > checks for generics, > overriding methods, closures, > annotations and other > newer features can be turned > off (if > they are to blame, > which I actually doubt from my > observations). > > I would really > appreciate your help in this > regard and I > think everyone would > benefit from any bugfix you > can offer > for this. > > BR, Jacob Wieland > > -- > -- > > ------------------------------ > ------------------------------------------- > Dr. Jacob Wieland > Senior Software Engineer > > Testing Technologies > IST GmbH > Michaelkirchstra?e 17/18 > 10179 Berlin, Germany > > Phone +49 30 726 19 19 > 34 > > Email > wieland at testingtech.com > > > > Fax +49 30 726 19 19 > 20 > > Internet > www.testingtech.com > > > > --------------------------------------------------------------------------------------------------------------- > > > > ----------------------------------------------------------------------------------------------------------------- > > > > UPCOMING EVENTS > > SUBMIT YOUR TOPIC for > the UCAAT 2015 > Deadline: May 30, 2015 > ucaat.etsi.org/2015/CallForPresentations.html > > > > Apr 21-23, 2015 | SAE > Conference & Exhibition > Detroit, Michigan, USA > www.sae.org/congress/ > > > > Apr 28-30, 2015 | iqnite > Dusseldorf, Germany > www.iqnite-conferences.com/de/index.aspx > > > ----------------------------------------------------------------------------------------------------------------- > > > Gesch?ftsf?hrung: > Theofanis Vassiliou-Gioles, > Stephan > Pietsch, Pete > Nicholson Handelsregister HRB > 77805 B, > Amtsgericht > Charlottenburg Ust ID Nr.: DE > 813 143 070 This > email may contain > confidential and privileged > material for > the sole use of the > intended recipient. Any > review, use, > distribution or > disclosure by others is strictly > prohibited. > If you are not the > intended recipient (or > authorized to > receive for the > recipient), please contact the > sender by > reply email and delete > all copies of this message. > > > > > > > -- > -- > ------------------------------ > ------------------------------------------- > Dr. Jacob Wieland > Senior Software Engineer > > Testing Technologies IST GmbH > Michaelkirchstra?e 17/18 > 10179 Berlin, Germany > > Phone +49 30 726 19 19 34 > > Email wieland at testingtech.com > > > > Fax +49 30 726 19 19 20 > > Internet www.testingtech.com > > > --------------------------------------------------------------------------------------------------------------- > > > > ----------------------------------------------------------------------------------------------------------------- > > > > UPCOMING EVENTS > > SUBMIT YOUR TOPIC for the UCAAT 2015 > Deadline: May 30, 2015 > ucaat.etsi.org/2015/CallForPresentations.html > > > > Apr 21-23, 2015 | SAE Conference & > Exhibition > Detroit, Michigan, USA > www.sae.org/congress/ > > > > Apr 28-30, 2015 | iqnite > Dusseldorf, Germany > www.iqnite-conferences.com/de/index.aspx > > > ----------------------------------------------------------------------------------------------------------------- > > > Gesch?ftsf?hrung: Theofanis > Vassiliou-Gioles, Stephan Pietsch, > Pete Nicholson Handelsregister HRB > 77805 B, Amtsgericht > Charlottenburg Ust ID Nr.: DE 813 > 143 070 This email may contain > confidential and privileged > material for the sole use of the > intended recipient. Any review, > use, distribution or > disclosure by > others is strictly prohibited. If > you are not the intended > recipient (or authorized to > receive for the recipient), please > contact the sender by reply email > and delete all copies of this > message. > > > > > > -- > -- > ------------------------------ > ------------------------------------------- > Dr. Jacob Wieland > Senior Software Engineer > > Testing Technologies IST GmbH > Michaelkirchstra?e 17/18 > 10179 Berlin, Germany > > Phone +49 30 726 19 19 34 > > Email wieland at testingtech.com > > > > Fax +49 30 726 19 19 20 > > Internet www.testingtech.com > > > --------------------------------------------------------------------------------------------------------------- > > > > ----------------------------------------------------------------------------------------------------------------- > > > > UPCOMING EVENTS > > SUBMIT YOUR TOPIC for the UCAAT 2015 > Deadline: May 30, 2015 > ucaat.etsi.org/2015/CallForPresentations.html > > > > Apr 21-23, 2015 | SAE Conference & Exhibition > Detroit, Michigan, USA > www.sae.org/congress/ > > > > Apr 28-30, 2015 | iqnite > Dusseldorf, Germany > www.iqnite-conferences.com/de/index.aspx > > > > > ----------------------------------------------------------------------------------------------------------------- > > > Gesch?ftsf?hrung: Theofanis > Vassiliou-Gioles, Stephan Pietsch, Pete > Nicholson Handelsregister HRB 77805 B, > Amtsgericht Charlottenburg > Ust ID > Nr.: DE 813 143 070 This email may contain > confidential and privileged > material for the sole use of the intended > recipient. Any review, use, > distribution or disclosure by others is > strictly prohibited. If you > are > not the intended recipient (or authorized > to receive for the > recipient), > please contact the sender by reply email > and delete all copies of this > message. > > > > > > > > > -- > -- > ------------------------------ > ------------------------------------------- > Dr. Jacob Wieland > Senior Software Engineer > > Testing Technologies IST GmbH > Michaelkirchstra?e 17/18 > 10179 Berlin, Germany > > Phone +49 30 726 19 19 34 Email wieland at testingtech.com > > Fax +49 30 726 19 19 20 Internet www.testingtech.com > > --------------------------------------------------------------------------------------------------------------- > > ----------------------------------------------------------------------------------------------------------------- > > UPCOMING EVENTS > > SUBMIT YOUR TOPIC for the UCAAT 2015 > Deadline: May 30, 2015 > ucaat.etsi.org/2015/CallForPresentations.html > > Apr 21-23, 2015 | SAE Conference & Exhibition > Detroit, Michigan, USA > www.sae.org/congress/ > > Apr 28-30, 2015 | iqnite > Dusseldorf, Germany > www.iqnite-conferences.com/de/index.aspx > ----------------------------------------------------------------------------------------------------------------- > Gesch?ftsf?hrung: Theofanis Vassiliou-Gioles, Stephan Pietsch, Pete > Nicholson Handelsregister HRB 77805 B, Amtsgericht Charlottenburg Ust > ID Nr.: DE 813 143 070 This email may contain confidential and > privileged material for the sole use of the intended recipient. Any > review, use, distribution or disclosure by others is strictly > prohibited. If you are not the intended recipient (or authorized to > receive for the recipient), please contact the sender by reply email > and delete all copies of this message. -------------- next part -------------- An HTML attachment was scrubbed... URL: