From headius at headius.com  Thu Dec  1 00:23:57 2011
From: headius at headius.com (Charles Oliver Nutter)
Date: Thu, 1 Dec 2011 02:23:57 -0600
Subject: JRuby invokedynamic updates for November
In-Reply-To: <p06240813cafcaa3b89c7@192.168.1.122>
References: <CAE-f1xT_Ku+4OhN1cCCXVO5Lanv=g_NFUxHi3n17mCMma1Jv2A@mail.gmail.com>
	<p0624080acafc352919a0@192.168.1.122>
	<CAE-f1xQiqyfWMqwTVSQRx5BFPgVAb8447YDHk_xwGLyxBi=AGQ@mail.gmail.com>
	<CAE-f1xRuf+SaZjriTFe_C3EPesj7zYH=u+i3h4VWGysuABwigg@mail.gmail.com>
	<p0624080ecafc4c3d8251@192.168.1.122>
	<CAE-f1xQ9xuOJ358aOe-=_=_Awm+eVwOa9ZdYBY_z23S4H9LQRw@mail.gmail.com>
	<p06240813cafcaa3b89c7@192.168.1.122>
Message-ID: <CAE-f1xSj9t==DYPZ+OD0Sy8_x-sAxoW7zEwu3xCrcjiYxPsftA@mail.gmail.com>

On Wed, Nov 30, 2011 at 9:52 PM, Stephen Bannasch
<stephen.bannasch at deanbrook.org> wrote:
> At 3:03 PM -0500 11/30/11, Stephen Bannasch wrote:
>>Benchmark code here: https://gist.github.com/1410423
>
> The hpricot benchmark is also slowed down consdierably.

I have found more issues in my invokedynamic binding logic and am in
the process of repairing them. I've pushed a couple changes to
master...can you re-test?

Here's updated REXML benchmark numbers.

*Parsing small document for 5 seconds          ... 3687 calls      (737.40/s)
Adding new elemnt for 5 seconds               ... 583686 calls    (116737.20/s)
Document creation for 5 seconds               ... 118806 calls    (23761.20/s)
Writing tree for 5 seconds                    ... 8299 calls      (1659.80/s)
By-hand search for 5 seconds                  ... 417629 calls    (83525.80/s)
XPath search for 5 seconds                    ... 1327 calls      (265.40/s)
*Parse large document for 5 seconds            ... 337 calls       (67.40/s)
Stream parsing for 5 seconds                  ... 590 calls       (118.00/s)
*Pull parsing for 5 seconds                    ... 534 calls       (106.80/s)
*SAX2 parsing for 5 seconds                    ... 170 calls       (34.00/s)
*Lightweight parsing for 5 seconds             ... 726 calls       (145.20/s)

The starred benchmarks are still slower than non-invokedynamic, but
the others are all faster...in some cases quite a bit faster. I'm
investigating the remaining anomalies...profiles don't look right,
especially on the initial "small" parse.

The five benchmarks at the bottom may be ok...they seem to vary wildly
regardless of invokedynamic, so they're probably at least on par.

I am *very* glad you brought this to my attention :) The logic I was
using to build a PIC and to completely invalidate call sites had a
critical flaw: if a PIC was built and one of those classes were
subsequently modified, the call site would get into a state where it
was repeatedly rebound each time those types were re-encountered. This
caused several heavily-hit call sites in the benchmark to rebind the
call site on nearly every invocation, which caused much of the
performance issues.

I am still seeing rebinding in profiles of some of the remaining slow
benchmarks, so I'm missing something.

- Charlie

From headius at headius.com  Thu Dec  1 01:29:08 2011
From: headius at headius.com (Charles Oliver Nutter)
Date: Thu, 1 Dec 2011 03:29:08 -0600
Subject: JRuby invokedynamic updates for November
In-Reply-To: <CAE-f1xSj9t==DYPZ+OD0Sy8_x-sAxoW7zEwu3xCrcjiYxPsftA@mail.gmail.com>
References: <CAE-f1xT_Ku+4OhN1cCCXVO5Lanv=g_NFUxHi3n17mCMma1Jv2A@mail.gmail.com>
	<p0624080acafc352919a0@192.168.1.122>
	<CAE-f1xQiqyfWMqwTVSQRx5BFPgVAb8447YDHk_xwGLyxBi=AGQ@mail.gmail.com>
	<CAE-f1xRuf+SaZjriTFe_C3EPesj7zYH=u+i3h4VWGysuABwigg@mail.gmail.com>
	<p0624080ecafc4c3d8251@192.168.1.122>
	<CAE-f1xQ9xuOJ358aOe-=_=_Awm+eVwOa9ZdYBY_z23S4H9LQRw@mail.gmail.com>
	<p06240813cafcaa3b89c7@192.168.1.122>
	<CAE-f1xSj9t==DYPZ+OD0Sy8_x-sAxoW7zEwu3xCrcjiYxPsftA@mail.gmail.com>
Message-ID: <CAE-f1xRy6AktwRKbGcMHSOCADj=8Mju=NRPk8+peoYSu0dS15A@mail.gmail.com>

On Thu, Dec 1, 2011 at 2:23 AM, Charles Oliver Nutter
<headius at headius.com> wrote:
> I am still seeing rebinding in profiles of some of the remaining slow
> benchmarks, so I'm missing something.

Bleah, I seem to have found the problem.

If I disable the use of SwitchPoint for class-mutation-originated call
site invalidation, performance matches or exceeds the non-indy numbers
pretty much across the board:

Parsing small document for 5 seconds          ... 5179 calls      (1035.80/s)
Adding new elemnt for 5 seconds               ... 582401 calls    (116480.40/s)
Document creation for 5 seconds               ... 123492 calls    (24698.40/s)
Writing tree for 5 seconds                    ... 8953 calls      (1790.60/s)
By-hand search for 5 seconds                  ... 428487 calls    (85697.40/s)
XPath search for 5 seconds                    ... 1470 calls      (294.00/s)
Parse large document for 5 seconds            ... 395 calls       (79.00/s)
Stream parsing for 5 seconds                  ... 633 calls       (126.60/s)
Pull parsing for 5 seconds                    ... 660 calls       (132.00/s)
SAX2 parsing for 5 seconds                    ... 222 calls       (44.40/s)
Lightweight parsing for 5 seconds             ... 843 calls       (168.60/s)

This is a pretty disappointing finding. I'm not yet sure whether it's
a bug in SwitchPoint optimization or in JRuby's use of SwitchPoint,
but I'd like to see your numbers with and without this flag.

- Charlie

From headius at headius.com  Thu Dec  1 01:34:45 2011
From: headius at headius.com (Charles Oliver Nutter)
Date: Thu, 1 Dec 2011 03:34:45 -0600
Subject: How to monitor SwitchPoint's impact?
Message-ID: <CAE-f1xTkez=8fVTzzm=fekwX3D3wUD4vfmBvUEtJS484WhHjwQ@mail.gmail.com>

My recent explorations into Stephen B's perf degradation on JRuby have
led me to an unfortunate conclusion: something SwitchPoint-related is
responsible for the remaining slowdown.

I had bugs I fixed, like the awful PIC-causes-repeat-rebinding issue
or the heavy-class-mutation-never-stabilizes issue, but the last "fix"
I found was to disable SwitchPoint-based call site invalidation
entirely.

Now I'm not (yet) saying this is a perf issue in SwitchPoint itself.
My problem is that it's very difficult to get any visibility into how
frequently SwitchPoints are being invalidated (though I could
instrument my own code, of course), or more importantly how
drastically those invalidations are affecting performance. Does a
single SwitchPoint invalidation happening repeatedly completely tank
performance across the entire system? Does it require SwitchPoint
invalidation that affects a broader surface area?

I will continue investigating on my side. I suspect the problem is
that there's just a handful of SwitchPoint locations that are
repeatedly invalidated, but that those invalidations are having a
drastic global impact. I'm not yet sure how to work around such a
situation, where one bad apple is spoiling the whole bunch...

- Charlie

From blackdrag at gmx.org  Thu Dec  1 03:10:44 2011
From: blackdrag at gmx.org (Jochen Theodorou)
Date: Thu, 01 Dec 2011 12:10:44 +0100
Subject: Process-level fork on OpenJDK...is it madness?
In-Reply-To: <4ED6BDC2.9060908@univ-mlv.fr>
References: <CAE-f1xTSEBEHNdVJ10iteGgu+HGiMC6_Txi7JWgwbsQes4G+0w@mail.gmail.com>	<OFB4A645FE.7D4F098B-ON88257957.006A348C-88257957.00765040@roos.com>	<4ED55B13.5090803@gmx.org>	<4ED55E03.90905@oracle.com>	<4ED56677.4060707@gmx.org>
	<4ED62958.3020307@univ-mlv.fr>	<4ED64BB8.30207@gmx.org>
	<4ED6BDC2.9060908@univ-mlv.fr>
Message-ID: <4ED760B4.9020100@gmx.org>

Am 01.12.2011 00:35, schrieb R?mi Forax:
[...]
> The only way I see to avoid that is to not load the meta-class
> until someone reference them so you can compile this example
> to System.out.println(2) and if there is a ref to a meta-class somewhere,
> discard the code and recompile it with meta-class check.

In my example it is more that I have to compile against 
groovy.lang.Script#println, but anyway... in the general case it is 
afaik impossible to compile directly if I have a x.y(a,b) in Groovy. 
Only in certain special cases this is different. And if I cannot go the 
direct path, I need the meta class. For my primitive opts in 1.8 I 
wanted to be able to run a program, that does only calculations and 
internal method calls, to be able to run without any meta class being 
loaded. I think I didn't succeed fully... time did run out kind of.

> I do something like that in PHP.reboot but my unit of compilation
> is the method and not the class, which also avoid to compile a
> code you never use.

true

bye Jochen

-- 
Jochen "blackdrag" Theodorou - Groovy Project Tech Lead
blog: http://blackdragsview.blogspot.com/
german groovy discussion newsgroup: de.comp.lang.misc
For Groovy programming sources visit http://groovy-lang.org


From stephen.bannasch at deanbrook.org  Thu Dec  1 07:47:33 2011
From: stephen.bannasch at deanbrook.org (Stephen Bannasch)
Date: Thu, 1 Dec 2011 10:47:33 -0500
Subject: JRuby invokedynamic updates for November
In-Reply-To: <CAE-f1xRy6AktwRKbGcMHSOCADj=8Mju=NRPk8+peoYSu0dS15A@mail.gmail.com>
References: <CAE-f1xT_Ku+4OhN1cCCXVO5Lanv=g_NFUxHi3n17mCMma1Jv2A@mail.gmail.com>	<p0624080acafc352919a0@192.168.1.122>
	<CAE-f1xQiqyfWMqwTVSQRx5BFPgVAb8447YDHk_xwGLyxBi=AGQ@mail.gmail.com>
	<CAE-f1xRuf+SaZjriTFe_C3EPesj7zYH=u+i3h4VWGysuABwigg@mail.gmail.com>	<p0624080ecafc4c3d8251@192.168.1.122>
	<CAE-f1xQ9xuOJ358aOe-=_=_Awm+eVwOa9ZdYBY_z23S4H9LQRw@mail.gmail.com>	<p06240813cafcaa3b89c7@192.168.1.122>
	<CAE-f1xSj9t==DYPZ+OD0Sy8_x-sAxoW7zEwu3xCrcjiYxPsftA@mail.gmail.com>
	<CAE-f1xRy6AktwRKbGcMHSOCADj=8Mju=NRPk8+peoYSu0dS15A@mail.gmail.com>
Message-ID: <p0624081fcafd4e50962c@[192.168.1.122]>

At 3:29 AM -0600 12/1/11, Charles Oliver Nutter wrote:
>This is a pretty disappointing finding. I'm not yet sure whether it's
>a bug in SwitchPoint optimization or in JRuby's use of SwitchPoint,
>but I'd like to see your numbers with and without this flag.

Latest JRuby master, on Java 1.6.0_27, macosx-port (1.7.0) amd mlvm (1.8.0)

Summary: without changing SwitchPoint mlvm performance is similar to 1.6.0_27 -- macosx-port (1.7.0) is about 50% slower onrexml test

Ruby version: jruby 1.7.0.dev (ruby-1.8.7-p352) (2011-12-01 586f44f) (Java HotSpot(TM) 64-Bit Server VM 1.6.0_27) [darwin-x86_64-java]

                              user     system      total        real
  rexml                   3.847000   0.000000   3.847000 (  3.847000)
  hpricot                 0.458000  0.000000   0.458000 (  0.458000)
  jdom_document_builder   0.240000   0.000000   0.240000 (  0.240000)
  nokogiri                0.413000   0.000000   0.413000 (  0.413000)


Ruby version: jruby 1.7.0.dev (ruby-1.8.7-p352) (2011-12-01 586f44f) (OpenJDK 64-Bit Server VM 1.7.0-internal) [darwin-x86_64-java]

                              user     system      total        real
  rexml                   6.469000   0.000000   6.469000 (  6.469000)
  hpricot                 0.522000  0.000000   0.522000 (  0.522000)
  jdom_document_builder   0.221000   0.000000   0.221000 (  0.221000)
  nokogiri                0.409000   0.000000   0.409000 (  0.409000)

Ruby version: jruby 1.7.0.dev (ruby-1.8.7-p352) (2011-12-01 586f44f) (OpenJDK 64-Bit Server VM 1.8.0-internal) [darwin-amd64-java]

                              user     system      total        real
  rexml                   4.093000   0.000000   4.093000 (  4.094000)
  hpricot                 0.434000  0.000000   0.434000 (  0.434000)
  jdom_document_builder   0.226000   0.000000   0.226000 (  0.226000)
  nokogiri                0.441000   0.000000   0.441000 (  0.441000)

Testing with: jruby -Xinvokedynamic.invocation.switchpoint=false

Summary: slightly faster on macosx-port (1.7.0), somewhat slower on mlvm

Ruby version: jruby 1.7.0.dev (ruby-1.8.7-p352) (2011-12-01 586f44f) (OpenJDK 64-Bit Server VM 1.7.0-internal) [darwin-x86_64-java]

                              user     system      total        real
  rexml                   6.120000   0.000000   6.120000 (  6.120000)
  hpricot                 0.555000  0.000000   0.555000 (  0.555000)
  jdom_document_builder   0.276000   0.000000   0.276000 (  0.276000)
  nokogiri                0.492000   0.000000   0.492000 (  0.492000)

Ruby version: jruby 1.7.0.dev (ruby-1.8.7-p352) (2011-12-01 586f44f) (OpenJDK 64-Bit Server VM 1.8.0-internal) [darwin-amd64-java]

                              user     system      total        real
  rexml                   4.580000   0.000000   4.580000 (  4.580000)
  hpricot                 0.495000  0.000000   0.495000 (  0.495000)
  jdom_document_builder   0.251000   0.000000   0.251000 (  0.252000)
  nokogiri                0.496000   0.000000   0.496000 (  0.496000)

From headius at headius.com  Thu Dec  1 12:48:43 2011
From: headius at headius.com (Charles Oliver Nutter)
Date: Thu, 1 Dec 2011 14:48:43 -0600
Subject: JRuby invokedynamic updates for November
In-Reply-To: <p0624081fcafd4e50962c@192.168.1.122>
References: <CAE-f1xT_Ku+4OhN1cCCXVO5Lanv=g_NFUxHi3n17mCMma1Jv2A@mail.gmail.com>
	<p0624080acafc352919a0@192.168.1.122>
	<CAE-f1xQiqyfWMqwTVSQRx5BFPgVAb8447YDHk_xwGLyxBi=AGQ@mail.gmail.com>
	<CAE-f1xRuf+SaZjriTFe_C3EPesj7zYH=u+i3h4VWGysuABwigg@mail.gmail.com>
	<p0624080ecafc4c3d8251@192.168.1.122>
	<CAE-f1xQ9xuOJ358aOe-=_=_Awm+eVwOa9ZdYBY_z23S4H9LQRw@mail.gmail.com>
	<p06240813cafcaa3b89c7@192.168.1.122>
	<CAE-f1xSj9t==DYPZ+OD0Sy8_x-sAxoW7zEwu3xCrcjiYxPsftA@mail.gmail.com>
	<CAE-f1xRy6AktwRKbGcMHSOCADj=8Mju=NRPk8+peoYSu0dS15A@mail.gmail.com>
	<p0624081fcafd4e50962c@192.168.1.122>
Message-ID: <CAE-f1xTSoFAyKKNTB+5ot9zh+JXfZOkXMYJcZU3LRJxVYF_BNQ@mail.gmail.com>

On Thu, Dec 1, 2011 at 9:47 AM, Stephen Bannasch
<stephen.bannasch at deanbrook.org> wrote:
> At 3:29 AM -0600 12/1/11, Charles Oliver Nutter wrote:
>>This is a pretty disappointing finding. I'm not yet sure whether it's
>>a bug in SwitchPoint optimization or in JRuby's use of SwitchPoint,
>>but I'd like to see your numbers with and without this flag.
>
> Latest JRuby master, on Java 1.6.0_27, macosx-port (1.7.0) amd mlvm (1.8.0)
>
> Summary: without changing SwitchPoint mlvm performance is similar to 1.6.0_27 -- macosx-port (1.7.0) is about 50% slower onrexml test

My numbers match yours for the first run, with 1.8.0 + indy being
about on par or slightly slower/faster than 1.6.0_27. However if I run
for more than one iteration, 1.8.0 wins across the board.

1.6 first iteration:

Rehearsal ---------------------------------------------------------
rexml                   5.517000   0.000000   5.517000 (  5.517000)
hpricot                 0.957000   0.000000   0.957000 (  0.958000)
jdom_document_builder   0.908000   0.000000   0.908000 (  0.908000)
nokogiri                1.328000   0.000000   1.328000 (  1.328000)
------------------------------------------------ total: 8.710000sec

                            user     system      total        real
rexml                   3.656000   0.000000   3.656000 (  3.656000)
hpricot                 0.481000   0.000000   0.481000 (  0.481000)
jdom_document_builder   0.195000   0.000000   0.195000 (  0.195000)
nokogiri                0.327000   0.000000   0.327000 (  0.327000)

1.8 first iteration:

Rehearsal ---------------------------------------------------------
rexml                   6.077000   0.000000   6.077000 (  6.077000)
hpricot                 1.013000   0.000000   1.013000 (  1.013000)
jdom_document_builder   0.964000   0.000000   0.964000 (  0.964000)
nokogiri                1.344000   0.000000   1.344000 (  1.344000)
------------------------------------------------ total: 9.398000sec

                            user     system      total        real
rexml                   3.322000   0.000000   3.322000 (  3.322000)
hpricot                 0.391000   0.000000   0.391000 (  0.391000)
jdom_document_builder   0.195000   0.000000   0.195000 (  0.195000)
nokogiri                0.323000   0.000000   0.323000 (  0.323000)

1.6 second iteration:

Rehearsal ---------------------------------------------------------
rexml                   3.446000   0.000000   3.446000 (  3.446000)
hpricot                 0.430000   0.000000   0.430000 (  0.430000)
jdom_document_builder   0.187000   0.000000   0.187000 (  0.187000)
nokogiri                0.323000   0.000000   0.323000 (  0.323000)
------------------------------------------------ total: 4.386000sec

                            user     system      total        real
rexml                   3.427000   0.000000   3.427000 (  3.427000)
hpricot                 0.435000   0.000000   0.435000 (  0.435000)
jdom_document_builder   0.190000   0.000000   0.190000 (  0.190000)
nokogiri                0.324000   0.000000   0.324000 (  0.324000)

1.8 second iteration:

Rehearsal ---------------------------------------------------------
rexml                   2.697000   0.000000   2.697000 (  2.697000)
hpricot                 0.378000   0.000000   0.378000 (  0.378000)
jdom_document_builder   0.186000   0.000000   0.186000 (  0.186000)
nokogiri                0.319000   0.000000   0.319000 (  0.319000)
------------------------------------------------ total: 3.580000sec

                            user     system      total        real
rexml                   2.711000   0.000000   2.711000 (  2.711000)
hpricot                 0.386000   0.000000   0.386000 (  0.386000)
jdom_document_builder   0.192000   0.000000   0.192000 (  0.192000)
nokogiri                0.318000   0.000000   0.318000 (  0.318000)

1.6 fifth iteration:

Rehearsal ---------------------------------------------------------
rexml                   3.386000   0.000000   3.386000 (  3.386000)
hpricot                 0.431000   0.000000   0.431000 (  0.431000)
jdom_document_builder   0.186000   0.000000   0.186000 (  0.186000)
nokogiri                0.321000   0.000000   0.321000 (  0.321000)
------------------------------------------------ total: 4.324000sec

                            user     system      total        real
rexml                   3.385000   0.000000   3.385000 (  3.385000)
hpricot                 0.434000   0.000000   0.434000 (  0.434000)
jdom_document_builder   0.187000   0.000000   0.187000 (  0.187000)
nokogiri                0.313000   0.000000   0.313000 (  0.313000)

1.8 fifth iteration:

Rehearsal ---------------------------------------------------------
rexml                   2.615000   0.000000   2.615000 (  2.615000)
hpricot                 0.369000   0.000000   0.369000 (  0.369000)
jdom_document_builder   0.180000   0.000000   0.180000 (  0.179000)
nokogiri                0.310000   0.000000   0.310000 (  0.310000)
------------------------------------------------ total: 3.474000sec

                            user     system      total        real
rexml                   2.679000   0.000000   2.679000 (  2.679000)
hpricot                 0.368000   0.000000   0.368000 (  0.368000)
jdom_document_builder   0.179000   0.000000   0.179000 (  0.179000)
nokogiri                0.320000   0.000000   0.320000 (  0.320000)

The nokogiri numbers probably won't vary over versions much, since
there's a lot of Java code involved. But it's the only one that's not
consistently faster on 1.8, so that's curious.

The rexml numbers seem to be convincingly faster, allowing for warmup.

If I pass -X+C it also helps reduce warmup time a bit:

I have more perf investigations happening on my end, so I'll keep
pushing fixes as they come up. If you have other loads that look like
they're slower on JRuby master + 1.8, let me know!

- Charlie

From tom.rodriguez at oracle.com  Thu Dec  1 13:21:40 2011
From: tom.rodriguez at oracle.com (Tom Rodriguez)
Date: Thu, 1 Dec 2011 13:21:40 -0800
Subject: How to monitor SwitchPoint's impact?
In-Reply-To: <CAE-f1xTkez=8fVTzzm=fekwX3D3wUD4vfmBvUEtJS484WhHjwQ@mail.gmail.com>
References: <CAE-f1xTkez=8fVTzzm=fekwX3D3wUD4vfmBvUEtJS484WhHjwQ@mail.gmail.com>
Message-ID: <6BC23874-5EE0-4CD5-A533-D4B6702042E7@oracle.com>


On Dec 1, 2011, at 1:34 AM, Charles Oliver Nutter wrote:

> My recent explorations into Stephen B's perf degradation on JRuby have
> led me to an unfortunate conclusion: something SwitchPoint-related is
> responsible for the remaining slowdown.
> 
> I had bugs I fixed, like the awful PIC-causes-repeat-rebinding issue
> or the heavy-class-mutation-never-stabilizes issue, but the last "fix"
> I found was to disable SwitchPoint-based call site invalidation
> entirely.

The upside of the push based invalidation of SwitchPoint is that the executed code runs quickly but the downside is that invalidation is expensive, possibly requiring a deopt and recompilation.  If you suspect that's happening, try running with -XX:+UnlockDiagnosticVMOptions -XX:+LogVMOutput -XX:+PrintCompilation and look for dependency_failed messages where type='call_site_target_value'.  Any following make_not_entrant messages are probably caused by the invalidation of the SwitchPoint.

tom

> 
> Now I'm not (yet) saying this is a perf issue in SwitchPoint itself.
> My problem is that it's very difficult to get any visibility into how
> frequently SwitchPoints are being invalidated (though I could
> instrument my own code, of course), or more importantly how
> drastically those invalidations are affecting performance. Does a
> single SwitchPoint invalidation happening repeatedly completely tank
> performance across the entire system? Does it require SwitchPoint
> invalidation that affects a broader surface area?
> 
> I will continue investigating on my side. I suspect the problem is
> that there's just a handful of SwitchPoint locations that are
> repeatedly invalidated, but that those invalidations are having a
> drastic global impact. I'm not yet sure how to work around such a
> situation, where one bad apple is spoiling the whole bunch...
> 
> - Charlie
> _______________________________________________
> mlvm-dev mailing list
> mlvm-dev at openjdk.java.net
> http://mail.openjdk.java.net/mailman/listinfo/mlvm-dev


From headius at headius.com  Thu Dec  1 13:55:24 2011
From: headius at headius.com (Charles Oliver Nutter)
Date: Thu, 1 Dec 2011 15:55:24 -0600
Subject: How to monitor SwitchPoint's impact?
In-Reply-To: <6BC23874-5EE0-4CD5-A533-D4B6702042E7@oracle.com>
References: <CAE-f1xTkez=8fVTzzm=fekwX3D3wUD4vfmBvUEtJS484WhHjwQ@mail.gmail.com>
	<6BC23874-5EE0-4CD5-A533-D4B6702042E7@oracle.com>
Message-ID: <CAE-f1xRSoX0wcg=K0fEAP38w6dMZ2=NToFwNNv0HypEdDtBLYg@mail.gmail.com>

Perfect, Tom, thanks.

On Thu, Dec 1, 2011 at 3:21 PM, Tom Rodriguez <tom.rodriguez at oracle.com> wrote:
>
> On Dec 1, 2011, at 1:34 AM, Charles Oliver Nutter wrote:
>
>> My recent explorations into Stephen B's perf degradation on JRuby have
>> led me to an unfortunate conclusion: something SwitchPoint-related is
>> responsible for the remaining slowdown.
>>
>> I had bugs I fixed, like the awful PIC-causes-repeat-rebinding issue
>> or the heavy-class-mutation-never-stabilizes issue, but the last "fix"
>> I found was to disable SwitchPoint-based call site invalidation
>> entirely.
>
> The upside of the push based invalidation of SwitchPoint is that the executed code runs quickly but the downside is that invalidation is expensive, possibly requiring a deopt and recompilation. ?If you suspect that's happening, try running with -XX:+UnlockDiagnosticVMOptions -XX:+LogVMOutput -XX:+PrintCompilation and look for dependency_failed messages where type='call_site_target_value'. ?Any following make_not_entrant messages are probably caused by the invalidation of the SwitchPoint.
>
> tom
>
>>
>> Now I'm not (yet) saying this is a perf issue in SwitchPoint itself.
>> My problem is that it's very difficult to get any visibility into how
>> frequently SwitchPoints are being invalidated (though I could
>> instrument my own code, of course), or more importantly how
>> drastically those invalidations are affecting performance. Does a
>> single SwitchPoint invalidation happening repeatedly completely tank
>> performance across the entire system? Does it require SwitchPoint
>> invalidation that affects a broader surface area?
>>
>> I will continue investigating on my side. I suspect the problem is
>> that there's just a handful of SwitchPoint locations that are
>> repeatedly invalidated, but that those invalidations are having a
>> drastic global impact. I'm not yet sure how to work around such a
>> situation, where one bad apple is spoiling the whole bunch...
>>
>> - Charlie
>> _______________________________________________
>> mlvm-dev mailing list
>> mlvm-dev at openjdk.java.net
>> http://mail.openjdk.java.net/mailman/listinfo/mlvm-dev
>
> _______________________________________________
> mlvm-dev mailing list
> mlvm-dev at openjdk.java.net
> http://mail.openjdk.java.net/mailman/listinfo/mlvm-dev

From john.r.rose at oracle.com  Sat Dec  3 19:50:22 2011
From: john.r.rose at oracle.com (john.r.rose at oracle.com)
Date: Sun, 04 Dec 2011 03:50:22 +0000
Subject: hg: mlvm/mlvm/jdk: update 7030453 to review version
Message-ID: <20111204035023.1C5644755B@hg.openjdk.java.net>

Changeset: 1b3a3c6c9fb0
Author:    jrose
Date:      2011-12-03 19:50 -0800
URL:       http://hg.openjdk.java.net/mlvm/mlvm/jdk/rev/1b3a3c6c9fb0

update 7030453 to review version

! cval-tune-7030453.patch


From john.r.rose at oracle.com  Sat Dec  3 19:59:25 2011
From: john.r.rose at oracle.com (John Rose)
Date: Sat, 3 Dec 2011 19:59:25 -0800
Subject: review request (L): 7030453: JSR 292 ClassValue.get method is too
	slow
In-Reply-To: <5AF69B9B-D7A9-47B0-8452-06308D824D70@oracle.com>
References: <BB480899-E037-4BF2-9407-039BEA8131EC@oracle.com>
	<5AF69B9B-D7A9-47B0-8452-06308D824D70@oracle.com>
Message-ID: <1A25D30F-204C-4CEE-95A1-F6FDB7925B7D@oracle.com>

The caching ClassValue is still under review.  I took the opportunity to remove -Xlint warnings (adding type parameters) and refactor for better readability.

Also, Mike Duigou pointed out that there was still a dependency on ClassValue.equals == Object.equals, which could create a side-channel between independent CVs.  I removed this by replacing ClassValue with ClassValue.Identity as the hash table key.

The cache access is racy, for speed, but all races are (thought to be) benign.  For cache hits, end-user values are obtained from non-racing accesses.  All cache state changes happen either under a per-class lock or during GC (nulling of weak references).

Could I have a re-review?

Thanks,
-- John

On Sep 20, 2011, at 4:10 PM, John Rose wrote:

> On Sep 19, 2011, at 2:58 PM, John Rose wrote:
> 
>> http://cr.openjdk.java.net/~jrose/7030453/webrev.00
> 
> After some comments from David Holmes (thanks David!) I added internal comments about race conditions.
> 
> I refreshed the webrev with the additional comments.  I also changed one variable to be volatile, with a paragraph of comments explaining why.
> 
> (The change to volatile will inhibit CSE of ClassValue.get calls, but I think such CSE was unlikely anyway.  There should be no other performance effects.)
> 
> -- John


From forax at univ-mlv.fr  Sun Dec  4 14:13:39 2011
From: forax at univ-mlv.fr (=?ISO-8859-1?Q?R=E9mi_Forax?=)
Date: Sun, 04 Dec 2011 23:13:39 +0100
Subject: review request (L): 7030453: JSR 292 ClassValue.get method is
	too slow
In-Reply-To: <4EDBCC2F.7030907@oracle.com>
References: <BB480899-E037-4BF2-9407-039BEA8131EC@oracle.com>	<5AF69B9B-D7A9-47B0-8452-06308D824D70@oracle.com>
	<1A25D30F-204C-4CEE-95A1-F6FDB7925B7D@oracle.com>
	<4EDBCC2F.7030907@oracle.com>
Message-ID: <4EDBF093.1000801@univ-mlv.fr>

On 12/04/2011 08:38 PM, Joe Darcy wrote:
> Hi John,
>
> Are there alternatives to adding two new fields to java.lang.Class?  I 
> assume most Class'es won't have ClassValue information associated with 
> them.
>
> -Joe

If you use Groovy, JRuby or Nashorn in your code, all visible classes 
will use this two fields.

Any alternative will slow down the access to the class value.

BTW, if we had to remove some fields, I vote for moving all fields 
related to the reflection
in a delegate class. Looking up for members is slow so most of the code 
that call reflection
methods use their own cache, so the performance impact will be small in 
my opinion
(I have no data to prove that :)

R?mi


From forax at univ-mlv.fr  Sun Dec  4 17:03:13 2011
From: forax at univ-mlv.fr (=?ISO-8859-1?Q?R=E9mi_Forax?=)
Date: Mon, 05 Dec 2011 02:03:13 +0100
Subject: review request (L): 7030453: JSR 292 ClassValue.get method is
	too slow
In-Reply-To: <4EDC0FFE.2080805@oracle.com>
References: <BB480899-E037-4BF2-9407-039BEA8131EC@oracle.com>	<5AF69B9B-D7A9-47B0-8452-06308D824D70@oracle.com>
	<1A25D30F-204C-4CEE-95A1-F6FDB7925B7D@oracle.com>
	<4EDBCC2F.7030907@oracle.com> <4EDBF093.1000801@univ-mlv.fr>
	<4EDC0FFE.2080805@oracle.com>
Message-ID: <4EDC1851.4060508@univ-mlv.fr>

On 12/05/2011 01:27 AM, Joe Darcy wrote:
> On 12/4/2011 2:13 PM, R?mi Forax wrote:
>> On 12/04/2011 08:38 PM, Joe Darcy wrote:
>>> Hi John,
>>>
>>> Are there alternatives to adding two new fields to java.lang.Class?  
>>> I assume most Class'es won't have ClassValue information associated 
>>> with them.
>>>
>>> -Joe
>>
>> If you use Groovy, JRuby or Nashorn in your code, all visible classes 
>> will use this two fields.
>>
>> Any alternative will slow down the access to the class value.
>
> In the mean time, all the non-Groovy, non-JRuby, non-Nashorn, etc. 
> uses of class Class and all the classes not visible in those 
> environments when they are being used will be larger.
>
> Adding the fields may be the right time/space trade-off, but I think 
> the point merits some discussion given how many Class objects get 
> created and the relative proportion of Java executions where 
> ClassValue is currently used.
>
> The more reasonable time/space trade-off can change over time of course.
>
> -Joe
>

I agree but as I said, in that case, I think it's better to take a look 
to the big picture
and see if not only class values fields but also annotations related 
fields or reflection related fields
can be moved.

Also, if we don't provide a fast ClassValue, people will create their 
own concurrent weak hash map
using class as key that will be worst because it seems that only few 
people knows
how to do that right. Personally, I don't.

R?mi


From forax at univ-mlv.fr  Mon Dec  5 07:28:32 2011
From: forax at univ-mlv.fr (=?ISO-8859-1?Q?R=E9mi_Forax?=)
Date: Mon, 05 Dec 2011 16:28:32 +0100
Subject: The NoClassDefFoundError bug is back :(
Message-ID: <4EDCE320.6060601@univ-mlv.fr>

John, Christian,
I was updating the multi-dispatch sample from the cookbook
when the NoClassDefFoundError bug re-appear.



[forax at localhost multi-dispatch]$ java -ea -cp classes:. Main2
skip java.lang.String.contentEquals/2
skip java.lang.String.replace/3
Exception in thread "main" java.lang.NoClassDefFoundError: Main2
     at java.lang.invoke.MethodHandle.invokeExact(MethodHandle.java)
     at java.lang.invoke.MethodHandle.invokeExact(MethodHandle.java)
     at Main2.test2(Main2.java:27)
     at Main2.main(Main2.java:42)

[forax at localhost src]$ java -version
java version "1.8.0-ea"
Java(TM) SE Runtime Environment (build 1.8.0-ea-b07)
Java HotSpot(TM) 64-Bit Server VM (build 22.0-b03, mixed mode)

The sample is here:
http://code.google.com/p/jsr292-cookbook/source/browse/trunk/multi-dispatch/src/jsr292/cookbook/mdispatch/

You can compile it from src,
javac -d ../classes jsr292/cookbook/mdispatch/*.java

The example that doesn't work is Main2, which is created by compiling 
Main2.java
and then by rewriting all invokevirtual/invokestatic to invokedynamic 
using this line
java -cp classes:../lib/asm-debug-all-4.0_RC1.jar 
jsr292.cookbook.mdispatch.Rewriter Main2.class

Note that Main2.class is already rewritten so you don't have to do it
if you just want to reproduce the bug without modifying Main2.java

I think the error is thrown when calling a method handle created by this 
line
https://code.google.com/p/jsr292-cookbook/source/browse/trunk/multi-dispatch/src/jsr292/cookbook/mdispatch/SmallSelectorMetadata.java?spec=svn13&r=13#121
at this callsite
http://code.google.com/p/jsr292-cookbook/source/browse/trunk/multi-dispatch/Main2.java#27

How the code works: it installs a bimorphic cache on the receiver
and an inlining cache on the parameters. After 3 cache misses, the 
implementation
switch to use a table based resolution that lookup for most specific 
method handle and
use an exactInvoker to call it. The bug appear when the exactInvoker is 
called.

regards,
R?mi


From mleodaalder at gmail.com  Mon Dec  5 11:06:30 2011
From: mleodaalder at gmail.com (MLeo)
Date: Mon, 5 Dec 2011 20:06:30 +0100
Subject: Looping construction using only MethodHandles
Message-ID: <CABA5GS5NswYi-a2gZPp71YtEyZSDGPmU=geZjPLrcw4tQ4LFVA@mail.gmail.com>

Hello all,

Over the past few days I've been thinkering with a bit of code that folds
over the values of some container using some MethodHandle and a 'zero'
value.

https://gist.github.com/1425706

It's actually an implementation of a strategy to encode higher order
functions (HOF) without introducing a plethoramorphic callsite (if I
remember the term correctly). If you squint at it right sort of resembles
inversion of control. Or it's more like turning a function inside out.

And it mostly works, with suprising result.
It's not entirely tail call optimized, it still eats stack, but not the
same amount as an naive unequivalent Java static method that is hand
optimized for the task (so no HOF).

I'm using the x64 update 2 jdk on Windows Vista with -Xss10m. And the naive
version bails after a depth of ~110000 and after some optimization it will
get to ~310000 (after the first invocation!). The fold, on the other hand,
only bails out after recursing for ~650000 times and gets there nearly
immediately, the first run is ~10000 lower, but the remainder of
invocations is consistent (or seems that way, rather, I think it's because
not enough optimizations have kicked in yet).

Execution time is another story, the POJM (Plain Old Java Method) takes
~5ms, while the fold takes ~30ms. Which, now I think some more about it, is
more or less in line with the previous initial result, but still seems to
be a bit slow to me.
After some optimizations have kicked in (when the POJM reaches a depth of
310000) the execution times stay roughly the same. So it would appear the
JVM inlined the POJM 2 or 3 times. Creating the stacktraces doesn't seem to
impact performance, if I make them recurse 100000 times (so it doesn't
overflow) then the execution time is roughly the same.

Both ways are tested by invoking them through invokeWithArguments (I
haven't yet managed to get ASM to produce a test class for me), and I let
both try to sum an array (turned into an Iterator through
Arrays.asList(...).iterator()) of a million elements, so both will cause a
StackOverflowException. The test is to invoke a methods 1000 times (I got a
bit tired of waiting for a million times) and that I do 6 times (so I take
6 samples and average across them).


Now my question, is there something I can do to make it completely tail
call optimized? I've tried to 'rotate' the call graph/tree but that
obviously wouldn't work (it's still a direct recursion, no matter if you do
it directly or in the combiner of a foldArguments). It seems that it is
almost completely TCO already, but I haven't found where it's leaking
stack. It's definitely leaking less stack than a simple recursion.
Or will we need a special looping combinator here? I initially tried to
create a while combinator, however it seems that guardWithTest does not
accept MethodHandles returning void (for the target and fallback).

If this can be made to work (so fully TCO'd, and maybe some massaging from
HotSpot engineers), then in theory it would allow for functional languages
a way to monomorphize HOFs.
For something like map (or fold) it will probably require a guard/PIC for
the implementations of that method, and those methods need to be compiled
in such a way so that they become bootstrap methods/factories (see the
constructor of MethodHandleFoldBuilder.java in the gist) which can then get
installed in the invocation of that fold.

Thank you for your time,
--Maarten Daalder

PS.

In case you haven't noticed, I'm not a statistics wizard, so the numbers
mentioned should be taken only as an early indication and not of any
statistical significance. YMWV.

Btw, if you were to implement 'map' with this then it would (if I noticed
it correctly) also reverse the list, or you're going to need to append each
element to the list, which will probably be slow. This is something I'm
going to try in the next few days or so.
-------------- next part --------------
An HTML attachment was scrubbed...
URL: http://mail.openjdk.java.net/pipermail/mlvm-dev/attachments/20111205/f3cab31a/attachment.html 

From headius at headius.com  Mon Dec  5 21:54:59 2011
From: headius at headius.com (Charles Oliver Nutter)
Date: Mon, 5 Dec 2011 23:54:59 -0600
Subject: Looping construction using only MethodHandles
In-Reply-To: <CABA5GS5NswYi-a2gZPp71YtEyZSDGPmU=geZjPLrcw4tQ4LFVA@mail.gmail.com>
References: <CABA5GS5NswYi-a2gZPp71YtEyZSDGPmU=geZjPLrcw4tQ4LFVA@mail.gmail.com>
Message-ID: <CAE-f1xQ5TDmNfXqvQAOPSKAc79qY3bd5PdxwBMb6XDCSp2xTYA@mail.gmail.com>

This actually came up early this past summer, when I tried to do the
same thing (after Ola Bini showed me the MutableCallSite invoker
trick. You can probably find the thread in July some time.

The bottom line was that even if the handles could get folded into
tail recursion, the call site ends up interfering with the
MethodHandle optimizer's visibility into the graph.

I was as disappointed as you are (or as you will be, once you've read
this). I wish I'd been more involved early in the JSR-292 process and
pushed for a looping construct. Unfortunately that train has sailed,
and I've found no way to do loops (or any sort of backward branching)
using the current set of method handle adapters :(

Here's the tease, for future work on invokedynamic: I believe that if
we had a way to do backward branches, method handles could become a
turing-complete, user-accessible IR for the JVM's JIT.

- Charlie

On Mon, Dec 5, 2011 at 1:06 PM, MLeo <mleodaalder at gmail.com> wrote:
> Hello all,
>
> Over the past few days I've been thinkering with a bit of code that folds
> over the values of some container using some MethodHandle and a 'zero'
> value.
>
> https://gist.github.com/1425706
>
> It's actually an implementation of a strategy to encode higher order
> functions (HOF) without introducing a plethoramorphic callsite (if I
> remember the term correctly). If you squint at it right sort of resembles
> inversion of control. Or it's more like turning a function inside out.
>
> And it mostly works, with suprising result.
> It's not entirely tail call optimized, it still eats stack, but not the same
> amount as an naive unequivalent Java static method that is hand optimized
> for the task (so no HOF).
>
> I'm using the x64 update 2 jdk on Windows Vista with -Xss10m. And the naive
> version bails after a depth of ~110000 and after some optimization it will
> get to ~310000 (after the first invocation!). The fold, on the other hand,
> only bails out after recursing for ~650000 times and gets there nearly
> immediately, the first run is ~10000 lower, but the remainder of invocations
> is consistent (or seems that way, rather, I think it's because not enough
> optimizations have kicked in yet).
>
> Execution time is another story, the POJM (Plain Old Java Method) takes
> ~5ms, while the fold takes ~30ms. Which, now I think some more about it, is
> more or less in line with the previous initial result, but still seems to be
> a bit slow to me.
> After some optimizations have kicked in (when the POJM reaches a depth of
> 310000) the execution times stay roughly the same. So it would appear the
> JVM inlined the POJM 2 or 3 times. Creating the stacktraces doesn't seem to
> impact performance, if I make them recurse 100000 times (so it doesn't
> overflow) then the execution time is roughly the same.
>
> Both ways are tested by invoking them through invokeWithArguments (I haven't
> yet managed to get ASM to produce a test class for me), and I let both try
> to sum an array (turned into an Iterator through
> Arrays.asList(...).iterator()) of a million elements, so both will cause a
> StackOverflowException. The test is to invoke a methods 1000 times (I got a
> bit tired of waiting for a million times) and that I do 6 times (so I take 6
> samples and average across them).
>
>
> Now my question, is there something I can do to make it completely tail call
> optimized? I've tried to 'rotate' the call graph/tree but that obviously
> wouldn't work (it's still a direct recursion, no matter if you do it
> directly or in the combiner of a foldArguments). It seems that it is almost
> completely TCO already, but I haven't found where it's leaking stack. It's
> definitely leaking less stack than a simple recursion.
> Or will we need a special looping combinator here? I initially tried to
> create a while combinator, however it seems that guardWithTest does not
> accept MethodHandles returning void (for the target and fallback).
>
> If this can be made to work (so fully TCO'd, and maybe some massaging from
> HotSpot engineers), then in theory it would allow for functional languages a
> way to monomorphize HOFs.
> For something like map (or fold) it will probably require a guard/PIC for
> the implementations of that method, and those methods need to be compiled in
> such a way so that they become bootstrap methods/factories (see the
> constructor of MethodHandleFoldBuilder.java in the gist) which can then get
> installed in the invocation of that fold.
>
> Thank you for your time,
> --Maarten Daalder
>
> PS.
>
> In case you haven't noticed, I'm not a statistics wizard, so the numbers
> mentioned should be taken only as an early indication and not of any
> statistical significance. YMWV.
>
> Btw, if you were to implement 'map' with this then it would (if I noticed it
> correctly) also reverse the list, or you're going to need to append each
> element to the list, which will probably be slow. This is something I'm
> going to try in the next few days or so.
>
> _______________________________________________
> mlvm-dev mailing list
> mlvm-dev at openjdk.java.net
> http://mail.openjdk.java.net/mailman/listinfo/mlvm-dev
>

From forax at univ-mlv.fr  Tue Dec  6 01:34:59 2011
From: forax at univ-mlv.fr (=?ISO-8859-1?Q?R=E9mi_Forax?=)
Date: Tue, 06 Dec 2011 10:34:59 +0100
Subject: The NoClassDefFoundError bug is back :(
In-Reply-To: <4EDCE320.6060601@univ-mlv.fr>
References: <4EDCE320.6060601@univ-mlv.fr>
Message-ID: <4EDDE1C3.9070104@univ-mlv.fr>

On 12/05/2011 04:28 PM, R?mi Forax wrote:
> John, Christian,
> I was updating the multi-dispatch sample from the cookbook
> when the NoClassDefFoundError bug re-appear.

I've reduced the code to this simple sample:
public class NoClassDefFoundSample {
   public void foo() {
     // do nothing
   }

   public static void main(String[] args) {
     NoClassDefFoundSample classDefFoundSample = new 
NoClassDefFoundSample();
     for(int i=0; i < 100000; i++) {
       classDefFoundSample.foo();   // invokedynamic
     }
   }
}

public class NoClassDefFoundBug {
   // bootstrap method
   public static CallSite invokevirtual(Lookup lookup, String name, 
MethodType type) throws Throwable {
     MethodHandle mh = lookup.findVirtual(type.parameterType(0), name,
         type.dropParameterTypes(0, 1));

     CallSite invokerCallSite = new MutableCallSite(mh);

     MethodHandle dynamicInvoker = invokerCallSite.dynamicInvoker();

     CallSite callSite = new MutableCallSite(type);
     callSite.setTarget(dynamicInvoker);

     return callSite;
   }
}

The sample triggers the JIT and the JITed code of the dynamicInvoker() 
throws
the NoClassDefFoundError.

Code is available here:
https://code.google.com/p/jsr292-cookbook/source/browse/#svn/trunk/multi-dispatch/src/bug
https://code.google.com/p/jsr292-cookbook/source/browse/trunk/multi-dispatch/NoClassDefFoundSample.java

cheers,
R?mi





From forax at univ-mlv.fr  Tue Dec  6 01:52:01 2011
From: forax at univ-mlv.fr (=?ISO-8859-1?Q?R=E9mi_Forax?=)
Date: Tue, 06 Dec 2011 10:52:01 +0100
Subject: The NoClassDefFoundError bug is back :(
In-Reply-To: <4EDDE1C3.9070104@univ-mlv.fr>
References: <4EDCE320.6060601@univ-mlv.fr> <4EDDE1C3.9070104@univ-mlv.fr>
Message-ID: <4EDDE5C1.5000800@univ-mlv.fr>

On 12/06/2011 10:34 AM, R?mi Forax wrote:
> On 12/05/2011 04:28 PM, R?mi Forax wrote:
>> John, Christian,
>> I was updating the multi-dispatch sample from the cookbook
>> when the NoClassDefFoundError bug re-appear.
> I've reduced the code to this simple sample:
> public class NoClassDefFoundSample {
>     public void foo() {
>       // do nothing
>     }
>
>     public static void main(String[] args) {
>       NoClassDefFoundSample classDefFoundSample = new
> NoClassDefFoundSample();
>       for(int i=0; i<  100000; i++) {
>         classDefFoundSample.foo();   // invokedynamic
>       }
>     }
> }
>
> public class NoClassDefFoundBug {
>     // bootstrap method
>     public static CallSite invokevirtual(Lookup lookup, String name,
> MethodType type) throws Throwable {
>       MethodHandle mh = lookup.findVirtual(type.parameterType(0), name,
>           type.dropParameterTypes(0, 1));
>
>       CallSite invokerCallSite = new MutableCallSite(mh);
>
>       MethodHandle dynamicInvoker = invokerCallSite.dynamicInvoker();
>
>       CallSite callSite = new MutableCallSite(type);
>       callSite.setTarget(dynamicInvoker);
>
>       return callSite;
>     }
> }
>
> The sample triggers the JIT and the JITed code of the dynamicInvoker()
> throws
> the NoClassDefFoundError.
>
> Code is available here:
> https://code.google.com/p/jsr292-cookbook/source/browse/#svn/trunk/multi-dispatch/src/bug
> https://code.google.com/p/jsr292-cookbook/source/browse/trunk/multi-dispatch/NoClassDefFoundSample.java

I've forget to say that a workaround is to erase the type used the callsite
before calling dynamicInvoker().

>
> cheers,
> R?mi

cheers,
R?mi


From david.holmes at oracle.com  Sun Dec  4 18:25:28 2011
From: david.holmes at oracle.com (David Holmes)
Date: Mon, 05 Dec 2011 12:25:28 +1000
Subject: review request (L): 7030453: JSR 292 ClassValue.get method is
	too slow
In-Reply-To: <4EDC1851.4060508@univ-mlv.fr>
References: <BB480899-E037-4BF2-9407-039BEA8131EC@oracle.com>	<5AF69B9B-D7A9-47B0-8452-06308D824D70@oracle.com>
	<1A25D30F-204C-4CEE-95A1-F6FDB7925B7D@oracle.com>
	<4EDBCC2F.7030907@oracle.com> <4EDBF093.1000801@univ-mlv.fr>
	<4EDC0FFE.2080805@oracle.com> <4EDC1851.4060508@univ-mlv.fr>
Message-ID: <4EDC2B98.6020101@oracle.com>

On 5/12/2011 11:03 AM, R?mi Forax wrote:
> On 12/05/2011 01:27 AM, Joe Darcy wrote:
>> On 12/4/2011 2:13 PM, R?mi Forax wrote:
>>> On 12/04/2011 08:38 PM, Joe Darcy wrote:
>>>>
>>>> Are there alternatives to adding two new fields to java.lang.Class?
>>>> I assume most Class'es won't have ClassValue information associated
>>>> with them.
>>>
>>> If you use Groovy, JRuby or Nashorn in your code, all visible classes
>>> will use this two fields.
>>>
>>> Any alternative will slow down the access to the class value.
>>
>> In the mean time, all the non-Groovy, non-JRuby, non-Nashorn, etc.
>> uses of class Class and all the classes not visible in those
>> environments when they are being used will be larger.
>>
>> Adding the fields may be the right time/space trade-off, but I think
>> the point merits some discussion given how many Class objects get
>> created and the relative proportion of Java executions where
>> ClassValue is currently used.
>>
>> The more reasonable time/space trade-off can change over time of course.
>>
>
> I agree but as I said, in that case, I think it's better to take a look
> to the big picture
> and see if not only class values fields but also annotations related
> fields or reflection related fields can be moved.

This is currently being looked at with a general aim of reducing the 
size of Class instances. So adding back some size would need strong 
justification.

David
-----


> Also, if we don't provide a fast ClassValue, people will create their
> own concurrent weak hash map
> using class as key that will be worst because it seems that only few
> people knows
> how to do that right. Personally, I don't.
>
> R?mi
>

From john.r.rose at oracle.com  Wed Dec  7 12:34:06 2011
From: john.r.rose at oracle.com (john.r.rose at oracle.com)
Date: Wed, 07 Dec 2011 20:34:06 +0000
Subject: hg: mlvm/mlvm/jdk: update 7030453 - unhoist Class.classValueCache for
	footprint
Message-ID: <20111207203406.92C6047624@hg.openjdk.java.net>

Changeset: dae30690913f
Author:    jrose
Date:      2011-12-07 12:34 -0800
URL:       http://hg.openjdk.java.net/mlvm/mlvm/jdk/rev/dae30690913f

update 7030453 - unhoist Class.classValueCache for footprint

! cval-tune-7030453.patch


From john.r.rose at oracle.com  Wed Dec  7 13:14:33 2011
From: john.r.rose at oracle.com (John Rose)
Date: Wed, 7 Dec 2011 13:14:33 -0800 (PST)
Subject: review request (L): 7030453: JSR 292 ClassValue.get method is too
	slow
In-Reply-To: <4EDC2B98.6020101@oracle.com>
References: <BB480899-E037-4BF2-9407-039BEA8131EC@oracle.com>
	<5AF69B9B-D7A9-47B0-8452-06308D824D70@oracle.com>
	<1A25D30F-204C-4CEE-95A1-F6FDB7925B7D@oracle.com>
	<4EDBCC2F.7030907@oracle.com> <4EDBF093.1000801@univ-mlv.fr>
	<4EDC0FFE.2080805@oracle.com> <4EDC1851.4060508@univ-mlv.fr>
	<4EDC2B98.6020101@oracle.com>
Message-ID: <2C2A1E0C-876C-4DC6-9812-E53E2619D272@oracle.com>


On Dec 4, 2011, at 6:25 PM, David Holmes wrote:

>>> In the mean time, all the non-Groovy, non-JRuby, non-Nashorn, etc.
>>> uses of class Class and all the classes not visible in those
>>> environments when they are being used will be larger.
>>> 
>>> Adding the fields may be the right time/space trade-off, but I think
>>> the point merits some discussion given how many Class objects get
>>> created and the relative proportion of Java executions where
>>> ClassValue is currently used.
>>> 
>>> The more reasonable time/space trade-off can change over time of course.
>>> 
>> 
>> I agree but as I said, in that case, I think it's better to take a look
>> to the big picture
>> and see if not only class values fields but also annotations related
>> fields or reflection related fields can be moved.
> 
> This is currently being looked at with a general aim of reducing the size of Class instances. So adding back some size would need strong justification.

At a minimum we need a single word to root the lookup in the Class itself, in order to avoid (a) many extra branches and indirections per lookup and (b) scaling problems associated with centralized hash tables.  (This is the same reasoning as with Thread.threadLocals.)

Since the number of classes in apps is typically in the range 10^3 to 10^4, the extra word is not going to hurt anybody.  The proof of this is that nobody has needed to touch the highly expansive implementation of reflective caches on Class, which was noted by Remi.  These things have existed since March of 2001 without causing enough pain to require fixing, and in those years memory has shrunk in cost by orders of magnitude.

But, in order to respect the "general aim" you are mentioning, I have unhoisted one of the two words from the Class instance itself.  This will cause a minor slowdown in JSR 292 use cases.  This is tolerable, since the next level of speedup will probably come from compiler optimizations, not data structure reorganization.

I have updated the webrev in place with this additional change:
  http://cr.openjdk.java.net/~jrose/7030453/webrev.01

There is a separate mini-diff for the unhoisting, which is trivial:
  http://cr.openjdk.java.net/~jrose/7030453/unhoist-cache.patch

-- John
-------------- next part --------------
An HTML attachment was scrubbed...
URL: http://mail.openjdk.java.net/pipermail/mlvm-dev/attachments/20111207/18720b50/attachment.html 

From headius at headius.com  Thu Dec  8 09:48:09 2011
From: headius at headius.com (Charles Oliver Nutter)
Date: Thu, 8 Dec 2011 11:48:09 -0600
Subject: Coro crash running specs
Message-ID: <CAE-f1xSy9rRXhvAiN56eMVF+0K4Hc6=WDd-rxNOVry9Jyow_2Q@mail.gmail.com>

I'm fixing what looks to be bugs in the coroutine-based Fiber impl,
but I saw this crash today. I think it's a result of an exception
being raised from inside the coroutine.

https://gist.github.com/1447787

To reproduce:

* clone JRuby
* ant jar install-gems fetch-stable-specs
* jruby spec/mspec/bin/mspec ci -T--1.9 -T-X-C
-T-Xfiber.coroutines=true spec/ruby/core/fiber

I saw that OpenJDK is using JIRA now...I wonder if coro could get a
JIRA project for issues?

- Charlie

From headius at headius.com  Thu Dec  8 09:58:10 2011
From: headius at headius.com (Charles Oliver Nutter)
Date: Thu, 8 Dec 2011 11:58:10 -0600
Subject: The NoClassDefFoundError bug is back :(
In-Reply-To: <4EDDE5C1.5000800@univ-mlv.fr>
References: <4EDCE320.6060601@univ-mlv.fr> <4EDDE1C3.9070104@univ-mlv.fr>
	<4EDDE5C1.5000800@univ-mlv.fr>
Message-ID: <CAE-f1xT5H2xo56Enk80Rfo1nmXApEwEKM8HFDz=70hLM+2_0Gg@mail.gmail.com>

I just had a report of the same error in 1.7.0GA, and I saw it on the
CI server last night after I set up a 1.7.0_01 build. Did it linger
into GA?

http://ci.jruby.org/job/jruby-test-matrix/jdk=sun-java-7,label=master/406/console

On Tue, Dec 6, 2011 at 3:52 AM, R?mi Forax <forax at univ-mlv.fr> wrote:
> I've forget to say that a workaround is to erase the type used the callsite
> before calling dynamicInvoker().

Can you elaborate on this? I'm not using dynamicInvoker...I bind the
site and use invokeWithArguments. Are you saying I could insert an
asType() that clears everything to Object?

If so, would that introduce overhead?

- Charlie

From john.r.rose at oracle.com  Thu Dec  8 11:57:17 2011
From: john.r.rose at oracle.com (John Rose)
Date: Thu, 8 Dec 2011 11:57:17 -0800
Subject: review request (L): 7030453: JSR 292 ClassValue.get method is too
	slow
In-Reply-To: <821usf2xsb.fsf@mid.bfk.de>
References: <BB480899-E037-4BF2-9407-039BEA8131EC@oracle.com>
	<5AF69B9B-D7A9-47B0-8452-06308D824D70@oracle.com>
	<1A25D30F-204C-4CEE-95A1-F6FDB7925B7D@oracle.com>
	<4EDBCC2F.7030907@oracle.com> <4EDBF093.1000801@univ-mlv.fr>
	<4EDC0FFE.2080805@oracle.com> <4EDC1851.4060508@univ-mlv.fr>
	<4EDC2B98.6020101@oracle.com>
	<2C2A1E0C-876C-4DC6-9812-E53E2619D272@oracle.com>
	<821usf2xsb.fsf@mid.bfk.de>
Message-ID: <DE9118E8-BFBE-4570-947C-9D346F29FE23@oracle.com>

On Dec 8, 2011, at 1:55 AM, Florian Weimer wrote:

> * John Rose:
> 
>> But, in order to respect the "general aim" you are mentioning, I have
>> unhoisted one of the two words from the Class instance itself.  This
>> will cause a minor slowdown in JSR 292 use cases.
> 
> What about using ClassValue for the various caches instead?
> enumConstants and enumConstantDirectory seem good candidates (callers
> cache the value anyway or do additional work while accessing the field).

That's a fine idea, Florian, especially when we are counting every word of fixed overhead.  (The alternative is keeping one root pointer in Class for the whole block of caches.)

Even the reflective caches are candidates for ClassValue treatment, since only a minority of classes are subjected to reflective operations.

ClassValue is good for any set of values associated sparsely with classes, as long as the query does not happen in a very tight loop.

The trade-off is whether to add another 4-5 cache line touches per use to buy the extra compactness.  To compare queries:

  Class cls = ...;
  Object val1 = cls.cache1;
  if (val1 != null) ... // fill cache
  test1(val1);

  load $t1, [$cls + #Class::cache1]
  test $t1
  jump,zero Lfillcache
  call test1($t1)

  ClassValue cval = ...
  Object val2 = cval.get(cls);
  test2(val2);

  load $t1, [$cls + #Class::classValueMap]
  load $t2array, [$t1 + #ClassValueMap::cacheArray]
    & implicit { test $t1; jump,zero Lfillcache }  // via trap handler
  load $t3, [$t2array + #Object[]::length]
  sub $t3, 1
  jump,negative Lfatal  // never taken; software invariant
  load $t4a, [$cval + #ClassValue::hashCodeForCache]
  load $t4b, [$cval + #ClassValue::version]
  and $t4a, $t3
  load $t5entry, [$t2array + $t4a*wordSize + #Object[]::base]
  load $t6a, [$t5entry + #Entry::value]
  load $t6b, [$t5entry + #Entry::version]
  cmp $t6b, $t5b
  jump,notEqual Lfillcache
  call test2($t6a)

The pointer dependencies for cache references are:
  ClassValue -> t4:{hashCodeForCache,version}
  Class -> t1:ClassValueMap -> t2:cacheArray -> ( t3:length,  t5:Entry -> t6:{value,version} )

The bracketed items are likely to be on a single cache line, so there are six cache references.  For a constant ClassValue, the t4 references can (in principle) be hoisted as constants into the code.  And the first indirection ($t1) could be removed by hoisting the cache array back into Class.

All this reminds me...

Somebody should experiment with re-implementing reflection and proxy creation on top of method handles.  It would let us cut out a bunch of old code (both C++ and Java), and standardize on a single high-leverage mechanism for runtime method composition.  (Look in the current JDK core where bytecode spinning is happening...  Those places are all candidates for refactoring with method handles.)

We are working on tuning up method handle invocation performance (e.g., 7023639).  When method handles are robustly performant, we will have the attractive option of refactoring older APIs on top of MHs.

It's not too early to start experimenting with a POC for this.  It would be a nice big open-source project.  Any takers?

-- John
-------------- next part --------------
An HTML attachment was scrubbed...
URL: http://mail.openjdk.java.net/pipermail/mlvm-dev/attachments/20111208/67feaf51/attachment.html 

From forax at univ-mlv.fr  Thu Dec  8 12:16:12 2011
From: forax at univ-mlv.fr (=?UTF-8?B?UsOpbWkgRm9yYXg=?=)
Date: Thu, 08 Dec 2011 21:16:12 +0100
Subject: The NoClassDefFoundError bug is back :(
In-Reply-To: <CAE-f1xT5H2xo56Enk80Rfo1nmXApEwEKM8HFDz=70hLM+2_0Gg@mail.gmail.com>
References: <4EDCE320.6060601@univ-mlv.fr> <4EDDE1C3.9070104@univ-mlv.fr>
	<4EDDE5C1.5000800@univ-mlv.fr>
	<CAE-f1xT5H2xo56Enk80Rfo1nmXApEwEKM8HFDz=70hLM+2_0Gg@mail.gmail.com>
Message-ID: <4EE11B0C.5050300@univ-mlv.fr>

On 12/08/2011 06:58 PM, Charles Oliver Nutter wrote:
> I just had a report of the same error in 1.7.0GA, and I saw it on the
> CI server last night after I set up a 1.7.0_01 build. Did it linger
> into GA?
>
> http://ci.jruby.org/job/jruby-test-matrix/jdk=sun-java-7,label=master/406/console
>
> On Tue, Dec 6, 2011 at 3:52 AM, R?mi Forax<forax at univ-mlv.fr>  wrote:
>> I've forget to say that a workaround is to erase the type used the callsite
>> before calling dynamicInvoker().
> Can you elaborate on this? I'm not using dynamicInvoker...I bind the
> site and use invokeWithArguments. Are you saying I could insert an
> asType() that clears everything to Object?

invokeWithArguments uses foldArgument and a generic invoker
that maybe why you see the same error.

about asType(), the idea is to erase the type before calling 
invokeWithArguments
instead of
   mh.invokeWithArgument(args);
you can use
   mh.asType(mh.type().erase()).invokeWithArguments(args);

and I have no idea if it works and about the perf cost if there is a cost.

>
> If so, would that introduce overhead?
>
> - Charlie

R?mi


From mleodaalder at gmail.com  Thu Dec  8 12:34:44 2011
From: mleodaalder at gmail.com (MLeo)
Date: Thu, 8 Dec 2011 21:34:44 +0100
Subject: Looping construction using only MethodHandles
In-Reply-To: <CAE-f1xQ5TDmNfXqvQAOPSKAc79qY3bd5PdxwBMb6XDCSp2xTYA@mail.gmail.com>
References: <CABA5GS5NswYi-a2gZPp71YtEyZSDGPmU=geZjPLrcw4tQ4LFVA@mail.gmail.com>
	<CAE-f1xQ5TDmNfXqvQAOPSKAc79qY3bd5PdxwBMb6XDCSp2xTYA@mail.gmail.com>
Message-ID: <CABA5GS7ny11dccOyEdtVOpANu=-4LQLtV2RuWfkq6QDgaQShRA@mail.gmail.com>

I do think I remember that (I've been subscribed to the list since
2009-01-21), this was mostly an attempt to see how far it would go (and to
get my brain wired for the MethodHandle API).

However, are the discounted code size budgets the only reason why the
MethodHandle 'loop' can recurse 2x to 6x more than a simpler POJM?
Obviously it doesn't generate stacktraces (at least not that show up), yet
it does consume stack.

One of the other things I'm going to attempt is to create my own While
adapter, with a static field, a bootstrap method and an invoke method that
does an invoke dynamic on the static field. I wonder how small I can get
the class file. I presume that the anonymous classloader in the sun package
will, at some point, be moved/finalized into a java.invoke or java.lang
form?

Thank you for taking a look,

--Maarten Daalder
On 6 December 2011 06:54, Charles Oliver Nutter <headius at headius.com> wrote:

> This actually came up early this past summer, when I tried to do the
> same thing (after Ola Bini showed me the MutableCallSite invoker
> trick. You can probably find the thread in July some time.
>
> The bottom line was that even if the handles could get folded into
> tail recursion, the call site ends up interfering with the
> MethodHandle optimizer's visibility into the graph.
>
> I was as disappointed as you are (or as you will be, once you've read
> this). I wish I'd been more involved early in the JSR-292 process and
> pushed for a looping construct. Unfortunately that train has sailed,
> and I've found no way to do loops (or any sort of backward branching)
> using the current set of method handle adapters :(
>
> Here's the tease, for future work on invokedynamic: I believe that if
> we had a way to do backward branches, method handles could become a
> turing-complete, user-accessible IR for the JVM's JIT.
>
> - Charlie
>
> On Mon, Dec 5, 2011 at 1:06 PM, MLeo <mleodaalder at gmail.com> wrote:
> > Hello all,
> >
> > Over the past few days I've been thinkering with a bit of code that folds
> > over the values of some container using some MethodHandle and a 'zero'
> > value.
> >
> > https://gist.github.com/1425706
> >
> > It's actually an implementation of a strategy to encode higher order
> > functions (HOF) without introducing a plethoramorphic callsite (if I
> > remember the term correctly). If you squint at it right sort of resembles
> > inversion of control. Or it's more like turning a function inside out.
> >
> > And it mostly works, with suprising result.
> > It's not entirely tail call optimized, it still eats stack, but not the
> same
> > amount as an naive unequivalent Java static method that is hand optimized
> > for the task (so no HOF).
> >
> > I'm using the x64 update 2 jdk on Windows Vista with -Xss10m. And the
> naive
> > version bails after a depth of ~110000 and after some optimization it
> will
> > get to ~310000 (after the first invocation!). The fold, on the other
> hand,
> > only bails out after recursing for ~650000 times and gets there nearly
> > immediately, the first run is ~10000 lower, but the remainder of
> invocations
> > is consistent (or seems that way, rather, I think it's because not enough
> > optimizations have kicked in yet).
> >
> > Execution time is another story, the POJM (Plain Old Java Method) takes
> > ~5ms, while the fold takes ~30ms. Which, now I think some more about it,
> is
> > more or less in line with the previous initial result, but still seems
> to be
> > a bit slow to me.
> > After some optimizations have kicked in (when the POJM reaches a depth of
> > 310000) the execution times stay roughly the same. So it would appear the
> > JVM inlined the POJM 2 or 3 times. Creating the stacktraces doesn't seem
> to
> > impact performance, if I make them recurse 100000 times (so it doesn't
> > overflow) then the execution time is roughly the same.
> >
> > Both ways are tested by invoking them through invokeWithArguments (I
> haven't
> > yet managed to get ASM to produce a test class for me), and I let both
> try
> > to sum an array (turned into an Iterator through
> > Arrays.asList(...).iterator()) of a million elements, so both will cause
> a
> > StackOverflowException. The test is to invoke a methods 1000 times (I
> got a
> > bit tired of waiting for a million times) and that I do 6 times (so I
> take 6
> > samples and average across them).
> >
> >
> > Now my question, is there something I can do to make it completely tail
> call
> > optimized? I've tried to 'rotate' the call graph/tree but that obviously
> > wouldn't work (it's still a direct recursion, no matter if you do it
> > directly or in the combiner of a foldArguments). It seems that it is
> almost
> > completely TCO already, but I haven't found where it's leaking stack.
> It's
> > definitely leaking less stack than a simple recursion.
> > Or will we need a special looping combinator here? I initially tried to
> > create a while combinator, however it seems that guardWithTest does not
> > accept MethodHandles returning void (for the target and fallback).
> >
> > If this can be made to work (so fully TCO'd, and maybe some massaging
> from
> > HotSpot engineers), then in theory it would allow for functional
> languages a
> > way to monomorphize HOFs.
> > For something like map (or fold) it will probably require a guard/PIC for
> > the implementations of that method, and those methods need to be
> compiled in
> > such a way so that they become bootstrap methods/factories (see the
> > constructor of MethodHandleFoldBuilder.java in the gist) which can then
> get
> > installed in the invocation of that fold.
> >
> > Thank you for your time,
> > --Maarten Daalder
> >
> > PS.
> >
> > In case you haven't noticed, I'm not a statistics wizard, so the numbers
> > mentioned should be taken only as an early indication and not of any
> > statistical significance. YMWV.
> >
> > Btw, if you were to implement 'map' with this then it would (if I
> noticed it
> > correctly) also reverse the list, or you're going to need to append each
> > element to the list, which will probably be slow. This is something I'm
> > going to try in the next few days or so.
> >
> > _______________________________________________
> > mlvm-dev mailing list
> > mlvm-dev at openjdk.java.net
> > http://mail.openjdk.java.net/mailman/listinfo/mlvm-dev
> >
> _______________________________________________
> mlvm-dev mailing list
> mlvm-dev at openjdk.java.net
> http://mail.openjdk.java.net/mailman/listinfo/mlvm-dev
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: http://mail.openjdk.java.net/pipermail/mlvm-dev/attachments/20111208/62dd733e/attachment.html 

From headius at headius.com  Thu Dec  8 23:35:30 2011
From: headius at headius.com (Charles Oliver Nutter)
Date: Fri, 9 Dec 2011 01:35:30 -0600
Subject: The NoClassDefFoundError bug is back :(
In-Reply-To: <4EE11B0C.5050300@univ-mlv.fr>
References: <4EDCE320.6060601@univ-mlv.fr> <4EDDE1C3.9070104@univ-mlv.fr>
	<4EDDE5C1.5000800@univ-mlv.fr>
	<CAE-f1xT5H2xo56Enk80Rfo1nmXApEwEKM8HFDz=70hLM+2_0Gg@mail.gmail.com>
	<4EE11B0C.5050300@univ-mlv.fr>
Message-ID: <CAE-f1xTnjch1f37z3xmNVr7O_4zA5DEVX5iDUcK3Uuad3KpJ6Q@mail.gmail.com>

On Thu, Dec 8, 2011 at 2:16 PM, R?mi Forax <forax at univ-mlv.fr> wrote:
> invokeWithArguments uses foldArgument and a generic invoker
> that maybe why you see the same error.
>
> about asType(), the idea is to erase the type before calling
> invokeWithArguments
> instead of
> ? mh.invokeWithArgument(args);
> you can use
> ? mh.asType(mh.type().erase()).invokeWithArguments(args);
>
> and I have no idea if it works and about the perf cost if there is a cost.

I'll try to give it a shot and see how it looks.

- Charlie

From john.r.rose at oracle.com  Sun Dec 11 00:03:08 2011
From: john.r.rose at oracle.com (john.r.rose at oracle.com)
Date: Sun, 11 Dec 2011 08:03:08 +0000
Subject: hg: mlvm/mlvm/hotspot: meth-lazy: checkpoint
Message-ID: <20111211080308.A41794766A@hg.openjdk.java.net>

Changeset: 395a651d3ac1
Author:    jrose
Date:      2011-12-11 00:03 -0800
URL:       http://hg.openjdk.java.net/mlvm/mlvm/hotspot/rev/395a651d3ac1

meth-lazy: checkpoint

! meth-lazy.patch


From john.r.rose at oracle.com  Sun Dec 11 00:04:02 2011
From: john.r.rose at oracle.com (john.r.rose at oracle.com)
Date: Sun, 11 Dec 2011 08:04:02 +0000
Subject: hg: mlvm/mlvm/jdk: 2 new changesets
Message-ID: <20111211080403.378F74766B@hg.openjdk.java.net>

Changeset: 2d9be279bfb5
Author:    jrose
Date:      2011-12-11 00:03 -0800
URL:       http://hg.openjdk.java.net/mlvm/mlvm/jdk/rev/2d9be279bfb5

warnings removal patch

! series
+ warnings-7117167.patch

Changeset: 7d35033c4f9f
Author:    jrose
Date:      2011-12-11 00:03 -0800
URL:       http://hg.openjdk.java.net/mlvm/mlvm/jdk/rev/7d35033c4f9f

meth-lazy: checkpoint

! meth-lazy.patch


From christian.thalinger at oracle.com  Mon Dec 12 04:04:00 2011
From: christian.thalinger at oracle.com (Christian Thalinger)
Date: Mon, 12 Dec 2011 13:04:00 +0100
Subject: The NoClassDefFoundError bug is back :(
In-Reply-To: <CAE-f1xT5H2xo56Enk80Rfo1nmXApEwEKM8HFDz=70hLM+2_0Gg@mail.gmail.com>
References: <4EDCE320.6060601@univ-mlv.fr> <4EDDE1C3.9070104@univ-mlv.fr>
	<4EDDE5C1.5000800@univ-mlv.fr>
	<CAE-f1xT5H2xo56Enk80Rfo1nmXApEwEKM8HFDz=70hLM+2_0Gg@mail.gmail.com>
Message-ID: <ABBC3B1E-511F-4926-8B02-462AD475CB4A@oracle.com>


On Dec 8, 2011, at 6:58 PM, Charles Oliver Nutter wrote:

> I just had a report of the same error in 1.7.0GA, and I saw it on the
> CI server last night after I set up a 1.7.0_01 build. Did it linger
> into GA?

We have (at least) one known bug with each early-adopter runtime which is not fixed yet:

7093795: JSR 292: Seph bench/bench_arithmetic.sp fails with: NoClassDefFoundError: seph/lang/SephObject
7093796: JSR 292: Rtalk test case fails with: NoClassDefFoundError: ri/core/rtalk/RtObject
7111583: JSR 292: JRuby test_respond_to.rb fails with: NoClassDefFoundError: org/jruby/runtime/ThreadContext

Some (all?) bugs root somewhere in the Java API and are difficult to fix (as far as I understand it).  All of this will go away eventually with the new implementation John is working on.

-- Chris

> 
> http://ci.jruby.org/job/jruby-test-matrix/jdk=sun-java-7,label=master/406/console
> 
> On Tue, Dec 6, 2011 at 3:52 AM, R?mi Forax <forax at univ-mlv.fr> wrote:
>> I've forget to say that a workaround is to erase the type used the callsite
>> before calling dynamicInvoker().
> 
> Can you elaborate on this? I'm not using dynamicInvoker...I bind the
> site and use invokeWithArguments. Are you saying I could insert an
> asType() that clears everything to Object?
> 
> If so, would that introduce overhead?
> 
> - Charlie
> _______________________________________________
> mlvm-dev mailing list
> mlvm-dev at openjdk.java.net
> http://mail.openjdk.java.net/mailman/listinfo/mlvm-dev


From christian.thalinger at oracle.com  Mon Dec 12 04:16:21 2011
From: christian.thalinger at oracle.com (Christian Thalinger)
Date: Mon, 12 Dec 2011 13:16:21 +0100
Subject: Coro patch
In-Reply-To: <4ED60760.5080401@jku.at>
References: <D5C30C24-04DD-41E3-91DC-0C267D22EE4D@gmail.com>	<9B961DA2-1064-4F14-8BCF-F94173C107BD@gmail.com>	<CALyUpY0BXBcE2Bw8sWCvGxiqjVDn+sRhn=aW7mBG6v5Ed3Nsxg@mail.gmail.com>	<CAE-f1xT4Y5rmec62tCao5DppdG_ALwQcJBr_P9-aQqGNOA5RsQ@mail.gmail.com>	<4ED4D70B.3010509@jku.at>	<CAE-f1xSiN4Cn_rRtJ25QK9cMgEEUQ1E06EpA7E+Y_P-LE5VwsA@mail.gmail.com>	<CALyUpY3hgM70-JcCSoao-m=8-F5i75N47YokQv7T-kTyvOhFyg@mail.gmail.com>	<CAE-f1xRR4VBz-Dp7W-x5UvY3+BSVpeG8mOr8vnD694ret8zU2A@mail.gmail.com>	<CAE-f1xSRa8Dh+cnJsjBYhwjee-qQ5cHpZy2T=zPrcTOzz5K9yQ@mail.gmail.com>	<4ED602B6.10101@jku.at>
	<CAE-f1xRfjiB3DVqY76YoEDF7ZWJpdRcyQNEG+So73i+xQ805Mw@mail.gmail.com>
	<4ED60760.5080401@jku.at>
Message-ID: <7392583D-ACF1-4A31-9DE7-F83032C9B50B@oracle.com>


On Nov 30, 2011, at 11:37 AM, Lukas Stadler wrote:

> On 2011-11-30 11:20, Charles Oliver Nutter wrote:
>> On Wed, Nov 30, 2011 at 4:17 AM, Lukas Stadler<lukas.stadler at jku.at>  wrote:
>>> Hm, maybe... the fix was really just a tiny tiny bugfix, so that
>>> shouldn't have caused any performance regressions, although, of course,
>>> I can't say for sure.
>>> But maybe something in invokedynamic has changed so that it's impacted
>>> by coro? I can reproduce it and I'll have to look into this.
>> I have not looked at compiler logs for indy at all...if you don't
>> suspect that indy is interfering with coro, then perhaps the execution
>> pattern is preventing indy from optimizing as well as it should.
> Exactly. It's still a bug in the coro patch, since it shouldn't impact 
> performance that way.
>>> Has your usage of invokedynamic changed a lot since the last "perfect"
>>> performance numbers with invokedynamic?
>> The numbers on your blog would not have been using invokedynamic at
>> all. What other numbers are you referring to?
>> 
>> JRuby is using invokedynamic more and more, but we're not doing
>> anything *unusual*.
> I thought that maybe you were refering to a measurement with indy that 
> showed the better numbers.
> But I guess it wouldn't have been a sudden decline in performance 
> anyway, since it's probably not one specific use of indy that exposes 
> the coro performance regression, but all of them together.

Could you look at the inlining first if we are good there?

-- Chris

> 
> - Lukas
> _______________________________________________
> mlvm-dev mailing list
> mlvm-dev at openjdk.java.net
> http://mail.openjdk.java.net/mailman/listinfo/mlvm-dev


From christian.thalinger at oracle.com  Mon Dec 12 04:25:09 2011
From: christian.thalinger at oracle.com (Christian Thalinger)
Date: Mon, 12 Dec 2011 13:25:09 +0100
Subject: How to monitor SwitchPoint's impact?
In-Reply-To: <CAE-f1xRSoX0wcg=K0fEAP38w6dMZ2=NToFwNNv0HypEdDtBLYg@mail.gmail.com>
References: <CAE-f1xTkez=8fVTzzm=fekwX3D3wUD4vfmBvUEtJS484WhHjwQ@mail.gmail.com>
	<6BC23874-5EE0-4CD5-A533-D4B6702042E7@oracle.com>
	<CAE-f1xRSoX0wcg=K0fEAP38w6dMZ2=NToFwNNv0HypEdDtBLYg@mail.gmail.com>
Message-ID: <A0693D27-4318-44EA-BEDC-C9F35460CA4A@oracle.com>

If the bugger here is a repeatedly invalidated SwitchPoint then we (reads: I) should fix:

7087838: JSR 292: add throttling logic for optimistic call site optimizations

I just don't have a good frequency-based logic yet...

-- Chris

On Dec 1, 2011, at 10:55 PM, Charles Oliver Nutter wrote:

> Perfect, Tom, thanks.
> 
> On Thu, Dec 1, 2011 at 3:21 PM, Tom Rodriguez <tom.rodriguez at oracle.com> wrote:
>> 
>> On Dec 1, 2011, at 1:34 AM, Charles Oliver Nutter wrote:
>> 
>>> My recent explorations into Stephen B's perf degradation on JRuby have
>>> led me to an unfortunate conclusion: something SwitchPoint-related is
>>> responsible for the remaining slowdown.
>>> 
>>> I had bugs I fixed, like the awful PIC-causes-repeat-rebinding issue
>>> or the heavy-class-mutation-never-stabilizes issue, but the last "fix"
>>> I found was to disable SwitchPoint-based call site invalidation
>>> entirely.
>> 
>> The upside of the push based invalidation of SwitchPoint is that the executed code runs quickly but the downside is that invalidation is expensive, possibly requiring a deopt and recompilation.  If you suspect that's happening, try running with -XX:+UnlockDiagnosticVMOptions -XX:+LogVMOutput -XX:+PrintCompilation and look for dependency_failed messages where type='call_site_target_value'.  Any following make_not_entrant messages are probably caused by the invalidation of the SwitchPoint.
>> 
>> tom
>> 
>>> 
>>> Now I'm not (yet) saying this is a perf issue in SwitchPoint itself.
>>> My problem is that it's very difficult to get any visibility into how
>>> frequently SwitchPoints are being invalidated (though I could
>>> instrument my own code, of course), or more importantly how
>>> drastically those invalidations are affecting performance. Does a
>>> single SwitchPoint invalidation happening repeatedly completely tank
>>> performance across the entire system? Does it require SwitchPoint
>>> invalidation that affects a broader surface area?
>>> 
>>> I will continue investigating on my side. I suspect the problem is
>>> that there's just a handful of SwitchPoint locations that are
>>> repeatedly invalidated, but that those invalidations are having a
>>> drastic global impact. I'm not yet sure how to work around such a
>>> situation, where one bad apple is spoiling the whole bunch...
>>> 
>>> - Charlie
>>> _______________________________________________
>>> mlvm-dev mailing list
>>> mlvm-dev at openjdk.java.net
>>> http://mail.openjdk.java.net/mailman/listinfo/mlvm-dev
>> 
>> _______________________________________________
>> mlvm-dev mailing list
>> mlvm-dev at openjdk.java.net
>> http://mail.openjdk.java.net/mailman/listinfo/mlvm-dev
> _______________________________________________
> mlvm-dev mailing list
> mlvm-dev at openjdk.java.net
> http://mail.openjdk.java.net/mailman/listinfo/mlvm-dev


From headius at headius.com  Mon Dec 12 08:38:23 2011
From: headius at headius.com (Charles Oliver Nutter)
Date: Mon, 12 Dec 2011 10:38:23 -0600
Subject: How to monitor SwitchPoint's impact?
In-Reply-To: <A0693D27-4318-44EA-BEDC-C9F35460CA4A@oracle.com>
References: <CAE-f1xTkez=8fVTzzm=fekwX3D3wUD4vfmBvUEtJS484WhHjwQ@mail.gmail.com>
	<6BC23874-5EE0-4CD5-A533-D4B6702042E7@oracle.com>
	<CAE-f1xRSoX0wcg=K0fEAP38w6dMZ2=NToFwNNv0HypEdDtBLYg@mail.gmail.com>
	<A0693D27-4318-44EA-BEDC-C9F35460CA4A@oracle.com>
Message-ID: <CAE-f1xTF8ezcYmhz_GiYD1MwUDnz6N2KpGq-Enj2XCvRepgr3w@mail.gmail.com>

I'm still of the opinion that you shouldn't fix it, or at least
shouldn't demote "noisy" call sites to never optimize. It's not
common, but there are Ruby programs that may define a new class or
method at runtime even at steady state. Not for every request, but
occasionally. I don't like the thought that a "read mostly" call site
that never completely settles down might get demoted and never
optimized when it's still mostly steady.

The cost of updating call sites or switch points is part of the fun :)
I consider that cost when I'm deciding how and when to update call
sites, how and when to failover my own logic to a demoted version, and
so on. I think it's my responsibility to deal with those costs, not
yours...

- Charlie

On Mon, Dec 12, 2011 at 6:25 AM, Christian Thalinger
<christian.thalinger at oracle.com> wrote:
> If the bugger here is a repeatedly invalidated SwitchPoint then we (reads: I) should fix:
>
> 7087838: JSR 292: add throttling logic for optimistic call site optimizations
>
> I just don't have a good frequency-based logic yet...
>
> -- Chris
>
> On Dec 1, 2011, at 10:55 PM, Charles Oliver Nutter wrote:
>
>> Perfect, Tom, thanks.
>>
>> On Thu, Dec 1, 2011 at 3:21 PM, Tom Rodriguez <tom.rodriguez at oracle.com> wrote:
>>>
>>> On Dec 1, 2011, at 1:34 AM, Charles Oliver Nutter wrote:
>>>
>>>> My recent explorations into Stephen B's perf degradation on JRuby have
>>>> led me to an unfortunate conclusion: something SwitchPoint-related is
>>>> responsible for the remaining slowdown.
>>>>
>>>> I had bugs I fixed, like the awful PIC-causes-repeat-rebinding issue
>>>> or the heavy-class-mutation-never-stabilizes issue, but the last "fix"
>>>> I found was to disable SwitchPoint-based call site invalidation
>>>> entirely.
>>>
>>> The upside of the push based invalidation of SwitchPoint is that the executed code runs quickly but the downside is that invalidation is expensive, possibly requiring a deopt and recompilation. ?If you suspect that's happening, try running with -XX:+UnlockDiagnosticVMOptions -XX:+LogVMOutput -XX:+PrintCompilation and look for dependency_failed messages where type='call_site_target_value'. ?Any following make_not_entrant messages are probably caused by the invalidation of the SwitchPoint.
>>>
>>> tom
>>>
>>>>
>>>> Now I'm not (yet) saying this is a perf issue in SwitchPoint itself.
>>>> My problem is that it's very difficult to get any visibility into how
>>>> frequently SwitchPoints are being invalidated (though I could
>>>> instrument my own code, of course), or more importantly how
>>>> drastically those invalidations are affecting performance. Does a
>>>> single SwitchPoint invalidation happening repeatedly completely tank
>>>> performance across the entire system? Does it require SwitchPoint
>>>> invalidation that affects a broader surface area?
>>>>
>>>> I will continue investigating on my side. I suspect the problem is
>>>> that there's just a handful of SwitchPoint locations that are
>>>> repeatedly invalidated, but that those invalidations are having a
>>>> drastic global impact. I'm not yet sure how to work around such a
>>>> situation, where one bad apple is spoiling the whole bunch...
>>>>
>>>> - Charlie
>>>> _______________________________________________
>>>> mlvm-dev mailing list
>>>> mlvm-dev at openjdk.java.net
>>>> http://mail.openjdk.java.net/mailman/listinfo/mlvm-dev
>>>
>>> _______________________________________________
>>> mlvm-dev mailing list
>>> mlvm-dev at openjdk.java.net
>>> http://mail.openjdk.java.net/mailman/listinfo/mlvm-dev
>> _______________________________________________
>> mlvm-dev mailing list
>> mlvm-dev at openjdk.java.net
>> http://mail.openjdk.java.net/mailman/listinfo/mlvm-dev
>
> _______________________________________________
> mlvm-dev mailing list
> mlvm-dev at openjdk.java.net
> http://mail.openjdk.java.net/mailman/listinfo/mlvm-dev

From headius at headius.com  Mon Dec 12 08:42:42 2011
From: headius at headius.com (Charles Oliver Nutter)
Date: Mon, 12 Dec 2011 10:42:42 -0600
Subject: How to monitor SwitchPoint's impact?
In-Reply-To: <CAE-f1xTF8ezcYmhz_GiYD1MwUDnz6N2KpGq-Enj2XCvRepgr3w@mail.gmail.com>
References: <CAE-f1xTkez=8fVTzzm=fekwX3D3wUD4vfmBvUEtJS484WhHjwQ@mail.gmail.com>
	<6BC23874-5EE0-4CD5-A533-D4B6702042E7@oracle.com>
	<CAE-f1xRSoX0wcg=K0fEAP38w6dMZ2=NToFwNNv0HypEdDtBLYg@mail.gmail.com>
	<A0693D27-4318-44EA-BEDC-C9F35460CA4A@oracle.com>
	<CAE-f1xTF8ezcYmhz_GiYD1MwUDnz6N2KpGq-Enj2XCvRepgr3w@mail.gmail.com>
Message-ID: <CAE-f1xTgcjRJQo5FbLVxPS1bo9pEM2UV3VaWMHQSygPc8pSqCQ@mail.gmail.com>

Oh, and for closure on this thread...

The benchmark I was looking at did indeed invalidate too frequently.
In this particular case, it was creating a new "singleton" class at
runtime for every iteration, and the subsequent method definitions
inside that class *each* triggered a SwitchPoint invalidation. I'm
looking at changes now that will only create (and invalidate) SPs for
invalidation after the SP has been bound into a call site. In other
words, during interpretation or early in execution, I should be able
to eliminate all SP use, and SPs will only come into play for
steady-state, JITed Ruby code.

- Charlie

On Mon, Dec 12, 2011 at 10:38 AM, Charles Oliver Nutter
<headius at headius.com> wrote:
> I'm still of the opinion that you shouldn't fix it, or at least
> shouldn't demote "noisy" call sites to never optimize. It's not
> common, but there are Ruby programs that may define a new class or
> method at runtime even at steady state. Not for every request, but
> occasionally. I don't like the thought that a "read mostly" call site
> that never completely settles down might get demoted and never
> optimized when it's still mostly steady.
>
> The cost of updating call sites or switch points is part of the fun :)
> I consider that cost when I'm deciding how and when to update call
> sites, how and when to failover my own logic to a demoted version, and
> so on. I think it's my responsibility to deal with those costs, not
> yours...
>
> - Charlie
>
> On Mon, Dec 12, 2011 at 6:25 AM, Christian Thalinger
> <christian.thalinger at oracle.com> wrote:
>> If the bugger here is a repeatedly invalidated SwitchPoint then we (reads: I) should fix:
>>
>> 7087838: JSR 292: add throttling logic for optimistic call site optimizations
>>
>> I just don't have a good frequency-based logic yet...
>>
>> -- Chris
>>
>> On Dec 1, 2011, at 10:55 PM, Charles Oliver Nutter wrote:
>>
>>> Perfect, Tom, thanks.
>>>
>>> On Thu, Dec 1, 2011 at 3:21 PM, Tom Rodriguez <tom.rodriguez at oracle.com> wrote:
>>>>
>>>> On Dec 1, 2011, at 1:34 AM, Charles Oliver Nutter wrote:
>>>>
>>>>> My recent explorations into Stephen B's perf degradation on JRuby have
>>>>> led me to an unfortunate conclusion: something SwitchPoint-related is
>>>>> responsible for the remaining slowdown.
>>>>>
>>>>> I had bugs I fixed, like the awful PIC-causes-repeat-rebinding issue
>>>>> or the heavy-class-mutation-never-stabilizes issue, but the last "fix"
>>>>> I found was to disable SwitchPoint-based call site invalidation
>>>>> entirely.
>>>>
>>>> The upside of the push based invalidation of SwitchPoint is that the executed code runs quickly but the downside is that invalidation is expensive, possibly requiring a deopt and recompilation. ?If you suspect that's happening, try running with -XX:+UnlockDiagnosticVMOptions -XX:+LogVMOutput -XX:+PrintCompilation and look for dependency_failed messages where type='call_site_target_value'. ?Any following make_not_entrant messages are probably caused by the invalidation of the SwitchPoint.
>>>>
>>>> tom
>>>>
>>>>>
>>>>> Now I'm not (yet) saying this is a perf issue in SwitchPoint itself.
>>>>> My problem is that it's very difficult to get any visibility into how
>>>>> frequently SwitchPoints are being invalidated (though I could
>>>>> instrument my own code, of course), or more importantly how
>>>>> drastically those invalidations are affecting performance. Does a
>>>>> single SwitchPoint invalidation happening repeatedly completely tank
>>>>> performance across the entire system? Does it require SwitchPoint
>>>>> invalidation that affects a broader surface area?
>>>>>
>>>>> I will continue investigating on my side. I suspect the problem is
>>>>> that there's just a handful of SwitchPoint locations that are
>>>>> repeatedly invalidated, but that those invalidations are having a
>>>>> drastic global impact. I'm not yet sure how to work around such a
>>>>> situation, where one bad apple is spoiling the whole bunch...
>>>>>
>>>>> - Charlie
>>>>> _______________________________________________
>>>>> mlvm-dev mailing list
>>>>> mlvm-dev at openjdk.java.net
>>>>> http://mail.openjdk.java.net/mailman/listinfo/mlvm-dev
>>>>
>>>> _______________________________________________
>>>> mlvm-dev mailing list
>>>> mlvm-dev at openjdk.java.net
>>>> http://mail.openjdk.java.net/mailman/listinfo/mlvm-dev
>>> _______________________________________________
>>> mlvm-dev mailing list
>>> mlvm-dev at openjdk.java.net
>>> http://mail.openjdk.java.net/mailman/listinfo/mlvm-dev
>>
>> _______________________________________________
>> mlvm-dev mailing list
>> mlvm-dev at openjdk.java.net
>> http://mail.openjdk.java.net/mailman/listinfo/mlvm-dev

From chen.xi at telecom-bretagne.eu  Mon Dec 12 08:56:28 2011
From: chen.xi at telecom-bretagne.eu (chen.xi at telecom-bretagne.eu)
Date: Mon, 12 Dec 2011 17:56:28 +0100
Subject: problem: fail to apply the patch
Message-ID: <20111212175628.25196jlejb6vxv0o@webmail.telecom-bretagne.eu>



I went to the /patches/make directory and executed the make command as follows.

1) make setup, to create symbolic link and set guard. The last output is:
------------------------
# report what happened:
cd ../..; bash patches/make/each-patch-repo.sh 'hg qselect; hg qunapplied'
+ (cd sources/.; hg qselect; hg qunapplied)
0b66a233bfb9
buildable
testable
+ (cd sources/hotspot; hg qselect; hg qunapplied)
buildable
d8cb48376797
testable
+ (cd sources/jdk; hg qselect; hg qunapplied)
7ab0d613cd1a
buildable
testable
meth-acc-7077803.patch
cval-tune-7030453.patch
meth.patch
meth-experiment.patch
continuation.patch
anonk.patch
coro.patch
+ (cd sources/langtools; hg qselect; hg qunapplied)
5df63fd8fa64
buildable
testable
------------------------

2) make check, I got errors, so I continued to do,

3) make force

4) make check, to review if the force command had been taken into effect. the output is:
------------------------
# check the difference betwen tip version and patch-required version, if there is difference,
# then create file current-diffs.txt in the source repo and exit with 1
# If this command fails, consider setting RELAX_CHECKS=true or FORCE_VERSIONS=true.
cd ../..; bash patches/make/each-patch-repo.sh \
        hg diff --git -I src -r "\$(bash `pwd`/patches/make/current-release.sh)" \
        "> current-diffs.txt;" \
        "if [ ! -s current-diffs.txt ]; then rm current-diffs.txt;" \
        "else grep '^diff' current-diffs.txt; exit 1; fi"
+ (cd sources/.; hg diff --git -I src -r $(bash /home/xc1427/continuation_research/davinci.v1/patches/make/current-release.sh) > current-diffs.txt; if [ ! -s current-diffs.txt ]; then rm current-diffs.txt; else grep '^diff' current-diffs.txt; exit 1; fi)
+ (cd sources/hotspot; hg diff --git -I src -r $(bash /home/xc1427/continuation_research/davinci.v1/patches/make/current-release.sh) > current-diffs.txt; if [ ! -s current-diffs.txt ]; then rm current-diffs.txt; else grep '^diff' current-diffs.txt; exit 1; fi)
mq status file refers to unknown node 4c9ea0147722
mq status file refers to unknown node daf841a4cc84
+ (cd sources/jdk; hg diff --git -I src -r $(bash /home/xc1427/continuation_research/davinci.v1/patches/make/current-release.sh) > current-diffs.txt; if [ ! -s current-diffs.txt ]; then rm current-diffs.txt; else grep '^diff' current-diffs.txt; exit 1; fi)
mq status file refers to unknown node f2029fe990b5
mq status file refers to unknown node f2029fe990b5
+ (cd sources/langtools; hg diff --git -I src -r $(bash /home/xc1427/continuation_research/davinci.v1/patches/make/current-release.sh) > current-diffs.txt; if [ ! -s current-diffs.txt ]; then rm current-diffs.txt; else grep '^diff' current-diffs.txt; exit 1; fi)
mq status file refers to unknown node 0001c23b2b4b
mq status file refers to unknown node 0001c23b2b4b
------------------------

we can notice the strange message as "refers to unknow node XXXX". However, there is no more error.

5) make patch. This is where I failed. the output is:
------------------------
cd ../..; bash patches/make/each-patch-repo.sh \
        '(! hg qunapplied | read something) || hg qpush -a'
+ (cd sources/.; (! hg qunapplied | read something) || hg qpush -a)
+ (cd sources/hotspot; (! hg qunapplied | read something) || hg qpush -a)
+ (cd sources/jdk; (! hg qunapplied | read something) || hg qpush -a)
mq status file refers to unknown node f2029fe990b5
(working directory not at a head)
abort: working directory revision is not qtip
*** Exit status 255.
+ (cd sources/langtools; (! hg qunapplied | read something) || hg qpush -a)
make: *** [mq-push-patches] Error 255
------------------------

As we see, the error comes out as "working directory revision is not qtip". I searched in google and mercurial wiki, but found few helpful informations.

== My point of view ==
I've ever tried "hg qapplied" in the sources/hotspot right after the first step. I got supprised by the output:
----------
mac-tweaks.patch
meth.patch
coro-meth.patch
coro.patch
----------
which means some patches had been applied before I did nothing. And recall the previous strange message "refers to unknow node XXXX".

I got frustrated at the very first stage. Can somebody help me?
-------------- next part --------------
An HTML attachment was scrubbed...
URL: http://mail.openjdk.java.net/pipermail/mlvm-dev/attachments/20111212/8fba55ca/attachment-0001.html 

From chen.xi at telecom-bretagne.eu  Mon Dec 12 09:17:08 2011
From: chen.xi at telecom-bretagne.eu (chen.xi at telecom-bretagne.eu)
Date: Mon, 12 Dec 2011 18:17:08 +0100
Subject: problem: fail to apply the patch [continue]
Message-ID: <20111212181708.93972jjz2maj8o4k@webmail.telecom-bretagne.eu>



I review the output message. I come to know what the error has come from.

In fact, when I did "make setup", some patches which had already been in the source rep was not popped at all. relative output is:
---------------------------------
cd ../..; bash patches/make/each-patch-repo.sh \
        "hg qpop -a; hg qselect buildable testable" \
            "$(uname -s | sed -n 's/Darwin/bsd-port/p')" \
            "\$(bash `pwd`/patches/make/current-release.sh)"
+ (cd sources/.; hg qpop -a; hg qselect buildable testable  $(bash /home/xc1427/continuation_research/davinci.v1/patches/make/current-release.sh))
no patches applied
+ (cd sources/hotspot; hg qpop -a; hg qselect buildable testable  $(bash /home/xc1427/continuation_research/davinci.v1/patches/make/current-release.sh))
abort: trying to pop unknown node daf841a4cc84
+ (cd sources/jdk; hg qpop -a; hg qselect buildable testable  $(bash /home/xc1427/continuation_research/davinci.v1/patches/make/current-release.sh))
abort: trying to pop unknown node f2029fe990b5
+ (cd sources/langtools; hg qpop -a; hg qselect buildable testable  $(bash /home/xc1427/continuation_research/davinci.v1/patches/make/current-release.sh))
abort: trying to pop unknown node 0001c23b2b4b
---------------------------------

Therefore, at the point that I updated the source revision to conform to patches, CHANGES WAS MADE BEFORE ALL THE PATCHES HAS BEEN POPPED.  So I got the error.

I think the causality is clear. Can anybody tell me how to pop all the patches ? (obviously qpop -a cannot do this....)
-------------- next part --------------
An HTML attachment was scrubbed...
URL: http://mail.openjdk.java.net/pipermail/mlvm-dev/attachments/20111212/caf73e5f/attachment.html 

From forax at univ-mlv.fr  Mon Dec 12 15:31:44 2011
From: forax at univ-mlv.fr (=?UTF-8?B?UsOpbWkgRm9yYXg=?=)
Date: Tue, 13 Dec 2011 00:31:44 +0100
Subject: How to monitor SwitchPoint's impact?
In-Reply-To: <CAE-f1xTF8ezcYmhz_GiYD1MwUDnz6N2KpGq-Enj2XCvRepgr3w@mail.gmail.com>
References: <CAE-f1xTkez=8fVTzzm=fekwX3D3wUD4vfmBvUEtJS484WhHjwQ@mail.gmail.com>
	<6BC23874-5EE0-4CD5-A533-D4B6702042E7@oracle.com>
	<CAE-f1xRSoX0wcg=K0fEAP38w6dMZ2=NToFwNNv0HypEdDtBLYg@mail.gmail.com>
	<A0693D27-4318-44EA-BEDC-C9F35460CA4A@oracle.com>
	<CAE-f1xTF8ezcYmhz_GiYD1MwUDnz6N2KpGq-Enj2XCvRepgr3w@mail.gmail.com>
Message-ID: <4EE68EE0.6020906@univ-mlv.fr>

On 12/12/2011 05:38 PM, Charles Oliver Nutter wrote:
> I'm still of the opinion that you shouldn't fix it, or at least
> shouldn't demote "noisy" call sites to never optimize. It's not
> common, but there are Ruby programs that may define a new class or
> method at runtime even at steady state. Not for every request, but
> occasionally. I don't like the thought that a "read mostly" call site
> that never completely settles down might get demoted and never
> optimized when it's still mostly steady.
>
> The cost of updating call sites or switch points is part of the fun :)
> I consider that cost when I'm deciding how and when to update call
> sites, how and when to failover my own logic to a demoted version, and
> so on. I think it's my responsibility to deal with those costs, not
> yours...
>
> - Charlie

I agree.

In my opinion, it's up to the language runtime designer to estimate
if it's a good idea to invalidate something at some point or not
because the runtime has more information than the VM.

Moreover any throttle logic just make the API less predictable,
the throttle logic should be removed to help the designers
to easily predict the behaviour of their runtimes.

R?mi

>
> On Mon, Dec 12, 2011 at 6:25 AM, Christian Thalinger
> <christian.thalinger at oracle.com>  wrote:
>> If the bugger here is a repeatedly invalidated SwitchPoint then we (reads: I) should fix:
>>
>> 7087838: JSR 292: add throttling logic for optimistic call site optimizations
>>
>> I just don't have a good frequency-based logic yet...
>>
>> -- Chris
>>
>> On Dec 1, 2011, at 10:55 PM, Charles Oliver Nutter wrote:
>>
>>> Perfect, Tom, thanks.
>>>
>>> On Thu, Dec 1, 2011 at 3:21 PM, Tom Rodriguez<tom.rodriguez at oracle.com>  wrote:
>>>> On Dec 1, 2011, at 1:34 AM, Charles Oliver Nutter wrote:
>>>>
>>>>> My recent explorations into Stephen B's perf degradation on JRuby have
>>>>> led me to an unfortunate conclusion: something SwitchPoint-related is
>>>>> responsible for the remaining slowdown.
>>>>>
>>>>> I had bugs I fixed, like the awful PIC-causes-repeat-rebinding issue
>>>>> or the heavy-class-mutation-never-stabilizes issue, but the last "fix"
>>>>> I found was to disable SwitchPoint-based call site invalidation
>>>>> entirely.
>>>> The upside of the push based invalidation of SwitchPoint is that the executed code runs quickly but the downside is that invalidation is expensive, possibly requiring a deopt and recompilation.  If you suspect that's happening, try running with -XX:+UnlockDiagnosticVMOptions -XX:+LogVMOutput -XX:+PrintCompilation and look for dependency_failed messages where type='call_site_target_value'.  Any following make_not_entrant messages are probably caused by the invalidation of the SwitchPoint.
>>>>
>>>> tom
>>>>
>>>>> Now I'm not (yet) saying this is a perf issue in SwitchPoint itself.
>>>>> My problem is that it's very difficult to get any visibility into how
>>>>> frequently SwitchPoints are being invalidated (though I could
>>>>> instrument my own code, of course), or more importantly how
>>>>> drastically those invalidations are affecting performance. Does a
>>>>> single SwitchPoint invalidation happening repeatedly completely tank
>>>>> performance across the entire system? Does it require SwitchPoint
>>>>> invalidation that affects a broader surface area?
>>>>>
>>>>> I will continue investigating on my side. I suspect the problem is
>>>>> that there's just a handful of SwitchPoint locations that are
>>>>> repeatedly invalidated, but that those invalidations are having a
>>>>> drastic global impact. I'm not yet sure how to work around such a
>>>>> situation, where one bad apple is spoiling the whole bunch...
>>>>>
>>>>> - Charlie
>>>>> _______________________________________________
>>>>> mlvm-dev mailing list
>>>>> mlvm-dev at openjdk.java.net
>>>>> http://mail.openjdk.java.net/mailman/listinfo/mlvm-dev
>>>> _______________________________________________
>>>> mlvm-dev mailing list
>>>> mlvm-dev at openjdk.java.net
>>>> http://mail.openjdk.java.net/mailman/listinfo/mlvm-dev
>>> _______________________________________________
>>> mlvm-dev mailing list
>>> mlvm-dev at openjdk.java.net
>>> http://mail.openjdk.java.net/mailman/listinfo/mlvm-dev
>> _______________________________________________
>> mlvm-dev mailing list
>> mlvm-dev at openjdk.java.net
>> http://mail.openjdk.java.net/mailman/listinfo/mlvm-dev
> _______________________________________________
> mlvm-dev mailing list
> mlvm-dev at openjdk.java.net
> http://mail.openjdk.java.net/mailman/listinfo/mlvm-dev


From fweimer at bfk.de  Thu Dec  8 01:55:00 2011
From: fweimer at bfk.de (Florian Weimer)
Date: Thu, 08 Dec 2011 09:55:00 +0000
Subject: review request (L): 7030453: JSR 292 ClassValue.get method is too
	slow
In-Reply-To: <2C2A1E0C-876C-4DC6-9812-E53E2619D272@oracle.com> (John Rose's
	message of "Wed, 7 Dec 2011 13:14:33 -0800 (PST)")
References: <BB480899-E037-4BF2-9407-039BEA8131EC@oracle.com>
	<5AF69B9B-D7A9-47B0-8452-06308D824D70@oracle.com>
	<1A25D30F-204C-4CEE-95A1-F6FDB7925B7D@oracle.com>
	<4EDBCC2F.7030907@oracle.com> <4EDBF093.1000801@univ-mlv.fr>
	<4EDC0FFE.2080805@oracle.com> <4EDC1851.4060508@univ-mlv.fr>
	<4EDC2B98.6020101@oracle.com>
	<2C2A1E0C-876C-4DC6-9812-E53E2619D272@oracle.com>
Message-ID: <821usf2xsb.fsf@mid.bfk.de>

* John Rose:

> But, in order to respect the "general aim" you are mentioning, I have
> unhoisted one of the two words from the Class instance itself.  This
> will cause a minor slowdown in JSR 292 use cases.

What about using ClassValue for the various caches instead?
enumConstants and enumConstantDirectory seem good candidates (callers
cache the value anyway or do additional work while accessing the field).

-- 
Florian Weimer                <fweimer at bfk.de>
BFK edv-consulting GmbH       http://www.bfk.de/
Kriegsstra?e 100              tel: +49-721-96201-1
D-76133 Karlsruhe             fax: +49-721-96201-99

From christian.thalinger at oracle.com  Tue Dec 13 00:24:37 2011
From: christian.thalinger at oracle.com (Christian Thalinger)
Date: Tue, 13 Dec 2011 09:24:37 +0100
Subject: How to monitor SwitchPoint's impact?
In-Reply-To: <4EE68EE0.6020906@univ-mlv.fr>
References: <CAE-f1xTkez=8fVTzzm=fekwX3D3wUD4vfmBvUEtJS484WhHjwQ@mail.gmail.com>
	<6BC23874-5EE0-4CD5-A533-D4B6702042E7@oracle.com>
	<CAE-f1xRSoX0wcg=K0fEAP38w6dMZ2=NToFwNNv0HypEdDtBLYg@mail.gmail.com>
	<A0693D27-4318-44EA-BEDC-C9F35460CA4A@oracle.com>
	<CAE-f1xTF8ezcYmhz_GiYD1MwUDnz6N2KpGq-Enj2XCvRepgr3w@mail.gmail.com>
	<4EE68EE0.6020906@univ-mlv.fr>
Message-ID: <24F8C43D-DCE5-4C58-9338-0F5F5708C028@oracle.com>


On Dec 13, 2011, at 12:31 AM, R?mi Forax wrote:

> On 12/12/2011 05:38 PM, Charles Oliver Nutter wrote:
>> I'm still of the opinion that you shouldn't fix it, or at least
>> shouldn't demote "noisy" call sites to never optimize. It's not
>> common, but there are Ruby programs that may define a new class or
>> method at runtime even at steady state. Not for every request, but
>> occasionally. I don't like the thought that a "read mostly" call site
>> that never completely settles down might get demoted and never
>> optimized when it's still mostly steady.
>> 
>> The cost of updating call sites or switch points is part of the fun :)
>> I consider that cost when I'm deciding how and when to update call
>> sites, how and when to failover my own logic to a demoted version, and
>> so on. I think it's my responsibility to deal with those costs, not
>> yours...
>> 
>> - Charlie
> 
> I agree.
> 
> In my opinion, it's up to the language runtime designer to estimate
> if it's a good idea to invalidate something at some point or not
> because the runtime has more information than the VM.
> 
> Moreover any throttle logic just make the API less predictable,
> the throttle logic should be removed to help the designers
> to easily predict the behaviour of their runtimes.

I'm fine with that.  Currently there is no such logic.

-- Chris

> 
> R?mi
> 
>> 
>> On Mon, Dec 12, 2011 at 6:25 AM, Christian Thalinger
>> <christian.thalinger at oracle.com>  wrote:
>>> If the bugger here is a repeatedly invalidated SwitchPoint then we (reads: I) should fix:
>>> 
>>> 7087838: JSR 292: add throttling logic for optimistic call site optimizations
>>> 
>>> I just don't have a good frequency-based logic yet...
>>> 
>>> -- Chris
>>> 
>>> On Dec 1, 2011, at 10:55 PM, Charles Oliver Nutter wrote:
>>> 
>>>> Perfect, Tom, thanks.
>>>> 
>>>> On Thu, Dec 1, 2011 at 3:21 PM, Tom Rodriguez<tom.rodriguez at oracle.com>  wrote:
>>>>> On Dec 1, 2011, at 1:34 AM, Charles Oliver Nutter wrote:
>>>>> 
>>>>>> My recent explorations into Stephen B's perf degradation on JRuby have
>>>>>> led me to an unfortunate conclusion: something SwitchPoint-related is
>>>>>> responsible for the remaining slowdown.
>>>>>> 
>>>>>> I had bugs I fixed, like the awful PIC-causes-repeat-rebinding issue
>>>>>> or the heavy-class-mutation-never-stabilizes issue, but the last "fix"
>>>>>> I found was to disable SwitchPoint-based call site invalidation
>>>>>> entirely.
>>>>> The upside of the push based invalidation of SwitchPoint is that the executed code runs quickly but the downside is that invalidation is expensive, possibly requiring a deopt and recompilation.  If you suspect that's happening, try running with -XX:+UnlockDiagnosticVMOptions -XX:+LogVMOutput -XX:+PrintCompilation and look for dependency_failed messages where type='call_site_target_value'.  Any following make_not_entrant messages are probably caused by the invalidation of the SwitchPoint.
>>>>> 
>>>>> tom
>>>>> 
>>>>>> Now I'm not (yet) saying this is a perf issue in SwitchPoint itself.
>>>>>> My problem is that it's very difficult to get any visibility into how
>>>>>> frequently SwitchPoints are being invalidated (though I could
>>>>>> instrument my own code, of course), or more importantly how
>>>>>> drastically those invalidations are affecting performance. Does a
>>>>>> single SwitchPoint invalidation happening repeatedly completely tank
>>>>>> performance across the entire system? Does it require SwitchPoint
>>>>>> invalidation that affects a broader surface area?
>>>>>> 
>>>>>> I will continue investigating on my side. I suspect the problem is
>>>>>> that there's just a handful of SwitchPoint locations that are
>>>>>> repeatedly invalidated, but that those invalidations are having a
>>>>>> drastic global impact. I'm not yet sure how to work around such a
>>>>>> situation, where one bad apple is spoiling the whole bunch...
>>>>>> 
>>>>>> - Charlie
>>>>>> _______________________________________________
>>>>>> mlvm-dev mailing list
>>>>>> mlvm-dev at openjdk.java.net
>>>>>> http://mail.openjdk.java.net/mailman/listinfo/mlvm-dev
>>>>> _______________________________________________
>>>>> mlvm-dev mailing list
>>>>> mlvm-dev at openjdk.java.net
>>>>> http://mail.openjdk.java.net/mailman/listinfo/mlvm-dev
>>>> _______________________________________________
>>>> mlvm-dev mailing list
>>>> mlvm-dev at openjdk.java.net
>>>> http://mail.openjdk.java.net/mailman/listinfo/mlvm-dev
>>> _______________________________________________
>>> mlvm-dev mailing list
>>> mlvm-dev at openjdk.java.net
>>> http://mail.openjdk.java.net/mailman/listinfo/mlvm-dev
>> _______________________________________________
>> mlvm-dev mailing list
>> mlvm-dev at openjdk.java.net
>> http://mail.openjdk.java.net/mailman/listinfo/mlvm-dev
> 
> _______________________________________________
> mlvm-dev mailing list
> mlvm-dev at openjdk.java.net
> http://mail.openjdk.java.net/mailman/listinfo/mlvm-dev


From christian.thalinger at oracle.com  Tue Dec 13 01:51:52 2011
From: christian.thalinger at oracle.com (Christian Thalinger)
Date: Tue, 13 Dec 2011 10:51:52 +0100
Subject: problem: fail to apply the patch [continue]
In-Reply-To: <20111212181708.93972jjz2maj8o4k@webmail.telecom-bretagne.eu>
References: <20111212181708.93972jjz2maj8o4k@webmail.telecom-bretagne.eu>
Message-ID: <E561015A-7021-4939-AD0C-144B3C76A71A@oracle.com>


On Dec 12, 2011, at 6:17 PM, chen.xi at telecom-bretagne.eu wrote:

> 
> I review the output message. I come to know what the error has come from.
> 
> In fact, when I did "make setup", some patches which had already been in the source rep was not popped at all. relative output is:
> ---------------------------------
> cd ../..; bash patches/make/each-patch-repo.sh \
>         "hg qpop -a; hg qselect buildable testable" \
>             "$(uname -s | sed -n 's/Darwin/bsd-port/p')" \
>             "\$(bash `pwd`/patches/make/current-release.sh)"
> + (cd sources/.; hg qpop -a; hg qselect buildable testable  $(bash /home/xc1427/continuation_research/davinci.v1/patches/make/current-release.sh))
> no patches applied
> + (cd sources/hotspot; hg qpop -a; hg qselect buildable testable  $(bash /home/xc1427/continuation_research/davinci.v1/patches/make/current-release.sh))
> abort: trying to pop unknown node daf841a4cc84
> + (cd sources/jdk; hg qpop -a; hg qselect buildable testable  $(bash /home/xc1427/continuation_research/davinci.v1/patches/make/current-release.sh))
> abort: trying to pop unknown node f2029fe990b5
> + (cd sources/langtools; hg qpop -a; hg qselect buildable testable  $(bash /home/xc1427/continuation_research/davinci.v1/patches/make/current-release.sh))
> abort: trying to pop unknown node 0001c23b2b4b
> ---------------------------------
> 
> Therefore, at the point that I updated the source revision to conform to patches, changes was made before all the patches has been popped.  So I got the error.
> 
> I think the causality is clear. Can anybody tell me how to pop all the patches ? (obviously qpop -a cannot do this....)

What OpenJDK forest did you clone?  If qpop doesn't work you can strip the changesets you don't want.

-- Chris

> _______________________________________________
> mlvm-dev mailing list
> mlvm-dev at openjdk.java.net
> http://mail.openjdk.java.net/mailman/listinfo/mlvm-dev

-------------- next part --------------
An HTML attachment was scrubbed...
URL: http://mail.openjdk.java.net/pipermail/mlvm-dev/attachments/20111213/7f7ccf13/attachment.html 

From john.r.rose at oracle.com  Tue Dec 13 12:41:13 2011
From: john.r.rose at oracle.com (john.r.rose at oracle.com)
Date: Tue, 13 Dec 2011 20:41:13 +0000
Subject: hg: mlvm/mlvm/jdk: 2 new changesets
Message-ID: <20111213204114.0E3DB4769C@hg.openjdk.java.net>

Changeset: b9c18afea366
Author:    jrose
Date:      2011-12-13 12:40 -0800
URL:       http://hg.openjdk.java.net/mlvm/mlvm/jdk/rev/b9c18afea366

remove indy-bsm-postjdk7.patch

! series

Changeset: 5d5463d1cb3e
Author:    jrose
Date:      2011-12-13 12:41 -0800
URL:       http://hg.openjdk.java.net/mlvm/mlvm/jdk/rev/5d5463d1cb3e

meth-lazy: remove BMH scaffolding from LzMH

! meth-lazy.patch


From linuxhippy at gmail.com  Tue Dec 13 14:52:40 2011
From: linuxhippy at gmail.com (Clemens Eisserer)
Date: Tue, 13 Dec 2011 23:52:40 +0100
Subject: JRuby startup costs [was: Process-level fork on OpenJDK...is it
	madness?]
In-Reply-To: <CAE-f1xSVsVB2J_k-0mmb2-b+J1+4wF9kE3JdQq7zgzVsfJ2UEA@mail.gmail.com>
References: <FAD15808DC12914D83C3793EEA3E556306F35DE3@BUDMLVEM09.e2k.ad.ge.com>
	<CAE-f1xSVsVB2J_k-0mmb2-b+J1+4wF9kE3JdQq7zgzVsfJ2UEA@mail.gmail.com>
Message-ID: <CAFvQSYSXcT_Xdt6Dw1V+3H6HkyYG9AsJ+zHS4Ejkec0TtRNgsQ@mail.gmail.com>

Hi Oliver,

> Another more subtle aspect is that our interpreter gets "hot" *very*
> quickly, since it's running the same code over and over again
> regardless of what Ruby code is executing. If all that Ruby code were
> JVM bytecode, each piece would have to get "hot" to run fast...so
> interpreting wins there too.

Hmm, from what I understand, hotspot's interpreter is already
"super-hot" right after startup.
Whereas JRuby's interpreter is jvm bytecode that needs to be compiled
by a compiler, hotspot's interpreter is *hand-tuned* low-level
assembly.

- Clemens

From headius at headius.com  Tue Dec 13 16:51:56 2011
From: headius at headius.com (Charles Oliver Nutter)
Date: Tue, 13 Dec 2011 18:51:56 -0600
Subject: JRuby startup costs [was: Process-level fork on OpenJDK...is it
	madness?]
In-Reply-To: <CAFvQSYSXcT_Xdt6Dw1V+3H6HkyYG9AsJ+zHS4Ejkec0TtRNgsQ@mail.gmail.com>
References: <FAD15808DC12914D83C3793EEA3E556306F35DE3@BUDMLVEM09.e2k.ad.ge.com>
	<CAE-f1xSVsVB2J_k-0mmb2-b+J1+4wF9kE3JdQq7zgzVsfJ2UEA@mail.gmail.com>
	<CAFvQSYSXcT_Xdt6Dw1V+3H6HkyYG9AsJ+zHS4Ejkec0TtRNgsQ@mail.gmail.com>
Message-ID: <CAE-f1xTd2Dz-t45Mat=-5GVoA=vQ+8=resEmrHqQ+7yeOQ392g@mail.gmail.com>

On Tue, Dec 13, 2011 at 4:52 PM, Clemens Eisserer <linuxhippy at gmail.com> wrote:
> Hmm, from what I understand, hotspot's interpreter is already
> "super-hot" right after startup.
> Whereas JRuby's interpreter is jvm bytecode that needs to be compiled
> by a compiler, hotspot's interpreter is *hand-tuned* low-level
> assembly.

This is all true. The subtle part is that our interpreter runs faster
when jitted by the JVM than the JVM's interpreter can run compiled
Ruby code.

Here's "fib", first interpreted (by JRuby) but with the interpreter
jitted (by the JVM), and second compiled (by JRuby) but interpreted
(by the JVM):

system ~/projects/jruby $ jruby -X-C bench/bench_fib_recursive.rb 5 35
  3.179000   0.000000   3.179000 (  3.150000)
  3.650000   0.000000   3.650000 (  3.650000)
  3.739000   0.000000   3.739000 (  3.740000)
  3.733000   0.000000   3.733000 (  3.733000)
  3.778000   0.000000   3.778000 (  3.778000)

system ~/projects/jruby $ jruby -X+C -J-Xint bench/bench_fib_recursive.rb 5 35
 75.825000   0.000000  75.825000 ( 75.805000)
 75.994000   0.000000  75.994000 ( 75.994000)

Now this isn't *quite* fair since more than just the Ruby code is
running interpreted in the second case, but it illustrates a
point...our interpreter actually performs surprisingly well compared
to fully-interpreted Java, and as a result having Ruby code interpret
first often leads to things "warming up" more quickly (with the
alternative being to only ever run compiled Ruby code, but accept it
will be "cold" for 10000 calls).

- Charlie

From john.r.rose at oracle.com  Tue Dec 13 17:11:33 2011
From: john.r.rose at oracle.com (john.r.rose at oracle.com)
Date: Wed, 14 Dec 2011 01:11:33 +0000
Subject: hg: mlvm/mlvm/jdk: 4 new changesets
Message-ID: <20111214011134.E7264476A2@hg.openjdk.java.net>

Changeset: ea20028ecbf4
Author:    jrose
Date:      2011-12-13 17:08 -0800
URL:       http://hg.openjdk.java.net/mlvm/mlvm/jdk/rev/ea20028ecbf4

cval: move ClassValueMap into ClassValue

! cval-tune-7030453.patch

Changeset: 4353b8785309
Author:    jrose
Date:      2011-12-13 17:08 -0800
URL:       http://hg.openjdk.java.net/mlvm/mlvm/jdk/rev/4353b8785309

warnings: update

! warnings-7117167.patch

Changeset: 51b9d460f0b0
Author:    jrose
Date:      2011-12-13 17:10 -0800
URL:       http://hg.openjdk.java.net/mlvm/mlvm/jdk/rev/51b9d460f0b0

rebase patches

! jdk7-b147-to-bsd-port.patch
! series

Changeset: 94b34e043bc2
Author:    jrose
Date:      2011-12-13 17:11 -0800
URL:       http://hg.openjdk.java.net/mlvm/mlvm/jdk/rev/94b34e043bc2

meth-lazy: update

+ meth-lazy-7023639.bcg.patch
+ meth-lazy-7023639.patch
- meth-lazy.patch
! series


From john.r.rose at oracle.com  Tue Dec 13 17:14:50 2011
From: john.r.rose at oracle.com (john.r.rose at oracle.com)
Date: Wed, 14 Dec 2011 01:14:50 +0000
Subject: hg: mlvm/mlvm/hotspot: 2 new changesets
Message-ID: <20111214011451.1C2EC476A3@hg.openjdk.java.net>

Changeset: 3dfb22ef25a5
Author:    jrose
Date:      2011-12-13 17:12 -0800
URL:       http://hg.openjdk.java.net/mlvm/mlvm/hotspot/rev/3dfb22ef25a5

rebase patches

! mac-tweaks.patch
! series

Changeset: 89129696dbc4
Author:    jrose
Date:      2011-12-13 17:14 -0800
URL:       http://hg.openjdk.java.net/mlvm/mlvm/hotspot/rev/89129696dbc4

meth-lazy: update

+ meth-lazy-7023639.bcg.patch
+ meth-lazy-7023639.jit.patch
+ meth-lazy-7023639.patch
- meth-lazy.patch
! meth-lazy.txt


From chen.xi at telecom-bretagne.eu  Tue Dec 13 17:24:16 2011
From: chen.xi at telecom-bretagne.eu (chen.xi at telecom-bretagne.eu)
Date: Wed, 14 Dec 2011 02:24:16 +0100
Subject: problem: fail to apply the patch [continue]
In-Reply-To: <E561015A-7021-4939-AD0C-144B3C76A71A@oracle.com>
References: <20111212181708.93972jjz2maj8o4k@webmail.telecom-bretagne.eu>
	<E561015A-7021-4939-AD0C-144B3C76A71A@oracle.com>
Message-ID: <20111214022416.10235ee1x809xa80@webmail.telecom-bretagne.eu>


The Openjdk I fclone is hsx/hotspot-comp. I just followed the
instruction on mlvm wiki.

I tried the strip command as you said. But it did not help any more.
E.g., the hotspot patch requires revision is "d8cb48376797". Its first
descendant is "cec1757a0134". So I do "hg strip cec1757a0134" in
order to remove all the changeset before "d8cb48376797". But this  
command failed. Output message is:
------------------
mq status file refers to unknown node 4c9ea0147722
mq status file refers to unknown node daf841a4cc84
abort: unknown revision 'qtip'!
------------------

As showed above, message like "unknown node XXXX" appears again.
What's more, how does Mercurial know the revision 'qtip'? I searched all
the tags and there is no qtip.

So let's simplify my problem. We just focus the error at the very
beginning. Let's only do qpop for all the source repo.
-------------------
xc1427 at xc1427-laptop:~/continuation_research/davinci.v1$ bash
patches/make/each-patch-repo.sh hg qpop -a
+ (cd sources/.; hg qpop -a)
no patches applied
+ (cd sources/hotspot; hg qpop -a)
abort: trying to pop unknown node daf841a4cc84
*** Exit status 255.
+ (cd sources/jdk; hg qpop -a)
abort: trying to pop unknown node f2029fe990b5
*** Exit status 255.
+ (cd sources/langtools; hg qpop -a)
abort: trying to pop unknown node 0001c23b2b4b
*** Exit status 255.
--------------------
Here is strange. Provided the source repo is clean, the expected
output should be *no patches applied*. So why does mq want to pop
*unknown node* ?

-- Chen

PS: why is the fclone command used to pull project as shown in the  
wiki, since the Forest Extension is not delivered with Mercurial any  
more ?

Christian Thalinger <christian.thalinger at oracle.com> a ?crit :

>
> On Dec 12, 2011, at 6:17 PM, chen.xi at telecom-bretagne.eu wrote:
>
>>
>> I review the output message. I come to know what the error has
come from.
>>
>> In fact, when I did "make setup", some patches which had already  
>> been in the source rep was not popped at all. relative output is:
>> ---------------------------------
>> cd ../..; bash patches/make/each-patch-repo.sh \
>>         "hg qpop -a; hg qselect buildable testable" \
>>             "$(uname -s | sed -n 's/Darwin/bsd-port/p')" \
>>             "\$(bash `pwd`/patches/make/current-release.sh)"
>> + (cd sources/.; hg qpop -a; hg qselect buildable testable  $(bash

>>
/home/xc1427/continuation_research/davinci.v1/patches/make/current-release.sh))
>> no patches applied
>> + (cd sources/hotspot; hg qpop -a; hg qselect buildable testable  $(bash
/home/xc1427/continuation_research/davinci.v1/patches/make/current-release.sh))
>> abort: trying to pop unknown node daf841a4cc84
>> + (cd sources/jdk; hg qpop -a; hg qselect buildable testable
$(bash
>>
/home/xc1427/continuation_research/davinci.v1/patches/make/current-release.sh))
>> abort: trying to pop unknown node f2029fe990b5
>> + (cd sources/langtools; hg qpop -a; hg qselect buildable testable

>> $(bash
/home/xc1427/continuation_research/davinci.v1/patches/make/current-release.sh))
>> abort: trying to pop unknown node 0001c23b2b4b
>> ---------------------------------
>>
>> Therefore, at the point that I updated the source revision to  
>> conform to patches, changes was made before all the patches has
been
>> popped.  So I got the error.
>>
>> I think the causality is clear. Can anybody tell me how to pop all

>> the patches ? (obviously qpop -a cannot do this....)
>
> What OpenJDK forest did you clone?  If qpop doesn't work you can  
> strip the changesets you don't want.
>
> -- Chris
>
>> _______________________________________________
>> mlvm-dev mailing list
>> mlvm-dev at openjdk.java.net
>> http://mail.openjdk.java.net/mailman/listinfo/mlvm-dev
>
>




From stephen.bannasch at deanbrook.org  Tue Dec 13 22:08:28 2011
From: stephen.bannasch at deanbrook.org (Stephen Bannasch)
Date: Wed, 14 Dec 2011 01:08:28 -0500
Subject: problem: fail to apply the patch [continue]
In-Reply-To: <20111214022416.10235ee1x809xa80@webmail.telecom-bretagne.eu>
References: <20111212181708.93972jjz2maj8o4k@webmail.telecom-bretagne.eu>	<E561015A-7021-4939-AD0C-144B3C76A71A@oracle.com>
	<20111214022416.10235ee1x809xa80@webmail.telecom-bretagne.eu>
Message-ID: <p06240826cb0ded846417@[192.168.1.122]>

At 2:24 AM +0100 12/14/11, chen.xi at telecom-bretagne.eu wrote:
>The Openjdk I fclone is hsx/hotspot-comp. I just followed the
>instruction on mlvm wiki.

mlvm is building fine for me and jtreg tests I run also work

my build scripts: http://gist.github.com/243072

From john.r.rose at oracle.com  Tue Dec 13 23:28:05 2011
From: john.r.rose at oracle.com (John Rose)
Date: Tue, 13 Dec 2011 23:28:05 -0800
Subject: problem: fail to apply the patch [continue]
In-Reply-To: <20111214022416.10235ee1x809xa80@webmail.telecom-bretagne.eu>
References: <20111212181708.93972jjz2maj8o4k@webmail.telecom-bretagne.eu>
	<E561015A-7021-4939-AD0C-144B3C76A71A@oracle.com>
	<20111214022416.10235ee1x809xa80@webmail.telecom-bretagne.eu>
Message-ID: <C3CBF83D-FFF8-4A9F-8C06-58844D9A603C@oracle.com>

On Dec 13, 2011, at 5:24 PM, chen.xi at telecom-bretagne.eu wrote:

> The Openjdk I fclone is hsx/hotspot-comp. I just followed the
> instruction on mlvm wiki.
> 
> I tried the strip command as you said. But it did not help any more.
> E.g., the hotspot patch requires revision is "d8cb48376797". Its first
> descendant is "cec1757a0134". So I do "hg strip cec1757a0134" in
> order to remove all the changeset before "d8cb48376797". But this  
> command failed. Output message is:
> ------------------
> mq status file refers to unknown node 4c9ea0147722
> mq status file refers to unknown node daf841a4cc84
> abort: unknown revision 'qtip'!
> ------------------
> 
> As showed above, message like "unknown node XXXX" appears again.
> What's more, how does Mercurial know the revision 'qtip'? I searched all
> the tags and there is no qtip.
> 
> So let's simplify my problem. We just focus the error at the very
> beginning. Let's only do qpop for all the source repo.

With problems like this, the quickest way out is to delete the whole repo and pull a fresh copy.

Note that mq uses the "strip" command internally to push and pop versions.  It uses the status file to keep track of where it is.  Therefore, if you do an "out of band" unexpected "strip", mq will no longer be able to track what's pushed.  That is the reason for some of the errors.  But it's not worth debugging.  Do an "hg diff" to save any uncommitted differences, move your dead repo. to a holding folder (or delete it if you feel lucky) and make a new one.

I hope this helps!

Also, I have rebased the 292-related patches, so there will be less chance of confusion with patch mismatches.  As people may have noticed, I do this about once a month, but missed November.

-- John

From christian.thalinger at oracle.com  Tue Dec 13 23:29:20 2011
From: christian.thalinger at oracle.com (Christian Thalinger)
Date: Wed, 14 Dec 2011 08:29:20 +0100
Subject: problem: fail to apply the patch [continue]
In-Reply-To: <20111214022416.10235ee1x809xa80@webmail.telecom-bretagne.eu>
References: <20111212181708.93972jjz2maj8o4k@webmail.telecom-bretagne.eu>
	<E561015A-7021-4939-AD0C-144B3C76A71A@oracle.com>
	<20111214022416.10235ee1x809xa80@webmail.telecom-bretagne.eu>
Message-ID: <6A77AD0F-B55C-4846-A16D-4D5B44C21DFB@oracle.com>


On Dec 14, 2011, at 2:24 AM, chen.xi at telecom-bretagne.eu wrote:

> 
> The Openjdk I fclone is hsx/hotspot-comp. I just followed the
> instruction on mlvm wiki.
> 
> I tried the strip command as you said. But it did not help any more.
> E.g., the hotspot patch requires revision is "d8cb48376797". Its first
> descendant is "cec1757a0134". So I do "hg strip cec1757a0134" in
> order to remove all the changeset before "d8cb48376797". But this  
> command failed. Output message is:
> ------------------
> mq status file refers to unknown node 4c9ea0147722
> mq status file refers to unknown node daf841a4cc84
> abort: unknown revision 'qtip'!
> ------------------
> 
> As showed above, message like "unknown node XXXX" appears again.
> What's more, how does Mercurial know the revision 'qtip'? I searched all
> the tags and there is no qtip.
> 
> So let's simplify my problem. We just focus the error at the very
> beginning. Let's only do qpop for all the source repo.
> -------------------
> xc1427 at xc1427-laptop:~/continuation_research/davinci.v1$ bash
> patches/make/each-patch-repo.sh hg qpop -a
> + (cd sources/.; hg qpop -a)
> no patches applied
> + (cd sources/hotspot; hg qpop -a)
> abort: trying to pop unknown node daf841a4cc84
> *** Exit status 255.
> + (cd sources/jdk; hg qpop -a)
> abort: trying to pop unknown node f2029fe990b5
> *** Exit status 255.
> + (cd sources/langtools; hg qpop -a)
> abort: trying to pop unknown node 0001c23b2b4b
> *** Exit status 255.
> --------------------
> Here is strange. Provided the source repo is clean, the expected
> output should be *no patches applied*. So why does mq want to pop
> *unknown node* ?

I have to guess here but I suspect that your status files are not clean (i.e. .hg/patches/status in hotspot, jdk, and langtools).  I think you can simply remove them and then try again.

-- Chris

> 
> -- Chen
> 
> PS: why is the fclone command used to pull project as shown in the  
> wiki, since the Forest Extension is not delivered with Mercurial any  
> more ?
> 
> Christian Thalinger <christian.thalinger at oracle.com> a ?crit :
> 
>> 
>> On Dec 12, 2011, at 6:17 PM, chen.xi at telecom-bretagne.eu wrote:
>> 
>>> 
>>> I review the output message. I come to know what the error has
> come from.
>>> 
>>> In fact, when I did "make setup", some patches which had already  
>>> been in the source rep was not popped at all. relative output is:
>>> ---------------------------------
>>> cd ../..; bash patches/make/each-patch-repo.sh \
>>>        "hg qpop -a; hg qselect buildable testable" \
>>>            "$(uname -s | sed -n 's/Darwin/bsd-port/p')" \
>>>            "\$(bash `pwd`/patches/make/current-release.sh)"
>>> + (cd sources/.; hg qpop -a; hg qselect buildable testable  $(bash
> 
>>> 
> /home/xc1427/continuation_research/davinci.v1/patches/make/current-release.sh))
>>> no patches applied
>>> + (cd sources/hotspot; hg qpop -a; hg qselect buildable testable  $(bash
> /home/xc1427/continuation_research/davinci.v1/patches/make/current-release.sh))
>>> abort: trying to pop unknown node daf841a4cc84
>>> + (cd sources/jdk; hg qpop -a; hg qselect buildable testable
> $(bash
>>> 
> /home/xc1427/continuation_research/davinci.v1/patches/make/current-release.sh))
>>> abort: trying to pop unknown node f2029fe990b5
>>> + (cd sources/langtools; hg qpop -a; hg qselect buildable testable
> 
>>> $(bash
> /home/xc1427/continuation_research/davinci.v1/patches/make/current-release.sh))
>>> abort: trying to pop unknown node 0001c23b2b4b
>>> ---------------------------------
>>> 
>>> Therefore, at the point that I updated the source revision to  
>>> conform to patches, changes was made before all the patches has
> been
>>> popped.  So I got the error.
>>> 
>>> I think the causality is clear. Can anybody tell me how to pop all
> 
>>> the patches ? (obviously qpop -a cannot do this....)
>> 
>> What OpenJDK forest did you clone?  If qpop doesn't work you can  
>> strip the changesets you don't want.
>> 
>> -- Chris
>> 
>>> _______________________________________________
>>> mlvm-dev mailing list
>>> mlvm-dev at openjdk.java.net
>>> http://mail.openjdk.java.net/mailman/listinfo/mlvm-dev
>> 
>> 
> 
> 
> 
> _______________________________________________
> mlvm-dev mailing list
> mlvm-dev at openjdk.java.net
> http://mail.openjdk.java.net/mailman/listinfo/mlvm-dev


From christian.thalinger at oracle.com  Wed Dec 14 02:30:56 2011
From: christian.thalinger at oracle.com (christian.thalinger at oracle.com)
Date: Wed, 14 Dec 2011 10:30:56 +0000
Subject: hg: mlvm/mlvm/hotspot: Fixed patch names
Message-ID: <20111214103057.A0899476A9@hg.openjdk.java.net>

Changeset: cddb66faf23f
Author:    twisti
Date:      2011-12-14 11:29 +0100
URL:       http://hg.openjdk.java.net/mlvm/mlvm/hotspot/rev/cddb66faf23f

Fixed patch names

! series


From chen.xi at telecom-bretagne.eu  Wed Dec 14 13:48:25 2011
From: chen.xi at telecom-bretagne.eu (chen.xi at telecom-bretagne.eu)
Date: Wed, 14 Dec 2011 22:48:25 +0100
Subject: problem: fail to apply the patch [continue]
In-Reply-To: <C3CBF83D-FFF8-4A9F-8C06-58844D9A603C@oracle.com>
References: <20111212181708.93972jjz2maj8o4k@webmail.telecom-bretagne.eu>
	<E561015A-7021-4939-AD0C-144B3C76A71A@oracle.com>
	<20111214022416.10235ee1x809xa80@webmail.telecom-bretagne.eu>
	<C3CBF83D-FFF8-4A9F-8C06-58844D9A603C@oracle.com>
Message-ID: <20111214224825.11007hvw9j29hcco@webmail.telecom-bretagne.eu>

Yeah, I solved the problem. As some of you have mentioned, the status  
file is not clean, because I occasionally did some "out-of-band" strip  
and confused the mq.

Thanks a lot.

-- Chen

John Rose <john.r.rose at oracle.com> a ?crit?:

> On Dec 13, 2011, at 5:24 PM, chen.xi at telecom-bretagne.eu wrote:
>
>> The Openjdk I fclone is hsx/hotspot-comp. I just followed the
>> instruction on mlvm wiki.
>>
>> I tried the strip command as you said. But it did not help any more.
>> E.g., the hotspot patch requires revision is "d8cb48376797". Its first
>> descendant is "cec1757a0134". So I do "hg strip cec1757a0134" in
>> order to remove all the changeset before "d8cb48376797". But this
>> command failed. Output message is:
>> ------------------
>> mq status file refers to unknown node 4c9ea0147722
>> mq status file refers to unknown node daf841a4cc84
>> abort: unknown revision 'qtip'!
>> ------------------
>>
>> As showed above, message like "unknown node XXXX" appears again.
>> What's more, how does Mercurial know the revision 'qtip'? I searched all
>> the tags and there is no qtip.
>>
>> So let's simplify my problem. We just focus the error at the very
>> beginning. Let's only do qpop for all the source repo.
>
> With problems like this, the quickest way out is to delete the whole  
> repo and pull a fresh copy.
>
> Note that mq uses the "strip" command internally to push and pop  
> versions.  It uses the status file to keep track of where it is.   
> Therefore, if you do an "out of band" unexpected "strip", mq will no  
> longer be able to track what's pushed.  That is the reason for some  
> of the errors.  But it's not worth debugging.  Do an "hg diff" to  
> save any uncommitted differences, move your dead repo. to a holding  
> folder (or delete it if you feel lucky) and make a new one.
>
> I hope this helps!
>
> Also, I have rebased the 292-related patches, so there will be less  
> chance of confusion with patch mismatches.  As people may have  
> noticed, I do this about once a month, but missed November.
>
> -- John
> _______________________________________________
> mlvm-dev mailing list
> mlvm-dev at openjdk.java.net
> http://mail.openjdk.java.net/mailman/listinfo/mlvm-dev
>
>




From forax at univ-mlv.fr  Sat Dec 17 10:33:48 2011
From: forax at univ-mlv.fr (=?ISO-8859-1?Q?R=E9mi_Forax?=)
Date: Sat, 17 Dec 2011 19:33:48 +0100
Subject: JSR 292 Goodness: Almost static final field
Message-ID: <4EECE08C.3020203@univ-mlv.fr>

I've written a blog entry about how to write a static volatile final 
variable, i.e. a variable which is considered as constant
by the compiler (the JIT) and if the variable changes, the JITed code is 
deoptimized and
jump back in interpreter mode to eventually optimize it again later.

http://weblogs.java.net/blog/forax/archive/2011/12/17/jsr-292-goodness-almost-static-final-field

The code is a little bit more convoluted than it should because there is 
no invokedynamic
in Java but you can emulate it.

R?mi


From christian.thalinger at oracle.com  Fri Dec 23 05:26:01 2011
From: christian.thalinger at oracle.com (christian.thalinger at oracle.com)
Date: Fri, 23 Dec 2011 13:26:01 +0000
Subject: hg: mlvm/mlvm/jdk: Use LFs: 17 MethodHandlesTests working
Message-ID: <20111223132602.9FC5A477C8@hg.openjdk.java.net>

Changeset: db9868860bdd
Author:    twisti
Date:      2011-12-23 14:24 +0100
URL:       http://hg.openjdk.java.net/mlvm/mlvm/jdk/rev/db9868860bdd

Use LFs: 17 MethodHandlesTests working

! meth-lazy-7023639.bcg.patch


From christian.thalinger at oracle.com  Fri Dec 30 06:26:03 2011
From: christian.thalinger at oracle.com (christian.thalinger at oracle.com)
Date: Fri, 30 Dec 2011 14:26:03 +0000
Subject: hg: mlvm/mlvm/jdk: LF'ified makeAllocator;
	18 MethodHandlesTests working
Message-ID: <20111230142603.C86E947851@hg.openjdk.java.net>

Changeset: 700540b4b52b
Author:    twisti
Date:      2011-12-30 15:25 +0100
URL:       http://hg.openjdk.java.net/mlvm/mlvm/jdk/rev/700540b4b52b

LF'ified makeAllocator; 18 MethodHandlesTests working

! meth-lazy-7023639.bcg.patch


From forax at univ-mlv.fr  Fri Dec 30 06:45:03 2011
From: forax at univ-mlv.fr (=?ISO-8859-1?Q?R=E9mi_Forax?=)
Date: Fri, 30 Dec 2011 15:45:03 +0100
Subject: hg: mlvm/mlvm/jdk: LF'ified makeAllocator; 18 MethodHandlesTests
	working
In-Reply-To: <20111230142603.C86E947851@hg.openjdk.java.net>
References: <20111230142603.C86E947851@hg.openjdk.java.net>
Message-ID: <4EFDCE6F.4090705@univ-mlv.fr>

On 12/30/2011 03:26 PM, christian.thalinger at oracle.com wrote:
> Changeset: 700540b4b52b
> Author:    twisti
> Date:      2011-12-30 15:25 +0100
> URL:       http://hg.openjdk.java.net/mlvm/mlvm/jdk/rev/700540b4b52b
>
> LF'ified makeAllocator; 18 MethodHandlesTests working
>
> ! meth-lazy-7023639.bcg.patch

Hi Christian,
I believe you should create the UNSAFE_ALLOCATEINSTANCE with
lookup.bind() instead of doing a findVirtual + a bindReceiver
to avoid to bind the receiver for each call of findConstructor.

BTW, nice to see that collect is not used anymore during the allocation.

cheers,
R?mi



From christian.thalinger at oracle.com  Fri Dec 30 07:25:43 2011
From: christian.thalinger at oracle.com (Christian Thalinger)
Date: Fri, 30 Dec 2011 16:25:43 +0100
Subject: hg: mlvm/mlvm/jdk: LF'ified makeAllocator;
	18 MethodHandlesTests working
In-Reply-To: <4EFDCE6F.4090705@univ-mlv.fr>
References: <20111230142603.C86E947851@hg.openjdk.java.net>
	<4EFDCE6F.4090705@univ-mlv.fr>
Message-ID: <ED227F00-221E-4EE4-AE95-764F6256DD14@oracle.com>


On Dec 30, 2011, at 3:45 PM, R?mi Forax wrote:

> On 12/30/2011 03:26 PM, christian.thalinger at oracle.com wrote:
>> Changeset: 700540b4b52b
>> Author:    twisti
>> Date:      2011-12-30 15:25 +0100
>> URL:       http://hg.openjdk.java.net/mlvm/mlvm/jdk/rev/700540b4b52b
>> 
>> LF'ified makeAllocator; 18 MethodHandlesTests working
>> 
>> ! meth-lazy-7023639.bcg.patch
> 
> Hi Christian,
> I believe you should create the UNSAFE_ALLOCATEINSTANCE with
> lookup.bind() instead of doing a findVirtual + a bindReceiver
> to avoid to bind the receiver for each call of findConstructor.

Good point.  The reason why I'm not using it now but rather use something like this:

            lzmh = (LazyMethodHandle) lzmh.bindReceiver(UNSAFE);
            lzmh = (LazyMethodHandle) lzmh.bindArgument(0, allocateClass);

is because MH.bindReceiver still uses BoundMethodHandle and I want to avoid BMHs since I want to get rid of them.  But as soon as DMHs have LazyMethodHandle functionality I will change that code.

-- Chris

> 
> BTW, nice to see that collect is not used anymore during the allocation.
> 
> cheers,
> R?mi
> 
> 
> _______________________________________________
> mlvm-dev mailing list
> mlvm-dev at openjdk.java.net
> http://mail.openjdk.java.net/mailman/listinfo/mlvm-dev