From lukas.stadler at oracle.com  Mon Aug  1 08:49:53 2016
From: lukas.stadler at oracle.com (Lukas Stadler)
Date: Mon, 1 Aug 2016 10:49:53 +0200
Subject: Nodes removal from the IR
In-Reply-To: <1469786420.6563.13.camel@ed.ac.uk>
References: <1469786420.6563.13.camel@ed.ac.uk>
Message-ID: <55E55373-A2DF-4DB1-A65F-BD870958BBB5@oracle.com>

Hi Juan,

how do you remove the LoadFieldNode?
replaceAtUsages(null) is a problem in this case, because this node is supposed to generate a value.
Do you replace it with a constant?

- Lukas

> On 29 Jul 2016, at 12:00, Juan Fumero <juan.fumero at ed.ac.uk> wrote:
> 
> Hi all, 
>   I am trying to remove some nodes from the IR that I do not need for
> the GPU execution. 
> 
> For instance, to remove some of the FixedGuard I do the following:
> 
> <code>
> if (node instanceof FixedGuardNode) {
>    node.replaceAtUsages(null);
>    graph.removeFixed((FixedWithNextNode));
> }
>         
> new CanonicalizerPhase().apply(graph, new PhaseContext(providers));
> new DeadCodeEliminationPhase().apply(graph);
> </code>
> 
> This works fine. However, when I try to remove other kind of nodes,
> like UnsafeLoadNode, I get the following error:
> 
> [thread:1] scope: main
>   [thread:1] scope: main.CleanPhase
>     [thread:1] scope: main.CleanPhase.InterceptException
>     Exception occurred in scope: main.CleanPhase.InterceptException
>     Context obj com.oracle.graal.graph.VerificationError: cannot delete
> node 74|UnsafeLoad because of predecessor: 72|LoadField#locals
>     	at node: 74|UnsafeLoad
>     Context obj com.oracle.truffle.r.library.gpu.phases.CleanPhase at 32dd
> cca
> internal error: com.oracle.truffle.r.runtime.RInternalError:
> com.oracle.graal.graph.VerificationError: cannot delete node
> 74|UnsafeLoad because of predecessor: 72|LoadField#locals
> 	at node: 74|UnsafeLoad (see fastr_errors.log)
> 
> 
> Because there is a dependency with LoadFieldNode. My question is, how
> could I get rid of these kind of nodes? I am trying also to remove its
> dependencies but I get the same error. I also tried to removed with
> graph.removeFixed(node) but I get the error as well. 
> 
> 
> Many thanks
> Juan
> 
> 
> -- 
> The University of Edinburgh is a charitable body, registered in
> Scotland, with registration number SC005336.
> 


From juan.fumero at ed.ac.uk  Mon Aug  1 09:28:53 2016
From: juan.fumero at ed.ac.uk (Juan Fumero)
Date: Mon, 01 Aug 2016 10:28:53 +0100
Subject: Nodes removal from the IR
In-Reply-To: <55E55373-A2DF-4DB1-A65F-BD870958BBB5@oracle.com>
References: <1469786420.6563.13.camel@ed.ac.uk>
	<55E55373-A2DF-4DB1-A65F-BD870958BBB5@oracle.com>
Message-ID: <1470043733.9657.8.camel@ed.ac.uk>

Hi Lukas,?

? I didn't actually replace LoadFieldNode. ?I was thinking I could
apply something similar to FixedGuard and replace it with null and then
remove it.?

So for these kind of nodes I should replace them with another node?
Could I replace it either with a?FloatingNode or FixedNode? ?

Thanks
Juan

On Mon, 2016-08-01 at 10:49 +0200, Lukas Stadler wrote:
> Hi Juan,
> 
> how do you remove the LoadFieldNode?
> replaceAtUsages(null) is a problem in this case, because this node is
> supposed to generate a value.
> Do you replace it with a constant?
> 
> - Lukas
> 
> > 
> > On 29 Jul 2016, at 12:00, Juan Fumero <juan.fumero at ed.ac.uk> wrote:
> > 
> > Hi all,?
> > ? I am trying to remove some nodes from the IR that I do not need
> > for
> > the GPU execution.?
> > 
> > For instance, to remove some of the FixedGuard I do the following:
> > 
> > <code>
> > if (node instanceof FixedGuardNode) {
> > ???node.replaceAtUsages(null);
> > ???graph.removeFixed((FixedWithNextNode));
> > }
> > ????????
> > new CanonicalizerPhase().apply(graph, new PhaseContext(providers));
> > new DeadCodeEliminationPhase().apply(graph);
> > </code>
> > 
> > This works fine. However, when I try to remove other kind of nodes,
> > like UnsafeLoadNode, I get the following error:
> > 
> > [thread:1] scope: main
> > ? [thread:1] scope: main.CleanPhase
> > ????[thread:1] scope: main.CleanPhase.InterceptException
> > ????Exception occurred in scope: main.CleanPhase.InterceptException
> > ????Context obj com.oracle.graal.graph.VerificationError: cannot
> > delete
> > node 74|UnsafeLoad because of predecessor: 72|LoadField#locals
> > ????	at node: 74|UnsafeLoad
> > ????Context obj com.oracle.truffle.r.library.gpu.phases.CleanPhase@
> > 32dd
> > cca
> > internal error: com.oracle.truffle.r.runtime.RInternalError:
> > com.oracle.graal.graph.VerificationError: cannot delete node
> > 74|UnsafeLoad because of predecessor: 72|LoadField#locals
> > 	at node: 74|UnsafeLoad (see fastr_errors.log)
> > 
> > 
> > Because there is a dependency with LoadFieldNode. My question is,
> > how
> > could I get rid of these kind of nodes? I am trying also to remove
> > its
> > dependencies but I get the same error. I also tried to removed with
> > graph.removeFixed(node) but I get the error as well.?
> > 
> > 
> > Many thanks
> > Juan
> > 
> > 

-- 
The University of Edinburgh is a charitable body, registered in
Scotland, with registration number SC005336.


From lukas.stadler at oracle.com  Mon Aug  1 09:43:18 2016
From: lukas.stadler at oracle.com (Lukas Stadler)
Date: Mon, 1 Aug 2016 11:43:18 +0200
Subject: Nodes removal from the IR
In-Reply-To: <1470043733.9657.8.camel@ed.ac.uk>
References: <1469786420.6563.13.camel@ed.ac.uk>
	<55E55373-A2DF-4DB1-A65F-BD870958BBB5@oracle.com>
	<1470043733.9657.8.camel@ed.ac.uk>
Message-ID: <BF23675C-7A9D-4E8A-892E-53F585417EA0@oracle.com>

It depends on what the semantics of your change should be - what does it mean that you remove the load? is it not necessary any more, because you already know the value?
If you remove a loadfield, and the loadfield has usages, then these usages (e.g., arithmetic operations) need some value to work with.

The FixedGuardNode is special, in that its usages are guard dependencies, and usually can work with a ?null? input (i.e., no node connected, not a ?null? constant).
FloatingGuardedNode, for example, uses an @OptionalInput, while the BinaryNodes use the non-optional @Input.

You can replaceAtUsages with any other node, fixed or floating, but its result needs to be available at the point of the loadfield.
You may get errors from the Scheduler if it isn?t.

- Lukas

> On 01 Aug 2016, at 11:28, Juan Fumero <juan.fumero at ed.ac.uk> wrote:
> 
> Hi Lukas, 
> 
>   I didn't actually replace LoadFieldNode.  I was thinking I could
> apply something similar to FixedGuard and replace it with null and then
> remove it. 
> 
> So for these kind of nodes I should replace them with another node?
> Could I replace it either with a FloatingNode or FixedNode?  
> 
> Thanks
> Juan
> 
> On Mon, 2016-08-01 at 10:49 +0200, Lukas Stadler wrote:
>> Hi Juan,
>> 
>> how do you remove the LoadFieldNode?
>> replaceAtUsages(null) is a problem in this case, because this node is
>> supposed to generate a value.
>> Do you replace it with a constant?
>> 
>> - Lukas
>> 
>>> 
>>> On 29 Jul 2016, at 12:00, Juan Fumero <juan.fumero at ed.ac.uk> wrote:
>>> 
>>> Hi all, 
>>>   I am trying to remove some nodes from the IR that I do not need
>>> for
>>> the GPU execution. 
>>> 
>>> For instance, to remove some of the FixedGuard I do the following:
>>> 
>>> <code>
>>> if (node instanceof FixedGuardNode) {
>>>    node.replaceAtUsages(null);
>>>    graph.removeFixed((FixedWithNextNode));
>>> }
>>>         
>>> new CanonicalizerPhase().apply(graph, new PhaseContext(providers));
>>> new DeadCodeEliminationPhase().apply(graph);
>>> </code>
>>> 
>>> This works fine. However, when I try to remove other kind of nodes,
>>> like UnsafeLoadNode, I get the following error:
>>> 
>>> [thread:1] scope: main
>>>   [thread:1] scope: main.CleanPhase
>>>     [thread:1] scope: main.CleanPhase.InterceptException
>>>     Exception occurred in scope: main.CleanPhase.InterceptException
>>>     Context obj com.oracle.graal.graph.VerificationError: cannot
>>> delete
>>> node 74|UnsafeLoad because of predecessor: 72|LoadField#locals
>>>     	at node: 74|UnsafeLoad
>>>     Context obj com.oracle.truffle.r.library.gpu.phases.CleanPhase@
>>> 32dd
>>> cca
>>> internal error: com.oracle.truffle.r.runtime.RInternalError:
>>> com.oracle.graal.graph.VerificationError: cannot delete node
>>> 74|UnsafeLoad because of predecessor: 72|LoadField#locals
>>> 	at node: 74|UnsafeLoad (see fastr_errors.log)
>>> 
>>> 
>>> Because there is a dependency with LoadFieldNode. My question is,
>>> how
>>> could I get rid of these kind of nodes? I am trying also to remove
>>> its
>>> dependencies but I get the same error. I also tried to removed with
>>> graph.removeFixed(node) but I get the error as well. 
>>> 
>>> 
>>> Many thanks
>>> Juan
>>> 
>>> 
> 
> -- 
> The University of Edinburgh is a charitable body, registered in
> Scotland, with registration number SC005336.
> 


From juan.fumero at ed.ac.uk  Mon Aug  1 10:02:34 2016
From: juan.fumero at ed.ac.uk (Juan Fumero)
Date: Mon, 01 Aug 2016 11:02:34 +0100
Subject: Nodes removal from the IR
In-Reply-To: <BF23675C-7A9D-4E8A-892E-53F585417EA0@oracle.com>
References: <1469786420.6563.13.camel@ed.ac.uk>
	<55E55373-A2DF-4DB1-A65F-BD870958BBB5@oracle.com>
	<1470043733.9657.8.camel@ed.ac.uk>
	<BF23675C-7A9D-4E8A-892E-53F585417EA0@oracle.com>
Message-ID: <1470045754.9657.16.camel@ed.ac.uk>

Hi,?

On Mon, 2016-08-01 at 11:43 +0200, Lukas Stadler wrote:
> It depends on what the semantics of your change should be - what does
> it mean that you remove the load? is it not necessary any more,
> because you already know the value?
> If you remove a loadfield, and the loadfield has usages, then these
> usages (e.g., arithmetic operations) need some value to work with.

Yes, that's exactly my case, I know the values for some of the
LoadField nodes (related to scope variables from R) so I do not
actually need to process some of the LoadFields when I generate code
for OpenCL.?

> 
> The FixedGuardNode is special, in that its usages are guard
> dependencies, and usually can work with a ?null? input (i.e., no node
> connected, not a ?null? constant).
> FloatingGuardedNode, for example, uses an @OptionalInput, while the
> BinaryNodes use the non-optional @Input.
> 
> You can replaceAtUsages with any other node, fixed or floating, but
> its result needs to be available at the point of the loadfield.
> You may get errors from the Scheduler if it isn?t.

Ok, thanks Lukas. This is very useful.?

Cheers
Juan

> 
> - Lukas
> 
> > 
> > On 01 Aug 2016, at 11:28, Juan Fumero <juan.fumero at ed.ac.uk> wrote:
> > 
> > Hi Lukas,?
> > 
> > ? I didn't actually replace LoadFieldNode.??I was thinking I could
> > apply something similar to FixedGuard and replace it with null and
> > then
> > remove it.?
> > 
> > So for these kind of nodes I should replace them with another node?
> > Could I replace it either with a FloatingNode or FixedNode???
> > 
> > Thanks
> > Juan
> > 
> > On Mon, 2016-08-01 at 10:49 +0200, Lukas Stadler wrote:
> > > 
> > > Hi Juan,
> > > 
> > > how do you remove the LoadFieldNode?
> > > replaceAtUsages(null) is a problem in this case, because this
> > > node is
> > > supposed to generate a value.
> > > Do you replace it with a constant?
> > > 
> > > - Lukas
> > > 
> > > > 
> > > > 
> > > > On 29 Jul 2016, at 12:00, Juan Fumero <juan.fumero at ed.ac.uk>
> > > > wrote:
> > > > 
> > > > Hi all,?
> > > > ? I am trying to remove some nodes from the IR that I do not
> > > > need
> > > > for
> > > > the GPU execution.?
> > > > 
> > > > For instance, to remove some of the FixedGuard I do the
> > > > following:
> > > > 
> > > > <code>
> > > > if (node instanceof FixedGuardNode) {
> > > > ???node.replaceAtUsages(null);
> > > > ???graph.removeFixed((FixedWithNextNode));
> > > > }
> > > > ????????
> > > > new CanonicalizerPhase().apply(graph, new
> > > > PhaseContext(providers));
> > > > new DeadCodeEliminationPhase().apply(graph);
> > > > </code>
> > > > 
> > > > This works fine. However, when I try to remove other kind of
> > > > nodes,
> > > > like UnsafeLoadNode, I get the following error:
> > > > 
> > > > [thread:1] scope: main
> > > > ? [thread:1] scope: main.CleanPhase
> > > > ????[thread:1] scope: main.CleanPhase.InterceptException
> > > > ????Exception occurred in scope:
> > > > main.CleanPhase.InterceptException
> > > > ????Context obj com.oracle.graal.graph.VerificationError:
> > > > cannot
> > > > delete
> > > > node 74|UnsafeLoad because of predecessor: 72|LoadField#locals
> > > > ????	at node: 74|UnsafeLoad
> > > > ????Context obj
> > > > com.oracle.truffle.r.library.gpu.phases.CleanPhase@
> > > > 32dd
> > > > cca
> > > > internal error: com.oracle.truffle.r.runtime.RInternalError:
> > > > com.oracle.graal.graph.VerificationError: cannot delete node
> > > > 74|UnsafeLoad because of predecessor: 72|LoadField#locals
> > > > 	at node: 74|UnsafeLoad (see fastr_errors.log)
> > > > 
> > > > 
> > > > Because there is a dependency with LoadFieldNode. My question
> > > > is,
> > > > how
> > > > could I get rid of these kind of nodes? I am trying also to
> > > > remove
> > > > its
> > > > dependencies but I get the same error. I also tried to removed
> > > > with
> > > > graph.removeFixed(node) but I get the error as well.?
> > > > 
> > > > 
> > > > Many thanks
> > > > Juan
> > > > 
> > > > 

-- 
The University of Edinburgh is a charitable body, registered in
Scotland, with registration number SC005336.


From christian.wimmer at oracle.com  Wed Aug  3 19:40:23 2016
From: christian.wimmer at oracle.com (Christian Wimmer)
Date: Wed, 3 Aug 2016 12:40:23 -0700
Subject: How can I call sulong from simplelanguage?
In-Reply-To: <CAHZjNtY=Xoeg=BVJS8ZPG8LbuN1LnbBFA6NY4aRvhJ4UrNo4Tg@mail.gmail.com>
References: <CAHZjNtY=Xoeg=BVJS8ZPG8LbuN1LnbBFA6NY4aRvhJ4UrNo4Tg@mail.gmail.com>
Message-ID: <ba186cef-5259-ca09-bd01-f20b06212836@oracle.com>

I can comment on the SimpleLanguage part of the question: 
SimpleLanguage does not provide convenient built-in functions right now 
to load other language code (it does not even allow to load addiional 
SimpleLanguage source files). That's why the example of calling Ruby 
from SimpleLanguage looks syntactically bad.

To make your use case work, you need to define new SimpleLanguage 
builtin that take a file name and loads it.

-Christian


On 07/30/2016 04:38 AM, Sidharth Kshatriya wrote:
> I want to be able to use the polyglot feature of the Truffle API to be able
> to call sulong from, say, simple language.
>
> How would I go about doing this?
>
> I have additionally filed a ticket on the sulong project at github
> https://github.com/graalvm/sulong/issues/350
>
> I was not sure about the best way to approach this question -- whether to
> file a ticket or write an email (as the answer will be useful to other
> users of truffle/graal) so please forgive the duplication!
>
> Thanks,
>
> Sidharth
>

From gerard at gerardkrol.nl  Thu Aug  4 19:46:17 2016
From: gerard at gerardkrol.nl (Gerard Krol)
Date: Thu, 4 Aug 2016 21:46:17 +0200
Subject: Truffle on-stack-replacement sometimes not working well with
	background compilation?
Message-ID: <CAFzGNhKPQyavqPq-evLBMLqNysH=P6CDoX0bRuCfWgyQp22gww@mail.gmail.com>

Hi,

I've had some performance problems with Cover[1] and a specific benchmark.
Yesterday I finally figured out what was going on. The problem seems to be
that on-stack-replacement doesn't always work well with background
compilation.

The benchmark (fannkuch, [2])  did take 311 seconds to run, while the GCC
version would finish in 25 seconds. Warming up (running the benchmark twice
and timing the second run) would reduce the runtime to 47 seconds. I like
to run my benchmarks without warmup though, as that is a good worst-case
scenario.

I then figured out that just adding
-Dgraal.TruffleBackgroundCompilation=false made a huge difference. The time
needed was reduced to 52 seconds.

Is this a bug? Expected behavior in some cases? Am I doing something wrong?

Reproduction:
1. Get Cover from [1] (don't forget the 5th installation step)
2. Run time ./cover tests/fannkuch-redux_gcc5.cover
3. Run time ./cover -benchmark tests/fannkuch-redux_gcc5.cover (this will
disable the background compilation)

Regards,

Gerard

[1] https://github.com/gerard-/cover
[2]
https://github.com/gerard-/cover/blob/master/tests/fannkuch-redux_gcc5.cover

From christian.humer at gmail.com  Thu Aug  4 19:56:07 2016
From: christian.humer at gmail.com (Christian Humer)
Date: Thu, 04 Aug 2016 19:56:07 +0000
Subject: Truffle on-stack-replacement sometimes not working well with
	background compilation?
In-Reply-To: <CAFzGNhKPQyavqPq-evLBMLqNysH=P6CDoX0bRuCfWgyQp22gww@mail.gmail.com>
References: <CAFzGNhKPQyavqPq-evLBMLqNysH=P6CDoX0bRuCfWgyQp22gww@mail.gmail.com>
Message-ID: <em52c27866-a46f-45d4-b728-2ced7c386393@grashalm-work>

Hi Gerard,

Thanks for the report. Unfortunately I am unable to have a look at it 
right now, but I will asap.
Can you, in the meantime, provide me with some info on which version of 
graalvm/truffle you are using so I can try to reproduce?

Thanks,
Christian

On 04.08.2016 21:46:17, "Gerard Krol" <gerard at gerardkrol.nl> wrote:

>Hi,
>
>I've had some performance problems with Cover[1] and a specific 
>benchmark.
>Yesterday I finally figured out what was going on. The problem seems to 
>be
>that on-stack-replacement doesn't always work well with background
>compilation.
>
>The benchmark (fannkuch, [2])  did take 311 seconds to run, while the 
>GCC
>version would finish in 25 seconds. Warming up (running the benchmark 
>twice
>and timing the second run) would reduce the runtime to 47 seconds. I 
>like
>to run my benchmarks without warmup though, as that is a good 
>worst-case
>scenario.
>
>I then figured out that just adding
>-Dgraal.TruffleBackgroundCompilation=false made a huge difference. The 
>time
>needed was reduced to 52 seconds.
>
>Is this a bug? Expected behavior in some cases? Am I doing something 
>wrong?
>
>Reproduction:
>1. Get Cover from [1] (don't forget the 5th installation step)
>2. Run time ./cover tests/fannkuch-redux_gcc5.cover
>3. Run time ./cover -benchmark tests/fannkuch-redux_gcc5.cover (this 
>will
>disable the background compilation)
>
>Regards,
>
>Gerard
>
>[1] https://github.com/gerard-/cover
>[2]
>https://github.com/gerard-/cover/blob/master/tests/fannkuch-redux_gcc5.cover


From gerard at gerardkrol.nl  Thu Aug  4 20:13:27 2016
From: gerard at gerardkrol.nl (Gerard Krol)
Date: Thu, 4 Aug 2016 22:13:27 +0200
Subject: Truffle on-stack-replacement sometimes not working well with
	background compilation?
In-Reply-To: <em52c27866-a46f-45d4-b728-2ced7c386393@grashalm-work>
References: <CAFzGNhKPQyavqPq-evLBMLqNysH=P6CDoX0bRuCfWgyQp22gww@mail.gmail.com>
	<em52c27866-a46f-45d4-b728-2ced7c386393@grashalm-work>
Message-ID: <CAFzGNhJwyUW+OCvRBsHov39x4voLy-0PEtupPacYttfzCA8aCg@mail.gmail.com>

Hi Christian,

I'm using graalvm version 0.12, with truffle-api 0.15. I'm perfectly happy
with background compilation disabled at the moment, so take your time.

Regards,

Gerard

On Thu, Aug 4, 2016 at 9:56 PM, Christian Humer <christian.humer at gmail.com>
wrote:

> Hi Gerard,
>
> Thanks for the report. Unfortunately I am unable to have a look at it
> right now, but I will asap.
> Can you, in the meantime, provide me with some info on which version of
> graalvm/truffle you are using so I can try to reproduce?
>
> Thanks,
> Christian
>
> On 04.08.2016 21:46:17, "Gerard Krol" <gerard at gerardkrol.nl> wrote:
>
> Hi,
>>
>> I've had some performance problems with Cover[1] and a specific benchmark.
>> Yesterday I finally figured out what was going on. The problem seems to be
>> that on-stack-replacement doesn't always work well with background
>> compilation.
>>
>> The benchmark (fannkuch, [2])  did take 311 seconds to run, while the GCC
>> version would finish in 25 seconds. Warming up (running the benchmark
>> twice
>> and timing the second run) would reduce the runtime to 47 seconds. I like
>> to run my benchmarks without warmup though, as that is a good worst-case
>> scenario.
>>
>> I then figured out that just adding
>> -Dgraal.TruffleBackgroundCompilation=false made a huge difference. The
>> time
>> needed was reduced to 52 seconds.
>>
>> Is this a bug? Expected behavior in some cases? Am I doing something
>> wrong?
>>
>> Reproduction:
>> 1. Get Cover from [1] (don't forget the 5th installation step)
>> 2. Run time ./cover tests/fannkuch-redux_gcc5.cover
>> 3. Run time ./cover -benchmark tests/fannkuch-redux_gcc5.cover (this will
>> disable the background compilation)
>>
>> Regards,
>>
>> Gerard
>>
>> [1] https://github.com/gerard-/cover
>> [2]
>> https://github.com/gerard-/cover/blob/master/tests/fannkuch-
>> redux_gcc5.cover
>>
>
>

From manuel.rigger at chello.at  Thu Aug  4 21:10:36 2016
From: manuel.rigger at chello.at (Manuel Rigger)
Date: Thu, 4 Aug 2016 23:10:36 +0200
Subject: Open Source C/C++ Truffle languages available?
In-Reply-To: <EB126D4A-44E9-4E9D-A442-7DDD5977D337@oracle.com>
References: <CAFzGNhKuf88CAhxAqAn4Xk_5bW-5mXKXCxO0cY_kDtyApCJ2ew@mail.gmail.com>
	<4FEFE529-C757-4AD3-885C-2B8B820C9FBB@oracle.com>
	<CAFzGNhLYhLDyk0nmoW1xzReos95gOe-R1QDLLnAPDkcZWX5s8w@mail.gmail.com>
	<EB126D4A-44E9-4E9D-A442-7DDD5977D337@oracle.com>
Message-ID: <57A3AF4C.7050402@chello.at>

Hi Gerard,

the Cover project sounds great!

We are also implementing a memory safe version of Sulong. The ideas for 
this version are described in the following paper: 
http://ssw.jku.at/General/Staff/ManuelRigger/ECOOP16-DS.pdf. Maybe you 
can apply some of the ideas from the paper to Cover, or give us feedback.

- Manuel

On 07/24/2016 05:07 PM, Chris Seaton wrote:
> That sounds like a great project!
>
> I think everything that would have been useful to you from TruffleC will have been re-implemented in Sulong. For example how to use our high performance native function interface, how to mix managed/native memory, and so on.
>
> TruffleC did have some interesting work to support ASTs that included constructs like ?goto?. We can explain how that works to you if you need that later on.
>
> Chris
>
>> On 24 Jul 2016, at 15:53, Gerard Krol <gerard at gerardkrol.nl> wrote:
>>
>> Hello Chris,
>>
>> Thanks for your quick response. I'm interested in building a C++ like
>> language using Truffle, that would support a "sensible" subset of C++. It
>> would use garbage collection and provide memory safety though, as well as
>> other things that in my opinion are missing from C++. The language is
>> called "Cover" (for now).
>>
>> I'm basing Cover on SimpleLanguage, and am currently using the Eclipse CDT
>> C++ parser. Currently it is in the "hello world + loops" phase. If you want
>> to follow along take a look at https://github.com/gerard-/cover .
>>
>> Regards,
>>
>> Gerard
>>
>>
>> On Sun, Jul 24, 2016 at 3:19 PM, Chris Seaton <chris.seaton at oracle.com>
>> wrote:
>>
>>> Hello Gerard,
>>>
>>> The old C implementation was called TruffleC. It interpreted the C
>>> language AST. Sulong interprets the LLVM IR instead. That?s not really a
>>> huge difference in practice. LLVM IR is a bit like a linearised version of
>>> the AST and doesn?t include much lowering or optimisation. TruffleC uses
>>> the same clever hacks as Sulong.
>>>
>>> JRuby's C extensions (which are still at an early stage) used to use
>>> TruffleC, but they now use Sulong.
>>>
>>> TruffleC isn?t open source, and there aren?t any plans that I?m aware of
>>> to open source it. Sulong is open source already. If you were happy with
>>> the TruffleC approach there?s not any reason that I know of that would mean
>>> Sulong wouldn?t also be appropriate, so I see TruffleC as deprecated by
>>> Sulong which is why there isn?t a great demand to open source it.
>>>
>>> Chris
>>>
>>>> On 24 Jul 2016, at 09:12, Gerard Krol <gerard at gerardkrol.nl> wrote:
>>>>
>>>> Hi,
>>>>
>>>> I've recently learned about Tuffle/Graal. It seems great technology.
>>>>
>>>> I'm interested in running C++ code on the JVM, without resorting to the
>>>> (admittedly clever) Sulong hack. I've seen some presentations and read
>>> some
>>>> papers about a C implementation, but that one doesn't seem to be open
>>>> source. The same goes for the C extensions for JRuby. Is that correct?
>>>>
>>>> If so, what are the reasons for not open sourcing it? Any plans to do so
>>> in
>>>> the future?
>>>>
>>>> Regards,
>>>>
>>>> Gerard
>>>
>


From cristian.esquivias at gmail.com  Thu Aug  4 23:03:59 2016
From: cristian.esquivias at gmail.com (Cristian Esquivias)
Date: Thu, 4 Aug 2016 16:03:59 -0700
Subject: NPE with -Dgraal.Dump flag
Message-ID: <CAH0=iZc+iTk+UCWbQWWaL_r_c+gsD1bMOnti+qHzO7ZVDTo7TQ@mail.gmail.com>

Hi all,

I'm getting stacktraces when I run my language with -Dgraal.Dump and the
relevant graphs don't seem to be showing up in IGV. I got everything set up
via the README on github.com/graalvm/graal-core and used `mx vm` and `mx
igv` to run the program and to see the AST, respectively.

Is there a bug in the dump or do I need to tweak language? I had a working
version last week but I had to switch computers and got the errors after I
rebuilt the stack. I haven't changed my language, but maybe there's some
extra information that's now needed?

I'm running on Ubuntu 16.04 on VirtualBox.

Thanks,
Cristian

Sample stacktrace:
CFGPrinter: Exception during output of PhaseSuite:
java.lang.NullPointerException
java.lang.NullPointerException
at
com.oracle.graal.printer.CFGPrinterObserver.dumpSandboxed(CFGPrinterObserver.java:229)
at
com.oracle.graal.printer.CFGPrinterObserver.dump(CFGPrinterObserver.java:83)
at com.oracle.graal.debug.internal.DebugScope.dump(DebugScope.java:303)
at com.oracle.graal.debug.Debug.dump(Debug.java:669)
at com.oracle.graal.phases.BasePhase.apply(BasePhase.java:180)
at com.oracle.graal.phases.BasePhase.apply(BasePhase.java:154)
at
com.oracle.graal.compiler.GraalCompiler.emitFrontEnd(GraalCompiler.java:210)
at com.oracle.graal.compiler.GraalCompiler.compile(GraalCompiler.java:175)
at
com.oracle.graal.compiler.GraalCompiler.compileGraph(GraalCompiler.java:162)
at
com.oracle.graal.truffle.hotspot.HotSpotTruffleRuntime.compileMethod(HotSpotTruffleRuntime.java:233)
at
com.oracle.graal.truffle.hotspot.HotSpotTruffleRuntime.installOptimizedCallTargetCallMethod(HotSpotTruffleRuntime.java:191)
at
com.oracle.graal.truffle.hotspot.HotSpotTruffleRuntime.reinstallStubs(HotSpotTruffleRuntime.java:292)
at com.oracle.graal.truffle.TruffleCompiler.<init>(TruffleCompiler.java:112)
at
com.oracle.graal.truffle.DefaultTruffleCompiler.<init>(DefaultTruffleCompiler.java:48)
at
com.oracle.graal.truffle.DefaultTruffleCompiler.create(DefaultTruffleCompiler.java:44)
at
com.oracle.graal.truffle.hotspot.HotSpotTruffleRuntime.initializeTruffleCompiler(HotSpotTruffleRuntime.java:158)
at
com.oracle.graal.truffle.hotspot.HotSpotTruffleRuntime.getTruffleCompiler(HotSpotTruffleRuntime.java:149)
at
com.oracle.graal.truffle.GraalTruffleRuntime.doCompile0(GraalTruffleRuntime.java:465)
at
com.oracle.graal.truffle.GraalTruffleRuntime.doCompile(GraalTruffleRuntime.java:451)
at
com.oracle.graal.truffle.GraalTruffleRuntime$1.run(GraalTruffleRuntime.java:483)
at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:511)
at java.util.concurrent.FutureTask.run(FutureTask.java:266)
at
java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142)
at
java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617)
at java.lang.Thread.run(Thread.java:745)
at com.oracle.graal.compiler.CompilerThread.run(CompilerThread.java:51)

From tom.rodriguez at oracle.com  Thu Aug  4 23:48:31 2016
From: tom.rodriguez at oracle.com (Tom Rodriguez)
Date: Thu, 4 Aug 2016 16:48:31 -0700
Subject: NPE with -Dgraal.Dump flag
In-Reply-To: <CAH0=iZc+iTk+UCWbQWWaL_r_c+gsD1bMOnti+qHzO7ZVDTo7TQ@mail.gmail.com>
References: <CAH0=iZc+iTk+UCWbQWWaL_r_c+gsD1bMOnti+qHzO7ZVDTo7TQ@mail.gmail.com>
Message-ID: <EC81F33C-560B-45BE-BB77-C27E4FEEE19C@oracle.com>

Sorry, that?s my fault.  I added a try/finally there but bound the wrong part.  I?ll push a fix for it.  The exception only happens in cases where CFGPrinterObserver wasn?t going to do anything so it?s relatively benign.

tom

> On Aug 4, 2016, at 4:03 PM, Cristian Esquivias <cristian.esquivias at gmail.com> wrote:
> 
> Hi all,
> 
> I'm getting stacktraces when I run my language with -Dgraal.Dump and the
> relevant graphs don't seem to be showing up in IGV. I got everything set up
> via the README on github.com/graalvm/graal-core and used `mx vm` and `mx
> igv` to run the program and to see the AST, respectively.
> 
> Is there a bug in the dump or do I need to tweak language? I had a working
> version last week but I had to switch computers and got the errors after I
> rebuilt the stack. I haven't changed my language, but maybe there's some
> extra information that's now needed?
> 
> I'm running on Ubuntu 16.04 on VirtualBox.
> 
> Thanks,
> Cristian
> 
> Sample stacktrace:
> CFGPrinter: Exception during output of PhaseSuite:
> java.lang.NullPointerException
> java.lang.NullPointerException
> at
> com.oracle.graal.printer.CFGPrinterObserver.dumpSandboxed(CFGPrinterObserver.java:229)
> at
> com.oracle.graal.printer.CFGPrinterObserver.dump(CFGPrinterObserver.java:83)
> at com.oracle.graal.debug.internal.DebugScope.dump(DebugScope.java:303)
> at com.oracle.graal.debug.Debug.dump(Debug.java:669)
> at com.oracle.graal.phases.BasePhase.apply(BasePhase.java:180)
> at com.oracle.graal.phases.BasePhase.apply(BasePhase.java:154)
> at
> com.oracle.graal.compiler.GraalCompiler.emitFrontEnd(GraalCompiler.java:210)
> at com.oracle.graal.compiler.GraalCompiler.compile(GraalCompiler.java:175)
> at
> com.oracle.graal.compiler.GraalCompiler.compileGraph(GraalCompiler.java:162)
> at
> com.oracle.graal.truffle.hotspot.HotSpotTruffleRuntime.compileMethod(HotSpotTruffleRuntime.java:233)
> at
> com.oracle.graal.truffle.hotspot.HotSpotTruffleRuntime.installOptimizedCallTargetCallMethod(HotSpotTruffleRuntime.java:191)
> at
> com.oracle.graal.truffle.hotspot.HotSpotTruffleRuntime.reinstallStubs(HotSpotTruffleRuntime.java:292)
> at com.oracle.graal.truffle.TruffleCompiler.<init>(TruffleCompiler.java:112)
> at
> com.oracle.graal.truffle.DefaultTruffleCompiler.<init>(DefaultTruffleCompiler.java:48)
> at
> com.oracle.graal.truffle.DefaultTruffleCompiler.create(DefaultTruffleCompiler.java:44)
> at
> com.oracle.graal.truffle.hotspot.HotSpotTruffleRuntime.initializeTruffleCompiler(HotSpotTruffleRuntime.java:158)
> at
> com.oracle.graal.truffle.hotspot.HotSpotTruffleRuntime.getTruffleCompiler(HotSpotTruffleRuntime.java:149)
> at
> com.oracle.graal.truffle.GraalTruffleRuntime.doCompile0(GraalTruffleRuntime.java:465)
> at
> com.oracle.graal.truffle.GraalTruffleRuntime.doCompile(GraalTruffleRuntime.java:451)
> at
> com.oracle.graal.truffle.GraalTruffleRuntime$1.run(GraalTruffleRuntime.java:483)
> at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:511)
> at java.util.concurrent.FutureTask.run(FutureTask.java:266)
> at
> java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142)
> at
> java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617)
> at java.lang.Thread.run(Thread.java:745)
> at com.oracle.graal.compiler.CompilerThread.run(CompilerThread.java:51)


From cristian.esquivias at gmail.com  Fri Aug  5 01:10:08 2016
From: cristian.esquivias at gmail.com (Cristian Esquivias)
Date: Thu, 4 Aug 2016 18:10:08 -0700
Subject: NPE with -Dgraal.Dump flag
In-Reply-To: <EC81F33C-560B-45BE-BB77-C27E4FEEE19C@oracle.com>
References: <CAH0=iZc+iTk+UCWbQWWaL_r_c+gsD1bMOnti+qHzO7ZVDTo7TQ@mail.gmail.com>
	<EC81F33C-560B-45BE-BB77-C27E4FEEE19C@oracle.com>
Message-ID: <CAH0=iZeaT+Ubm_3Zw8KdY5C4Q5xrXC4cLUjrcBErFRQ=eYXkvQ@mail.gmail.com>

Thanks, Tom. I feel like the output to IGV changed at the same time I
started seeing the stacktraces, but that could've been my imagination. I'll
circle back once I rebuild and verify a change.

- Cristian

On Thu, Aug 4, 2016 at 4:48 PM, Tom Rodriguez <tom.rodriguez at oracle.com>
wrote:

> Sorry, that?s my fault.  I added a try/finally there but bound the wrong
> part.  I?ll push a fix for it.  The exception only happens in cases where
> CFGPrinterObserver wasn?t going to do anything so it?s relatively benign.
>
> tom
>
> > On Aug 4, 2016, at 4:03 PM, Cristian Esquivias <
> cristian.esquivias at gmail.com> wrote:
> >
> > Hi all,
> >
> > I'm getting stacktraces when I run my language with -Dgraal.Dump and the
> > relevant graphs don't seem to be showing up in IGV. I got everything set
> up
> > via the README on github.com/graalvm/graal-core and used `mx vm` and `mx
> > igv` to run the program and to see the AST, respectively.
> >
> > Is there a bug in the dump or do I need to tweak language? I had a
> working
> > version last week but I had to switch computers and got the errors after
> I
> > rebuilt the stack. I haven't changed my language, but maybe there's some
> > extra information that's now needed?
> >
> > I'm running on Ubuntu 16.04 on VirtualBox.
> >
> > Thanks,
> > Cristian
> >
> > Sample stacktrace:
> > CFGPrinter: Exception during output of PhaseSuite:
> > java.lang.NullPointerException
> > java.lang.NullPointerException
> > at
> > com.oracle.graal.printer.CFGPrinterObserver.dumpSandboxed(
> CFGPrinterObserver.java:229)
> > at
> > com.oracle.graal.printer.CFGPrinterObserver.dump(
> CFGPrinterObserver.java:83)
> > at com.oracle.graal.debug.internal.DebugScope.dump(DebugScope.java:303)
> > at com.oracle.graal.debug.Debug.dump(Debug.java:669)
> > at com.oracle.graal.phases.BasePhase.apply(BasePhase.java:180)
> > at com.oracle.graal.phases.BasePhase.apply(BasePhase.java:154)
> > at
> > com.oracle.graal.compiler.GraalCompiler.emitFrontEnd(
> GraalCompiler.java:210)
> > at com.oracle.graal.compiler.GraalCompiler.compile(
> GraalCompiler.java:175)
> > at
> > com.oracle.graal.compiler.GraalCompiler.compileGraph(
> GraalCompiler.java:162)
> > at
> > com.oracle.graal.truffle.hotspot.HotSpotTruffleRuntime.compileMethod(
> HotSpotTruffleRuntime.java:233)
> > at
> > com.oracle.graal.truffle.hotspot.HotSpotTruffleRuntime.
> installOptimizedCallTargetCallMethod(HotSpotTruffleRuntime.java:191)
> > at
> > com.oracle.graal.truffle.hotspot.HotSpotTruffleRuntime.reinstallStubs(
> HotSpotTruffleRuntime.java:292)
> > at com.oracle.graal.truffle.TruffleCompiler.<init>(
> TruffleCompiler.java:112)
> > at
> > com.oracle.graal.truffle.DefaultTruffleCompiler.<init>(
> DefaultTruffleCompiler.java:48)
> > at
> > com.oracle.graal.truffle.DefaultTruffleCompiler.create(
> DefaultTruffleCompiler.java:44)
> > at
> > com.oracle.graal.truffle.hotspot.HotSpotTruffleRuntime.
> initializeTruffleCompiler(HotSpotTruffleRuntime.java:158)
> > at
> > com.oracle.graal.truffle.hotspot.HotSpotTruffleRuntime.
> getTruffleCompiler(HotSpotTruffleRuntime.java:149)
> > at
> > com.oracle.graal.truffle.GraalTruffleRuntime.doCompile0(
> GraalTruffleRuntime.java:465)
> > at
> > com.oracle.graal.truffle.GraalTruffleRuntime.doCompile(
> GraalTruffleRuntime.java:451)
> > at
> > com.oracle.graal.truffle.GraalTruffleRuntime$1.run(
> GraalTruffleRuntime.java:483)
> > at java.util.concurrent.Executors$RunnableAdapter.
> call(Executors.java:511)
> > at java.util.concurrent.FutureTask.run(FutureTask.java:266)
> > at
> > java.util.concurrent.ThreadPoolExecutor.runWorker(
> ThreadPoolExecutor.java:1142)
> > at
> > java.util.concurrent.ThreadPoolExecutor$Worker.run(
> ThreadPoolExecutor.java:617)
> > at java.lang.Thread.run(Thread.java:745)
> > at com.oracle.graal.compiler.CompilerThread.run(CompilerThread.java:51)
>
>

From doug.simon at oracle.com  Fri Aug  5 09:00:51 2016
From: doug.simon at oracle.com (Doug Simon)
Date: Fri, 5 Aug 2016 11:00:51 +0200
Subject: Updating truffle import in graal-core
Message-ID: <C414FDE0-D02D-46D2-BF05-13CAC7FEEB9A@oracle.com>

I?m planning on updating graal-core with the latest Truffle changes. This is in preparation for a GraalVM 0.15 release soon since 0.14 missed out on the latest graal-core and graal-enterprise bits.

Please let me know asap if I should hold off on this. The commits included in the update are listed below.

-Doug

commit bd163128ec958b97ebc68b33ac5b4fae376a37b5
Merge: 64858a0 a23d950
Author: Andreas Woess <andreas.woess at oracle.com>
Date:   Wed Aug 3 05:38:31 2016 -0700

    Merge pull request #155 in G/truffle from ~ANDREAS.WOESS_ORACLE.COM/truffle:pr/interop-gen-codestyle to master

    * commit 'a23d950104592eb9ff7f5d55da309b40cb4b7b88':
      Fix checkstyle issues in generated code

commit 64858a010dcce60422e0bcc49171a6e9ea862076
Merge: 534a039 af32b3f
Author: Christian Humer <christian.humer at oracle.com>
Date:   Thu Jul 28 07:17:55 2016 -0700

    Merge pull request #153 in G/truffle from cwi/Truffle-compile-immediately-test to master

    * commit 'af32b3ff953bbe8b176bf37385fe62ec8e883371':
      Make SLTestSuite more flexible

commit af32b3ff953bbe8b176bf37385fe62ec8e883371
Author: Christian Wimmer <christian.wimmer at oracle.com>
Date:   Tue Jul 26 12:00:23 2016 -0700

    Make SLTestSuite more flexible

commit a23d950104592eb9ff7f5d55da309b40cb4b7b88
Author: Stefan Marr <git at stefan-marr.de>
Date:   Thu Jun 30 22:37:33 2016 +0200

    Fix checkstyle issues in generated code

    ?static final? is the recommended order by JLS.

    Signed-off-by: Stefan Marr <git at stefan-marr.de>

From gerard at gerardkrol.nl  Fri Aug  5 20:48:07 2016
From: gerard at gerardkrol.nl (Gerard Krol)
Date: Fri, 5 Aug 2016 22:48:07 +0200
Subject: Open Source C/C++ Truffle languages available?
In-Reply-To: <57A3AF4C.7050402@chello.at>
References: <CAFzGNhKuf88CAhxAqAn4Xk_5bW-5mXKXCxO0cY_kDtyApCJ2ew@mail.gmail.com>
	<4FEFE529-C757-4AD3-885C-2B8B820C9FBB@oracle.com>
	<CAFzGNhLYhLDyk0nmoW1xzReos95gOe-R1QDLLnAPDkcZWX5s8w@mail.gmail.com>
	<EB126D4A-44E9-4E9D-A442-7DDD5977D337@oracle.com>
	<57A3AF4C.7050402@chello.at>
Message-ID: <CAFzGNhJ0gGmQSJ965CTWZWpfb4hh91TrcyL=Ehb1RqY_N3BAnA@mail.gmail.com>

Hi Manuel,

Thanks for the link. I'm trying to get away with just disallowing the
unsafe stuff, but if I find I need to add support I'll certainly see what I
can learn from Sulong!

Regards,

Gerard

On Thu, Aug 4, 2016 at 11:10 PM, Manuel Rigger <manuel.rigger at chello.at>
wrote:

> Hi Gerard,
>
> the Cover project sounds great!
>
> We are also implementing a memory safe version of Sulong. The ideas for
> this version are described in the following paper:
> http://ssw.jku.at/General/Staff/ManuelRigger/ECOOP16-DS.pdf. Maybe you
> can apply some of the ideas from the paper to Cover, or give us feedback.
>
> - Manuel
>
> On 07/24/2016 05:07 PM, Chris Seaton wrote:
>
>> That sounds like a great project!
>>
>> I think everything that would have been useful to you from TruffleC will
>> have been re-implemented in Sulong. For example how to use our high
>> performance native function interface, how to mix managed/native memory,
>> and so on.
>>
>> TruffleC did have some interesting work to support ASTs that included
>> constructs like ?goto?. We can explain how that works to you if you need
>> that later on.
>>
>> Chris
>>
>> On 24 Jul 2016, at 15:53, Gerard Krol <gerard at gerardkrol.nl> wrote:
>>>
>>> Hello Chris,
>>>
>>> Thanks for your quick response. I'm interested in building a C++ like
>>> language using Truffle, that would support a "sensible" subset of C++. It
>>> would use garbage collection and provide memory safety though, as well as
>>> other things that in my opinion are missing from C++. The language is
>>> called "Cover" (for now).
>>>
>>> I'm basing Cover on SimpleLanguage, and am currently using the Eclipse
>>> CDT
>>> C++ parser. Currently it is in the "hello world + loops" phase. If you
>>> want
>>> to follow along take a look at https://github.com/gerard-/cover .
>>>
>>> Regards,
>>>
>>> Gerard
>>>
>>>
>>> On Sun, Jul 24, 2016 at 3:19 PM, Chris Seaton <chris.seaton at oracle.com>
>>> wrote:
>>>
>>> Hello Gerard,
>>>>
>>>> The old C implementation was called TruffleC. It interpreted the C
>>>> language AST. Sulong interprets the LLVM IR instead. That?s not really a
>>>> huge difference in practice. LLVM IR is a bit like a linearised version
>>>> of
>>>> the AST and doesn?t include much lowering or optimisation. TruffleC uses
>>>> the same clever hacks as Sulong.
>>>>
>>>> JRuby's C extensions (which are still at an early stage) used to use
>>>> TruffleC, but they now use Sulong.
>>>>
>>>> TruffleC isn?t open source, and there aren?t any plans that I?m aware of
>>>> to open source it. Sulong is open source already. If you were happy with
>>>> the TruffleC approach there?s not any reason that I know of that would
>>>> mean
>>>> Sulong wouldn?t also be appropriate, so I see TruffleC as deprecated by
>>>> Sulong which is why there isn?t a great demand to open source it.
>>>>
>>>> Chris
>>>>
>>>> On 24 Jul 2016, at 09:12, Gerard Krol <gerard at gerardkrol.nl> wrote:
>>>>>
>>>>> Hi,
>>>>>
>>>>> I've recently learned about Tuffle/Graal. It seems great technology.
>>>>>
>>>>> I'm interested in running C++ code on the JVM, without resorting to the
>>>>> (admittedly clever) Sulong hack. I've seen some presentations and read
>>>>>
>>>> some
>>>>
>>>>> papers about a C implementation, but that one doesn't seem to be open
>>>>> source. The same goes for the C extensions for JRuby. Is that correct?
>>>>>
>>>>> If so, what are the reasons for not open sourcing it? Any plans to do
>>>>> so
>>>>>
>>>> in
>>>>
>>>>> the future?
>>>>>
>>>>> Regards,
>>>>>
>>>>> Gerard
>>>>>
>>>>
>>>>
>>
>

From headius at headius.com  Sat Aug  6 00:35:53 2016
From: headius at headius.com (Charles Oliver Nutter)
Date: Fri, 5 Aug 2016 17:35:53 -0700
Subject: Poor performance with invokedynamic on Graal
Message-ID: <CAE-f1xRtUvFH0Z6DKnTyDut7u4bkHcRD=-AYqQrSyqM-SJ+2pw@mail.gmail.com>

Hey all!

I've started to poke at running JRuby on Graal and I've been pretty
disappointed with the results so far. It appears that Graal is not doing a
good job of optimizing through invokedynamic call sites, which largely
defeats our best opportunity for optimization.

I've tried two benchmarks today against graal-core HEAD.

The first is trivial: a recursive fib benchmark. JRuby+indy on Hotspot
(8u92) runs fib(35) in around 0.32s. JRuby+indy on Graal takes 0.52s or so
per iteration.

https://gist.github.com/headius/492fb995e15b50d3db42145339f21968

The second benchmark is more robust: a pure-Ruby red/black tree
implementation. The benchmark builds up a large tree, searches it, deletes
from it, etc. JRuby+indy on Hotspot can run an iteration in about 0.41s. On
Graal, it takes 1s or more.

This is all especially unfortunate because some time ago I was able to see
a 5x improvement on a mandelbrot benchmark. I don't see that on recent
Graal builds.

Here's how to reproduce...

* Grab a master build of JRuby from http://ci.jruby.org. Look for
9.1.3.0-snapshot "bin" tarball.
* Unpack, add bin/ to PATH so JRuby gets picked up.
* Run the benchmarks I've gisted below.

https://gist.github.com/headius/5eb47848fbe486266942695891b0d9f6

Let me know if you have question! I want to see this run as well as I
believe it should!

- Charlie

From thomas.wuerthinger at oracle.com  Mon Aug  8 17:49:13 2016
From: thomas.wuerthinger at oracle.com (Thomas Wuerthinger)
Date: Mon, 8 Aug 2016 20:49:13 +0300
Subject: Poor performance with invokedynamic on Graal
In-Reply-To: <CAE-f1xRtUvFH0Z6DKnTyDut7u4bkHcRD=-AYqQrSyqM-SJ+2pw@mail.gmail.com>
References: <CAE-f1xRtUvFH0Z6DKnTyDut7u4bkHcRD=-AYqQrSyqM-SJ+2pw@mail.gmail.com>
Message-ID: <07C4E739-3422-4A53-AB6A-3E543F6A2601@oracle.com>

Hi Charlie!

Thanks for the regression report. Our policy with Graal is to not accept any significant peak slow-down on realistic workloads compared to C2. We will create a ticket to track progress. Can you also specify the version of the JDK used with Graal (i.e., the full ?-version? output of the JVM)?

We did not (wittingly) change the behavior of Graal related to invokedynamic. Maybe some VM internals have changed that we did not yet adopt for.

Obviously, we recommend on the long run to use Truffle+JRuby. We see fairly convincing numbers with Truffle enabled for the mentioned red-black tree benchmark on our performance tracking infrastructure.

- thomas


> On 06 Aug 2016, at 03:35, Charles Oliver Nutter <headius at headius.com> wrote:
> 
> Hey all!
> 
> I've started to poke at running JRuby on Graal and I've been pretty
> disappointed with the results so far. It appears that Graal is not doing a
> good job of optimizing through invokedynamic call sites, which largely
> defeats our best opportunity for optimization.
> 
> I've tried two benchmarks today against graal-core HEAD.
> 
> The first is trivial: a recursive fib benchmark. JRuby+indy on Hotspot
> (8u92) runs fib(35) in around 0.32s. JRuby+indy on Graal takes 0.52s or so
> per iteration.
> 
> https://gist.github.com/headius/492fb995e15b50d3db42145339f21968
> 
> The second benchmark is more robust: a pure-Ruby red/black tree
> implementation. The benchmark builds up a large tree, searches it, deletes
> from it, etc. JRuby+indy on Hotspot can run an iteration in about 0.41s. On
> Graal, it takes 1s or more.
> 
> This is all especially unfortunate because some time ago I was able to see
> a 5x improvement on a mandelbrot benchmark. I don't see that on recent
> Graal builds.
> 
> Here's how to reproduce...
> 
> * Grab a master build of JRuby from http://ci.jruby.org. Look for
> 9.1.3.0-snapshot "bin" tarball.
> * Unpack, add bin/ to PATH so JRuby gets picked up.
> * Run the benchmarks I've gisted below.
> 
> https://gist.github.com/headius/5eb47848fbe486266942695891b0d9f6
> 
> Let me know if you have question! I want to see this run as well as I
> believe it should!
> 
> - Charlie


From headius at headius.com  Mon Aug  8 17:53:00 2016
From: headius at headius.com (Charles Oliver Nutter)
Date: Mon, 8 Aug 2016 12:53:00 -0500
Subject: Poor performance with invokedynamic on Graal
In-Reply-To: <07C4E739-3422-4A53-AB6A-3E543F6A2601@oracle.com>
References: <CAE-f1xRtUvFH0Z6DKnTyDut7u4bkHcRD=-AYqQrSyqM-SJ+2pw@mail.gmail.com>
	<07C4E739-3422-4A53-AB6A-3E543F6A2601@oracle.com>
Message-ID: <CAE-f1xTVPy6_PyZ0R05MSSJqJbv6XwVuDks44Fsv0xztbkUp8g@mail.gmail.com>

On Mon, Aug 8, 2016 at 10:49 AM, Thomas Wuerthinger <
thomas.wuerthinger at oracle.com> wrote:

> Thanks for the regression report. Our policy with Graal is to not accept
> any significant peak slow-down on realistic workloads compared to C2. We
> will create a ticket to track progress. Can you also specify the version of
> the JDK used with Graal (i.e., the full ?-version? output of the JVM)?
>

Here's the full -version:

java version "1.8.0_92"

Java(TM) SE Runtime Environment (build 1.8.0_92-b14)

OpenJDK 64-Bit Server VM (build 25.71-b01-internal-jvmci-0.19-dev, mixed
mode)

> We did not (wittingly) change the behavior of Graal related to
> invokedynamic. Maybe some VM internals have changed that we did not yet
> adopt for.
>

At the moment I'm digging into PEA to see why it's not doing anything for a
simple Ruby loop. Everything inlines, but boxes are apparently still being
created. PEA finishes without making any changes.


> Obviously, we recommend on the long run to use Truffle+JRuby. We see
> fairly convincing numbers with Truffle enabled for the mentioned red-black
> tree benchmark on our performance tracking infrastructure.
>

Yes, I'm sure you do :-) However I'm interested in getting current JRuby
users better performance...I'm interested in near-term production Ruby use
cases.

I also believe that with better object shaping and a bit of method
specialization (both works-in-progress) in JRuby's IR, we can be
competitive with JRuby+Truffle on any compiler that provides partial escape
analysis.

- Charlie

From doug.simon at oracle.com  Mon Aug  8 17:56:19 2016
From: doug.simon at oracle.com (Doug Simon)
Date: Mon, 8 Aug 2016 19:56:19 +0200
Subject: Poor performance with invokedynamic on Graal
In-Reply-To: <07C4E739-3422-4A53-AB6A-3E543F6A2601@oracle.com>
References: <CAE-f1xRtUvFH0Z6DKnTyDut7u4bkHcRD=-AYqQrSyqM-SJ+2pw@mail.gmail.com>
	<07C4E739-3422-4A53-AB6A-3E543F6A2601@oracle.com>
Message-ID: <A210156E-838C-4AEB-BC34-3C10C55182D1@oracle.com>

Hi Charlie,

I?ve created an issue to track our investigation into the reported performance issue:

https://github.com/graalvm/graal-core/issues/202

Feel free to add an extra discoveries you make there.

Thanks,

-Doug

> On 08 Aug 2016, at 19:49, Thomas Wuerthinger <thomas.wuerthinger at oracle.com> wrote:
> 
> Hi Charlie!
> 
> Thanks for the regression report. Our policy with Graal is to not accept any significant peak slow-down on realistic workloads compared to C2. We will create a ticket to track progress. Can you also specify the version of the JDK used with Graal (i.e., the full ?-version? output of the JVM)?
> 
> We did not (wittingly) change the behavior of Graal related to invokedynamic. Maybe some VM internals have changed that we did not yet adopt for.
> 
> Obviously, we recommend on the long run to use Truffle+JRuby. We see fairly convincing numbers with Truffle enabled for the mentioned red-black tree benchmark on our performance tracking infrastructure.
> 
> - thomas
> 
> 
>> On 06 Aug 2016, at 03:35, Charles Oliver Nutter <headius at headius.com> wrote:
>> 
>> Hey all!
>> 
>> I've started to poke at running JRuby on Graal and I've been pretty
>> disappointed with the results so far. It appears that Graal is not doing a
>> good job of optimizing through invokedynamic call sites, which largely
>> defeats our best opportunity for optimization.
>> 
>> I've tried two benchmarks today against graal-core HEAD.
>> 
>> The first is trivial: a recursive fib benchmark. JRuby+indy on Hotspot
>> (8u92) runs fib(35) in around 0.32s. JRuby+indy on Graal takes 0.52s or so
>> per iteration.
>> 
>> https://gist.github.com/headius/492fb995e15b50d3db42145339f21968
>> 
>> The second benchmark is more robust: a pure-Ruby red/black tree
>> implementation. The benchmark builds up a large tree, searches it, deletes
>> from it, etc. JRuby+indy on Hotspot can run an iteration in about 0.41s. On
>> Graal, it takes 1s or more.
>> 
>> This is all especially unfortunate because some time ago I was able to see
>> a 5x improvement on a mandelbrot benchmark. I don't see that on recent
>> Graal builds.
>> 
>> Here's how to reproduce...
>> 
>> * Grab a master build of JRuby from http://ci.jruby.org. Look for
>> 9.1.3.0-snapshot "bin" tarball.
>> * Unpack, add bin/ to PATH so JRuby gets picked up.
>> * Run the benchmarks I've gisted below.
>> 
>> https://gist.github.com/headius/5eb47848fbe486266942695891b0d9f6
>> 
>> Let me know if you have question! I want to see this run as well as I
>> believe it should!
>> 
>> - Charlie
> 


From christian.wimmer at oracle.com  Mon Aug  8 18:40:10 2016
From: christian.wimmer at oracle.com (Christian Wimmer)
Date: Mon, 8 Aug 2016 11:40:10 -0700
Subject: SubstrateVM Questions
In-Reply-To: <CAHZjNtaBpSfWTw7kNYB-t-kEgwPS=mgraf8hX0HchBPoETztbA@mail.gmail.com>
References: <CAHZjNtaBpSfWTw7kNYB-t-kEgwPS=mgraf8hX0HchBPoETztbA@mail.gmail.com>
Message-ID: <6a2fe427-6422-830c-ddb6-f338111f35d8@oracle.com>


On 07/30/2016 05:48 AM, Sidharth Kshatriya wrote:
> Graal/Truffle with Sulong seems to be a really compelling platform to build
> a programming language! Uniquely, there are also a large number of papers
> that go into detail on how things work. Its well documented too!
>
> There is one point of concern and that is the warm up time of any
> programming language JIT built on truffle/graal. Many dynamic scripting
> languages need to start up quickly and do their job. So while long running
> scripts will do the job well I'm concerned about the merits of making a
> dynamic language port using graal/truffle with the long warmup.
>
> I learnt about SubstrateVM in which you AOT compile some code which fixes
> startup time issues. Is that something that will be available to external
> implementers? Or will this just be something for scripting engines
> "blessed" by Oracle?

Our first step will be to put Substrate VM binaries on OTN, where we 
also release Graal VM.

> Also this SubstrateVM sounds like its going to have an embedded hotspot VM
> (if I can imagine correctly). Does that have implications for any external
> party wanting to distribute their scripting engine executable?

There are no parts from the HotSpot VM inside an executable created for 
a Truffle language (or other Java application).

The build process of the executable currently runs only on the Graal VM 
(or in the future on Java 9, since we do not depend on Graal but only on 
the JVMCI API that HotSpot will have built in soon).

-Christian

From sid.kshatriya at gmail.com  Mon Aug  8 20:06:13 2016
From: sid.kshatriya at gmail.com (Sidharth Kshatriya)
Date: Tue, 9 Aug 2016 01:36:13 +0530
Subject: SubstrateVM Questions
In-Reply-To: <6a2fe427-6422-830c-ddb6-f338111f35d8@oracle.com>
References: <CAHZjNtaBpSfWTw7kNYB-t-kEgwPS=mgraf8hX0HchBPoETztbA@mail.gmail.com>
	<6a2fe427-6422-830c-ddb6-f338111f35d8@oracle.com>
Message-ID: <CAHZjNtbq40iEkreT=WQBDfxWg-qQ4b-8iCGm8qotjKj6foydGQ@mail.gmail.com>

Thanks for your reply. I guess I don't understand the technology stack
intimately yet so I found the answer a bit confusing. My main concern is :
will I be able to construct a VM (which has very fast start up time) for my
custom language myself?

P.S. links to some documentation on substrate VM will be useful. Not much
stuff is available on the net about it.

On Aug 9, 2016 12:10 AM, "Christian Wimmer" <christian.wimmer at oracle.com>
wrote:

>
> On 07/30/2016 05:48 AM, Sidharth Kshatriya wrote:
>
>> Graal/Truffle with Sulong seems to be a really compelling platform to
>> build
>> a programming language! Uniquely, there are also a large number of papers
>> that go into detail on how things work. Its well documented too!
>>
>> There is one point of concern and that is the warm up time of any
>> programming language JIT built on truffle/graal. Many dynamic scripting
>> languages need to start up quickly and do their job. So while long running
>> scripts will do the job well I'm concerned about the merits of making a
>> dynamic language port using graal/truffle with the long warmup.
>>
>> I learnt about SubstrateVM in which you AOT compile some code which fixes
>> startup time issues. Is that something that will be available to external
>> implementers? Or will this just be something for scripting engines
>> "blessed" by Oracle?
>>
>
> Our first step will be to put Substrate VM binaries on OTN, where we also
> release Graal VM.
>
> Also this SubstrateVM sounds like its going to have an embedded hotspot VM
>> (if I can imagine correctly). Does that have implications for any external
>> party wanting to distribute their scripting engine executable?
>>
>
> There are no parts from the HotSpot VM inside an executable created for a
> Truffle language (or other Java application).
>
> The build process of the executable currently runs only on the Graal VM
> (or in the future on Java 9, since we do not depend on Graal but only on
> the JVMCI API that HotSpot will have built in soon).
>
> -Christian
>

From christian.wimmer at oracle.com  Mon Aug  8 20:22:34 2016
From: christian.wimmer at oracle.com (Christian Wimmer)
Date: Mon, 8 Aug 2016 13:22:34 -0700
Subject: SubstrateVM Questions
In-Reply-To: <CAHZjNtbq40iEkreT=WQBDfxWg-qQ4b-8iCGm8qotjKj6foydGQ@mail.gmail.com>
References: <CAHZjNtaBpSfWTw7kNYB-t-kEgwPS=mgraf8hX0HchBPoETztbA@mail.gmail.com>
	<6a2fe427-6422-830c-ddb6-f338111f35d8@oracle.com>
	<CAHZjNtbq40iEkreT=WQBDfxWg-qQ4b-8iCGm8qotjKj6foydGQ@mail.gmail.com>
Message-ID: <f33c9c25-8620-2900-a9fb-e87bf08e5fcd@oracle.com>



On 08/08/2016 01:06 PM, Sidharth Kshatriya wrote:
> Thanks for your reply. I guess I don't understand the technology stack
> intimately yet so I found the answer a bit confusing. My main concern is
> : will I be able to construct a VM (which has very fast start up time)
> for my custom language myself?

Yes, that is the plan.  Of course your language implementation will need 
to adhere to all the restrictions that Substrate VM has, e.g., your 
language implementation must not use reflection.

> P.S. links to some documentation on substrate VM will be useful. Not
> much stuff is available on the net about it.

I know, and releasing the binaries also means that we will write proper 
documentation on how to use it and how it works.

-Christian


> On Aug 9, 2016 12:10 AM, "Christian Wimmer" <christian.wimmer at oracle.com
> <mailto:christian.wimmer at oracle.com>> wrote:
>
>
>     On 07/30/2016 05:48 AM, Sidharth Kshatriya wrote:
>
>         Graal/Truffle with Sulong seems to be a really compelling
>         platform to build
>         a programming language! Uniquely, there are also a large number
>         of papers
>         that go into detail on how things work. Its well documented too!
>
>         There is one point of concern and that is the warm up time of any
>         programming language JIT built on truffle/graal. Many dynamic
>         scripting
>         languages need to start up quickly and do their job. So while
>         long running
>         scripts will do the job well I'm concerned about the merits of
>         making a
>         dynamic language port using graal/truffle with the long warmup.
>
>         I learnt about SubstrateVM in which you AOT compile some code
>         which fixes
>         startup time issues. Is that something that will be available to
>         external
>         implementers? Or will this just be something for scripting engines
>         "blessed" by Oracle?
>
>
>     Our first step will be to put Substrate VM binaries on OTN, where we
>     also release Graal VM.
>
>         Also this SubstrateVM sounds like its going to have an embedded
>         hotspot VM
>         (if I can imagine correctly). Does that have implications for
>         any external
>         party wanting to distribute their scripting engine executable?
>
>
>     There are no parts from the HotSpot VM inside an executable created
>     for a Truffle language (or other Java application).
>
>     The build process of the executable currently runs only on the Graal
>     VM (or in the future on Java 9, since we do not depend on Graal but
>     only on the JVMCI API that HotSpot will have built in soon).
>
>     -Christian
>

From christian.humer at gmail.com  Tue Aug  9 13:23:43 2016
From: christian.humer at gmail.com (Christian Humer)
Date: Tue, 09 Aug 2016 13:23:43 +0000
Subject: GraalVM 0.15 Available on OTN
Message-ID: <emf88fee36-c77f-45a2-884c-e4dbea5b91a9@grashalm-work>

Hi guys,

Just a short notice that a new version of GraalVM 0.15 is available on 
OTN:
http://www.oracle.com/technetwork/oracle-labs/program-languages/downloads/index.html

It includes the latest binary bits of Graal, its enterprise extensions 
and all the languages for Mac and Linux.
All repositories were tagged with "graal-vm-0.15" to mark the exact 
commit that was used for this build.

Have fun!

Cheers,
Christian Humer


From forax at univ-mlv.fr  Tue Aug  9 19:03:05 2016
From: forax at univ-mlv.fr (Remi Forax)
Date: Tue, 09 Aug 2016 21:03:05 +0200
Subject: GraalVM 0.15 Available on OTN
In-Reply-To: <emf88fee36-c77f-45a2-884c-e4dbea5b91a9@grashalm-work>
References: <emf88fee36-c77f-45a2-884c-e4dbea5b91a9@grashalm-work>
Message-ID: <9E5A8F76-437F-45C0-B8C3-D35DD69FF01C@univ-mlv.fr>

What the difference between the open source version of Graal and this binary version ?

Remi




On August 9, 2016 3:23:43 PM GMT+02:00, Christian Humer <christian.humer at gmail.com> wrote:
>Hi guys,
>
>Just a short notice that a new version of GraalVM 0.15 is available on 
>OTN:
>http://www.oracle.com/technetwork/oracle-labs/program-languages/downloads/index.html
>
>It includes the latest binary bits of Graal, its enterprise extensions 
>and all the languages for Mac and Linux.
>All repositories were tagged with "graal-vm-0.15" to mark the exact 
>commit that was used for this build.
>
>Have fun!
>
>Cheers,
>Christian Humer

-- 
Sent from my Android device with K-9 Mail. Please excuse my brevity.

From doug.simon at oracle.com  Tue Aug  9 19:22:55 2016
From: doug.simon at oracle.com (Doug Simon)
Date: Tue, 9 Aug 2016 21:22:55 +0200
Subject: GraalVM 0.15 Available on OTN
In-Reply-To: <9E5A8F76-437F-45C0-B8C3-D35DD69FF01C@univ-mlv.fr>
References: <emf88fee36-c77f-45a2-884c-e4dbea5b91a9@grashalm-work>
	<9E5A8F76-437F-45C0-B8C3-D35DD69FF01C@univ-mlv.fr>
Message-ID: <5DAA8DAB-0FD5-404C-9362-4BB50C8BFAEB@oracle.com>

Hi Remi,

The GraalVM binaries include the JavaScript, Ruby, and R engines in addition to the Graal compiler.

-Doug

> On 09 Aug 2016, at 21:03, Remi Forax <forax at univ-mlv.fr> wrote:
> 
> What the difference between the open source version of Graal and this binary version ?
> 
> Remi
> 
> 
> 
> 
> On August 9, 2016 3:23:43 PM GMT+02:00, Christian Humer <christian.humer at gmail.com> wrote:
>> Hi guys,
>> 
>> Just a short notice that a new version of GraalVM 0.15 is available on 
>> OTN:
>> http://www.oracle.com/technetwork/oracle-labs/program-languages/downloads/index.html
>> 
>> It includes the latest binary bits of Graal, its enterprise extensions 
>> and all the languages for Mac and Linux.
>> All repositories were tagged with "graal-vm-0.15" to mark the exact 
>> commit that was used for this build.
>> 
>> Have fun!
>> 
>> Cheers,
>> Christian Humer
> 
> -- 
> Sent from my Android device with K-9 Mail. Please excuse my brevity.


From headius at headius.com  Tue Aug  9 22:52:54 2016
From: headius at headius.com (Charles Oliver Nutter)
Date: Tue, 9 Aug 2016 17:52:54 -0500
Subject: GraalVM 0.15 Available on OTN
In-Reply-To: <5DAA8DAB-0FD5-404C-9362-4BB50C8BFAEB@oracle.com>
References: <emf88fee36-c77f-45a2-884c-e4dbea5b91a9@grashalm-work>
	<9E5A8F76-437F-45C0-B8C3-D35DD69FF01C@univ-mlv.fr>
	<5DAA8DAB-0FD5-404C-9362-4BB50C8BFAEB@oracle.com>
Message-ID: <CAE-f1xQQ1aPABL79tJLaYLpr8+ARkkf9QoqTpH69oA+yn3ghMw@mail.gmail.com>

I thought I'd heard there's optimizations in the closed source Graal that
aren't in the OSS repo. Was that just a vicious rumor?

- Charlie (mobile)

On Aug 9, 2016 2:53 PM, "Doug Simon" <doug.simon at oracle.com> wrote:

> Hi Remi,
>
> The GraalVM binaries include the JavaScript, Ruby, and R engines in
> addition to the Graal compiler.
>
> -Doug
>
> > On 09 Aug 2016, at 21:03, Remi Forax <forax at univ-mlv.fr> wrote:
> >
> > What the difference between the open source version of Graal and this
> binary version ?
> >
> > Remi
> >
> >
> >
> >
> > On August 9, 2016 3:23:43 PM GMT+02:00, Christian Humer <
> christian.humer at gmail.com> wrote:
> >> Hi guys,
> >>
> >> Just a short notice that a new version of GraalVM 0.15 is available on
> >> OTN:
> >> http://www.oracle.com/technetwork/oracle-labs/
> program-languages/downloads/index.html
> >>
> >> It includes the latest binary bits of Graal, its enterprise extensions
> >> and all the languages for Mac and Linux.
> >> All repositories were tagged with "graal-vm-0.15" to mark the exact
> >> commit that was used for this build.
> >>
> >> Have fun!
> >>
> >> Cheers,
> >> Christian Humer
> >
> > --
> > Sent from my Android device with K-9 Mail. Please excuse my brevity.
>
>

From rednaxelafx at gmail.com  Tue Aug  9 22:56:31 2016
From: rednaxelafx at gmail.com (Krystal Mok)
Date: Tue, 9 Aug 2016 15:56:31 -0700
Subject: GraalVM 0.15 Available on OTN
In-Reply-To: <5DAA8DAB-0FD5-404C-9362-4BB50C8BFAEB@oracle.com>
References: <emf88fee36-c77f-45a2-884c-e4dbea5b91a9@grashalm-work>
	<9E5A8F76-437F-45C0-B8C3-D35DD69FF01C@univ-mlv.fr>
	<5DAA8DAB-0FD5-404C-9362-4BB50C8BFAEB@oracle.com>
Message-ID: <CA+cQ+tR+gUx-9bcUyCx=QpY0_SS-Mt4tuMBeM43B2VC7cLznVg@mail.gmail.com>

Hi Doig and Chris,

So is this this binary release a Enterprise Graal release or graal-core +
the three Truffle languages?

Thanks,
Kris

On Tuesday, August 9, 2016, Doug Simon <doug.simon at oracle.com> wrote:

> Hi Remi,
>
> The GraalVM binaries include the JavaScript, Ruby, and R engines in
> addition to the Graal compiler.
>
> -Doug
>
> > On 09 Aug 2016, at 21:03, Remi Forax <forax at univ-mlv.fr <javascript:;>>
> wrote:
> >
> > What the difference between the open source version of Graal and this
> binary version ?
> >
> > Remi
> >
> >
> >
> >
> > On August 9, 2016 3:23:43 PM GMT+02:00, Christian Humer <
> christian.humer at gmail.com <javascript:;>> wrote:
> >> Hi guys,
> >>
> >> Just a short notice that a new version of GraalVM 0.15 is available on
> >> OTN:
> >> http://www.oracle.com/technetwork/oracle-labs/
> program-languages/downloads/index.html
> >>
> >> It includes the latest binary bits of Graal, its enterprise extensions
> >> and all the languages for Mac and Linux.
> >> All repositories were tagged with "graal-vm-0.15" to mark the exact
> >> commit that was used for this build.
> >>
> >> Have fun!
> >>
> >> Cheers,
> >> Christian Humer
> >
> > --
> > Sent from my Android device with K-9 Mail. Please excuse my brevity.
>
>

From doug.simon at oracle.com  Wed Aug 10 06:08:45 2016
From: doug.simon at oracle.com (Doug Simon)
Date: Wed, 10 Aug 2016 08:08:45 +0200
Subject: GraalVM 0.15 Available on OTN
In-Reply-To: <CA+cQ+tR+gUx-9bcUyCx=QpY0_SS-Mt4tuMBeM43B2VC7cLznVg@mail.gmail.com>
References: <emf88fee36-c77f-45a2-884c-e4dbea5b91a9@grashalm-work>
	<9E5A8F76-437F-45C0-B8C3-D35DD69FF01C@univ-mlv.fr>
	<5DAA8DAB-0FD5-404C-9362-4BB50C8BFAEB@oracle.com>
	<CA+cQ+tR+gUx-9bcUyCx=QpY0_SS-Mt4tuMBeM43B2VC7cLznVg@mail.gmail.com>
Message-ID: <B96E4FB4-44C3-4643-8C7F-FBD4AD796384@oracle.com>


> On 10 Aug 2016, at 00:56, Krystal Mok <rednaxelafx at gmail.com> wrote:
> 
> Hi Doig and Chris,
> 
> So is this this binary release a Enterprise Graal release or graal-core + the three Truffle languages?

The binary indeed contains Enterprise Graal + 3 Truffle languages. Sorry for the ambiguity.

-Doug

> 
> Thanks,
> Kris
> 
> On Tuesday, August 9, 2016, Doug Simon <doug.simon at oracle.com> wrote:
> Hi Remi,
> 
> The GraalVM binaries include the JavaScript, Ruby, and R engines in addition to the Graal compiler.
> 
> -Doug
> 
> > On 09 Aug 2016, at 21:03, Remi Forax <forax at univ-mlv.fr> wrote:
> >
> > What the difference between the open source version of Graal and this binary version ?
> >
> > Remi
> >
> >
> >
> >
> > On August 9, 2016 3:23:43 PM GMT+02:00, Christian Humer <christian.humer at gmail.com> wrote:
> >> Hi guys,
> >>
> >> Just a short notice that a new version of GraalVM 0.15 is available on
> >> OTN:
> >> http://www.oracle.com/technetwork/oracle-labs/program-languages/downloads/index.html
> >>
> >> It includes the latest binary bits of Graal, its enterprise extensions
> >> and all the languages for Mac and Linux.
> >> All repositories were tagged with "graal-vm-0.15" to mark the exact
> >> commit that was used for this build.
> >>
> >> Have fun!
> >>
> >> Cheers,
> >> Christian Humer
> >
> > --
> > Sent from my Android device with K-9 Mail. Please excuse my brevity.
> 


From thomas.wuerthinger at oracle.com  Wed Aug 10 09:13:42 2016
From: thomas.wuerthinger at oracle.com (Thomas Wuerthinger)
Date: Wed, 10 Aug 2016 12:13:42 +0300
Subject: GraalVM 0.15 Available on OTN
In-Reply-To: <CAE-f1xQQ1aPABL79tJLaYLpr8+ARkkf9QoqTpH69oA+yn3ghMw@mail.gmail.com>
References: <emf88fee36-c77f-45a2-884c-e4dbea5b91a9@grashalm-work>
	<9E5A8F76-437F-45C0-B8C3-D35DD69FF01C@univ-mlv.fr>
	<5DAA8DAB-0FD5-404C-9362-4BB50C8BFAEB@oracle.com>
	<CAE-f1xQQ1aPABL79tJLaYLpr8+ARkkf9QoqTpH69oA+yn3ghMw@mail.gmail.com>
Message-ID: <D43DC97E-2473-466C-8235-0495ADA53258@oracle.com>

There are additional features in the OTN download relevant for enterprise grade deployment. The speed-up of JRuby+Truffle over stock JRuby of 2-10x is however already obtained by a fully available open source stack.

- thomas

> On 10 Aug 2016, at 01:52, Charles Oliver Nutter <headius at headius.com> wrote:
> 
> I thought I'd heard there's optimizations in the closed source Graal that
> aren't in the OSS repo. Was that just a vicious rumor?
> 
> - Charlie (mobile)
> 
> On Aug 9, 2016 2:53 PM, "Doug Simon" <doug.simon at oracle.com> wrote:
> 
>> Hi Remi,
>> 
>> The GraalVM binaries include the JavaScript, Ruby, and R engines in
>> addition to the Graal compiler.
>> 
>> -Doug
>> 
>>> On 09 Aug 2016, at 21:03, Remi Forax <forax at univ-mlv.fr> wrote:
>>> 
>>> What the difference between the open source version of Graal and this
>> binary version ?
>>> 
>>> Remi
>>> 
>>> 
>>> 
>>> 
>>> On August 9, 2016 3:23:43 PM GMT+02:00, Christian Humer <
>> christian.humer at gmail.com> wrote:
>>>> Hi guys,
>>>> 
>>>> Just a short notice that a new version of GraalVM 0.15 is available on
>>>> OTN:
>>>> http://www.oracle.com/technetwork/oracle-labs/
>> program-languages/downloads/index.html
>>>> 
>>>> It includes the latest binary bits of Graal, its enterprise extensions
>>>> and all the languages for Mac and Linux.
>>>> All repositories were tagged with "graal-vm-0.15" to mark the exact
>>>> commit that was used for this build.
>>>> 
>>>> Have fun!
>>>> 
>>>> Cheers,
>>>> Christian Humer
>>> 
>>> --
>>> Sent from my Android device with K-9 Mail. Please excuse my brevity.
>> 
>> 


From java at stefan-marr.de  Wed Aug 10 11:17:47 2016
From: java at stefan-marr.de (Stefan Marr)
Date: Wed, 10 Aug 2016 13:17:47 +0200
Subject: Can we get the IDE for free,
	too? What do we need for full IDE Integration for Truffle Languages?
Message-ID: <9CA5D2F5-0633-4CCA-9837-A0CAE6803FF9@stefan-marr.de>

Hi:

I recently experiment with IDE support for my Truffle-based languages, and wrote up a few notes on what I see Truffle could do.
Long post below.

If you prefer the HTML version, it?s here: http://stefan-marr.de/2016/08/can-we-get-the-ide-for-free-too/

Comments welcome.

Best regards
Stefan


# Can we get the IDE for free, too?
# What do we need for full IDE Integration for Truffle Languages?

With the [Truffle language implementation framework][4], we got a powerful
foundation for implementing languages as simple interpreters. In combination
with the Graal compiler, Truffle interpreters execute their programs as [very
efficient native code][5].

Now that we got just-in-time compilation essentially "[for free][6]", _can we
get IDE integration for our Truffle languages as well?_

In case you wonder, this is inspired by the [language server protocol][3]
project of Microsoft, RedHat, Eclipse Che, and others. Their goal is to develop
a language-agnostic protocol that connects so-called _language servers_ to IDEs.
That made me wonder whether we could provide the infrastructure needed for such
a language server as part of Truffle.

In the remainder of this post, I'll briefly discuss what IDE features would be
desirable as a start, what of that Truffle currently could support, and how far
we could got with a language-agnostic API as part of Truffle.

## 1. Which IDE Features would be desirable?

Generally, I am thinking about features available in IDEs such as Eclipse,
Visual Studio, NetBeans, or Pharo. On the one hand, there are tools that help
to understand the execution of a program. Typically, this includes debugging
support, inspecting of values, but it can also be about profiling to identify
performance issues. Such execution-related aspects are covered by Truffle
already today. The framework comes with support for a debugger and a profiler.
The debugger can be used across Truffle languages for instance in [NetBeans][7]
or in [a web-based experiment of mine][8].

Features that are not strictly related to execution are however not supported.
In the research community, this area is something where [Language Workbench
projects][1] [[1]] excel. They often come with their own domain-specific
languages to define language grammars, and use transformation or compilation
approaches to generate a wide ranges of tools from such specification.

The tools I find most essential for an IDE include:
 - support for highlighting (syntactic and semantic)
 - code browsing, structural representation
 - code completion (incl. signature information, and API documentation)
 - reporting of parsing and compilation errors
 - reporting of potential bugs and code quality issues 
 - quick fix functionality
 - refactoring support

![Code Completion for SOMns in VS Code](figures/code-completion.png)

For code completion, as in the figure above, one needs of course
language-specific support, ideally taking the current context into account, and
adjusting the set of proposals to what is possible at that lexical location.

![Go to Definition for SOMns in VS Code](figures/go-to-definition.png)

Similarly, for the _go to definition_ example above, it is most useful if the
language semantics are taken into account. My prototype currently does not take
into account that the receiver of a `println` in this case is a literal, which
would allow it to reduce the set of possible definitions to the one in the
`String` class.

![Parsing Error in SOMns in VS Code](figures/parse-error.png)

Other features are more simple mappings of already available functionality. The
error message above is shown on parsing errors, and helps to understand why the
parser complains. That type of error is also shown in typical languages for
instance on the command line to enable basic programming.

![Navigating a file based on its program entities](figures/symbol-info.png)

The other features are clearly more language-specific, as is the last example
above, the browsing of a file based on the entities it defines. However, most
of these elements can be mapped onto a common set of concepts that can be
handled in a language-agnostic way in an IDE.

While the [DynSem][9] project might bring all these generation-based features
to Truffle languages, I wonder whether we can do more in a bottom-up approach
based on the interpreters we already got. [Ens?][2], a self-describing DSL
workbench, seems to go the route of interpretation over transformation as well.

## 2. What does Truffle currently support? 

As mentioned already above, Truffle currently focuses on providing a framework
geared towards tooling for language execution. This focuses mainly on
providing the implementation framework for the languages themselves, but
includes also support for [language instrumentation][10] that can be used to
implement debuggers, profilers, tools for collecting dynamic metrics, [coverage
tracking][11], [dynamic code reloading][12], etc.

The framework is based on the idea that AST nodes, i.e., the basic executable
elements to which a parser transforms an input program, can be _tagged_. An
instrumentation tool can then act based on these tags and for instance add
extra behavior to a program or track its execution. AST nodes are correlated to
their lexical representation in a program by so-called `SourceSections`. A
`SourceSection` encodes the coordinates in the program file.

Unfortunately, this is where the support from the Truffle framework ends.
Tooling aspects revolving around the lexical aspects of programs are currently
not supported.

## 3. Can we provide a language-agnostic API to implement tooling focused on lexical aspects? 

The most basic aspect that is currently missing in Truffle for any kind of
lexical support is mapping any location in a source to a corresponding semantic
entity. There are two underlying features that are currently missing for that.
First, we would need to actually retain information about code that is not part
of a method, i.e., is not part of the ASTs that are built for Truffle. Second,
we would need a simple lookup data structure from a location in the source to
the corresponding element. To implement for instance code completion, we need
to identify the code entity located at the current position in the source, as
well as its context so that we can propose sensible completion options.

Let's go over the list of things I wanted.

### 3.1 Support for Highlighting (Syntactic and Semantic)

This needs two things. First, a set of common tags to identify concepts such as
keywords, literals, or method definitions. Second, it needs an API to
communicate that information during parsing to Truffle so that it is stored
along with the `Source` information for instance, and can be used for
highlighting.

To support semantic instead of purely syntactic highlighting, Truffle would
furthermore need support for dynamic tags. Currently, Truffle assumes that tags
are not going to change after AST creation. For many dynamic languages, this is
however too early. Only executing the code will determine whether an
operation is for instance a field read or an access to a global.

### 3.2 Code Browsing, Structural Representation

Communicating structural information might be a bit more challenging in a
language-agnostic way. One could got with the choice of a superset of concepts
that is common to various languages and then provide an API that records these
information on a per-`Source` basis, which could then be queried from an IDE.

One challenge here, especially from the perspective of editing would be to
chose data structures that are easily and efficiently _evolvable/updatable_
during the editing of a source file. Assuming that the editor provides the
information about only parts of a source having changed, it should be possible
to leverage that.

Note, this also requires a parser that is flexible enough to parse such chunks.
This is however something that would be language-specific, especially since
Truffle leaves the parsing aspect completely to the languages.

### 3.3 Code Completion (incl. signature information, and API documentation)

For code completion, one needs a mapping from the source locations to the
'lexically dominating' entities. With that I mean, not necessarily the
structure of an AST, but as with the highlighting, a mapping from the source
location to the most relevant element from a user's perspective. Assuming we
got that for highlighting already, we would need language-specific _lookup_
routines to determine the relevant elements for code completion. And those
should then probably also return all the language-specific information about
signatures (name and properties of arguments, e.g., types) as well as API
documentation.

### 3.4 Reporting of Parsing and Compilation Errors

Depending on how far one wants to take that, this could be as simple as an API
to report one or more parsing or compilation exceptions.

I see two relevant aspects here that should be considered. The first is that
the current `PolyglotEngine` design of Truffle does not actually expose a
`parse()` operation. It is kept sealed off under the hood. Second, depending on
the language and the degree of convenience one would want to provide to users,
the parser might want to continue parsing after the first error and report
multiple issues in one go. This might make the parser much more complex, but for
compilation, i.e., structural or typing issues unrelated to the syntax, one
might want to report all issues, instead of aborting after the first one. Such
features might require very different engineering decisions compared to
implementations that abort on the first error, but it would improve the
programming experience dramatically.

### 3.5 Reporting of Potential Bugs and Code Quality Issues

This doesn't seem to be fundamentally different from the previous issue. The
question is whether an API for querying such information is something that
belongs into the `PolyglotEngine`, or whether there should be another entry
point for such tooling altogether. Since I have a strong dynamic languages
background, I'd argue the `PolyglotEngine` is the right place. I want to
execute code to learn more about the behavior of a program. I want to run unit
tests to get the best feedback and information (including types and semantic
highlighting) about my code. So, I'd say it belongs all in there.

### 3.6 Quick-Fix Functionality and Refactoring Support

I haven't really experimented with these aspects, but there seems to be a
language-specific and a language-agnostic component to it. The
language-specific component would be to identify the entities that need to be
changed by a quick fix or refactoring, as well as determining the replacement.
The actual operation however seems to be fairly language-independent and could
be a service provided by a common infrastructure to change `Source`
objects/files.

## 4. Conclusion

To me it seems that there is huge potential for Truffle to provide more
language-agnostic infrastructure to realize standard and perhaps non-standard
IDE features by providing additional APIs to be implemented by languages.
Getting basic features is something reasonably straight forward and would help
anyone using a language that doesn't already have IDE support traditionally.

However, there are also a couple of challenges that might be at odds with
Truffle as a framework for languages that are mostly tuned for peak
performance. In my own experience, adapting the SOMns parser to provide all the
necessary information for highlighting, code completion, go to definition,
etc., requires quite a few design decisions that depart from the
straight-forward parsing that I did before to just directly construct the AST.
On the one hand, I need to retain much more information than before. My Truffle
ASTs are very basic and contain only elements relevant for execution. For
editing in an IDE however, we want all the _declarative_ elements as well. On
the other hand, one probably wants a parser that is incremental, and perhaps
works on the chunk that the editor identified as changed. If the parser wasn't
designed from the start to work like that, this seems to require quite
pervasive changes to the parser. Similarly, one would need a different approach
to parsing to continue after a parse error was found. On top of that comes the
aspect of storing the desired information in an efficient data structure.
Perhaps that is something where persistent data structures would be handy.

While there are challenges and tradeoffs, for language implementers like me,
this would be a great thing. I'd love to experiment with my language and still
get the benefits of an IDE. Perhaps not exactly for free, but with a reasonable
effort. While Language Workbenches provide such features, I personally prefer
the bottom up approach. Instead of specifying my language, I'd rather express
the one truth of its semantics as an interpreter. In the end, I want to execute
programs. So, let's start there.

### PS

I'd like to thank [Michael L. Van De Vanter][14] for discussions on the topic.
The code for my experiments with the language server protocol and SOMns are
available on GitHub as part of the [SOMns-vscode][15] project. So far, I
haven't tried to integrate it with Truffle however.

And I have to admit, I am currently mostly focusing on the [MetaConc][16] 
project, where we aim a little higher and try to go beyond what people come to
expect when debugging concurrent applications.

### References

1. Erdweg, S.; van der Storm, T.; V?lter, M.; Boersma, M.; Bosman, R.; Cook, W. R.;
   Gerritsen, A.; Hulshout, A.; Kelly, S.; Loh, A.; Konat, G. D. P.; Molina, P. J.;
   Palatnik, M.; Pohjonen, R.; Schindler, E.; Schindler, K.; Solmi, R.; Vergu, V. A.;
   Visser, E.; van der Vlist, K.; Wachsmuth, G. H. & van der Woning, J. (2013),
   [The State of the Art in Language Workbenches][1], Springer, pp. 197-217.

[1]: http://homepages.cwi.nl/~storm/publications/lwc13paper.pdf
[2]: http://enso-lang.org/
[3]: https://github.com/Microsoft/language-server-protocol/
[4]: https://github.com/graalvm/truffle#readme
[5]: https://github.com/smarr/are-we-fast-yet/blob/master/docs/performance.md#performance-results
[6]: http://stefan-marr.de/papers/oopsla-marr-ducasse-meta-tracing-vs-partial-evaluation/
[7]: https://youtu.be/ewdzDqPsn38
[8]: https://vimeo.com/smarr/web-debugger-01
[9]: https://github.com/metaborg/dynsem
[10]: http://lafo.ssw.jku.at/javadoc/truffle/latest/com/oracle/truffle/api/instrumentation/package-summary.html
[11]: https://github.com/MetaConc/CoverallsTruffle#readme
[12]: http://2016.ecoop.org/event/icooolps-2016-trufflereloader-a-low-overhead-language-neutral-reloader
[13]: https://github.com/smarr/SOMns
[14]: http://vandevanter.net/mlvdv/
[15]: https://github.com/smarr/SOMns-vscode/tree/master/server
[16]: http://ssw.jku.at/Research/Projects/MetaConc/


-- 
Stefan Marr
Johannes Kepler Universit?t Linz
http://stefan-marr.de/research/




From christian.humer at gmail.com  Thu Aug 11 18:06:57 2016
From: christian.humer at gmail.com (Christian Humer)
Date: Thu, 11 Aug 2016 18:06:57 +0000
Subject: Truffle on-stack-replacement sometimes not working well with
	background compilation?
In-Reply-To: <CAFzGNhJwyUW+OCvRBsHov39x4voLy-0PEtupPacYttfzCA8aCg@mail.gmail.com>
References: <CAFzGNhKPQyavqPq-evLBMLqNysH=P6CDoX0bRuCfWgyQp22gww@mail.gmail.com>
	<em52c27866-a46f-45d4-b728-2ced7c386393@grashalm-work>
	<CAFzGNhJwyUW+OCvRBsHov39x4voLy-0PEtupPacYttfzCA8aCg@mail.gmail.com>
Message-ID: <em49dae66f-09f8-4a5e-8717-f753878e4a9d@grashalm-work>

Hi Gerard,

I could reproduce and fix your problem. The outer loop compilation was 
indeed blocking the inner loop compilation when background compilation 
was enabled.

Its fixed in 
https://github.com/graalvm/graal-core/commit/e46c8d69e4a12fce8a7ca6853a5e60294af262de 
if you are building your own graal-core.
If not, its going to be in the next GraalVM release 0.17 coming to OTN, 
early next month.

Cheers,
Christian


On 04.08.2016 22:13:27, "Gerard Krol" <gerard at gerardkrol.nl> wrote:

>Hi Christian,
>
>I'm using graalvm version 0.12, with truffle-api 0.15. I'm perfectly 
>happy
>with background compilation disabled at the moment, so take your time.
>
>Regards,
>
>Gerard
>
>On Thu, Aug 4, 2016 at 9:56 PM, Christian Humer 
><christian.humer at gmail.com>
>wrote:
>
>>  Hi Gerard,
>>
>>  Thanks for the report. Unfortunately I am unable to have a look at it
>>  right now, but I will asap.
>>  Can you, in the meantime, provide me with some info on which version 
>>of
>>  graalvm/truffle you are using so I can try to reproduce?
>>
>>  Thanks,
>>  Christian
>>
>>  On 04.08.2016 21:46:17, "Gerard Krol" <gerard at gerardkrol.nl> wrote:
>>
>>  Hi,
>>>
>>>  I've had some performance problems with Cover[1] and a specific 
>>>benchmark.
>>>  Yesterday I finally figured out what was going on. The problem seems 
>>>to be
>>>  that on-stack-replacement doesn't always work well with background
>>>  compilation.
>>>
>>>  The benchmark (fannkuch, [2])  did take 311 seconds to run, while 
>>>the GCC
>>>  version would finish in 25 seconds. Warming up (running the 
>>>benchmark
>>>  twice
>>>  and timing the second run) would reduce the runtime to 47 seconds. I 
>>>like
>>>  to run my benchmarks without warmup though, as that is a good 
>>>worst-case
>>>  scenario.
>>>
>>>  I then figured out that just adding
>>>  -Dgraal.TruffleBackgroundCompilation=false made a huge difference. 
>>>The
>>>  time
>>>  needed was reduced to 52 seconds.
>>>
>>>  Is this a bug? Expected behavior in some cases? Am I doing something
>>>  wrong?
>>>
>>>  Reproduction:
>>>  1. Get Cover from [1] (don't forget the 5th installation step)
>>>  2. Run time ./cover tests/fannkuch-redux_gcc5.cover
>>>  3. Run time ./cover -benchmark tests/fannkuch-redux_gcc5.cover (this 
>>>will
>>>  disable the background compilation)
>>>
>>>  Regards,
>>>
>>>  Gerard
>>>
>>>  [1] https://github.com/gerard-/cover
>>>  [2]
>>>  https://github.com/gerard-/cover/blob/master/tests/fannkuch-
>>>  redux_gcc5.cover
>>>
>>
>>


From gerard at gerardkrol.nl  Thu Aug 11 20:28:26 2016
From: gerard at gerardkrol.nl (Gerard Krol)
Date: Thu, 11 Aug 2016 22:28:26 +0200
Subject: Truffle on-stack-replacement sometimes not working well with
	background compilation?
In-Reply-To: <em49dae66f-09f8-4a5e-8717-f753878e4a9d@grashalm-work>
References: <CAFzGNhKPQyavqPq-evLBMLqNysH=P6CDoX0bRuCfWgyQp22gww@mail.gmail.com>
	<em52c27866-a46f-45d4-b728-2ced7c386393@grashalm-work>
	<CAFzGNhJwyUW+OCvRBsHov39x4voLy-0PEtupPacYttfzCA8aCg@mail.gmail.com>
	<em49dae66f-09f8-4a5e-8717-f753878e4a9d@grashalm-work>
Message-ID: <CAFzGNhKy1oo3T9pr06GGxhtbQwKFLf2Xn76vC9fBUMjvymcvjg@mail.gmail.com>

Hi Christian,

Great, thanks!

Regards,

Gerard

On Thu, Aug 11, 2016 at 8:06 PM, Christian Humer <christian.humer at gmail.com>
wrote:

> Hi Gerard,
>
> I could reproduce and fix your problem. The outer loop compilation was
> indeed blocking the inner loop compilation when background compilation was
> enabled.
>
> Its fixed in https://github.com/graalvm/graal-core/commit/e46c8d69e4a12fc
> e8a7ca6853a5e60294af262de if you are building your own graal-core.
> If not, its going to be in the next GraalVM release 0.17 coming to OTN,
> early next month.
>
> Cheers,
> Christian
>
>
> On 04.08.2016 22:13:27, "Gerard Krol" <gerard at gerardkrol.nl> wrote:
>
> Hi Christian,
>>
>> I'm using graalvm version 0.12, with truffle-api 0.15. I'm perfectly happy
>> with background compilation disabled at the moment, so take your time.
>>
>> Regards,
>>
>> Gerard
>>
>> On Thu, Aug 4, 2016 at 9:56 PM, Christian Humer <
>> christian.humer at gmail.com>
>> wrote:
>>
>>  Hi Gerard,
>>>
>>>  Thanks for the report. Unfortunately I am unable to have a look at it
>>>  right now, but I will asap.
>>>  Can you, in the meantime, provide me with some info on which version of
>>>  graalvm/truffle you are using so I can try to reproduce?
>>>
>>>  Thanks,
>>>  Christian
>>>
>>>  On 04.08.2016 21:46:17, "Gerard Krol" <gerard at gerardkrol.nl> wrote:
>>>
>>>  Hi,
>>>
>>>>
>>>>  I've had some performance problems with Cover[1] and a specific
>>>> benchmark.
>>>>  Yesterday I finally figured out what was going on. The problem seems
>>>> to be
>>>>  that on-stack-replacement doesn't always work well with background
>>>>  compilation.
>>>>
>>>>  The benchmark (fannkuch, [2])  did take 311 seconds to run, while the
>>>> GCC
>>>>  version would finish in 25 seconds. Warming up (running the benchmark
>>>>  twice
>>>>  and timing the second run) would reduce the runtime to 47 seconds. I
>>>> like
>>>>  to run my benchmarks without warmup though, as that is a good
>>>> worst-case
>>>>  scenario.
>>>>
>>>>  I then figured out that just adding
>>>>  -Dgraal.TruffleBackgroundCompilation=false made a huge difference. The
>>>>  time
>>>>  needed was reduced to 52 seconds.
>>>>
>>>>  Is this a bug? Expected behavior in some cases? Am I doing something
>>>>  wrong?
>>>>
>>>>  Reproduction:
>>>>  1. Get Cover from [1] (don't forget the 5th installation step)
>>>>  2. Run time ./cover tests/fannkuch-redux_gcc5.cover
>>>>  3. Run time ./cover -benchmark tests/fannkuch-redux_gcc5.cover (this
>>>> will
>>>>  disable the background compilation)
>>>>
>>>>  Regards,
>>>>
>>>>  Gerard
>>>>
>>>>  [1] https://github.com/gerard-/cover
>>>>  [2]
>>>>  https://github.com/gerard-/cover/blob/master/tests/fannkuch-
>>>>  redux_gcc5.cover
>>>>
>>>>
>>>
>>>
>

From roland.schatz at oracle.com  Tue Aug 16 13:32:34 2016
From: roland.schatz at oracle.com (Roland Schatz)
Date: Tue, 16 Aug 2016 15:32:34 +0200
Subject: new JVMCI binaries
Message-ID: <f2cced68-986f-1dcb-06f5-13ca70b1e8a6@oracle.com>

Hi,

We have released version 0.19 of the JVMCI JDK 8 binaries. You can 
download the Oracle JDK build from OTN [1].
OpenJDK builds of the new version can be downloaded from github [2].

The new version contains backports of changes from JDK-9, including a 
minor API change.

Starting from graal-core revision 
4c735d3ab78edda879c4169d7af5eec04e40e2a5, the minimum required JVMCI 
version is 0.19.

- Roland

[1] 
http://www.oracle.com/technetwork/oracle-labs/program-languages/downloads/index.htm
[2] https://github.com/dougxc/openjdk8-jvmci-builder/releases


From jaroslav.tulach at oracle.com  Wed Aug 17 11:02:16 2016
From: jaroslav.tulach at oracle.com (Jaroslav Tulach)
Date: Wed, 17 Aug 2016 13:02:16 +0200
Subject: Can we get the IDE for free,
	too? What do we need for full IDE Integration for Truffle Languages?
In-Reply-To: <9CA5D2F5-0633-4CCA-9837-A0CAE6803FF9@stefan-marr.de>
References: <9CA5D2F5-0633-4CCA-9837-A0CAE6803FF9@stefan-marr.de>
Message-ID: <7294827.ddKJsvEOjJ@pracovni>

On st?eda 10. srpna 2016 13:17:47 CEST Stefan Marr wrote:
> Hi:
> 
> I recently experiment with IDE support for my Truffle-based languages, and
> wrote up a few notes on what I see Truffle could do. Long post below.
> 
> If you prefer the HTML version, it?s here:
> http://stefan-marr.de/2016/08/can-we-get-the-ide-for-free-too/
> 
> Comments welcome.

Nice. What are the next steps? Here are those that I have in mind:

== Implement Language Server Protocol ==

Having a trivial implementation of https://github.com/Microsoft/language-server-protocol/ in graal would be natural start, right? It would give IDEs 
something to connect to and display. I know you can get some structural info 
about SOM programs that could be used for code completion. Having that working 
could inspire other people to join.

== Generate Java API for the Language Server Protocol ==

I could probably generate Java API bindings for the protocol, so we don't have 
to write that by hand and update everytime the protocol evolves. There is 
[TypeScript to Java transpiler](https://dukescript.com/update/2016/07/01/
transcript-to-java.html) and this could be a nice test.

== Use Language Server Protocol in NetBeans ==

I know guys in NetBeans who work on the editor infrastructure and could call 
the protocol and display the necessary information in the IDE.

-jt


> Best regards
> Stefan
> 
> 
> # Can we get the IDE for free, too?
> # What do we need for full IDE Integration for Truffle Languages?
> 
> With the [Truffle language implementation framework][4], we got a powerful
> foundation for implementing languages as simple interpreters. In combination
> with the Graal compiler, Truffle interpreters execute their programs as
> [very efficient native code][5].
> 
> Now that we got just-in-time compilation essentially "[for free][6]", _can
> we get IDE integration for our Truffle languages as well?_
> 
> In case you wonder, this is inspired by the [language server protocol][3]
> project of Microsoft, RedHat, Eclipse Che, and others. Their goal is to
> develop a language-agnostic protocol that connects so-called _language
> servers_ to IDEs. That made me wonder whether we could provide the
> infrastructure needed for such a language server as part of Truffle.
> 
> In the remainder of this post, I'll briefly discuss what IDE features would
> be desirable as a start, what of that Truffle currently could support, and
> how far we could got with a language-agnostic API as part of Truffle.
> 
> ## 1. Which IDE Features would be desirable?
> 
> Generally, I am thinking about features available in IDEs such as Eclipse,
> Visual Studio, NetBeans, or Pharo. On the one hand, there are tools that
> help to understand the execution of a program. Typically, this includes
> debugging support, inspecting of values, but it can also be about profiling
> to identify performance issues. Such execution-related aspects are covered
> by Truffle already today. The framework comes with support for a debugger
> and a profiler. The debugger can be used across Truffle languages for
> instance in [NetBeans][7] or in [a web-based experiment of mine][8].
> 
> Features that are not strictly related to execution are however not
> supported. In the research community, this area is something where
> [Language Workbench projects][1] [[1]] excel. They often come with their
> own domain-specific languages to define language grammars, and use
> transformation or compilation approaches to generate a wide ranges of tools
> from such specification.
> 
> The tools I find most essential for an IDE include:
>  - support for highlighting (syntactic and semantic)
>  - code browsing, structural representation
>  - code completion (incl. signature information, and API documentation)
>  - reporting of parsing and compilation errors
>  - reporting of potential bugs and code quality issues
>  - quick fix functionality
>  - refactoring support
> 
> ![Code Completion for SOMns in VS Code](figures/code-completion.png)
> 
> For code completion, as in the figure above, one needs of course
> language-specific support, ideally taking the current context into account,
> and adjusting the set of proposals to what is possible at that lexical
> location.
> 
> ![Go to Definition for SOMns in VS Code](figures/go-to-definition.png)
> 
> Similarly, for the _go to definition_ example above, it is most useful if
> the language semantics are taken into account. My prototype currently does
> not take into account that the receiver of a `println` in this case is a
> literal, which would allow it to reduce the set of possible definitions to
> the one in the `String` class.
> 
> ![Parsing Error in SOMns in VS Code](figures/parse-error.png)
> 
> Other features are more simple mappings of already available functionality.
> The error message above is shown on parsing errors, and helps to understand
> why the parser complains. That type of error is also shown in typical
> languages for instance on the command line to enable basic programming.
> 
> ![Navigating a file based on its program entities](figures/symbol-info.png)
> 
> The other features are clearly more language-specific, as is the last
> example above, the browsing of a file based on the entities it defines.
> However, most of these elements can be mapped onto a common set of concepts
> that can be handled in a language-agnostic way in an IDE.
> 
> While the [DynSem][9] project might bring all these generation-based
> features to Truffle languages, I wonder whether we can do more in a
> bottom-up approach based on the interpreters we already got. [Ens?][2], a
> self-describing DSL workbench, seems to go the route of interpretation over
> transformation as well.
> 
> ## 2. What does Truffle currently support?
> 
> As mentioned already above, Truffle currently focuses on providing a
> framework geared towards tooling for language execution. This focuses
> mainly on providing the implementation framework for the languages
> themselves, but includes also support for [language instrumentation][10]
> that can be used to implement debuggers, profilers, tools for collecting
> dynamic metrics, [coverage tracking][11], [dynamic code reloading][12],
> etc.
> 
> The framework is based on the idea that AST nodes, i.e., the basic
> executable elements to which a parser transforms an input program, can be
> _tagged_. An instrumentation tool can then act based on these tags and for
> instance add extra behavior to a program or track its execution. AST nodes
> are correlated to their lexical representation in a program by so-called
> `SourceSections`. A `SourceSection` encodes the coordinates in the program
> file.
> 
> Unfortunately, this is where the support from the Truffle framework ends.
> Tooling aspects revolving around the lexical aspects of programs are
> currently not supported.
> 
> ## 3. Can we provide a language-agnostic API to implement tooling focused on
> lexical aspects?
> 
> The most basic aspect that is currently missing in Truffle for any kind of
> lexical support is mapping any location in a source to a corresponding
> semantic entity. There are two underlying features that are currently
> missing for that. First, we would need to actually retain information about
> code that is not part of a method, i.e., is not part of the ASTs that are
> built for Truffle. Second, we would need a simple lookup data structure
> from a location in the source to the corresponding element. To implement
> for instance code completion, we need to identify the code entity located
> at the current position in the source, as well as its context so that we
> can propose sensible completion options.
> 
> Let's go over the list of things I wanted.
> 
> ### 3.1 Support for Highlighting (Syntactic and Semantic)
> 
> This needs two things. First, a set of common tags to identify concepts such
> as keywords, literals, or method definitions. Second, it needs an API to
> communicate that information during parsing to Truffle so that it is stored
> along with the `Source` information for instance, and can be used for
> highlighting.
> 
> To support semantic instead of purely syntactic highlighting, Truffle would
> furthermore need support for dynamic tags. Currently, Truffle assumes that
> tags are not going to change after AST creation. For many dynamic
> languages, this is however too early. Only executing the code will
> determine whether an operation is for instance a field read or an access to
> a global.
> 
> ### 3.2 Code Browsing, Structural Representation
> 
> Communicating structural information might be a bit more challenging in a
> language-agnostic way. One could got with the choice of a superset of
> concepts that is common to various languages and then provide an API that
> records these information on a per-`Source` basis, which could then be
> queried from an IDE.
> 
> One challenge here, especially from the perspective of editing would be to
> chose data structures that are easily and efficiently _evolvable/updatable_
> during the editing of a source file. Assuming that the editor provides the
> information about only parts of a source having changed, it should be
> possible to leverage that.
> 
> Note, this also requires a parser that is flexible enough to parse such
> chunks. This is however something that would be language-specific,
> especially since Truffle leaves the parsing aspect completely to the
> languages.
> 
> ### 3.3 Code Completion (incl. signature information, and API documentation)
> 
> For code completion, one needs a mapping from the source locations to the
> 'lexically dominating' entities. With that I mean, not necessarily the
> structure of an AST, but as with the highlighting, a mapping from the source
> location to the most relevant element from a user's perspective. Assuming
> we got that for highlighting already, we would need language-specific
> _lookup_ routines to determine the relevant elements for code completion.
> And those should then probably also return all the language-specific
> information about signatures (name and properties of arguments, e.g.,
> types) as well as API documentation.
> 
> ### 3.4 Reporting of Parsing and Compilation Errors
> 
> Depending on how far one wants to take that, this could be as simple as an
> API to report one or more parsing or compilation exceptions.
> 
> I see two relevant aspects here that should be considered. The first is that
> the current `PolyglotEngine` design of Truffle does not actually expose a
> `parse()` operation. It is kept sealed off under the hood. Second,
> depending on the language and the degree of convenience one would want to
> provide to users, the parser might want to continue parsing after the first
> error and report multiple issues in one go. This might make the parser much
> more complex, but for compilation, i.e., structural or typing issues
> unrelated to the syntax, one might want to report all issues, instead of
> aborting after the first one. Such features might require very different
> engineering decisions compared to implementations that abort on the first
> error, but it would improve the programming experience dramatically.
> 
> ### 3.5 Reporting of Potential Bugs and Code Quality Issues
> 
> This doesn't seem to be fundamentally different from the previous issue. The
> question is whether an API for querying such information is something that
> belongs into the `PolyglotEngine`, or whether there should be another entry
> point for such tooling altogether. Since I have a strong dynamic languages
> background, I'd argue the `PolyglotEngine` is the right place. I want to
> execute code to learn more about the behavior of a program. I want to run
> unit tests to get the best feedback and information (including types and
> semantic highlighting) about my code. So, I'd say it belongs all in there.
> 
> ### 3.6 Quick-Fix Functionality and Refactoring Support
> 
> I haven't really experimented with these aspects, but there seems to be a
> language-specific and a language-agnostic component to it. The
> language-specific component would be to identify the entities that need to
> be changed by a quick fix or refactoring, as well as determining the
> replacement. The actual operation however seems to be fairly
> language-independent and could be a service provided by a common
> infrastructure to change `Source` objects/files.
> 
> ## 4. Conclusion
> 
> To me it seems that there is huge potential for Truffle to provide more
> language-agnostic infrastructure to realize standard and perhaps
> non-standard IDE features by providing additional APIs to be implemented by
> languages. Getting basic features is something reasonably straight forward
> and would help anyone using a language that doesn't already have IDE
> support traditionally.
> 
> However, there are also a couple of challenges that might be at odds with
> Truffle as a framework for languages that are mostly tuned for peak
> performance. In my own experience, adapting the SOMns parser to provide all
> the necessary information for highlighting, code completion, go to
> definition, etc., requires quite a few design decisions that depart from
> the
> straight-forward parsing that I did before to just directly construct the
> AST. On the one hand, I need to retain much more information than before.
> My Truffle ASTs are very basic and contain only elements relevant for
> execution. For editing in an IDE however, we want all the _declarative_
> elements as well. On the other hand, one probably wants a parser that is
> incremental, and perhaps works on the chunk that the editor identified as
> changed. If the parser wasn't designed from the start to work like that,
> this seems to require quite pervasive changes to the parser. Similarly, one
> would need a different approach to parsing to continue after a parse error
> was found. On top of that comes the aspect of storing the desired
> information in an efficient data structure. Perhaps that is something where
> persistent data structures would be handy.
> 
> While there are challenges and tradeoffs, for language implementers like me,
> this would be a great thing. I'd love to experiment with my language and
> still get the benefits of an IDE. Perhaps not exactly for free, but with a
> reasonable effort. While Language Workbenches provide such features, I
> personally prefer the bottom up approach. Instead of specifying my
> language, I'd rather express the one truth of its semantics as an
> interpreter. In the end, I want to execute programs. So, let's start there.
> 
> ### PS
> 
> I'd like to thank [Michael L. Van De Vanter][14] for discussions on the
> topic. The code for my experiments with the language server protocol and
> SOMns are available on GitHub as part of the [SOMns-vscode][15] project. So
> far, I haven't tried to integrate it with Truffle however.
> 
> And I have to admit, I am currently mostly focusing on the [MetaConc][16]
> project, where we aim a little higher and try to go beyond what people come
> to expect when debugging concurrent applications.
> 
> ### References
> 
> 1. Erdweg, S.; van der Storm, T.; V?lter, M.; Boersma, M.; Bosman, R.; Cook,
> W. R.; Gerritsen, A.; Hulshout, A.; Kelly, S.; Loh, A.; Konat, G. D. P.;
> Molina, P. J.; Palatnik, M.; Pohjonen, R.; Schindler, E.; Schindler, K.;
> Solmi, R.; Vergu, V. A.; Visser, E.; van der Vlist, K.; Wachsmuth, G. H. &
> van der Woning, J. (2013), [The State of the Art in Language
> Workbenches][1], Springer, pp. 197-217.
> 
> [1]: http://homepages.cwi.nl/~storm/publications/lwc13paper.pdf
> [2]: http://enso-lang.org/
> [3]: https://github.com/Microsoft/language-server-protocol/
> [4]: https://github.com/graalvm/truffle#readme
> [5]:
> https://github.com/smarr/are-we-fast-yet/blob/master/docs/performance.md#pe
> rformance-results [6]:
> http://stefan-marr.de/papers/oopsla-marr-ducasse-meta-tracing-vs-partial-ev
> aluation/ [7]: https://youtu.be/ewdzDqPsn38
> [8]: https://vimeo.com/smarr/web-debugger-01
> [9]: https://github.com/metaborg/dynsem
> [10]:
> http://lafo.ssw.jku.at/javadoc/truffle/latest/com/oracle/truffle/api/instru
> mentation/package-summary.html [11]:
> https://github.com/MetaConc/CoverallsTruffle#readme
> [12]:
> http://2016.ecoop.org/event/icooolps-2016-trufflereloader-a-low-overhead-la
> nguage-neutral-reloader [13]: https://github.com/smarr/SOMns
> [14]: http://vandevanter.net/mlvdv/
> [15]: https://github.com/smarr/SOMns-vscode/tree/master/server
> [16]: http://ssw.jku.at/Research/Projects/MetaConc/



From cthalinger at twitter.com  Wed Aug 17 19:01:02 2016
From: cthalinger at twitter.com (Christian Thalinger)
Date: Wed, 17 Aug 2016 09:01:02 -1000
Subject: GraalVM 0.15 Available on OTN
In-Reply-To: <D43DC97E-2473-466C-8235-0495ADA53258@oracle.com>
References: <emf88fee36-c77f-45a2-884c-e4dbea5b91a9@grashalm-work>
	<9E5A8F76-437F-45C0-B8C3-D35DD69FF01C@univ-mlv.fr>
	<5DAA8DAB-0FD5-404C-9362-4BB50C8BFAEB@oracle.com>
	<CAE-f1xQQ1aPABL79tJLaYLpr8+ARkkf9QoqTpH69oA+yn3ghMw@mail.gmail.com>
	<D43DC97E-2473-466C-8235-0495ADA53258@oracle.com>
Message-ID: <EA778FDC-A87C-49EB-84C3-E8F547A88D1D@twitter.com>


> On Aug 9, 2016, at 11:13 PM, Thomas Wuerthinger <thomas.wuerthinger at oracle.com> wrote:
> 
> There are additional features in the OTN download relevant for enterprise grade deployment.

Can you name these ?additional features? so everyone knows what we could get?

> The speed-up of JRuby+Truffle over stock JRuby of 2-10x is however already obtained by a fully available open source stack.
> 
> - thomas
> 
>> On 10 Aug 2016, at 01:52, Charles Oliver Nutter <headius at headius.com> wrote:
>> 
>> I thought I'd heard there's optimizations in the closed source Graal that
>> aren't in the OSS repo. Was that just a vicious rumor?
>> 
>> - Charlie (mobile)
>> 
>> On Aug 9, 2016 2:53 PM, "Doug Simon" <doug.simon at oracle.com> wrote:
>> 
>>> Hi Remi,
>>> 
>>> The GraalVM binaries include the JavaScript, Ruby, and R engines in
>>> addition to the Graal compiler.
>>> 
>>> -Doug
>>> 
>>>> On 09 Aug 2016, at 21:03, Remi Forax <forax at univ-mlv.fr> wrote:
>>>> 
>>>> What the difference between the open source version of Graal and this
>>> binary version ?
>>>> 
>>>> Remi
>>>> 
>>>> 
>>>> 
>>>> 
>>>> On August 9, 2016 3:23:43 PM GMT+02:00, Christian Humer <
>>> christian.humer at gmail.com> wrote:
>>>>> Hi guys,
>>>>> 
>>>>> Just a short notice that a new version of GraalVM 0.15 is available on
>>>>> OTN:
>>>>> http://www.oracle.com/technetwork/oracle-labs/
>>> program-languages/downloads/index.html
>>>>> 
>>>>> It includes the latest binary bits of Graal, its enterprise extensions
>>>>> and all the languages for Mac and Linux.
>>>>> All repositories were tagged with "graal-vm-0.15" to mark the exact
>>>>> commit that was used for this build.
>>>>> 
>>>>> Have fun!
>>>>> 
>>>>> Cheers,
>>>>> Christian Humer
>>>> 
>>>> --
>>>> Sent from my Android device with K-9 Mail. Please excuse my brevity.
>>> 
>>> 
> 


From java at stefan-marr.de  Thu Aug 18 22:27:47 2016
From: java at stefan-marr.de (Stefan Marr)
Date: Fri, 19 Aug 2016 00:27:47 +0200
Subject: Can we get the IDE for free,
	too? What do we need for full IDE Integration for Truffle Languages?
In-Reply-To: <7294827.ddKJsvEOjJ@pracovni>
References: <9CA5D2F5-0633-4CCA-9837-A0CAE6803FF9@stefan-marr.de>
	<7294827.ddKJsvEOjJ@pracovni>
Message-ID: <3D72BF50-3A9C-44CA-A4EA-A0651274D88C@stefan-marr.de>

Hi Jaroslav:

> On 17 Aug 2016, at 13:02, Jaroslav Tulach <jaroslav.tulach at oracle.com> wrote:
> 
> Nice. What are the next steps? Here are those that I have in mind:
> 
> == Implement Language Server Protocol ==
> 
> Having a trivial implementation of https://github.com/Microsoft/language-server-protocol/ in graal would be natural start, right?

Well, I guess one would need to decide what the minimal set of functionality would be that is useful.
And, then I?d say one would need the Truffle support for the non-execution related information.

> It would give IDEs 
> something to connect to and display. I know you can get some structural info 
> about SOM programs that could be used for code completion. Having that working 
> could inspire other people to join.

Yeah, well, so the thing is that one would need to find language-agnostic notions that make sense and perhaps define some general adapters that can be used to access the language-specific information/data structures, for instance for class structure, methods, etc.


> == Generate Java API for the Language Server Protocol ==
> 
> I could probably generate Java API bindings for the protocol, so we don't have 
> to write that by hand and update everytime the protocol evolves. There is 
> [TypeScript to Java transpiler](https://dukescript.com/update/2016/07/01/
> transcript-to-java.html) and this could be a nice test.

There?s also ls-api, which I have been using: https://github.com/TypeFox/ls-api
But it?s somewhat heavy on library dependencies.

> == Use Language Server Protocol in NetBeans ==
> 
> I know guys in NetBeans who work on the editor infrastructure and could call 
> the protocol and display the necessary information in the IDE.

There?s also the start for Eclipse support: https://github.com/eclipselabs/eclipse-language-service ;)

Best regards
Stefan

-- 
Stefan Marr
Johannes Kepler Universit?t Linz
http://stefan-marr.de/research/




From andreas.woess at oracle.com  Fri Aug 19 11:06:17 2016
From: andreas.woess at oracle.com (Andreas Woess)
Date: Fri, 19 Aug 2016 13:06:17 +0200
Subject: Updating truffle import in graal-core
Message-ID: <57B6E829.4030706@oracle.com>

Hi,
I'm going to update graal-core to the latest truffle version (42f2bdc).
Please let me know asap if I should hold off on this for some reason.
Notable changes in truffle are:

* Deprecate support for the "identifier" associated with each SourceSection
* Remove deprecated instrumentation API package
com.oracle.truffle.api.instrument and all its classes.
* Remove deprecated API method TruffleLanguage#isInstrumentable(Node),
TruffleLanguage#getVisualizer(), TruffleLanguage#createWrapperNode(),
TruffleLanguage.Env#instrumenter(), RootNode#applyInstrumentation()
* Remove deprecated API Debugger#setTagBreakpoint
* Remove deprecated API RootNode#applyInstrumentation
* Remove deprecated tagging API in SourceSection and Source.
* new TCK tests

- andreas

From doug.simon at oracle.com  Thu Aug 25 21:43:47 2016
From: doug.simon at oracle.com (Doug Simon)
Date: Thu, 25 Aug 2016 23:43:47 +0200
Subject: new JVMCI binaries (0.20)
Message-ID: <9153F74B-79F4-4611-B6BB-E05475028FCB@oracle.com>

Hi,

We have released version 0.20 of the JVMCI JDK 8 binaries. You can download the Oracle JDK build from OTN [1].
OpenJDK builds of the new version can be downloaded from github [2].

New API has been added to JVMCI to allow Graal to query the set of intrinsics supported by HotSpot [3].

Starting from graal-core revision 61351aa5409e67bb25fe859c5dbab5d67f619f83, the minimum required JVMCI version is 0.20.

-Doug

[1] http://www.oracle.com/technetwork/oracle-labs/program-languages/downloads/index.htm
[2] https://github.com/dougxc/openjdk8-jvmci-builder/releases/tag/jvmci-0.20
[3] https://bugs.openjdk.java.net/browse/JDK-8164358