From mark.reinhold at oracle.com  Thu Oct 13 17:56:32 2022
From: mark.reinhold at oracle.com (Mark Reinhold)
Date: Thu, 13 Oct 2022 17:56:32 +0000
Subject: Selectively Shifting and Constraining Computation
Message-ID: <20221013105630.787752083@eggemoggin.niobe.net>

For a few months now I?ve been pondering, and brainstorming with a few
of you, how best to frame the work of Project Leyden conceptually.
Here?s my proposal:

  https://openjdk.org/projects/leyden/notes/02-shift-and-constrain

Short summary:

  The goal of Project Leyden is to improve the startup time, time to
  peak performance, and footprint of Java programs.  In this note we
  propose to work toward that goal by extending the Java programming
  model with features for _selectively shifting and constraining
  computation_ by means of _condensing code_.  We also propose an
  initial research and development roadmap.

Comments, questions, and suggestions welcome!

- Mark

From forax at univ-mlv.fr  Thu Oct 13 18:43:29 2022
From: forax at univ-mlv.fr (Remi Forax)
Date: Thu, 13 Oct 2022 20:43:29 +0200 (CEST)
Subject: Selectively Shifting and Constraining Computation
In-Reply-To: <20221013105630.787752083@eggemoggin.niobe.net>
References: <20221013105630.787752083@eggemoggin.niobe.net>
Message-ID: <1690520167.25120630.1665686609004.JavaMail.zimbra@u-pem.fr>

----- Original Message -----
> From: "mark reinhold" <mark.reinhold at oracle.com>
> To: leyden-dev at openjdk.org
> Sent: Thursday, October 13, 2022 7:56:32 PM
> Subject: Selectively Shifting and Constraining Computation

> For a few months now I?ve been pondering, and brainstorming with a few
> of you, how best to frame the work of Project Leyden conceptually.
> Here?s my proposal:
> 
>  https://openjdk.org/projects/leyden/notes/02-shift-and-constrain
> 
> Short summary:
> 
>  The goal of Project Leyden is to improve the startup time, time to
>  peak performance, and footprint of Java programs.  In this note we
>  propose to work toward that goal by extending the Java programming
>  model with features for _selectively shifting and constraining
>  computation_ by means of _condensing code_.  We also propose an
>  initial research and development roadmap.
> 
> Comments, questions, and suggestions welcome!

I will argue that moving from compile time to runtime is also a form of condensation, by example transforming an invokevirtual call to an invokedynamic allows to consider some arguments as effectively constants helping the JIT to generate better assembly code (often more compact) by delaying the linking until the actual values are seen. Here is an example of library implementing such kind of runtime macros [1].

> 
> - Mark

R?mi

[1] https://github.com/forax/macro

From mark.reinhold at oracle.com  Thu Oct 13 19:04:25 2022
From: mark.reinhold at oracle.com (Mark Reinhold)
Date: Thu, 13 Oct 2022 19:04:25 +0000
Subject: Selectively Shifting and Constraining Computation
In-Reply-To: <1690520167.25120630.1665686609004.JavaMail.zimbra@u-pem.fr>
References: <20221013105630.787752083@eggemoggin.niobe.net>
 <1690520167.25120630.1665686609004.JavaMail.zimbra@u-pem.fr>
Message-ID: <20221013120424.298245311@eggemoggin.niobe.net>

2022/10/13 11:43:29 -0700, Remi Forax <forax at univ-mlv.fr>:
> I will argue that moving from compile time to runtime is also a form
> of condensation, by example transforming an invokevirtual call to an
> invokedynamic allows to consider some arguments as effectively
> constants helping the JIT to generate better assembly code (often more
> compact) by delaying the linking until the actual values are
> seen.

That?s certainly a form of shifting, and we could generalize the
condenser concept slightly to allow such forward shifting should
interesting use cases arise.

- Mark

From john.r.rose at oracle.com  Tue Oct 18 17:58:55 2022
From: john.r.rose at oracle.com (John Rose)
Date: Tue, 18 Oct 2022 17:58:55 +0000
Subject: Selectively Shifting and Constraining Computation
In-Reply-To: <20221013120424.298245311@eggemoggin.niobe.net>
References: <20221013105630.787752083@eggemoggin.niobe.net>
 <1690520167.25120630.1665686609004.JavaMail.zimbra@u-pem.fr>
 <20221013120424.298245311@eggemoggin.niobe.net>
Message-ID: <78C8DA0A-FB04-46F2-9512-255F31B775EB@oracle.com>

Replacing a vanilla invoke by something more specific strikes me as an example of adding a runtime constraint. The call site is constrained to process only particular argument types or values, depending on what the indy turns into.  It could be open-coded as well, but that gives the JIT less info.  

The JIT makes good use of all such constraints. More generally the JIT benefits from type restrictions and constant replacement everywhere. 

> On Oct 13, 2022, at 12:04 PM, Mark Reinhold <mark.reinhold at oracle.com> wrote:
> 
> ?2022/10/13 11:43:29 -0700, Remi Forax <forax at univ-mlv.fr>:
>> I will argue that moving from compile time to runtime is also a form
>> of condensation, by example transforming an invokevirtual call to an
>> invokedynamic allows to consider some arguments as effectively
>> constants helping the JIT to generate better assembly code (often more
>> compact) by delaying the linking until the actual values are
>> seen.
> 
> That?s certainly a form of shifting, and we could generalize the
> condenser concept slightly to allow such forward shifting should
> interesting use cases arise.
> 
> - Mark

From john.r.rose at oracle.com  Tue Oct 18 18:18:08 2022
From: john.r.rose at oracle.com (John Rose)
Date: Tue, 18 Oct 2022 11:18:08 -0700
Subject: Selectively Shifting and Constraining Computation
In-Reply-To: <78C8DA0A-FB04-46F2-9512-255F31B775EB@oracle.com>
References: <20221013105630.787752083@eggemoggin.niobe.net>
 <1690520167.25120630.1665686609004.JavaMail.zimbra@u-pem.fr>
 <20221013120424.298245311@eggemoggin.niobe.net>
 <78C8DA0A-FB04-46F2-9512-255F31B775EB@oracle.com>
Message-ID: <15CF9BD7-D9F3-44C8-9BA8-E1B07AD5217F@oracle.com>

And backing out of those particular points raised by Remi, I?d like to 
say that this was a good read; thanks Mark.

Time-shifting is IMO a deeply fruitful notion, and will probably carry 
beyond the specific use cases of adjusting class loading order and 
static-final constant values.  I hope we can find a few tiny spots in 
the language and VM specs where we can remove unimportant restrictions 
that will make various forms of time-shifting (forward and backward, 
global and local) semantically neutral, with respect to specified 
semantics.  To do that we will have to make some parts of the 
specification more lenient towards reordering of expressions that might 
have side effects, and/or some way to constrain side effects.  But we 
must do so with exquisite care, and (often) with opt-in from the user.  
That?s how lazy statics are conceived currently:  They can be 
reordered because they are uncoupled from their class?s `<clinit>`, 
and the uncoupling must be explicitly permitted by the user.  It?s a 
very interesting problem, and a very worth-while one.

I like the way you link time-shifting to the introduction of 
constraints.  I already mentioned how the idea of constraints might tie 
to call sites.  Maybe also data structure fields; who knows.  I?m not 
in a hurry to figure it all out; it feels like we are at the start of 
some years of excellent investigation.

? John
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <https://mail.openjdk.org/pipermail/leyden-dev/attachments/20221018/c8e06f47/attachment.htm>

From heidinga at redhat.com  Sat Oct 22 02:49:36 2022
From: heidinga at redhat.com (Dan Heidinga)
Date: Fri, 21 Oct 2022 22:49:36 -0400
Subject: Selectively Shifting and Constraining Computation
In-Reply-To: <20221013105630.787752083@eggemoggin.niobe.net>
References: <20221013105630.787752083@eggemoggin.niobe.net>
Message-ID: <CAJq4Gi7GF2k-Bkx8Q3L1_v5J6mEzfBX0a3BZ8iXmePBssPj79w@mail.gmail.com>

Hi Mark,

Thanks for posting this document.  It provides a clear starting point for
us to move forward to start to address Leyden.

A few questions based on my read through:

> As long as a condenser preserves program meaning and does not impose
constraints
> other than those accepted by the developer, an implementation has wide
latitude to
> optimize the result.

Does ?an implementation? refer only to the JDK or are other parts of the
ecosystem - frameworks, libraries, etc - able to add their own condensers
as well?

> If a developer chooses condensers that shift sufficient computation from
run time to
> earlier phases ? which may require accepting many constraints ? then a
conforming
> Java implementation could even produce a fully-static platform-specific
executable.

Great to read this!  Especially in light of the recent announcement around
GraalCE moving under the OpenJDK umbrella, this provides wide latitude to
use something like Graal?s native image as a final condenser.

> In Project Leyden we?ll explore additional techniques for shifting
computation, and
> where necessary we?ll evolve the Platform Specification to accommodate
them so
> that shifting computation preserves a program?s meaning.

+1 to preserving meaning!  The developer indicating what they want - ie:
when something should be initialized - is key as only the developer knows
the intent behind their code.

> Some of these techniques might not require any specification changes,
e.g.,
> expanding dynamic-proxy call sites into ordinary bytecode prior to run
time.

My experiments in this area have shown that the specification doesn?t need
to change, but that there are user-visible changes when doing this (ie:
Class:getNestMembers and others).  Is there an intention to document the
user visible aspects of changes from Condensers?

> Others will definitely require specification changes, e.g., resolving
classes prior to run
>  time. Yet others will take the form of new platform features that allow
developers to
> express temporal shifting directly in source code, e.g., lazy static
final fields.

Glad to hear that language changes are in the scope of this proposal.
Helping developers say what they mean makes it easier to reason about what
needs to change versus trying to impose new semantics on top of the soup
that is <clinit>.

> So let?s generalize, and allow an arbitrary number of phases in which
time-shifting
> transformations and related optimizations can be applied.

To confirm: this is saying that a ?condensed? Java application may act as
an input to a later phase of ?condensation??

> Condensation is meaning preserving.

Have you given thought to precisely specifying and then validating this
property?  It?s definitely something we want but it may be challenging to
enforce?.

> To ensure that condensers run code correctly, and transform and constrain
code
> only in ways that preserve meaning, we must add the concept of condensers
to
> the Java Platform Specification.

I?m unclear reading this if the set of Condensers will be limited to what
an implementation chooses to provide or if others (frameworks) can provide
their own Condensers as well?

Currently, implementing a jlink plugin from outside the platform is a pain
(though possible with enough ?add-opens and reflective hacks) but I?d
prefer to see Condensers have a more supported api so the ecosystem can
extend the concept.

> We can, however, enable other Java implementations (e.g., some future
version
> of the GraalVM native-image tool) to do so.

This is prescient given the recent announcement that Graal CE is moving to
OpenJDK =)

> Eliminate apparently-unused classes and class members (i.e., stripping,
which
> requires the constraint that selected classes and members cannot be
reflected upon)

Being slightly pedantic, it means the removed members can?t be reflected
on, correct?  A class that?s been stripped of unused methods / fields can
still have its remaining members reflectively looked up is my assumption.

> jlink

Using jlink is the natural starting point for this work.  It would be
unfortunate if condensers were limited to only work on modules given the
(unfortunately) slow update of them.  Is the intention to extend jlink to
work with classpath entities (ie: non-modular code) as well?  We foresee
the need for condensers to be able to ?condense? non-modules as well.

We?re excited to start working with you on this approach.  When do you
think we?d be ready to create a Leyden repo and start developing the code
for this approach?

Thanks,
?Dan



On Thu, Oct 13, 2022 at 1:56 PM Mark Reinhold <mark.reinhold at oracle.com>
wrote:

> For a few months now I?ve been pondering, and brainstorming with a few
> of you, how best to frame the work of Project Leyden conceptually.
> Here?s my proposal:
>
>   https://openjdk.org/projects/leyden/notes/02-shift-and-constrain
>
> Short summary:
>
>   The goal of Project Leyden is to improve the startup time, time to
>   peak performance, and footprint of Java programs.  In this note we
>   propose to work toward that goal by extending the Java programming
>   model with features for _selectively shifting and constraining
>   computation_ by means of _condensing code_.  We also propose an
>   initial research and development roadmap.
>
> Comments, questions, and suggestions welcome!
>
> - Mark
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <https://mail.openjdk.org/pipermail/leyden-dev/attachments/20221021/b1f3c632/attachment.htm>

From forax at univ-mlv.fr  Sat Oct 22 05:56:54 2022
From: forax at univ-mlv.fr (forax at univ-mlv.fr)
Date: Sat, 22 Oct 2022 07:56:54 +0200 (CEST)
Subject: Selectively Shifting and Constraining Computation
In-Reply-To: <15CF9BD7-D9F3-44C8-9BA8-E1B07AD5217F@oracle.com>
References: <20221013105630.787752083@eggemoggin.niobe.net>
 <1690520167.25120630.1665686609004.JavaMail.zimbra@u-pem.fr>
 <20221013120424.298245311@eggemoggin.niobe.net>
 <78C8DA0A-FB04-46F2-9512-255F31B775EB@oracle.com>
 <15CF9BD7-D9F3-44C8-9BA8-E1B07AD5217F@oracle.com>
Message-ID: <381835350.31599386.1666418214406.JavaMail.zimbra@u-pem.fr>

> From: "John Rose" <john.r.rose at oracle.com>
> To: "mark reinhold" <mark.reinhold at oracle.com>
> Cc: "Remi Forax" <forax at univ-mlv.fr>, "leyden-dev" <leyden-dev at openjdk.org>
> Sent: Tuesday, October 18, 2022 8:18:08 PM
> Subject: Re: Selectively Shifting and Constraining Computation

> And backing out of those particular points raised by Remi, I?d like to say that
> this was a good read; thanks Mark.

> Time-shifting is IMO a deeply fruitful notion, and will probably carry beyond
> the specific use cases of adjusting class loading order and static-final
> constant values. I hope we can find a few tiny spots in the language and VM
> specs where we can remove unimportant restrictions that will make various forms
> of time-shifting (forward and backward, global and local) semantically neutral,
> with respect to specified semantics. To do that we will have to make some parts
> of the specification more lenient towards reordering of expressions that might
> have side effects, and/or some way to constrain side effects. But we must do so
> with exquisite care, and (often) with opt-in from the user. That?s how lazy
> statics are conceived currently: They can be reordered because they are
> uncoupled from their class?s <clinit> , and the uncoupling must be explicitly
> permitted by the user. It?s a very interesting problem, and a very worth-while
> one.

I'm a great fan of the lazy statics, but you do not only get reordering of the side effects, you may also get double executions of the initialization code, double executions of the side effects. 
So the constraint on the side effects when initializing a lazy static is quite radical, do not do them . 

> I like the way you link time-shifting to the introduction of constraints. I
> already mentioned how the idea of constraints might tie to call sites. Maybe
> also data structure fields; who knows. I?m not in a hurry to figure it all out;
> it feels like we are at the start of some years of excellent investigation.

> ? John

R?mi 
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <https://mail.openjdk.org/pipermail/leyden-dev/attachments/20221022/bce86294/attachment.htm>

From john.r.rose at oracle.com  Tue Oct 25 00:39:20 2022
From: john.r.rose at oracle.com (John Rose)
Date: Mon, 24 Oct 2022 17:39:20 -0700
Subject: Selectively Shifting and Constraining Computation
In-Reply-To: <381835350.31599386.1666418214406.JavaMail.zimbra@u-pem.fr>
References: <20221013105630.787752083@eggemoggin.niobe.net>
 <1690520167.25120630.1665686609004.JavaMail.zimbra@u-pem.fr>
 <20221013120424.298245311@eggemoggin.niobe.net>
 <78C8DA0A-FB04-46F2-9512-255F31B775EB@oracle.com>
 <15CF9BD7-D9F3-44C8-9BA8-E1B07AD5217F@oracle.com>
 <381835350.31599386.1666418214406.JavaMail.zimbra@u-pem.fr>
Message-ID: <9A1EF932-96AE-455B-B50B-BB7ED7C6CCD8@oracle.com>

On 21 Oct 2022, at 22:56, forax at univ-mlv.fr wrote:

> I'm a great fan of the lazy statics, but you do not only get reordering of the side effects, you may also get double executions of the initialization code, double executions of the side effects.
> So the constraint on the side effects when initializing a lazy static is quite radical, do not do them .

I think you are referring to the draft semantics of lazies which piggy-back on top of CONSTANT_Dynamic, which indeed allows initialization races and therefore double side effects.  I?m getting rid of this in the newest draft of the JEP, so that refactoring to lazy will be easier to say ?yes? to.

A lazy static should never double its initializer, regardless of init-races.  My current plan is to insist that the same class locking be done for each lazy init as for <clinit> itself (which is the big lazy init of the overall class).  And on the same CL object mentioned in JVMS 5.5.

Also, errors in lazy inits will be recorded as ExceptionInInitializerError, not BootstrapMethodError.  Both effects require some subtle tweaking of the bootstrapping of lazies, which I?m working on?

I just updated the JEP to reflect this:  https://bugs.openjdk.org/browse/JDK-8209964

There?s a new section on translation strategy.  It hints at adjustments to the BSM protocol, which I?m currently investigating.  The basic idea there is that a BSM can ask that the JVM pass it exactly one argument, which reifies the entire state of the relevant condy entry, and allows the BSM to update that state appropriately as it arbitrates error and race conditions.

See also a slide deck mentioned in the comments.

From david.lloyd at redhat.com  Tue Oct 25 02:06:05 2022
From: david.lloyd at redhat.com (David Lloyd)
Date: Mon, 24 Oct 2022 21:06:05 -0500
Subject: Selectively Shifting and Constraining Computation
In-Reply-To: <9A1EF932-96AE-455B-B50B-BB7ED7C6CCD8@oracle.com>
References: <20221013105630.787752083@eggemoggin.niobe.net>
 <1690520167.25120630.1665686609004.JavaMail.zimbra@u-pem.fr>
 <20221013120424.298245311@eggemoggin.niobe.net>
 <78C8DA0A-FB04-46F2-9512-255F31B775EB@oracle.com>
 <15CF9BD7-D9F3-44C8-9BA8-E1B07AD5217F@oracle.com>
 <381835350.31599386.1666418214406.JavaMail.zimbra@u-pem.fr>
 <9A1EF932-96AE-455B-B50B-BB7ED7C6CCD8@oracle.com>
Message-ID: <CANghgrRpNo2PN8pH5MAwBoJAMCQ1WDpGpgNZmrsHSwoa=T3TEA@mail.gmail.com>

On Mon, Oct 24, 2022 at 7:39 PM John Rose <john.r.rose at oracle.com> wrote:

> On 21 Oct 2022, at 22:56, forax at univ-mlv.fr wrote:
>
> > I'm a great fan of the lazy statics, but you do not only get reordering
> of the side effects, you may also get double executions of the
> initialization code, double executions of the side effects.
> > So the constraint on the side effects when initializing a lazy static is
> quite radical, do not do them .
>
> I think you are referring to the draft semantics of lazies which
> piggy-back on top of CONSTANT_Dynamic, which indeed allows initialization
> races and therefore double side effects.  I?m getting rid of this in the
> newest draft of the JEP, so that refactoring to lazy will be easier to say
> ?yes? to.
>
> A lazy static should never double its initializer, regardless of
> init-races.  My current plan is to insist that the same class locking be
> done for each lazy init as for <clinit> itself (which is the big lazy init
> of the overall class).  And on the same CL object mentioned in JVMS 5.5.
>

Wouldn't it be preferable to use a lock per lazy static? One of the main
downsides of a static final field, apart from the lack of initialization
granularity, is how the initialization granularity interacts with the
corresponding lock granularity. So if I naively have fields that seem
"constant" with only "constant" dependencies (in the form of other static
final fields), and those classes may be initialized by multiple threads in
an indeterminate order, I may introduce a deadlock. It would be unfortunate
if lazy statics were prone to the same kind of deadlock due to the coarse
granularity that I would assume would have been left behind.

By having a lock per field, I believe it would be significantly less likely
to introduce a deadlock inadvertently (maybe even impossible if
initialization cycles could be detected somehow, but it's late here and
maybe my tired brain didn't reason this correctly).

-- 
- DML ? he/him
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <https://mail.openjdk.org/pipermail/leyden-dev/attachments/20221024/b9881db3/attachment.htm>

From john.r.rose at oracle.com  Tue Oct 25 02:40:16 2022
From: john.r.rose at oracle.com (John Rose)
Date: Mon, 24 Oct 2022 19:40:16 -0700
Subject: Selectively Shifting and Constraining Computation
In-Reply-To: <CANghgrRpNo2PN8pH5MAwBoJAMCQ1WDpGpgNZmrsHSwoa=T3TEA@mail.gmail.com>
References: <20221013105630.787752083@eggemoggin.niobe.net>
 <1690520167.25120630.1665686609004.JavaMail.zimbra@u-pem.fr>
 <20221013120424.298245311@eggemoggin.niobe.net>
 <78C8DA0A-FB04-46F2-9512-255F31B775EB@oracle.com>
 <15CF9BD7-D9F3-44C8-9BA8-E1B07AD5217F@oracle.com>
 <381835350.31599386.1666418214406.JavaMail.zimbra@u-pem.fr>
 <9A1EF932-96AE-455B-B50B-BB7ED7C6CCD8@oracle.com>
 <CANghgrRpNo2PN8pH5MAwBoJAMCQ1WDpGpgNZmrsHSwoa=T3TEA@mail.gmail.com>
Message-ID: <1D014E3D-279B-4B22-AFA1-C6F2C766DAA0@oracle.com>

On 24 Oct 2022, at 19:06, David Lloyd wrote:

> On Mon, Oct 24, 2022 at 7:39 PM John Rose <john.r.rose at oracle.com> wrote:
>
>> On 21 Oct 2022, at 22:56, forax at univ-mlv.fr wrote:
>>
>>> I'm a great fan of the lazy statics, but you do not only get reordering
>> of the side effects, you may also get double executions of the
>> initialization code, double executions of the side effects.
>>> So the constraint on the side effects when initializing a lazy static is
>> quite radical, do not do them .
>>
>> I think you are referring to the draft semantics of lazies which
>> piggy-back on top of CONSTANT_Dynamic, which indeed allows initialization
>> races and therefore double side effects.  I?m getting rid of this in the
>> newest draft of the JEP, so that refactoring to lazy will be easier to say
>> ?yes? to.
>>
>> A lazy static should never double its initializer, regardless of
>> init-races.  My current plan is to insist that the same class locking be
>> done for each lazy init as for <clinit> itself (which is the big lazy init
>> of the overall class).  And on the same CL object mentioned in JVMS 5.5.
>>
>
> Wouldn't it be preferable to use a lock per lazy static?

In short, probably not.  You?d have lots of extra footprint (all those monitor states) for no demonstrable gain.  (?Despite the speculations you made, which I also made but w/o coming to any definite benefit.)  You?d lose something too:  The sequencing of initialization (lazy and not-lazy) of a single class would lose coherence.  This IMO is a risk:  The more lazies differ from not-lazies, the more possibility of bugs when adopting lazies.  And allowing a class to run initialization expressions concurrently is surely much less compatible with the current state, where initialization expressions are run in sequence.

So to me it was a no-brainer to use the existing lock ?LC?.

> One of the main
> downsides of a static final field, apart from the lack of initialization
> granularity, is how the initialization granularity interacts with the
> corresponding lock granularity. So if I naively have fields that seem
> "constant" with only "constant" dependencies (in the form of other static
> final fields), and those classes may be initialized by multiple threads in
> an indeterminate order, I may introduce a deadlock. It would be unfortunate
> if lazy statics were prone to the same kind of deadlock due to the coarse
> granularity that I would assume would have been left behind.
>
> By having a lock per field, I believe it would be significantly less likely
> to introduce a deadlock inadvertently (maybe even impossible if
> initialization cycles could be detected somehow, but it's late here and
> maybe my tired brain didn't reason this correctly).
>
> -- 
> - DML ? he/him

From forax at univ-mlv.fr  Tue Oct 25 06:54:12 2022
From: forax at univ-mlv.fr (forax at univ-mlv.fr)
Date: Tue, 25 Oct 2022 08:54:12 +0200 (CEST)
Subject: Selectively Shifting and Constraining Computation
In-Reply-To: <9A1EF932-96AE-455B-B50B-BB7ED7C6CCD8@oracle.com>
References: <20221013105630.787752083@eggemoggin.niobe.net>
 <1690520167.25120630.1665686609004.JavaMail.zimbra@u-pem.fr>
 <20221013120424.298245311@eggemoggin.niobe.net>
 <78C8DA0A-FB04-46F2-9512-255F31B775EB@oracle.com>
 <15CF9BD7-D9F3-44C8-9BA8-E1B07AD5217F@oracle.com>
 <381835350.31599386.1666418214406.JavaMail.zimbra@u-pem.fr>
 <9A1EF932-96AE-455B-B50B-BB7ED7C6CCD8@oracle.com>
Message-ID: <1856700346.33013309.1666680852837.JavaMail.zimbra@u-pem.fr>

----- Original Message -----
> From: "John Rose" <john.r.rose at oracle.com>
> To: "Remi Forax" <forax at univ-mlv.fr>
> Cc: "mark reinhold" <mark.reinhold at oracle.com>, "leyden-dev" <leyden-dev at openjdk.org>
> Sent: Tuesday, October 25, 2022 2:39:20 AM
> Subject: Re: Selectively Shifting and Constraining Computation

> On 21 Oct 2022, at 22:56, forax at univ-mlv.fr wrote:
> 
>> I'm a great fan of the lazy statics, but you do not only get reordering of the
>> side effects, you may also get double executions of the initialization code,
>> double executions of the side effects.
>> So the constraint on the side effects when initializing a lazy static is quite
>> radical, do not do them .
> 
> I think you are referring to the draft semantics of lazies which piggy-back on
> top of CONSTANT_Dynamic, which indeed allows initialization races and therefore
> double side effects.  I?m getting rid of this in the newest draft of the JEP,
> so that refactoring to lazy will be easier to say ?yes? to.
> 
> A lazy static should never double its initializer, regardless of init-races.  My
> current plan is to insist that the same class locking be done for each lazy
> init as for <clinit> itself (which is the big lazy init of the overall class).
> And on the same CL object mentioned in JVMS 5.5.
> 
> Also, errors in lazy inits will be recorded as ExceptionInInitializerError, not
> BootstrapMethodError.  Both effects require some subtle tweaking of the
> bootstrapping of lazies, which I?m working on?
> 
> I just updated the JEP to reflect this:
> https://bugs.openjdk.org/browse/JDK-8209964
> 
> There?s a new section on translation strategy.  It hints at adjustments to the
> BSM protocol, which I?m currently investigating.  The basic idea there is that
> a BSM can ask that the JVM pass it exactly one argument, which reifies the
> entire state of the relevant condy entry, and allows the BSM to update that
> state appropriately as it arbitrates error and race conditions.
> 
> See also a slide deck mentioned in the comments.

I've just read it, i like it, it's far better than the previous semantics and using the same lock as <clinit> will allow IDEs to provide nice refactoring without having to think too much.

One question: should the new BSM protocol also work with indy BSM, it does has to given that condy update a the constant pool constant while indy update a method handle target, so not the same data structure in memory but it may be nice to use the same umbrella.

Also, it will not only benefit Java but also Clojure or any other languages on top of the JVM that use reified objects to represent its code.

R?mi

From david.lloyd at redhat.com  Tue Oct 25 14:04:28 2022
From: david.lloyd at redhat.com (David Lloyd)
Date: Tue, 25 Oct 2022 09:04:28 -0500
Subject: Selectively Shifting and Constraining Computation
In-Reply-To: <1D014E3D-279B-4B22-AFA1-C6F2C766DAA0@oracle.com>
References: <20221013105630.787752083@eggemoggin.niobe.net>
 <1690520167.25120630.1665686609004.JavaMail.zimbra@u-pem.fr>
 <20221013120424.298245311@eggemoggin.niobe.net>
 <78C8DA0A-FB04-46F2-9512-255F31B775EB@oracle.com>
 <15CF9BD7-D9F3-44C8-9BA8-E1B07AD5217F@oracle.com>
 <381835350.31599386.1666418214406.JavaMail.zimbra@u-pem.fr>
 <9A1EF932-96AE-455B-B50B-BB7ED7C6CCD8@oracle.com>
 <CANghgrRpNo2PN8pH5MAwBoJAMCQ1WDpGpgNZmrsHSwoa=T3TEA@mail.gmail.com>
 <1D014E3D-279B-4B22-AFA1-C6F2C766DAA0@oracle.com>
Message-ID: <CANghgrSnnEL1Kb3nZFb8O-OOy1VCWRVVOcbAnEFEGjesVXaSTQ@mail.gmail.com>

On Mon, Oct 24, 2022 at 9:40 PM John Rose <john.r.rose at oracle.com> wrote:
>
> On 24 Oct 2022, at 19:06, David Lloyd wrote:
>
> > On Mon, Oct 24, 2022 at 7:39 PM John Rose <john.r.rose at oracle.com> wrote:
> >
> >> On 21 Oct 2022, at 22:56, forax at univ-mlv.fr wrote:
> >>
> >>> I'm a great fan of the lazy statics, but you do not only get reordering
> >> of the side effects, you may also get double executions of the
> >> initialization code, double executions of the side effects.
> >>> So the constraint on the side effects when initializing a lazy static is
> >> quite radical, do not do them .
> >>
> >> I think you are referring to the draft semantics of lazies which
> >> piggy-back on top of CONSTANT_Dynamic, which indeed allows initialization
> >> races and therefore double side effects.  I?m getting rid of this in the
> >> newest draft of the JEP, so that refactoring to lazy will be easier to say
> >> ?yes? to.
> >>
> >> A lazy static should never double its initializer, regardless of
> >> init-races.  My current plan is to insist that the same class locking be
> >> done for each lazy init as for <clinit> itself (which is the big lazy init
> >> of the overall class).  And on the same CL object mentioned in JVMS 5.5.
> >>
> >
> > Wouldn't it be preferable to use a lock per lazy static?
>
> In short, probably not.  You?d have lots of extra footprint (all those monitor states) for no demonstrable gain.

Wouldn't the monitor only actually be necessary during the actual
initialization? Otherwise the number of initialization states per
field would be identical, I would expect. Since lazy static
initialization would be "rare", I would therefore expect the monitor
(or its equivalent) to be stored out-of-line with respect to the
semi-permanent class structures, and only retained until
initialization is complete, whereas when the init lock LC is shared,
it must be created no later than the first initialization event and
destroyed no sooner than after every field and the class itself have
been initialized.

> (?Despite the speculations you made, which I also made but w/o coming to any definite benefit.)

The avoidance of deadlock is a definite benefit IMO :-). Class init
deadlocks have historically existed in the wild. Since the invisible
class initialization lock also is coupled to the class inheritance
initialization sequence (super before sub), the same kinds of
dependencies which can cause class init deadlocks could cause lazy
static deadlocks, except it might be less clear what is happening, and
why, due to the independent initialization of the lazy statics
themselves.

And, since it seems I owe you some speculation: concurrent
initialization can have performance benefits as well, relating to
startup time. I also speculate that fully independent initialization
would have benefits for native image generators of the present and
future.

> You?d lose something too:  The sequencing of initialization (lazy and not-lazy) of a single class would lose coherence.  This IMO is a risk:  The more lazies differ from not-lazies, the more possibility of bugs when adopting lazies.  And allowing a class to run initialization expressions concurrently is surely much less compatible with the current state, where initialization expressions are run in sequence.

I would argue that independently-locked lazies would be _less_
different from not-lazies because the thing that lazy statics is
really replacing, AFAICT, is the "lazy holder" pattern, much in the
same way that lambdas replace many usages of anonymous classes. Giving
them independent locks is, from this perspective, less different from
the status quo than sharing a lock.

--
- DML ? he/him


From david.lloyd at redhat.com  Tue Oct 25 14:07:09 2022
From: david.lloyd at redhat.com (David Lloyd)
Date: Tue, 25 Oct 2022 09:07:09 -0500
Subject: Selectively Shifting and Constraining Computation
In-Reply-To: <1856700346.33013309.1666680852837.JavaMail.zimbra@u-pem.fr>
References: <20221013105630.787752083@eggemoggin.niobe.net>
 <1690520167.25120630.1665686609004.JavaMail.zimbra@u-pem.fr>
 <20221013120424.298245311@eggemoggin.niobe.net>
 <78C8DA0A-FB04-46F2-9512-255F31B775EB@oracle.com>
 <15CF9BD7-D9F3-44C8-9BA8-E1B07AD5217F@oracle.com>
 <381835350.31599386.1666418214406.JavaMail.zimbra@u-pem.fr>
 <9A1EF932-96AE-455B-B50B-BB7ED7C6CCD8@oracle.com>
 <1856700346.33013309.1666680852837.JavaMail.zimbra@u-pem.fr>
Message-ID: <CANghgrRQeSKShg52UTKdmQvohbknCFuLoPnNeENz7ZpsfT+d0w@mail.gmail.com>

On Tue, Oct 25, 2022 at 1:56 AM <forax at univ-mlv.fr> wrote:
> I've just read it, i like it, it's far better than the previous semantics and using the same lock as <clinit> will allow IDEs to provide nice refactoring without having to think too much.

Just out of curiosity, why might the behavior of the initialization
lock make a difference for IDE refactoring?
-- 
- DML ? he/him


From forax at univ-mlv.fr  Tue Oct 25 15:06:01 2022
From: forax at univ-mlv.fr (forax at univ-mlv.fr)
Date: Tue, 25 Oct 2022 17:06:01 +0200 (CEST)
Subject: Selectively Shifting and Constraining Computation
In-Reply-To: <CANghgrRQeSKShg52UTKdmQvohbknCFuLoPnNeENz7ZpsfT+d0w@mail.gmail.com>
References: <20221013105630.787752083@eggemoggin.niobe.net>
 <20221013120424.298245311@eggemoggin.niobe.net>
 <78C8DA0A-FB04-46F2-9512-255F31B775EB@oracle.com>
 <15CF9BD7-D9F3-44C8-9BA8-E1B07AD5217F@oracle.com>
 <381835350.31599386.1666418214406.JavaMail.zimbra@u-pem.fr>
 <9A1EF932-96AE-455B-B50B-BB7ED7C6CCD8@oracle.com>
 <1856700346.33013309.1666680852837.JavaMail.zimbra@u-pem.fr>
 <CANghgrRQeSKShg52UTKdmQvohbknCFuLoPnNeENz7ZpsfT+d0w@mail.gmail.com>
Message-ID: <1871497607.33616758.1666710361027.JavaMail.zimbra@u-pem.fr>

----- Original Message -----
> From: "David Lloyd" <david.lloyd at redhat.com>
> To: "leyden-dev" <leyden-dev at openjdk.org>
> Cc: "Remi Forax" <forax at univ-mlv.fr>
> Sent: Tuesday, October 25, 2022 4:07:09 PM
> Subject: Re: Selectively Shifting and Constraining Computation

Hi David,

> On Tue, Oct 25, 2022 at 1:56 AM <forax at univ-mlv.fr> wrote:
>> I've just read it, i like it, it's far better than the previous semantics and
>> using the same lock as <clinit> will allow IDEs to provide nice refactoring
>> without having to think too much.
> 
> Just out of curiosity, why might the behavior of the initialization
> lock make a difference for IDE refactoring?

Because it's under a lock, so you can not have a double initialization, so the IDE do not have to prove that there is no side effect before doing a refactoring that introduces a lazy constant.

This does not mean that the lock of the <clinit> and the lock used by the initialization of the lazy constant has to be the same, having a transient lock for the lazy initialization mey be enough. As you said, in the lazy holder pattern, the lock of <clinit> of the class is not used but the lock of the <clinit> of the holder class is used instead.

> --
> - DML ? he/him


R?mi

From mark.reinhold at oracle.com  Fri Oct 28 00:28:41 2022
From: mark.reinhold at oracle.com (Mark Reinhold)
Date: Fri, 28 Oct 2022 00:28:41 +0000
Subject: Selectively Shifting and Constraining Computation
In-Reply-To: <CAJq4Gi7GF2k-Bkx8Q3L1_v5J6mEzfBX0a3BZ8iXmePBssPj79w@mail.gmail.com>
References: <20221013105630.787752083@eggemoggin.niobe.net>
 <CAJq4Gi7GF2k-Bkx8Q3L1_v5J6mEzfBX0a3BZ8iXmePBssPj79w@mail.gmail.com>
Message-ID: <20221027172839.945357016@eggemoggin.niobe.net>

2022/10/21 19:49:36 -0700, Dan Heidinga <heidinga at redhat.com>:
> Thanks for posting this document.  It provides a clear starting point for
> us to move forward to start to address Leyden.

Thanks, Dan, and thanks for your comments and questions.

> A few questions based on my read through:
> 
>> As long as a condenser preserves program meaning and does not impose
>> constraints other than those accepted by the developer, an implementation has
>> wide latitude to optimize the result.
> 
> Does ?an implementation? refer only to the JDK or are other parts of the
> ecosystem - frameworks, libraries, etc - able to add their own condensers
> as well?

Good question!

We?re concerned here primarily with any software system that?s claimed
to be a conformant implementation of the Java Platform -- the JDK is
merely the dominant and best-known example.  The components of such a
system -- including any condensers -- can be held to the rigors of the
Platform Specification and the JCK, thereby ensuring that they preserve
the meanings of Java programs.

Some people will, no doubt, want to write their own condensers, and
obviously we?d like all condensers to preserve program meaning.  The
current conformance regime (specification/RI/JCK), however, applies only
to entire Platform implementations.  If we want to enable developers of
external condensers to make certifiable claims of safety then we?ll need
to extend the conformance regime to test standalone condensers.
(Whether that needs to be a priority isn?t clear to me at the moment.)

Enabling the foundational tooling of the JDK (jlink, or whatever) to run
arbitrary condensers is attractive, but not without risk.  People could
ship non-conformant condensers even if we extend the conformance regime,
since there?d be no conformance requirement.  Developers could, perhaps
unwittingly, use such condensers and wind up with broken programs.  So
maybe we?ll support external condensers in the JDK toolchain but emit a
warning when they?re used.

>> If a developer chooses condensers that shift sufficient computation from run
>> time to earlier phases ? which may require accepting many constraints ? then
>> a conforming Java implementation could even produce a fully-static
>> platform-specific executable.
> 
> Great to read this!  Especially in light of the recent announcement around
> GraalCE moving under the OpenJDK umbrella, this provides wide latitude to
> use something like Graal?s native image as a final condenser.

Exactly.

> ...
> 
>> Some of these techniques might not require any specification changes, e.g.,
>> expanding dynamic-proxy call sites into ordinary bytecode prior to run time.
> 
> My experiments in this area have shown that the specification doesn?t need
> to change, but that there are user-visible changes when doing this (ie:
> Class:getNestMembers and others).  Is there an intention to document the
> user visible aspects of changes from Condensers?

To the extent that a condenser changes the behavior of a program within
the bounds of the (possibly enhanced) Platform Specification then yes,
we should document those changes.

>> Others will definitely require specification changes, e.g., resolving classes
>> prior to run time. Yet others will take the form of new platform features
>> that allow developers to express temporal shifting directly in source code,
>> e.g., lazy static final fields.
> 
> Glad to hear that language changes are in the scope of this proposal.
> Helping developers say what they mean makes it easier to reason about what
> needs to change versus trying to impose new semantics on top of the soup
> that is <clinit>.

Indeed!  Applying new semantics to existing code always leads to tears.

>> So let?s generalize, and allow an arbitrary number of phases in which
>> time-shifting transformations and related optimizations can be applied.
> 
> To confirm: this is saying that a ?condensed? Java application may act as
> an input to a later phase of ?condensation??

Yes, for some but likely not all definitions of ?condensed.?  If you?ve
condensed all the way down to a platform-specific executable, e.g., then
it?s likely not worth the effort for us to support further condensation.
(I suspect it?ll be acceptable to consider condensation to such formats
to be terminal steps, as you suggest above.)

>> Condensation is meaning preserving.
> 
> Have you given thought to precisely specifying and then validating this
> property?  It?s definitely something we want but it may be challenging to
> enforce?.

The meaning of a Java program is already given by the Specification, and
the JCK already validates a Java implementation by running a large
collection of Specification-derived tests.  My initial thought here is
to extend the JCK to test a condenser by applying the condenser to each
JCK test and then verifying that the condensed test still passes.  (To
keep JCK run times tractable, do this in large batches.)

>> To ensure that condensers run code correctly, and transform and constrain
>> code only in ways that preserve meaning, we must add the concept of
>> condensers to the Java Platform Specification.
> 
> I?m unclear reading this if the set of Condensers will be limited to what
> an implementation chooses to provide or if others (frameworks) can provide
> their own Condensers as well?

(See above.)

> Currently, implementing a jlink plugin from outside the platform is a pain
> (though possible with enough ?add-opens and reflective hacks) but I?d
> prefer to see Condensers have a more supported api so the ecosystem can
> extend the concept.

Agreed.  We probably do want an open condenser API, though designing a
good one will take considerable effort.  (We didn?t open the jlink
plugin API because we knew that it wasn?t ready.)

>> We can, however, enable other Java implementations (e.g., some future version
>> of the GraalVM native-image tool) to do so.
> 
> This is prescient given the recent announcement that Graal CE is moving to
> OpenJDK =)

What a coincidence!

>> Eliminate apparently-unused classes and class members (i.e., stripping, which
>> requires the constraint that selected classes and members cannot be reflected
>> upon)
> 
> Being slightly pedantic, it means the removed members can?t be reflected
> on, correct?

Yes, but ...

>               A class that?s been stripped of unused methods / fields can
> still have its remaining members reflectively looked up is my assumption.

That?s a reasonable assumption.

In detail, though, we could in principle retain reflective metadata even
for removed program elements, if there?s a need to do so.  We could,
likewise, remove reflective metadata for some retained program elements
if we?re confident that those elements will never be reflected upon.
More exploration required ...

>> jlink
> 
> Using jlink is the natural starting point for this work.  It would be
> unfortunate if condensers were limited to only work on modules given the
> (unfortunately) slow update of them.  Is the intention to extend jlink to
> work with classpath entities (ie: non-modular code) as well?  We foresee
> the need for condensers to be able to ?condense? non-modules as well.

The jlink tool doesn?t support arbitrary class path code, or automatic
modules, in order to guarantee that the resulting image is structurally
correct by construction.  I?m reluctant to give up on that guarantee.
Viable workarounds exist for non-modular code (jdeps, ModiTect, etc.),
so developers who want to risk building a potentially-broken image from
a potentially-broken class path can do so, at their option.

I do expect that, in Leyden, we?ll find interesting ways to leverage the
constraints already imposed by modules.  Hopefully that will motivate
the further adoption of modules.

> We?re excited to start working with you on this approach.  When do you
> think we?d be ready to create a Leyden repo and start developing the code
> for this approach?

I?m in the process of setting up a repo; more on that soon.

- Mark

From karsten.silz at gmail.com  Fri Oct 28 07:19:19 2022
From: karsten.silz at gmail.com (Karsten Silz)
Date: Fri, 28 Oct 2022 08:19:19 +0100
Subject: Selectively Shifting and Constraining Computation
Message-ID: <1173CBA7-C57E-4AD8-BC1C-0C9C868F7717@gmail.com>

HI!

This is my first posting here, so please bear with me.

The proposal gives two reasons why working with static executables is inconvenient. These reasons may have influenced the decision to delay the standardizing of static executables in favor of optimizing Java first. I'm not sure how valid these two reasons will be when the first results of Leyden become broadly available in three years. So I'd like to get your feedback!

I assume 95+% of users run LTS Java releases in production (https://youtu.be/SYO-LmA647E?t=186), Leyden won't be in Java 21 but will ship (at least partially) in Java 25.

The first reason was that static executables require "developers to construct and maintain fragile configuration files". We need these configuration files if a library doesn't work with static executables out of the box and our Java framework can't configure that library (GraalVM strongly recommends using a framework for static executables). Or if our own code does something neither GraalVM nor the framework can figure out. The libraries shipping with GraalVM-supporting Java frameworks work, so this compatibility issue is mainly about that "random library from Maven Central".

VMware announced last year that Spring Boot 3 will support GraalVM static executables. I think that'll tremendously increase the compatibility of the Java ecosystem with static executables. Why? Because everybody wants to integrate with Spring Boot. 

And compatibility is now a community effort: GraalVM, Spring, and Micronaut launched the GraalVM Reachability Metadata Repository. Helidon may join soon, according to its lead. And since everybody can contribute reachability data, this is now even partially crowd-sourced.

Will that work perfectly next month with Spring Boot 3? Nope. Will it be good enough for most new projects with Spring Boot 3.6/4.0 in 3 years? I believe so. Now, this is just my opinion. So, how do other people predict the degree of static executable compatibility in Java in three years?

BTW: Spring Boot can use Java method calls and annotations for configuring static executables, not "fragile configuration files" (https://www.youtube.com/watch?v=TS4DpYSmfXk&t=2307s).

The second reason is that developers "endure unusually long build times" with GraalVM static executables. Unusually long build times? Yes. Developers? Not so often. Why?

Because the best practice seems to be to NOT work with static executables during development but use Java instead. I think this mirrors Java development for Kubernetes, where it's often impractical to run a whole Kubernetes cluster with all the nodes and all the data on the developer's machine. The difference between "local Java" and "running application" is more significant with static executables, but it appears manageable for daily development. 

In that approach, it's generally the CI/CD pipeline that builds static executables. Developers only build static executables "before a significant feature is about to be merged or when a problem arises that requires debugging", to quote a "lessons learned from using static executables" InfoQ article I edited (https://www.infoq.com/articles/native-java-real-world).

So it seems long build times are mostly not an issue today and will be even less so in three years: At least so far, static executable build times have decreased continuously with GraalVM releases.

Now, I have just one data point with my article. What does everybody else think about the impact of build times on static executables?


Regards,
Karsten Silz



-------------- next part --------------
An HTML attachment was scrubbed...
URL: <https://mail.openjdk.org/pipermail/leyden-dev/attachments/20221028/82ec6fca/attachment.htm>

From ron.pressler at oracle.com  Fri Oct 28 21:10:57 2022
From: ron.pressler at oracle.com (Ron Pressler)
Date: Fri, 28 Oct 2022 21:10:57 +0000
Subject: Selectively Shifting and Constraining Computation
In-Reply-To: <1173CBA7-C57E-4AD8-BC1C-0C9C868F7717@gmail.com>
References: <1173CBA7-C57E-4AD8-BC1C-0C9C868F7717@gmail.com>
Message-ID: <105359C2-4C21-4262-B9EA-31CAD896889D@oracle.com>



> On 28 Oct 2022, at 08:19, Karsten Silz <karsten.silz at gmail.com> wrote:
> 
> These reasons may have influenced the decision to delay the standardizing of static executables in favor of optimizing Java first. 

I couldn?t find anything in the document that says that the implementation of some condensers will be delayed in favour of others. It outlines the mechanism that (we hope and expect) will make it possible to ?standardise static executables?. That is the same mechanism that will also support the work, that has already been ongoing for years, to continue optimising other points along the dynamism spectrum. The work on thinking about, and eventually specifying condensers *is* the work of ?standardising" static executables (as well as other flavours), and it?s the first item mentioned in the roadmap.

? Ron


From ron.pressler at oracle.com  Fri Oct 28 21:40:39 2022
From: ron.pressler at oracle.com (Ron Pressler)
Date: Fri, 28 Oct 2022 21:40:39 +0000
Subject: Selectively Shifting and Constraining Computation
In-Reply-To: <105359C2-4C21-4262-B9EA-31CAD896889D@oracle.com>
References: <1173CBA7-C57E-4AD8-BC1C-0C9C868F7717@gmail.com>
 <105359C2-4C21-4262-B9EA-31CAD896889D@oracle.com>
Message-ID: <EF348FFC-49AF-47EC-B80F-F12FF03F305C@oracle.com>



> On 28 Oct 2022, at 22:10, Ron Pressler <ron.pressler at oracle.com> wrote:
> 
> 
> 
>> On 28 Oct 2022, at 08:19, Karsten Silz <karsten.silz at gmail.com> wrote:
>> 
>> These reasons may have influenced the decision to delay the standardizing of static executables in favor of optimizing Java first. 
> 
> I couldn?t find anything in the document that says that the implementation of some condensers will be delayed in favour of others. It outlines the mechanism that (we hope and expect) will make it possible to ?standardise static executables?. That is the same mechanism that will also support the work, that has already been ongoing for years, to continue optimising other points along the dynamism spectrum. The work on thinking about, and eventually specifying condensers *is* the work of ?standardising" static executables (as well as other flavours), and it?s the first item mentioned in the roadmap.
> 
> ? Ron
> 


P.S.

Just to clarify, given that Java is today on the most dynamic end of the spectrum hinted at in the document, I would imagine that the specification work on condensers would naturally start at the point where Java is today, and gradually work its way from there toward allowing more ?static? forms of condensers. To get to the end we have to start at the beginning (and will probably learn a lot along the way and adjust accordingly).


From karsten.silz at gmail.com  Fri Oct 28 22:25:03 2022
From: karsten.silz at gmail.com (Karsten Silz)
Date: Fri, 28 Oct 2022 23:25:03 +0100
Subject: Selectively Shifting and Constraining Computation
In-Reply-To: <105359C2-4C21-4262-B9EA-31CAD896889D@oracle.com>
References: <1173CBA7-C57E-4AD8-BC1C-0C9C868F7717@gmail.com>
 <105359C2-4C21-4262-B9EA-31CAD896889D@oracle.com>
Message-ID: <34CFCCC2-B68B-439B-A103-827B30621194@gmail.com>


> On 28 Oct 2022, at 22:10, Ron Pressler <ron.pressler at oracle.com> wrote:
> 
>> On 28 Oct 2022, at 08:19, Karsten Silz <karsten.silz at gmail.com> wrote:
>> 
>> These reasons may have influenced the decision to delay the standardizing of static executables in favor of optimizing Java first. 
> 
> I couldn?t find anything in the document that says that the implementation of some condensers will be delayed in favour of others.

Sorry if that wasn?t clear: I was referring to the May announcement there: https://openjdk.org/projects/leyden/notes/01-beginnings <https://openjdk.org/projects/leyden/notes/01-beginnings>

"So rather than adopt the closed-world constraint at the start, I propose that we instead pursue a gradual, incremental approach. We will explore a spectrum of constraints, weaker than the closed-world constraint, and discover what optimizations they enable. [?] In the long run we will likely embrace the full closed-world constraint in order to produce fully-static images. Between now and then, however, we will develop and deliver incremental improvements which developers can use sooner rather than later.?

So I understood this as ?Leyden will optimize Java first and later standardize fully-static images?. Am I wrong?

And my understanding of the October proposal is that it explains how these goals will be achieved. But it didn?t seem to change the plan of optimizing Java first. Please let me know if I missed something there!


Regards,
Karsten Silz

-------------- next part --------------
An HTML attachment was scrubbed...
URL: <https://mail.openjdk.org/pipermail/leyden-dev/attachments/20221028/dfafa8d6/attachment.htm>

From ron.pressler at oracle.com  Fri Oct 28 23:40:36 2022
From: ron.pressler at oracle.com (Ron Pressler)
Date: Fri, 28 Oct 2022 23:40:36 +0000
Subject: [External] : Re: Selectively Shifting and Constraining Computation
In-Reply-To: <34CFCCC2-B68B-439B-A103-827B30621194@gmail.com>
References: <1173CBA7-C57E-4AD8-BC1C-0C9C868F7717@gmail.com>
 <105359C2-4C21-4262-B9EA-31CAD896889D@oracle.com>
 <34CFCCC2-B68B-439B-A103-827B30621194@gmail.com>
Message-ID: <3C558EE1-BD03-4749-B690-36AE7EC1D71B@oracle.com>


On 28 Oct 2022, at 23:25, Karsten Silz <karsten.silz at gmail.com<mailto:karsten.silz at gmail.com>> wrote:


On 28 Oct 2022, at 22:10, Ron Pressler <ron.pressler at oracle.com<mailto:ron.pressler at oracle.com>> wrote:

On 28 Oct 2022, at 08:19, Karsten Silz <karsten.silz at gmail.com<mailto:karsten.silz at gmail.com>> wrote:

These reasons may have influenced the decision to delay the standardizing of static executables in favor of optimizing Java first.

I couldn?t find anything in the document that says that the implementation of some condensers will be delayed in favour of others.

Sorry if that wasn?t clear: I was referring to the May announcement there: https://openjdk.org/projects/leyden/notes/01-beginnings

"So rather than adopt the closed-world constraint at the start, I propose that we instead pursue a gradual, incremental approach. We will explore a spectrum of constraints, weaker than the closed-world constraint, and discover what optimizations they enable. [?] In the long run we will likely embrace the full closed-world constraint in order to produce fully-static images. Between now and then, however, we will develop and deliver incremental improvements which developers can use sooner rather than later.?

So I understood this as ?Leyden will optimize Java first and later standardize fully-static images?. Am I wrong?

And my understanding of the October proposal is that it explains how these goals will be achieved. But it didn?t seem to change the plan of optimizing Java first. Please let me know if I missed something there!


Regards,
Karsten Silz


I don?t think you?re wrong in expecting that ?standardising static images? would probably not be the *first* deliverable from Leyden, but I do think you?re wrong in concluding that that means it?s *delayed* (e.g. Project Loom delivered JEPs 353 and 373 in JDKs 13 and 15 respectively, so JEP 425, virtual threads, wasn?t Loom?s first deliverable, but it certainly wasn?t delayed by those JEPs). You?re reading ?we believe we?ve found a way to deliver value at various points on the road to X? as ?X is delayed?, and that?s a misinterpretation.

The document suggests that the work on investigating the concept of condensers ? the thing that could eventually enable static images ? will start right away (and may be done in parallel to developing specific condensers). Obviously, that work will need to get from where we are today to some point that?s some ways off, but the great news is that we may be able to deliver value on the way there.

I think your real question might be, ?but I want feature X *now*; could you get there faster?? Like with many other OpenJDK projects, there's probably some ?there? that we could get to faster, but that?s usually not where we?d like to end up.

BTW, I don?t understand what ?optimising Java first? means. The work on optimising OpenJDK is constant; it neither starts nor ends with Leyden. Project Leyden, among other things, will support it.


? Ron
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <https://mail.openjdk.org/pipermail/leyden-dev/attachments/20221028/976741c8/attachment.htm>