From chf at redhat.com  Thu Feb  4 17:16:18 2021
From: chf at redhat.com (Christine Flood)
Date: Thu, 4 Feb 2021 12:16:18 -0500
Subject: create a fork under openjdk
In-Reply-To: <031901d6f4a5$c6a59590$53f0c0b0$@alibaba-inc.com>
References: <031901d6f4a5$c6a59590$53f0c0b0$@alibaba-inc.com>
Message-ID: <CALKUemzyFgF4VKK9UiDRNawaM+CNwvgPxiodgKkZJS-KDej8mQ@mail.gmail.com>

Have you guys looked at CRIU at all?  If you are running on Linux it gives
you the capability of firing up a warmed up Java application in minutes.

For example the TODO app (https://github.com/cescoffier/quarkus-todo-app)
takes 1.837 seconds to start but only  0.11 seconds to restore.

I've attached a console log showing the commands I used.

If you want greater control, I've written a Java API you can tinker with:
https://github.com/chflood/JavaCriuJar.

Unfortunately this is a Linux only capability.


Christine




On Wed, Jan 27, 2021 at 7:14 AM ???(??) <sanhong.lsh at alibaba-inc.com> wrote:

> Hello,
>
> We are working on the fast-startup related development based on JDK11u. To
> facilitate community collaboration(sharing backports, bug fixing, patches,
> etc. across different companies/parties based on the repo), we are
> exploring
> creating the fork of JDK11u under the OpenJDK group [1]
>
>
>
> Can anyone help with this, or is there a way/process we can follow to do
> that? - Your help or any input is much appreciated.
>
>
>
> [1]  <https://github.com/openjdk> https://github.com/openjdk
>
>
>
> Thanks!
>
> Sanhong
>
>
>
>

From sanhong.lsh at alibaba-inc.com  Tue Feb  9 07:03:15 2021
From: sanhong.lsh at alibaba-inc.com (=?UTF-8?B?5p2O5LiJ57qiKOS4iee6oik=?=)
Date: Tue, 09 Feb 2021 15:03:15 +0800
Subject: =?utf-8?Q?=E7=AD=94=E5=A4=8D:_create_a_fork_under_openjdk?=
In-Reply-To: <CALKUemzyFgF4VKK9UiDRNawaM+CNwvgPxiodgKkZJS-KDej8mQ@mail.gmail.com>
References: <031901d6f4a5$c6a59590$53f0c0b0$@alibaba-inc.com>
 <CALKUemzyFgF4VKK9UiDRNawaM+CNwvgPxiodgKkZJS-KDej8mQ@mail.gmail.com>
Message-ID: <00c501d6feb1$a3408770$e9c19650$@alibaba-inc.com>

Hi Christine,

Thanks for your comments  - the challenge of using CRIU is we can not handle the state migration of the java application.

The situation will become more worse when the application evolves forward, and we don?t have a good mechanism(may consider a middleware framework?) to enforce the ?stateless constraints? for their development.

 

Thanks!

Sanhong

???: Christine Flood <chf at redhat.com> 
????: 2021?2?5? 1:16
???: ???(??) <sanhong.lsh at alibaba-inc.com>
??: discuss at openjdk.java.net
??: Re: create a fork under openjdk

 

Have you guys looked at CRIU at all?  If you are running on Linux it gives you the capability of firing up a warmed up Java application in minutes.

 

For example the TODO app (https://github.com/cescoffier/quarkus-todo-app) takes 1.837 seconds to start but only  0.11 seconds to restore.

 

I've attached a console log showing the commands I used.

 

If you want greater control, I've written a Java API you can tinker with:  https://github.com/chflood/JavaCriuJar.

 

Unfortunately this is a Linux only capability.

 

 

Christine

 

 

 

 

On Wed, Jan 27, 2021 at 7:14 AM ???(??) <sanhong.lsh at alibaba-inc.com <mailto:sanhong.lsh at alibaba-inc.com> > wrote:

Hello,

We are working on the fast-startup related development based on JDK11u. To
facilitate community collaboration(sharing backports, bug fixing, patches,
etc. across different companies/parties based on the repo), we are exploring
creating the fork of JDK11u under the OpenJDK group [1]



Can anyone help with this, or is there a way/process we can follow to do
that? - Your help or any input is much appreciated.



[1]  <https://github.com/openjdk> https://github.com/openjdk 



Thanks!

Sanhong





From forax at univ-mlv.fr  Tue Feb  9 09:39:08 2021
From: forax at univ-mlv.fr (Remi Forax)
Date: Tue, 9 Feb 2021 10:39:08 +0100 (CET)
Subject: =?utf-8?Q?Re:_=E7=AD=94=E5=A4=8D:_create_a_fork_under_openjdk?=
In-Reply-To: <00c501d6feb1$a3408770$e9c19650$@alibaba-inc.com>
References: <031901d6f4a5$c6a59590$53f0c0b0$@alibaba-inc.com>
 <CALKUemzyFgF4VKK9UiDRNawaM+CNwvgPxiodgKkZJS-KDej8mQ@mail.gmail.com>
 <00c501d6feb1$a3408770$e9c19650$@alibaba-inc.com>
Message-ID: <648998894.1020386.1612863548859.JavaMail.zimbra@u-pem.fr>



----- Mail original -----
> De: "???(??)" <sanhong.lsh at alibaba-inc.com>
> ?: "Christine Flood" <chf at redhat.com>
> Cc: "discuss" <discuss at openjdk.java.net>
> Envoy?: Mardi 9 F?vrier 2021 08:03:15
> Objet: ??: create a fork under openjdk

> Hi Christine,
> 
> Thanks for your comments  - the challenge of using CRIU is we can not handle the
> state migration of the java application.
> 
> The situation will become more worse when the application evolves forward, and
> we don?t have a good mechanism(may consider a middleware framework?) to enforce
> the ?stateless constraints? for their development.
> 
> 
> 
> Thanks!
> 
> Sanhong


Hi Sanhong,
this seems orthogonal, why do you want state migration on top of CRIU ?

You can use CRIU just as a way to start your application in steady state skipping the warmup phase.
So you build your app, then you run it with fake traffic then save the states with CRIU.
If you update the app, instead of trying to migrate states, you just rebuild it, etc.

regards,
R?mi

> 
> ???: Christine Flood <chf at redhat.com>
> ????: 2021?2?5? 1:16
> ???: ???(??) <sanhong.lsh at alibaba-inc.com>
> ??: discuss at openjdk.java.net
> ??: Re: create a fork under openjdk
> 
> 
> 
> Have you guys looked at CRIU at all?  If you are running on Linux it gives you
> the capability of firing up a warmed up Java application in minutes.
> 
> 
> 
> For example the TODO app (https://github.com/cescoffier/quarkus-todo-app) takes
> 1.837 seconds to start but only  0.11 seconds to restore.
> 
> 
> 
> I've attached a console log showing the commands I used.
> 
> 
> 
> If you want greater control, I've written a Java API you can tinker with:
> https://github.com/chflood/JavaCriuJar.
> 
> 
> 
> Unfortunately this is a Linux only capability.
> 
> 
> 
> 
> 
> Christine
> 
> 
> 
> 
> 
> 
> 
> 
> 
> On Wed, Jan 27, 2021 at 7:14 AM ???(??) <sanhong.lsh at alibaba-inc.com
> <mailto:sanhong.lsh at alibaba-inc.com> > wrote:
> 
> Hello,
> 
> We are working on the fast-startup related development based on JDK11u. To
> facilitate community collaboration(sharing backports, bug fixing, patches,
> etc. across different companies/parties based on the repo), we are exploring
> creating the fork of JDK11u under the OpenJDK group [1]
> 
> 
> 
> Can anyone help with this, or is there a way/process we can follow to do
> that? - Your help or any input is much appreciated.
> 
> 
> 
> [1]  <https://github.com/openjdk> https://github.com/openjdk
> 
> 
> 
> Thanks!
> 
> Sanhong

From neugens at redhat.com  Tue Feb  9 12:51:18 2021
From: neugens at redhat.com (Mario Torre)
Date: Tue, 9 Feb 2021 13:51:18 +0100
Subject: Broken links between Jira and mercurial
Message-ID: <CAJwKKmb_BR6gF37Jb0r02cnHgnKzDw2oFuRjRWyzJKygyVhUQQ@mail.gmail.com>

Hi all!

I was going through some old bugs and noticed that the links to the
commit with the resolution is sometimes wrong and results in an error.

For example:

https://bugs.openjdk.java.net/browse/JDK-8131760

points to:

http://hg.openjdk.java.net/jdk9/client/jdk/rev/314ce60cae98

Which then results in "Mercurial repository not found"

Does anyone know why is that the case? I know I can work around it by
searching for the bug id in the commit but having the direct link is
usually faster.

Thanks!

Cheers,
Mario
-- 
Mario Torre
Manager, Software Engineering
Red Hat GmbH <https://www.redhat.com>
9704 A60C B4BE A8B8 0F30  9205 5D7E 4952 3F65 7898


From kevin.rushforth at oracle.com  Tue Feb  9 12:56:41 2021
From: kevin.rushforth at oracle.com (Kevin Rushforth)
Date: Tue, 9 Feb 2021 04:56:41 -0800
Subject: Broken links between Jira and mercurial
In-Reply-To: <CAJwKKmb_BR6gF37Jb0r02cnHgnKzDw2oFuRjRWyzJKygyVhUQQ@mail.gmail.com>
References: <CAJwKKmb_BR6gF37Jb0r02cnHgnKzDw2oFuRjRWyzJKygyVhUQQ@mail.gmail.com>
Message-ID: <b102176e-9064-102d-4602-fb826c481184@oracle.com>

This particular link was added manually, and points to a team repo 
(jdk9/client) that has been deleted as redundant. If you go to the 
actual Bug ID used to commit the change:

https://bugs.openjdk.java.net/browse/JDK-8143849

You will see a link to the commit in the main jdk9/jdk9 repo that you 
can follow:

http://hg.openjdk.java.net/jdk9/jdk9/jdk/rev/314ce60cae98

-- Kevin

On 2/9/2021 4:51 AM, Mario Torre wrote:
> Hi all!
>
> I was going through some old bugs and noticed that the links to the
> commit with the resolution is sometimes wrong and results in an error.
>
> For example:
>
> https://bugs.openjdk.java.net/browse/JDK-8131760
>
> points to:
>
> http://hg.openjdk.java.net/jdk9/client/jdk/rev/314ce60cae98
>
> Which then results in "Mercurial repository not found"
>
> Does anyone know why is that the case? I know I can work around it by
> searching for the bug id in the commit but having the direct link is
> usually faster.
>
> Thanks!
>
> Cheers,
> Mario


From Alan.Bateman at oracle.com  Tue Feb  9 12:59:44 2021
From: Alan.Bateman at oracle.com (Alan Bateman)
Date: Tue, 9 Feb 2021 12:59:44 +0000
Subject: Broken links between Jira and mercurial
In-Reply-To: <CAJwKKmb_BR6gF37Jb0r02cnHgnKzDw2oFuRjRWyzJKygyVhUQQ@mail.gmail.com>
References: <CAJwKKmb_BR6gF37Jb0r02cnHgnKzDw2oFuRjRWyzJKygyVhUQQ@mail.gmail.com>
Message-ID: <bd146d98-ea30-82a9-86c3-65bfece68317@oracle.com>


On 09/02/2021 12:51, Mario Torre wrote:
> Hi all!
>
> I was going through some old bugs and noticed that the links to the
> commit with the resolution is sometimes wrong and results in an error.
>
> For example:
>
> https://bugs.openjdk.java.net/browse/JDK-8131760
>
> points to:
>
> http://hg.openjdk.java.net/jdk9/client/jdk/rev/314ce60cae98
>
> Which then results in "Mercurial repository not found"
>
jdk9/client was a staging repo and may have been archived to save space. 
The change-sets will have been pushed to the master (jdk9/jdk9 in this 
case) and surprised there isn't another comment in JIRA so that link. In 
this one at least, it should work if you replace "client" with "jdk9".

-Alan

From neugens at redhat.com  Tue Feb  9 14:54:34 2021
From: neugens at redhat.com (Mario Torre)
Date: Tue, 9 Feb 2021 15:54:34 +0100
Subject: Broken links between Jira and mercurial
In-Reply-To: <bd146d98-ea30-82a9-86c3-65bfece68317@oracle.com>
References: <CAJwKKmb_BR6gF37Jb0r02cnHgnKzDw2oFuRjRWyzJKygyVhUQQ@mail.gmail.com>
 <bd146d98-ea30-82a9-86c3-65bfece68317@oracle.com>
Message-ID: <CAJwKKmbhE8nt7XTw4XdoDLJt823YcAAds7RayoSCJAx=N=-HLQ@mail.gmail.com>

Thanks Alan and Kevin,

I think all the examples of similarly broken links I've seen were also
on the staging repos, so this makes sense, although I was surprised to
see jdk9/client disappear.

Cheers,
Mario

On Tue, Feb 9, 2021 at 2:00 PM Alan Bateman <Alan.Bateman at oracle.com> wrote:
>
>
> On 09/02/2021 12:51, Mario Torre wrote:
> > Hi all!
> >
> > I was going through some old bugs and noticed that the links to the
> > commit with the resolution is sometimes wrong and results in an error.
> >
> > For example:
> >
> > https://bugs.openjdk.java.net/browse/JDK-8131760
> >
> > points to:
> >
> > http://hg.openjdk.java.net/jdk9/client/jdk/rev/314ce60cae98
> >
> > Which then results in "Mercurial repository not found"
> >
> jdk9/client was a staging repo and may have been archived to save space.
> The change-sets will have been pushed to the master (jdk9/jdk9 in this
> case) and surprised there isn't another comment in JIRA so that link. In
> this one at least, it should work if you replace "client" with "jdk9".
>
> -Alan
>


-- 
Mario Torre
Manager, Software Engineering
Red Hat GmbH <https://www.redhat.com>
9704 A60C B4BE A8B8 0F30  9205 5D7E 4952 3F65 7898


From jianglizhou at google.com  Wed Feb 10 03:42:46 2021
From: jianglizhou at google.com (Jiangli Zhou)
Date: Tue, 9 Feb 2021 19:42:46 -0800
Subject: create a fork under openjdk
In-Reply-To: <65ef2498-965f-efbe-48c4-3f5d8ed01d35@oracle.com>
References: <031901d6f4a5$c6a59590$53f0c0b0$@alibaba-inc.com>
 <b9a1c4b3-e797-89bf-2e67-8eef3bef6a5c@oracle.com>
 <066501d6f640$f3ca7580$db5f6080$@alibaba-inc.com>
 <CALrW1jz_7wqkbJdRps2_tCZrzQG30zPxUOY7M85v4+Ua4nraiQ@mail.gmail.com>
 <65ef2498-965f-efbe-48c4-3f5d8ed01d35@oracle.com>
Message-ID: <CALrW1jw+E=KfktWo-KqvryYYHp_G1UZZxhCQ6H6LG9c7fxq3=Q@mail.gmail.com>

Hi Claes,

Thanks a lot for the feedback (as always :)). We have been exploring
and trying to find a workable solution for this. We are legging 2+
years behind currently. So we are looking for guidance from OpenJDK
team on a workable approach for collaborating with others in the
community.

Best,
Jiangli

On Sat, Jan 30, 2021 at 5:56 AM Claes Redestad
<claes.redestad at oracle.com> wrote:
>
> Hi Jiangli,
>
> I will be happy to review changes when they are upstreamed to the JDK
> head, and implore you to make sure that the changes you are exploring
> can be upstreamed promptly to avoid potential conflicts with ongoing and
> existing enhancement work in either head or the endorsed projects.
>
> I will personally not participate in any discussion or collaborate with
> changes outside of the JDK mainline or a project that tracks the
> mainline repo (panama, valhalla, leyden...).
>
> Best regards
>
> /Claes
>
> On 2021-01-30 05:34, Jiangli Zhou wrote:
> > Hi Alan and Sanhong,
> >
> > We are in a similar situation as Alibaba. Our developers are using JDK 11
> > or are being migrated to JDK 11. JDK 11 is our primarily supported release.
> >
> > We are interested in working with Alibaba and others in the community on
> > the areas that Sanhong mentioned. We have started investigating and
> > prototyping
> > <http://cr.openjdk.java.net/~jiangli/Leyden/general_class_pre_initialization_core_mechanism/>
> > on JDK 11. Some of the work is currently being tested in production. Using
> > a JDK 11 based collaboration repository is a good starting point and avoids
> > the need/cost for merging (non-trivial) existing work into JDK head up
> > front before the work is ripe enough. It also helps us to validate any new
> > work in production as early as possible, while using JDK head based repo
> > (e.g. sandbox) would not achieve that in the short term future. Our end
> > goal is to rebase all the work from JDK 11 to JDK head, when it is mature
> > and ready for contributing back (with review and approval by reviewers) to
> > the mainline and potentially feed into the Leyden project.
> >
> > Best,
> > Jiangli
> >
> >
> > On Fri, Jan 29, 2021 at 5:18 AM ???(??) <sanhong.lsh at alibaba-inc.com> wrote:
> >
> >> Hi Alan,
> >> Thanks for your reply.
> >> Let me elaborate a little bit on this -
> >> 1. Our cloud workloads are running on the top of JDK11(hard to keep them at
> >> the head version due to enormous upgrade cost).
> >> 2. Based on these real use-cases,  we want to do the startup-related
> >> enhancements(neither exploration nor prototyping, just resolving the actual
> >> requirements) on JDK11
> >> - At this moment, we are considering AppCDS(+upstream backports),
> >> pre-initialization, and more optimization, e.g., AOT, in the future if
> >> better for facilitating startup.
> >> 3. We plan to start with JDK11, but I think we can contribute them(or some
> >> of them) up to the master later.
> >>
> >> Thanks!
> >> Sanhong
> >> -----????-----
> >> ???: Alan Bateman <Alan.Bateman at oracle.com>
> >> ????: 2021?1?27? 20:45
> >> ???: ???(??) <sanhong.lsh at alibaba-inc.com>; discuss at openjdk.java.net
> >> ??: Re: create a fork under openjdk
> >>
> >> On 27/01/2021 12:13, ???(??) wrote:
> >>> Hello,
> >>>
> >>> We are working on the fast-startup related development based on
> >>> JDK11u. To facilitate community collaboration(sharing backports, bug
> >>> fixing, patches, etc. across different companies/parties based on the
> >>> repo), we are exploring creating the fork of JDK11u under the OpenJDK
> >>> group [1]
> >>>
> >>>
> >>>
> >>> Can anyone help with this, or is there a way/process we can follow to
> >>> do that? - Your help or any input is much appreciated.
> >>
> >> Is this a collection of small improvements that you are looking to
> >> upstream?
> >> Is there any reason not to work with a fork from the main line as it
> >> already
> >> has a lot of startup improvements since JDK 11. The jdk-sandbox repo [1]
> >> and
> >> existing mailing lists could be used in that case. Some of the JEPs in
> >> recent releases have accumulated changes in the sandbox during their
> >> development.
> >>
> >> Or maybe you have a larger multi-year project in mind? OpenJDK has Projects
> >> [2] for collaborative efforts. Projects will usually involve exploration
> >> and
> >> prototyping. Is that the sort of thing you had in mind?
> >> It might be useful to expand a bit on what you have in mind as it is hard
> >> to
> >> know if this is about upstreaming changes or an effort that will explore
> >> many directions.
> >>
> >> -Alan
> >>
> >> [1] https://github.com/openjdk/jdk-sandbox
> >> [2] https://openjdk.java.net/projects
> >>
> >>

From jianglizhou at google.com  Wed Feb 10 03:56:39 2021
From: jianglizhou at google.com (Jiangli Zhou)
Date: Tue, 9 Feb 2021 19:56:39 -0800
Subject: create a fork under openjdk
In-Reply-To: <CALKUemzyFgF4VKK9UiDRNawaM+CNwvgPxiodgKkZJS-KDej8mQ@mail.gmail.com>
References: <031901d6f4a5$c6a59590$53f0c0b0$@alibaba-inc.com>
 <CALKUemzyFgF4VKK9UiDRNawaM+CNwvgPxiodgKkZJS-KDej8mQ@mail.gmail.com>
Message-ID: <CALrW1jwGgSmbbDSjkx+8MGbPaSJo68LhM48e2R=FAh6v7zVDMQ@mail.gmail.com>

Liam explored using CRIU for taking process snapshots for Javac in the
past. His finding (based on preliminary measurements) indicated the
image size could have a heavier impact on startup performance, and
worse startup time could be observed due to larger images. The
environment was a cloud build & testing environment.

We want to understand potential security issues associated with taking
and storing snapshots of live Java processes in production
environments.

Best,
Jiangli

On Thu, Feb 4, 2021 at 9:16 AM Christine Flood <chf at redhat.com> wrote:
>
> Have you guys looked at CRIU at all?  If you are running on Linux it gives
> you the capability of firing up a warmed up Java application in minutes.
>
> For example the TODO app (https://github.com/cescoffier/quarkus-todo-app)
> takes 1.837 seconds to start but only  0.11 seconds to restore.
>
> I've attached a console log showing the commands I used.
>
> If you want greater control, I've written a Java API you can tinker with:
> https://github.com/chflood/JavaCriuJar.
>
> Unfortunately this is a Linux only capability.
>
>
> Christine
>
>
>
>
> On Wed, Jan 27, 2021 at 7:14 AM ???(??) <sanhong.lsh at alibaba-inc.com> wrote:
>
> > Hello,
> >
> > We are working on the fast-startup related development based on JDK11u. To
> > facilitate community collaboration(sharing backports, bug fixing, patches,
> > etc. across different companies/parties based on the repo), we are
> > exploring
> > creating the fork of JDK11u under the OpenJDK group [1]
> >
> >
> >
> > Can anyone help with this, or is there a way/process we can follow to do
> > that? - Your help or any input is much appreciated.
> >
> >
> >
> > [1]  <https://github.com/openjdk> https://github.com/openjdk
> >
> >
> >
> > Thanks!
> >
> > Sanhong
> >
> >
> >
> >

From akozlov at azul.com  Thu Feb 11 16:08:25 2021
From: akozlov at azul.com (Anton Kozlov)
Date: Thu, 11 Feb 2021 19:08:25 +0300
Subject: =?UTF-8?B?UmU6IOetlOWkjTogY3JlYXRlIGEgZm9yayB1bmRlciBvcGVuamRr?=
In-Reply-To: <00c501d6feb1$a3408770$e9c19650$@alibaba-inc.com>
References: <031901d6f4a5$c6a59590$53f0c0b0$@alibaba-inc.com>
 <CALKUemzyFgF4VKK9UiDRNawaM+CNwvgPxiodgKkZJS-KDej8mQ@mail.gmail.com>
 <00c501d6feb1$a3408770$e9c19650$@alibaba-inc.com>
Message-ID: <afa6af14-f47d-9b62-5931-147046c9547c@azul.com>

(Continuing this little off-topic)

On 09.02.2021 10:03, ???(??) wrote:
> The challenge of using CRIU is we can not handle the state migration
> of the java application.
> 
> The situation will become more worse when the application evolves
> forward, and we don?t have a good mechanism(may consider a middleware
> framework?) to enforce the ?stateless constraints? for their
> development.

State management and stateless constraints are the main objectives of the
Coordinated Restore at Checkpoint project [1].  It suggests java itself be a
foundation for such tasks.

One part of the state are resources external to the Java process.  These are
never allowed in the image, it is checked in runtime when the image is created.
It can be used as a part of the development process, so once you succeeded to
create an image of an application in a certain state, you can be sure running
from the image will succeed (or at least will not fail because some external
resource is gone).

Another part is a java application state.  CRaC provides a mechanism for user
code to manage the state.  It is possible before the image is created and after
the java has started from the image.  This enables the application state update
that cannot be updated automatically by the CRaC.

The CRaC is far from complete, but even in the current form, it enables useful
applications.  Demos were done with network services, but CRaC is not limited
to them.

[1] https://mail.openjdk.java.net/pipermail/discuss/2020-September/005594.html

Thanks,
Anton


From akozlov at azul.com  Thu Feb 11 16:19:21 2021
From: akozlov at azul.com (Anton Kozlov)
Date: Thu, 11 Feb 2021 19:19:21 +0300
Subject: create a fork under openjdk
In-Reply-To: <CALrW1jwGgSmbbDSjkx+8MGbPaSJo68LhM48e2R=FAh6v7zVDMQ@mail.gmail.com>
References: <031901d6f4a5$c6a59590$53f0c0b0$@alibaba-inc.com>
 <CALKUemzyFgF4VKK9UiDRNawaM+CNwvgPxiodgKkZJS-KDej8mQ@mail.gmail.com>
 <CALrW1jwGgSmbbDSjkx+8MGbPaSJo68LhM48e2R=FAh6v7zVDMQ@mail.gmail.com>
Message-ID: <47fb3b6a-df56-e98b-5921-46662bb388b7@azul.com>



On 10.02.2021 06:56, Jiangli Zhou wrote:
> Liam explored using CRIU for taking process snapshots for Javac in the
> past. 

Interesting! So such snapshot could be a drop-in replacement of javac but
warmed-up for faster compilations.

> His finding (based on preliminary measurements) indicated the
> image size could have a heavier impact on startup performance, and
> worse startup time could be observed due to larger images. The
> environment was a cloud build & testing environment.

For now, we use the on-demand page-in of the image.  It makes start-up less
dependent on the image size, especially if some memory regions are not needed
immediately.  Please take a look at graphs [1]. 

[1] https://github.com/crac/docs

Thanks,
Anton


From sanhong.lsh at alibaba-inc.com  Fri Feb 12 04:16:28 2021
From: sanhong.lsh at alibaba-inc.com (=?UTF-8?B?5p2O5LiJ57qiKOS4iee6oik=?=)
Date: Fri, 12 Feb 2021 12:16:28 +0800
Subject: =?UTF-8?B?562U5aSNOiDnrZTlpI06IGNyZWF0ZSBhIGZvcmsgdW5kZXIgb3Blbmpkaw==?=
In-Reply-To: <648998894.1020386.1612863548859.JavaMail.zimbra@u-pem.fr>
References: <031901d6f4a5$c6a59590$53f0c0b0$@alibaba-inc.com>
 <CALKUemzyFgF4VKK9UiDRNawaM+CNwvgPxiodgKkZJS-KDej8mQ@mail.gmail.com>
 <00c501d6feb1$a3408770$e9c19650$@alibaba-inc.com>
 <648998894.1020386.1612863548859.JavaMail.zimbra@u-pem.fr>
Message-ID: <02dc01d700f5$d627d080$82777180$@alibaba-inc.com>

Hi R?mi,
The next run requires the 'clean' states in the previous snapshot.
However, the challenge is that even the application owners cannot clearly decide when the application will reach a 'clean' state(due to the code/dependency complexity).
Their application may reply on some 3rd party libraries(e.g., netty, spring),   or some parts/services are developed by different teams.   
In these external code dependencies,  we cannot tell how many state have been maintained, which should be saved(or skipped) before making a snapshot.
If some 'runtime dependent' states have been saved in the snapshot, for example:
- unique message-id for client/server communication(can be only used for this channel talk).
- some on-the-fly system information, e.g System.currentTimeMillis

These states could bring logic error for the next run forked from the previous snapshot.

Thanks!
Sanhong
-----????-----
???: Remi Forax <forax at univ-mlv.fr> 
????: 2021?2?9? 17:39
???: ???(??) <sanhong.lsh at alibaba-inc.com>
??: Christine Flood <chf at redhat.com>; discuss <discuss at openjdk.java.net>
??: Re: ??: create a fork under openjdk



----- Mail original -----
> De: "???(??)" <sanhong.lsh at alibaba-inc.com>
> ?: "Christine Flood" <chf at redhat.com>
> Cc: "discuss" <discuss at openjdk.java.net>
> Envoy?: Mardi 9 F?vrier 2021 08:03:15
> Objet: ??: create a fork under openjdk

> Hi Christine,
> 
> Thanks for your comments  - the challenge of using CRIU is we can not 
> handle the state migration of the java application.
> 
> The situation will become more worse when the application evolves 
> forward, and we don?t have a good mechanism(may consider a middleware 
> framework?) to enforce the ?stateless constraints? for their development.
> 
> 
> 
> Thanks!
> 
> Sanhong


Hi Sanhong,
this seems orthogonal, why do you want state migration on top of CRIU ?

You can use CRIU just as a way to start your application in steady state skipping the warmup phase.
So you build your app, then you run it with fake traffic then save the states with CRIU.
If you update the app, instead of trying to migrate states, you just rebuild it, etc.

regards,
R?mi

> 
> ???: Christine Flood <chf at redhat.com>
> ????: 2021?2?5? 1:16
> ???: ???(??) <sanhong.lsh at alibaba-inc.com>
> ??: discuss at openjdk.java.net
> ??: Re: create a fork under openjdk
> 
> 
> 
> Have you guys looked at CRIU at all?  If you are running on Linux it 
> gives you the capability of firing up a warmed up Java application in minutes.
> 
> 
> 
> For example the TODO app 
> (https://github.com/cescoffier/quarkus-todo-app) takes
> 1.837 seconds to start but only  0.11 seconds to restore.
> 
> 
> 
> I've attached a console log showing the commands I used.
> 
> 
> 
> If you want greater control, I've written a Java API you can tinker with:
> https://github.com/chflood/JavaCriuJar.
> 
> 
> 
> Unfortunately this is a Linux only capability.
> 
> 
> 
> 
> 
> Christine
> 
> 
> 
> 
> 
> 
> 
> 
> 
> On Wed, Jan 27, 2021 at 7:14 AM ???(??) <sanhong.lsh at alibaba-inc.com 
> <mailto:sanhong.lsh at alibaba-inc.com> > wrote:
> 
> Hello,
> 
> We are working on the fast-startup related development based on 
> JDK11u. To facilitate community collaboration(sharing backports, bug 
> fixing, patches, etc. across different companies/parties based on the 
> repo), we are exploring creating the fork of JDK11u under the OpenJDK 
> group [1]
> 
> 
> 
> Can anyone help with this, or is there a way/process we can follow to 
> do that? - Your help or any input is much appreciated.
> 
> 
> 
> [1]  <https://github.com/openjdk> https://github.com/openjdk
> 
> 
> 
> Thanks!
> 
> Sanhong


From chf at redhat.com  Wed Feb 17 17:51:34 2021
From: chf at redhat.com (Christine Flood)
Date: Wed, 17 Feb 2021 12:51:34 -0500
Subject: create a fork under openjdk
In-Reply-To: <CALrW1jwGgSmbbDSjkx+8MGbPaSJo68LhM48e2R=FAh6v7zVDMQ@mail.gmail.com>
References: <031901d6f4a5$c6a59590$53f0c0b0$@alibaba-inc.com>
 <CALKUemzyFgF4VKK9UiDRNawaM+CNwvgPxiodgKkZJS-KDej8mQ@mail.gmail.com>
 <CALrW1jwGgSmbbDSjkx+8MGbPaSJo68LhM48e2R=FAh6v7zVDMQ@mail.gmail.com>
Message-ID: <CALKUemyMFshkZ9qOpQTkfKT2L0Co93BQ36-QrMtt4XeSCUjGYQ@mail.gmail.com>

I would be very interested in reproducing a test case where criu startup is
slower than jvm class loading and warm up for anything more than a trivial
application and even then I'd still be curious.  I have not seen that in
any of my experiments.   One optimization that we are working on is having
a full gc and a heap shrinkage to just the working set prior to
checkpointing.  That might help with image size issues.

The security issues as I understand them are:

1) Active secrets, like cached passwords
2) Inactive secrets, like cached passwords which haven't been gc'd yet
3) Address layout consistency

I would argue that programs which are responsible for things like active
secrets should be able to address this issue through save and restore
hooks.

Inactive secrets will be handled by a full gc and heap shrink before the
checkpoint.

This leaves us with restored images having identical heap layouts.  This
can be a great benefit.  I have seen cases where the kernel is smart enough
to share the underlying pages.  However it can be a security risk.  If this
is an important issue we can look at randomizing the java heap upon
restore.  If we did a fullgc and tight compression of the heap before
checkpointing we have all the information we need to do a fast random
permutation on restore.  Please let me know if this is an important issue.


The nuisance issues are:
1) Things like processor count changed so work stealing queues are no
longer working efficiently
2) If you are timing an operation it may look like something that should
take ms is actually taking days because it started before the checkpoint
and finished after the restore.

We are working on restoring the jvm with appropriately sized work stealing
queues.  For GC this should be easy since we forced a full gc before saving
and the queues would be empty.
For everything else it would require copying the work elements into the new
queues and performing some bookkeeping.

I don't have an answer to the timing issue, but I see this as a nuisance
and not a show stopper.


Christine



On Tue, Feb 9, 2021 at 10:56 PM Jiangli Zhou <jianglizhou at google.com> wrote:

> Liam explored using CRIU for taking process snapshots for Javac in the
> past. His finding (based on preliminary measurements) indicated the
> image size could have a heavier impact on startup performance, and
> worse startup time could be observed due to larger images. The
> environment was a cloud build & testing environment.
>
> We want to understand potential security issues associated with taking
> and storing snapshots of live Java processes in production
> environments.
>
> Best,
> Jiangli
>
> On Thu, Feb 4, 2021 at 9:16 AM Christine Flood <chf at redhat.com> wrote:
> >
> > Have you guys looked at CRIU at all?  If you are running on Linux it
> gives
> > you the capability of firing up a warmed up Java application in minutes.
> >
> > For example the TODO app (https://github.com/cescoffier/quarkus-todo-app
> )
> > takes 1.837 seconds to start but only  0.11 seconds to restore.
> >
> > I've attached a console log showing the commands I used.
> >
> > If you want greater control, I've written a Java API you can tinker with:
> > https://github.com/chflood/JavaCriuJar.
> >
> > Unfortunately this is a Linux only capability.
> >
> >
> > Christine
> >
> >
> >
> >
> > On Wed, Jan 27, 2021 at 7:14 AM ???(??) <sanhong.lsh at alibaba-inc.com>
> wrote:
> >
> > > Hello,
> > >
> > > We are working on the fast-startup related development based on
> JDK11u. To
> > > facilitate community collaboration(sharing backports, bug fixing,
> patches,
> > > etc. across different companies/parties based on the repo), we are
> > > exploring
> > > creating the fork of JDK11u under the OpenJDK group [1]
> > >
> > >
> > >
> > > Can anyone help with this, or is there a way/process we can follow to
> do
> > > that? - Your help or any input is much appreciated.
> > >
> > >
> > >
> > > [1]  <https://github.com/openjdk> https://github.com/openjdk
> > >
> > >
> > >
> > > Thanks!
> > >
> > > Sanhong
> > >
> > >
> > >
> > >
>
>

From fweimer at redhat.com  Wed Feb 17 18:28:49 2021
From: fweimer at redhat.com (Florian Weimer)
Date: Wed, 17 Feb 2021 19:28:49 +0100
Subject: create a fork under openjdk
In-Reply-To: <CALKUemyMFshkZ9qOpQTkfKT2L0Co93BQ36-QrMtt4XeSCUjGYQ@mail.gmail.com>
 (Christine Flood's message of "Wed, 17 Feb 2021 12:51:34 -0500")
References: <031901d6f4a5$c6a59590$53f0c0b0$@alibaba-inc.com>
 <CALKUemzyFgF4VKK9UiDRNawaM+CNwvgPxiodgKkZJS-KDej8mQ@mail.gmail.com>
 <CALrW1jwGgSmbbDSjkx+8MGbPaSJo68LhM48e2R=FAh6v7zVDMQ@mail.gmail.com>
 <CALKUemyMFshkZ9qOpQTkfKT2L0Co93BQ36-QrMtt4XeSCUjGYQ@mail.gmail.com>
Message-ID: <87lfbmsg4e.fsf@oldenburg.str.redhat.com>

* Christine Flood:

> This leaves us with restored images having identical heap layouts.  This
> can be a great benefit.  I have seen cases where the kernel is smart enough
> to share the underlying pages.  However it can be a security risk.  If this
> is an important issue we can look at randomizing the java heap upon
> restore.  If we did a fullgc and tight compression of the heap before
> checkpointing we have all the information we need to do a fast random
> permutation on restore.  Please let me know if this is an important issue.

There's plenty of bytecode mapped at predictable addresses, and even one
RWX mapping:

/proc/3566599/maps:800000000-800003000 rwxp 00001000 103:02 91670724                        /usr/lib/jvm/java-15-openjdk-15.0.2.0.7-0.rolling.fc33.x86_64/lib/server/classes.jsa
/proc/3566599/maps:7f943475a000-7f9434a4a000 rwxp 00000000 00:00 0 
/proc/3566599/maps:7f9434e9b000-7f943537b000 rwxp 00000000 00:00 0 
/proc/3566599/maps:7f943c2fa000-7f943c56a000 rwxp 00000000 00:00 0 
/proc/3566619/maps:800000000-800003000 rwxp 00001000 103:02 91670724                        /usr/lib/jvm/java-15-openjdk-15.0.2.0.7-0.rolling.fc33.x86_64/lib/server/classes.jsa
/proc/3566619/maps:7f443075a000-7f44309ca000 rwxp 00000000 00:00 0 
/proc/3566619/maps:7f4430e9b000-7f443110b000 rwxp 00000000 00:00 0 
/proc/3566619/maps:7f44382fa000-7f443856a000 rwxp 00000000 00:00 0 

I understand the concern, but Hotspot is simply not hardened against
these kinds of issues.  The focus is more on prevent complete exploit
categories by ensuring both spatial and temporal memory safety in the
first place.  Not on post-exploitation countermeasures that rarely work
reliably in practice anyway.

Thanks,
Florian