From shade at redhat.com  Fri Nov 18 13:50:30 2016
From: shade at redhat.com (Aleksey Shipilev)
Date: Fri, 18 Nov 2016 14:50:30 +0100
Subject: Repository?
Message-ID: <52f370b5-79a7-7b11-b77c-0c1314812e9e@redhat.com>

Hi,

It is very exciting to see the JDK 10 mailing list!

When can we expect open forests (or maybe a monorepo that was discussed
at jdk9-dev some time ago [1]) for JDK 10? :)

Thanks,
-Aleksey

[1] http://openjdk.java.net/jeps/296



From mark.reinhold at oracle.com  Fri Nov 18 15:56:03 2016
From: mark.reinhold at oracle.com (mark.reinhold at oracle.com)
Date: Fri, 18 Nov 2016 07:56:03 -0800
Subject: Repository?
In-Reply-To: <52f370b5-79a7-7b11-b77c-0c1314812e9e@redhat.com>
References: <52f370b5-79a7-7b11-b77c-0c1314812e9e@redhat.com>
Message-ID: <20161118075603.631406469@eggemoggin.niobe.net>

2016/11/18 5:50:30 -0800, Aleksey Shipilev <shade at redhat.com>:
> It is very exciting to see the JDK 10 mailing list!
> 
> When can we expect open forests (or maybe a monorepo that was discussed
> at jdk9-dev some time ago [1]) for JDK 10? :)

Working on it ... stay tuned.

- Mark

From joe.darcy at oracle.com  Fri Nov 18 16:33:25 2016
From: joe.darcy at oracle.com (joe darcy)
Date: Fri, 18 Nov 2016 08:33:25 -0800
Subject: Repository? -- How many lines of development?
In-Reply-To: <52f370b5-79a7-7b11-b77c-0c1314812e9e@redhat.com>
References: <52f370b5-79a7-7b11-b77c-0c1314812e9e@redhat.com>
Message-ID: <11f471fa-87a6-f6e3-86b2-05467c076a5a@oracle.com>

Hello,

On 11/18/2016 5:50 AM, Aleksey Shipilev wrote:
> Hi,
>
> It is very exciting to see the JDK 10 mailing list!
>
> When can we expect open forests (or maybe a monorepo that was discussed
> at jdk9-dev some time ago [1]) for JDK 10? :)
>
> Thanks,
> -Aleksey
>
> [1] http://openjdk.java.net/jeps/296

And thus will commence the first thread in jdk10-dev, how many lines of 
development where "line is development" means either a forest or a monorepo.

For a few reasons including not holding up the start of JDK 10 
development for further discussion about and administrative advancement 
of JEP 296 and to give more time to work on internal-only details of the 
repo consolidation (such as how the various closed repos are handled), 
the JDK 10 lines of development won't start out as monorepos. They will 
at least initially use the existing multi-repo structure as in JDK 9. 
However, we'll return to JEP 296 later in the release.

Regardless of many repos used for a line of the development, there is a 
larger question of how many lines of development to have. For JDK 10 I 
propose three lines of development:

* A master forest, serving the roles master and dev play today in 9.

With a few exceptions, in JDK 9 master was just time-delayed copy of dev 
so we can implement recording the  information about which set of 
sources correspond to a promoted build without using a whole other forest.

Rather than using a separate line of development for client-libs work as 
in 9, I think this should be done in the same line of development as all 
other libs work in 10.

* Single HotSpot forest.

Over of the course of JDK 9, the HotSpot team went from using multiple 
forests for their work to using a single one.

* Sandbox

The JDK 9 sandbox (http://hg.openjdk.java.net/jdk9/sandbox/) allows 
collaboration and publication of sources of small projects outside the 
main line of development. This ability should continue in JDK 10.

Comments?

Thanks,

-Joe

From joe.darcy at oracle.com  Mon Nov 28 17:55:15 2016
From: joe.darcy at oracle.com (joe darcy)
Date: Mon, 28 Nov 2016 09:55:15 -0800
Subject: Repository? -- How many lines of development?
In-Reply-To: <11f471fa-87a6-f6e3-86b2-05467c076a5a@oracle.com>
References: <52f370b5-79a7-7b11-b77c-0c1314812e9e@redhat.com>
	<11f471fa-87a6-f6e3-86b2-05467c076a5a@oracle.com>
Message-ID: <633ced12-ded6-29b4-e367-a26de70012b6@oracle.com>

Hello again,

Last call for comments, in summary the proposal is for JDK 10 to have 
three externally visible lines of development:

* master
* hotspot
* sandbox

If there are no objections by Nov. 30, I think we should go ahead with 
this arrangement.

Thanks,

-Joe


On 11/18/2016 8:33 AM, joe darcy wrote:
> Hello,
>
> On 11/18/2016 5:50 AM, Aleksey Shipilev wrote:
>> Hi,
>>
>> It is very exciting to see the JDK 10 mailing list!
>>
>> When can we expect open forests (or maybe a monorepo that was discussed
>> at jdk9-dev some time ago [1]) for JDK 10? :)
>>
>> Thanks,
>> -Aleksey
>>
>> [1] http://openjdk.java.net/jeps/296
>
> And thus will commence the first thread in jdk10-dev, how many lines 
> of development where "line is development" means either a forest or a 
> monorepo.
>
> For a few reasons including not holding up the start of JDK 10 
> development for further discussion about and administrative 
> advancement of JEP 296 and to give more time to work on internal-only 
> details of the repo consolidation (such as how the various closed 
> repos are handled), the JDK 10 lines of development won't start out as 
> monorepos. They will at least initially use the existing multi-repo 
> structure as in JDK 9. However, we'll return to JEP 296 later in the 
> release.
>
> Regardless of many repos used for a line of the development, there is 
> a larger question of how many lines of development to have. For JDK 10 
> I propose three lines of development:
>
> * A master forest, serving the roles master and dev play today in 9.
>
> With a few exceptions, in JDK 9 master was just time-delayed copy of 
> dev so we can implement recording the  information about which set of 
> sources correspond to a promoted build without using a whole other 
> forest.
>
> Rather than using a separate line of development for client-libs work 
> as in 9, I think this should be done in the same line of development 
> as all other libs work in 10.
>
> * Single HotSpot forest.
>
> Over of the course of JDK 9, the HotSpot team went from using multiple 
> forests for their work to using a single one.
>
> * Sandbox
>
> The JDK 9 sandbox (http://hg.openjdk.java.net/jdk9/sandbox/) allows 
> collaboration and publication of sources of small projects outside the 
> main line of development. This ability should continue in JDK 10.
>
> Comments?
>
> Thanks,
>
> -Joe


From stefan.karlsson at oracle.com  Mon Nov 28 19:01:59 2016
From: stefan.karlsson at oracle.com (Stefan Karlsson)
Date: Mon, 28 Nov 2016 20:01:59 +0100
Subject: Repository? -- How many lines of development?
In-Reply-To: <633ced12-ded6-29b4-e367-a26de70012b6@oracle.com>
References: <52f370b5-79a7-7b11-b77c-0c1314812e9e@redhat.com>
	<11f471fa-87a6-f6e3-86b2-05467c076a5a@oracle.com>
	<633ced12-ded6-29b4-e367-a26de70012b6@oracle.com>
Message-ID: <7025d324-68dd-aad7-1f2b-22b5b1836d11@oracle.com>

Hi Joe,

On 2016-11-28 18:55, joe darcy wrote:
> Hello again,
>
> Last call for comments, in summary the proposal is for JDK 10 to have 
> three externally visible lines of development:
>
> * master
> * hotspot

Not an objection, but rather an observation.

This would mean that we have two things that people will refer to as the 
hotspot repository. The jdk10/hotspot top repository and the 
jdk10/hotspot/hotspot repository with the hotspot code. Maybe it would 
be better to stick with hs? Of course, this problem will go away when 
the repository consolidation is in place, so it might be a short-lived 
issue.


Thanks,
StefanK

> * sandbox
>
> If there are no objections by Nov. 30, I think we should go ahead with 
> this arrangement.
>
> Thanks,
>
> -Joe
>
>
> On 11/18/2016 8:33 AM, joe darcy wrote:
>> Hello,
>>
>> On 11/18/2016 5:50 AM, Aleksey Shipilev wrote:
>>> Hi,
>>>
>>> It is very exciting to see the JDK 10 mailing list!
>>>
>>> When can we expect open forests (or maybe a monorepo that was discussed
>>> at jdk9-dev some time ago [1]) for JDK 10? :)
>>>
>>> Thanks,
>>> -Aleksey
>>>
>>> [1] http://openjdk.java.net/jeps/296
>>
>> And thus will commence the first thread in jdk10-dev, how many lines 
>> of development where "line is development" means either a forest or a 
>> monorepo.
>>
>> For a few reasons including not holding up the start of JDK 10 
>> development for further discussion about and administrative 
>> advancement of JEP 296 and to give more time to work on internal-only 
>> details of the repo consolidation (such as how the various closed 
>> repos are handled), the JDK 10 lines of development won't start out 
>> as monorepos. They will at least initially use the existing 
>> multi-repo structure as in JDK 9. However, we'll return to JEP 296 
>> later in the release.
>>
>> Regardless of many repos used for a line of the development, there is 
>> a larger question of how many lines of development to have. For JDK 
>> 10 I propose three lines of development:
>>
>> * A master forest, serving the roles master and dev play today in 9.
>>
>> With a few exceptions, in JDK 9 master was just time-delayed copy of 
>> dev so we can implement recording the  information about which set of 
>> sources correspond to a promoted build without using a whole other 
>> forest.
>>
>> Rather than using a separate line of development for client-libs work 
>> as in 9, I think this should be done in the same line of development 
>> as all other libs work in 10.
>>
>> * Single HotSpot forest.
>>
>> Over of the course of JDK 9, the HotSpot team went from using 
>> multiple forests for their work to using a single one.
>>
>> * Sandbox
>>
>> The JDK 9 sandbox (http://hg.openjdk.java.net/jdk9/sandbox/) allows 
>> collaboration and publication of sources of small projects outside 
>> the main line of development. This ability should continue in JDK 10.
>>
>> Comments?
>>
>> Thanks,
>>
>> -Joe
>


From shade at redhat.com  Mon Nov 28 20:05:45 2016
From: shade at redhat.com (Aleksey Shipilev)
Date: Mon, 28 Nov 2016 21:05:45 +0100
Subject: Repository? -- How many lines of development?
In-Reply-To: <11f471fa-87a6-f6e3-86b2-05467c076a5a@oracle.com>
References: <52f370b5-79a7-7b11-b77c-0c1314812e9e@redhat.com>
	<11f471fa-87a6-f6e3-86b2-05467c076a5a@oracle.com>
Message-ID: <e4e9d89f-9bbf-6679-5743-fd4f094f9515@redhat.com>

On 11/18/2016 05:33 PM, joe darcy wrote:
> On 11/18/2016 5:50 AM, Aleksey Shipilev wrote:
> Regardless of many repos used for a line of the development, there is a larger question of how many
> lines of development to have. For JDK 10 I propose three lines of development:
> 
> * A master forest, serving the roles master and dev play today in 9.
> 
> With a few exceptions, in JDK 9 master was just time-delayed copy of dev so we can implement
> recording the  information about which set of sources correspond to a promoted build without using a
> whole other forest.
> 
> Rather than using a separate line of development for client-libs work as in 9, I think this should
> be done in the same line of development as all other libs work in 10.
> 
> * Single HotSpot forest.
> 
> Over of the course of JDK 9, the HotSpot team went from using multiple forests for their work to
> using a single one.
> 
> * Sandbox
> 
> The JDK 9 sandbox (http://hg.openjdk.java.net/jdk9/sandbox/) allows collaboration and publication of
> sources of small projects outside the main line of development. This ability should continue in JDK 10.
> 
> Comments?

(silly me, haven't sent the reply)

This looks okay to me. As long as we have a forest to push pending JDK 10 changes into.

Thanks,
-Aleksey


From martinrb at google.com  Mon Nov 28 20:15:44 2016
From: martinrb at google.com (Martin Buchholz)
Date: Mon, 28 Nov 2016 12:15:44 -0800
Subject: Repository? -- How many lines of development?
In-Reply-To: <11f471fa-87a6-f6e3-86b2-05467c076a5a@oracle.com>
References: <52f370b5-79a7-7b11-b77c-0c1314812e9e@redhat.com>
	<11f471fa-87a6-f6e3-86b2-05467c076a5a@oracle.com>
Message-ID: <CA+kOe0_7=CmP9WbEGfHM_kNirjf1dzjaTo4Z_Rz0wR0svv+Vxg@mail.gmail.com>

On Fri, Nov 18, 2016 at 8:33 AM, joe darcy <joe.darcy at oracle.com> wrote:

> * A master forest, serving the roles master and dev play today in 9.
>
> With a few exceptions, in JDK 9 master was just time-delayed copy of dev
> so we can implement recording the  information about which set of sources
> correspond to a promoted build without using a whole other forest.
>
> Rather than using a separate line of development for client-libs work as
> in 9, I think this should be done in the same line of development as all
> other libs work in 10.
>

For many years, I've been advocating having a guaranteed always-working,
never regressing master and also always a place for developers to
submit-and-forget their (possibly slightly buggy) changes.  All regressions
that could be caught by a test are 100% guaranteed to be caught by a
competent trusted release engineer who is the only one ever moving changes
into the master forest.  Based on this idea, it seems essential to have
something like a jdk10-dev forest (it could also be implemented using
mercurial branches, but that would be a break with many decades of
tradition).

I notice today the message
http://mail.openjdk.java.net/pipermail/quality-discuss/2016-November/000596.html
where regressions have crept into a jdk9 build, which is disappointing.
The whole point of regression testing is to ensure that regressions don't
happen!  And I recall having that job myself back in 2005!

From neugens.limasoftware at gmail.com  Mon Nov 28 20:52:13 2016
From: neugens.limasoftware at gmail.com (Mario Torre)
Date: Mon, 28 Nov 2016 21:52:13 +0100
Subject: Repository? -- How many lines of development?
In-Reply-To: <11f471fa-87a6-f6e3-86b2-05467c076a5a@oracle.com>
References: <52f370b5-79a7-7b11-b77c-0c1314812e9e@redhat.com>
	<11f471fa-87a6-f6e3-86b2-05467c076a5a@oracle.com>
Message-ID: <CAGUMyaQUtugkLm3JfzHOBgB7mY0W9TxFYfXZgf4AVTfk3BLjiw@mail.gmail.com>

2016-11-18 17:33 GMT+01:00 joe darcy <joe.darcy at oracle.com>:
> Hello,
>
> On 11/18/2016 5:50 AM, Aleksey Shipilev wrote:
>>
>> Hi,
>>
>> It is very exciting to see the JDK 10 mailing list!
>>
>> When can we expect open forests (or maybe a monorepo that was discussed
>> at jdk9-dev some time ago [1]) for JDK 10? :)
>>
>> Thanks,
>> -Aleksey
>>
>> [1] http://openjdk.java.net/jeps/296
>
>
> And thus will commence the first thread in jdk10-dev, how many lines of
> development where "line is development" means either a forest or a monorepo.
>
> For a few reasons including not holding up the start of JDK 10 development
> for further discussion about and administrative advancement of JEP 296 and
> to give more time to work on internal-only details of the repo consolidation
> (such as how the various closed repos are handled), the JDK 10 lines of
> development won't start out as monorepos. They will at least initially use
> the existing multi-repo structure as in JDK 9. However, we'll return to JEP
> 296 later in the release.
>
> Regardless of many repos used for a line of the development, there is a
> larger question of how many lines of development to have. For JDK 10 I
> propose three lines of development:
>
> * A master forest, serving the roles master and dev play today in 9.
>
> With a few exceptions, in JDK 9 master was just time-delayed copy of dev so
> we can implement recording the  information about which set of sources
> correspond to a promoted build without using a whole other forest.
>
> Rather than using a separate line of development for client-libs work as in
> 9, I think this should be done in the same line of development as all other
> libs work in 10.
>
> * Single HotSpot forest.
>
> Over of the course of JDK 9, the HotSpot team went from using multiple
> forests for their work to using a single one.
>
> * Sandbox
>
> The JDK 9 sandbox (http://hg.openjdk.java.net/jdk9/sandbox/) allows
> collaboration and publication of sources of small projects outside the main
> line of development. This ability should continue in JDK 10.
>
> Comments?
>
> Thanks,
>
> -Joe

Looks a lot easier to manage and will surely create less confusion, so +1.

I agree with Stefan that perhaps it would be better to call the
hotspot "hs", although this seems to me a minor thing.

Cheers,
Mario

-- 
pgp key: http://subkeys.pgp.net/ PGP Key ID: 80F240CF
Fingerprint: BA39 9666 94EC 8B73 27FA  FC7C 4086 63E3 80F2 40CF

Java Champion - Blog: http://neugens.wordpress.com - Twitter: @neugens
Proud GNU Classpath developer: http://www.classpath.org/
OpenJDK: http://openjdk.java.net/projects/caciocavallo/

Please, support open standards:
http://endsoftpatents.org/

From david.holmes at oracle.com  Mon Nov 28 21:47:59 2016
From: david.holmes at oracle.com (David Holmes)
Date: Tue, 29 Nov 2016 07:47:59 +1000
Subject: Repository? -- How many lines of development?
In-Reply-To: <11f471fa-87a6-f6e3-86b2-05467c076a5a@oracle.com>
References: <52f370b5-79a7-7b11-b77c-0c1314812e9e@redhat.com>
	<11f471fa-87a6-f6e3-86b2-05467c076a5a@oracle.com>
Message-ID: <32589a52-5e9f-dc65-3c99-bd5f9c475b06@oracle.com>

Sorry Joe this got forgotten ...

On 19/11/2016 2:33 AM, joe darcy wrote:
> Hello,
>
> On 11/18/2016 5:50 AM, Aleksey Shipilev wrote:
>> Hi,
>>
>> It is very exciting to see the JDK 10 mailing list!
>>
>> When can we expect open forests (or maybe a monorepo that was discussed
>> at jdk9-dev some time ago [1]) for JDK 10? :)
>>
>> Thanks,
>> -Aleksey
>>
>> [1] http://openjdk.java.net/jeps/296
>
> And thus will commence the first thread in jdk10-dev, how many lines of
> development where "line is development" means either a forest or a
> monorepo.
>
> For a few reasons including not holding up the start of JDK 10
> development for further discussion about and administrative advancement
> of JEP 296 and to give more time to work on internal-only details of the
> repo consolidation (such as how the various closed repos are handled),
> the JDK 10 lines of development won't start out as monorepos. They will
> at least initially use the existing multi-repo structure as in JDK 9.
> However, we'll return to JEP 296 later in the release.
>
> Regardless of many repos used for a line of the development, there is a
> larger question of how many lines of development to have. For JDK 10 I
> propose three lines of development:
>
> * A master forest, serving the roles master and dev play today in 9.
>
> With a few exceptions, in JDK 9 master was just time-delayed copy of dev
> so we can implement recording the  information about which set of
> sources correspond to a promoted build without using a whole other forest.

I agree with the description, but how does someone get a stable snapshot 
of master at a given "level"? How will the tagging work?

I also think the name "master" (due to historical usage) suggests a 
level of stability that won't actually be present. Maybe it is better to 
call it "dev"?

David
-----

> Rather than using a separate line of development for client-libs work as
> in 9, I think this should be done in the same line of development as all
> other libs work in 10.
>
> * Single HotSpot forest.
>
> Over of the course of JDK 9, the HotSpot team went from using multiple
> forests for their work to using a single one.
>
> * Sandbox
>
> The JDK 9 sandbox (http://hg.openjdk.java.net/jdk9/sandbox/) allows
> collaboration and publication of sources of small projects outside the
> main line of development. This ability should continue in JDK 10.
>
> Comments?
>
> Thanks,
>
> -Joe

From joe.darcy at oracle.com  Mon Nov 28 22:07:59 2016
From: joe.darcy at oracle.com (Joseph D. Darcy)
Date: Mon, 28 Nov 2016 14:07:59 -0800
Subject: Repository? -- How many lines of development?
In-Reply-To: <32589a52-5e9f-dc65-3c99-bd5f9c475b06@oracle.com>
References: <52f370b5-79a7-7b11-b77c-0c1314812e9e@redhat.com>
	<11f471fa-87a6-f6e3-86b2-05467c076a5a@oracle.com>
	<32589a52-5e9f-dc65-3c99-bd5f9c475b06@oracle.com>
Message-ID: <583CAABF.6000102@oracle.com>

Hi David,

On 11/28/2016 1:47 PM, David Holmes wrote:
> Sorry Joe this got forgotten ...
>
> On 19/11/2016 2:33 AM, joe darcy wrote:
>> Hello,
>>
>> On 11/18/2016 5:50 AM, Aleksey Shipilev wrote:
>>> Hi,
>>>
>>> It is very exciting to see the JDK 10 mailing list!
>>>
>>> When can we expect open forests (or maybe a monorepo that was discussed
>>> at jdk9-dev some time ago [1]) for JDK 10? :)
>>>
>>> Thanks,
>>> -Aleksey
>>>
>>> [1] http://openjdk.java.net/jeps/296
>>
>> And thus will commence the first thread in jdk10-dev, how many lines of
>> development where "line is development" means either a forest or a
>> monorepo.
>>
>> For a few reasons including not holding up the start of JDK 10
>> development for further discussion about and administrative advancement
>> of JEP 296 and to give more time to work on internal-only details of the
>> repo consolidation (such as how the various closed repos are handled),
>> the JDK 10 lines of development won't start out as monorepos. They will
>> at least initially use the existing multi-repo structure as in JDK 9.
>> However, we'll return to JEP 296 later in the release.
>>
>> Regardless of many repos used for a line of the development, there is a
>> larger question of how many lines of development to have. For JDK 10 I
>> propose three lines of development:
>>
>> * A master forest, serving the roles master and dev play today in 9.
>>
>> With a few exceptions, in JDK 9 master was just time-delayed copy of dev
>> so we can implement recording the  information about which set of
>> sources correspond to a promoted build without using a whole other 
>> forest.
>
> I agree with the description, but how does someone get a stable 
> snapshot of master at a given "level"? How will the tagging work?

For the combined dev/master forest, the most recent integration tag will 
have the same stability guarantees we have today so "pull the most 
recent jdk-10+XYZ tag" to get a stable snapshot.

As an aside, for JDK 10 I'd also like to see promoted builds on a more 
frequent schedule than once a week.

>
> I also think the name "master" (due to historical usage) suggests a 
> level of stability that won't actually be present. Maybe it is better 
> to call it "dev"?
>

Much of the stability we see in master today is because the forest only 
changes once a week with the dev -> master integration ;-)

In a nutshell, the proposal is to replace tracking the known-good state 
in 9 via integrate dev -> master, tag in master, pull down to dev 
process today with just "tag known good in combined dev/master" in 10.

Cheers,

-Joe

From joe.darcy at oracle.com  Mon Nov 28 22:16:25 2016
From: joe.darcy at oracle.com (Joseph D. Darcy)
Date: Mon, 28 Nov 2016 14:16:25 -0800
Subject: Repository? -- How many lines of development?
In-Reply-To: <7025d324-68dd-aad7-1f2b-22b5b1836d11@oracle.com>
References: <52f370b5-79a7-7b11-b77c-0c1314812e9e@redhat.com>	<11f471fa-87a6-f6e3-86b2-05467c076a5a@oracle.com>	<633ced12-ded6-29b4-e367-a26de70012b6@oracle.com>
	<7025d324-68dd-aad7-1f2b-22b5b1836d11@oracle.com>
Message-ID: <583CACB9.7040008@oracle.com>

Hi Stefan,

On 11/28/2016 11:01 AM, Stefan Karlsson wrote:
> Hi Joe,
>
> On 2016-11-28 18:55, joe darcy wrote:
>> Hello again,
>>
>> Last call for comments, in summary the proposal is for JDK 10 to have 
>> three externally visible lines of development:
>>
>> * master
>> * hotspot
>
> Not an objection, but rather an observation.
>
> This would mean that we have two things that people will refer to as 
> the hotspot repository. The jdk10/hotspot top repository and the 
> jdk10/hotspot/hotspot repository with the hotspot code. Maybe it would 
> be better to stick with hs? Of course, this problem will go away when 
> the repository consolidation is in place, so it might be a short-lived 
> issue.

As JEP 296: Consolidate the JDK Forest into a Single Repository 
advances, we will return to discussions on whether to consolidate the 
repositories and, if consolidating, when to do so.

Thanks,

-Joe

From martinrb at google.com  Mon Nov 28 23:30:16 2016
From: martinrb at google.com (Martin Buchholz)
Date: Mon, 28 Nov 2016 15:30:16 -0800
Subject: Repository? -- How many lines of development?
In-Reply-To: <583CAABF.6000102@oracle.com>
References: <52f370b5-79a7-7b11-b77c-0c1314812e9e@redhat.com>
	<11f471fa-87a6-f6e3-86b2-05467c076a5a@oracle.com>
	<32589a52-5e9f-dc65-3c99-bd5f9c475b06@oracle.com>
	<583CAABF.6000102@oracle.com>
Message-ID: <CA+kOe0_6WaKaAkCYMpD6KwOp2XHGYF7_hYy2xeEa9OoGE5LEag@mail.gmail.com>

On Mon, Nov 28, 2016 at 2:07 PM, Joseph D. Darcy <joe.darcy at oracle.com>
wrote:

>
> For the combined dev/master forest, the most recent integration tag will
> have the same stability guarantees we have today so "pull the most recent
> jdk-10+XYZ tag" to get a stable snapshot.
>
>
But ... I want better than the stability guarantees we have today!

I want to obsolete those messages to quality-discuss with "new failures"
because it should not even be possible for master to get into a state where
regression tests have regressed (except for flakes or external
dependencies).


> As an aside, for JDK 10 I'd also like to see promoted builds on a more
> frequent schedule than once a week.
>

People do "continuous testing and integration" these days.  Set up your
integration pipeline so that it is always running.  The pipeline
automatically promotes changesets to master when all tests pass.  Easy!
Then every master changeset is equally stable to a "promoted build".

In principle, the same could be accomplished via running every single
commit through a jprt-like process, but I'm afraid that would be too
expensive.  Back in my day, full build + test took on the order of 10
hours, so changesets would need to be batched, or you need to throw more
hardware at the problem (give every changeset a dedicated machine for
testing?)

From martinrb at google.com  Mon Nov 28 23:36:27 2016
From: martinrb at google.com (Martin Buchholz)
Date: Mon, 28 Nov 2016 15:36:27 -0800
Subject: Repository? -- How many lines of development?
In-Reply-To: <583CAABF.6000102@oracle.com>
References: <52f370b5-79a7-7b11-b77c-0c1314812e9e@redhat.com>
	<11f471fa-87a6-f6e3-86b2-05467c076a5a@oracle.com>
	<32589a52-5e9f-dc65-3c99-bd5f9c475b06@oracle.com>
	<583CAABF.6000102@oracle.com>
Message-ID: <CA+kOe09v4pc6GbRhZo1v2gnvwxM6qVNEcQqKrhBOM=aSwXTjSw@mail.gmail.com>

On Mon, Nov 28, 2016 at 2:07 PM, Joseph D. Darcy <joe.darcy at oracle.com>
wrote:

> For the combined dev/master forest, the most recent integration tag will
> have the same stability guarantees we have today so "pull the most recent
> jdk-10+XYZ tag" to get a stable snapshot.
>

Another problem is not having any easy/automated way to get the latest
stable snapshot.  You have to look around, or tail -1 .hgtags to guess at
what might be the latest, because there is no single moving tag that
represents the desired stability level.  Why can't I

hg tupdate jdk-9-latest-build

From jcoomes at twitter.com  Mon Nov 28 23:51:59 2016
From: jcoomes at twitter.com (John Coomes)
Date: Mon, 28 Nov 2016 15:51:59 -0800
Subject: Repository? -- How many lines of development?
In-Reply-To: <583CAABF.6000102@oracle.com>
References: <52f370b5-79a7-7b11-b77c-0c1314812e9e@redhat.com>
	<11f471fa-87a6-f6e3-86b2-05467c076a5a@oracle.com>
	<32589a52-5e9f-dc65-3c99-bd5f9c475b06@oracle.com>
	<583CAABF.6000102@oracle.com>
Message-ID: <CAG2Byqmu9MFUUTWwrQ0UW4x=L8WWk6J4JvywW0PfseN95uY2aQ@mail.gmail.com>

On Mon, Nov 28, 2016 at 2:07 PM, Joseph D. Darcy <joe.darcy at oracle.com>
wrote:

> Hi David,
>
>
> On 11/28/2016 1:47 PM, David Holmes wrote:
>
>> Sorry Joe this got forgotten ...
>>
>> On 19/11/2016 2:33 AM, joe darcy wrote:
>>
>>> Hello,
>>>
>>> On 11/18/2016 5:50 AM, Aleksey Shipilev wrote:
>>>
>>>> Hi,
>>>>
>>>> It is very exciting to see the JDK 10 mailing list!
>>>>
>>>> When can we expect open forests (or maybe a monorepo that was discussed
>>>> at jdk9-dev some time ago [1]) for JDK 10? :)
>>>>
>>>> Thanks,
>>>> -Aleksey
>>>>
>>>> [1] http://openjdk.java.net/jeps/296
>>>>
>>>
>>> And thus will commence the first thread in jdk10-dev, how many lines of
>>> development where "line is development" means either a forest or a
>>> monorepo.
>>>
>>> For a few reasons including not holding up the start of JDK 10
>>> development for further discussion about and administrative advancement
>>> of JEP 296 and to give more time to work on internal-only details of the
>>> repo consolidation (such as how the various closed repos are handled),
>>> the JDK 10 lines of development won't start out as monorepos. They will
>>> at least initially use the existing multi-repo structure as in JDK 9.
>>> However, we'll return to JEP 296 later in the release.
>>>
>>> Regardless of many repos used for a line of the development, there is a
>>> larger question of how many lines of development to have. For JDK 10 I
>>> propose three lines of development:
>>>
>>> * A master forest, serving the roles master and dev play today in 9.
>>>
>>> With a few exceptions, in JDK 9 master was just time-delayed copy of dev
>>> so we can implement recording the  information about which set of
>>> sources correspond to a promoted build without using a whole other
>>> forest.
>>>
>>
>> I agree with the description, but how does someone get a stable snapshot
>> of master at a given "level"? How will the tagging work?
>>
>
> For the combined dev/master forest, the most recent integration tag will
> have the same stability guarantees we have today so "pull the most recent
> jdk-10+XYZ tag" to get a stable snapshot.
>
> As an aside, for JDK 10 I'd also like to see promoted builds on a more
> frequent schedule than once a week.
>
>
>> I also think the name "master" (due to historical usage) suggests a level
>> of stability that won't actually be present. Maybe it is better to call it
>> "dev"?
>>
>>
> Much of the stability we see in master today is because the forest only
> changes once a week with the dev -> master integration ;-)
>
> In a nutshell, the proposal is to replace tracking the known-good state in
> 9 via integrate dev -> master, tag in master, pull down to dev process
> today with just "tag known good in combined dev/master" in 10.
>
>
Hi Joe,

This would make cloning or pulling the "last known good" state more
difficult.  In jdk9, I can always clone/pull from a fixed location
(http://.../jdk9/jdk9).
With the proposal, I would have to first clone/pull everything, then
examine the tags and sort them (which programatically means decoding
release and build numbers, ugh), find the right tag, then pull/update to
get the working dir in the right state.

Since my clone would have all the changesets, including those beyond the
"last known good" state, it also makes the use of 'tip' and 'default' error
prone.  If I do some archaeology and run "hg update <some_older_rev>", then
when I'm done, I can't go back to "last known good" with a simple "hg
update default".  I have to look at the tags again, sort them, and so on.

I think it's more important that cloning or pulling the "last known good"
state be simple, rather than promoting a build.

Mercurial bookmarks might be a compromise solution - you can have a
bookmark with a fixed name (e.g., "stable", "jdk10-stable",
"last-promoted", whatever) that is updated each time a build is promoted.
With validation from jcheck, it could be made reasonably safe.

-John

From joe.darcy at oracle.com  Tue Nov 29 00:50:32 2016
From: joe.darcy at oracle.com (Joseph D. Darcy)
Date: Mon, 28 Nov 2016 16:50:32 -0800
Subject: Repository? -- How many lines of development?
In-Reply-To: <CA+kOe0_6WaKaAkCYMpD6KwOp2XHGYF7_hYy2xeEa9OoGE5LEag@mail.gmail.com>
References: <52f370b5-79a7-7b11-b77c-0c1314812e9e@redhat.com>
	<11f471fa-87a6-f6e3-86b2-05467c076a5a@oracle.com>
	<32589a52-5e9f-dc65-3c99-bd5f9c475b06@oracle.com>
	<583CAABF.6000102@oracle.com>
	<CA+kOe0_6WaKaAkCYMpD6KwOp2XHGYF7_hYy2xeEa9OoGE5LEag@mail.gmail.com>
Message-ID: <583CD0D8.3000105@oracle.com>

On 11/28/2016 3:30 PM, Martin Buchholz wrote:
>
>
> On Mon, Nov 28, 2016 at 2:07 PM, Joseph D. Darcy <joe.darcy at oracle.com 
> <mailto:joe.darcy at oracle.com>> wrote:
>
>
>     For the combined dev/master forest, the most recent integration
>     tag will have the same stability guarantees we have today so "pull
>     the most recent jdk-10+XYZ tag" to get a stable snapshot.
>
>
> But ... I want better than the stability guarantees we have today!

Over the course of JDK 9 we've made marked improvements to the 
regression test stability, at least on metrics we track internally on 
internal systems, so unfortunately am I not at liberty to share them. 
Briefly, the tier 1 regression tests in promoted builds are very stable, 
the tier 2 tests somewhat less stable, and so on.

>
> I want to obsolete those messages to quality-discuss with "new 
> failures" because it should not even be possible for master to get 
> into a state where regression tests have regressed (except for flakes 
> or external dependencies).

I don't know if those particular failures are actually in the promoted 
build or just an artifact of how that particular batch of tests was run. 
IIRC, we did not see a corresponding set of failures before the integration.

>     As an aside, for JDK 10 I'd also like to see promoted builds on a
>     more frequent schedule than once a week.
>
>
> People do "continuous testing and integration" these days.  Set up 
> your integration pipeline so that it is always running.  The pipeline 
> automatically promotes changesets to master when all tests pass.  
> Easy!  Then every master changeset is equally stable to a "promoted 
> build".

One of the internal systems mentioned above is a CI system that run 
regression tests after a change is pushed. The approximate integration 
model is then to promote known-good states of sources as vetted by the 
CI system.

If problems are fixed soon after they are identified, then a post-push 
system gives good results with avoiding the need more complexity on the 
front end to manage a series of in-flight patches.

Cheers,

-Joe

From joe.darcy at oracle.com  Tue Nov 29 01:07:50 2016
From: joe.darcy at oracle.com (Joseph D. Darcy)
Date: Mon, 28 Nov 2016 17:07:50 -0800
Subject: Repository? -- How many lines of development?
In-Reply-To: <CAG2Byqmu9MFUUTWwrQ0UW4x=L8WWk6J4JvywW0PfseN95uY2aQ@mail.gmail.com>
References: <52f370b5-79a7-7b11-b77c-0c1314812e9e@redhat.com>
	<11f471fa-87a6-f6e3-86b2-05467c076a5a@oracle.com>
	<32589a52-5e9f-dc65-3c99-bd5f9c475b06@oracle.com>
	<583CAABF.6000102@oracle.com>
	<CAG2Byqmu9MFUUTWwrQ0UW4x=L8WWk6J4JvywW0PfseN95uY2aQ@mail.gmail.com>
Message-ID: <583CD4E6.2090703@oracle.com>

Hi John,

On 11/28/2016 3:51 PM, John Coomes wrote:
> On Mon, Nov 28, 2016 at 2:07 PM, Joseph D. Darcy <joe.darcy at oracle.com 
> <mailto:joe.darcy at oracle.com>> wrote:
>
>
>

[snip]

>     In a nutshell, the proposal is to replace tracking the known-good
>     state in 9 via integrate dev -> master, tag in master, pull down
>     to dev process today with just "tag known good in combined
>     dev/master" in 10.
>
>
> Hi Joe,
>
> This would make cloning or pulling the "last known good" state more 
> difficult.  In jdk9, I can always clone/pull from a fixed location 
> (http://.../jdk9/jdk9).  With the proposal, I would have to first 
> clone/pull everything, then examine the tags and sort them (which 
> programatically means decoding release and build numbers, ugh), find 
> the right tag, then pull/update to get the working dir in the right state.
>
> Since my clone would have all the changesets, including those beyond 
> the "last known good" state, it also makes the use of 'tip' and 
> 'default' error prone.  If I do some archaeology and run "hg update 
> <some_older_rev>", then when I'm done, I can't go back to "last known 
> good" with a simple "hg update default".  I have to look at the tags 
> again, sort them, and so on.
>
> I think it's more important that cloning or pulling the "last known 
> good" state be simple, rather than promoting a build.
>
> Mercurial bookmarks might be a compromise solution - you can have a 
> bookmark with a fixed name (e.g., "stable", "jdk10-stable", 
> "last-promoted", whatever) that is updated each time a build is 
> promoted. With validation from jcheck, it could be made reasonably safe.
>

I don't object to having a convenient way to get the last known good 
state of the sources, but I think the way we do this in 9, using a 
separate 700 MB+ Hg forest, has too much overhead to store a relative 
paucity of information.

In terms of evaluating alternatives, are there other ways you see to 
record this information in Mercurial?

Thanks,

-Joe


From philip.race at oracle.com  Tue Nov 29 22:02:53 2016
From: philip.race at oracle.com (Phil Race)
Date: Tue, 29 Nov 2016 14:02:53 -0800
Subject: Repository? -- How many lines of development?
In-Reply-To: <633ced12-ded6-29b4-e367-a26de70012b6@oracle.com>
References: <52f370b5-79a7-7b11-b77c-0c1314812e9e@redhat.com>
	<11f471fa-87a6-f6e3-86b2-05467c076a5a@oracle.com>
	<633ced12-ded6-29b4-e367-a26de70012b6@oracle.com>
Message-ID: <e0edc443-aaf8-2362-493d-cb0f0b94f8a7@oracle.com>

Hi Joe,

I need to register my concerns for SE client (the open part, not closed)
I don't think it is feasible to collapse the current lines of development
unless all our current testing done at PIT is able to be run for every push
and we are nowhere near that. Maybe even further away than we were before.

-phil.



On 11/28/2016 09:55 AM, joe darcy wrote:
> Hello again,
>
> Last call for comments, in summary the proposal is for JDK 10 to have 
> three externally visible lines of development:
>
> * master
> * hotspot
> * sandbox
>
> If there are no objections by Nov. 30, I think we should go ahead with 
> this arrangement.
>
> Thanks,
>
> -Joe
>
>
> On 11/18/2016 8:33 AM, joe darcy wrote:
>> Hello,
>>
>> On 11/18/2016 5:50 AM, Aleksey Shipilev wrote:
>>> Hi,
>>>
>>> It is very exciting to see the JDK 10 mailing list!
>>>
>>> When can we expect open forests (or maybe a monorepo that was discussed
>>> at jdk9-dev some time ago [1]) for JDK 10? :)
>>>
>>> Thanks,
>>> -Aleksey
>>>
>>> [1] http://openjdk.java.net/jeps/296
>>
>> And thus will commence the first thread in jdk10-dev, how many lines 
>> of development where "line is development" means either a forest or a 
>> monorepo.
>>
>> For a few reasons including not holding up the start of JDK 10 
>> development for further discussion about and administrative 
>> advancement of JEP 296 and to give more time to work on internal-only 
>> details of the repo consolidation (such as how the various closed 
>> repos are handled), the JDK 10 lines of development won't start out 
>> as monorepos. They will at least initially use the existing 
>> multi-repo structure as in JDK 9. However, we'll return to JEP 296 
>> later in the release.
>>
>> Regardless of many repos used for a line of the development, there is 
>> a larger question of how many lines of development to have. For JDK 
>> 10 I propose three lines of development:
>>
>> * A master forest, serving the roles master and dev play today in 9.
>>
>> With a few exceptions, in JDK 9 master was just time-delayed copy of 
>> dev so we can implement recording the  information about which set of 
>> sources correspond to a promoted build without using a whole other 
>> forest.
>>
>> Rather than using a separate line of development for client-libs work 
>> as in 9, I think this should be done in the same line of development 
>> as all other libs work in 10.
>>
>> * Single HotSpot forest.
>>
>> Over of the course of JDK 9, the HotSpot team went from using 
>> multiple forests for their work to using a single one.
>>
>> * Sandbox
>>
>> The JDK 9 sandbox (http://hg.openjdk.java.net/jdk9/sandbox/) allows 
>> collaboration and publication of sources of small projects outside 
>> the main line of development. This ability should continue in JDK 10.
>>
>> Comments?
>>
>> Thanks,
>>
>> -Joe
>