From shade at redhat.com Fri Nov 18 13:50:30 2016 From: shade at redhat.com (Aleksey Shipilev) Date: Fri, 18 Nov 2016 14:50:30 +0100 Subject: Repository? Message-ID: <52f370b5-79a7-7b11-b77c-0c1314812e9e@redhat.com> Hi, It is very exciting to see the JDK 10 mailing list! When can we expect open forests (or maybe a monorepo that was discussed at jdk9-dev some time ago [1]) for JDK 10? :) Thanks, -Aleksey [1] http://openjdk.java.net/jeps/296 From mark.reinhold at oracle.com Fri Nov 18 15:56:03 2016 From: mark.reinhold at oracle.com (mark.reinhold at oracle.com) Date: Fri, 18 Nov 2016 07:56:03 -0800 Subject: Repository? In-Reply-To: <52f370b5-79a7-7b11-b77c-0c1314812e9e@redhat.com> References: <52f370b5-79a7-7b11-b77c-0c1314812e9e@redhat.com> Message-ID: <20161118075603.631406469@eggemoggin.niobe.net> 2016/11/18 5:50:30 -0800, Aleksey Shipilev : > It is very exciting to see the JDK 10 mailing list! > > When can we expect open forests (or maybe a monorepo that was discussed > at jdk9-dev some time ago [1]) for JDK 10? :) Working on it ... stay tuned. - Mark From joe.darcy at oracle.com Fri Nov 18 16:33:25 2016 From: joe.darcy at oracle.com (joe darcy) Date: Fri, 18 Nov 2016 08:33:25 -0800 Subject: Repository? -- How many lines of development? In-Reply-To: <52f370b5-79a7-7b11-b77c-0c1314812e9e@redhat.com> References: <52f370b5-79a7-7b11-b77c-0c1314812e9e@redhat.com> Message-ID: <11f471fa-87a6-f6e3-86b2-05467c076a5a@oracle.com> Hello, On 11/18/2016 5:50 AM, Aleksey Shipilev wrote: > Hi, > > It is very exciting to see the JDK 10 mailing list! > > When can we expect open forests (or maybe a monorepo that was discussed > at jdk9-dev some time ago [1]) for JDK 10? :) > > Thanks, > -Aleksey > > [1] http://openjdk.java.net/jeps/296 And thus will commence the first thread in jdk10-dev, how many lines of development where "line is development" means either a forest or a monorepo. For a few reasons including not holding up the start of JDK 10 development for further discussion about and administrative advancement of JEP 296 and to give more time to work on internal-only details of the repo consolidation (such as how the various closed repos are handled), the JDK 10 lines of development won't start out as monorepos. They will at least initially use the existing multi-repo structure as in JDK 9. However, we'll return to JEP 296 later in the release. Regardless of many repos used for a line of the development, there is a larger question of how many lines of development to have. For JDK 10 I propose three lines of development: * A master forest, serving the roles master and dev play today in 9. With a few exceptions, in JDK 9 master was just time-delayed copy of dev so we can implement recording the information about which set of sources correspond to a promoted build without using a whole other forest. Rather than using a separate line of development for client-libs work as in 9, I think this should be done in the same line of development as all other libs work in 10. * Single HotSpot forest. Over of the course of JDK 9, the HotSpot team went from using multiple forests for their work to using a single one. * Sandbox The JDK 9 sandbox (http://hg.openjdk.java.net/jdk9/sandbox/) allows collaboration and publication of sources of small projects outside the main line of development. This ability should continue in JDK 10. Comments? Thanks, -Joe From joe.darcy at oracle.com Mon Nov 28 17:55:15 2016 From: joe.darcy at oracle.com (joe darcy) Date: Mon, 28 Nov 2016 09:55:15 -0800 Subject: Repository? -- How many lines of development? In-Reply-To: <11f471fa-87a6-f6e3-86b2-05467c076a5a@oracle.com> References: <52f370b5-79a7-7b11-b77c-0c1314812e9e@redhat.com> <11f471fa-87a6-f6e3-86b2-05467c076a5a@oracle.com> Message-ID: <633ced12-ded6-29b4-e367-a26de70012b6@oracle.com> Hello again, Last call for comments, in summary the proposal is for JDK 10 to have three externally visible lines of development: * master * hotspot * sandbox If there are no objections by Nov. 30, I think we should go ahead with this arrangement. Thanks, -Joe On 11/18/2016 8:33 AM, joe darcy wrote: > Hello, > > On 11/18/2016 5:50 AM, Aleksey Shipilev wrote: >> Hi, >> >> It is very exciting to see the JDK 10 mailing list! >> >> When can we expect open forests (or maybe a monorepo that was discussed >> at jdk9-dev some time ago [1]) for JDK 10? :) >> >> Thanks, >> -Aleksey >> >> [1] http://openjdk.java.net/jeps/296 > > And thus will commence the first thread in jdk10-dev, how many lines > of development where "line is development" means either a forest or a > monorepo. > > For a few reasons including not holding up the start of JDK 10 > development for further discussion about and administrative > advancement of JEP 296 and to give more time to work on internal-only > details of the repo consolidation (such as how the various closed > repos are handled), the JDK 10 lines of development won't start out as > monorepos. They will at least initially use the existing multi-repo > structure as in JDK 9. However, we'll return to JEP 296 later in the > release. > > Regardless of many repos used for a line of the development, there is > a larger question of how many lines of development to have. For JDK 10 > I propose three lines of development: > > * A master forest, serving the roles master and dev play today in 9. > > With a few exceptions, in JDK 9 master was just time-delayed copy of > dev so we can implement recording the information about which set of > sources correspond to a promoted build without using a whole other > forest. > > Rather than using a separate line of development for client-libs work > as in 9, I think this should be done in the same line of development > as all other libs work in 10. > > * Single HotSpot forest. > > Over of the course of JDK 9, the HotSpot team went from using multiple > forests for their work to using a single one. > > * Sandbox > > The JDK 9 sandbox (http://hg.openjdk.java.net/jdk9/sandbox/) allows > collaboration and publication of sources of small projects outside the > main line of development. This ability should continue in JDK 10. > > Comments? > > Thanks, > > -Joe From stefan.karlsson at oracle.com Mon Nov 28 19:01:59 2016 From: stefan.karlsson at oracle.com (Stefan Karlsson) Date: Mon, 28 Nov 2016 20:01:59 +0100 Subject: Repository? -- How many lines of development? In-Reply-To: <633ced12-ded6-29b4-e367-a26de70012b6@oracle.com> References: <52f370b5-79a7-7b11-b77c-0c1314812e9e@redhat.com> <11f471fa-87a6-f6e3-86b2-05467c076a5a@oracle.com> <633ced12-ded6-29b4-e367-a26de70012b6@oracle.com> Message-ID: <7025d324-68dd-aad7-1f2b-22b5b1836d11@oracle.com> Hi Joe, On 2016-11-28 18:55, joe darcy wrote: > Hello again, > > Last call for comments, in summary the proposal is for JDK 10 to have > three externally visible lines of development: > > * master > * hotspot Not an objection, but rather an observation. This would mean that we have two things that people will refer to as the hotspot repository. The jdk10/hotspot top repository and the jdk10/hotspot/hotspot repository with the hotspot code. Maybe it would be better to stick with hs? Of course, this problem will go away when the repository consolidation is in place, so it might be a short-lived issue. Thanks, StefanK > * sandbox > > If there are no objections by Nov. 30, I think we should go ahead with > this arrangement. > > Thanks, > > -Joe > > > On 11/18/2016 8:33 AM, joe darcy wrote: >> Hello, >> >> On 11/18/2016 5:50 AM, Aleksey Shipilev wrote: >>> Hi, >>> >>> It is very exciting to see the JDK 10 mailing list! >>> >>> When can we expect open forests (or maybe a monorepo that was discussed >>> at jdk9-dev some time ago [1]) for JDK 10? :) >>> >>> Thanks, >>> -Aleksey >>> >>> [1] http://openjdk.java.net/jeps/296 >> >> And thus will commence the first thread in jdk10-dev, how many lines >> of development where "line is development" means either a forest or a >> monorepo. >> >> For a few reasons including not holding up the start of JDK 10 >> development for further discussion about and administrative >> advancement of JEP 296 and to give more time to work on internal-only >> details of the repo consolidation (such as how the various closed >> repos are handled), the JDK 10 lines of development won't start out >> as monorepos. They will at least initially use the existing >> multi-repo structure as in JDK 9. However, we'll return to JEP 296 >> later in the release. >> >> Regardless of many repos used for a line of the development, there is >> a larger question of how many lines of development to have. For JDK >> 10 I propose three lines of development: >> >> * A master forest, serving the roles master and dev play today in 9. >> >> With a few exceptions, in JDK 9 master was just time-delayed copy of >> dev so we can implement recording the information about which set of >> sources correspond to a promoted build without using a whole other >> forest. >> >> Rather than using a separate line of development for client-libs work >> as in 9, I think this should be done in the same line of development >> as all other libs work in 10. >> >> * Single HotSpot forest. >> >> Over of the course of JDK 9, the HotSpot team went from using >> multiple forests for their work to using a single one. >> >> * Sandbox >> >> The JDK 9 sandbox (http://hg.openjdk.java.net/jdk9/sandbox/) allows >> collaboration and publication of sources of small projects outside >> the main line of development. This ability should continue in JDK 10. >> >> Comments? >> >> Thanks, >> >> -Joe > From shade at redhat.com Mon Nov 28 20:05:45 2016 From: shade at redhat.com (Aleksey Shipilev) Date: Mon, 28 Nov 2016 21:05:45 +0100 Subject: Repository? -- How many lines of development? In-Reply-To: <11f471fa-87a6-f6e3-86b2-05467c076a5a@oracle.com> References: <52f370b5-79a7-7b11-b77c-0c1314812e9e@redhat.com> <11f471fa-87a6-f6e3-86b2-05467c076a5a@oracle.com> Message-ID: On 11/18/2016 05:33 PM, joe darcy wrote: > On 11/18/2016 5:50 AM, Aleksey Shipilev wrote: > Regardless of many repos used for a line of the development, there is a larger question of how many > lines of development to have. For JDK 10 I propose three lines of development: > > * A master forest, serving the roles master and dev play today in 9. > > With a few exceptions, in JDK 9 master was just time-delayed copy of dev so we can implement > recording the information about which set of sources correspond to a promoted build without using a > whole other forest. > > Rather than using a separate line of development for client-libs work as in 9, I think this should > be done in the same line of development as all other libs work in 10. > > * Single HotSpot forest. > > Over of the course of JDK 9, the HotSpot team went from using multiple forests for their work to > using a single one. > > * Sandbox > > The JDK 9 sandbox (http://hg.openjdk.java.net/jdk9/sandbox/) allows collaboration and publication of > sources of small projects outside the main line of development. This ability should continue in JDK 10. > > Comments? (silly me, haven't sent the reply) This looks okay to me. As long as we have a forest to push pending JDK 10 changes into. Thanks, -Aleksey From martinrb at google.com Mon Nov 28 20:15:44 2016 From: martinrb at google.com (Martin Buchholz) Date: Mon, 28 Nov 2016 12:15:44 -0800 Subject: Repository? -- How many lines of development? In-Reply-To: <11f471fa-87a6-f6e3-86b2-05467c076a5a@oracle.com> References: <52f370b5-79a7-7b11-b77c-0c1314812e9e@redhat.com> <11f471fa-87a6-f6e3-86b2-05467c076a5a@oracle.com> Message-ID: On Fri, Nov 18, 2016 at 8:33 AM, joe darcy wrote: > * A master forest, serving the roles master and dev play today in 9. > > With a few exceptions, in JDK 9 master was just time-delayed copy of dev > so we can implement recording the information about which set of sources > correspond to a promoted build without using a whole other forest. > > Rather than using a separate line of development for client-libs work as > in 9, I think this should be done in the same line of development as all > other libs work in 10. > For many years, I've been advocating having a guaranteed always-working, never regressing master and also always a place for developers to submit-and-forget their (possibly slightly buggy) changes. All regressions that could be caught by a test are 100% guaranteed to be caught by a competent trusted release engineer who is the only one ever moving changes into the master forest. Based on this idea, it seems essential to have something like a jdk10-dev forest (it could also be implemented using mercurial branches, but that would be a break with many decades of tradition). I notice today the message http://mail.openjdk.java.net/pipermail/quality-discuss/2016-November/000596.html where regressions have crept into a jdk9 build, which is disappointing. The whole point of regression testing is to ensure that regressions don't happen! And I recall having that job myself back in 2005! From neugens.limasoftware at gmail.com Mon Nov 28 20:52:13 2016 From: neugens.limasoftware at gmail.com (Mario Torre) Date: Mon, 28 Nov 2016 21:52:13 +0100 Subject: Repository? -- How many lines of development? In-Reply-To: <11f471fa-87a6-f6e3-86b2-05467c076a5a@oracle.com> References: <52f370b5-79a7-7b11-b77c-0c1314812e9e@redhat.com> <11f471fa-87a6-f6e3-86b2-05467c076a5a@oracle.com> Message-ID: 2016-11-18 17:33 GMT+01:00 joe darcy : > Hello, > > On 11/18/2016 5:50 AM, Aleksey Shipilev wrote: >> >> Hi, >> >> It is very exciting to see the JDK 10 mailing list! >> >> When can we expect open forests (or maybe a monorepo that was discussed >> at jdk9-dev some time ago [1]) for JDK 10? :) >> >> Thanks, >> -Aleksey >> >> [1] http://openjdk.java.net/jeps/296 > > > And thus will commence the first thread in jdk10-dev, how many lines of > development where "line is development" means either a forest or a monorepo. > > For a few reasons including not holding up the start of JDK 10 development > for further discussion about and administrative advancement of JEP 296 and > to give more time to work on internal-only details of the repo consolidation > (such as how the various closed repos are handled), the JDK 10 lines of > development won't start out as monorepos. They will at least initially use > the existing multi-repo structure as in JDK 9. However, we'll return to JEP > 296 later in the release. > > Regardless of many repos used for a line of the development, there is a > larger question of how many lines of development to have. For JDK 10 I > propose three lines of development: > > * A master forest, serving the roles master and dev play today in 9. > > With a few exceptions, in JDK 9 master was just time-delayed copy of dev so > we can implement recording the information about which set of sources > correspond to a promoted build without using a whole other forest. > > Rather than using a separate line of development for client-libs work as in > 9, I think this should be done in the same line of development as all other > libs work in 10. > > * Single HotSpot forest. > > Over of the course of JDK 9, the HotSpot team went from using multiple > forests for their work to using a single one. > > * Sandbox > > The JDK 9 sandbox (http://hg.openjdk.java.net/jdk9/sandbox/) allows > collaboration and publication of sources of small projects outside the main > line of development. This ability should continue in JDK 10. > > Comments? > > Thanks, > > -Joe Looks a lot easier to manage and will surely create less confusion, so +1. I agree with Stefan that perhaps it would be better to call the hotspot "hs", although this seems to me a minor thing. Cheers, Mario -- pgp key: http://subkeys.pgp.net/ PGP Key ID: 80F240CF Fingerprint: BA39 9666 94EC 8B73 27FA FC7C 4086 63E3 80F2 40CF Java Champion - Blog: http://neugens.wordpress.com - Twitter: @neugens Proud GNU Classpath developer: http://www.classpath.org/ OpenJDK: http://openjdk.java.net/projects/caciocavallo/ Please, support open standards: http://endsoftpatents.org/ From david.holmes at oracle.com Mon Nov 28 21:47:59 2016 From: david.holmes at oracle.com (David Holmes) Date: Tue, 29 Nov 2016 07:47:59 +1000 Subject: Repository? -- How many lines of development? In-Reply-To: <11f471fa-87a6-f6e3-86b2-05467c076a5a@oracle.com> References: <52f370b5-79a7-7b11-b77c-0c1314812e9e@redhat.com> <11f471fa-87a6-f6e3-86b2-05467c076a5a@oracle.com> Message-ID: <32589a52-5e9f-dc65-3c99-bd5f9c475b06@oracle.com> Sorry Joe this got forgotten ... On 19/11/2016 2:33 AM, joe darcy wrote: > Hello, > > On 11/18/2016 5:50 AM, Aleksey Shipilev wrote: >> Hi, >> >> It is very exciting to see the JDK 10 mailing list! >> >> When can we expect open forests (or maybe a monorepo that was discussed >> at jdk9-dev some time ago [1]) for JDK 10? :) >> >> Thanks, >> -Aleksey >> >> [1] http://openjdk.java.net/jeps/296 > > And thus will commence the first thread in jdk10-dev, how many lines of > development where "line is development" means either a forest or a > monorepo. > > For a few reasons including not holding up the start of JDK 10 > development for further discussion about and administrative advancement > of JEP 296 and to give more time to work on internal-only details of the > repo consolidation (such as how the various closed repos are handled), > the JDK 10 lines of development won't start out as monorepos. They will > at least initially use the existing multi-repo structure as in JDK 9. > However, we'll return to JEP 296 later in the release. > > Regardless of many repos used for a line of the development, there is a > larger question of how many lines of development to have. For JDK 10 I > propose three lines of development: > > * A master forest, serving the roles master and dev play today in 9. > > With a few exceptions, in JDK 9 master was just time-delayed copy of dev > so we can implement recording the information about which set of > sources correspond to a promoted build without using a whole other forest. I agree with the description, but how does someone get a stable snapshot of master at a given "level"? How will the tagging work? I also think the name "master" (due to historical usage) suggests a level of stability that won't actually be present. Maybe it is better to call it "dev"? David ----- > Rather than using a separate line of development for client-libs work as > in 9, I think this should be done in the same line of development as all > other libs work in 10. > > * Single HotSpot forest. > > Over of the course of JDK 9, the HotSpot team went from using multiple > forests for their work to using a single one. > > * Sandbox > > The JDK 9 sandbox (http://hg.openjdk.java.net/jdk9/sandbox/) allows > collaboration and publication of sources of small projects outside the > main line of development. This ability should continue in JDK 10. > > Comments? > > Thanks, > > -Joe From joe.darcy at oracle.com Mon Nov 28 22:07:59 2016 From: joe.darcy at oracle.com (Joseph D. Darcy) Date: Mon, 28 Nov 2016 14:07:59 -0800 Subject: Repository? -- How many lines of development? In-Reply-To: <32589a52-5e9f-dc65-3c99-bd5f9c475b06@oracle.com> References: <52f370b5-79a7-7b11-b77c-0c1314812e9e@redhat.com> <11f471fa-87a6-f6e3-86b2-05467c076a5a@oracle.com> <32589a52-5e9f-dc65-3c99-bd5f9c475b06@oracle.com> Message-ID: <583CAABF.6000102@oracle.com> Hi David, On 11/28/2016 1:47 PM, David Holmes wrote: > Sorry Joe this got forgotten ... > > On 19/11/2016 2:33 AM, joe darcy wrote: >> Hello, >> >> On 11/18/2016 5:50 AM, Aleksey Shipilev wrote: >>> Hi, >>> >>> It is very exciting to see the JDK 10 mailing list! >>> >>> When can we expect open forests (or maybe a monorepo that was discussed >>> at jdk9-dev some time ago [1]) for JDK 10? :) >>> >>> Thanks, >>> -Aleksey >>> >>> [1] http://openjdk.java.net/jeps/296 >> >> And thus will commence the first thread in jdk10-dev, how many lines of >> development where "line is development" means either a forest or a >> monorepo. >> >> For a few reasons including not holding up the start of JDK 10 >> development for further discussion about and administrative advancement >> of JEP 296 and to give more time to work on internal-only details of the >> repo consolidation (such as how the various closed repos are handled), >> the JDK 10 lines of development won't start out as monorepos. They will >> at least initially use the existing multi-repo structure as in JDK 9. >> However, we'll return to JEP 296 later in the release. >> >> Regardless of many repos used for a line of the development, there is a >> larger question of how many lines of development to have. For JDK 10 I >> propose three lines of development: >> >> * A master forest, serving the roles master and dev play today in 9. >> >> With a few exceptions, in JDK 9 master was just time-delayed copy of dev >> so we can implement recording the information about which set of >> sources correspond to a promoted build without using a whole other >> forest. > > I agree with the description, but how does someone get a stable > snapshot of master at a given "level"? How will the tagging work? For the combined dev/master forest, the most recent integration tag will have the same stability guarantees we have today so "pull the most recent jdk-10+XYZ tag" to get a stable snapshot. As an aside, for JDK 10 I'd also like to see promoted builds on a more frequent schedule than once a week. > > I also think the name "master" (due to historical usage) suggests a > level of stability that won't actually be present. Maybe it is better > to call it "dev"? > Much of the stability we see in master today is because the forest only changes once a week with the dev -> master integration ;-) In a nutshell, the proposal is to replace tracking the known-good state in 9 via integrate dev -> master, tag in master, pull down to dev process today with just "tag known good in combined dev/master" in 10. Cheers, -Joe From joe.darcy at oracle.com Mon Nov 28 22:16:25 2016 From: joe.darcy at oracle.com (Joseph D. Darcy) Date: Mon, 28 Nov 2016 14:16:25 -0800 Subject: Repository? -- How many lines of development? In-Reply-To: <7025d324-68dd-aad7-1f2b-22b5b1836d11@oracle.com> References: <52f370b5-79a7-7b11-b77c-0c1314812e9e@redhat.com> <11f471fa-87a6-f6e3-86b2-05467c076a5a@oracle.com> <633ced12-ded6-29b4-e367-a26de70012b6@oracle.com> <7025d324-68dd-aad7-1f2b-22b5b1836d11@oracle.com> Message-ID: <583CACB9.7040008@oracle.com> Hi Stefan, On 11/28/2016 11:01 AM, Stefan Karlsson wrote: > Hi Joe, > > On 2016-11-28 18:55, joe darcy wrote: >> Hello again, >> >> Last call for comments, in summary the proposal is for JDK 10 to have >> three externally visible lines of development: >> >> * master >> * hotspot > > Not an objection, but rather an observation. > > This would mean that we have two things that people will refer to as > the hotspot repository. The jdk10/hotspot top repository and the > jdk10/hotspot/hotspot repository with the hotspot code. Maybe it would > be better to stick with hs? Of course, this problem will go away when > the repository consolidation is in place, so it might be a short-lived > issue. As JEP 296: Consolidate the JDK Forest into a Single Repository advances, we will return to discussions on whether to consolidate the repositories and, if consolidating, when to do so. Thanks, -Joe From martinrb at google.com Mon Nov 28 23:30:16 2016 From: martinrb at google.com (Martin Buchholz) Date: Mon, 28 Nov 2016 15:30:16 -0800 Subject: Repository? -- How many lines of development? In-Reply-To: <583CAABF.6000102@oracle.com> References: <52f370b5-79a7-7b11-b77c-0c1314812e9e@redhat.com> <11f471fa-87a6-f6e3-86b2-05467c076a5a@oracle.com> <32589a52-5e9f-dc65-3c99-bd5f9c475b06@oracle.com> <583CAABF.6000102@oracle.com> Message-ID: On Mon, Nov 28, 2016 at 2:07 PM, Joseph D. Darcy wrote: > > For the combined dev/master forest, the most recent integration tag will > have the same stability guarantees we have today so "pull the most recent > jdk-10+XYZ tag" to get a stable snapshot. > > But ... I want better than the stability guarantees we have today! I want to obsolete those messages to quality-discuss with "new failures" because it should not even be possible for master to get into a state where regression tests have regressed (except for flakes or external dependencies). > As an aside, for JDK 10 I'd also like to see promoted builds on a more > frequent schedule than once a week. > People do "continuous testing and integration" these days. Set up your integration pipeline so that it is always running. The pipeline automatically promotes changesets to master when all tests pass. Easy! Then every master changeset is equally stable to a "promoted build". In principle, the same could be accomplished via running every single commit through a jprt-like process, but I'm afraid that would be too expensive. Back in my day, full build + test took on the order of 10 hours, so changesets would need to be batched, or you need to throw more hardware at the problem (give every changeset a dedicated machine for testing?) From martinrb at google.com Mon Nov 28 23:36:27 2016 From: martinrb at google.com (Martin Buchholz) Date: Mon, 28 Nov 2016 15:36:27 -0800 Subject: Repository? -- How many lines of development? In-Reply-To: <583CAABF.6000102@oracle.com> References: <52f370b5-79a7-7b11-b77c-0c1314812e9e@redhat.com> <11f471fa-87a6-f6e3-86b2-05467c076a5a@oracle.com> <32589a52-5e9f-dc65-3c99-bd5f9c475b06@oracle.com> <583CAABF.6000102@oracle.com> Message-ID: On Mon, Nov 28, 2016 at 2:07 PM, Joseph D. Darcy wrote: > For the combined dev/master forest, the most recent integration tag will > have the same stability guarantees we have today so "pull the most recent > jdk-10+XYZ tag" to get a stable snapshot. > Another problem is not having any easy/automated way to get the latest stable snapshot. You have to look around, or tail -1 .hgtags to guess at what might be the latest, because there is no single moving tag that represents the desired stability level. Why can't I hg tupdate jdk-9-latest-build From jcoomes at twitter.com Mon Nov 28 23:51:59 2016 From: jcoomes at twitter.com (John Coomes) Date: Mon, 28 Nov 2016 15:51:59 -0800 Subject: Repository? -- How many lines of development? In-Reply-To: <583CAABF.6000102@oracle.com> References: <52f370b5-79a7-7b11-b77c-0c1314812e9e@redhat.com> <11f471fa-87a6-f6e3-86b2-05467c076a5a@oracle.com> <32589a52-5e9f-dc65-3c99-bd5f9c475b06@oracle.com> <583CAABF.6000102@oracle.com> Message-ID: On Mon, Nov 28, 2016 at 2:07 PM, Joseph D. Darcy wrote: > Hi David, > > > On 11/28/2016 1:47 PM, David Holmes wrote: > >> Sorry Joe this got forgotten ... >> >> On 19/11/2016 2:33 AM, joe darcy wrote: >> >>> Hello, >>> >>> On 11/18/2016 5:50 AM, Aleksey Shipilev wrote: >>> >>>> Hi, >>>> >>>> It is very exciting to see the JDK 10 mailing list! >>>> >>>> When can we expect open forests (or maybe a monorepo that was discussed >>>> at jdk9-dev some time ago [1]) for JDK 10? :) >>>> >>>> Thanks, >>>> -Aleksey >>>> >>>> [1] http://openjdk.java.net/jeps/296 >>>> >>> >>> And thus will commence the first thread in jdk10-dev, how many lines of >>> development where "line is development" means either a forest or a >>> monorepo. >>> >>> For a few reasons including not holding up the start of JDK 10 >>> development for further discussion about and administrative advancement >>> of JEP 296 and to give more time to work on internal-only details of the >>> repo consolidation (such as how the various closed repos are handled), >>> the JDK 10 lines of development won't start out as monorepos. They will >>> at least initially use the existing multi-repo structure as in JDK 9. >>> However, we'll return to JEP 296 later in the release. >>> >>> Regardless of many repos used for a line of the development, there is a >>> larger question of how many lines of development to have. For JDK 10 I >>> propose three lines of development: >>> >>> * A master forest, serving the roles master and dev play today in 9. >>> >>> With a few exceptions, in JDK 9 master was just time-delayed copy of dev >>> so we can implement recording the information about which set of >>> sources correspond to a promoted build without using a whole other >>> forest. >>> >> >> I agree with the description, but how does someone get a stable snapshot >> of master at a given "level"? How will the tagging work? >> > > For the combined dev/master forest, the most recent integration tag will > have the same stability guarantees we have today so "pull the most recent > jdk-10+XYZ tag" to get a stable snapshot. > > As an aside, for JDK 10 I'd also like to see promoted builds on a more > frequent schedule than once a week. > > >> I also think the name "master" (due to historical usage) suggests a level >> of stability that won't actually be present. Maybe it is better to call it >> "dev"? >> >> > Much of the stability we see in master today is because the forest only > changes once a week with the dev -> master integration ;-) > > In a nutshell, the proposal is to replace tracking the known-good state in > 9 via integrate dev -> master, tag in master, pull down to dev process > today with just "tag known good in combined dev/master" in 10. > > Hi Joe, This would make cloning or pulling the "last known good" state more difficult. In jdk9, I can always clone/pull from a fixed location (http://.../jdk9/jdk9). With the proposal, I would have to first clone/pull everything, then examine the tags and sort them (which programatically means decoding release and build numbers, ugh), find the right tag, then pull/update to get the working dir in the right state. Since my clone would have all the changesets, including those beyond the "last known good" state, it also makes the use of 'tip' and 'default' error prone. If I do some archaeology and run "hg update ", then when I'm done, I can't go back to "last known good" with a simple "hg update default". I have to look at the tags again, sort them, and so on. I think it's more important that cloning or pulling the "last known good" state be simple, rather than promoting a build. Mercurial bookmarks might be a compromise solution - you can have a bookmark with a fixed name (e.g., "stable", "jdk10-stable", "last-promoted", whatever) that is updated each time a build is promoted. With validation from jcheck, it could be made reasonably safe. -John From joe.darcy at oracle.com Tue Nov 29 00:50:32 2016 From: joe.darcy at oracle.com (Joseph D. Darcy) Date: Mon, 28 Nov 2016 16:50:32 -0800 Subject: Repository? -- How many lines of development? In-Reply-To: References: <52f370b5-79a7-7b11-b77c-0c1314812e9e@redhat.com> <11f471fa-87a6-f6e3-86b2-05467c076a5a@oracle.com> <32589a52-5e9f-dc65-3c99-bd5f9c475b06@oracle.com> <583CAABF.6000102@oracle.com> Message-ID: <583CD0D8.3000105@oracle.com> On 11/28/2016 3:30 PM, Martin Buchholz wrote: > > > On Mon, Nov 28, 2016 at 2:07 PM, Joseph D. Darcy > wrote: > > > For the combined dev/master forest, the most recent integration > tag will have the same stability guarantees we have today so "pull > the most recent jdk-10+XYZ tag" to get a stable snapshot. > > > But ... I want better than the stability guarantees we have today! Over the course of JDK 9 we've made marked improvements to the regression test stability, at least on metrics we track internally on internal systems, so unfortunately am I not at liberty to share them. Briefly, the tier 1 regression tests in promoted builds are very stable, the tier 2 tests somewhat less stable, and so on. > > I want to obsolete those messages to quality-discuss with "new > failures" because it should not even be possible for master to get > into a state where regression tests have regressed (except for flakes > or external dependencies). I don't know if those particular failures are actually in the promoted build or just an artifact of how that particular batch of tests was run. IIRC, we did not see a corresponding set of failures before the integration. > As an aside, for JDK 10 I'd also like to see promoted builds on a > more frequent schedule than once a week. > > > People do "continuous testing and integration" these days. Set up > your integration pipeline so that it is always running. The pipeline > automatically promotes changesets to master when all tests pass. > Easy! Then every master changeset is equally stable to a "promoted > build". One of the internal systems mentioned above is a CI system that run regression tests after a change is pushed. The approximate integration model is then to promote known-good states of sources as vetted by the CI system. If problems are fixed soon after they are identified, then a post-push system gives good results with avoiding the need more complexity on the front end to manage a series of in-flight patches. Cheers, -Joe From joe.darcy at oracle.com Tue Nov 29 01:07:50 2016 From: joe.darcy at oracle.com (Joseph D. Darcy) Date: Mon, 28 Nov 2016 17:07:50 -0800 Subject: Repository? -- How many lines of development? In-Reply-To: References: <52f370b5-79a7-7b11-b77c-0c1314812e9e@redhat.com> <11f471fa-87a6-f6e3-86b2-05467c076a5a@oracle.com> <32589a52-5e9f-dc65-3c99-bd5f9c475b06@oracle.com> <583CAABF.6000102@oracle.com> Message-ID: <583CD4E6.2090703@oracle.com> Hi John, On 11/28/2016 3:51 PM, John Coomes wrote: > On Mon, Nov 28, 2016 at 2:07 PM, Joseph D. Darcy > wrote: > > > [snip] > In a nutshell, the proposal is to replace tracking the known-good > state in 9 via integrate dev -> master, tag in master, pull down > to dev process today with just "tag known good in combined > dev/master" in 10. > > > Hi Joe, > > This would make cloning or pulling the "last known good" state more > difficult. In jdk9, I can always clone/pull from a fixed location > (http://.../jdk9/jdk9). With the proposal, I would have to first > clone/pull everything, then examine the tags and sort them (which > programatically means decoding release and build numbers, ugh), find > the right tag, then pull/update to get the working dir in the right state. > > Since my clone would have all the changesets, including those beyond > the "last known good" state, it also makes the use of 'tip' and > 'default' error prone. If I do some archaeology and run "hg update > ", then when I'm done, I can't go back to "last known > good" with a simple "hg update default". I have to look at the tags > again, sort them, and so on. > > I think it's more important that cloning or pulling the "last known > good" state be simple, rather than promoting a build. > > Mercurial bookmarks might be a compromise solution - you can have a > bookmark with a fixed name (e.g., "stable", "jdk10-stable", > "last-promoted", whatever) that is updated each time a build is > promoted. With validation from jcheck, it could be made reasonably safe. > I don't object to having a convenient way to get the last known good state of the sources, but I think the way we do this in 9, using a separate 700 MB+ Hg forest, has too much overhead to store a relative paucity of information. In terms of evaluating alternatives, are there other ways you see to record this information in Mercurial? Thanks, -Joe From philip.race at oracle.com Tue Nov 29 22:02:53 2016 From: philip.race at oracle.com (Phil Race) Date: Tue, 29 Nov 2016 14:02:53 -0800 Subject: Repository? -- How many lines of development? In-Reply-To: <633ced12-ded6-29b4-e367-a26de70012b6@oracle.com> References: <52f370b5-79a7-7b11-b77c-0c1314812e9e@redhat.com> <11f471fa-87a6-f6e3-86b2-05467c076a5a@oracle.com> <633ced12-ded6-29b4-e367-a26de70012b6@oracle.com> Message-ID: Hi Joe, I need to register my concerns for SE client (the open part, not closed) I don't think it is feasible to collapse the current lines of development unless all our current testing done at PIT is able to be run for every push and we are nowhere near that. Maybe even further away than we were before. -phil. On 11/28/2016 09:55 AM, joe darcy wrote: > Hello again, > > Last call for comments, in summary the proposal is for JDK 10 to have > three externally visible lines of development: > > * master > * hotspot > * sandbox > > If there are no objections by Nov. 30, I think we should go ahead with > this arrangement. > > Thanks, > > -Joe > > > On 11/18/2016 8:33 AM, joe darcy wrote: >> Hello, >> >> On 11/18/2016 5:50 AM, Aleksey Shipilev wrote: >>> Hi, >>> >>> It is very exciting to see the JDK 10 mailing list! >>> >>> When can we expect open forests (or maybe a monorepo that was discussed >>> at jdk9-dev some time ago [1]) for JDK 10? :) >>> >>> Thanks, >>> -Aleksey >>> >>> [1] http://openjdk.java.net/jeps/296 >> >> And thus will commence the first thread in jdk10-dev, how many lines >> of development where "line is development" means either a forest or a >> monorepo. >> >> For a few reasons including not holding up the start of JDK 10 >> development for further discussion about and administrative >> advancement of JEP 296 and to give more time to work on internal-only >> details of the repo consolidation (such as how the various closed >> repos are handled), the JDK 10 lines of development won't start out >> as monorepos. They will at least initially use the existing >> multi-repo structure as in JDK 9. However, we'll return to JEP 296 >> later in the release. >> >> Regardless of many repos used for a line of the development, there is >> a larger question of how many lines of development to have. For JDK >> 10 I propose three lines of development: >> >> * A master forest, serving the roles master and dev play today in 9. >> >> With a few exceptions, in JDK 9 master was just time-delayed copy of >> dev so we can implement recording the information about which set of >> sources correspond to a promoted build without using a whole other >> forest. >> >> Rather than using a separate line of development for client-libs work >> as in 9, I think this should be done in the same line of development >> as all other libs work in 10. >> >> * Single HotSpot forest. >> >> Over of the course of JDK 9, the HotSpot team went from using >> multiple forests for their work to using a single one. >> >> * Sandbox >> >> The JDK 9 sandbox (http://hg.openjdk.java.net/jdk9/sandbox/) allows >> collaboration and publication of sources of small projects outside >> the main line of development. This ability should continue in JDK 10. >> >> Comments? >> >> Thanks, >> >> -Joe >