From joe.darcy at oracle.com Sat Sep 15 14:42:05 2018 From: joe.darcy at oracle.com (joe darcy) Date: Sat, 15 Sep 2018 07:42:05 -0700 Subject: Welcome to skara-dev! Message-ID: Hello, Welcome to the skara-dev mailing list. In the near term as part of exploring SCM alternatives, we'll discuss a proposal for a revised format for commit messages as well as publish a prototype git repo where the commit messages in the existing hg repo have been rewritten to use the new format. Cheers, -Joe From joe.darcy at oracle.com Mon Sep 17 04:27:51 2018 From: joe.darcy at oracle.com (joe darcy) Date: Sun, 16 Sep 2018 21:27:51 -0700 Subject: Proposed revised format for JDK commit messages Message-ID: <7bd451d6-1cb4-c869-c76c-16c0943974da@oracle.com> Hello, As documented in http://openjdk.java.net/guide/producingChangeset.html, the format for a Mercurial changeset comment has been : Summary: Reviewed-by: + Contributed-by: where the "Summary:" and "Contributed-by: " lines are syntactically optional. In more detail from the Developers Guide: > There may be more than one bugid line, but there must be at least one. > > ?The summary line is optional, but authors are strongly encouraged to > include one if the nature of the change is not obvious from the > synopsis. It's just one line, meant to give the reader a clue as to > how the code changed. A more complete description of the change > belongs in the bug report. > > A reviewed-by line is required. Reviewers must have the ability to > deal with any adverse consequences of the change, and so must > themselves be authors. They are therefore identified by their OpenJDK > usernames rather than full e-mail addresses. > > The contributed-by line is optional. If present, it is a list of > comma-separated email addresses. It should be included only when the > author or authors of the change do not have commit rights to the > target repository and thus would not otherwise receive acknowledgment. [...] > The required format of the comments will be enforced whenever the > changeset is pushed into the JDK master or team repository forests. > Other Projects may copy these conventions, adopt some other > conventions, or have no conventions, depending upon their goals The syntax check of this format is enforced on the JDK repos using jcheck. Slightly more formally, the existing commit message can be described by an EBNF-style grammar as: ? JdkHgCommitMessage : BugIdLine+ SummaryLine? ReviewersLine ConstributedByLine? ? BugIdLine : /[0-9]{8}/ ": " Text ? SummaryLine : "Summary: " Text ? ReviewersLine : "Reviewed-by: " Username (", " Username)* "\n" ? ContributedByLine : "Contributed-by: " Text ? Username : /[a-z]+/ ? Text : /[^\n]+/ "\n" This format has served adequately over the years; although it doesn't accurately model some more complex scenarios, such as a patch co-authored by several people. The commit message is part of the hash of a change. Therefore, changing an SCM is one of the few times the format of the message can be uniformly changed without triggering a new set of hashes (since there will already by a new set of hashes from changing the SCM) Git tooling generally expects commits to have a subject and git requires an email field. This argues for adjustments in the commit format. One possibility is: ? ProposedJdkGitCommitMessage : BugIdLine+ BlankLine SummaryLine? ReviewersLine CoAuthorsLine+ ? BugIdLine : /[0-9]{8}/ ": " Text ? SummaryLine : "Summary: " Text ? ReviewersLine : "Reviewed-by: " Username (", " Username)* "\n" ? ContributedByLine : "Contributed-by: " Text ? BlankLine = "\n" ? Username : /[a-z]+/ ? Text : /[^\n]+/ "\n" A simple example: Current: ??? 8210283: Support git as an SCM alternative in the build ??? Reviewed-by: ihse, ehelin Proposed: ??? 8210283: Support git as an SCM alternative in the build ??? Reviewed-by: ihse, ehelin The only difference in a simple case is the addition of a blank line between the bug ids and the reviewer information. The initial bug id line(s) before the the first blank line serve as the git subject field. A more complicated example: Current: ??? 8167108: inconsistent handling of SR_lock can lead to crashes ??? Summary: Add Thread Safe Memory Reclamation (Thread-SMR) mechanism. ??? Reviewed-by: coleenp, dcubed, dholmes, eosterlund, gthornbr, kbarrett, rehn, sspitsyn, stefank ??? Contributed-by: daniel.daugherty at oracle.com, erik.osterlund at oracle.com, robbin.ehn at oracle.com Proposed: ??? 8167108: inconsistent handling of SR_lock can lead to crashes ??? Add Thread Safe Memory Reclamation (Thread-SMR) mechanism. ??? Reviewed-by: coleenp, dcubed, dholmes, eosterlund, gthornbr, kbarrett, rehn, sspitsyn, stefank ??? Co-authored-by: Erik Osterlund ??? Co-authored-by: Robbin Ehn ??? Co-authored-by: Igor Ignatyev A "Co-authored-by" field is recognized by popular git hosting providers. A JDK 12 repo whose commits have been converted to this format will be available shortly. Comments? Thanks, -Joe From Alan.Bateman at oracle.com Mon Sep 17 09:18:04 2018 From: Alan.Bateman at oracle.com (Alan Bateman) Date: Mon, 17 Sep 2018 10:18:04 +0100 Subject: Proposed revised format for JDK commit messages In-Reply-To: <7bd451d6-1cb4-c869-c76c-16c0943974da@oracle.com> References: <7bd451d6-1cb4-c869-c76c-16c0943974da@oracle.com> Message-ID: <7d10c97c-59c3-b1e3-225b-c657203362d2@oracle.com> On 17/09/2018 05:27, joe darcy wrote: > : > > Comments? > Joe - can you provide an example for sponsored contributions? I assume the change author will be the sponsor and there will be a Co-authored-by trailer with the contributor email address. I'm just wondering if this will be incorrectly interpreted as being authored by both the sponsor and contributor when the contributor is the sole author. -Alan From erik.helin at oracle.com Mon Sep 17 13:30:41 2018 From: erik.helin at oracle.com (Erik Helin) Date: Mon, 17 Sep 2018 15:30:41 +0200 Subject: Proposed revised format for JDK commit messages In-Reply-To: <7bd451d6-1cb4-c869-c76c-16c0943974da@oracle.com> References: <7bd451d6-1cb4-c869-c76c-16c0943974da@oracle.com> Message-ID: <3fd787cd-4e8b-880e-7f02-27a45b3baa56@oracle.com> On 9/17/18 6:27 AM, joe darcy wrote: > A JDK 12 repo whose commits have been converted to this format will be > available shortly. An automatically updated read-only mirror of jdk/jdk [0] with the proposed commit format is now available at: https://github.com/openjdk/jdk Please note that this is a read-only mirror, any pull requests will be closed (and the author directed towards http://openjdk.java.net). Thanks, Erik [0]: http://hg.openjdk.java.net/jdk/jdk From erik.helin at oracle.com Mon Sep 17 13:32:59 2018 From: erik.helin at oracle.com (Erik Helin) Date: Mon, 17 Sep 2018 15:32:59 +0200 Subject: Proposed revised format for JDK commit messages In-Reply-To: <7d10c97c-59c3-b1e3-225b-c657203362d2@oracle.com> References: <7bd451d6-1cb4-c869-c76c-16c0943974da@oracle.com> <7d10c97c-59c3-b1e3-225b-c657203362d2@oracle.com> Message-ID: On 9/17/18 11:18 AM, Alan Bateman wrote: > On 17/09/2018 05:27, joe darcy wrote: >> : >> >> Comments? >> > Joe - can you provide an example for sponsored contributions? I assume > the change author will be the sponsor and there will be a Co-authored-by > trailer with the contributor email address. I'm just wondering if this > will be incorrectly interpreted as being authored by both the sponsor > and contributor when the contributor is the sole author. Hi Alan, thanks for reading through the proposal and good question! I did think about this case, and right now a "Contributed-by" line with a single contributor is being translated so that the single contributor becomes the sole author (the sponsor is not listed anywhere). When I read through a number of commit messages, it seemed to be common practice to list both the sponsor and the contributor when the sponsor also has contributed to the patch (and not "only" sponsored it). This is also how I personally have used "Contributed-by" when sponsoring HotSpot patches. For an example, please see the patch for JDK-8210724 [0]. The patch was authored (and proposed) by Man Cao on hotspot-gc-dev [1], reviewed by Kim Barrett and Stefan Johansson and sponsored by JC Beyler. Looking at the commit in the hg repository [2] we have exactly the case you are thinking of: Man Cao is listed as the sole contributor on the "Contributed-by" line and jcbeyler is the author. Looking at the translated commit [3] you will see that Man is listed as the author (and JC isn't mentioned at all). Please note that the commit author field says "Man Cao ", but Man seems to have a GitHub account with the manc at google.com email registered to it, so GitHub will display Man's GitHub username "mancao". So, just to be clear, the git repository contains no GitHub usernames. The motivation for this change is that no SCM tooling understands OpenJDK's "Contributed-by" line, so it seemed easier to just let the author field actually denote the author of the patch. This way commands such as e.g. 'git shortlog' will work correctly. Please let us know if you have any more questions! Erik [0]: https://bugs.openjdk.java.net/browse/JDK-8210724 [1]: http://mail.openjdk.java.net/pipermail/hotspot-gc-dev/2018-September/023256.html [2]: http://hg.openjdk.java.net/jdk/jdk/rev/43323ced5e40 [3]: https://github.com/openjdk/jdk/commit/4e1a13e673e3efe9ad46766160c75ba1b16cfdff From erik.helin at oracle.com Mon Sep 17 13:33:24 2018 From: erik.helin at oracle.com (Erik Helin) Date: Mon, 17 Sep 2018 15:33:24 +0200 Subject: Proposed revised format for JDK commit messages In-Reply-To: <7bd451d6-1cb4-c869-c76c-16c0943974da@oracle.com> References: <7bd451d6-1cb4-c869-c76c-16c0943974da@oracle.com> Message-ID: <313e5273-ff78-c363-d954-ec5d672a9f91@oracle.com> On 9/17/18 6:27 AM, joe darcy wrote: > One possibility is: > > ? ProposedJdkGitCommitMessage : BugIdLine+ BlankLine SummaryLine? > ReviewersLine CoAuthorsLine+ > > ? BugIdLine : /[0-9]{8}/ ": " Text > > ? SummaryLine : "Summary: " Text This is actually SummaryLines, you can have as many summary lines as you want :) Furthermore, the "Summary:" prefix is not needed, the summary lines will always be surrounded by blank lines. My EBNF is a bit rusty, but I think the above paragraph would be expressed as: SummaryLines: (Text)+ "\n" > ? ReviewersLine : "Reviewed-by: " Username (", " Username)* "\n" > > ? ContributedByLine : "Contributed-by: " Text > > ? BlankLine = "\n" > > ? Username : /[a-z]+/ > > ? Text : /[^\n]+/ "\n" Thanks, Erik From mark.reinhold at oracle.com Mon Sep 17 15:03:02 2018 From: mark.reinhold at oracle.com (mark.reinhold at oracle.com) Date: Mon, 17 Sep 2018 08:03:02 -0700 Subject: Proposed revised format for JDK commit messages In-Reply-To: References: <7bd451d6-1cb4-c869-c76c-16c0943974da@oracle.com> <7d10c97c-59c3-b1e3-225b-c657203362d2@oracle.com> Message-ID: <20180917080302.317332872@eggemoggin.niobe.net> 2018/9/17 6:32:59 -0700, erik.helin at oracle.com: > ... > > For an example, please see the patch for JDK-8210724 [0]. The patch was > authored (and proposed) by Man Cao on hotspot-gc-dev [1], reviewed by > Kim Barrett and Stefan Johansson and sponsored by JC Beyler. > > Looking at the commit in the hg repository [2] we have exactly the case > you are thinking of: Man Cao is listed as the sole contributor on the > "Contributed-by" line and jcbeyler is the author. Looking at the > translated commit [3] you will see that Man is listed as the author (and > JC isn't mentioned at all). That seems like a bug. It?s important to record the sponsorship relation, since the sponsor of a changeset is responsible for any adverse consequences of that changeset. > Please note that the commit author field says > "Man Cao ", but Man seems to have a GitHub account with > the manc at google.com email registered to it, so GitHub will display Man's > GitHub username "mancao". So, just to be clear, the git repository > contains no GitHub usernames. > > The motivation for this change is that no SCM tooling understands > OpenJDK's "Contributed-by" line, so it seemed easier to just let the > author field actually denote the author of the patch. This way commands > such as e.g. 'git shortlog' will work correctly. Does common SCM tooling understand the ?Reviewed-by? line? If not, then the argument against keeping the ?Contributed-by? line for sponsored changes is pretty weak. - Mark > [0]: https://bugs.openjdk.java.net/browse/JDK-8210724 > [1]: http://mail.openjdk.java.net/pipermail/hotspot-gc-dev/2018-September/023256.html > [2]: http://hg.openjdk.java.net/jdk/jdk/rev/43323ced5e40 > [3]: https://github.com/openjdk/jdk/commit/4e1a13e673e3efe9ad46766160c75ba1b16cfdff From erik.helin at oracle.com Mon Sep 17 16:29:11 2018 From: erik.helin at oracle.com (Erik Helin) Date: Mon, 17 Sep 2018 18:29:11 +0200 Subject: Proposed revised format for JDK commit messages In-Reply-To: <20180917080302.317332872@eggemoggin.niobe.net> References: <7bd451d6-1cb4-c869-c76c-16c0943974da@oracle.com> <7d10c97c-59c3-b1e3-225b-c657203362d2@oracle.com> <20180917080302.317332872@eggemoggin.niobe.net> Message-ID: On 09/17/2018 05:03 PM, mark.reinhold at oracle.com wrote: > 2018/9/17 6:32:59 -0700, erik.helin at oracle.com: >> ... >> >> For an example, please see the patch for JDK-8210724 [0]. The patch was >> authored (and proposed) by Man Cao on hotspot-gc-dev [1], reviewed by >> Kim Barrett and Stefan Johansson and sponsored by JC Beyler. >> >> Looking at the commit in the hg repository [2] we have exactly the case >> you are thinking of: Man Cao is listed as the sole contributor on the >> "Contributed-by" line and jcbeyler is the author. Looking at the >> translated commit [3] you will see that Man is listed as the author (and >> JC isn't mentioned at all). > > That seems like a bug. It?s important to record the sponsorship > relation, since the sponsor of a changeset is responsible for any > adverse consequences of that changeset. There are ways we can record the sponshorship, see my reply further down. Just for my understanding: this only applies to contributions from persons with the "Participant" role [4]? Persons with role "Author" [5] still needs a sponsor but we don't record that sponsorship, right? >> Please note that the commit author field says >> "Man Cao ", but Man seems to have a GitHub account with >> the manc at google.com email registered to it, so GitHub will display Man's >> GitHub username "mancao". So, just to be clear, the git repository >> contains no GitHub usernames. >> >> The motivation for this change is that no SCM tooling understands >> OpenJDK's "Contributed-by" line, so it seemed easier to just let the >> author field actually denote the author of the patch. This way commands >> such as e.g. 'git shortlog' will work correctly. > > Does common SCM tooling understand the ?Reviewed-by? line? No, I am not aware of any tooling that understands the "Reviewed-by" line. > If not, then the argument against keeping the ?Contributed-by? line > for sponsored changes is pretty weak. What I don't like about recording sponsors as authors is that not only does tools not understand the "Contributed-by" line, they will interpret the author field incorrectly (the author is just the sponsor, the real author is on the "Contributed-by" line). How about changing whats get recorded for a sponsored commit? We could let the author be recorded in the author field and then have a "Sponsored-by" line? This way we could also encode the sponshorship for commits from a person with role "Author" (the "Author" would of course be the author and we would add an "Sponsored-by" line for the sponsor). With the concept of "Sponsored-by", the commit at [3] would look like: 8210724: Change the verbosity threshold of logging for [oopstorage,ref] Improve logging verbosity levels in oopStorage.cpp Reviewed-by: kbarrett, sjohanss Sponsored-by: jcbeyler and the author would be "Man Cao ". What do you think? I'm open for suggestions here, my initial goal was just to let author mean author :) Thanks for your feedback! Erik [0]: https://bugs.openjdk.java.net/browse/JDK-8210724 [1]: http://mail.openjdk.java.net/pipermail/hotspot-gc-dev/2018-September/023256.html [2]: http://hg.openjdk.java.net/jdk/jdk/rev/43323ced5e40 [3]: https://github.com/openjdk/jdk/commit/4e1a13e673e3efe9ad46766160c75ba1b16cfdff [4]: http://openjdk.java.net/bylaws#participant [5]: http://openjdk.java.net/bylaws#author From Alan.Bateman at oracle.com Mon Sep 17 16:48:12 2018 From: Alan.Bateman at oracle.com (Alan Bateman) Date: Mon, 17 Sep 2018 17:48:12 +0100 Subject: Proposed revised format for JDK commit messages In-Reply-To: References: <7bd451d6-1cb4-c869-c76c-16c0943974da@oracle.com> <7d10c97c-59c3-b1e3-225b-c657203362d2@oracle.com> <20180917080302.317332872@eggemoggin.niobe.net> Message-ID: On 17/09/2018 17:29, Erik Helin wrote: > > There are ways we can record the sponshorship, see my reply further > down. Just for my understanding: this only applies to contributions > from persons with the "Participant" role [4]? Persons with role > "Author" [5] still needs a sponsor but we don't record that > sponsorship, right? It depends on who creates the change-set. Sometimes a contributor with Author role will create the change-set themselves and the sponsor will do the push. The sponsor will usually have reviewed the change so will be listed in the Reviewed-by line but you might find a few cases where the sponsor wasn't involved in the review, instead they are just helping out. At other times, the sponsor will create the change-set (and so be the change-set author) and list the contributor in the Contributed-by line today. In general there isn't any specific to contributors with Author role here - I can think of a few cases where a Committer or Reviewer developed patches that were turned into change-sets and pushed by someone else (with the appropriate Contributed-by line of course). -Alan From philip.race at oracle.com Mon Sep 17 16:59:07 2018 From: philip.race at oracle.com (Phil Race) Date: Mon, 17 Sep 2018 09:59:07 -0700 Subject: Proposed revised format for JDK commit messages In-Reply-To: References: <7bd451d6-1cb4-c869-c76c-16c0943974da@oracle.com> <7d10c97c-59c3-b1e3-225b-c657203362d2@oracle.com> <20180917080302.317332872@eggemoggin.niobe.net> Message-ID: On 9/17/2018 9:48 AM, Alan Bateman wrote: > On 17/09/2018 17:29, Erik Helin wrote: >> >> There are ways we can record the sponshorship, see my reply further >> down. Just for my understanding: this only applies to contributions >> from persons with the "Participant" role [4]? Persons with role >> "Author" [5] still needs a sponsor but we don't record that >> sponsorship, right? > It depends on who creates the change-set. Sometimes a contributor with > Author role will create the change-set themselves and the sponsor will > do the push. Can that practice stay the same with git ? -phil. > The sponsor will usually have reviewed the change so will be listed in > the Reviewed-by line but you might find a few cases where the sponsor > wasn't involved in the review, instead they are just helping out. At > other times, the sponsor will create the change-set (and so be the > change-set author) and list the contributor in the Contributed-by line > today. In general there isn't any specific to contributors with Author > role here - I can think of a few cases where a Committer or Reviewer > developed patches that were turned into change-sets and pushed by > someone else (with the appropriate Contributed-by line of course). > > -Alan From erik.helin at oracle.com Mon Sep 17 18:02:42 2018 From: erik.helin at oracle.com (Erik Helin) Date: Mon, 17 Sep 2018 20:02:42 +0200 Subject: Proposed revised format for JDK commit messages In-Reply-To: References: <7bd451d6-1cb4-c869-c76c-16c0943974da@oracle.com> <7d10c97c-59c3-b1e3-225b-c657203362d2@oracle.com> <20180917080302.317332872@eggemoggin.niobe.net> Message-ID: <96157d4e-b153-cfd0-6bd2-963e1aca353c@oracle.com> On 09/17/2018 06:29 PM, Erik Helin wrote: > How about changing whats get recorded for a sponsored commit? We could > let the author be recorded in the author field and then have a > "Sponsored-by" line? This way we could also encode the sponshorship for > commits from a person with role "Author" (the "Author" would of course > be the author and we would add an "Sponsored-by" line for the sponsor). ...and, as usual when not thinking about a problem for a while, a third idea popped up :) We could utilize the fact that git has two different fields in a commit for recording different kinds of authorship: - committer The person who applied, ev. merged and then pushed the patch - author The person who created the patch Git's notion of committer is fairly close to our concept of sponsor (a sponsor has a bit more responsibility than just committing and pushing the patch). So another suggestion would therefore be to utilize the committer field for recording the sponsor. For "regular" pushes from a person with role "Committer" or above, the committer field and the author field would be the same (assuming the person with "Committer" status also wrote the patch). So, two suggestions for encoding sponsorship while still letting author mean author: - add a "Sponsored-by" line - use the committer field for recording the sponsor What do you think? Thanks, Erik From joe.darcy at oracle.com Mon Sep 17 18:45:18 2018 From: joe.darcy at oracle.com (joe darcy) Date: Mon, 17 Sep 2018 11:45:18 -0700 Subject: Proposed revised format for JDK commit messages In-Reply-To: <96157d4e-b153-cfd0-6bd2-963e1aca353c@oracle.com> References: <7bd451d6-1cb4-c869-c76c-16c0943974da@oracle.com> <7d10c97c-59c3-b1e3-225b-c657203362d2@oracle.com> <20180917080302.317332872@eggemoggin.niobe.net> <96157d4e-b153-cfd0-6bd2-963e1aca353c@oracle.com> Message-ID: <85dd33bc-86c8-300a-7c76-48448eb71e19@oracle.com> On 9/17/2018 11:02 AM, Erik Helin wrote: > On 09/17/2018 06:29 PM, Erik Helin wrote: >> How about changing whats get recorded for a sponsored commit? We >> could let the author be recorded in the author field and then have a >> "Sponsored-by" line? This way we could also encode the sponshorship >> for commits from a person with role "Author" (the "Author" would of >> course be the author and we would add an "Sponsored-by" line for the >> sponsor). > > ...and, as usual when not thinking about a problem for a while, a > third idea popped up :) > > We could utilize the fact that git has two different fields in a > commit for recording different kinds of authorship: > - committer > ? The person who applied, ev. merged and then pushed the patch > - author > ? The person who created the patch > > Git's notion of committer is fairly close to our concept of sponsor (a > sponsor has a bit more responsibility than just committing and pushing > the patch). So another suggestion would therefore be to utilize the > committer field for recording the sponsor. > > For "regular" pushes from a person with role "Committer" or above, the > committer field and the author field would be the same (assuming the > person with "Committer" status also wrote the patch). > > So, two suggestions for encoding sponsorship while still letting > author mean author: > - add a "Sponsored-by" line > - use the committer field for recording the sponsor > > Right; there are several notions we want to capture: * The set of people who wrote and reviewed the changeset in question. * The person(s) responsible for a fix going into a particular line of development/release In the common case the sole author is the person deciding the fix should go into a release by virtue of being the person who pushes the changeset. In certain cases, the existing fields don't model what what is going on. For example, a changeset in the mainline release is written by an non-author and thus has a sponsor as author and a contributed-by line. If such a change is backported to an update release, there isn't a good way to record who is doing the backport. I think using the existing git field as suggested is worth further consideration. -Joe From erik.helin at oracle.com Tue Sep 18 06:36:02 2018 From: erik.helin at oracle.com (Erik Helin) Date: Tue, 18 Sep 2018 08:36:02 +0200 Subject: Proposed revised format for JDK commit messages In-Reply-To: References: <7bd451d6-1cb4-c869-c76c-16c0943974da@oracle.com> <7d10c97c-59c3-b1e3-225b-c657203362d2@oracle.com> <20180917080302.317332872@eggemoggin.niobe.net> Message-ID: <5afd7a51-1d6e-8e6e-f92d-95500062f99d@oracle.com> On 9/17/18 6:59 PM, Phil Race wrote: > On 9/17/2018 9:48 AM, Alan Bateman wrote: >> On 17/09/2018 17:29, Erik Helin wrote: >>> >>> There are ways we can record the sponshorship, see my reply further >>> down. Just for my understanding: this only applies to contributions >>> from persons with the "Participant" role [4]? Persons with role >>> "Author" [5] still needs a sponsor but we don't record that >>> sponsorship, right? >> It depends on who creates the change-set. Sometimes a contributor with >> Author role will create the change-set themselves and the sponsor will >> do the push. > > Can that practice stay the same with git ? Absolutely, this is a use-case that is very well supported in both hg and git. For hg, the commands are: - contributor: hg export -r ref > foo.patch - sponsor: hg import --exact foo.patch For git, the commands are: - contributor: git format-patch --stdout ref > foo.patch - sponsor: git am foo.patch As you can see, the concept is the same and the commands are fairly similar. Both hg and git support a plethora of options when generating the patches, see `hg help export` and `git help format-patch` for more details. Thanks, Erik From erik.helin at oracle.com Tue Sep 18 07:49:37 2018 From: erik.helin at oracle.com (Erik Helin) Date: Tue, 18 Sep 2018 09:49:37 +0200 Subject: Proposed revised format for JDK commit messages In-Reply-To: References: <7bd451d6-1cb4-c869-c76c-16c0943974da@oracle.com> <7d10c97c-59c3-b1e3-225b-c657203362d2@oracle.com> <20180917080302.317332872@eggemoggin.niobe.net> Message-ID: <38b281c6-702e-aeac-05a1-ee74afe01e74@oracle.com> On 9/17/18 6:48 PM, Alan Bateman wrote: > On 17/09/2018 17:29, Erik Helin wrote: >> There are ways we can record the sponshorship, see my reply further >> down. Just for my understanding: this only applies to contributions >> from persons with the "Participant" role [4]? Persons with role >> "Author" [5] still needs a sponsor but we don't record that >> sponsorship, right? > It depends on who creates the change-set. Sometimes a contributor with > Author role will create the change-set themselves and the sponsor will > do the push. The sponsor will usually have reviewed the change so will > be listed in the Reviewed-by line but you might find a few cases where > the sponsor wasn't involved in the review, instead they are just helping > out. At other times, the sponsor will create the change-set (and so be > the change-set author) and list the contributor in the Contributed-by > line today. In general there isn't any specific to contributors with > Author role here - I can think of a few cases where a Committer or > Reviewer developed patches that were turned into change-sets and pushed > by someone else (with the appropriate Contributed-by line of course). Thanks Alan for your thoughts on this matter, I agree that in almost all cases the sponsor for an "Author" patch will be one of the reviewers (at least that is my personal experience). As in wrote in my reply to Mark (and myself), I think it makes sense to encode the sponshorship using either a "Sponsored-by" line or using the committer field in git. What do you think about this? Thanks for taking your time to read through this proposal and providing feedback, much appreciated! Erik From per.liden at oracle.com Wed Sep 19 07:20:47 2018 From: per.liden at oracle.com (Per Liden) Date: Wed, 19 Sep 2018 09:20:47 +0200 Subject: Proposed revised format for JDK commit messages In-Reply-To: <7bd451d6-1cb4-c869-c76c-16c0943974da@oracle.com> References: <7bd451d6-1cb4-c869-c76c-16c0943974da@oracle.com> Message-ID: <359ff054-194e-76c3-7050-166179a56710@oracle.com> Hi, On 09/17/2018 06:27 AM, joe darcy wrote: [...] > > ? ReviewersLine : "Reviewed-by: " Username (", " Username)* "\n" > > ? ContributedByLine : "Contributed-by: " Text > [...] > Proposed: > > ??? 8167108: inconsistent handling of SR_lock can lead to crashes > > ??? Add Thread Safe Memory Reclamation (Thread-SMR) mechanism. > > ??? Reviewed-by: coleenp, dcubed, dholmes, eosterlund, gthornbr, > kbarrett, rehn, sspitsyn, stefank > ??? Co-authored-by: Erik Osterlund > ??? Co-authored-by: Robbin Ehn > ??? Co-authored-by: Igor Ignatyev > It would feel natural to have all *-by lines follow the same syntax. The Reviewed-by line sticks out in this regard. Is there a reason why we don't just do: Reviewed-by: Some Name Reviewed-by: Some Other Name ... and so on? cheers, Per From Alan.Bateman at oracle.com Wed Sep 19 08:30:15 2018 From: Alan.Bateman at oracle.com (Alan Bateman) Date: Wed, 19 Sep 2018 09:30:15 +0100 Subject: Proposed revised format for JDK commit messages In-Reply-To: <359ff054-194e-76c3-7050-166179a56710@oracle.com> References: <7bd451d6-1cb4-c869-c76c-16c0943974da@oracle.com> <359ff054-194e-76c3-7050-166179a56710@oracle.com> Message-ID: <193ce6c8-a6e9-8085-91b8-fd8d78df7e09@oracle.com> On 19/09/2018 08:20, Per Liden wrote: > > It would feel natural to have all *-by lines follow the same syntax. > The Reviewed-by line sticks out in this regard. Is there a reason why > we don't just do: > > Reviewed-by: Some Name > Reviewed-by: Some Other Name > > ... and so on? I assume the names specified to the Reviewed-by line(s) will need to be checked against the set of members of the project and their roles. I don't know the details of the validation that is done today but it might be less reliable to try to map anything other than their OpenJDK user name. -Alan From magnus.ihse.bursie at oracle.com Wed Sep 19 10:57:33 2018 From: magnus.ihse.bursie at oracle.com (Magnus Ihse Bursie) Date: Wed, 19 Sep 2018 12:57:33 +0200 Subject: Proposed revised format for JDK commit messages In-Reply-To: <359ff054-194e-76c3-7050-166179a56710@oracle.com> References: <7bd451d6-1cb4-c869-c76c-16c0943974da@oracle.com> <359ff054-194e-76c3-7050-166179a56710@oracle.com> Message-ID: <74449375-B601-4183-88D8-944B1DCD4F8B@oracle.com> /Magnus > 19 sep. 2018 kl. 09:20 skrev Per Liden : > > Hi, > >> On 09/17/2018 06:27 AM, joe darcy wrote: >> [...] >> ReviewersLine : "Reviewed-by: " Username (", " Username)* "\n" >> ContributedByLine : "Contributed-by: " Text > [...] >> Proposed: >> 8167108: inconsistent handling of SR_lock can lead to crashes >> Add Thread Safe Memory Reclamation (Thread-SMR) mechanism. >> Reviewed-by: coleenp, dcubed, dholmes, eosterlund, gthornbr, kbarrett, rehn, sspitsyn, stefank >> Co-authored-by: Erik Osterlund >> Co-authored-by: Robbin Ehn >> Co-authored-by: Igor Ignatyev > > It would feel natural to have all *-by lines follow the same syntax. The Reviewed-by line sticks out in this regard. Is there a reason why we don't just do: > > Reviewed-by: Some Name > Reviewed-by: Some Other Name > > ... and so on? Per, This is just as the case is today. The Contributed-by follows different rules than Reviewed-by, since anyone can contribute a patch, but only OpenJDK Reviewers can review a patch. /Magnus > > cheers, > Per From per.liden at oracle.com Wed Sep 19 11:22:51 2018 From: per.liden at oracle.com (Per Liden) Date: Wed, 19 Sep 2018 04:22:51 -0700 (PDT) Subject: Proposed revised format for JDK commit messages In-Reply-To: <74449375-B601-4183-88D8-944B1DCD4F8B@oracle.com> References: <7bd451d6-1cb4-c869-c76c-16c0943974da@oracle.com> <359ff054-194e-76c3-7050-166179a56710@oracle.com> <74449375-B601-4183-88D8-944B1DCD4F8B@oracle.com> Message-ID: <19347469-e6a2-d418-a7ac-672c6678c300@oracle.com> On 09/19/2018 12:57 PM, Magnus Ihse Bursie wrote: > > > /Magnus > >> 19 sep. 2018 kl. 09:20 skrev Per Liden : >> >> Hi, >> >>> On 09/17/2018 06:27 AM, joe darcy wrote: >>> [...] >>> ReviewersLine : "Reviewed-by: " Username (", " Username)* "\n" >>> ContributedByLine : "Contributed-by: " Text >> [...] >>> Proposed: >>> 8167108: inconsistent handling of SR_lock can lead to crashes >>> Add Thread Safe Memory Reclamation (Thread-SMR) mechanism. >>> Reviewed-by: coleenp, dcubed, dholmes, eosterlund, gthornbr, kbarrett, rehn, sspitsyn, stefank >>> Co-authored-by: Erik Osterlund >>> Co-authored-by: Robbin Ehn >>> Co-authored-by: Igor Ignatyev >> >> It would feel natural to have all *-by lines follow the same syntax. The Reviewed-by line sticks out in this regard. Is there a reason why we don't just do: >> >> Reviewed-by: Some Name >> Reviewed-by: Some Other Name >> >> ... and so on? > > Per, > > This is just as the case is today. The Contributed-by follows different rules than Reviewed-by, since anyone can contribute a patch, but only OpenJDK Reviewers can review a patch. I know, but I was just thinking this might a good time to revise that. Nothing stops us from continuing using OpenJDK user names if we want, that instead of email addresses. For example: Reviewed-by: Some Name Reviewed-by: Some Name or just: Reviewed-by: someOpenJDKId Reviewed-by: someOtherOpenJDKId It was mostly reacting to the inconsistency if having multiple values in a Reviewed-by tags, when all(?) other tags have a single value. cheers, Per > > /Magnus > >> >> cheers, >> Per > From erik.helin at oracle.com Wed Sep 19 13:11:41 2018 From: erik.helin at oracle.com (Erik Helin) Date: Wed, 19 Sep 2018 15:11:41 +0200 Subject: Proposed revised format for JDK commit messages In-Reply-To: <193ce6c8-a6e9-8085-91b8-fd8d78df7e09@oracle.com> References: <7bd451d6-1cb4-c869-c76c-16c0943974da@oracle.com> <359ff054-194e-76c3-7050-166179a56710@oracle.com> <193ce6c8-a6e9-8085-91b8-fd8d78df7e09@oracle.com> Message-ID: On 09/19/2018 10:30 AM, Alan Bateman wrote: > On 19/09/2018 08:20, Per Liden wrote: >> >> It would feel natural to have all *-by lines follow the same syntax. >> The Reviewed-by line sticks out in this regard. Is there a reason why >> we don't just do: >> >> Reviewed-by: Some Name >> Reviewed-by: Some Other Name >> >> ... and so on? > I assume the names specified to the Reviewed-by line(s) will need to be > checked against the set of members of the project and their roles. I > don't know the details of the validation that is done today but it might > be less reliable to try to map anything other than their OpenJDK user name. I don't have any strong opinion on whether to go with with one "Reviewed-by" line or multiple, I just want to point out that we _can_ go with multiple "Reviewed-by" lines if we want to. We can utilize what we already have done for the author field: "Real Name " For an example have a look at [0], which has "Per Liden " as the author. This way you know that this commit was done by an OpenJDK author and you also get their real name (please note that GitHub will not show the email address). Personally I also think it reads nicely, the commit was done by "Per Liden" who is also known as "pliden" on openjdk.java.net. The motivation for going with one line was because it is more concise and because reviewers must always be OpenJDK authors (every reviewer must have an OpenJDK username). For shorter commit messages, with a single author and one to two reviewers, this becomes nice and succinct: 8210283: Support git as an SCM alternative in the build Reviewed-by: ihse, ehelin On the other hand I can see where Per is coming from for larger commits with multiple authors and potentially many reviewers. For those commits, having multiple "Reviewed-by" lines fit in well: 8167108: inconsistent handling of SR_lock can lead to crashes Add Thread Safe Memory Reclamation (Thread-SMR) mechanism. Reviewed-by: Coleen Phillimore Reviewed-by: Daniel Daugherty Reviewed-by: David Holmes Reviewed-by: Erik Osterlund Reviewed-by: Gerald Thornbrugh Reviewed-by: Kim Barrett Reviewed-by: Robbin Ehn Reviewed-by: Serguei Spitsyn Reviewed-by: Stefan Karlsson Co-authored-by: Erik Osterlund Co-authored-by: Robbin Ehn Co-authored-by: Igor Ignatyev With these many reviewers, another benefit is that each individual line becomes shorter (even though there are more of them). For making a decision here, it might be worthwhile to keep in mind that the vast majority of commits in jdk/jdk have a single author and one to two reviewers, that is by far the most common case. Again, I don't have strong preference here, I just want to highlight what can be done. Thanks, Erik [0]: https://github.com/openjdk/jdk/commit/300babb5d7d28463e79c18b63f182149e66eca65 From iris.clark at oracle.com Wed Sep 19 17:43:35 2018 From: iris.clark at oracle.com (Iris Clark) Date: Wed, 19 Sep 2018 10:43:35 -0700 (PDT) Subject: Proposed revised format for JDK commit messages In-Reply-To: <193ce6c8-a6e9-8085-91b8-fd8d78df7e09@oracle.com> References: <7bd451d6-1cb4-c869-c76c-16c0943974da@oracle.com> <359ff054-194e-76c3-7050-166179a56710@oracle.com> <193ce6c8-a6e9-8085-91b8-fd8d78df7e09@oracle.com> Message-ID: <4a03e6c5-1299-4963-80f3-8a7544e334f3@default> > On 19/09/2018 08:20, Per Liden wrote: >> >> It would feel natural to have all *-by lines follow the same syntax. >> The Reviewed-by line sticks out in this regard. Is there a reason why >> we don't just do: >> >> Reviewed-by: Some Name >> Reviewed-by: Some Other Name >> >> ... and so on? > I assume the names specified to the Reviewed-by line(s) will need to > be checked against the set of members of the project and their roles. > I don't know the details of the validation that is done today but it > might be less reliable to try to map anything other than their > OpenJDK user name. OpenJDK usernames are stable. e-mail addresses are not. iris From shade at redhat.com Wed Sep 26 20:11:47 2018 From: shade at redhat.com (Aleksey Shipilev) Date: Wed, 26 Sep 2018 22:11:47 +0200 Subject: Mercurial clonebundles for hg.openjdk.java.net Message-ID: Hi, This might not be under the umbrella of Skara project, and it may be irrelevant if/when we migrate to Git, but can we repeat what Mozilla does for their monorepo and enable clonebundles [1] on hg.openjdk.java.net, at least on busy repositories? It seems to be easy to configure: enable extension on server side, "hg bundle" the current state of repository, put the bundle somewhere, list the bundles in .hg/clonebundles.manifest on server side, and everything works magically from there. Clients would choose the bundle they can understand, or fallback to regular clone. Clients would also pull all the new changesets after the bundle was created, which means bundles can be refreshed only once in a while. My dirty experiments with clonebundles are very promising. Even within-Europe clones have improved quite a bit [2][3], almost down to the clone performance of local seeding [4]. I think that means: a) clonebundles test bottlenecks on hg client unbundling performance; b) non-clonebundles test bottlenecks on hgserver CPU. With clonebundles, hgserver CPU utilization drops to almost zero. The bundle is streamed by the usual HTTP(S) servers, which can be the part of geo-local CDN. Of course, doing similar thing with Git and Skara's GitHub mirror is much faster [5], quite probably because Git server+client is faster at dealing with the repo. Bottom-line: Enabling clonebundles would simplify our development lives before Skara arrives, and it would certainly be usable on repositories which Skara migration would not touch (e.g. all jdk-updates/*?). It should get CPU costs for hg.openjdk.java.net down too. Thanks, -Aleksey [1] https://www.mercurial-scm.org/wiki/ClonebundlesExtension [2] ------- remote hgserver, clonebundles disabled $ time hg clone --config ui.clonebundles=false http://shipilev.net:8080/ jdk-jdk requesting all changes adding changesets adding manifests adding file changes added 51849 changesets with 448820 changes to 180327 files new changesets fd16c54261b3:54afe70c50b6 updating to branch default 66512 files updated, 0 files merged, 0 files removed, 0 files unresolved real 7m18.963s user 2m27.424s sys 0m16.004s [3] ------- remote hgserver, clonebundles enabled, remote seed $ time hg clone http://shipilev.net:8080/ jdk-jdk applying clone bundle from https://builds.shipilev.net/scratch/jdk-jdk.zstd-max.hg adding changesets adding manifests adding file changes added 51849 changesets with 448820 changes to 180327 files finished applying clone bundle searching for changes no changes found updating to branch default 66512 files updated, 0 files merged, 0 files removed, 0 files unresolved real 1m59.017s user 2m10.728s sys 0m14.323s [4] ------- remote hgserver, clonebundles enabled, local seed $ time hg clone http://shipilev.net:8080/ jdk-jdk applying clone bundle from http://localhost:8000/jdk-jdk.zstd-max.hg adding changesets adding manifests adding file changes added 51849 changesets with 448820 changes to 180327 files finished applying clone bundle searching for changes no changes found updating to branch default 66512 files updated, 0 files merged, 0 files removed, 0 files unresolved real 1m55.545s user 2m9.893s sys 0m14.186s [5] ------- remote git, GitHub mirrors $ time git clone https://github.com/Project-Skara/jdk.git Cloning into 'jdk'... remote: Enumerating objects: 881253, done. remote: Counting objects: 100% (881253/881253), done. remote: Compressing objects: 100% (160363/160363), done. remote: Total 881253 (delta 665729), reused 881253 (delta 665729), pack-reused 0 Receiving objects: 100% (881253/881253), 258.61 MiB | 7.16 MiB/s, done. Resolving deltas: 100% (665729/665729), done. Checking out files: 100% (66258/66258), done. real 1m1.359s user 0m47.248s sys 0m6.129s