From mark.reinhold at oracle.com Tue May 7 13:33:15 2013 From: mark.reinhold at oracle.com (mark.reinhold at oracle.com) Date: Tue, 07 May 2013 13:33:15 -0700 Subject: [PATCH] jcheck: allow bug id line to be ignored In-Reply-To: <51802CF7.7010809@redhat.com> References: <514B5A6C.9050708@redhat.com>, <20130408155342.3301@eggemoggin.niobe.net>, <51802CF7.7010809@redhat.com> Message-ID: <20130507133315.949928@eggemoggin.niobe.net> 2013/4/30 6:43 -0700, omajid at redhat.com: > On 04/08/2013 06:53 PM, mark.reinhold at oracle.com wrote: >> I think it'd be cleaner just to load the bugids lazily. As a side >> benefit that will save a bit of time too. > > Done. Updated webrev (including tests) is here: > http://cr.openjdk.java.net/~omajid/webrevs/jcheck-ignore-bug-line/01/ As much as I'm a fan of higher-order programming, passing a lambda into checker.__init__ for populate_repo_bugids() seems like overkill. The __init__ method can just call repo_bugids(ui, repo) directly when self.bugids_ignore turns out to be false. On line 143, I'd make the regex a bit more precise, something like: bug_ident = re.compile("(([A-Z][A-Z0-9]+-)?[0-9]+):") The new unit tests look fine. - Mark From mark.reinhold at oracle.com Tue May 7 13:47:30 2013 From: mark.reinhold at oracle.com (mark.reinhold at oracle.com) Date: Tue, 07 May 2013 13:47:30 -0700 Subject: [PATCH] jcheck: allow bug id line to be ignored In-Reply-To: <516886B6.5040304@redhat.com> References: <514B5A6C.9050708@redhat.com>, <20130408151913.75918@eggemoggin.niobe.net>, <516886B6.5040304@redhat.com> Message-ID: <20130507134730.190976@eggemoggin.niobe.net> 2013/4/12 8:12 -0700, omajid at redhat.com: > Thanks for pointing me to the right place to add tests. I had two > questions about tests in jcheck that I couldn't find answers to: > > 1. What's the difference between 'make test' and 'make tests'? Should I > be running both all the time or prefer one over the other? "make test" runs all the tests. "make tests" creates the prebuilt test cases by running mktests.sh; it doesn't actually run any tests. > 2. Looking at runtests.sh, some tests set up jcheck: > > 259 hg add -R z z/.jcheck/conf > 260 cat >z/.hg/hgrc <<___ > 261 [extensions] > 262 jcheck = $(pwd)/jcheck.py > 263 [hooks] > 264 pretxncommit.jcheck=python:jcheck.hook > 265 ___ > > While others dont: > > 208 # Lax bugids > 209 echo "-- $r lax bug ids" > 210 rm -rf z > 211 hg init z > 212 mkdir z/.jcheck > 213 cat >z/.jcheck/conf <<___ > 214 project=jdk7 > 215 comments=lax > 216 bugids=lax > 217 ___ > 218 hg add -R z z/.jcheck/conf > 219 > 220 if HGUSER=$setup_author hg ci -R z -m '1234: Silly bugid'; then > true; else fail; fi > 221 r=$(expr $r + 1) Hmm, looks like you found a bug in some existing tests. > Is it expected that jcheck will be hooked into the global ~/.hgrc file? > (Is that a good expectation for unit tests?) No, not at all, and that would be an inappropriate expectation. The "export HGRCPATH=" at the top of runtests.sh prevents that. The lax bugid test you quoted above passes because it just doesn't run with jcheck enabled. I'll fix that. - Mark From omajid at redhat.com Tue May 7 16:58:27 2013 From: omajid at redhat.com (Omair Majid) Date: Tue, 07 May 2013 19:58:27 -0400 Subject: [PATCH] jcheck: allow bug id line to be ignored In-Reply-To: <20130507133315.949928@eggemoggin.niobe.net> References: <514B5A6C.9050708@redhat.com>, <20130408155342.3301@eggemoggin.niobe.net>, <51802CF7.7010809@redhat.com> <20130507133315.949928@eggemoggin.niobe.net> Message-ID: <51899523.5080302@redhat.com> Hi Mark, On 05/07/2013 04:33 PM, mark.reinhold at oracle.com wrote: > 2013/4/30 6:43 -0700, omajid at redhat.com: >> On 04/08/2013 06:53 PM, mark.reinhold at oracle.com wrote: >>> I think it'd be cleaner just to load the bugids lazily. As a side >>> benefit that will save a bit of time too. >> >> Done. Updated webrev (including tests) is here: >> http://cr.openjdk.java.net/~omajid/webrevs/jcheck-ignore-bug-line/01/ > > As much as I'm a fan of higher-order programming, passing a lambda > into checker.__init__ for populate_repo_bugids() seems like overkill. > The __init__ method can just call repo_bugids(ui, repo) directly when > self.bugids_ignore turns out to be false. Fair enough. I figured that the code was placed outside the class for a reason, but calling repo_bugids in the __init__ method definitely looks cleaner. > On line 143, I'd make the regex a bit more precise, something like: > > bug_ident = re.compile("(([A-Z][A-Z0-9]+-)?[0-9]+):") I went with your suggestion, but do you know of any others who might be interested in (or affected by) this regex change? > The new unit tests look fine. Thanks for the comments. New webrev: http://cr.openjdk.java.net/~omajid/webrevs/jcheck-ignore-bug-line/02/ Thanks, Omair -- PGP Key: 66484681 (http://pgp.mit.edu/) Fingerprint = F072 555B 0A17 3957 4E95 0056 F286 F14F 6648 4681 From mark.reinhold at oracle.com Tue May 7 17:20:15 2013 From: mark.reinhold at oracle.com (mark.reinhold at oracle.com) Date: Tue, 07 May 2013 17:20:15 -0700 Subject: [PATCH] jcheck: allow bug id line to be ignored In-Reply-To: <51899523.5080302@redhat.com> References: <514B5A6C.9050708@redhat.com>, <20130507133315.949928@eggemoggin.niobe.net>, <51899523.5080302@redhat.com> Message-ID: <20130507172015.768588@eggemoggin.niobe.net> 2013/5/7 9:58 -0700, omajid at redhat.com: > On 05/07/2013 04:33 PM, mark.reinhold at oracle.com wrote: >> 2013/4/30 6:43 -0700, omajid at redhat.com: >>> On 04/08/2013 06:53 PM, mark.reinhold at oracle.com wrote: >> On line 143, I'd make the regex a bit more precise, something like: >> >> bug_ident = re.compile("(([A-Z][A-Z0-9]+-)?[0-9]+):") > > I went with your suggestion, but do you know of any others who might be > interested in (or affected by) this regex change? No, not offhand. So far jcheck has only been used in repos with purely numeric bugids. >> The new unit tests look fine. > > Thanks for the comments. > > New webrev: > http://cr.openjdk.java.net/~omajid/webrevs/jcheck-ignore-bug-line/02/ Looks good. Shall I push that patch for you, or do you want to qfinish it and send me an hg bundle? (If the latter, please hg qref -u omajid to get the committer name set properly.) - Mark From omajid at redhat.com Wed May 8 07:13:25 2013 From: omajid at redhat.com (Omair Majid) Date: Wed, 08 May 2013 10:13:25 -0400 Subject: [PATCH] jcheck: allow bug id line to be ignored In-Reply-To: <20130507172015.768588@eggemoggin.niobe.net> References: <514B5A6C.9050708@redhat.com>, <20130507133315.949928@eggemoggin.niobe.net>, <51899523.5080302@redhat.com> <20130507172015.768588@eggemoggin.niobe.net> Message-ID: <518A5D85.5060307@redhat.com> Hi, On 05/07/2013 08:20 PM, mark.reinhold at oracle.com wrote: > Looks good. Shall I push that patch for you, or do you want to qfinish > it and send me an hg bundle? (If the latter, please hg qref -u omajid > to get the committer name set properly.) I am attaching the bundle, but please do what's most convenient for you. Thanks, Omair -- PGP Key: 66484681 (http://pgp.mit.edu/) Fingerprint = F072 555B 0A17 3957 4E95 0056 F286 F14F 6648 4681 From mark.reinhold at oracle.com Wed May 8 14:56:59 2013 From: mark.reinhold at oracle.com (mark.reinhold at oracle.com) Date: Wed, 08 May 2013 21:56:59 +0000 Subject: hg: code-tools/jcheck: Allow bug id line to be ignored Message-ID: <20130508215659.812C648905@hg.openjdk.java.net> Changeset: 73390716f288 Author: omajid Date: 2013-05-08 10:00 -0400 URL: http://hg.openjdk.java.net/code-tools/jcheck/rev/73390716f288 Allow bug id line to be ignored ! jcheck.py ! runtests.sh From mark.reinhold at oracle.com Wed May 8 14:58:43 2013 From: mark.reinhold at oracle.com (mark.reinhold at oracle.com) Date: Wed, 08 May 2013 21:58:43 +0000 Subject: hg: code-tools/jcheck/dist: jcheck 73390716f288 2013-05-08 10:00:56 -0400 Message-ID: <20130508215843.EF9BD48906@hg.openjdk.java.net> Changeset: 0ee86975a5c3 Author: mr Date: 2013-05-08 10:00 -0400 URL: http://hg.openjdk.java.net/code-tools/jcheck/dist/rev/0ee86975a5c3 jcheck 73390716f288 2013-05-08 10:00:56 -0400 ! jcheck.py From mark.reinhold at oracle.com Wed May 8 14:59:39 2013 From: mark.reinhold at oracle.com (mark.reinhold at oracle.com) Date: Wed, 08 May 2013 14:59:39 -0700 Subject: [PATCH] jcheck: allow bug id line to be ignored In-Reply-To: <518A5D85.5060307@redhat.com> References: <514B5A6C.9050708@redhat.com>, <20130507172015.768588@eggemoggin.niobe.net>, <518A5D85.5060307@redhat.com> Message-ID: <20130508145939.53645@eggemoggin.niobe.net> 2013/5/8 0:13 -0700, omajid at redhat.com: > I am attaching the bundle, but please do what's most convenient for you. Bundle pushed, and new version of jcheck installed on hg.openjdk.java.net. - Mark From omajid at redhat.com Thu May 9 08:43:05 2013 From: omajid at redhat.com (Omair Majid) Date: Thu, 09 May 2013 11:43:05 -0400 Subject: [PATCH] jcheck: allow bug id line to be ignored In-Reply-To: <20130508145939.53645@eggemoggin.niobe.net> References: <514B5A6C.9050708@redhat.com>, <20130507172015.768588@eggemoggin.niobe.net>, <518A5D85.5060307@redhat.com> <20130508145939.53645@eggemoggin.niobe.net> Message-ID: <518BC409.1080700@redhat.com> On 05/08/2013 05:59 PM, mark.reinhold at oracle.com wrote: > Bundle pushed, and new version of jcheck installed on hg.openjdk.java.net. Thank you! -- PGP Key: 66484681 (http://pgp.mit.edu/) Fingerprint = F072 555B 0A17 3957 4E95 0056 F286 F14F 6648 4681 From mark.reinhold at oracle.com Thu May 9 14:23:25 2013 From: mark.reinhold at oracle.com (mark.reinhold at oracle.com) Date: Thu, 09 May 2013 21:23:25 +0000 Subject: hg: code-tools/jcheck/dist: jcheck c45a917ff927 2013-05-09 14:17:37 -0700 Message-ID: <20130509212325.8F0104895C@hg.openjdk.java.net> Changeset: d6c70b3b5d98 Author: mr Date: 2013-05-09 14:17 -0700 URL: http://hg.openjdk.java.net/code-tools/jcheck/dist/rev/d6c70b3b5d98 jcheck c45a917ff927 2013-05-09 14:17:37 -0700 ! jcheck.py From mark.reinhold at oracle.com Thu May 9 14:23:15 2013 From: mark.reinhold at oracle.com (mark.reinhold at oracle.com) Date: Thu, 09 May 2013 21:23:15 +0000 Subject: hg: code-tools/jcheck: Don't collect non-numeric bugids Message-ID: <20130509212315.504874895B@hg.openjdk.java.net> Changeset: c45a917ff927 Author: mr Date: 2013-05-09 14:17 -0700 URL: http://hg.openjdk.java.net/code-tools/jcheck/rev/c45a917ff927 Don't collect non-numeric bugids ! jcheck.py ! runtests.sh From omajid at redhat.com Thu May 9 14:50:34 2013 From: omajid at redhat.com (Omair Majid) Date: Thu, 09 May 2013 17:50:34 -0400 Subject: hg: code-tools/jcheck: Don't collect non-numeric bugids In-Reply-To: <20130509212315.504874895B@hg.openjdk.java.net> References: <20130509212315.504874895B@hg.openjdk.java.net> Message-ID: <518C1A2A.6050202@redhat.com> On 05/09/2013 05:23 PM, mark.reinhold at oracle.com wrote: > Changeset: c45a917ff927 > Author: mr > Date: 2013-05-09 14:17 -0700 > URL: http://hg.openjdk.java.net/code-tools/jcheck/rev/c45a917ff927 > > Don't collect non-numeric bugids > > ! jcheck.py > ! runtests.sh > I guess I made a mistake in my patch? Hope I didn't break anything. I am a little confused about this change in the patch, though: > -if HGUSER=$setup_author hg ci -R z -m "6-2: test separate bugids > -Reviewed-by: $pass_author"; then fail; else true; fi > +echo "-- $r ignore bug ids 4" > +rm -rf z > +hg init z Doesn't that prevent test 3 from running? Thanks, Omair -- PGP Key: 66484681 (http://pgp.mit.edu/) Fingerprint = F072 555B 0A17 3957 4E95 0056 F286 F14F 6648 4681 From mark.reinhold at oracle.com Thu May 9 15:06:52 2013 From: mark.reinhold at oracle.com (mark.reinhold at oracle.com) Date: Thu, 09 May 2013 15:06:52 -0700 Subject: hg: code-tools/jcheck: Don't collect non-numeric bugids In-Reply-To: <518C1A2A.6050202@redhat.com> References: <20130509212315.504874895B@hg.openjdk.java.net>, <518C1A2A.6050202@redhat.com> Message-ID: <20130509150652.39756@eggemoggin.niobe.net> 2013/5/9 7:50 -0700, omajid at redhat.com: > On 05/09/2013 05:23 PM, mark.reinhold at oracle.com wrote: >> Changeset: c45a917ff927 >> Author: mr >> Date: 2013-05-09 14:17 -0700 >> URL: http://hg.openjdk.java.net/code-tools/jcheck/rev/c45a917ff927 >> >> Don't collect non-numeric bugids >> >> ! jcheck.py >> ! runtests.sh > > I guess I made a mistake in my patch? Hope I didn't break anything. Neither of us considered the case of a repo with JIRA-style bugids in which bugids are not ignored, and just such a repo turned up today. > I am a little confused about this change in the patch, though: > >> -if HGUSER=$setup_author hg ci -R z -m "6-2: test separate bugids >> -Reviewed-by: $pass_author"; then fail; else true; fi >> +echo "-- $r ignore bug ids 4" >> +rm -rf z >> +hg init z > > Doesn't that prevent test 3 from running? Duh, my mistake. I'll fix that; thanks for noticing it. - Mark From mark.reinhold at oracle.com Thu May 9 15:09:22 2013 From: mark.reinhold at oracle.com (mark.reinhold at oracle.com) Date: Thu, 09 May 2013 22:09:22 +0000 Subject: hg: code-tools/jcheck: Restore missing lines to runtest.sh Message-ID: <20130509220922.E151B48963@hg.openjdk.java.net> Changeset: d447b345b98a Author: mr Date: 2013-05-09 15:09 -0700 URL: http://hg.openjdk.java.net/code-tools/jcheck/rev/d447b345b98a Restore missing lines to runtest.sh ! runtests.sh From chris.hegarty at oracle.com Tue May 14 03:53:39 2013 From: chris.hegarty at oracle.com (Chris Hegarty) Date: Tue, 14 May 2013 11:53:39 +0100 Subject: Proposal: jcheck restrict addition of subrepo files Message-ID: <519217B3.9080601@oracle.com> Hi, Recently I ran into an issue in a staging forest where a developer, when working on a partial forest, accidentally created a directory structure in a repository that conflicted with that of one of the sub/inner forest repositories. For example, hypothetically, if a developer was working solely on the top level repo, they could accidentally add jdk/Test.java. This would cause problems when trying to clone the full forest, that contains the jdk repo. Granted this doesn't occur that often, but when it does it consumes unnecessary cycles, and can be disruptive. I've prototyped a very limited solution whereby a repo, through its jcheck conf, can define what inner|sub|containing repos are part of its forest. jcheck can then trivially prevent files being added that would conflict with this. Trivial jcheck webrev (python level:novice) : http://cr.openjdk.java.net/~chegar/jcheck_subrepos_webrev.00/webrev/jcheck.py.udiff.html Example update to .jcheck/conf in the jdk8 top level repo: diff -r 69b773a221b9 .jcheck/conf --- a/.jcheck/conf Thu May 09 10:03:08 2013 -0700 +++ b/.jcheck/conf Tue May 14 11:25:33 2013 +0100 @@ -1,1 +1,2 @@ project=jdk8 project=jdk8 +subrepos=corba,jaxp,jaxws,langtools,jdk,hotspot,nashorn Note: Since jcheck current ignores name/value pairs in the conf that are not needed, no special co-ordination would be required between updating the jcheck/conf in individual forests and the release of an updated jcheck version. The enforcement will only happen over time, when both of the above have occurred. Thanks, -Chris. From mark.reinhold at oracle.com Thu May 16 08:47:37 2013 From: mark.reinhold at oracle.com (mark.reinhold at oracle.com) Date: Thu, 16 May 2013 08:47:37 -0700 Subject: Proposal: jcheck restrict addition of subrepo files In-Reply-To: <519217B3.9080601@oracle.com> References: <519217B3.9080601@oracle.com> Message-ID: <20130516084737.950263@eggemoggin.niobe.net> 2013/5/13 20:53 -0700, chris.hegarty at oracle.com: > Recently I ran into an issue in a staging forest where a developer, when > working on a partial forest, accidentally created a directory structure > in a repository that conflicted with that of one of the sub/inner forest > repositories. > > ... > > I've prototyped a very limited solution whereby a repo, through its > jcheck conf, can define what inner|sub|containing repos are part of its > forest. jcheck can then trivially prevent files being added that would > conflict with this. The incident to which you refer involved the creation of a directory named jdk/test/closed, which conflicts with an Oracle-internal sub-repo of closed regression tests. It wouldn't be appropriate for a .jcheck/conf file in an open repo to refer to a private repo, so I don't think this is the right approach. I'm not convinced that it's worth trying to prevent this sort of mistake is in jcheck. As far as I know it's happened only a few times over the last seven years. - Mark From chris.hegarty at oracle.com Thu May 16 09:05:52 2013 From: chris.hegarty at oracle.com (Chris Hegarty) Date: Thu, 16 May 2013 17:05:52 +0100 Subject: Proposal: jcheck restrict addition of subrepo files In-Reply-To: <20130516084737.950263@eggemoggin.niobe.net> References: <519217B3.9080601@oracle.com> <20130516084737.950263@eggemoggin.niobe.net> Message-ID: <519503E0.4090203@oracle.com> On 05/16/2013 04:47 PM, mark.reinhold at oracle.com wrote: > ... > The incident to which you refer involved the creation of a directory > named jdk/test/closed, which conflicts with an Oracle-internal sub-repo > of closed regression tests. It wouldn't be appropriate for a .jcheck/conf > file in an open repo to refer to a private repo, so I don't think this is > the right approach. Yes, the problem is with down stream projects of OpenJDK. I don't see that the problem is limited to Oracle's sub-repos, but I don't know if other projects are overlaying sub-repos. If they are then they could specify them also. > I'm not convinced that it's worth trying to prevent this sort of mistake > is in jcheck. As far as I know it's happened only a few times over the > last seven years. True, it is not that common. Maybe not worth trying to fix. Thanks, -Chris.