From erikj at openjdk.org Mon Aug 1 21:38:33 2022 From: erikj at openjdk.org (Erik Joelsson) Date: Mon, 1 Aug 2022 21:38:33 GMT Subject: RFR: 1519: MailmanListReader misses new month on the 1st Message-ID: After expanding the tests for MailmanListReader in [SKARA-1513](https://bugs.openjdk.org/browse/SKARA-1513), I've discovered that some tests started failing on the 1st of the month. These failures actually uncovered a real problem, where the current logic would not start querying a new month archive until the 2nd of the new month. This is basically an off by one error in the loop condition where we generate the list of months to iterate over. ------------- Commit messages: - SKARA-1519 Changes: https://git.openjdk.org/skara/pull/1345/files Webrev: https://webrevs.openjdk.org/?repo=skara&pr=1345&range=00 Issue: https://bugs.openjdk.org/browse/SKARA-1519 Stats: 2 lines in 1 file changed: 1 ins; 0 del; 1 mod Patch: https://git.openjdk.org/skara/pull/1345.diff Fetch: git fetch https://git.openjdk.org/skara pull/1345/head:pull/1345 PR: https://git.openjdk.org/skara/pull/1345 From ihse at openjdk.org Mon Aug 1 21:42:03 2022 From: ihse at openjdk.org (Magnus Ihse Bursie) Date: Mon, 1 Aug 2022 21:42:03 GMT Subject: RFR: 1519: MailmanListReader misses new month on the 1st In-Reply-To: References: Message-ID: On Mon, 1 Aug 2022 21:32:25 GMT, Erik Joelsson wrote: > After expanding the tests for MailmanListReader in [SKARA-1513](https://bugs.openjdk.org/browse/SKARA-1513), I've discovered that some tests started failing on the 1st of the month. These failures actually uncovered a real problem, where the current logic would not start querying a new month archive until the 2nd of the new month. This is basically an off by one error in the loop condition where we generate the list of months to iterate over. Marked as reviewed by ihse (Reviewer). ------------- PR: https://git.openjdk.org/skara/pull/1345 From erikj at openjdk.org Mon Aug 1 22:16:31 2022 From: erikj at openjdk.org (Erik Joelsson) Date: Mon, 1 Aug 2022 22:16:31 GMT Subject: Integrated: 1519: MailmanListReader misses new month on the 1st In-Reply-To: References: Message-ID: On Mon, 1 Aug 2022 21:32:25 GMT, Erik Joelsson wrote: > After expanding the tests for MailmanListReader in [SKARA-1513](https://bugs.openjdk.org/browse/SKARA-1513), I've discovered that some tests started failing on the 1st of the month. These failures actually uncovered a real problem, where the current logic would not start querying a new month archive until the 2nd of the new month. This is basically an off by one error in the loop condition where we generate the list of months to iterate over. This pull request has now been integrated. Changeset: 0f737485 Author: Erik Joelsson URL: https://git.openjdk.org/skara/commit/0f737485aea2da1740db62547e8e17a8f7b64c80 Stats: 2 lines in 1 file changed: 1 ins; 0 del; 1 mod 1519: MailmanListReader misses new month on the 1st Reviewed-by: ihse ------------- PR: https://git.openjdk.org/skara/pull/1345 From erikj at openjdk.org Mon Aug 1 22:57:27 2022 From: erikj at openjdk.org (Erik Joelsson) Date: Mon, 1 Aug 2022 22:57:27 GMT Subject: RFR: 1515: Mailing list parsing of PR links broken since openjdk.org transition Message-ID: The mlbridge bot uses PR links in emails to identify which PR an email thread belongs to. This is then used to correctly post emails from the archive as comments on PRs. The PR links in the emails are transformed to an openjdk style URL: github.com/openjdk -> git.openjdk.org The URL matching logic knows about this transformation and tries to reverse it before matching PR links. The problem is that before June 9, the rewritten URLs were git.openjdk.java.net. The current logic can't handle multiple different transformation patterns, so any emails found with the old URL can't be matched correctly to PRs. In the github host config for the mlbridge bot, we currently have this config: "weburl": { "pattern": "^https://github.com/openjdk/(.*)$", "replacement": "https://git.openjdk.org/$1", } This patch adds the ability to add alternate replacement patterns. By using this, we can search for both the old openjdk.java.net URLs as well as the new openjdk.org one. It will look like this: "weburl": { "pattern": "^https://github.com/openjdk/(.*)$", "replacement": "https://git.openjdk.org/$1", "altreplacements": [ "https://git.openjdk.java.net/$1", ], } ------------- Commit messages: - SKARA-1515 Changes: https://git.openjdk.org/skara/pull/1346/files Webrev: https://webrevs.openjdk.org/?repo=skara&pr=1346&range=00 Issue: https://bugs.openjdk.org/browse/SKARA-1515 Stats: 90 lines in 5 files changed: 57 ins; 3 del; 30 mod Patch: https://git.openjdk.org/skara/pull/1346.diff Fetch: git fetch https://git.openjdk.org/skara pull/1346/head:pull/1346 PR: https://git.openjdk.org/skara/pull/1346 From erikj at openjdk.org Thu Aug 4 13:57:02 2022 From: erikj at openjdk.org (Erik Joelsson) Date: Thu, 4 Aug 2022 13:57:02 GMT Subject: RFR: Improve logging in PR bot Message-ID: This very minor patch adds the repo names to log messages in `getPeriodicItems()`. I'm going to look into performance issues with the bots, and just seeing a bunch of Fetching all open pull requests Fetching all open pull requests in the log without any further context isn't helping. I also stumbled over an unused import as well as a bad use of LinkedList while in the area. ------------- Commit messages: - Improve logging Changes: https://git.openjdk.org/skara/pull/1347/files Webrev: https://webrevs.openjdk.org/?repo=skara&pr=1347&range=00 Stats: 4 lines in 1 file changed: 0 ins; 1 del; 3 mod Patch: https://git.openjdk.org/skara/pull/1347.diff Fetch: git fetch https://git.openjdk.org/skara pull/1347/head:pull/1347 PR: https://git.openjdk.org/skara/pull/1347 From kcr at openjdk.org Thu Aug 4 14:10:42 2022 From: kcr at openjdk.org (Kevin Rushforth) Date: Thu, 4 Aug 2022 14:10:42 GMT Subject: RFR: Improve logging in PR bot In-Reply-To: References: Message-ID: On Thu, 4 Aug 2022 13:52:40 GMT, Erik Joelsson wrote: > This very minor patch adds the repo names to log messages in `getPeriodicItems()`. I'm going to look into performance issues with the bots, and just seeing a bunch of > > > Fetching all open pull requests > Fetching all open pull requests > > > in the log without any further context isn't helping. > > I also stumbled over an unused import as well as a bad use of LinkedList while in the area. Marked as reviewed by kcr (Reviewer). ------------- PR: https://git.openjdk.org/skara/pull/1347 From erikj at openjdk.org Thu Aug 4 21:08:50 2022 From: erikj at openjdk.org (Erik Joelsson) Date: Thu, 4 Aug 2022 21:08:50 GMT Subject: Integrated: Improve logging in PR bot In-Reply-To: References: Message-ID: On Thu, 4 Aug 2022 13:52:40 GMT, Erik Joelsson wrote: > This very minor patch adds the repo names to log messages in `getPeriodicItems()`. I'm going to look into performance issues with the bots, and just seeing a bunch of > > > Fetching all open pull requests > Fetching all open pull requests > > > in the log without any further context isn't helping. > > I also stumbled over an unused import as well as a bad use of LinkedList while in the area. This pull request has now been integrated. Changeset: b327f717 Author: Erik Joelsson URL: https://git.openjdk.org/skara/commit/b327f717d00efa36d7acc1730316dc8da61592cf Stats: 4 lines in 1 file changed: 0 ins; 1 del; 3 mod Improve logging in PR bot Reviewed-by: kcr ------------- PR: https://git.openjdk.org/skara/pull/1347 From kcr at openjdk.org Fri Aug 5 13:17:28 2022 From: kcr at openjdk.org (Kevin Rushforth) Date: Fri, 5 Aug 2022 13:17:28 GMT Subject: RFR: 1515: Mailing list parsing of PR links broken since openjdk.org transition In-Reply-To: References: Message-ID: On Mon, 1 Aug 2022 22:52:29 GMT, Erik Joelsson wrote: > The mlbridge bot uses PR links in emails to identify which PR an email thread belongs to. This is then used to correctly post emails from the archive as comments on PRs. The PR links in the emails are transformed to an openjdk style URL: > > github.com/openjdk -> git.openjdk.org > > The URL matching logic knows about this transformation and tries to reverse it before matching PR links. The problem is that before June 9, the rewritten URLs were git.openjdk.java.net. The current logic can't handle multiple different transformation patterns, so any emails found with the old URL can't be matched correctly to PRs. > > In the github host config for the mlbridge bot, we currently have this config: > > > "weburl": { > "pattern": "^https://github.com/openjdk/(.*)$", > "replacement": "https://git.openjdk.org/$1", > } > > This patch adds the ability to add alternate replacement patterns. By using this, we can search for both the old openjdk.java.net URLs as well as the new openjdk.org one. It will look like this: > > > "weburl": { > "pattern": "^https://github.com/openjdk/(.*)$", > "replacement": "https://git.openjdk.org/$1", > "altreplacements": [ > "https://git.openjdk.java.net/$1", > ], > } Looks good. ------------- Marked as reviewed by kcr (Reviewer). PR: https://git.openjdk.org/skara/pull/1346 From erikj at openjdk.org Fri Aug 5 13:27:44 2022 From: erikj at openjdk.org (Erik Joelsson) Date: Fri, 5 Aug 2022 13:27:44 GMT Subject: Integrated: 1515: Mailing list parsing of PR links broken since openjdk.org transition In-Reply-To: References: Message-ID: On Mon, 1 Aug 2022 22:52:29 GMT, Erik Joelsson wrote: > The mlbridge bot uses PR links in emails to identify which PR an email thread belongs to. This is then used to correctly post emails from the archive as comments on PRs. The PR links in the emails are transformed to an openjdk style URL: > > github.com/openjdk -> git.openjdk.org > > The URL matching logic knows about this transformation and tries to reverse it before matching PR links. The problem is that before June 9, the rewritten URLs were git.openjdk.java.net. The current logic can't handle multiple different transformation patterns, so any emails found with the old URL can't be matched correctly to PRs. > > In the github host config for the mlbridge bot, we currently have this config: > > > "weburl": { > "pattern": "^https://github.com/openjdk/(.*)$", > "replacement": "https://git.openjdk.org/$1", > } > > This patch adds the ability to add alternate replacement patterns. By using this, we can search for both the old openjdk.java.net URLs as well as the new openjdk.org one. It will look like this: > > > "weburl": { > "pattern": "^https://github.com/openjdk/(.*)$", > "replacement": "https://git.openjdk.org/$1", > "altreplacements": [ > "https://git.openjdk.java.net/$1", > ], > } This pull request has now been integrated. Changeset: da062e7b Author: Erik Joelsson URL: https://git.openjdk.org/skara/commit/da062e7bd6f7716e66135c0248cb2dab7cfd4651 Stats: 90 lines in 5 files changed: 57 ins; 3 del; 30 mod 1515: Mailing list parsing of PR links broken since openjdk.org transition Reviewed-by: kcr ------------- PR: https://git.openjdk.org/skara/pull/1346 From erikj at openjdk.org Fri Aug 5 17:57:45 2022 From: erikj at openjdk.org (Erik Joelsson) Date: Fri, 5 Aug 2022 17:57:45 GMT Subject: RFR: 1529: Optimize Census namespace instantiation Message-ID: This patch adds a new factory method `Census::parseNamespace`, which enables initialization of a single Namespace directly from a HostedRepository (without having to clone the census repo to local disk first). The motivation behind this is bot performance. The Census is loaded in many WorkItems. Cloning/fetching the complete census repo to local disk is relatively expensive compared to a couple of REST calls to just read files directly from the remote repo. Especially as our REST client already leverages caching through ETag. This particular optimization can however only be applied to the IssueNotifier in the NotifyBot, where I doubt it will make that much of a difference. The main performance gain is realized in a different bot which currently isn't public (but these gains will impact all Skara users). I chose to at least for now, only apply this for initializing a single Namespace as that can be done by reading only 2 files from the remote, so the performance gain is very clear. Other bots either need to read considerably more files, or have the need to update the census. This optimization idea may be expanded in the future if I see a need and noticeable gains from it. In order to make HostedRepository available to the Census class, I had to get rid of several existing module dependencies to avoid circular dependencies. Fortunately this wasn't too hard. It made no sense to me that `org.openjdk.skara.forge` would require `org.openjdk.skara.census`. There was just one static method `PullRequestUtils` in the forge module that used a reference to the census module, and this static method was only used from one module `org.openjdk.skara.bots.pr`, so I simply moved it to `CheckablePullRequest` which is kind of a utility class in that module. The rest of the offending dependencies were just unused, so I cleaned out a fair few of those. In the `IssueNotifier`, in addition to using the new initialization method, I also added lazy initialization and reusing of the `CensusInstance` class, to avoid re-initializing the same data for every round in loops. ------------- Commit messages: - SKARA-1529 Changes: https://git.openjdk.org/skara/pull/1348/files Webrev: https://webrevs.openjdk.org/?repo=skara&pr=1348&range=00 Issue: https://bugs.openjdk.org/browse/SKARA-1529 Stats: 209 lines in 18 files changed: 120 ins; 64 del; 25 mod Patch: https://git.openjdk.org/skara/pull/1348.diff Fetch: git fetch https://git.openjdk.org/skara pull/1348/head:pull/1348 PR: https://git.openjdk.org/skara/pull/1348 From erikj at openjdk.org Fri Aug 5 21:53:23 2022 From: erikj at openjdk.org (Erik Joelsson) Date: Fri, 5 Aug 2022 21:53:23 GMT Subject: RFR: 1530: Race with CheckWorkItem in PR bot Message-ID: This patch fixes a race in the PR bot. Currently, all WorkItems that process PRs get an instance of the PullRequest that was fetched before the WorkItem instance was created. The data in that PullRequest instance (the parts that aren't dynamically fetched on method calls) may then be stale before the WorkItem even starts running. With this patch, I'm changing PullRequestWorkItem (the super class of all WorkItems that handle PRs), to only be instantiated with the ID of the PR, and then fetches the actual PullRequest object at the start of the `run()` method. When the `run()` method is called, we are guaranteed to be the only WorkItem currently processing the particular PR, so fetching data there should guarantee a coherent and current view. In many cases where PullRequestWorkItems were created, the creator first refreshed the PullRequest object to provide an updated view, reflecting any changes made by the creator. These refreshes are no longer necessary, so I've removed them all. ------------- Commit messages: - SKARA-1530 Changes: https://git.openjdk.org/skara/pull/1349/files Webrev: https://webrevs.openjdk.org/?repo=skara&pr=1349&range=00 Issue: https://bugs.openjdk.org/browse/SKARA-1530 Stats: 54 lines in 5 files changed: 19 ins; 8 del; 27 mod Patch: https://git.openjdk.org/skara/pull/1349.diff Fetch: git fetch https://git.openjdk.org/skara pull/1349/head:pull/1349 PR: https://git.openjdk.org/skara/pull/1349 From kcr at openjdk.org Fri Aug 5 22:07:05 2022 From: kcr at openjdk.org (Kevin Rushforth) Date: Fri, 5 Aug 2022 22:07:05 GMT Subject: RFR: 1530: Race with CheckWorkItem in PR bot In-Reply-To: References: Message-ID: On Fri, 5 Aug 2022 21:49:18 GMT, Erik Joelsson wrote: > This patch fixes a race in the PR bot. > > Currently, all WorkItems that process PRs get an instance of the PullRequest that was fetched before the WorkItem instance was created. The data in that PullRequest instance (the parts that aren't dynamically fetched on method calls) may then be stale before the WorkItem even starts running. > > With this patch, I'm changing PullRequestWorkItem (the super class of all WorkItems that handle PRs), to only be instantiated with the ID of the PR, and then fetches the actual PullRequest object at the start of the `run()` method. When the `run()` method is called, we are guaranteed to be the only WorkItem currently processing the particular PR, so fetching data there should guarantee a coherent and current view. > > In many cases where PullRequestWorkItems were created, the creator first refreshed the PullRequest object to provide an updated view, reflecting any changes made by the creator. These refreshes are no longer necessary, so I've removed them all. Looks like a good solution to me. bots/pr/src/main/java/org/openjdk/skara/bots/pr/PullRequestWorkItem.java line 49: > 47: return true; > 48: } > 49: if (!(prId.equals(otherItem.prId) && bot.repo().isSame(otherItem.bot.repo()))) { This means that two work requests can run concurrently if either the `prId` is different or the `bot repo` is different. I presume this is because the `prId` is relative to the repo (which makes sense), right? ------------- Marked as reviewed by kcr (Reviewer). PR: https://git.openjdk.org/skara/pull/1349 From erikj at openjdk.org Fri Aug 5 22:20:04 2022 From: erikj at openjdk.org (Erik Joelsson) Date: Fri, 5 Aug 2022 22:20:04 GMT Subject: RFR: 1530: Race with CheckWorkItem in PR bot In-Reply-To: References: Message-ID: On Fri, 5 Aug 2022 22:03:04 GMT, Kevin Rushforth wrote: >> This patch fixes a race in the PR bot. >> >> Currently, all WorkItems that process PRs get an instance of the PullRequest that was fetched before the WorkItem instance was created. The data in that PullRequest instance (the parts that aren't dynamically fetched on method calls) may then be stale before the WorkItem even starts running. >> >> With this patch, I'm changing PullRequestWorkItem (the super class of all WorkItems that handle PRs), to only be instantiated with the ID of the PR, and then fetches the actual PullRequest object at the start of the `run()` method. When the `run()` method is called, we are guaranteed to be the only WorkItem currently processing the particular PR, so fetching data there should guarantee a coherent and current view. >> >> In many cases where PullRequestWorkItems were created, the creator first refreshed the PullRequest object to provide an updated view, reflecting any changes made by the creator. These refreshes are no longer necessary, so I've removed them all. > > bots/pr/src/main/java/org/openjdk/skara/bots/pr/PullRequestWorkItem.java line 49: > >> 47: return true; >> 48: } >> 49: if (!(prId.equals(otherItem.prId) && bot.repo().isSame(otherItem.bot.repo()))) { > > This means that two work requests can run concurrently if either the `prId` is different or the `bot repo` is different. I presume this is because the `prId` is relative to the repo (which makes sense), right? Correct. The prId is the number we see in PR links, e.g. 1349 for this one. The implementation of `PullRequest::isSame` is equivalent to the new code. The problem was that I can no longer guarantee to have the PullRequest object available when `concurrentWith` is called (rather, it's guaranteed to not be there), so I had to basically copy the code, and get the `HostedRepository` from the bot instead of the pr. This condition is supposed to be exactly the same as it was pre patch. ------------- PR: https://git.openjdk.org/skara/pull/1349 From gli at openjdk.org Sat Aug 6 10:25:32 2022 From: gli at openjdk.org (Guoxiong Li) Date: Sat, 6 Aug 2022 10:25:32 GMT Subject: RFR: 1463: Include starting branch in back port instructions [v2] In-Reply-To: References: Message-ID: > Hi all, > > This patch adds the suggested git command about fetching the newest code > from the target branch before solving the conflicts. > > Thanks for taking the time to review. > > Best Regards, > -- Guoxiong Guoxiong Li has updated the pull request incrementally with one additional commit since the last revision: Tweak the message. ------------- Changes: - all: https://git.openjdk.org/skara/pull/1344/files - new: https://git.openjdk.org/skara/pull/1344/files/246b583f..15f19d29 Webrevs: - full: https://webrevs.openjdk.org/?repo=skara&pr=1344&range=01 - incr: https://webrevs.openjdk.org/?repo=skara&pr=1344&range=00-01 Stats: 2 lines in 1 file changed: 1 ins; 0 del; 1 mod Patch: https://git.openjdk.org/skara/pull/1344.diff Fetch: git fetch https://git.openjdk.org/skara pull/1344/head:pull/1344 PR: https://git.openjdk.org/skara/pull/1344 From gli at openjdk.org Sat Aug 6 10:30:26 2022 From: gli at openjdk.org (Guoxiong Li) Date: Sat, 6 Aug 2022 10:30:26 GMT Subject: RFR: 1463: Include starting branch in back port instructions In-Reply-To: References: Message-ID: <-s6ov-ovIIKCoCD5m-a3NyK-R4zeKaCwRKm7TtJ8FH0=.1e4c5225-dba5-4ff4-b98b-fee3f70e65e8@github.com> On Sun, 31 Jul 2022 01:43:55 GMT, David Holmes wrote: > Having just done a couple of such backports I would have found the proposed addition to be extremely confusing. I think all you need to do is tweak: > > > To manually resolve these conflicts run the following commands in your personal fork of $REPO > > to include mention of the appropriate branch, and/or have a separate instruction telling you have to get the right branch if you don't already have it. But don't change the existing steps which worked perfectly fine. @dholmes-ora Thanks for the comment. I tweak the message you mentioned. But it seems not be necessary to separate the commands into two parts, and the newly added steps seems good because someone may miss such steps sometimes. These steps are some notes and can remind the developers if they forget the related git commands. ------------- PR: https://git.openjdk.org/skara/pull/1344 From gli at openjdk.org Sat Aug 6 10:39:44 2022 From: gli at openjdk.org (Guoxiong Li) Date: Sat, 6 Aug 2022 10:39:44 GMT Subject: RFR: 1463: Include starting branch in back port instructions [v3] In-Reply-To: References: Message-ID: > Hi all, > > This patch adds the suggested git command about fetching the newest code > from the target branch before solving the conflicts. > > Thanks for taking the time to review. > > Best Regards, > -- Guoxiong Guoxiong Li has updated the pull request incrementally with one additional commit since the last revision: Tweak the message again. ------------- Changes: - all: https://git.openjdk.org/skara/pull/1344/files - new: https://git.openjdk.org/skara/pull/1344/files/15f19d29..44335603 Webrevs: - full: https://webrevs.openjdk.org/?repo=skara&pr=1344&range=02 - incr: https://webrevs.openjdk.org/?repo=skara&pr=1344&range=01-02 Stats: 1 line in 1 file changed: 0 ins; 0 del; 1 mod Patch: https://git.openjdk.org/skara/pull/1344.diff Fetch: git fetch https://git.openjdk.org/skara pull/1344/head:pull/1344 PR: https://git.openjdk.org/skara/pull/1344 From dholmes at openjdk.org Sun Aug 7 22:49:06 2022 From: dholmes at openjdk.org (David Holmes) Date: Sun, 7 Aug 2022 22:49:06 GMT Subject: RFR: 1463: Include starting branch in back port instructions [v3] In-Reply-To: References: Message-ID: <9lVqc8M_7h4J-h-MH5GX2ieMsOdhuhBGP5IXxZoqDJ4=.37202726-825d-473b-bd4d-048cda8ebe8e@github.com> On Sat, 6 Aug 2022 10:39:44 GMT, Guoxiong Li wrote: >> Hi all, >> >> This patch adds the suggested git command about fetching the newest code >> from the target branch before solving the conflicts. >> >> Thanks for taking the time to review. >> >> Best Regards, >> -- Guoxiong > > Guoxiong Li has updated the pull request incrementally with one additional commit since the last revision: > > Tweak the message again. Sorry but I find this new step creates confusion. Would it normally be subsumed by using `git sync`? ------------- PR: https://git.openjdk.org/skara/pull/1344 From ihse at openjdk.org Mon Aug 8 08:52:03 2022 From: ihse at openjdk.org (Magnus Ihse Bursie) Date: Mon, 8 Aug 2022 08:52:03 GMT Subject: RFR: 1463: Include starting branch in back port instructions [v3] In-Reply-To: <9lVqc8M_7h4J-h-MH5GX2ieMsOdhuhBGP5IXxZoqDJ4=.37202726-825d-473b-bd4d-048cda8ebe8e@github.com> References: <9lVqc8M_7h4J-h-MH5GX2ieMsOdhuhBGP5IXxZoqDJ4=.37202726-825d-473b-bd4d-048cda8ebe8e@github.com> Message-ID: <3b1Y0GLHR3Uc6eDS0sVu5MadxlKsSGnVVJQe9uRrVR4=.4e4be2ef-2d31-4170-9673-956e7b986859@github.com> On Sun, 7 Aug 2022 22:45:58 GMT, David Holmes wrote: >> Guoxiong Li has updated the pull request incrementally with one additional commit since the last revision: >> >> Tweak the message again. > > Sorry but I find this new step creates confusion. Would it normally be subsumed by using `git sync`? @dholmes-ora `git sync` is a Skara addition. We can certainly have instructions that suggest using them, but we should also have instructions for how to do without the additions. ------------- PR: https://git.openjdk.org/skara/pull/1344 From ihse at openjdk.org Mon Aug 8 09:00:15 2022 From: ihse at openjdk.org (Magnus Ihse Bursie) Date: Mon, 8 Aug 2022 09:00:15 GMT Subject: RFR: 1463: Include starting branch in back port instructions [v3] In-Reply-To: <9lVqc8M_7h4J-h-MH5GX2ieMsOdhuhBGP5IXxZoqDJ4=.37202726-825d-473b-bd4d-048cda8ebe8e@github.com> References: <9lVqc8M_7h4J-h-MH5GX2ieMsOdhuhBGP5IXxZoqDJ4=.37202726-825d-473b-bd4d-048cda8ebe8e@github.com> Message-ID: On Sun, 7 Aug 2022 22:45:58 GMT, David Holmes wrote: >> Guoxiong Li has updated the pull request incrementally with one additional commit since the last revision: >> >> Tweak the message again. > > Sorry but I find this new step creates confusion. Would it normally be subsumed by using `git sync`? @dholmes-ora The JBS report by Mikael says that an important step is missing -- we need to make sure we start with an up-to-date view of the target branch. This aspect was missing from the instructions before. It seem to me that @lgxbslgx's patch will add this. Of course adding more instructions adds to the complexity, but that is the price to pay to get a fail-safe process. At the risk of being more verbose, the instructions could possibly be made clearer, by listing the additions as a separate step, like: "First make sure you have an up to date revision of the target branch: $ git checkout.. $ git fetch --no-tags ... Then get this backport PR: ..." ------------- PR: https://git.openjdk.org/skara/pull/1344 From ihse at openjdk.org Mon Aug 8 09:07:14 2022 From: ihse at openjdk.org (Magnus Ihse Bursie) Date: Mon, 8 Aug 2022 09:07:14 GMT Subject: RFR: 1530: Race with CheckWorkItem in PR bot In-Reply-To: References: Message-ID: On Fri, 5 Aug 2022 21:49:18 GMT, Erik Joelsson wrote: > This patch fixes a race in the PR bot. > > Currently, all WorkItems that process PRs get an instance of the PullRequest that was fetched before the WorkItem instance was created. The data in that PullRequest instance (the parts that aren't dynamically fetched on method calls) may then be stale before the WorkItem even starts running. > > With this patch, I'm changing PullRequestWorkItem (the super class of all WorkItems that handle PRs), to only be instantiated with the ID of the PR, and then fetches the actual PullRequest object at the start of the `run()` method. When the `run()` method is called, we are guaranteed to be the only WorkItem currently processing the particular PR, so fetching data there should guarantee a coherent and current view. > > In many cases where PullRequestWorkItems were created, the creator first refreshed the PullRequest object to provide an updated view, reflecting any changes made by the creator. These refreshes are no longer necessary, so I've removed them all. Looks good. Tricky one! ------------- Marked as reviewed by ihse (Reviewer). PR: https://git.openjdk.org/skara/pull/1349 From dholmes at openjdk.org Mon Aug 8 09:18:32 2022 From: dholmes at openjdk.org (David Holmes) Date: Mon, 8 Aug 2022 09:18:32 GMT Subject: RFR: 1463: Include starting branch in back port instructions [v3] In-Reply-To: References: <9lVqc8M_7h4J-h-MH5GX2ieMsOdhuhBGP5IXxZoqDJ4=.37202726-825d-473b-bd4d-048cda8ebe8e@github.com> Message-ID: On Mon, 8 Aug 2022 08:57:35 GMT, Magnus Ihse Bursie wrote: >> Sorry but I find this new step creates confusion. Would it normally be subsumed by using `git sync`? > > @dholmes-ora The JBS report by Mikael says that an important step is missing -- we need to make sure we start with an up-to-date view of the target branch. This aspect was missing from the instructions before. It seem to me that @lgxbslgx's patch will add this. Of course adding more instructions adds to the complexity, but that is the price to pay to get a fail-safe process. > > At the risk of being more verbose, the instructions could possibly be made clearer, by listing the additions as a separate step, like: > "First make sure you have an up to date revision of the target branch: > $ git checkout.. > $ git fetch --no-tags ... > > Then get this backport PR: > ..." @magicus as I said I recently followed the existing instructions for a couple of backports and with no problem and nothing missing. So while I accept if you start from some specific condition there may be a step missing, if you are already in the right position then this new extra step is potentially confusing. Hence it would be clearer to me if this step was added with some context for when the step is needed. I asked about skara sync because I will always run it before starting anything of this nature. So I wanted to know if that is what already takes care of this additional step. Not everyone uses skara but these instructions have to make sense to both sets of users. ------------- PR: https://git.openjdk.org/skara/pull/1344 From ihse at openjdk.org Mon Aug 8 10:38:31 2022 From: ihse at openjdk.org (Magnus Ihse Bursie) Date: Mon, 8 Aug 2022 10:38:31 GMT Subject: RFR: 1463: Include starting branch in back port instructions [v3] In-Reply-To: References: Message-ID: On Sat, 6 Aug 2022 10:39:44 GMT, Guoxiong Li wrote: >> Hi all, >> >> This patch adds the suggested git command about fetching the newest code >> from the target branch before solving the conflicts. >> >> Thanks for taking the time to review. >> >> Best Regards, >> -- Guoxiong > > Guoxiong Li has updated the pull request incrementally with one additional commit since the last revision: > > Tweak the message again. `skara sync` will sync your personal github fork with it's upstream. It will not fetch those changes to your local repo, which is needed to resolve the conflict locally. Also, if you read the JBS issue, the problem is if you start out with a repo that is out of date. It is likely that things have worked out well for you (and as for most people, most of the time) due to sheer luck, or rather, the fact that your repo is likely already up to date. The request by Mikael was to add instructions that make sure that by following the instructions, you will succeed, always, and not just if you are lucky. ------------- PR: https://git.openjdk.org/skara/pull/1344 From dholmes at openjdk.org Mon Aug 8 12:01:40 2022 From: dholmes at openjdk.org (David Holmes) Date: Mon, 8 Aug 2022 12:01:40 GMT Subject: RFR: 1463: Include starting branch in back port instructions [v3] In-Reply-To: References: Message-ID: On Mon, 8 Aug 2022 10:34:20 GMT, Magnus Ihse Bursie wrote: > skara sync will sync your personal github fork with it's upstream. It will not fetch those changes to your local repo, which is needed to resolve the conflict locally. skara sync can update your local repo if you use the fast-forward option, which I always do. ------------- PR: https://git.openjdk.org/skara/pull/1344 From dholmes at openjdk.org Mon Aug 8 12:07:37 2022 From: dholmes at openjdk.org (David Holmes) Date: Mon, 8 Aug 2022 12:07:37 GMT Subject: RFR: 1463: Include starting branch in back port instructions [v3] In-Reply-To: References: Message-ID: On Mon, 8 Aug 2022 10:36:07 GMT, Magnus Ihse Bursie wrote: > The request by Mikael was to add instructions that make sure that by following the instructions, you will succeed, always, and not just if you are lucky. I understand that. But those instructions could simply be in the form of: > Make sure your repo is update to date Telling somehow how to make their repo up-to-date when there are different styles of repo management (e.g. skara and non-skara) is presuming what style is being used. You can add the fetch as an example, but as I said (and please remember I am a member of your target audience here) simply stating you must do this can be confusing when there is no need to do it at all. ------------- PR: https://git.openjdk.org/skara/pull/1344 From ihse at openjdk.org Mon Aug 8 12:27:40 2022 From: ihse at openjdk.org (Magnus Ihse Bursie) Date: Mon, 8 Aug 2022 12:27:40 GMT Subject: RFR: 1463: Include starting branch in back port instructions [v3] In-Reply-To: References: Message-ID: On Sat, 6 Aug 2022 10:39:44 GMT, Guoxiong Li wrote: >> Hi all, >> >> This patch adds the suggested git command about fetching the newest code >> from the target branch before solving the conflicts. >> >> Thanks for taking the time to review. >> >> Best Regards, >> -- Guoxiong > > Guoxiong Li has updated the pull request incrementally with one additional commit since the last revision: > > Tweak the message again. Fair enough. The instructions need to tell you to make sure you are up to date, but we can leave out the details of that. ------------- PR: https://git.openjdk.org/skara/pull/1344 From erikj at openjdk.org Mon Aug 8 13:45:33 2022 From: erikj at openjdk.org (Erik Joelsson) Date: Mon, 8 Aug 2022 13:45:33 GMT Subject: Integrated: 1530: Race with CheckWorkItem in PR bot In-Reply-To: References: Message-ID: On Fri, 5 Aug 2022 21:49:18 GMT, Erik Joelsson wrote: > This patch fixes a race in the PR bot. > > Currently, all WorkItems that process PRs get an instance of the PullRequest that was fetched before the WorkItem instance was created. The data in that PullRequest instance (the parts that aren't dynamically fetched on method calls) may then be stale before the WorkItem even starts running. > > With this patch, I'm changing PullRequestWorkItem (the super class of all WorkItems that handle PRs), to only be instantiated with the ID of the PR, and then fetches the actual PullRequest object at the start of the `run()` method. When the `run()` method is called, we are guaranteed to be the only WorkItem currently processing the particular PR, so fetching data there should guarantee a coherent and current view. > > In many cases where PullRequestWorkItems were created, the creator first refreshed the PullRequest object to provide an updated view, reflecting any changes made by the creator. These refreshes are no longer necessary, so I've removed them all. This pull request has now been integrated. Changeset: d7b59c6f Author: Erik Joelsson URL: https://git.openjdk.org/skara/commit/d7b59c6fca8b30fe35575afeb3e4ee13f310e797 Stats: 54 lines in 5 files changed: 19 ins; 8 del; 27 mod 1530: Race with CheckWorkItem in PR bot Reviewed-by: kcr, ihse ------------- PR: https://git.openjdk.org/skara/pull/1349 From erikj at openjdk.org Mon Aug 8 17:18:18 2022 From: erikj at openjdk.org (Erik Joelsson) Date: Mon, 8 Aug 2022 17:18:18 GMT Subject: RFR: 1463: Include starting branch in back port instructions [v3] In-Reply-To: References: Message-ID: On Sat, 6 Aug 2022 10:39:44 GMT, Guoxiong Li wrote: >> Hi all, >> >> This patch adds the suggested git command about fetching the newest code >> from the target branch before solving the conflicts. >> >> Thanks for taking the time to review. >> >> Best Regards, >> -- Guoxiong > > Guoxiong Li has updated the pull request incrementally with one additional commit since the last revision: > > Tweak the message again. There are two issues being addressed in this change. 1. The implicit assumption that you need to be checked out on the target branch in your personal fork before starting to run the suggested commands. 2. Your local repo needs to be reasonably up to date on that target branch. I definitely think that 1 needs to be included first in the current list of commands, as otherwise you risk doing some pretty weird things. As for 2, this can be achieved in a number of different ways, of which various incarnations of `git sync` are probably common, but the basic way of achieving it is through `git fetch`. My suggestion would be to simply state the recommendation that you should be up to date with the target branch of your upstream and give a separate suggested command to achieve that. The fetch command in the current version of the patch works for me. ------------- PR: https://git.openjdk.org/skara/pull/1344 From gli at openjdk.org Mon Aug 8 18:12:35 2022 From: gli at openjdk.org (Guoxiong Li) Date: Mon, 8 Aug 2022 18:12:35 GMT Subject: RFR: 1463: Include starting branch in back port instructions [v4] In-Reply-To: References: Message-ID: <-b1RvFCiQJ-vadwsSbB4Zh5zoTmi5oQUvq9SQShjclg=.eb3f91c3-8ec6-4c3f-8575-4acb3aafae96@github.com> > Hi all, > > This patch adds the suggested git command about fetching the newest code > from the target branch before solving the conflicts. > > Thanks for taking the time to review. > > Best Regards, > -- Guoxiong Guoxiong Li has updated the pull request incrementally with two additional commits since the last revision: - Add more comments to state the commands - Fix test cases. ------------- Changes: - all: https://git.openjdk.org/skara/pull/1344/files - new: https://git.openjdk.org/skara/pull/1344/files/44335603..5932229a Webrevs: - full: https://webrevs.openjdk.org/?repo=skara&pr=1344&range=03 - incr: https://webrevs.openjdk.org/?repo=skara&pr=1344&range=02-03 Stats: 14 lines in 3 files changed: 10 ins; 0 del; 4 mod Patch: https://git.openjdk.org/skara/pull/1344.diff Fetch: git fetch https://git.openjdk.org/skara pull/1344/head:pull/1344 PR: https://git.openjdk.org/skara/pull/1344 From gli at openjdk.org Mon Aug 8 18:16:19 2022 From: gli at openjdk.org (Guoxiong Li) Date: Mon, 8 Aug 2022 18:16:19 GMT Subject: RFR: 1463: Include starting branch in back port instructions [v3] In-Reply-To: References: Message-ID: On Sat, 6 Aug 2022 10:39:44 GMT, Guoxiong Li wrote: >> Hi all, >> >> This patch adds the suggested git command about fetching the newest code >> from the target branch before solving the conflicts. >> >> Thanks for taking the time to review. >> >> Best Regards, >> -- Guoxiong > > Guoxiong Li has updated the pull request incrementally with one additional commit since the last revision: > > Tweak the message again. Thanks for your comments. I add some comments to state the commands and to clarify the usage. Please see the newest code. One issue is that the content may be too long. ------------- PR: https://git.openjdk.org/skara/pull/1344 From lgxbslgx at gmail.com Mon Aug 8 20:25:25 2022 From: lgxbslgx at gmail.com (Guoxiong Li) Date: Tue, 9 Aug 2022 04:25:25 +0800 Subject: [Proposal] Maintainer Approvals and SKARA In-Reply-To: References: <639d9498-c764-2005-30ef-87351d639d9d@oracle.com> Message-ID: Hi all, I re-read your comments about this enhancement recently. And I also made a mistake when I backport a patch to jdk11u-dev [1] although I have backported several times. So SKARA must direct the developers accurately and help developers complete the duplicated work to avoid the unnecessary mistakes. Now I propose the following dev flow. Please only focus on the dev flow instead of the code implementation here. *- When a backport PR is created in Github.* 1. A new checkbox item `[ ] Change must be properly approved by the maintainers` will be added to the `Progress` part of the current PR body. 2. A comment is posted by the bot like below (the italic content) to direct the developers: *This is a backport pull request. Please add a comment to the main issue [JDK-XXXX](link)* *to state the related condition (the backport reason, the risk of the patch, the amount of* *work and so on). Below is an example for you:* *```* *Fix Request(jdk17u-dev)* *The code applies cleanly and the test in this change fails without the patch and succeeds after applying it.* *The risk of this backport is low because the change is little and the issue fixed by this change also exists in jdk xy.* *```* *If you don't have permission to add a comment in JBS. Please use the command `request-approval` to provide* *the related content, then the bot can help you add a comment by using the content you provided. Below is an example for you:* *```* */request-approval* *Fix Request(jdk17u-dev)* *The code applies cleanly and the test in this change fails without the patch and succeeds after applying it.* *The risk of this backport is low because the change is little and the issue fixed by this change also exists in jdk xy.* *```* *Please note you have to contact the maintainers **directly **in the issue [JDK-XXXX](link)* *or by using the command **`request-approval` **repeatedly** instead of in this pull request.* *And you don't need to add the fix request label manually to the issue like before,* *now the bot can help you add the label automatically when this pull request is ready **for maintainers to approve.* *- When the backport PR is ready for approval *(other checks have succeeded and other progresses have been done) 1. The bot adds a label named `jdkXXu-fix-request` to *all the issues* of the PR 2. The bot adds a blocked label named `approval` in PR, like `csr` or other blocked labels These two labels are convenient for maintainers to filter the issues and PRs which need to be handled now. *- It is time for maintainers to approve* The maintainers can add the label `jdkXXu-fix-yes` or `jdkXXu-fix-no` in the issue or use the command `/approval yes|no|y|n` in the PR to approve or object the patch. This newly added command `approval` can be very convenient if one backport PR has many corresponding issues. After using this command, the bot will add the related label to *all the issues* just like the label `jdkXXu-fix-request` it had added. *- Then the commit or sponsor flow will continue as usual.* After approval, if the maintainer said 'yes', the PR will become ready to be integrated. If the maintainer said 'no', the PR can be closed by the bot with a comment like "The maintainers disapproved of this patch so this pull request will be closed automatically". No matter 'yes' or 'no', the label `approval` in the PR will be removed. Above is all the dev flow. And next, I will give more information about the new commands `*request-approval` and `approval`.* *- "request-approval" command* It is similar to the `summary` command which permits multiline content. And this command can be used multi times if the author of the PR wants to. Each time this command is used, the bot will post the content to a new comment in the issue. Because the author who has no permission may want to contact/talk with the maintainers. The related record will be recorded in the issue instead of PR which is your intention in your previous comments. *- "approval" command* The command `/approval yes` or `/approval y` mean `approved`. And the command `/approval no` or `/approval n` mean `dispproved`. This command can be used multiple times and only the last time is valid. Please note when the `/approval no` or `/approval n` is used, the bot will close the PR, and when the `/approval yes` or `/approval y` later is used, the bot will open the PR automatically. That is all. Thanks for reading and providing feedback. And I will try to implement these features if I have time and hear no objection after discussion. [1] https://github.com/openjdk/jdk11u-dev/pull/1218#issuecomment-1184235601 Best Regards, -- Guoxiong -------------- next part -------------- An HTML attachment was scrubbed... URL: From erikj at openjdk.org Mon Aug 8 21:21:04 2022 From: erikj at openjdk.org (Erik Joelsson) Date: Mon, 8 Aug 2022 21:21:04 GMT Subject: RFR: Fix NPE in toString Message-ID: My patch in SKARA-1530 did unfortunately trigger NPE in the toString() methods of all subclasses of PullRequestWorkItem. This is because the pr field isn't initialized in the constructor anymore. The repository is available through the bot field however. ------------- Commit messages: - Fix NPE in toString Changes: https://git.openjdk.org/skara/pull/1351/files Webrev: https://webrevs.openjdk.org/?repo=skara&pr=1351&range=00 Stats: 3 lines in 3 files changed: 0 ins; 0 del; 3 mod Patch: https://git.openjdk.org/skara/pull/1351.diff Fetch: git fetch https://git.openjdk.org/skara pull/1351/head:pull/1351 PR: https://git.openjdk.org/skara/pull/1351 From kcr at openjdk.org Mon Aug 8 21:21:04 2022 From: kcr at openjdk.org (Kevin Rushforth) Date: Mon, 8 Aug 2022 21:21:04 GMT Subject: RFR: Fix NPE in toString In-Reply-To: References: Message-ID: <_Int_LXayxsFYFX-wAbjLXpankDTns-vy6ySGN5JuAc=.43c5dafb-1d9d-4434-a108-4287873623e7@github.com> On Mon, 8 Aug 2022 21:04:54 GMT, Erik Joelsson wrote: > My patch in SKARA-1530 did unfortunately trigger NPE in the toString() methods of all subclasses of PullRequestWorkItem. This is because the pr field isn't initialized in the constructor anymore. The repository is available through the bot field however. Marked as reviewed by kcr (Reviewer). ------------- PR: https://git.openjdk.org/skara/pull/1351 From erikj at openjdk.org Mon Aug 8 21:21:04 2022 From: erikj at openjdk.org (Erik Joelsson) Date: Mon, 8 Aug 2022 21:21:04 GMT Subject: RFR: Fix NPE in toString In-Reply-To: References: Message-ID: On Mon, 8 Aug 2022 21:04:54 GMT, Erik Joelsson wrote: > My patch in SKARA-1530 did unfortunately trigger NPE in the toString() methods of all subclasses of PullRequestWorkItem. This is because the pr field isn't initialized in the constructor anymore. The repository is available through the bot field however. Thanks! ------------- PR: https://git.openjdk.org/skara/pull/1351 From erikj at openjdk.org Mon Aug 8 21:22:12 2022 From: erikj at openjdk.org (Erik Joelsson) Date: Mon, 8 Aug 2022 21:22:12 GMT Subject: Integrated: Fix NPE in toString In-Reply-To: References: Message-ID: On Mon, 8 Aug 2022 21:04:54 GMT, Erik Joelsson wrote: > My patch in SKARA-1530 did unfortunately trigger NPE in the toString() methods of all subclasses of PullRequestWorkItem. This is because the pr field isn't initialized in the constructor anymore. The repository is available through the bot field however. This pull request has now been integrated. Changeset: 990ac8fb Author: Erik Joelsson URL: https://git.openjdk.org/skara/commit/990ac8fb40d1c2943b680631d7dac7aa30844236 Stats: 3 lines in 3 files changed: 0 ins; 0 del; 3 mod Fix NPE in toString Reviewed-by: kcr ------------- PR: https://git.openjdk.org/skara/pull/1351 From dholmes at openjdk.org Tue Aug 9 03:11:12 2022 From: dholmes at openjdk.org (David Holmes) Date: Tue, 9 Aug 2022 03:11:12 GMT Subject: RFR: 1463: Include starting branch in back port instructions [v4] In-Reply-To: <-b1RvFCiQJ-vadwsSbB4Zh5zoTmi5oQUvq9SQShjclg=.eb3f91c3-8ec6-4c3f-8575-4acb3aafae96@github.com> References: <-b1RvFCiQJ-vadwsSbB4Zh5zoTmi5oQUvq9SQShjclg=.eb3f91c3-8ec6-4c3f-8575-4acb3aafae96@github.com> Message-ID: On Mon, 8 Aug 2022 18:12:35 GMT, Guoxiong Li wrote: >> Hi all, >> >> This patch adds the suggested git command about fetching the newest code >> from the target branch before solving the conflicts. >> >> Thanks for taking the time to review. >> >> Best Regards, >> -- Guoxiong > > Guoxiong Li has updated the pull request incrementally with two additional commits since the last revision: > > - Add more comments to state the commands > - Fix test cases. The latest version with additional comments seems okay. FWIW I normally use `git pull`, `git sync` and `git switch` rather than `git fetch` and `git checkout`. So I have to interpret these instructions anyway, and the new comments help with that. Thanks. ------------- PR: https://git.openjdk.org/skara/pull/1344 From ihse at openjdk.org Tue Aug 9 10:50:21 2022 From: ihse at openjdk.org (Magnus Ihse Bursie) Date: Tue, 9 Aug 2022 10:50:21 GMT Subject: RFR: 1463: Include starting branch in back port instructions [v4] In-Reply-To: <-b1RvFCiQJ-vadwsSbB4Zh5zoTmi5oQUvq9SQShjclg=.eb3f91c3-8ec6-4c3f-8575-4acb3aafae96@github.com> References: <-b1RvFCiQJ-vadwsSbB4Zh5zoTmi5oQUvq9SQShjclg=.eb3f91c3-8ec6-4c3f-8575-4acb3aafae96@github.com> Message-ID: On Mon, 8 Aug 2022 18:12:35 GMT, Guoxiong Li wrote: >> Hi all, >> >> This patch adds the suggested git command about fetching the newest code >> from the target branch before solving the conflicts. >> >> Thanks for taking the time to review. >> >> Best Regards, >> -- Guoxiong > > Guoxiong Li has updated the pull request incrementally with two additional commits since the last revision: > > - Add more comments to state the commands > - Fix test cases. Marked as reviewed by ihse (Reviewer). ------------- PR: https://git.openjdk.org/skara/pull/1344 From erikj at openjdk.org Tue Aug 9 12:51:44 2022 From: erikj at openjdk.org (Erik Joelsson) Date: Tue, 9 Aug 2022 12:51:44 GMT Subject: RFR: 1463: Include starting branch in back port instructions [v4] In-Reply-To: <-b1RvFCiQJ-vadwsSbB4Zh5zoTmi5oQUvq9SQShjclg=.eb3f91c3-8ec6-4c3f-8575-4acb3aafae96@github.com> References: <-b1RvFCiQJ-vadwsSbB4Zh5zoTmi5oQUvq9SQShjclg=.eb3f91c3-8ec6-4c3f-8575-4acb3aafae96@github.com> Message-ID: On Mon, 8 Aug 2022 18:12:35 GMT, Guoxiong Li wrote: >> Hi all, >> >> This patch adds the suggested git command about fetching the newest code >> from the target branch before solving the conflicts. >> >> Thanks for taking the time to review. >> >> Best Regards, >> -- Guoxiong > > Guoxiong Li has updated the pull request incrementally with two additional commits since the last revision: > > - Add more comments to state the commands > - Fix test cases. Marked as reviewed by erikj (Lead). ------------- PR: https://git.openjdk.org/skara/pull/1344 From gli at openjdk.org Tue Aug 9 12:55:53 2022 From: gli at openjdk.org (Guoxiong Li) Date: Tue, 9 Aug 2022 12:55:53 GMT Subject: RFR: 1463: Include starting branch in back port instructions [v4] In-Reply-To: <-b1RvFCiQJ-vadwsSbB4Zh5zoTmi5oQUvq9SQShjclg=.eb3f91c3-8ec6-4c3f-8575-4acb3aafae96@github.com> References: <-b1RvFCiQJ-vadwsSbB4Zh5zoTmi5oQUvq9SQShjclg=.eb3f91c3-8ec6-4c3f-8575-4acb3aafae96@github.com> Message-ID: On Mon, 8 Aug 2022 18:12:35 GMT, Guoxiong Li wrote: >> Hi all, >> >> This patch adds the suggested git command about fetching the newest code >> from the target branch before solving the conflicts. >> >> Thanks for taking the time to review. >> >> Best Regards, >> -- Guoxiong > > Guoxiong Li has updated the pull request incrementally with two additional commits since the last revision: > > - Add more comments to state the commands > - Fix test cases. Thanks for the reviews. Integrating. ------------- PR: https://git.openjdk.org/skara/pull/1344 From gli at openjdk.org Tue Aug 9 12:55:53 2022 From: gli at openjdk.org (Guoxiong Li) Date: Tue, 9 Aug 2022 12:55:53 GMT Subject: Integrated: 1463: Include starting branch in back port instructions In-Reply-To: References: Message-ID: On Sat, 30 Jul 2022 12:27:40 GMT, Guoxiong Li wrote: > Hi all, > > This patch adds the suggested git command about fetching the newest code > from the target branch before solving the conflicts. > > Thanks for taking the time to review. > > Best Regards, > -- Guoxiong This pull request has now been integrated. Changeset: 35b23b88 Author: Guoxiong Li URL: https://git.openjdk.org/skara/commit/35b23b882393c33565e49ec0399adba7e8eb2f5b Stats: 20 lines in 3 files changed: 16 ins; 0 del; 4 mod 1463: Include starting branch in back port instructions Reviewed-by: ihse, erikj ------------- PR: https://git.openjdk.org/skara/pull/1344 From gli at openjdk.org Tue Aug 9 13:10:06 2022 From: gli at openjdk.org (Guoxiong Li) Date: Tue, 9 Aug 2022 13:10:06 GMT Subject: RFR: 1516: Command 'git pr integrate' gets error message 'time out' Message-ID: Hi all, I tried the command `git pr integrate 1341` to integrate SKARA-1495. The command replies `error: timed out waiting for response to /integrate command`. It is because the bot added a comment `@lgxbslgx Pushed as commit XXX.` so that this line is not started with `Pushed as commit`. This patch fixes the bug by using a pattern matcher instead of judging the beginning of the line. And I verified this patch by using the command `git pr integrate 1344` to integrate SKARA-1463 just now. It works well and replies `@lgxbslgx Pushed as commit 35b23b882393c33565e49ec0399adba7e8eb2f5b`. But I have not verified the sponsor flow because I don't have a change to do that now. Thanks for taking the time to review. Best Regards, -- Guoxion ------------- Commit messages: - SKARA-1516 Changes: https://git.openjdk.org/skara/pull/1352/files Webrev: https://webrevs.openjdk.org/?repo=skara&pr=1352&range=00 Issue: https://bugs.openjdk.org/browse/SKARA-1516 Stats: 11 lines in 2 files changed: 5 ins; 1 del; 5 mod Patch: https://git.openjdk.org/skara/pull/1352.diff Fetch: git fetch https://git.openjdk.org/skara pull/1352/head:pull/1352 PR: https://git.openjdk.org/skara/pull/1352 From ihse at openjdk.org Tue Aug 9 15:34:40 2022 From: ihse at openjdk.org (Magnus Ihse Bursie) Date: Tue, 9 Aug 2022 15:34:40 GMT Subject: RFR: 1516: Command 'git pr integrate' gets error message 'time out' In-Reply-To: References: Message-ID: On Tue, 9 Aug 2022 13:05:17 GMT, Guoxiong Li wrote: > Hi all, > > I tried the command `git pr integrate 1341` to integrate SKARA-1495. The command replies `error: timed out waiting for response to /integrate command`. It is because the bot added a comment `@lgxbslgx Pushed as commit XXX.` so that this line is not started with `Pushed as commit`. > > This patch fixes the bug by using a pattern matcher instead of judging the beginning of the line. And I verified this patch by using the command `git pr integrate 1344` to integrate SKARA-1463 just now. It works well and replies `@lgxbslgx Pushed as commit 35b23b882393c33565e49ec0399adba7e8eb2f5b`. But I have not verified the sponsor flow because I don't have a change to do that now. > > Thanks for taking the time to review. > > Best Regards, > -- Guoxion Is there no tests for this? Can you please add one, if not, or update it, if it does not test this properly. ------------- PR: https://git.openjdk.org/skara/pull/1352 From gli at openjdk.org Tue Aug 9 16:27:05 2022 From: gli at openjdk.org (Guoxiong Li) Date: Tue, 9 Aug 2022 16:27:05 GMT Subject: RFR: 1516: Command 'git pr integrate' gets error message 'time out' In-Reply-To: References: Message-ID: On Tue, 9 Aug 2022 13:05:17 GMT, Guoxiong Li wrote: > Hi all, > > I tried the command `git pr integrate 1341` to integrate SKARA-1495. The command replies `error: timed out waiting for response to /integrate command`. It is because the bot added a comment `@lgxbslgx Pushed as commit XXX.` so that this line is not started with `Pushed as commit`. > > This patch fixes the bug by using a pattern matcher instead of judging the beginning of the line. And I verified this patch by using the command `git pr integrate 1344` to integrate SKARA-1463 just now. It works well and replies `@lgxbslgx Pushed as commit 35b23b882393c33565e49ec0399adba7e8eb2f5b`. But I have not verified the sponsor flow because I don't have a chance to do that now. > > Thanks for taking the time to review. > > Best Regards, > -- Guoxion All the code in cli module don?t have tests now. It seems be hard to add tests to verify the client commands so the previous developers didn?t add any test for it. I don?t really know whether it has a good way to add such test, but I am still thinking. ------------- PR: https://git.openjdk.org/skara/pull/1352 From erikj at openjdk.org Tue Aug 9 16:36:07 2022 From: erikj at openjdk.org (Erik Joelsson) Date: Tue, 9 Aug 2022 16:36:07 GMT Subject: RFR: 1516: Command 'git pr integrate' gets error message 'time out' In-Reply-To: References: Message-ID: <_kHwJbirppUGeGlZFPiU9gOD4ko30PiAwlbCu6WOV58=.09438f57-49a1-479e-9e26-64c472056d0d@github.com> On Tue, 9 Aug 2022 13:05:17 GMT, Guoxiong Li wrote: > Hi all, > > I tried the command `git pr integrate 1341` to integrate SKARA-1495. The command replies `error: timed out waiting for response to /integrate command`. It is because the bot added a comment `@lgxbslgx Pushed as commit XXX.` so that this line is not started with `Pushed as commit`. > > This patch fixes the bug by using a pattern matcher instead of judging the beginning of the line. And I verified this patch by using the command `git pr integrate 1344` to integrate SKARA-1463 just now. It works well and replies `@lgxbslgx Pushed as commit 35b23b882393c33565e49ec0399adba7e8eb2f5b`. But I have not verified the sponsor flow because I don't have a chance to do that now. > > Thanks for taking the time to review. > > Best Regards, > -- Guoxion Marked as reviewed by erikj (Lead). ------------- PR: https://git.openjdk.org/skara/pull/1352 From ihse at openjdk.org Tue Aug 9 16:40:38 2022 From: ihse at openjdk.org (Magnus Ihse Bursie) Date: Tue, 9 Aug 2022 16:40:38 GMT Subject: RFR: 1516: Command 'git pr integrate' gets error message 'time out' In-Reply-To: References: Message-ID: <54JgKaXtjrEQCtOLUYAUYSa-iH7bUjYwmddl7qfMODI=.9a14ee19-b0bc-4a35-a2e8-e76c57dd5357@github.com> On Tue, 9 Aug 2022 13:05:17 GMT, Guoxiong Li wrote: > Hi all, > > I tried the command `git pr integrate 1341` to integrate SKARA-1495. The command replies `error: timed out waiting for response to /integrate command`. It is because the bot added a comment `@lgxbslgx Pushed as commit XXX.` so that this line is not started with `Pushed as commit`. > > This patch fixes the bug by using a pattern matcher instead of judging the beginning of the line. And I verified this patch by using the command `git pr integrate 1344` to integrate SKARA-1463 just now. It works well and replies `@lgxbslgx Pushed as commit 35b23b882393c33565e49ec0399adba7e8eb2f5b`. But I have not verified the sponsor flow because I don't have a chance to do that now. > > Thanks for taking the time to review. > > Best Regards, > -- Guoxion Marked as reviewed by ihse (Reviewer). Ah, right, it's the CLI. *duh* I do think we should have tests for them, but setting up testing scaffolding for CLI testing is way beyond this PR. But if you feel like doing it, please go ahead! Especially a popular command like "git skara sync" would benefit from having automated testing. ------------- PR: https://git.openjdk.org/skara/pull/1352 From gli at openjdk.org Tue Aug 9 17:40:32 2022 From: gli at openjdk.org (Guoxiong Li) Date: Tue, 9 Aug 2022 17:40:32 GMT Subject: RFR: 1516: Command 'git pr integrate' gets error message 'time out' In-Reply-To: References: Message-ID: On Tue, 9 Aug 2022 13:05:17 GMT, Guoxiong Li wrote: > Hi all, > > I tried the command `git pr integrate 1341` to integrate SKARA-1495. The command replies `error: timed out waiting for response to /integrate command`. It is because the bot added a comment `@lgxbslgx Pushed as commit XXX.` so that this line is not started with `Pushed as commit`. > > This patch fixes the bug by using a pattern matcher instead of judging the beginning of the line. And I verified this patch by using the command `git pr integrate 1344` to integrate SKARA-1463 just now. It works well and replies `@lgxbslgx Pushed as commit 35b23b882393c33565e49ec0399adba7e8eb2f5b`. But I have not verified the sponsor flow because I don't have a chance to do that now. > > Thanks for taking the time to review. > > Best Regards, > -- Guoxion Thanks for the reviews. Integrating. ------------- PR: https://git.openjdk.org/skara/pull/1352 From gli at openjdk.org Tue Aug 9 17:40:32 2022 From: gli at openjdk.org (Guoxiong Li) Date: Tue, 9 Aug 2022 17:40:32 GMT Subject: Integrated: 1516: Command 'git pr integrate' gets error message 'time out' In-Reply-To: References: Message-ID: On Tue, 9 Aug 2022 13:05:17 GMT, Guoxiong Li wrote: > Hi all, > > I tried the command `git pr integrate 1341` to integrate SKARA-1495. The command replies `error: timed out waiting for response to /integrate command`. It is because the bot added a comment `@lgxbslgx Pushed as commit XXX.` so that this line is not started with `Pushed as commit`. > > This patch fixes the bug by using a pattern matcher instead of judging the beginning of the line. And I verified this patch by using the command `git pr integrate 1344` to integrate SKARA-1463 just now. It works well and replies `@lgxbslgx Pushed as commit 35b23b882393c33565e49ec0399adba7e8eb2f5b`. But I have not verified the sponsor flow because I don't have a chance to do that now. > > Thanks for taking the time to review. > > Best Regards, > -- Guoxion This pull request has now been integrated. Changeset: 98ca8935 Author: Guoxiong Li URL: https://git.openjdk.org/skara/commit/98ca89358932bddab90af35531f0860639a89aa9 Stats: 11 lines in 2 files changed: 5 ins; 1 del; 5 mod 1516: Command 'git pr integrate' gets error message 'time out' Reviewed-by: erikj, ihse ------------- PR: https://git.openjdk.org/skara/pull/1352 From gli at openjdk.org Wed Aug 10 14:51:00 2022 From: gli at openjdk.org (Guoxiong Li) Date: Wed, 10 Aug 2022 14:51:00 GMT Subject: RFR: 1536: The method "JiraIssue#setState" is invalid by using the argument "State.CLOSED" Message-ID: <06dbiZl5-7Cy30t4NETNqMaHx9gLtEJvUfFWBr800TY=.a927a549-73ce-45e2-984b-08ef341c1791@github.com> Hi all, Currently, when the method "JiraIssue#setState" is invoked with the argument "State.CLOSED", it can't work as expected. This patch fixes it and add a manual test case for it. And I run the newly added test munually to close SKARA-1463 [1] and SKARA-1516 [2] so that this patch can be verified. Thanks for taking the time to review. Best Regards, -- Guoxiong [1] https://bugs.openjdk.org/browse/SKARA-1463 [2] https://bugs.openjdk.org/browse/SKARA-1516 ------------- Commit messages: - SKARA-1536 Changes: https://git.openjdk.org/skara/pull/1353/files Webrev: https://webrevs.openjdk.org/?repo=skara&pr=1353&range=00 Issue: https://bugs.openjdk.org/browse/SKARA-1536 Stats: 47 lines in 4 files changed: 43 ins; 1 del; 3 mod Patch: https://git.openjdk.org/skara/pull/1353.diff Fetch: git fetch https://git.openjdk.org/skara pull/1353/head:pull/1353 PR: https://git.openjdk.org/skara/pull/1353 From gli at openjdk.org Wed Aug 10 15:18:01 2022 From: gli at openjdk.org (Guoxiong Li) Date: Wed, 10 Aug 2022 15:18:01 GMT Subject: RFR: 1536: The method "JiraIssue#setState" is invalid when using the argument "State.CLOSED" [v2] In-Reply-To: <06dbiZl5-7Cy30t4NETNqMaHx9gLtEJvUfFWBr800TY=.a927a549-73ce-45e2-984b-08ef341c1791@github.com> References: <06dbiZl5-7Cy30t4NETNqMaHx9gLtEJvUfFWBr800TY=.a927a549-73ce-45e2-984b-08ef341c1791@github.com> Message-ID: > Hi all, > > Currently, when the method "JiraIssue#setState" is invoked with the argument "State.CLOSED", it can't work as expected. This patch fixes it and add a manual test case for it. And I run the newly added test munually to close SKARA-1463 [1] and SKARA-1516 [2] so that this patch can be verified. > > Thanks for taking the time to review. > > Best Regards, > -- Guoxiong > > [1] https://bugs.openjdk.org/browse/SKARA-1463 > [2] https://bugs.openjdk.org/browse/SKARA-1516 Guoxiong Li has updated the pull request incrementally with one additional commit since the last revision: Fix test case. ------------- Changes: - all: https://git.openjdk.org/skara/pull/1353/files - new: https://git.openjdk.org/skara/pull/1353/files/41dc9046..558561f0 Webrevs: - full: https://webrevs.openjdk.org/?repo=skara&pr=1353&range=01 - incr: https://webrevs.openjdk.org/?repo=skara&pr=1353&range=00-01 Stats: 2 lines in 1 file changed: 1 ins; 0 del; 1 mod Patch: https://git.openjdk.org/skara/pull/1353.diff Fetch: git fetch https://git.openjdk.org/skara pull/1353/head:pull/1353 PR: https://git.openjdk.org/skara/pull/1353 From ihse at openjdk.org Wed Aug 10 15:18:01 2022 From: ihse at openjdk.org (Magnus Ihse Bursie) Date: Wed, 10 Aug 2022 15:18:01 GMT Subject: RFR: 1536: The method "JiraIssue#setState" is invalid when using the argument "State.CLOSED" [v2] In-Reply-To: References: <06dbiZl5-7Cy30t4NETNqMaHx9gLtEJvUfFWBr800TY=.a927a549-73ce-45e2-984b-08ef341c1791@github.com> Message-ID: On Wed, 10 Aug 2022 15:14:24 GMT, Guoxiong Li wrote: >> Hi all, >> >> Currently, when the method "JiraIssue#setState" is invoked with the argument "State.CLOSED", it can't work as expected. This patch fixes it and add a manual test case for it. And I run the newly added test munually to close SKARA-1463 [1] and SKARA-1516 [2] so that this patch can be verified. >> >> Thanks for taking the time to review. >> >> Best Regards, >> -- Guoxiong >> >> [1] https://bugs.openjdk.org/browse/SKARA-1463 >> [2] https://bugs.openjdk.org/browse/SKARA-1516 > > Guoxiong Li has updated the pull request incrementally with one additional commit since the last revision: > > Fix test case. Looks good to me. That's the problem with too deep nesting levels, bugs like this creep in. ------------- Marked as reviewed by ihse (Reviewer). PR: https://git.openjdk.org/skara/pull/1353 From ihse at openjdk.org Wed Aug 10 15:35:15 2022 From: ihse at openjdk.org (Magnus Ihse Bursie) Date: Wed, 10 Aug 2022 15:35:15 GMT Subject: RFR: 1529: Optimize Census namespace instantiation In-Reply-To: References: Message-ID: On Fri, 5 Aug 2022 17:52:55 GMT, Erik Joelsson wrote: > This patch adds a new factory method `Census::parseNamespace`, which enables initialization of a single Namespace directly from a HostedRepository (without having to clone the census repo to local disk first). > > The motivation behind this is bot performance. The Census is loaded in many WorkItems. Cloning/fetching the complete census repo to local disk is relatively expensive compared to a couple of REST calls to just read files directly from the remote repo. Especially as our REST client already leverages caching through ETag. This particular optimization can however only be applied to the IssueNotifier in the NotifyBot, where I doubt it will make that much of a difference. The main performance gain is realized in a different bot which currently isn't public (but these gains will impact all Skara users). > > I chose to at least for now, only apply this for initializing a single Namespace as that can be done by reading only 2 files from the remote, so the performance gain is very clear. Other bots either need to read considerably more files, or have the need to update the census. This optimization idea may be expanded in the future if I see a need and noticeable gains from it. > > In order to make HostedRepository available to the Census class, I had to get rid of several existing module dependencies to avoid circular dependencies. Fortunately this wasn't too hard. It made no sense to me that `org.openjdk.skara.forge` would require `org.openjdk.skara.census`. There was just one static method `PullRequestUtils` in the forge module that used a reference to the census module, and this static method was only used from one module `org.openjdk.skara.bots.pr`, so I simply moved it to `CheckablePullRequest` which is kind of a utility class in that module. The rest of the offending dependencies were just unused, so I cleaned out a fair few of those. > > In the `IssueNotifier`, in addition to using the new initialization method, I also added lazy initialization and reusing of the `CensusInstance` class, to avoid re-initializing the same data for every round in loops. census/src/main/java/module-info.java line 25: > 23: module org.openjdk.skara.census { > 24: requires org.openjdk.skara.xml; > 25: requires static org.openjdk.skara.forge; What does `requires static` mean in a `module-info.java` file? ------------- PR: https://git.openjdk.org/skara/pull/1348 From gli at openjdk.org Wed Aug 10 15:39:56 2022 From: gli at openjdk.org (Guoxiong Li) Date: Wed, 10 Aug 2022 15:39:56 GMT Subject: RFR: 1536: The method "JiraIssue#setState" is invalid when using the argument "State.CLOSED" [v3] In-Reply-To: <06dbiZl5-7Cy30t4NETNqMaHx9gLtEJvUfFWBr800TY=.a927a549-73ce-45e2-984b-08ef341c1791@github.com> References: <06dbiZl5-7Cy30t4NETNqMaHx9gLtEJvUfFWBr800TY=.a927a549-73ce-45e2-984b-08ef341c1791@github.com> Message-ID: > Hi all, > > Currently, when the method "JiraIssue#setState" is invoked with the argument "State.CLOSED", it can't work as expected. This patch fixes it and add a manual test case for it. And I run the newly added test munually to close SKARA-1463 [1] and SKARA-1516 [2] so that this patch can be verified. > > Thanks for taking the time to review. > > Best Regards, > -- Guoxiong > > [1] https://bugs.openjdk.org/browse/SKARA-1463 > [2] https://bugs.openjdk.org/browse/SKARA-1516 Guoxiong Li has updated the pull request incrementally with one additional commit since the last revision: Adjust the import statement. ------------- Changes: - all: https://git.openjdk.org/skara/pull/1353/files - new: https://git.openjdk.org/skara/pull/1353/files/558561f0..0601a308 Webrevs: - full: https://webrevs.openjdk.org/?repo=skara&pr=1353&range=02 - incr: https://webrevs.openjdk.org/?repo=skara&pr=1353&range=01-02 Stats: 2 lines in 1 file changed: 1 ins; 1 del; 0 mod Patch: https://git.openjdk.org/skara/pull/1353.diff Fetch: git fetch https://git.openjdk.org/skara pull/1353/head:pull/1353 PR: https://git.openjdk.org/skara/pull/1353 From ihse at openjdk.org Wed Aug 10 15:39:56 2022 From: ihse at openjdk.org (Magnus Ihse Bursie) Date: Wed, 10 Aug 2022 15:39:56 GMT Subject: RFR: 1529: Optimize Census namespace instantiation In-Reply-To: References: Message-ID: On Fri, 5 Aug 2022 17:52:55 GMT, Erik Joelsson wrote: > This patch adds a new factory method `Census::parseNamespace`, which enables initialization of a single Namespace directly from a HostedRepository (without having to clone the census repo to local disk first). > > The motivation behind this is bot performance. The Census is loaded in many WorkItems. Cloning/fetching the complete census repo to local disk is relatively expensive compared to a couple of REST calls to just read files directly from the remote repo. Especially as our REST client already leverages caching through ETag. This particular optimization can however only be applied to the IssueNotifier in the NotifyBot, where I doubt it will make that much of a difference. The main performance gain is realized in a different bot which currently isn't public (but these gains will impact all Skara users). > > I chose to at least for now, only apply this for initializing a single Namespace as that can be done by reading only 2 files from the remote, so the performance gain is very clear. Other bots either need to read considerably more files, or have the need to update the census. This optimization idea may be expanded in the future if I see a need and noticeable gains from it. > > In order to make HostedRepository available to the Census class, I had to get rid of several existing module dependencies to avoid circular dependencies. Fortunately this wasn't too hard. It made no sense to me that `org.openjdk.skara.forge` would require `org.openjdk.skara.census`. There was just one static method `PullRequestUtils` in the forge module that used a reference to the census module, and this static method was only used from one module `org.openjdk.skara.bots.pr`, so I simply moved it to `CheckablePullRequest` which is kind of a utility class in that module. The rest of the offending dependencies were just unused, so I cleaned out a fair few of those. > > In the `IssueNotifier`, in addition to using the new initialization method, I also added lazy initialization and reusing of the `CensusInstance` class, to avoid re-initializing the same data for every round in loops. Looks good to me. ------------- Marked as reviewed by ihse (Reviewer). PR: https://git.openjdk.org/skara/pull/1348 From erikj at openjdk.org Wed Aug 10 16:43:56 2022 From: erikj at openjdk.org (Erik Joelsson) Date: Wed, 10 Aug 2022 16:43:56 GMT Subject: RFR: 1529: Optimize Census namespace instantiation In-Reply-To: References: Message-ID: On Wed, 10 Aug 2022 15:32:08 GMT, Magnus Ihse Bursie wrote: >> This patch adds a new factory method `Census::parseNamespace`, which enables initialization of a single Namespace directly from a HostedRepository (without having to clone the census repo to local disk first). >> >> The motivation behind this is bot performance. The Census is loaded in many WorkItems. Cloning/fetching the complete census repo to local disk is relatively expensive compared to a couple of REST calls to just read files directly from the remote repo. Especially as our REST client already leverages caching through ETag. This particular optimization can however only be applied to the IssueNotifier in the NotifyBot, where I doubt it will make that much of a difference. The main performance gain is realized in a different bot which currently isn't public (but these gains will impact all Skara users). >> >> I chose to at least for now, only apply this for initializing a single Namespace as that can be done by reading only 2 files from the remote, so the performance gain is very clear. Other bots either need to read considerably more files, or have the need to update the census. This optimization idea may be expanded in the future if I see a need and noticeable gains from it. >> >> In order to make HostedRepository available to the Census class, I had to get rid of several existing module dependencies to avoid circular dependencies. Fortunately this wasn't too hard. It made no sense to me that `org.openjdk.skara.forge` would require `org.openjdk.skara.census`. There was just one static method `PullRequestUtils` in the forge module that used a reference to the census module, and this static method was only used from one module `org.openjdk.skara.bots.pr`, so I simply moved it to `CheckablePullRequest` which is kind of a utility class in that module. The rest of the offending dependencies were just unused, so I cleaned out a fair few of those. >> >> In the `IssueNotifier`, in addition to using the new initialization method, I also added lazy initialization and reusing of the `CensusInstance` class, to avoid re-initializing the same data for every round in loops. > > census/src/main/java/module-info.java line 25: > >> 23: module org.openjdk.skara.census { >> 24: requires org.openjdk.skara.xml; >> 25: requires static org.openjdk.skara.forge; > > What does `requires static` mean in a `module-info.java` file? It's the java modules way of declaring a dependency optional. It's only needed at compile time, but not at runtime. At least in this case, the dependency is only used in a static method, which means any types from the forge module would only be used if you call that static method, and to be able to do that, you would already have had to load those classes. I don't think it matters in Skara currently, as I think all modules that require census also require forge, but it feels a bit cleaner to me that pulling in forge is optional here. ------------- PR: https://git.openjdk.org/skara/pull/1348 From erikj at openjdk.org Wed Aug 10 16:47:17 2022 From: erikj at openjdk.org (Erik Joelsson) Date: Wed, 10 Aug 2022 16:47:17 GMT Subject: RFR: 1536: The method "JiraIssue#setState" is invalid when using the argument "State.CLOSED" [v3] In-Reply-To: References: <06dbiZl5-7Cy30t4NETNqMaHx9gLtEJvUfFWBr800TY=.a927a549-73ce-45e2-984b-08ef341c1791@github.com> Message-ID: On Wed, 10 Aug 2022 15:39:56 GMT, Guoxiong Li wrote: >> Hi all, >> >> Currently, when the method "JiraIssue#setState" is invoked with the argument "State.CLOSED", it can't work as expected. This patch fixes it and add a manual test case for it. And I run the newly added test munually to close SKARA-1463 [1] and SKARA-1516 [2] so that this patch can be verified. >> >> Thanks for taking the time to review. >> >> Best Regards, >> -- Guoxiong >> >> [1] https://bugs.openjdk.org/browse/SKARA-1463 >> [2] https://bugs.openjdk.org/browse/SKARA-1516 > > Guoxiong Li has updated the pull request incrementally with one additional commit since the last revision: > > Adjust the import statement. Nice find. How did you find this? ------------- Marked as reviewed by erikj (Lead). PR: https://git.openjdk.org/skara/pull/1353 From gli at openjdk.org Wed Aug 10 17:12:14 2022 From: gli at openjdk.org (Guoxiong Li) Date: Wed, 10 Aug 2022 17:12:14 GMT Subject: RFR: 1536: The method "JiraIssue#setState" is invalid when using the argument "State.CLOSED" [v3] In-Reply-To: References: <06dbiZl5-7Cy30t4NETNqMaHx9gLtEJvUfFWBr800TY=.a927a549-73ce-45e2-984b-08ef341c1791@github.com> Message-ID: On Wed, 10 Aug 2022 16:44:59 GMT, Erik Joelsson wrote: > Nice find. How did you find this? After Joe announcing the "In Review" substate is available in JBS [1], it is time for me to start working on SKARA-803 [2]. So I need to read the related code about the issue status, including the method `JiraIssue#setState`, then I find this bug. [1] https://mail.openjdk.org/pipermail/jdk-dev/2022-August/006829.html [2] https://bugs.openjdk.org/browse/SKARA-803 ------------- PR: https://git.openjdk.org/skara/pull/1353 From gli at openjdk.org Wed Aug 10 17:12:14 2022 From: gli at openjdk.org (Guoxiong Li) Date: Wed, 10 Aug 2022 17:12:14 GMT Subject: RFR: 1536: The method "JiraIssue#setState" is invalid when using the argument "State.CLOSED" [v3] In-Reply-To: References: <06dbiZl5-7Cy30t4NETNqMaHx9gLtEJvUfFWBr800TY=.a927a549-73ce-45e2-984b-08ef341c1791@github.com> Message-ID: On Wed, 10 Aug 2022 15:39:56 GMT, Guoxiong Li wrote: >> Hi all, >> >> Currently, when the method "JiraIssue#setState" is invoked with the argument "State.CLOSED", it can't work as expected. This patch fixes it and add a manual test case for it. And I run the newly added test munually to close SKARA-1463 [1] and SKARA-1516 [2] so that this patch can be verified. >> >> Thanks for taking the time to review. >> >> Best Regards, >> -- Guoxiong >> >> [1] https://bugs.openjdk.org/browse/SKARA-1463 >> [2] https://bugs.openjdk.org/browse/SKARA-1516 > > Guoxiong Li has updated the pull request incrementally with one additional commit since the last revision: > > Adjust the import statement. Thanks for the review. Integrating. ------------- PR: https://git.openjdk.org/skara/pull/1353 From gli at openjdk.org Wed Aug 10 17:12:14 2022 From: gli at openjdk.org (Guoxiong Li) Date: Wed, 10 Aug 2022 17:12:14 GMT Subject: Integrated: 1536: The method "JiraIssue#setState" is invalid when using the argument "State.CLOSED" In-Reply-To: <06dbiZl5-7Cy30t4NETNqMaHx9gLtEJvUfFWBr800TY=.a927a549-73ce-45e2-984b-08ef341c1791@github.com> References: <06dbiZl5-7Cy30t4NETNqMaHx9gLtEJvUfFWBr800TY=.a927a549-73ce-45e2-984b-08ef341c1791@github.com> Message-ID: On Wed, 10 Aug 2022 14:46:18 GMT, Guoxiong Li wrote: > Hi all, > > Currently, when the method "JiraIssue#setState" is invoked with the argument "State.CLOSED", it can't work as expected. This patch fixes it and add a manual test case for it. And I run the newly added test munually to close SKARA-1463 [1] and SKARA-1516 [2] so that this patch can be verified. > > Thanks for taking the time to review. > > Best Regards, > -- Guoxiong > > [1] https://bugs.openjdk.org/browse/SKARA-1463 > [2] https://bugs.openjdk.org/browse/SKARA-1516 This pull request has now been integrated. Changeset: 670ec1e5 Author: Guoxiong Li URL: https://git.openjdk.org/skara/commit/670ec1e53a33037ea6f484f1336070c4718d0094 Stats: 48 lines in 4 files changed: 44 ins; 1 del; 3 mod 1536: The method "JiraIssue#setState" is invalid when using the argument "State.CLOSED" Reviewed-by: ihse, erikj ------------- PR: https://git.openjdk.org/skara/pull/1353 From lgxbslgx at gmail.com Wed Aug 10 18:31:05 2022 From: lgxbslgx at gmail.com (Guoxiong Li) Date: Thu, 11 Aug 2022 02:31:05 +0800 Subject: Request for comment on adding "Under Review" as a substate of bugs In-Reply-To: References: <98d35bbf-9431-902a-3a63-d926e74cdfcf@oracle.com> <558af4a0-c758-9150-32bc-a9287a928f54@oracle.com> Message-ID: Hi Joe, Thanks for the work. It is very convenient to edit the `understanding` field directly. But I find it is unable to click the `Start Work` button if someone is not the assignee of the issue. So when solving SKARA-803 [1], I find we need to assign the issue to the bot, then "click" the `Start Work` button to transfer the status to `In Progress`, and at last assign the issue back to the actual developer. This situation is similar to the previous `understanding` field. The three steps (self-assign, Start work, assign back) solution is a workaround but it seems to be not very elegant. So I want to ask: can this be improved? Or can the bot accounts or other administrators' accounts click the `Start Work` button directly without as an assignee? [1] https://bugs.openjdk.org/browse/SKARA-803 Best Regards, -- Guoxiong -------------- next part -------------- An HTML attachment was scrubbed... URL: From erikj at openjdk.org Wed Aug 10 21:26:31 2022 From: erikj at openjdk.org (Erik Joelsson) Date: Wed, 10 Aug 2022 21:26:31 GMT Subject: RFR: 1537: SponsorCommand can fail indefinitely with '/integrate auto' Message-ID: This patch fixes two issues. The main one is an Optional "No value present" issue in SponsorCommand which can happen in combination with '/integrate auto' if a sponsor is too quick. With this patch, appropriate error replies are added by the bot when this happens, asking the sponsor to try again when the PR is in a suitable state. The commands on PRs and commits are designed to be evaluated in order, so there is really no way around this if they aren't issued in the right order. Related to this, I'm also fixing SKARA-1534, where CheckWorkItem would continuously post new '/integrate' comments when '/integrate auto' is activated. This would typically happen in conjunction with the above sponsor problem, and did actually cause my new test to fail (as it found 2 replies to /integrate instead of 1), so I decided to fix that in the same patch. To do that, I exposed some static utility methods from PullRequestCommandWorkItem to be able to identify if any unhandled '/integrate' commands were already present in the PR. ------------- Commit messages: - SKARA-1537 Changes: https://git.openjdk.org/skara/pull/1354/files Webrev: https://webrevs.openjdk.org/?repo=skara&pr=1354&range=00 Issue: https://bugs.openjdk.org/browse/SKARA-1537 Stats: 131 lines in 4 files changed: 109 ins; 13 del; 9 mod Patch: https://git.openjdk.org/skara/pull/1354.diff Fetch: git fetch https://git.openjdk.org/skara pull/1354/head:pull/1354 PR: https://git.openjdk.org/skara/pull/1354 From erikj at openjdk.org Wed Aug 10 21:34:35 2022 From: erikj at openjdk.org (Erik Joelsson) Date: Wed, 10 Aug 2022 21:34:35 GMT Subject: RFR: 1537: SponsorCommand can fail indefinitely with '/integrate auto' In-Reply-To: References: Message-ID: On Wed, 10 Aug 2022 21:21:40 GMT, Erik Joelsson wrote: > This patch fixes two issues. The main one is an Optional "No value present" issue in SponsorCommand which can happen in combination with '/integrate auto' if a sponsor is too quick. With this patch, appropriate error replies are added by the bot when this happens, asking the sponsor to try again when the PR is in a suitable state. The commands on PRs and commits are designed to be evaluated in order, so there is really no way around this if they aren't issued in the right order. > > Related to this, I'm also fixing SKARA-1534, where CheckWorkItem would continuously post new '/integrate' comments when '/integrate auto' is activated. This would typically happen in conjunction with the above sponsor problem, and did actually cause my new test to fail (as it found 2 replies to /integrate instead of 1), so I decided to fix that in the same patch. To do that, I exposed some static utility methods from PullRequestCommandWorkItem to be able to identify if any unhandled '/integrate' commands were already present in the PR. The previous behavior for the SponsorCommand lead to eternal retries in processing the /sponsor command, while CheckWorkItem would continue to post '/integrate' over and over again, so it's pretty serious to get this fixed. We currently have two PRs suffering from this live. ------------- PR: https://git.openjdk.org/skara/pull/1354 From erikj at openjdk.org Wed Aug 10 21:48:34 2022 From: erikj at openjdk.org (Erik Joelsson) Date: Wed, 10 Aug 2022 21:48:34 GMT Subject: RFR: 1537: SponsorCommand can fail indefinitely with '/integrate auto' [v2] In-Reply-To: References: Message-ID: > This patch fixes two issues. The main one is an Optional "No value present" issue in SponsorCommand which can happen in combination with '/integrate auto' if a sponsor is too quick. With this patch, appropriate error replies are added by the bot when this happens, asking the sponsor to try again when the PR is in a suitable state. The commands on PRs and commits are designed to be evaluated in order, so there is really no way around this if they aren't issued in the right order. > > Related to this, I'm also fixing SKARA-1534, where CheckWorkItem would continuously post new '/integrate' comments when '/integrate auto' is activated. This would typically happen in conjunction with the above sponsor problem, and did actually cause my new test to fail (as it found 2 replies to /integrate instead of 1), so I decided to fix that in the same patch. To do that, I exposed some static utility methods from PullRequestCommandWorkItem to be able to identify if any unhandled '/integrate' commands were already present in the PR. Erik Joelsson has updated the pull request with a new target base due to a merge or a rebase. The incremental webrev excludes the unrelated changes brought in by the merge/rebase. The pull request contains two additional commits since the last revision: - Merge commit '990ac8fb40d1c2943b680631d7dac7aa30844236' into SKARA-1537-sponsor-auto - SKARA-1537 ------------- Changes: - all: https://git.openjdk.org/skara/pull/1354/files - new: https://git.openjdk.org/skara/pull/1354/files/fe9726c4..2c3e7360 Webrevs: - full: https://webrevs.openjdk.org/?repo=skara&pr=1354&range=01 - incr: https://webrevs.openjdk.org/?repo=skara&pr=1354&range=00-01 Stats: 3 lines in 3 files changed: 0 ins; 0 del; 3 mod Patch: https://git.openjdk.org/skara/pull/1354.diff Fetch: git fetch https://git.openjdk.org/skara pull/1354/head:pull/1354 PR: https://git.openjdk.org/skara/pull/1354 From erikj at openjdk.org Thu Aug 11 13:55:36 2022 From: erikj at openjdk.org (Erik Joelsson) Date: Thu, 11 Aug 2022 13:55:36 GMT Subject: RFR: 1539: Allow dots in merge PR branch name Message-ID: This patch makes it possible to create merge style PRs based on branches with periods `.` in the branch name. We quite commonly put periods in branch names as we like to base branches on version strings. Modified an existing test to cover this case. ------------- Commit messages: - SKARA-1539 Changes: https://git.openjdk.org/skara/pull/1355/files Webrev: https://webrevs.openjdk.org/?repo=skara&pr=1355&range=00 Issue: https://bugs.openjdk.org/browse/SKARA-1539 Stats: 9 lines in 2 files changed: 0 ins; 0 del; 9 mod Patch: https://git.openjdk.org/skara/pull/1355.diff Fetch: git fetch https://git.openjdk.org/skara pull/1355/head:pull/1355 PR: https://git.openjdk.org/skara/pull/1355 From kcr at openjdk.org Thu Aug 11 14:02:35 2022 From: kcr at openjdk.org (Kevin Rushforth) Date: Thu, 11 Aug 2022 14:02:35 GMT Subject: RFR: 1539: Allow dots in merge PR branch name In-Reply-To: References: Message-ID: On Thu, 11 Aug 2022 13:51:02 GMT, Erik Joelsson wrote: > This patch makes it possible to create merge style PRs based on branches with periods `.` in the branch name. We quite commonly put periods in branch names as we like to base branches on version strings. > > Modified an existing test to cover this case. I left a question about the fix to the pattern string. forge/src/main/java/org/openjdk/skara/forge/PullRequestUtils.java line 42: > 40: } > 41: > 42: private final static Pattern mergeSourcePattern = Pattern.compile("^Merge ([-/.\\w:+]+)$"); This will match any character, which may not be what you intend. Shouldn't that be `\.` instead of just `.`? ------------- PR: https://git.openjdk.org/skara/pull/1355 From erikj at openjdk.org Thu Aug 11 14:09:17 2022 From: erikj at openjdk.org (Erik Joelsson) Date: Thu, 11 Aug 2022 14:09:17 GMT Subject: RFR: 1539: Allow dots in merge PR branch name In-Reply-To: References: Message-ID: On Thu, 11 Aug 2022 13:58:52 GMT, Kevin Rushforth wrote: >> This patch makes it possible to create merge style PRs based on branches with periods `.` in the branch name. We quite commonly put periods in branch names as we like to base branches on version strings. >> >> Modified an existing test to cover this case. > > forge/src/main/java/org/openjdk/skara/forge/PullRequestUtils.java line 42: > >> 40: } >> 41: >> 42: private final static Pattern mergeSourcePattern = Pattern.compile("^Merge ([-/.\\w:+]+)$"); > > This will match any character, which may not be what you intend. Shouldn't that be `\.` instead of just `.`? My understanding is that the only characters you need to escape inside a character class are `^`, `-`, `]` or ``, everything else is a literal. Also Intellij gave me a warning when I tried to escape it saying it was unnecessary. ------------- PR: https://git.openjdk.org/skara/pull/1355 From kcr at openjdk.org Thu Aug 11 14:17:39 2022 From: kcr at openjdk.org (Kevin Rushforth) Date: Thu, 11 Aug 2022 14:17:39 GMT Subject: RFR: 1539: Allow dots in merge PR branch name In-Reply-To: References: Message-ID: <8U0zedkvUUNcW09AKD_3UGA0JUF3mm3nWiEOseCCATY=.57486b5e-894e-41df-a9f9-7a9e735a35ab@github.com> On Thu, 11 Aug 2022 13:51:02 GMT, Erik Joelsson wrote: > This patch makes it possible to create merge style PRs based on branches with periods `.` in the branch name. We quite commonly put periods in branch names as we like to base branches on version strings. > > Modified an existing test to cover this case. Marked as reviewed by kcr (Reviewer). ------------- PR: https://git.openjdk.org/skara/pull/1355 From kcr at openjdk.org Thu Aug 11 14:17:39 2022 From: kcr at openjdk.org (Kevin Rushforth) Date: Thu, 11 Aug 2022 14:17:39 GMT Subject: RFR: 1539: Allow dots in merge PR branch name In-Reply-To: References: Message-ID: On Thu, 11 Aug 2022 14:05:32 GMT, Erik Joelsson wrote: >> forge/src/main/java/org/openjdk/skara/forge/PullRequestUtils.java line 42: >> >>> 40: } >>> 41: >>> 42: private final static Pattern mergeSourcePattern = Pattern.compile("^Merge ([-/.\\w:+]+)$"); >> >> This will match any character, which may not be what you intend. Shouldn't that be `\.` instead of just `.`? > > My understanding is that the only characters you need to escape inside a character class are `^`, `-`, `]` or ``, everything else is a literal. Also Intellij gave me a warning when I tried to escape it saying it was unnecessary. I stand corrected. I now see that `.` is a literal dot when it is inside `[]` (when not inside `[]` it matches any char). ------------- PR: https://git.openjdk.org/skara/pull/1355 From erikj at openjdk.org Thu Aug 11 17:09:11 2022 From: erikj at openjdk.org (Erik Joelsson) Date: Thu, 11 Aug 2022 17:09:11 GMT Subject: RFR: 1539: Allow dots in merge PR branch name [v2] In-Reply-To: References: Message-ID: > This patch makes it possible to create merge style PRs based on branches with periods `.` in the branch name. We quite commonly put periods in branch names as we like to base branches on version strings. > > Modified an existing test to cover this case. Erik Joelsson has updated the pull request incrementally with one additional commit since the last revision: Fix test and expand verification of valid branch names ------------- Changes: - all: https://git.openjdk.org/skara/pull/1355/files - new: https://git.openjdk.org/skara/pull/1355/files/bb6825b9..020d07f2 Webrevs: - full: https://webrevs.openjdk.org/?repo=skara&pr=1355&range=01 - incr: https://webrevs.openjdk.org/?repo=skara&pr=1355&range=00-01 Stats: 9 lines in 1 file changed: 0 ins; 0 del; 9 mod Patch: https://git.openjdk.org/skara/pull/1355.diff Fetch: git fetch https://git.openjdk.org/skara pull/1355/head:pull/1355 PR: https://git.openjdk.org/skara/pull/1355 From erikj at openjdk.org Thu Aug 11 17:45:04 2022 From: erikj at openjdk.org (Erik Joelsson) Date: Thu, 11 Aug 2022 17:45:04 GMT Subject: Integrated: 1539: Allow dots in merge PR branch name In-Reply-To: References: Message-ID: On Thu, 11 Aug 2022 13:51:02 GMT, Erik Joelsson wrote: > This patch makes it possible to create merge style PRs based on branches with periods `.` in the branch name. We quite commonly put periods in branch names as we like to base branches on version strings. > > Modified an existing test to cover this case. This pull request has now been integrated. Changeset: 1b79f292 Author: Erik Joelsson URL: https://git.openjdk.org/skara/commit/1b79f292f76562be87681fbd115e031ae667dfc0 Stats: 10 lines in 2 files changed: 0 ins; 0 del; 10 mod 1539: Allow dots in merge PR branch name Reviewed-by: kcr ------------- PR: https://git.openjdk.org/skara/pull/1355 From erikj at openjdk.org Thu Aug 11 17:47:04 2022 From: erikj at openjdk.org (Erik Joelsson) Date: Thu, 11 Aug 2022 17:47:04 GMT Subject: Integrated: 1529: Optimize Census namespace instantiation In-Reply-To: References: Message-ID: <92jxhYULOv2fxbQNdEnXYKtzcXcoLiZf4Asx-ZulJDQ=.9fe0cfef-8edb-465b-9951-6113a48205c4@github.com> On Fri, 5 Aug 2022 17:52:55 GMT, Erik Joelsson wrote: > This patch adds a new factory method `Census::parseNamespace`, which enables initialization of a single Namespace directly from a HostedRepository (without having to clone the census repo to local disk first). > > The motivation behind this is bot performance. The Census is loaded in many WorkItems. Cloning/fetching the complete census repo to local disk is relatively expensive compared to a couple of REST calls to just read files directly from the remote repo. Especially as our REST client already leverages caching through ETag. This particular optimization can however only be applied to the IssueNotifier in the NotifyBot, where I doubt it will make that much of a difference. The main performance gain is realized in a different bot which currently isn't public (but these gains will impact all Skara users). > > I chose to at least for now, only apply this for initializing a single Namespace as that can be done by reading only 2 files from the remote, so the performance gain is very clear. Other bots either need to read considerably more files, or have the need to update the census. This optimization idea may be expanded in the future if I see a need and noticeable gains from it. > > In order to make HostedRepository available to the Census class, I had to get rid of several existing module dependencies to avoid circular dependencies. Fortunately this wasn't too hard. It made no sense to me that `org.openjdk.skara.forge` would require `org.openjdk.skara.census`. There was just one static method `PullRequestUtils` in the forge module that used a reference to the census module, and this static method was only used from one module `org.openjdk.skara.bots.pr`, so I simply moved it to `CheckablePullRequest` which is kind of a utility class in that module. The rest of the offending dependencies were just unused, so I cleaned out a fair few of those. > > In the `IssueNotifier`, in addition to using the new initialization method, I also added lazy initialization and reusing of the `CensusInstance` class, to avoid re-initializing the same data for every round in loops. This pull request has now been integrated. Changeset: f9d5585f Author: Erik Joelsson URL: https://git.openjdk.org/skara/commit/f9d5585f97354abe575839d7f7d1bab0d05a4d06 Stats: 209 lines in 18 files changed: 120 ins; 64 del; 25 mod 1529: Optimize Census namespace instantiation Reviewed-by: ihse ------------- PR: https://git.openjdk.org/skara/pull/1348 From erikj at openjdk.org Thu Aug 11 20:45:37 2022 From: erikj at openjdk.org (Erik Joelsson) Date: Thu, 11 Aug 2022 20:45:37 GMT Subject: RFR: 1542: Census Namespace missing hashCode and equals Message-ID: The Namespace class is missing hashCode and equals. This patch adds implementations for those two methods. The other data classes in the Census module already have them. This is preventing the change in [SKARA-1529](https://bugs.openjdk.org/browse/SKARA-1529) from actually taking effect, as we can't compare two instances of Namespace to see if they have the same contents. ------------- Commit messages: - SKARA-1542 Changes: https://git.openjdk.org/skara/pull/1356/files Webrev: https://webrevs.openjdk.org/?repo=skara&pr=1356&range=00 Issue: https://bugs.openjdk.org/browse/SKARA-1542 Stats: 17 lines in 1 file changed: 17 ins; 0 del; 0 mod Patch: https://git.openjdk.org/skara/pull/1356.diff Fetch: git fetch https://git.openjdk.org/skara pull/1356/head:pull/1356 PR: https://git.openjdk.org/skara/pull/1356 From kcr at openjdk.org Thu Aug 11 21:28:13 2022 From: kcr at openjdk.org (Kevin Rushforth) Date: Thu, 11 Aug 2022 21:28:13 GMT Subject: RFR: 1542: Census Namespace missing hashCode and equals In-Reply-To: References: Message-ID: On Thu, 11 Aug 2022 20:40:10 GMT, Erik Joelsson wrote: > The Namespace class is missing hashCode and equals. This patch adds implementations for those two methods. The other data classes in the Census module already have them. > > This is preventing the change in [SKARA-1529](https://bugs.openjdk.org/browse/SKARA-1529) from actually taking effect, as we can't compare two instances of Namespace to see if they have the same contents. This looks fine presuming that name, mapping, and reverse are always non-null. ------------- Marked as reviewed by kcr (Reviewer). PR: https://git.openjdk.org/skara/pull/1356 From darcy at openjdk.org Thu Aug 11 21:28:13 2022 From: darcy at openjdk.org (Joe Darcy) Date: Thu, 11 Aug 2022 21:28:13 GMT Subject: RFR: 1542: Census Namespace missing hashCode and equals In-Reply-To: References: Message-ID: On Thu, 11 Aug 2022 20:40:10 GMT, Erik Joelsson wrote: > The Namespace class is missing hashCode and equals. This patch adds implementations for those two methods. The other data classes in the Census module already have them. > > This is preventing the change in [SKARA-1529](https://bugs.openjdk.org/browse/SKARA-1529) from actually taking effect, as we can't compare two instances of Namespace to see if they have the same contents. Marked as reviewed by darcy (Reviewer). census/src/main/java/org/openjdk/skara/census/Namespace.java line 100: > 98: return true; > 99: } > 100: if (o == null || getClass() != o.getClass()) { The "same type" check could be phrased differently using instanceof, but what you have here should work okay too. ------------- PR: https://git.openjdk.org/skara/pull/1356 From erikj at openjdk.org Thu Aug 11 21:33:16 2022 From: erikj at openjdk.org (Erik Joelsson) Date: Thu, 11 Aug 2022 21:33:16 GMT Subject: RFR: 1542: Census Namespace missing hashCode and equals In-Reply-To: References: Message-ID: On Thu, 11 Aug 2022 21:24:47 GMT, Joe Darcy wrote: >> The Namespace class is missing hashCode and equals. This patch adds implementations for those two methods. The other data classes in the Census module already have them. >> >> This is preventing the change in [SKARA-1529](https://bugs.openjdk.org/browse/SKARA-1529) from actually taking effect, as we can't compare two instances of Namespace to see if they have the same contents. > > census/src/main/java/org/openjdk/skara/census/Namespace.java line 100: > >> 98: return true; >> 99: } >> 100: if (o == null || getClass() != o.getClass()) { > > The "same type" check could be phrased differently using instanceof, but what you have here should work okay too. Hm, I let the IDE generate these for me so didn't think much of it. Aren't there subtle differences with instanceof, if there are subtypes involved? Not that it matters in this particular instance. ------------- PR: https://git.openjdk.org/skara/pull/1356 From erikj at openjdk.org Thu Aug 11 21:42:22 2022 From: erikj at openjdk.org (Erik Joelsson) Date: Thu, 11 Aug 2022 21:42:22 GMT Subject: RFR: 1542: Census Namespace missing hashCode and equals [v2] In-Reply-To: References: Message-ID: > The Namespace class is missing hashCode and equals. This patch adds implementations for those two methods. The other data classes in the Census module already have them. > > This is preventing the change in [SKARA-1529](https://bugs.openjdk.org/browse/SKARA-1529) from actually taking effect, as we can't compare two instances of Namespace to see if they have the same contents. Erik Joelsson has updated the pull request incrementally with one additional commit since the last revision: Use Object.equals ------------- Changes: - all: https://git.openjdk.org/skara/pull/1356/files - new: https://git.openjdk.org/skara/pull/1356/files/e0f5d6ea..afcdb1f8 Webrevs: - full: https://webrevs.openjdk.org/?repo=skara&pr=1356&range=01 - incr: https://webrevs.openjdk.org/?repo=skara&pr=1356&range=00-01 Stats: 3 lines in 1 file changed: 2 ins; 0 del; 1 mod Patch: https://git.openjdk.org/skara/pull/1356.diff Fetch: git fetch https://git.openjdk.org/skara pull/1356/head:pull/1356 PR: https://git.openjdk.org/skara/pull/1356 From erikj at openjdk.org Thu Aug 11 21:42:22 2022 From: erikj at openjdk.org (Erik Joelsson) Date: Thu, 11 Aug 2022 21:42:22 GMT Subject: RFR: 1542: Census Namespace missing hashCode and equals [v2] In-Reply-To: References: Message-ID: On Thu, 11 Aug 2022 21:22:11 GMT, Kevin Rushforth wrote: > This looks fine presuming that name, mapping, and reverse are always non-null. Right, I checked the code again, and while the two maps will currently never be null, the name field could potentially end up as null if given bad input data. I think it's good to be defensive and just use the static Object.equals for all the fields. I did this by just rerunning the intellij wizard. ------------- PR: https://git.openjdk.org/skara/pull/1356 From erikj at openjdk.org Thu Aug 11 21:42:23 2022 From: erikj at openjdk.org (Erik Joelsson) Date: Thu, 11 Aug 2022 21:42:23 GMT Subject: RFR: 1542: Census Namespace missing hashCode and equals [v2] In-Reply-To: References: Message-ID: On Thu, 11 Aug 2022 21:29:39 GMT, Erik Joelsson wrote: >> census/src/main/java/org/openjdk/skara/census/Namespace.java line 100: >> >>> 98: return true; >>> 99: } >>> 100: if (o == null || getClass() != o.getClass()) { >> >> The "same type" check could be phrased differently using instanceof, but what you have here should work okay too. > > Hm, I let the IDE generate these for me so didn't think much of it. Aren't there subtle differences with instanceof, if there are subtypes involved? Not that it matters in this particular instance. So I experimented some more. In the intellij wizard for this, if I check "accept subclasses", it generates code with `instanceof` instead. I don't think this equals method should do that. ------------- PR: https://git.openjdk.org/skara/pull/1356 From erikj at openjdk.org Thu Aug 11 21:45:54 2022 From: erikj at openjdk.org (Erik Joelsson) Date: Thu, 11 Aug 2022 21:45:54 GMT Subject: Integrated: 1542: Census Namespace missing hashCode and equals In-Reply-To: References: Message-ID: On Thu, 11 Aug 2022 20:40:10 GMT, Erik Joelsson wrote: > The Namespace class is missing hashCode and equals. This patch adds implementations for those two methods. The other data classes in the Census module already have them. > > This is preventing the change in [SKARA-1529](https://bugs.openjdk.org/browse/SKARA-1529) from actually taking effect, as we can't compare two instances of Namespace to see if they have the same contents. This pull request has now been integrated. Changeset: 4ac1e709 Author: Erik Joelsson URL: https://git.openjdk.org/skara/commit/4ac1e7097f69435ede9eb8aaa7b662ba7eea8679 Stats: 19 lines in 1 file changed: 19 ins; 0 del; 0 mod 1542: Census Namespace missing hashCode and equals Reviewed-by: kcr, darcy ------------- PR: https://git.openjdk.org/skara/pull/1356 From kcr at openjdk.org Thu Aug 11 22:22:23 2022 From: kcr at openjdk.org (Kevin Rushforth) Date: Thu, 11 Aug 2022 22:22:23 GMT Subject: RFR: 1537: SponsorCommand can fail indefinitely with '/integrate auto' [v2] In-Reply-To: References: Message-ID: On Wed, 10 Aug 2022 21:48:34 GMT, Erik Joelsson wrote: >> This patch fixes two issues. The main one is an Optional "No value present" issue in SponsorCommand which can happen in combination with '/integrate auto' if a sponsor is too quick. With this patch, appropriate error replies are added by the bot when this happens, asking the sponsor to try again when the PR is in a suitable state. The commands on PRs and commits are designed to be evaluated in order, so there is really no way around this if they aren't issued in the right order. >> >> Related to this, I'm also fixing SKARA-1534, where CheckWorkItem would continuously post new '/integrate' comments when '/integrate auto' is activated. This would typically happen in conjunction with the above sponsor problem, and did actually cause my new test to fail (as it found 2 replies to /integrate instead of 1), so I decided to fix that in the same patch. To do that, I exposed some static utility methods from PullRequestCommandWorkItem to be able to identify if any unhandled '/integrate' commands were already present in the PR. > > Erik Joelsson has updated the pull request with a new target base due to a merge or a rebase. The incremental webrev excludes the unrelated changes brought in by the merge/rebase. The pull request contains two additional commits since the last revision: > > - Merge commit '990ac8fb40d1c2943b680631d7dac7aa30844236' into SKARA-1537-sponsor-auto > - SKARA-1537 The changes look good to me. ------------- Marked as reviewed by kcr (Reviewer). PR: https://git.openjdk.org/skara/pull/1354 From erikj at openjdk.org Thu Aug 11 22:30:22 2022 From: erikj at openjdk.org (Erik Joelsson) Date: Thu, 11 Aug 2022 22:30:22 GMT Subject: RFR: 1537: SponsorCommand can fail indefinitely with '/integrate auto' [v2] In-Reply-To: References: Message-ID: On Wed, 10 Aug 2022 21:48:34 GMT, Erik Joelsson wrote: >> This patch fixes two issues. The main one is an Optional "No value present" issue in SponsorCommand which can happen in combination with '/integrate auto' if a sponsor is too quick. With this patch, appropriate error replies are added by the bot when this happens, asking the sponsor to try again when the PR is in a suitable state. The commands on PRs and commits are designed to be evaluated in order, so there is really no way around this if they aren't issued in the right order. >> >> Related to this, I'm also fixing SKARA-1534, where CheckWorkItem would continuously post new '/integrate' comments when '/integrate auto' is activated. This would typically happen in conjunction with the above sponsor problem, and did actually cause my new test to fail (as it found 2 replies to /integrate instead of 1), so I decided to fix that in the same patch. To do that, I exposed some static utility methods from PullRequestCommandWorkItem to be able to identify if any unhandled '/integrate' commands were already present in the PR. > > Erik Joelsson has updated the pull request with a new target base due to a merge or a rebase. The incremental webrev excludes the unrelated changes brought in by the merge/rebase. The pull request contains two additional commits since the last revision: > > - Merge commit '990ac8fb40d1c2943b680631d7dac7aa30844236' into SKARA-1537-sponsor-auto > - SKARA-1537 Thanks! ------------- PR: https://git.openjdk.org/skara/pull/1354 From erikj at openjdk.org Thu Aug 11 22:30:22 2022 From: erikj at openjdk.org (Erik Joelsson) Date: Thu, 11 Aug 2022 22:30:22 GMT Subject: Integrated: 1537: SponsorCommand can fail indefinitely with '/integrate auto' In-Reply-To: References: Message-ID: On Wed, 10 Aug 2022 21:21:40 GMT, Erik Joelsson wrote: > This patch fixes two issues. The main one is an Optional "No value present" issue in SponsorCommand which can happen in combination with '/integrate auto' if a sponsor is too quick. With this patch, appropriate error replies are added by the bot when this happens, asking the sponsor to try again when the PR is in a suitable state. The commands on PRs and commits are designed to be evaluated in order, so there is really no way around this if they aren't issued in the right order. > > Related to this, I'm also fixing SKARA-1534, where CheckWorkItem would continuously post new '/integrate' comments when '/integrate auto' is activated. This would typically happen in conjunction with the above sponsor problem, and did actually cause my new test to fail (as it found 2 replies to /integrate instead of 1), so I decided to fix that in the same patch. To do that, I exposed some static utility methods from PullRequestCommandWorkItem to be able to identify if any unhandled '/integrate' commands were already present in the PR. This pull request has now been integrated. Changeset: 16025165 Author: Erik Joelsson URL: https://git.openjdk.org/skara/commit/16025165f07d96d3a698c651d229d6980bc0e6ee Stats: 131 lines in 4 files changed: 109 ins; 13 del; 9 mod 1537: SponsorCommand can fail indefinitely with '/integrate auto' 1534: Bot posts multiple /integrate commands Reviewed-by: kcr ------------- PR: https://git.openjdk.org/skara/pull/1354 From darcy at openjdk.org Thu Aug 11 22:52:23 2022 From: darcy at openjdk.org (Joe Darcy) Date: Thu, 11 Aug 2022 22:52:23 GMT Subject: RFR: 1542: Census Namespace missing hashCode and equals [v2] In-Reply-To: References: Message-ID: On Thu, 11 Aug 2022 21:36:33 GMT, Erik Joelsson wrote: >> Hm, I let the IDE generate these for me so didn't think much of it. Aren't there subtle differences with instanceof, if there are subtypes involved? Not that it matters in this particular instance. > > So I experimented some more. In the intellij wizard for this, if I check "accept subclasses", it generates code with `instanceof` instead. I don't think this equals method should do that. To circle back on this, there is a school of thought that getClass should not be used in this fashion in an equals method since it violates the Liskov substitution principle -- instances of a subclass can be used where instances of the superclass can used. "Effective Java, 3rd edition" covers this topic in item 10. However, in a case like this where (I assume) subclasses are not expected, it doesn't matter very much in practice. ------------- PR: https://git.openjdk.org/skara/pull/1356 From erikj at openjdk.org Fri Aug 12 22:28:12 2022 From: erikj at openjdk.org (Erik Joelsson) Date: Fri, 12 Aug 2022 22:28:12 GMT Subject: RFR: 1532: CSRBot is too inefficient Message-ID: <_2T-daSUsHi9kbLO4LlSGvmAmk2DmfarQGSj5rmgflc=.785e2a6d-5844-483a-9b3c-046547f82c40@github.com> This patch is a pretty major redesign of how the CSRBot polls for work. The motivation for this is described in the bug description. It's quite big, so I will try to break down the major changes. 1. The `CSRBot` has been split into `CSRIssueBot` and `CSRPullRequestBot`. The issue bot polls the issue tracker (JBS) for work, while the pull request bot polls the forge (github). The WorkItem part of the old CSRBot has been moved to the new `PullRequestWorkItem`, and except for now only handling one single PR per WorkItem instance (instead of everything in a repo), has been left pretty much intact. All actual updates on PRs are (still) handled by this WorkItem. The new `IssueWorkItem` is created when a change to a CSR Issue has been detected (in JBS). It walks the JBS issue and PR links to identify all PRs that could possibly be related to that CSR, and creates `PullRequestWorkItem`s for them. 2. To be able to find PRs from JBS issues, the CSRBot needs to know how to parse the PR comment links. To better share this knowledge, I moved the creation and deletion logic from the NotifyBot to PullRequestUtils (in the forge module), which is available to all bots that need it. I also added a method for parsing the URL from such a comment. 3. To efficiently query for updated CSR issues from Jira, I added two new query methods on IssueProject. A longer explanation of the new polling mechanism for issues can be found in the first bug comment. I've also tried to reasonably explain things in comments in the code. 4. In `JiraProject`, there was a pretty serious bug in all query methods for Issues. The `JiraIssue` constructor takes a `RestRequest` as a parameter. This request object is supposed to be used by `JiraIssue` for generating all the subqueries for the Jira REST API. The problem was that the query methods in `JiraProject` just sent in whatever `RestRequest` they used to find issues, which broke all methods that tried to make further calls through the returned `JiraIssue` to Jira. I fixed this by always supplying a `RestRequest` object with the correct URL when creating a JiraIssue. 5. To be able to call Jira with absolute timestamps in queries, we need to know the timezone of the user we are calling as. Luckily this information was available with a simple call, so I added this to the `JiraHost` class. 6. Similar to how care needs to be taken when polling for updates to issues in Jira, I applied a similar solution when polling PRs. This required a new method `HostedRepository.openPullRequestsAfter`. Unfortunately, it doesn't help much on Github, where the API doesn't support such a parameter, but it works for Gitlab. I have modified the tests that are affected by this change so that they still pass. This does verify that changes to issues in the IssueTracker are detected. Perhaps I should add some new tests to specifically verify the polling logic. I have verified the new queries and complete functionality manually by running the bot against the playground repo and bugs-stage. ------------- Commit messages: - SKARA-1532 Changes: https://git.openjdk.org/skara/pull/1357/files Webrev: https://webrevs.openjdk.org/?repo=skara&pr=1357&range=00 Issue: https://bugs.openjdk.org/browse/SKARA-1532 Stats: 1099 lines in 21 files changed: 798 ins; 256 del; 45 mod Patch: https://git.openjdk.org/skara/pull/1357.diff Fetch: git fetch https://git.openjdk.org/skara pull/1357/head:pull/1357 PR: https://git.openjdk.org/skara/pull/1357 From joe.darcy at oracle.com Fri Aug 12 23:08:23 2022 From: joe.darcy at oracle.com (Joe Darcy) Date: Fri, 12 Aug 2022 16:08:23 -0700 Subject: Request for comment on adding "Under Review" as a substate of bugs In-Reply-To: References: <98d35bbf-9431-902a-3a63-d926e74cdfcf@oracle.com> <558af4a0-c758-9150-32bc-a9287a928f54@oracle.com> Message-ID: Hi Guoxiong, Feedback noted; we'll look into changing the workflow to make setting the state in question easier for the Skara bots. Thanks, -Joe On 8/10/2022 11:31 AM, Guoxiong Li wrote: > Hi Joe, > > Thanks for the work. It is very convenient?to edit the `understanding` > field directly. > > But I find it is unable to click the `Start Work` button if someone is > not the assignee of the issue. > So when solving SKARA-803 [1], I find we need to assign the issue to > the bot, then "click" > the `Start Work` button to transfer the status to `In Progress`, > and at last assign the issue back to the actual developer. > This situation is similar to the previous `understanding` field. > > The three?steps (self-assign, Start work, assign back) solution is a > workaround > but it seems to be not very elegant. > So I want to ask: can this be improved? > Or can the bot accounts or other administrators' accounts > click the?`Start Work` button directly without as an assignee? > > [1] https://bugs.openjdk.org/browse/SKARA-803 > > Best Regards, > -- Guoxiong -------------- next part -------------- An HTML attachment was scrubbed... URL: From christoph.langer at sap.com Mon Aug 15 06:01:55 2022 From: christoph.langer at sap.com (Langer, Christoph) Date: Mon, 15 Aug 2022 06:01:55 +0000 Subject: [Proposal] Maintainer Approvals and SKARA In-Reply-To: <158074e04fcf69679b242b4e6e1f79e377447662.camel@redhat.com> References: <639d9498-c764-2005-30ef-87351d639d9d@oracle.com> <158074e04fcf69679b242b4e6e1f79e377447662.camel@redhat.com> Message-ID: Hi Guoxiong, +1 from me as well. Would be absolutely great if this part of the backport process could be improved soon. ? Best regards Christoph > -----Original Message----- > From: jdk-updates-dev On Behalf Of > Severin Gehwolf > Sent: Dienstag, 9. August 2022 10:43 > To: Guoxiong Li ; jdk8u-dev at openjdk.org; jdk-updates- > dev at openjdk.org; skara-dev at openjdk.org > Subject: Re: [Proposal] Maintainer Approvals and SKARA > > Hi, > > On Tue, 2022-08-09 at 04:25 +0800, Guoxiong Li wrote: > > Hi all, > > > > I re-read your comments about this enhancement recently. > > And I also made a mistake when I backport a patch to jdk11u-dev [1] > > although I have backported several times. > > So SKARA must direct the developers accurately and help developers > > complete the duplicated work to avoid the unnecessary mistakes. > > > > Now I propose the following dev flow. Please only focus on the dev > > flow instead of the code implementation here. > > > > - When a backport PR is created in Github. > > 1. A new checkbox item `[ ] Change must be properly approved by the > > maintainers` will be added to the `Progress` part of the current PR > > body. > > 2. A comment is posted by the bot like below (the italic content) to > > direct the developers: > > This is a backport pull request. Please add a comment to the main > > issue [JDK-XXXX](link) to state the?related condition (the backport > > reason, the risk of the patch, the amount of work and so on). Below is > > an example for you: > > ``` > > Fix Request(jdk17u-dev) > > The code applies cleanly and the test in this change fails without the > > patch and succeeds after applying it. > > The risk of this backport is low because the change is little and the > > issue fixed by this change also exists in jdk xy. > > ``` > > If you don't have permission to add a comment in JBS. Please use the > > command `request-approval` to provide the related content, then?the > > bot can help you add a comment by using the content you provided. > > Below is an example for you: > > ``` > > /request-approval > > Fix Request(jdk17u-dev) > > The code applies cleanly and the test in this change fails without the > > patch and succeeds after applying it. > > The risk of this backport is low because the change is little and the > > issue fixed by this change also exists in jdk xy. > > ``` > > Please note you have to contact the maintainers?directly?in the issue > > [JDK-XXXX](link) or by using the command?`request-approval`?repeatedly > > instead of in this pull request. > > And you don't need to add the fix request label manually to the issue > > like before, now the bot can help you add the label automatically when > > this pull request is ready?for maintainers to approve. > > > > - When the backport PR is ready for approval (other checks have > > succeeded and other progresses have been done) 1. The bot adds a label > > named `jdkXXu-fix-request` to all the issues of the PR 2. The bot adds > > a blocked label named `approval` in PR, like `csr` or other blocked > > labels These two labels are convenient for maintainers to filter the > > issues and PRs which need to be handled now. > > > > - It is time for maintainers to approve The maintainers can add the > > label `jdkXXu-fix-yes` or `jdkXXu-fix-no` in the issue or use the > > command `/approval yes|no|y|n` in the PR to approve or object the > > patch. This newly added command `approval` can be very convenient if > > one backport PR has many corresponding issues. After using this > > command, the bot will add the related label to all the issues just > > like the label `jdkXXu-fix-request`?it had added. > > > > - Then the commit or sponsor flow will continue as usual. > > After approval, if the maintainer said 'yes', the PR will become ready > > to be integrated. > > If the maintainer said 'no', the PR can be closed?by the bot with a > > comment like "The maintainers disapproved of this patch so this pull > > request will be closed automatically". > > No matter 'yes' or 'no', the label `approval` in the PR will be > > removed. > > > > Above is all the dev flow. And next, I will give more information > > about the new commands `request-approval` and `approval`. > > > > - "request-approval" command > > It is similar to the `summary` command which permits multiline > > content. And this command can be used multi times if the author of the > > PR wants to. Each time this command is used, the bot will post the > > content to a new comment in the issue. Because the author who has no > > permission may want to contact/talk with the maintainers. > > The related record will be recorded in the issue instead of PR which > > is your intention in your previous comments. > > > > - "approval" command > > The command `/approval yes` or `/approval y` mean `approved`. > > And the command `/approval no` or `/approval n` mean `dispproved`. > > This command can be used multiple times and only the last time is > > valid. > > Please note when the `/approval no` or `/approval n` is used, the bot > > will close the PR, and when the `/approval yes` or `/approval y` later > > is used, the bot will open the PR automatically. > > > > > > That is all. Thanks for reading and providing feedback. And I will try > > to implement these features if I have time and hear no objection?after > > discussion. > > > > This sounds good to me. +1. Perhaps we should move this discussion and > implementation proposal to the bug tracking this issue as it is hard to find > those mailing list posts later on? > https://bugs.openjdk.org/browse/SKARA-1199 > > It also seems to be in-line with what Andrew Hughes suggested in [i]. > > Thanks, > Severin > > [i] https://mail.openjdk.org/pipermail/jdk-updates-dev/2022- > March/013278.html > > > [1] > > https://github.com/openjdk/jdk11u-dev/pull/1218#issuecomment-11842356 > > 01 > > > > Best Regards, > > -- Guoxiong From goetz.lindenmaier at sap.com Mon Aug 15 06:11:04 2022 From: goetz.lindenmaier at sap.com (Lindenmaier, Goetz) Date: Mon, 15 Aug 2022 06:11:04 +0000 Subject: [Proposal] Maintainer Approvals and SKARA In-Reply-To: References: <639d9498-c764-2005-30ef-87351d639d9d@oracle.com> <158074e04fcf69679b242b4e6e1f79e377447662.camel@redhat.com> Message-ID: Hi, +1, This is a good improval. Please remember that the instructions should not talk about backports, but about changes to an updates repository. Sometimes there are genuine changes for an update repository, these have to follow the very same approval rules. > > > This is a backport pull request. Please add a comment to the main > > > issue [JDK-XXXX](link) to state the related condition (the backport > > > reason, the risk of the patch, the amount of work and so on). Below is > > > an example for you: "This is a change to an updates repository. Please add a comment to the JBS issue, in case of a backport to the existing main issue [JDK-XXXX](link). State the related condition (Why you do the change/backport, the risk of the patch, the amount of work and so on). Below is an example for you: ... Best regards, Goetz > -----Original Message----- > From: skara-dev On Behalf Of Langer, Christoph > Sent: Montag, 15. August 2022 08:02 > To: Severin Gehwolf ; Guoxiong Li > ; jdk8u-dev at openjdk.org; jdk-updates- > dev at openjdk.org; skara-dev at openjdk.org > Subject: RE: [Proposal] Maintainer Approvals and SKARA > > Hi Guoxiong, > > +1 from me as well. Would be absolutely great if this part of the backport > process could be improved soon. ? > > Best regards > Christoph > > > -----Original Message----- > > From: jdk-updates-dev On Behalf Of > > Severin Gehwolf > > Sent: Dienstag, 9. August 2022 10:43 > > To: Guoxiong Li ; jdk8u-dev at openjdk.org; jdk-updates- > > dev at openjdk.org; skara-dev at openjdk.org > > Subject: Re: [Proposal] Maintainer Approvals and SKARA > > > > Hi, > > > > On Tue, 2022-08-09 at 04:25 +0800, Guoxiong Li wrote: > > > Hi all, > > > > > > I re-read your comments about this enhancement recently. > > > And I also made a mistake when I backport a patch to jdk11u-dev [1] > > > although I have backported several times. > > > So SKARA must direct the developers accurately and help developers > > > complete the duplicated work to avoid the unnecessary mistakes. > > > > > > Now I propose the following dev flow. Please only focus on the dev > > > flow instead of the code implementation here. > > > > > > - When a backport PR is created in Github. > > > 1. A new checkbox item `[ ] Change must be properly approved by the > > > maintainers` will be added to the `Progress` part of the current PR > > > body. > > > 2. A comment is posted by the bot like below (the italic content) to > > > direct the developers: > > > This is a backport pull request. Please add a comment to the main > > > issue [JDK-XXXX](link) to state the?related condition (the backport > > > reason, the risk of the patch, the amount of work and so on). Below is > > > an example for you: > > > ``` > > > Fix Request(jdk17u-dev) > > > The code applies cleanly and the test in this change fails without the > > > patch and succeeds after applying it. > > > The risk of this backport is low because the change is little and the > > > issue fixed by this change also exists in jdk xy. > > > ``` > > > If you don't have permission to add a comment in JBS. Please use the > > > command `request-approval` to provide the related content, then?the > > > bot can help you add a comment by using the content you provided. > > > Below is an example for you: > > > ``` > > > /request-approval > > > Fix Request(jdk17u-dev) > > > The code applies cleanly and the test in this change fails without the > > > patch and succeeds after applying it. > > > The risk of this backport is low because the change is little and the > > > issue fixed by this change also exists in jdk xy. > > > ``` > > > Please note you have to contact the maintainers?directly?in the issue > > > [JDK-XXXX](link) or by using the command?`request-approval`?repeatedly > > > instead of in this pull request. > > > And you don't need to add the fix request label manually to the issue > > > like before, now the bot can help you add the label automatically when > > > this pull request is ready?for maintainers to approve. > > > > > > - When the backport PR is ready for approval (other checks have > > > succeeded and other progresses have been done) 1. The bot adds a label > > > named `jdkXXu-fix-request` to all the issues of the PR 2. The bot adds > > > a blocked label named `approval` in PR, like `csr` or other blocked > > > labels These two labels are convenient for maintainers to filter the > > > issues and PRs which need to be handled now. > > > > > > - It is time for maintainers to approve The maintainers can add the > > > label `jdkXXu-fix-yes` or `jdkXXu-fix-no` in the issue or use the > > > command `/approval yes|no|y|n` in the PR to approve or object the > > > patch. This newly added command `approval` can be very convenient if > > > one backport PR has many corresponding issues. After using this > > > command, the bot will add the related label to all the issues just > > > like the label `jdkXXu-fix-request`?it had added. > > > > > > - Then the commit or sponsor flow will continue as usual. > > > After approval, if the maintainer said 'yes', the PR will become ready > > > to be integrated. > > > If the maintainer said 'no', the PR can be closed?by the bot with a > > > comment like "The maintainers disapproved of this patch so this pull > > > request will be closed automatically". > > > No matter 'yes' or 'no', the label `approval` in the PR will be > > > removed. > > > > > > Above is all the dev flow. And next, I will give more information > > > about the new commands `request-approval` and `approval`. > > > > > > - "request-approval" command > > > It is similar to the `summary` command which permits multiline > > > content. And this command can be used multi times if the author of the > > > PR wants to. Each time this command is used, the bot will post the > > > content to a new comment in the issue. Because the author who has no > > > permission may want to contact/talk with the maintainers. > > > The related record will be recorded in the issue instead of PR which > > > is your intention in your previous comments. > > > > > > - "approval" command > > > The command `/approval yes` or `/approval y` mean `approved`. > > > And the command `/approval no` or `/approval n` mean `dispproved`. > > > This command can be used multiple times and only the last time is > > > valid. > > > Please note when the `/approval no` or `/approval n` is used, the bot > > > will close the PR, and when the `/approval yes` or `/approval y` later > > > is used, the bot will open the PR automatically. > > > > > > > > > That is all. Thanks for reading and providing feedback. And I will try > > > to implement these features if I have time and hear no objection?after > > > discussion. > > > > > > > This sounds good to me. +1. Perhaps we should move this discussion and > > implementation proposal to the bug tracking this issue as it is hard to find > > those mailing list posts later on? > > https://bugs.openjdk.org/browse/SKARA-1199 > > > > It also seems to be in-line with what Andrew Hughes suggested in [i]. > > > > Thanks, > > Severin > > > > [i] https://mail.openjdk.org/pipermail/jdk-updates-dev/2022- > > March/013278.html > > > > > [1] > > > https://github.com/openjdk/jdk11u-dev/pull/1218#issuecomment- > 11842356 > > > 01 > > > > > > Best Regards, > > > -- Guoxiong From gli at openjdk.org Mon Aug 15 14:04:30 2022 From: gli at openjdk.org (Guoxiong Li) Date: Mon, 15 Aug 2022 14:04:30 GMT Subject: RFR: 1532: CSRBot is too inefficient In-Reply-To: <_2T-daSUsHi9kbLO4LlSGvmAmk2DmfarQGSj5rmgflc=.785e2a6d-5844-483a-9b3c-046547f82c40@github.com> References: <_2T-daSUsHi9kbLO4LlSGvmAmk2DmfarQGSj5rmgflc=.785e2a6d-5844-483a-9b3c-046547f82c40@github.com> Message-ID: On Fri, 12 Aug 2022 22:19:58 GMT, Erik Joelsson wrote: > This patch is a pretty major redesign of how the CSRBot polls for work. The motivation for this is described in the bug description. It's quite big, so I will try to break down the major changes. > > 1. The `CSRBot` has been split into `CSRIssueBot` and `CSRPullRequestBot`. The issue bot polls the issue tracker (JBS) for work, while the pull request bot polls the forge (github). The WorkItem part of the old CSRBot has been moved to the new `PullRequestWorkItem`, and except for now only handling one single PR per WorkItem instance (instead of everything in a repo), has been left pretty much intact. All actual updates on PRs are (still) handled by this WorkItem. The new `IssueWorkItem` is created when a change to a CSR Issue has been detected (in JBS). It walks the JBS issue and PR links to identify all PRs that could possibly be related to that CSR, and creates `PullRequestWorkItem`s for them. > > 2. To be able to find PRs from JBS issues, the CSRBot needs to know how to parse the PR comment links. To better share this knowledge, I moved the creation and deletion logic from the NotifyBot to PullRequestUtils (in the forge module), which is available to all bots that need it. I also added a method for parsing the URL from such a comment. > > 3. To efficiently query for updated CSR issues from Jira, I added two new query methods on IssueProject. A longer explanation of the new polling mechanism for issues can be found in the first bug comment. I've also tried to reasonably explain things in comments in the code. > > 4. In `JiraProject`, there was a pretty serious bug in all query methods for Issues. The `JiraIssue` constructor takes a `RestRequest` as a parameter. This request object is supposed to be used by `JiraIssue` for generating all the subqueries for the Jira REST API. The problem was that the query methods in `JiraProject` just sent in whatever `RestRequest` they used to find issues, which broke all methods that tried to make further calls through the returned `JiraIssue` to Jira. I fixed this by always supplying a `RestRequest` object with the correct URL when creating a JiraIssue. > > 5. To be able to call Jira with absolute timestamps in queries, we need to know the timezone of the user we are calling as. Luckily this information was available with a simple call, so I added this to the `JiraHost` class. > > 6. Similar to how care needs to be taken when polling for updates to issues in Jira, I applied a similar solution when polling PRs. This required a new method `HostedRepository.openPullRequestsAfter`. Unfortunately, it doesn't help much on Github, where the API doesn't support such a parameter, but it works for Gitlab. > > I have modified the tests that are affected by this change so that they still pass. This does verify that changes to issues in the IssueTracker are detected. Perhaps I should add some new tests to specifically verify the polling logic. I have verified the new queries and complete functionality manually by running the bot against the playground repo and bugs-stage. I don't really know whether it is a good idea to has both CSRIssueBot and CSRPullRequestBot. It seems that one CSRBot contains two work items, `IssueWorkItem` and `PullRequestWorkItem`, works well too. Or we can seperate the `CSRIssueBot` to a new module, like `IssueBot`, to be reused in the future. But it may exceed the range of this patch. bots/csr/src/main/java/org/openjdk/skara/bots/csr/CSRBotFactory.java line 33: > 31: import java.util.logging.Logger; > 32: import org.openjdk.skara.forge.HostedRepository; > 33: import org.openjdk.skara.issuetracker.IssueProject; The import statements can be polished. The same/similar package could be put nearly. bots/csr/src/main/java/org/openjdk/skara/bots/csr/CSRIssueBot.java line 1: > 1: package org.openjdk.skara.bots.csr; Need a copyright here. bots/csr/src/main/java/org/openjdk/skara/bots/csr/CSRIssueBot.java line 56: > 54: issueUpdatedAt.put(issue.id(), issue.updatedAt()); > 55: log.fine("Setting lastUpdatedAt from last updated issue " + issue.id() + " updated at " + lastUpdatedAt); > 56: } else { When setting the `lastUpdateAt` to the date of the last updated issue firstly, the method `issueProject.csrIssues` can not get the previous csr issues at the beginning. Is it a intent? bots/csr/src/main/java/org/openjdk/skara/bots/csr/CSRIssueBot.java line 73: > 71: if (issue.updatedAt().isAfter(lastUpdatedAt)) { > 72: lastUpdatedAt = issue.updatedAt(); > 73: } A race condition may let the bot ignore some updated issues. Considering the following situation: step 1: the method `issueProject.csrIssues` get the issue-1 and issue-2 at time 00:00:001 step 2: the issue-3 is updated at 00:00:002 and the issue-2 is updated at 00:00:003 step 3: at time 00:00:004, the statement `lastUpdatedAt = issue.updatedAt();` is run and updates the lastUpdatedAt to 00:00:003, then in the next round, the issue-3 will be ignored. bots/csr/src/main/java/org/openjdk/skara/bots/csr/CSRPullRequestBot.java line 76: > 74: if (lastUpdatedAt == null || pr.updatedAt().isAfter(lastUpdatedAt)) { > 75: lastUpdatedAt = pr.updatedAt(); > 76: } The logic is same as `CSRIssueBot`. The same race condition seems also exist. bots/csr/src/main/java/org/openjdk/skara/bots/csr/IssueWorkItem.java line 1: > 1: package org.openjdk.skara.bots.csr; Need a copyright here. bots/csr/src/main/java/org/openjdk/skara/bots/csr/IssueWorkItem.java line 51: > 49: .filter(l -> l.relationship().isPresent() && "csr of".equals(l.relationship().get())).findAny(); > 50: var issue = link.flatMap(Link::issue); > 51: var mainIssue = issue.flatMap(Backports::findMainIssue); These two `flatMap` methods seem can be put into the first stream statement because the variables `link` and `issue` never be used twice. bots/csr/src/main/java/org/openjdk/skara/bots/csr/PullRequestWorkItem.java line 1: > 1: package org.openjdk.skara.bots.csr; Need a copyright here. forge/src/test/java/org/openjdk/skara/forge/PullRequestUtilsTests.java line 1: > 1: package org.openjdk.skara.forge; Need a copyright here. ------------- Changes requested by gli (Committer). PR: https://git.openjdk.org/skara/pull/1357 From erikj at openjdk.org Mon Aug 15 16:58:26 2022 From: erikj at openjdk.org (Erik Joelsson) Date: Mon, 15 Aug 2022 16:58:26 GMT Subject: RFR: 1532: CSRBot is too inefficient In-Reply-To: References: <_2T-daSUsHi9kbLO4LlSGvmAmk2DmfarQGSj5rmgflc=.785e2a6d-5844-483a-9b3c-046547f82c40@github.com> Message-ID: On Mon, 15 Aug 2022 14:02:10 GMT, Guoxiong Li wrote: > I don't really know whether it is a good idea to has both CSRIssueBot and CSRPullRequestBot. It seems that one CSRBot contains two work items, `IssueWorkItem` and `PullRequestWorkItem`, works well too. The reason for having two separate bots is that the CSRPullRequestBot has one instance for each repo, while the CSRIssueBot has one instance for each IssueProject. There are generally multiple repos that share the same IssueProject. It would be possible to have one single CSRBot for each IssueProject and just loop over all the repos in it. I chose to split them to keep each "bot" more focused on polling one kind of thing. Compare with the existing `MailingListArchiveReaderBot` and `MailingListBridgeBot`. > Or we can seperate the `CSRIssueBot` to a new module, like `IssueBot`, to be reused in the future. But it may exceed the range of this patch. A separate module would be tricky as we need WorkItems that are specialized for CSR, but those wouldn't be accessible in the other module. I do agree that part of the implementation here would be nice to be able to reuse, but until we know how some other bot would like to reuse it, I would rather not complicate the current patch further. > bots/csr/src/main/java/org/openjdk/skara/bots/csr/CSRBotFactory.java line 33: > >> 31: import java.util.logging.Logger; >> 32: import org.openjdk.skara.forge.HostedRepository; >> 33: import org.openjdk.skara.issuetracker.IssueProject; > > The import statements can be polished. The same/similar package could be put nearly. That does look pretty bad. I never really care for import statements as the IDE handles it automatically, but I will see what can be done. ------------- PR: https://git.openjdk.org/skara/pull/1357 From erikj at openjdk.org Mon Aug 15 17:04:10 2022 From: erikj at openjdk.org (Erik Joelsson) Date: Mon, 15 Aug 2022 17:04:10 GMT Subject: RFR: 1532: CSRBot is too inefficient In-Reply-To: References: <_2T-daSUsHi9kbLO4LlSGvmAmk2DmfarQGSj5rmgflc=.785e2a6d-5844-483a-9b3c-046547f82c40@github.com> Message-ID: On Mon, 15 Aug 2022 13:38:00 GMT, Guoxiong Li wrote: >> This patch is a pretty major redesign of how the CSRBot polls for work. The motivation for this is described in the bug description. It's quite big, so I will try to break down the major changes. >> >> 1. The `CSRBot` has been split into `CSRIssueBot` and `CSRPullRequestBot`. The issue bot polls the issue tracker (JBS) for work, while the pull request bot polls the forge (github). The WorkItem part of the old CSRBot has been moved to the new `PullRequestWorkItem`, and except for now only handling one single PR per WorkItem instance (instead of everything in a repo), has been left pretty much intact. All actual updates on PRs are (still) handled by this WorkItem. The new `IssueWorkItem` is created when a change to a CSR Issue has been detected (in JBS). It walks the JBS issue and PR links to identify all PRs that could possibly be related to that CSR, and creates `PullRequestWorkItem`s for them. >> >> 2. To be able to find PRs from JBS issues, the CSRBot needs to know how to parse the PR comment links. To better share this knowledge, I moved the creation and deletion logic from the NotifyBot to PullRequestUtils (in the forge module), which is available to all bots that need it. I also added a method for parsing the URL from such a comment. >> >> 3. To efficiently query for updated CSR issues from Jira, I added two new query methods on IssueProject. A longer explanation of the new polling mechanism for issues can be found in the first bug comment. I've also tried to reasonably explain things in comments in the code. >> >> 4. In `JiraProject`, there was a pretty serious bug in all query methods for Issues. The `JiraIssue` constructor takes a `RestRequest` as a parameter. This request object is supposed to be used by `JiraIssue` for generating all the subqueries for the Jira REST API. The problem was that the query methods in `JiraProject` just sent in whatever `RestRequest` they used to find issues, which broke all methods that tried to make further calls through the returned `JiraIssue` to Jira. I fixed this by always supplying a `RestRequest` object with the correct URL when creating a JiraIssue. >> >> 5. To be able to call Jira with absolute timestamps in queries, we need to know the timezone of the user we are calling as. Luckily this information was available with a simple call, so I added this to the `JiraHost` class. >> >> 6. Similar to how care needs to be taken when polling for updates to issues in Jira, I applied a similar solution when polling PRs. This required a new method `HostedRepository.openPullRequestsAfter`. Unfortunately, it doesn't help much on Github, where the API doesn't support such a parameter, but it works for Gitlab. >> >> I have modified the tests that are affected by this change so that they still pass. This does verify that changes to issues in the IssueTracker are detected. Perhaps I should add some new tests to specifically verify the polling logic. I have verified the new queries and complete functionality manually by running the bot against the playground repo and bugs-stage. > > bots/csr/src/main/java/org/openjdk/skara/bots/csr/CSRIssueBot.java line 73: > >> 71: if (issue.updatedAt().isAfter(lastUpdatedAt)) { >> 72: lastUpdatedAt = issue.updatedAt(); >> 73: } > > A race condition may let the bot ignore some updated issues. Considering the following situation: > > step 1: the method `issueProject.csrIssues` get the issue-1 and issue-2 at time 00:00:001 > step 2: the issue-3 is updated at 00:00:002 and the issue-2 is updated at 00:00:003 > step 3: at time 00:00:004, the statement `lastUpdatedAt = issue.updatedAt();` is run and updates the lastUpdatedAt to 00:00:003, then in the next round, the issue-3 will be ignored. Your scenario should not be possible. If Issue-2 is updated at 00:00:003, then that update will not be included in the call in step 1, so issue-2.updatedAt() will not return 00:00:003, it will return an older value, as it was recorded at 00:00:001. This is why I only use the issue.updatedAt() values for `lastUpdatedAt`, so that I can be sure that the timestamps originate from the remote service (Jira) and can never be newer/later than when the data was gathered. ------------- PR: https://git.openjdk.org/skara/pull/1357 From erikj at openjdk.org Mon Aug 15 17:11:51 2022 From: erikj at openjdk.org (Erik Joelsson) Date: Mon, 15 Aug 2022 17:11:51 GMT Subject: RFR: 1532: CSRBot is too inefficient In-Reply-To: References: <_2T-daSUsHi9kbLO4LlSGvmAmk2DmfarQGSj5rmgflc=.785e2a6d-5844-483a-9b3c-046547f82c40@github.com> Message-ID: On Mon, 15 Aug 2022 13:42:05 GMT, Guoxiong Li wrote: >> This patch is a pretty major redesign of how the CSRBot polls for work. The motivation for this is described in the bug description. It's quite big, so I will try to break down the major changes. >> >> 1. The `CSRBot` has been split into `CSRIssueBot` and `CSRPullRequestBot`. The issue bot polls the issue tracker (JBS) for work, while the pull request bot polls the forge (github). The WorkItem part of the old CSRBot has been moved to the new `PullRequestWorkItem`, and except for now only handling one single PR per WorkItem instance (instead of everything in a repo), has been left pretty much intact. All actual updates on PRs are (still) handled by this WorkItem. The new `IssueWorkItem` is created when a change to a CSR Issue has been detected (in JBS). It walks the JBS issue and PR links to identify all PRs that could possibly be related to that CSR, and creates `PullRequestWorkItem`s for them. >> >> 2. To be able to find PRs from JBS issues, the CSRBot needs to know how to parse the PR comment links. To better share this knowledge, I moved the creation and deletion logic from the NotifyBot to PullRequestUtils (in the forge module), which is available to all bots that need it. I also added a method for parsing the URL from such a comment. >> >> 3. To efficiently query for updated CSR issues from Jira, I added two new query methods on IssueProject. A longer explanation of the new polling mechanism for issues can be found in the first bug comment. I've also tried to reasonably explain things in comments in the code. >> >> 4. In `JiraProject`, there was a pretty serious bug in all query methods for Issues. The `JiraIssue` constructor takes a `RestRequest` as a parameter. This request object is supposed to be used by `JiraIssue` for generating all the subqueries for the Jira REST API. The problem was that the query methods in `JiraProject` just sent in whatever `RestRequest` they used to find issues, which broke all methods that tried to make further calls through the returned `JiraIssue` to Jira. I fixed this by always supplying a `RestRequest` object with the correct URL when creating a JiraIssue. >> >> 5. To be able to call Jira with absolute timestamps in queries, we need to know the timezone of the user we are calling as. Luckily this information was available with a simple call, so I added this to the `JiraHost` class. >> >> 6. Similar to how care needs to be taken when polling for updates to issues in Jira, I applied a similar solution when polling PRs. This required a new method `HostedRepository.openPullRequestsAfter`. Unfortunately, it doesn't help much on Github, where the API doesn't support such a parameter, but it works for Gitlab. >> >> I have modified the tests that are affected by this change so that they still pass. This does verify that changes to issues in the IssueTracker are detected. Perhaps I should add some new tests to specifically verify the polling logic. I have verified the new queries and complete functionality manually by running the bot against the playground repo and bugs-stage. > > bots/csr/src/main/java/org/openjdk/skara/bots/csr/CSRIssueBot.java line 56: > >> 54: issueUpdatedAt.put(issue.id(), issue.updatedAt()); >> 55: log.fine("Setting lastUpdatedAt from last updated issue " + issue.id() + " updated at " + lastUpdatedAt); >> 56: } else { > > When setting the `lastUpdateAt` to the date of the last updated issue firstly, the method `issueProject.csrIssues` can not get the previous csr issues at the beginning. Is it a intent? Yes, this is the intent. When the bots start up, the CSRPullRequestBot will always go through every open PR and update the CSR state, so any updates in CSR issues will be found. The CSRIssueBot only needs to react to CSR issue updates going forward. I will try to clarify this in a comment. > bots/csr/src/main/java/org/openjdk/skara/bots/csr/IssueWorkItem.java line 51: > >> 49: .filter(l -> l.relationship().isPresent() && "csr of".equals(l.relationship().get())).findAny(); >> 50: var issue = link.flatMap(Link::issue); >> 51: var mainIssue = issue.flatMap(Backports::findMainIssue); > > These two `flatMap` methods seem can be put into the first stream statement because the variables `link` and `issue` never be used twice. That is true. I believe I left them out to more easily set breakpoints to verify state ------------- PR: https://git.openjdk.org/skara/pull/1357 From erikj at openjdk.org Mon Aug 15 17:17:50 2022 From: erikj at openjdk.org (Erik Joelsson) Date: Mon, 15 Aug 2022 17:17:50 GMT Subject: RFR: 1532: CSRBot is too inefficient In-Reply-To: References: <_2T-daSUsHi9kbLO4LlSGvmAmk2DmfarQGSj5rmgflc=.785e2a6d-5844-483a-9b3c-046547f82c40@github.com> Message-ID: On Mon, 15 Aug 2022 16:54:13 GMT, Erik Joelsson wrote: > > I don't really know whether it is a good idea to has both CSRIssueBot and CSRPullRequestBot. It seems that one CSRBot contains two work items, `IssueWorkItem` and `PullRequestWorkItem`, works well too. > > The reason for having two separate bots is that the CSRPullRequestBot has one instance for each repo, while the CSRIssueBot has one instance for each IssueProject. There are generally multiple repos that share the same IssueProject. It would be possible to have one single CSRBot for each IssueProject and just loop over all the repos in it. I chose to split them to keep each "bot" more focused on polling one kind of thing. Compare with the existing `MailingListArchiveReaderBot` and `MailingListBridgeBot`. > > > Or we can seperate the `CSRIssueBot` to a new module, like `IssueBot`, to be reused in the future. But it may exceed the range of this patch. > > A separate module would be tricky as we need WorkItems that are specialized for CSR, but those wouldn't be accessible in the other module. I do agree that part of the implementation here would be nice to be able to reuse, but until we know how some other bot would like to reuse it, I would rather not complicate the current patch further. Also, thank you for taking the time to review this! ------------- PR: https://git.openjdk.org/skara/pull/1357 From erikj at openjdk.org Mon Aug 15 17:23:34 2022 From: erikj at openjdk.org (Erik Joelsson) Date: Mon, 15 Aug 2022 17:23:34 GMT Subject: RFR: 1532: CSRBot is too inefficient [v2] In-Reply-To: <_2T-daSUsHi9kbLO4LlSGvmAmk2DmfarQGSj5rmgflc=.785e2a6d-5844-483a-9b3c-046547f82c40@github.com> References: <_2T-daSUsHi9kbLO4LlSGvmAmk2DmfarQGSj5rmgflc=.785e2a6d-5844-483a-9b3c-046547f82c40@github.com> Message-ID: > This patch is a pretty major redesign of how the CSRBot polls for work. The motivation for this is described in the bug description. It's quite big, so I will try to break down the major changes. > > 1. The `CSRBot` has been split into `CSRIssueBot` and `CSRPullRequestBot`. The issue bot polls the issue tracker (JBS) for work, while the pull request bot polls the forge (github). The WorkItem part of the old CSRBot has been moved to the new `PullRequestWorkItem`, and except for now only handling one single PR per WorkItem instance (instead of everything in a repo), has been left pretty much intact. All actual updates on PRs are (still) handled by this WorkItem. The new `IssueWorkItem` is created when a change to a CSR Issue has been detected (in JBS). It walks the JBS issue and PR links to identify all PRs that could possibly be related to that CSR, and creates `PullRequestWorkItem`s for them. > > 2. To be able to find PRs from JBS issues, the CSRBot needs to know how to parse the PR comment links. To better share this knowledge, I moved the creation and deletion logic from the NotifyBot to PullRequestUtils (in the forge module), which is available to all bots that need it. I also added a method for parsing the URL from such a comment. > > 3. To efficiently query for updated CSR issues from Jira, I added two new query methods on IssueProject. A longer explanation of the new polling mechanism for issues can be found in the first bug comment. I've also tried to reasonably explain things in comments in the code. > > 4. In `JiraProject`, there was a pretty serious bug in all query methods for Issues. The `JiraIssue` constructor takes a `RestRequest` as a parameter. This request object is supposed to be used by `JiraIssue` for generating all the subqueries for the Jira REST API. The problem was that the query methods in `JiraProject` just sent in whatever `RestRequest` they used to find issues, which broke all methods that tried to make further calls through the returned `JiraIssue` to Jira. I fixed this by always supplying a `RestRequest` object with the correct URL when creating a JiraIssue. > > 5. To be able to call Jira with absolute timestamps in queries, we need to know the timezone of the user we are calling as. Luckily this information was available with a simple call, so I added this to the `JiraHost` class. > > 6. Similar to how care needs to be taken when polling for updates to issues in Jira, I applied a similar solution when polling PRs. This required a new method `HostedRepository.openPullRequestsAfter`. Unfortunately, it doesn't help much on Github, where the API doesn't support such a parameter, but it works for Gitlab. > > I have modified the tests that are affected by this change so that they still pass. This does verify that changes to issues in the IssueTracker are detected. Perhaps I should add some new tests to specifically verify the polling logic. I have verified the new queries and complete functionality manually by running the bot against the playground repo and bugs-stage. Erik Joelsson has updated the pull request incrementally with one additional commit since the last revision: Added missing copyrights and expanded comment ------------- Changes: - all: https://git.openjdk.org/skara/pull/1357/files - new: https://git.openjdk.org/skara/pull/1357/files/5f2f96b8..4dbccc45 Webrevs: - full: https://webrevs.openjdk.org/?repo=skara&pr=1357&range=01 - incr: https://webrevs.openjdk.org/?repo=skara&pr=1357&range=00-01 Stats: 91 lines in 4 files changed: 90 ins; 0 del; 1 mod Patch: https://git.openjdk.org/skara/pull/1357.diff Fetch: git fetch https://git.openjdk.org/skara pull/1357/head:pull/1357 PR: https://git.openjdk.org/skara/pull/1357 From erikj at openjdk.org Mon Aug 15 17:23:34 2022 From: erikj at openjdk.org (Erik Joelsson) Date: Mon, 15 Aug 2022 17:23:34 GMT Subject: RFR: 1532: CSRBot is too inefficient [v2] In-Reply-To: References: <_2T-daSUsHi9kbLO4LlSGvmAmk2DmfarQGSj5rmgflc=.785e2a6d-5844-483a-9b3c-046547f82c40@github.com> Message-ID: On Mon, 15 Aug 2022 17:00:53 GMT, Erik Joelsson wrote: >> bots/csr/src/main/java/org/openjdk/skara/bots/csr/CSRIssueBot.java line 73: >> >>> 71: if (issue.updatedAt().isAfter(lastUpdatedAt)) { >>> 72: lastUpdatedAt = issue.updatedAt(); >>> 73: } >> >> A race condition may let the bot ignore some updated issues. Considering the following situation: >> >> step 1: the method `issueProject.csrIssues` get the issue-1 and issue-2 at time 00:00:001 >> step 2: the issue-3 is updated at 00:00:002 and the issue-2 is updated at 00:00:003 >> step 3: at time 00:00:004, the statement `lastUpdatedAt = issue.updatedAt();` is run and updates the lastUpdatedAt to 00:00:003, then in the next round, the issue-3 will be ignored. > > Your scenario should not be possible. If Issue-2 is updated at 00:00:003, then that update will not be included in the call in step 1, so issue-2.updatedAt() will not return 00:00:003, it will return an older value, as it was recorded at 00:00:001. This is why I only use the issue.updatedAt() values for `lastUpdatedAt`, so that I can be sure that the timestamps originate from the remote service (Jira) and can never be newer/later than when the data was gathered. In case you missed it, `JiraIssue.updatedAt()` does not call out to the remote service, it just reads the cached json. ------------- PR: https://git.openjdk.org/skara/pull/1357 From erikj at openjdk.org Mon Aug 15 20:19:24 2022 From: erikj at openjdk.org (Erik Joelsson) Date: Mon, 15 Aug 2022 20:19:24 GMT Subject: RFR: 1355: Mirror bot needs branch selection based on patterns Message-ID: <3DbGyKFljJIQJztWlg4YHsKX01wEIlLgfgEoulBO-9E=.c5a900c4-9670-4eaa-a246-3693486da960@github.com> This patch makes it possible to configure mirrored branches with regex patterns. For what I think is improved readability, I made it possible to supply the patterns as an array of strings, so we can have multiple separate patterns. It's still possible to define "branches" as a comma separated list, for backwards compatibility. When deploying this change, I will need to delete the existing local repos used for mirroring on the bot runner machine, because the method by which repos are cloned is changing (for repos that do not just mirror "everything"). Going forward, I think it's good to always clone all repos the same way, regardless of configuration. There is another behavioral change, where I'm adding `--force` when pushing branches with `includeTags`. It happens a bit too often that a repo maintainer changes tags, which currently causes error for any mirroring job because we aren't force pushing tags. I don't think we ever want to not force update when mirroring repos. ------------- Commit messages: - SKARA-1355 Changes: https://git.openjdk.org/skara/pull/1358/files Webrev: https://webrevs.openjdk.org/?repo=skara&pr=1358&range=00 Issue: https://bugs.openjdk.org/browse/SKARA-1355 Stats: 116 lines in 4 files changed: 88 ins; 7 del; 21 mod Patch: https://git.openjdk.org/skara/pull/1358.diff Fetch: git fetch https://git.openjdk.org/skara pull/1358/head:pull/1358 PR: https://git.openjdk.org/skara/pull/1358 From kcr at openjdk.org Tue Aug 16 14:19:14 2022 From: kcr at openjdk.org (Kevin Rushforth) Date: Tue, 16 Aug 2022 14:19:14 GMT Subject: RFR: 1355: Mirror bot needs branch selection based on patterns In-Reply-To: <3DbGyKFljJIQJztWlg4YHsKX01wEIlLgfgEoulBO-9E=.c5a900c4-9670-4eaa-a246-3693486da960@github.com> References: <3DbGyKFljJIQJztWlg4YHsKX01wEIlLgfgEoulBO-9E=.c5a900c4-9670-4eaa-a246-3693486da960@github.com> Message-ID: <9w1EqRHl007G41o-WSis3iYJhRDGeZ_HtoN3EUqqO9c=.4ba959bd-8885-4a97-9f62-3982b7986aba@github.com> On Mon, 15 Aug 2022 20:14:49 GMT, Erik Joelsson wrote: > This patch makes it possible to configure mirrored branches with regex patterns. For what I think is improved readability, I made it possible to supply the patterns as an array of strings, so we can have multiple separate patterns. It's still possible to define "branches" as a comma separated list, for backwards compatibility. > > When deploying this change, I will need to delete the existing local repos used for mirroring on the bot runner machine, because the method by which repos are cloned is changing (for repos that do not just mirror "everything"). Going forward, I think it's good to always clone all repos the same way, regardless of configuration. > > There is another behavioral change, where I'm adding `--force` when pushing branches with `includeTags`. It happens a bit too often that a repo maintainer changes tags, which currently causes error for any mirroring job because we aren't force pushing tags. I don't think we ever want to not force update when mirroring repos. Looks good with one question. vcs/src/main/java/org/openjdk/skara/vcs/git/GitRepository.java line 598: > 596: cmd.add("--tags"); > 597: if (force) { > 598: cmd.add("--force"); Is there a reason not to ignore the `force` flag when not pushing tags? It would seem more consistent to always look at the `force` flag and add `--force` is set. That way a repo configured not to push tags would still be able to handle a source branch that was force-pushed. ------------- PR: https://git.openjdk.org/skara/pull/1358 From erikj at openjdk.org Tue Aug 16 18:55:56 2022 From: erikj at openjdk.org (Erik Joelsson) Date: Tue, 16 Aug 2022 18:55:56 GMT Subject: RFR: 1355: Mirror bot needs branch selection based on patterns In-Reply-To: <9w1EqRHl007G41o-WSis3iYJhRDGeZ_HtoN3EUqqO9c=.4ba959bd-8885-4a97-9f62-3982b7986aba@github.com> References: <3DbGyKFljJIQJztWlg4YHsKX01wEIlLgfgEoulBO-9E=.c5a900c4-9670-4eaa-a246-3693486da960@github.com> <9w1EqRHl007G41o-WSis3iYJhRDGeZ_HtoN3EUqqO9c=.4ba959bd-8885-4a97-9f62-3982b7986aba@github.com> Message-ID: On Tue, 16 Aug 2022 14:13:15 GMT, Kevin Rushforth wrote: >> This patch makes it possible to configure mirrored branches with regex patterns. For what I think is improved readability, I made it possible to supply the patterns as an array of strings, so we can have multiple separate patterns. It's still possible to define "branches" as a comma separated list, for backwards compatibility. >> >> When deploying this change, I will need to delete the existing local repos used for mirroring on the bot runner machine, because the method by which repos are cloned is changing (for repos that do not just mirror "everything"). Going forward, I think it's good to always clone all repos the same way, regardless of configuration. >> >> There is another behavioral change, where I'm adding `--force` when pushing branches with `includeTags`. It happens a bit too often that a repo maintainer changes tags, which currently causes error for any mirroring job because we aren't force pushing tags. I don't think we ever want to not force update when mirroring repos. > > vcs/src/main/java/org/openjdk/skara/vcs/git/GitRepository.java line 598: > >> 596: cmd.add("--tags"); >> 597: if (force) { >> 598: cmd.add("--force"); > > Is there a reason not to ignore the `force` flag when not pushing tags? It would seem more consistent to always look at the `force` flag and add `--force` is set. That way a repo configured not to push tags would still be able to handle a source branch that was force-pushed. I was torn about this. I went this way to change as little current behavior as possible, mainly because I don't fully understand the nuances of git behavior here. Note that further down, `force==true` is already handled for branches by prepending `+` to the refs. On the other hand, the current proposal may be inconsistent, which is also bad long term. Perhaps the best solution would be to get rid of the `+` below and just add `--force` when `force==true`? ------------- PR: https://git.openjdk.org/skara/pull/1358 From kcr at openjdk.org Tue Aug 16 20:17:44 2022 From: kcr at openjdk.org (Kevin Rushforth) Date: Tue, 16 Aug 2022 20:17:44 GMT Subject: RFR: 1355: Mirror bot needs branch selection based on patterns In-Reply-To: <3DbGyKFljJIQJztWlg4YHsKX01wEIlLgfgEoulBO-9E=.c5a900c4-9670-4eaa-a246-3693486da960@github.com> References: <3DbGyKFljJIQJztWlg4YHsKX01wEIlLgfgEoulBO-9E=.c5a900c4-9670-4eaa-a246-3693486da960@github.com> Message-ID: On Mon, 15 Aug 2022 20:14:49 GMT, Erik Joelsson wrote: > This patch makes it possible to configure mirrored branches with regex patterns. For what I think is improved readability, I made it possible to supply the patterns as an array of strings, so we can have multiple separate patterns. It's still possible to define "branches" as a comma separated list, for backwards compatibility. > > When deploying this change, I will need to delete the existing local repos used for mirroring on the bot runner machine, because the method by which repos are cloned is changing (for repos that do not just mirror "everything"). Going forward, I think it's good to always clone all repos the same way, regardless of configuration. > > There is another behavioral change, where I'm adding `--force` when pushing branches with `includeTags`. It happens a bit too often that a repo maintainer changes tags, which currently causes error for any mirroring job because we aren't force pushing tags. I don't think we ever want to not force update when mirroring repos. Marked as reviewed by kcr (Reviewer). ------------- PR: https://git.openjdk.org/skara/pull/1358 From kcr at openjdk.org Tue Aug 16 20:17:45 2022 From: kcr at openjdk.org (Kevin Rushforth) Date: Tue, 16 Aug 2022 20:17:45 GMT Subject: RFR: 1355: Mirror bot needs branch selection based on patterns In-Reply-To: References: <3DbGyKFljJIQJztWlg4YHsKX01wEIlLgfgEoulBO-9E=.c5a900c4-9670-4eaa-a246-3693486da960@github.com> <9w1EqRHl007G41o-WSis3iYJhRDGeZ_HtoN3EUqqO9c=.4ba959bd-8885-4a97-9f62-3982b7986aba@github.com> Message-ID: On Tue, 16 Aug 2022 18:52:38 GMT, Erik Joelsson wrote: >> vcs/src/main/java/org/openjdk/skara/vcs/git/GitRepository.java line 598: >> >>> 596: cmd.add("--tags"); >>> 597: if (force) { >>> 598: cmd.add("--force"); >> >> Is there a reason not to ignore the `force` flag when not pushing tags? It would seem more consistent to always look at the `force` flag and add `--force` is set. That way a repo configured not to push tags would still be able to handle a source branch that was force-pushed. > > I was torn about this. I went this way to change as little current behavior as possible, mainly because I don't fully understand the nuances of git behavior here. Note that further down, `force==true` is already handled for branches by prepending `+` to the refs. On the other hand, the current proposal may be inconsistent, which is also bad long term. > > Perhaps the best solution would be to get rid of the `+` below and just add `--force` when `force==true`? Using `--force` consistently sounds cleanest, but if you want to limit the scope of this PR and do that in a follow-up PR, that would be good. ------------- PR: https://git.openjdk.org/skara/pull/1358 From gli at openjdk.org Wed Aug 17 06:24:32 2022 From: gli at openjdk.org (Guoxiong Li) Date: Wed, 17 Aug 2022 06:24:32 GMT Subject: RFR: 1532: CSRBot is too inefficient In-Reply-To: References: <_2T-daSUsHi9kbLO4LlSGvmAmk2DmfarQGSj5rmgflc=.785e2a6d-5844-483a-9b3c-046547f82c40@github.com> Message-ID: On Mon, 15 Aug 2022 17:14:56 GMT, Erik Joelsson wrote: > > I don't really know whether it is a good idea to has both CSRIssueBot and CSRPullRequestBot. It seems that one CSRBot contains two work items, `IssueWorkItem` and `PullRequestWorkItem`, works well too. > > The reason for having two separate bots is that the CSRPullRequestBot has one instance for each repo, while the CSRIssueBot has one instance for each IssueProject. There are generally multiple repos that share the same IssueProject. It would be possible to have one single CSRBot for each IssueProject and just loop over all the repos in it. I chose to split them to keep each "bot" more focused on polling one kind of thing. Compare with the existing `MailingListArchiveReaderBot` and `MailingListBridgeBot`. > > > Or we can seperate the `CSRIssueBot` to a new module, like `IssueBot`, to be reused in the future. But it may exceed the range of this patch. > > A separate module would be tricky as we need WorkItems that are specialized for CSR, but those wouldn't be accessible in the other module. I do agree that part of the implementation here would be nice to be able to reuse, but until we know how some other bot would like to reuse it, I would rather not complicate the current patch further. Looks good to keep these two bots. I agree that the issue bot may be considered to be reused in the future, but not in this patch. ------------- PR: https://git.openjdk.org/skara/pull/1357 From gli at openjdk.org Wed Aug 17 08:34:18 2022 From: gli at openjdk.org (Guoxiong Li) Date: Wed, 17 Aug 2022 08:34:18 GMT Subject: RFR: 1532: CSRBot is too inefficient [v2] In-Reply-To: References: <_2T-daSUsHi9kbLO4LlSGvmAmk2DmfarQGSj5rmgflc=.785e2a6d-5844-483a-9b3c-046547f82c40@github.com> Message-ID: On Mon, 15 Aug 2022 17:23:34 GMT, Erik Joelsson wrote: >> This patch is a pretty major redesign of how the CSRBot polls for work. The motivation for this is described in the bug description. It's quite big, so I will try to break down the major changes. >> >> 1. The `CSRBot` has been split into `CSRIssueBot` and `CSRPullRequestBot`. The issue bot polls the issue tracker (JBS) for work, while the pull request bot polls the forge (github). The WorkItem part of the old CSRBot has been moved to the new `PullRequestWorkItem`, and except for now only handling one single PR per WorkItem instance (instead of everything in a repo), has been left pretty much intact. All actual updates on PRs are (still) handled by this WorkItem. The new `IssueWorkItem` is created when a change to a CSR Issue has been detected (in JBS). It walks the JBS issue and PR links to identify all PRs that could possibly be related to that CSR, and creates `PullRequestWorkItem`s for them. >> >> 2. To be able to find PRs from JBS issues, the CSRBot needs to know how to parse the PR comment links. To better share this knowledge, I moved the creation and deletion logic from the NotifyBot to PullRequestUtils (in the forge module), which is available to all bots that need it. I also added a method for parsing the URL from such a comment. >> >> 3. To efficiently query for updated CSR issues from Jira, I added two new query methods on IssueProject. A longer explanation of the new polling mechanism for issues can be found in the first bug comment. I've also tried to reasonably explain things in comments in the code. >> >> 4. In `JiraProject`, there was a pretty serious bug in all query methods for Issues. The `JiraIssue` constructor takes a `RestRequest` as a parameter. This request object is supposed to be used by `JiraIssue` for generating all the subqueries for the Jira REST API. The problem was that the query methods in `JiraProject` just sent in whatever `RestRequest` they used to find issues, which broke all methods that tried to make further calls through the returned `JiraIssue` to Jira. I fixed this by always supplying a `RestRequest` object with the correct URL when creating a JiraIssue. >> >> 5. To be able to call Jira with absolute timestamps in queries, we need to know the timezone of the user we are calling as. Luckily this information was available with a simple call, so I added this to the `JiraHost` class. >> >> 6. Similar to how care needs to be taken when polling for updates to issues in Jira, I applied a similar solution when polling PRs. This required a new method `HostedRepository.openPullRequestsAfter`. Unfortunately, it doesn't help much on Github, where the API doesn't support such a parameter, but it works for Gitlab. >> >> I have modified the tests that are affected by this change so that they still pass. This does verify that changes to issues in the IssueTracker are detected. Perhaps I should add some new tests to specifically verify the polling logic. I have verified the new queries and complete functionality manually by running the bot against the playground repo and bugs-stage. > > Erik Joelsson has updated the pull request incrementally with one additional commit since the last revision: > > Added missing copyrights and expanded comment LGTM ------------- Marked as reviewed by gli (Committer). PR: https://git.openjdk.org/skara/pull/1357 From gli at openjdk.org Wed Aug 17 08:34:18 2022 From: gli at openjdk.org (Guoxiong Li) Date: Wed, 17 Aug 2022 08:34:18 GMT Subject: RFR: 1532: CSRBot is too inefficient [v2] In-Reply-To: References: <_2T-daSUsHi9kbLO4LlSGvmAmk2DmfarQGSj5rmgflc=.785e2a6d-5844-483a-9b3c-046547f82c40@github.com> Message-ID: On Mon, 15 Aug 2022 17:20:18 GMT, Erik Joelsson wrote: > In case you missed it, `JiraIssue.updatedAt()` does not call out to the remote service, it just reads the cached json. Good caught. ------------- PR: https://git.openjdk.org/skara/pull/1357 From ihse at openjdk.org Wed Aug 17 10:12:27 2022 From: ihse at openjdk.org (Magnus Ihse Bursie) Date: Wed, 17 Aug 2022 10:12:27 GMT Subject: RFR: 1355: Mirror bot needs branch selection based on patterns In-Reply-To: <3DbGyKFljJIQJztWlg4YHsKX01wEIlLgfgEoulBO-9E=.c5a900c4-9670-4eaa-a246-3693486da960@github.com> References: <3DbGyKFljJIQJztWlg4YHsKX01wEIlLgfgEoulBO-9E=.c5a900c4-9670-4eaa-a246-3693486da960@github.com> Message-ID: <2lkdRuYmOw8T0JbIQt9AWu4e4Bc_8XfwiefiUPMYcKQ=.c4be0d3e-222b-4650-af7b-2ed4b7ce44f0@github.com> On Mon, 15 Aug 2022 20:14:49 GMT, Erik Joelsson wrote: > This patch makes it possible to configure mirrored branches with regex patterns. For what I think is improved readability, I made it possible to supply the patterns as an array of strings, so we can have multiple separate patterns. It's still possible to define "branches" as a comma separated list, for backwards compatibility. > > When deploying this change, I will need to delete the existing local repos used for mirroring on the bot runner machine, because the method by which repos are cloned is changing (for repos that do not just mirror "everything"). Going forward, I think it's good to always clone all repos the same way, regardless of configuration. > > There is another behavioral change, where I'm adding `--force` when pushing branches with `includeTags`. It happens a bit too often that a repo maintainer changes tags, which currently causes error for any mirroring job because we aren't force pushing tags. I don't think we ever want to not force update when mirroring repos. LGTM ------------- Marked as reviewed by ihse (Reviewer). PR: https://git.openjdk.org/skara/pull/1358 From ihse at openjdk.org Thu Aug 18 11:23:07 2022 From: ihse at openjdk.org (Magnus Ihse Bursie) Date: Thu, 18 Aug 2022 11:23:07 GMT Subject: RFR: 1532: CSRBot is too inefficient [v2] In-Reply-To: References: <_2T-daSUsHi9kbLO4LlSGvmAmk2DmfarQGSj5rmgflc=.785e2a6d-5844-483a-9b3c-046547f82c40@github.com> Message-ID: <_wnAN6vD9mUD--XZjPmiPHNTn1wvideRcKe5edTKiuI=.7d1b1257-e7db-4042-b2ec-778311324d43@github.com> On Mon, 15 Aug 2022 17:23:34 GMT, Erik Joelsson wrote: >> This patch is a pretty major redesign of how the CSRBot polls for work. The motivation for this is described in the bug description. It's quite big, so I will try to break down the major changes. >> >> 1. The `CSRBot` has been split into `CSRIssueBot` and `CSRPullRequestBot`. The issue bot polls the issue tracker (JBS) for work, while the pull request bot polls the forge (github). The WorkItem part of the old CSRBot has been moved to the new `PullRequestWorkItem`, and except for now only handling one single PR per WorkItem instance (instead of everything in a repo), has been left pretty much intact. All actual updates on PRs are (still) handled by this WorkItem. The new `IssueWorkItem` is created when a change to a CSR Issue has been detected (in JBS). It walks the JBS issue and PR links to identify all PRs that could possibly be related to that CSR, and creates `PullRequestWorkItem`s for them. >> >> 2. To be able to find PRs from JBS issues, the CSRBot needs to know how to parse the PR comment links. To better share this knowledge, I moved the creation and deletion logic from the NotifyBot to PullRequestUtils (in the forge module), which is available to all bots that need it. I also added a method for parsing the URL from such a comment. >> >> 3. To efficiently query for updated CSR issues from Jira, I added two new query methods on IssueProject. A longer explanation of the new polling mechanism for issues can be found in the first bug comment. I've also tried to reasonably explain things in comments in the code. >> >> 4. In `JiraProject`, there was a pretty serious bug in all query methods for Issues. The `JiraIssue` constructor takes a `RestRequest` as a parameter. This request object is supposed to be used by `JiraIssue` for generating all the subqueries for the Jira REST API. The problem was that the query methods in `JiraProject` just sent in whatever `RestRequest` they used to find issues, which broke all methods that tried to make further calls through the returned `JiraIssue` to Jira. I fixed this by always supplying a `RestRequest` object with the correct URL when creating a JiraIssue. >> >> 5. To be able to call Jira with absolute timestamps in queries, we need to know the timezone of the user we are calling as. Luckily this information was available with a simple call, so I added this to the `JiraHost` class. >> >> 6. Similar to how care needs to be taken when polling for updates to issues in Jira, I applied a similar solution when polling PRs. This required a new method `HostedRepository.openPullRequestsAfter`. Unfortunately, it doesn't help much on Github, where the API doesn't support such a parameter, but it works for Gitlab. >> >> I have modified the tests that are affected by this change so that they still pass. This does verify that changes to issues in the IssueTracker are detected. Perhaps I should add some new tests to specifically verify the polling logic. I have verified the new queries and complete functionality manually by running the bot against the playground repo and bugs-stage. > > Erik Joelsson has updated the pull request incrementally with one additional commit since the last revision: > > Added missing copyrights and expanded comment bots/csr/src/main/java/org/openjdk/skara/bots/csr/PullRequestWorkItem.java line 48: > 46: private final static String CSR_LABEL = "csr"; > 47: private final static String CSR_UPDATE_MARKER = ""; > 48: private static final String PROGRESS_MARKER = ""; It's a bit unfortunate to duplicate string literals, especially those used as markers. Can you import this from `CheckRun.progressMarker` instead? bots/csr/src/test/java/org/openjdk/skara/bots/csr/CSRBotTests.java line 448: > 446: // Add the notification link to the PR in the issue. This is needed for the CSRIssueBot to > 447: // be able to trigger on CSR issue updates > 448: PullRequestUtils.postPullRequestLinkComment(issue, pr); Just a word of appreciation. It is great that you document so clearly all these testing steps. It makes all the difference in understanding what the test is trying to do, and why it should succeed. ------------- PR: https://git.openjdk.org/skara/pull/1357 From ihse at openjdk.org Thu Aug 18 11:27:46 2022 From: ihse at openjdk.org (Magnus Ihse Bursie) Date: Thu, 18 Aug 2022 11:27:46 GMT Subject: RFR: 1532: CSRBot is too inefficient [v2] In-Reply-To: References: <_2T-daSUsHi9kbLO4LlSGvmAmk2DmfarQGSj5rmgflc=.785e2a6d-5844-483a-9b3c-046547f82c40@github.com> Message-ID: On Mon, 15 Aug 2022 17:23:34 GMT, Erik Joelsson wrote: >> This patch is a pretty major redesign of how the CSRBot polls for work. The motivation for this is described in the bug description. It's quite big, so I will try to break down the major changes. >> >> 1. The `CSRBot` has been split into `CSRIssueBot` and `CSRPullRequestBot`. The issue bot polls the issue tracker (JBS) for work, while the pull request bot polls the forge (github). The WorkItem part of the old CSRBot has been moved to the new `PullRequestWorkItem`, and except for now only handling one single PR per WorkItem instance (instead of everything in a repo), has been left pretty much intact. All actual updates on PRs are (still) handled by this WorkItem. The new `IssueWorkItem` is created when a change to a CSR Issue has been detected (in JBS). It walks the JBS issue and PR links to identify all PRs that could possibly be related to that CSR, and creates `PullRequestWorkItem`s for them. >> >> 2. To be able to find PRs from JBS issues, the CSRBot needs to know how to parse the PR comment links. To better share this knowledge, I moved the creation and deletion logic from the NotifyBot to PullRequestUtils (in the forge module), which is available to all bots that need it. I also added a method for parsing the URL from such a comment. >> >> 3. To efficiently query for updated CSR issues from Jira, I added two new query methods on IssueProject. A longer explanation of the new polling mechanism for issues can be found in the first bug comment. I've also tried to reasonably explain things in comments in the code. >> >> 4. In `JiraProject`, there was a pretty serious bug in all query methods for Issues. The `JiraIssue` constructor takes a `RestRequest` as a parameter. This request object is supposed to be used by `JiraIssue` for generating all the subqueries for the Jira REST API. The problem was that the query methods in `JiraProject` just sent in whatever `RestRequest` they used to find issues, which broke all methods that tried to make further calls through the returned `JiraIssue` to Jira. I fixed this by always supplying a `RestRequest` object with the correct URL when creating a JiraIssue. >> >> 5. To be able to call Jira with absolute timestamps in queries, we need to know the timezone of the user we are calling as. Luckily this information was available with a simple call, so I added this to the `JiraHost` class. >> >> 6. Similar to how care needs to be taken when polling for updates to issues in Jira, I applied a similar solution when polling PRs. This required a new method `HostedRepository.openPullRequestsAfter`. Unfortunately, it doesn't help much on Github, where the API doesn't support such a parameter, but it works for Gitlab. >> >> I have modified the tests that are affected by this change so that they still pass. This does verify that changes to issues in the IssueTracker are detected. Perhaps I should add some new tests to specifically verify the polling logic. I have verified the new queries and complete functionality manually by running the bot against the playground repo and bugs-stage. > > Erik Joelsson has updated the pull request incrementally with one additional commit since the last revision: > > Added missing copyrights and expanded comment Overall it looks good. I think your general approach is sane to split the bots like this. Code reviewing code that's been moved around is always tricky; I made some samples to verify critical behavior but I'm mostly assuming that you have not really changed behavior that is not pertinent to this change. ------------- Marked as reviewed by ihse (Reviewer). PR: https://git.openjdk.org/skara/pull/1357 From erikj at openjdk.org Thu Aug 18 13:42:52 2022 From: erikj at openjdk.org (Erik Joelsson) Date: Thu, 18 Aug 2022 13:42:52 GMT Subject: RFR: 1532: CSRBot is too inefficient [v2] In-Reply-To: <_wnAN6vD9mUD--XZjPmiPHNTn1wvideRcKe5edTKiuI=.7d1b1257-e7db-4042-b2ec-778311324d43@github.com> References: <_2T-daSUsHi9kbLO4LlSGvmAmk2DmfarQGSj5rmgflc=.785e2a6d-5844-483a-9b3c-046547f82c40@github.com> <_wnAN6vD9mUD--XZjPmiPHNTn1wvideRcKe5edTKiuI=.7d1b1257-e7db-4042-b2ec-778311324d43@github.com> Message-ID: <1kVNIlv3ECZATH42EbzHqzQ7nwMwiV0KUrZIErQCNNM=.6ef3469c-630d-4ea1-8507-1c9dd5020208@github.com> On Thu, 18 Aug 2022 11:16:54 GMT, Magnus Ihse Bursie wrote: >> Erik Joelsson has updated the pull request incrementally with one additional commit since the last revision: >> >> Added missing copyrights and expanded comment > > bots/csr/src/main/java/org/openjdk/skara/bots/csr/PullRequestWorkItem.java line 48: > >> 46: private final static String CSR_LABEL = "csr"; >> 47: private final static String CSR_UPDATE_MARKER = ""; >> 48: private static final String PROGRESS_MARKER = ""; > > It's a bit unfortunate to duplicate string literals, especially those used as markers. Can you import this from `CheckRun.progressMarker` instead? I completely agree that duplicated strings are bad. Importing directly from another bot module wouldn't be great IMO. I will look into moving this into some common class in a shared module instead. Note though that this isn't new in this patch. ------------- PR: https://git.openjdk.org/skara/pull/1357 From erikj at openjdk.org Thu Aug 18 13:48:15 2022 From: erikj at openjdk.org (Erik Joelsson) Date: Thu, 18 Aug 2022 13:48:15 GMT Subject: RFR: 1532: CSRBot is too inefficient [v2] In-Reply-To: <1kVNIlv3ECZATH42EbzHqzQ7nwMwiV0KUrZIErQCNNM=.6ef3469c-630d-4ea1-8507-1c9dd5020208@github.com> References: <_2T-daSUsHi9kbLO4LlSGvmAmk2DmfarQGSj5rmgflc=.785e2a6d-5844-483a-9b3c-046547f82c40@github.com> <_wnAN6vD9mUD--XZjPmiPHNTn1wvideRcKe5edTKiuI=.7d1b1257-e7db-4042-b2ec-778311324d43@github.com> <1kVNIlv3ECZATH42EbzHqzQ7nwMwiV0KUrZIErQCNNM=.6ef3469c-630d-4ea1-8507-1c9dd5020208@github.com> Message-ID: On Thu, 18 Aug 2022 13:39:42 GMT, Erik Joelsson wrote: >> bots/csr/src/main/java/org/openjdk/skara/bots/csr/PullRequestWorkItem.java line 48: >> >>> 46: private final static String CSR_LABEL = "csr"; >>> 47: private final static String CSR_UPDATE_MARKER = ""; >>> 48: private static final String PROGRESS_MARKER = ""; >> >> It's a bit unfortunate to duplicate string literals, especially those used as markers. Can you import this from `CheckRun.progressMarker` instead? > > I completely agree that duplicated strings are bad. Importing directly from another bot module wouldn't be great IMO. I will look into moving this into some common class in a shared module instead. Note though that this isn't new in this patch. Actually, thinking on this some more, can I file a followup to attack constants shared between bots in a more uniform way? I think many of these should be shared knowledge between bots, including label names, so adding a PullRequestContants class in the forge module (next to PullRequestUtils) would be a good idea I think. I don't want to pollute this change with this though. ------------- PR: https://git.openjdk.org/skara/pull/1357 From erikj at openjdk.org Thu Aug 18 13:52:29 2022 From: erikj at openjdk.org (Erik Joelsson) Date: Thu, 18 Aug 2022 13:52:29 GMT Subject: RFR: 1532: CSRBot is too inefficient [v2] In-Reply-To: References: <_2T-daSUsHi9kbLO4LlSGvmAmk2DmfarQGSj5rmgflc=.785e2a6d-5844-483a-9b3c-046547f82c40@github.com> <_wnAN6vD9mUD--XZjPmiPHNTn1wvideRcKe5edTKiuI=.7d1b1257-e7db-4042-b2ec-778311324d43@github.com> <1kVNIlv3ECZATH42EbzHqzQ7nwMwiV0KUrZIErQCNNM=.6ef3469c-630d-4ea1-8507-1c9dd5020208@github.com> Message-ID: On Thu, 18 Aug 2022 13:45:00 GMT, Erik Joelsson wrote: >> I completely agree that duplicated strings are bad. Importing directly from another bot module wouldn't be great IMO. I will look into moving this into some common class in a shared module instead. Note though that this isn't new in this patch. > > Actually, thinking on this some more, can I file a followup to attack constants shared between bots in a more uniform way? I think many of these should be shared knowledge between bots, including label names, so adding a PullRequestContants class in the forge module (next to PullRequestUtils) would be a good idea I think. I don't want to pollute this change with this though. Filed https://bugs.openjdk.org/browse/SKARA-1547 ------------- PR: https://git.openjdk.org/skara/pull/1357 From erikj at openjdk.org Thu Aug 18 18:15:28 2022 From: erikj at openjdk.org (Erik Joelsson) Date: Thu, 18 Aug 2022 18:15:28 GMT Subject: Integrated: 1355: Mirror bot needs branch selection based on patterns In-Reply-To: <3DbGyKFljJIQJztWlg4YHsKX01wEIlLgfgEoulBO-9E=.c5a900c4-9670-4eaa-a246-3693486da960@github.com> References: <3DbGyKFljJIQJztWlg4YHsKX01wEIlLgfgEoulBO-9E=.c5a900c4-9670-4eaa-a246-3693486da960@github.com> Message-ID: On Mon, 15 Aug 2022 20:14:49 GMT, Erik Joelsson wrote: > This patch makes it possible to configure mirrored branches with regex patterns. For what I think is improved readability, I made it possible to supply the patterns as an array of strings, so we can have multiple separate patterns. It's still possible to define "branches" as a comma separated list, for backwards compatibility. > > When deploying this change, I will need to delete the existing local repos used for mirroring on the bot runner machine, because the method by which repos are cloned is changing (for repos that do not just mirror "everything"). Going forward, I think it's good to always clone all repos the same way, regardless of configuration. > > There is another behavioral change, where I'm adding `--force` when pushing branches with `includeTags`. It happens a bit too often that a repo maintainer changes tags, which currently causes error for any mirroring job because we aren't force pushing tags. I don't think we ever want to not force update when mirroring repos. This pull request has now been integrated. Changeset: f1227d0c Author: Erik Joelsson URL: https://git.openjdk.org/skara/commit/f1227d0cbb6ed47b6e6d0c411697fd7b008e283e Stats: 116 lines in 4 files changed: 88 ins; 7 del; 21 mod 1355: Mirror bot needs branch selection based on patterns Reviewed-by: kcr, ihse ------------- PR: https://git.openjdk.org/skara/pull/1358 From magnus.ihse.bursie at oracle.com Fri Aug 19 11:07:34 2022 From: magnus.ihse.bursie at oracle.com (Magnus Ihse Bursie) Date: Fri, 19 Aug 2022 13:07:34 +0200 Subject: RFR: 1355: Mirror bot needs branch selection based on patterns In-Reply-To: <3DbGyKFljJIQJztWlg4YHsKX01wEIlLgfgEoulBO-9E=.c5a900c4-9670-4eaa-a246-3693486da960@github.com> References: <3DbGyKFljJIQJztWlg4YHsKX01wEIlLgfgEoulBO-9E=.c5a900c4-9670-4eaa-a246-3693486da960@github.com> Message-ID: There never was a "Integrated: ..." mail about this, even though it was pushed 17 hours ago. Are the bots stuck? /Magnus On 2022-08-15 22:19, Erik Joelsson wrote: > This patch makes it possible to configure mirrored branches with regex patterns. For what I think is improved readability, I made it possible to supply the patterns as an array of strings, so we can have multiple separate patterns. It's still possible to define "branches" as a comma separated list, for backwards compatibility. > > When deploying this change, I will need to delete the existing local repos used for mirroring on the bot runner machine, because the method by which repos are cloned is changing (for repos that do not just mirror "everything"). Going forward, I think it's good to always clone all repos the same way, regardless of configuration. > > There is another behavioral change, where I'm adding `--force` when pushing branches with `includeTags`. It happens a bit too often that a repo maintainer changes tags, which currently causes error for any mirroring job because we aren't force pushing tags. I don't think we ever want to not force update when mirroring repos. > > ------------- > > Commit messages: > - SKARA-1355 > > Changes: https://git.openjdk.org/skara/pull/1358/files > Webrev: https://webrevs.openjdk.org/?repo=skara&pr=1358&range=00 > Issue: https://bugs.openjdk.org/browse/SKARA-1355 > Stats: 116 lines in 4 files changed: 88 ins; 7 del; 21 mod > Patch: https://git.openjdk.org/skara/pull/1358.diff > Fetch: git fetch https://git.openjdk.org/skara pull/1358/head:pull/1358 > > PR: https://git.openjdk.org/skara/pull/1358 From ihse at openjdk.org Fri Aug 19 11:09:12 2022 From: ihse at openjdk.org (Magnus Ihse Bursie) Date: Fri, 19 Aug 2022 11:09:12 GMT Subject: RFR: 1532: CSRBot is too inefficient [v2] In-Reply-To: References: <_2T-daSUsHi9kbLO4LlSGvmAmk2DmfarQGSj5rmgflc=.785e2a6d-5844-483a-9b3c-046547f82c40@github.com> <_wnAN6vD9mUD--XZjPmiPHNTn1wvideRcKe5edTKiuI=.7d1b1257-e7db-4042-b2ec-778311324d43@github.com> <1kVNIlv3ECZATH42EbzHqzQ7nwMwiV0KUrZIErQCNNM=.6ef3469c-630d-4ea1-8507-1c9dd5020208@github.com> Message-ID: On Thu, 18 Aug 2022 13:49:57 GMT, Erik Joelsson wrote: >> Actually, thinking on this some more, can I file a followup to attack constants shared between bots in a more uniform way? I think many of these should be shared knowledge between bots, including label names, so adding a PullRequestContants class in the forge module (next to PullRequestUtils) would be a good idea I think. I don't want to pollute this change with this though. > > Filed https://bugs.openjdk.org/browse/SKARA-1547 I saw that this was not new. Agree that it's better to fix in a separate PR. ------------- PR: https://git.openjdk.org/skara/pull/1357 From erik.joelsson at oracle.com Fri Aug 19 13:59:49 2022 From: erik.joelsson at oracle.com (erik.joelsson at oracle.com) Date: Fri, 19 Aug 2022 06:59:49 -0700 Subject: RFR: 1355: Mirror bot needs branch selection based on patterns In-Reply-To: References: <3DbGyKFljJIQJztWlg4YHsKX01wEIlLgfgEoulBO-9E=.c5a900c4-9670-4eaa-a246-3693486da960@github.com> Message-ID: I didn't receive it either, but it's in the mail archive: https://mail.openjdk.org/pipermail/skara-dev/2022-August/006327.html /Erik On 8/19/22 4:07 AM, Magnus Ihse Bursie wrote: > There never was a "Integrated: ..." mail about this, even though it > was pushed 17 hours ago. Are the bots stuck? > > /Magnus > > On 2022-08-15 22:19, Erik Joelsson wrote: >> This patch makes it possible to configure mirrored branches with >> regex patterns. For what I think is improved readability, I made it >> possible to supply the patterns as an array of strings, so we can >> have multiple separate patterns. It's still possible to define >> "branches" as a comma separated list, for backwards compatibility. >> >> When deploying this change, I will need to delete the existing local >> repos used for mirroring on the bot runner machine, because the >> method by which repos are cloned is changing (for repos that do not >> just mirror "everything"). Going forward, I think it's good to always >> clone all repos the same way, regardless of configuration. >> >> There is another behavioral change, where I'm adding `--force` when >> pushing branches with `includeTags`. It happens a bit too often that >> a repo maintainer changes tags, which currently causes error for any >> mirroring job because we aren't force pushing tags. I don't think we >> ever want to not force update when mirroring repos. >> >> ------------- >> >> Commit messages: >> ? - SKARA-1355 >> >> Changes: https://git.openjdk.org/skara/pull/1358/files >> ? Webrev: https://webrevs.openjdk.org/?repo=skara&pr=1358&range=00 >> ?? Issue: https://bugs.openjdk.org/browse/SKARA-1355 >> ?? Stats: 116 lines in 4 files changed: 88 ins; 7 del; 21 mod >> ?? Patch: https://git.openjdk.org/skara/pull/1358.diff >> ?? Fetch: git fetch https://git.openjdk.org/skara >> pull/1358/head:pull/1358 >> >> PR: https://git.openjdk.org/skara/pull/1358 From magnus.ihse.bursie at oracle.com Fri Aug 19 14:42:33 2022 From: magnus.ihse.bursie at oracle.com (Magnus Ihse Bursie) Date: Fri, 19 Aug 2022 16:42:33 +0200 Subject: RFR: 1355: Mirror bot needs branch selection based on patterns In-Reply-To: References: <3DbGyKFljJIQJztWlg4YHsKX01wEIlLgfgEoulBO-9E=.c5a900c4-9670-4eaa-a246-3693486da960@github.com> Message-ID: <1f8a962f-8569-885e-52ab-550513282d60@oracle.com> On 2022-08-19 15:59, erik.joelsson at oracle.com wrote: > I didn't receive it either, but it's in the mail archive: > https://mail.openjdk.org/pipermail/skara-dev/2022-August/006327.html Good finding. Then it is a problem with the Oracle mail servers, rather than Skara. It is much better that way. =) /Magnus > > /Erik > > On 8/19/22 4:07 AM, Magnus Ihse Bursie wrote: >> There never was a "Integrated: ..." mail about this, even though it >> was pushed 17 hours ago. Are the bots stuck? >> >> /Magnus >> >> On 2022-08-15 22:19, Erik Joelsson wrote: >>> This patch makes it possible to configure mirrored branches with >>> regex patterns. For what I think is improved readability, I made it >>> possible to supply the patterns as an array of strings, so we can >>> have multiple separate patterns. It's still possible to define >>> "branches" as a comma separated list, for backwards compatibility. >>> >>> When deploying this change, I will need to delete the existing local >>> repos used for mirroring on the bot runner machine, because the >>> method by which repos are cloned is changing (for repos that do not >>> just mirror "everything"). Going forward, I think it's good to >>> always clone all repos the same way, regardless of configuration. >>> >>> There is another behavioral change, where I'm adding `--force` when >>> pushing branches with `includeTags`. It happens a bit too often that >>> a repo maintainer changes tags, which currently causes error for any >>> mirroring job because we aren't force pushing tags. I don't think we >>> ever want to not force update when mirroring repos. >>> >>> ------------- >>> >>> Commit messages: >>> ? - SKARA-1355 >>> >>> Changes: https://git.openjdk.org/skara/pull/1358/files >>> ? Webrev: https://webrevs.openjdk.org/?repo=skara&pr=1358&range=00 >>> ?? Issue: https://bugs.openjdk.org/browse/SKARA-1355 >>> ?? Stats: 116 lines in 4 files changed: 88 ins; 7 del; 21 mod >>> ?? Patch: https://git.openjdk.org/skara/pull/1358.diff >>> ?? Fetch: git fetch https://git.openjdk.org/skara >>> pull/1358/head:pull/1358 >>> >>> PR: https://git.openjdk.org/skara/pull/1358 From erikj at openjdk.org Fri Aug 19 16:55:26 2022 From: erikj at openjdk.org (Erik Joelsson) Date: Fri, 19 Aug 2022 16:55:26 GMT Subject: RFR: 1532: CSRBot is too inefficient [v2] In-Reply-To: References: <_2T-daSUsHi9kbLO4LlSGvmAmk2DmfarQGSj5rmgflc=.785e2a6d-5844-483a-9b3c-046547f82c40@github.com> Message-ID: On Mon, 15 Aug 2022 17:23:34 GMT, Erik Joelsson wrote: >> This patch is a pretty major redesign of how the CSRBot polls for work. The motivation for this is described in the bug description. It's quite big, so I will try to break down the major changes. >> >> 1. The `CSRBot` has been split into `CSRIssueBot` and `CSRPullRequestBot`. The issue bot polls the issue tracker (JBS) for work, while the pull request bot polls the forge (github). The WorkItem part of the old CSRBot has been moved to the new `PullRequestWorkItem`, and except for now only handling one single PR per WorkItem instance (instead of everything in a repo), has been left pretty much intact. All actual updates on PRs are (still) handled by this WorkItem. The new `IssueWorkItem` is created when a change to a CSR Issue has been detected (in JBS). It walks the JBS issue and PR links to identify all PRs that could possibly be related to that CSR, and creates `PullRequestWorkItem`s for them. >> >> 2. To be able to find PRs from JBS issues, the CSRBot needs to know how to parse the PR comment links. To better share this knowledge, I moved the creation and deletion logic from the NotifyBot to PullRequestUtils (in the forge module), which is available to all bots that need it. I also added a method for parsing the URL from such a comment. >> >> 3. To efficiently query for updated CSR issues from Jira, I added two new query methods on IssueProject. A longer explanation of the new polling mechanism for issues can be found in the first bug comment. I've also tried to reasonably explain things in comments in the code. >> >> 4. In `JiraProject`, there was a pretty serious bug in all query methods for Issues. The `JiraIssue` constructor takes a `RestRequest` as a parameter. This request object is supposed to be used by `JiraIssue` for generating all the subqueries for the Jira REST API. The problem was that the query methods in `JiraProject` just sent in whatever `RestRequest` they used to find issues, which broke all methods that tried to make further calls through the returned `JiraIssue` to Jira. I fixed this by always supplying a `RestRequest` object with the correct URL when creating a JiraIssue. >> >> 5. To be able to call Jira with absolute timestamps in queries, we need to know the timezone of the user we are calling as. Luckily this information was available with a simple call, so I added this to the `JiraHost` class. >> >> 6. Similar to how care needs to be taken when polling for updates to issues in Jira, I applied a similar solution when polling PRs. This required a new method `HostedRepository.openPullRequestsAfter`. Unfortunately, it doesn't help much on Github, where the API doesn't support such a parameter, but it works for Gitlab. >> >> I have modified the tests that are affected by this change so that they still pass. This does verify that changes to issues in the IssueTracker are detected. Perhaps I should add some new tests to specifically verify the polling logic. I have verified the new queries and complete functionality manually by running the bot against the playground repo and bugs-stage. > > Erik Joelsson has updated the pull request incrementally with one additional commit since the last revision: > > Added missing copyrights and expanded comment Thanks! ------------- PR: https://git.openjdk.org/skara/pull/1357 From erikj at openjdk.org Fri Aug 19 16:55:27 2022 From: erikj at openjdk.org (Erik Joelsson) Date: Fri, 19 Aug 2022 16:55:27 GMT Subject: Integrated: 1532: CSRBot is too inefficient In-Reply-To: <_2T-daSUsHi9kbLO4LlSGvmAmk2DmfarQGSj5rmgflc=.785e2a6d-5844-483a-9b3c-046547f82c40@github.com> References: <_2T-daSUsHi9kbLO4LlSGvmAmk2DmfarQGSj5rmgflc=.785e2a6d-5844-483a-9b3c-046547f82c40@github.com> Message-ID: On Fri, 12 Aug 2022 22:19:58 GMT, Erik Joelsson wrote: > This patch is a pretty major redesign of how the CSRBot polls for work. The motivation for this is described in the bug description. It's quite big, so I will try to break down the major changes. > > 1. The `CSRBot` has been split into `CSRIssueBot` and `CSRPullRequestBot`. The issue bot polls the issue tracker (JBS) for work, while the pull request bot polls the forge (github). The WorkItem part of the old CSRBot has been moved to the new `PullRequestWorkItem`, and except for now only handling one single PR per WorkItem instance (instead of everything in a repo), has been left pretty much intact. All actual updates on PRs are (still) handled by this WorkItem. The new `IssueWorkItem` is created when a change to a CSR Issue has been detected (in JBS). It walks the JBS issue and PR links to identify all PRs that could possibly be related to that CSR, and creates `PullRequestWorkItem`s for them. > > 2. To be able to find PRs from JBS issues, the CSRBot needs to know how to parse the PR comment links. To better share this knowledge, I moved the creation and deletion logic from the NotifyBot to PullRequestUtils (in the forge module), which is available to all bots that need it. I also added a method for parsing the URL from such a comment. > > 3. To efficiently query for updated CSR issues from Jira, I added two new query methods on IssueProject. A longer explanation of the new polling mechanism for issues can be found in the first bug comment. I've also tried to reasonably explain things in comments in the code. > > 4. In `JiraProject`, there was a pretty serious bug in all query methods for Issues. The `JiraIssue` constructor takes a `RestRequest` as a parameter. This request object is supposed to be used by `JiraIssue` for generating all the subqueries for the Jira REST API. The problem was that the query methods in `JiraProject` just sent in whatever `RestRequest` they used to find issues, which broke all methods that tried to make further calls through the returned `JiraIssue` to Jira. I fixed this by always supplying a `RestRequest` object with the correct URL when creating a JiraIssue. > > 5. To be able to call Jira with absolute timestamps in queries, we need to know the timezone of the user we are calling as. Luckily this information was available with a simple call, so I added this to the `JiraHost` class. > > 6. Similar to how care needs to be taken when polling for updates to issues in Jira, I applied a similar solution when polling PRs. This required a new method `HostedRepository.openPullRequestsAfter`. Unfortunately, it doesn't help much on Github, where the API doesn't support such a parameter, but it works for Gitlab. > > I have modified the tests that are affected by this change so that they still pass. This does verify that changes to issues in the IssueTracker are detected. Perhaps I should add some new tests to specifically verify the polling logic. I have verified the new queries and complete functionality manually by running the bot against the playground repo and bugs-stage. This pull request has now been integrated. Changeset: 972cbb68 Author: Erik Joelsson URL: https://git.openjdk.org/skara/commit/972cbb685e9406ec9d245fd201e4182f7a2c0a6f Stats: 1189 lines in 21 files changed: 888 ins; 256 del; 45 mod 1532: CSRBot is too inefficient Reviewed-by: gli, ihse ------------- PR: https://git.openjdk.org/skara/pull/1357 From erikj at openjdk.org Fri Aug 19 19:57:06 2022 From: erikj at openjdk.org (Erik Joelsson) Date: Fri, 19 Aug 2022 19:57:06 GMT Subject: RFR: 1551: Improve logging and metrics for BotRunner Message-ID: I'm trying to better understand bot performance. I'm sure there are lots of things we could improve performance wise, but first we need to figure out how. We already have some logging and metrics gathering, but it's still not providing me with the information I need. With this change, I'm aiming to gather better metrics and logging around the getPeriodicItems() call, as well as refining the metrics for running WorkItems. The thread user time measurements from MX beans is useless to me, so I'm removing it. The cpu time is probably also useless, but I'm keeping it for now. What I really want is wall clock time for WorkItems, so I'm adding that. The gauges for measuring any kind of running time aren't useful at all. At least with counters, we can get some kind of idea how much time is spent on each kind of WorkItem over some time frame (using a rate function in the presentation layer). To complement the timing counters, I'm adding simple WorkItem counters as well, to keep track how many are scheduled, added as pending, submitted and finished. For the BotRunner.checkPeriodicItems() method, I'm adding two metrics. Since the whole method is running only one at a time, a gauge for tracking how long each round takes does make sense. Then for each sub call to Bot.getPeriodicItems there is a counter that basically gives the breakdown on how much of the time is spent in each kind of bot. For logging, I'm adding some more log messages to better track which Bot.getPeriodic item is called when, and a LogContext, so that all log messages are tagged with the bot toString, just like WorkItems are already. To support this I had to add toString() methods to all Bot implementations that were missing it. ------------- Commit messages: - Rename counter - More counters - Adding log context for each getPeriodicItems() call and toString() to all Bots - More metrics Changes: https://git.openjdk.org/skara/pull/1359/files Webrev: https://webrevs.openjdk.org/?repo=skara&pr=1359&range=00 Issue: https://bugs.openjdk.org/browse/SKARA-1551 Stats: 117 lines in 12 files changed: 76 ins; 29 del; 12 mod Patch: https://git.openjdk.org/skara/pull/1359.diff Fetch: git fetch https://git.openjdk.org/skara pull/1359/head:pull/1359 PR: https://git.openjdk.org/skara/pull/1359 From erikj at openjdk.org Fri Aug 19 20:01:34 2022 From: erikj at openjdk.org (Erik Joelsson) Date: Fri, 19 Aug 2022 20:01:34 GMT Subject: RFR: 1551: Improve logging and metrics for BotRunner [v2] In-Reply-To: References: Message-ID: > I'm trying to better understand bot performance. I'm sure there are lots of things we could improve performance wise, but first we need to figure out how. We already have some logging and metrics gathering, but it's still not providing me with the information I need. > > With this change, I'm aiming to gather better metrics and logging around the getPeriodicItems() call, as well as refining the metrics for running WorkItems. > > The thread user time measurements from MX beans is useless to me, so I'm removing it. The cpu time is probably also useless, but I'm keeping it for now. What I really want is wall clock time for WorkItems, so I'm adding that. The gauges for measuring any kind of running time aren't useful at all. At least with counters, we can get some kind of idea how much time is spent on each kind of WorkItem over some time frame (using a rate function in the presentation layer). > > To complement the timing counters, I'm adding simple WorkItem counters as well, to keep track how many are scheduled, added as pending, submitted and finished. > > For the BotRunner.checkPeriodicItems() method, I'm adding two metrics. Since the whole method is running only one at a time, a gauge for tracking how long each round takes does make sense. Then for each sub call to Bot.getPeriodicItems there is a counter that basically gives the breakdown on how much of the time is spent in each kind of bot. > > For logging, I'm adding some more log messages to better track which Bot.getPeriodic item is called when, and a LogContext, so that all log messages are tagged with the bot toString, just like WorkItems are already. To support this I had to add toString() methods to all Bot implementations that were missing it. Erik Joelsson has updated the pull request incrementally with one additional commit since the last revision: Fix failing test ------------- Changes: - all: https://git.openjdk.org/skara/pull/1359/files - new: https://git.openjdk.org/skara/pull/1359/files/af64c33a..0d2a52f7 Webrevs: - full: https://webrevs.openjdk.org/?repo=skara&pr=1359&range=01 - incr: https://webrevs.openjdk.org/?repo=skara&pr=1359&range=00-01 Stats: 1 line in 1 file changed: 0 ins; 0 del; 1 mod Patch: https://git.openjdk.org/skara/pull/1359.diff Fetch: git fetch https://git.openjdk.org/skara pull/1359/head:pull/1359 PR: https://git.openjdk.org/skara/pull/1359 From kcr at openjdk.org Fri Aug 19 20:57:15 2022 From: kcr at openjdk.org (Kevin Rushforth) Date: Fri, 19 Aug 2022 20:57:15 GMT Subject: RFR: 1551: Improve logging and metrics for BotRunner [v2] In-Reply-To: References: Message-ID: On Fri, 19 Aug 2022 20:01:34 GMT, Erik Joelsson wrote: >> I'm trying to better understand bot performance. I'm sure there are lots of things we could improve performance wise, but first we need to figure out how. We already have some logging and metrics gathering, but it's still not providing me with the information I need. >> >> With this change, I'm aiming to gather better metrics and logging around the getPeriodicItems() call, as well as refining the metrics for running WorkItems. >> >> The thread user time measurements from MX beans is useless to me, so I'm removing it. The cpu time is probably also useless, but I'm keeping it for now. What I really want is wall clock time for WorkItems, so I'm adding that. The gauges for measuring any kind of running time aren't useful at all. At least with counters, we can get some kind of idea how much time is spent on each kind of WorkItem over some time frame (using a rate function in the presentation layer). >> >> To complement the timing counters, I'm adding simple WorkItem counters as well, to keep track how many are scheduled, added as pending, submitted and finished. >> >> For the BotRunner.checkPeriodicItems() method, I'm adding two metrics. Since the whole method is running only one at a time, a gauge for tracking how long each round takes does make sense. Then for each sub call to Bot.getPeriodicItems there is a counter that basically gives the breakdown on how much of the time is spent in each kind of bot. >> >> For logging, I'm adding some more log messages to better track which Bot.getPeriodic item is called when, and a LogContext, so that all log messages are tagged with the bot toString, just like WorkItems are already. To support this I had to add toString() methods to all Bot implementations that were missing it. > > Erik Joelsson has updated the pull request incrementally with one additional commit since the last revision: > > Fix failing test Changes looks reasonable. ------------- Marked as reviewed by kcr (Reviewer). PR: https://git.openjdk.org/skara/pull/1359 From erikj at openjdk.org Fri Aug 19 21:13:20 2022 From: erikj at openjdk.org (Erik Joelsson) Date: Fri, 19 Aug 2022 21:13:20 GMT Subject: Integrated: 1551: Improve logging and metrics for BotRunner In-Reply-To: References: Message-ID: On Fri, 19 Aug 2022 19:52:30 GMT, Erik Joelsson wrote: > I'm trying to better understand bot performance. I'm sure there are lots of things we could improve performance wise, but first we need to figure out how. We already have some logging and metrics gathering, but it's still not providing me with the information I need. > > With this change, I'm aiming to gather better metrics and logging around the getPeriodicItems() call, as well as refining the metrics for running WorkItems. > > The thread user time measurements from MX beans is useless to me, so I'm removing it. The cpu time is probably also useless, but I'm keeping it for now. What I really want is wall clock time for WorkItems, so I'm adding that. The gauges for measuring any kind of running time aren't useful at all. At least with counters, we can get some kind of idea how much time is spent on each kind of WorkItem over some time frame (using a rate function in the presentation layer). > > To complement the timing counters, I'm adding simple WorkItem counters as well, to keep track how many are scheduled, added as pending, submitted and finished. > > For the BotRunner.checkPeriodicItems() method, I'm adding two metrics. Since the whole method is running only one at a time, a gauge for tracking how long each round takes does make sense. Then for each sub call to Bot.getPeriodicItems there is a counter that basically gives the breakdown on how much of the time is spent in each kind of bot. > > For logging, I'm adding some more log messages to better track which Bot.getPeriodic item is called when, and a LogContext, so that all log messages are tagged with the bot toString, just like WorkItems are already. To support this I had to add toString() methods to all Bot implementations that were missing it. This pull request has now been integrated. Changeset: fd16a1d4 Author: Erik Joelsson URL: https://git.openjdk.org/skara/commit/fd16a1d49b1035126eefa7d77c54891276891538 Stats: 118 lines in 13 files changed: 76 ins; 29 del; 13 mod 1551: Improve logging and metrics for BotRunner Reviewed-by: kcr ------------- PR: https://git.openjdk.org/skara/pull/1359 From gli at openjdk.org Sat Aug 20 07:58:06 2022 From: gli at openjdk.org (Guoxiong Li) Date: Sat, 20 Aug 2022 07:58:06 GMT Subject: RFR: 1431: The method TestPullRequest#diff sometimes returns wrong information Message-ID: Hi all, The method `TestPullRequest#diff` always passes the head hash of the target branch to the method `Repository#diff` as the base hash. But it may be wrong when the target branch was submitted some new commits after creating the pull request. `TestPullRequest#diff` should find the last common hash of the source and target branch as the base hash. This patch fixes the bug and adds the test case. I don't know whether it is good to place the test case at the class `CSRBotTests`, but the method `CSRBotTests#testBackportCsr` indeed contains the situation of this bug. Thanks for taking the time to review. Best Regards, -- Gxuoxiong ------------- Commit messages: - SKARA-1431 Changes: https://git.openjdk.org/skara/pull/1360/files Webrev: https://webrevs.openjdk.org/?repo=skara&pr=1360&range=00 Issue: https://bugs.openjdk.org/browse/SKARA-1431 Stats: 25 lines in 2 files changed: 17 ins; 3 del; 5 mod Patch: https://git.openjdk.org/skara/pull/1360.diff Fetch: git fetch https://git.openjdk.org/skara pull/1360/head:pull/1360 PR: https://git.openjdk.org/skara/pull/1360 From ihse at openjdk.org Mon Aug 22 07:54:21 2022 From: ihse at openjdk.org (Magnus Ihse Bursie) Date: Mon, 22 Aug 2022 07:54:21 GMT Subject: RFR: 1431: The method TestPullRequest#diff sometimes returns wrong information In-Reply-To: References: Message-ID: On Sat, 20 Aug 2022 07:53:23 GMT, Guoxiong Li wrote: > Hi all, > > The method `TestPullRequest#diff` always passes the head hash of the target branch to the method `Repository#diff` as the base hash. But it may be wrong when the target branch was submitted some new commits after creating the pull request. `TestPullRequest#diff` should find the last common hash of the source and target branch as the base hash. > > This patch fixes the bug and adds the test case. I don't know whether it is good to place the test case at the class `CSRBotTests`, but the method `CSRBotTests#testBackportCsr` indeed contains the situation of this bug. > > Thanks for taking the time to review. > > Best Regards, > -- Guoxiong test/src/main/java/org/openjdk/skara/test/TestPullRequest.java line 257: > 255: var baseHash = sourceHash; > 256: var targetBranch = new Branch(targetRef()); > 257: while (!"0".repeat(40).equals(baseHash.hex())) { So if the common commit is more than 40 commits away, return "no diff"? :-) This is only used in testing, right? If so, this you could get away with it, but I think a comment describing the limitations would be needed. ------------- PR: https://git.openjdk.org/skara/pull/1360 From erikj at openjdk.org Mon Aug 22 12:50:41 2022 From: erikj at openjdk.org (Erik Joelsson) Date: Mon, 22 Aug 2022 12:50:41 GMT Subject: RFR: 1431: The method TestPullRequest#diff sometimes returns wrong information In-Reply-To: References: Message-ID: On Mon, 22 Aug 2022 07:51:04 GMT, Magnus Ihse Bursie wrote: >> Hi all, >> >> The method `TestPullRequest#diff` always passes the head hash of the target branch to the method `Repository#diff` as the base hash. But it may be wrong when the target branch was submitted some new commits after creating the pull request. `TestPullRequest#diff` should find the last common hash of the source and target branch as the base hash. >> >> This patch fixes the bug and adds the test case. I don't know whether it is good to place the test case at the class `CSRBotTests`, but the method `CSRBotTests#testBackportCsr` indeed contains the situation of this bug. >> >> Thanks for taking the time to review. >> >> Best Regards, >> -- Guoxiong > > test/src/main/java/org/openjdk/skara/test/TestPullRequest.java line 257: > >> 255: var baseHash = sourceHash; >> 256: var targetBranch = new Branch(targetRef()); >> 257: while (!"0".repeat(40).equals(baseHash.hex())) { > > So if the common commit is more than 40 commits away, return "no diff"? :-) > > This is only used in testing, right? If so, this you could get away with it, but I think a comment describing the limitations would be needed. I think you are misunderstanding the code. `"0".repeat(40)` just generates a string with 40 zeros, basically a null hash. Looping to find the common ancestor is one way of doing it, but I would recommend checking out `git merge-base`. There is a method for it already on `GitRepository`. ------------- PR: https://git.openjdk.org/skara/pull/1360 From ihse at openjdk.org Mon Aug 22 13:00:19 2022 From: ihse at openjdk.org (Magnus Ihse Bursie) Date: Mon, 22 Aug 2022 13:00:19 GMT Subject: RFR: 1431: The method TestPullRequest#diff sometimes returns wrong information In-Reply-To: References: Message-ID: On Mon, 22 Aug 2022 12:47:05 GMT, Erik Joelsson wrote: >> test/src/main/java/org/openjdk/skara/test/TestPullRequest.java line 257: >> >>> 255: var baseHash = sourceHash; >>> 256: var targetBranch = new Branch(targetRef()); >>> 257: while (!"0".repeat(40).equals(baseHash.hex())) { >> >> So if the common commit is more than 40 commits away, return "no diff"? :-) >> >> This is only used in testing, right? If so, this you could get away with it, but I think a comment describing the limitations would be needed. > > I think you are misunderstanding the code. `"0".repeat(40)` just generates a string with 40 zeros, basically a null hash. > > Looping to find the common ancestor is one way of doing it, but I would recommend checking out `git merge-base`. There is a method for it already on `GitRepository`. I see. Yes, I misunderstood the code as looping backwards for a maximum of 40. Don't know how I got to that conclusion. My brain is apparently still partially stuck in vacation mode. I don't think I'm used to methods being called on a string literal. ------------- PR: https://git.openjdk.org/skara/pull/1360 From gli at openjdk.org Mon Aug 22 13:32:41 2022 From: gli at openjdk.org (Guoxiong Li) Date: Mon, 22 Aug 2022 13:32:41 GMT Subject: RFR: 1431: The method TestPullRequest#diff sometimes returns wrong information [v2] In-Reply-To: References: Message-ID: <0bBxa-U6rdayeueYcF9SH0p0O1QnSDzSKHG8nIGhYbU=.0893305d-2b5c-419d-8b93-b0091e973599@github.com> > Hi all, > > The method `TestPullRequest#diff` always passes the head hash of the target branch to the method `Repository#diff` as the base hash. But it may be wrong when the target branch was submitted some new commits after creating the pull request. `TestPullRequest#diff` should find the last common hash of the source and target branch as the base hash. > > This patch fixes the bug and adds the test case. I don't know whether it is good to place the test case at the class `CSRBotTests`, but the method `CSRBotTests#testBackportCsr` indeed contains the situation of this bug. > > Thanks for taking the time to review. > > Best Regards, > -- Guoxiong Guoxiong Li has updated the pull request incrementally with one additional commit since the last revision: Use the method 'metgeBase'. ------------- Changes: - all: https://git.openjdk.org/skara/pull/1360/files - new: https://git.openjdk.org/skara/pull/1360/files/19b5f654..52a4d4f8 Webrevs: - full: https://webrevs.openjdk.org/?repo=skara&pr=1360&range=01 - incr: https://webrevs.openjdk.org/?repo=skara&pr=1360&range=00-01 Stats: 8 lines in 1 file changed: 0 ins; 7 del; 1 mod Patch: https://git.openjdk.org/skara/pull/1360.diff Fetch: git fetch https://git.openjdk.org/skara pull/1360/head:pull/1360 PR: https://git.openjdk.org/skara/pull/1360 From gli at openjdk.org Mon Aug 22 13:32:42 2022 From: gli at openjdk.org (Guoxiong Li) Date: Mon, 22 Aug 2022 13:32:42 GMT Subject: RFR: 1431: The method TestPullRequest#diff sometimes returns wrong information [v2] In-Reply-To: References: Message-ID: On Mon, 22 Aug 2022 12:57:17 GMT, Magnus Ihse Bursie wrote: >> I think you are misunderstanding the code. `"0".repeat(40)` just generates a string with 40 zeros, basically a null hash. >> >> Looping to find the common ancestor is one way of doing it, but I would recommend checking out `git merge-base`. There is a method for it already on `GitRepository`. > > I see. Yes, I misunderstood the code as looping backwards for a maximum of 40. Don't know how I got to that conclusion. My brain is apparently still partially stuck in vacation mode. I don't think I'm used to methods being called on a string literal. > Looping to find the common ancestor is one way of doing it, but I would recommend checking out `git merge-base`. There is a method for it already on `GitRepository`. Thanks for reminding me. I change the code to use the method `mergeBase` just now. ------------- PR: https://git.openjdk.org/skara/pull/1360 From ihse at openjdk.org Mon Aug 22 14:12:30 2022 From: ihse at openjdk.org (Magnus Ihse Bursie) Date: Mon, 22 Aug 2022 14:12:30 GMT Subject: RFR: 1431: The method TestPullRequest#diff sometimes returns wrong information [v2] In-Reply-To: <0bBxa-U6rdayeueYcF9SH0p0O1QnSDzSKHG8nIGhYbU=.0893305d-2b5c-419d-8b93-b0091e973599@github.com> References: <0bBxa-U6rdayeueYcF9SH0p0O1QnSDzSKHG8nIGhYbU=.0893305d-2b5c-419d-8b93-b0091e973599@github.com> Message-ID: <070k8lPheBGkqy80B0S33xBE1hdWKEC0AoARlDKVAXE=.8c7f54b5-32d8-497a-a48c-dc10385faa2d@github.com> On Mon, 22 Aug 2022 13:32:41 GMT, Guoxiong Li wrote: >> Hi all, >> >> The method `TestPullRequest#diff` always passes the head hash of the target branch to the method `Repository#diff` as the base hash. But it may be wrong when the target branch was submitted some new commits after creating the pull request. `TestPullRequest#diff` should find the last common hash of the source and target branch as the base hash. >> >> This patch fixes the bug and adds the test case. I don't know whether it is good to place the test case at the class `CSRBotTests`, but the method `CSRBotTests#testBackportCsr` indeed contains the situation of this bug. >> >> Thanks for taking the time to review. >> >> Best Regards, >> -- Guoxiong > > Guoxiong Li has updated the pull request incrementally with one additional commit since the last revision: > > Use the method 'metgeBase'. Much better. ------------- Marked as reviewed by ihse (Reviewer). PR: https://git.openjdk.org/skara/pull/1360 From erikj at openjdk.org Mon Aug 22 18:48:48 2022 From: erikj at openjdk.org (Erik Joelsson) Date: Mon, 22 Aug 2022 18:48:48 GMT Subject: RFR: 1431: The method TestPullRequest#diff sometimes returns wrong information [v2] In-Reply-To: <0bBxa-U6rdayeueYcF9SH0p0O1QnSDzSKHG8nIGhYbU=.0893305d-2b5c-419d-8b93-b0091e973599@github.com> References: <0bBxa-U6rdayeueYcF9SH0p0O1QnSDzSKHG8nIGhYbU=.0893305d-2b5c-419d-8b93-b0091e973599@github.com> Message-ID: On Mon, 22 Aug 2022 13:32:41 GMT, Guoxiong Li wrote: >> Hi all, >> >> The method `TestPullRequest#diff` always passes the head hash of the target branch to the method `Repository#diff` as the base hash. But it may be wrong when the target branch was submitted some new commits after creating the pull request. `TestPullRequest#diff` should find the last common hash of the source and target branch as the base hash. >> >> This patch fixes the bug and adds the test case. I don't know whether it is good to place the test case at the class `CSRBotTests`, but the method `CSRBotTests#testBackportCsr` indeed contains the situation of this bug. >> >> Thanks for taking the time to review. >> >> Best Regards, >> -- Guoxiong > > Guoxiong Li has updated the pull request incrementally with one additional commit since the last revision: > > Use the method 'metgeBase'. Marked as reviewed by erikj (Lead). ------------- PR: https://git.openjdk.org/skara/pull/1360 From gli at openjdk.org Tue Aug 23 02:08:38 2022 From: gli at openjdk.org (Guoxiong Li) Date: Tue, 23 Aug 2022 02:08:38 GMT Subject: RFR: 1431: The method TestPullRequest#diff sometimes returns wrong information [v2] In-Reply-To: <0bBxa-U6rdayeueYcF9SH0p0O1QnSDzSKHG8nIGhYbU=.0893305d-2b5c-419d-8b93-b0091e973599@github.com> References: <0bBxa-U6rdayeueYcF9SH0p0O1QnSDzSKHG8nIGhYbU=.0893305d-2b5c-419d-8b93-b0091e973599@github.com> Message-ID: <4aLtDhINwLvOECL3MGgjSjQYhMLvZed34B3FiPParxA=.b12f35f7-5155-4886-aaeb-9e1461fc6606@github.com> On Mon, 22 Aug 2022 13:32:41 GMT, Guoxiong Li wrote: >> Hi all, >> >> The method `TestPullRequest#diff` always passes the head hash of the target branch to the method `Repository#diff` as the base hash. But it may be wrong when the target branch was submitted some new commits after creating the pull request. `TestPullRequest#diff` should find the last common hash of the source and target branch as the base hash. >> >> This patch fixes the bug and adds the test case. I don't know whether it is good to place the test case at the class `CSRBotTests`, but the method `CSRBotTests#testBackportCsr` indeed contains the situation of this bug. >> >> Thanks for taking the time to review. >> >> Best Regards, >> -- Guoxiong > > Guoxiong Li has updated the pull request incrementally with one additional commit since the last revision: > > Use the method 'metgeBase'. Thanks for the review. Integrating. ------------- PR: https://git.openjdk.org/skara/pull/1360 From gli at openjdk.org Tue Aug 23 02:08:38 2022 From: gli at openjdk.org (Guoxiong Li) Date: Tue, 23 Aug 2022 02:08:38 GMT Subject: Integrated: 1431: The method TestPullRequest#diff sometimes returns wrong information In-Reply-To: References: Message-ID: On Sat, 20 Aug 2022 07:53:23 GMT, Guoxiong Li wrote: > Hi all, > > The method `TestPullRequest#diff` always passes the head hash of the target branch to the method `Repository#diff` as the base hash. But it may be wrong when the target branch was submitted some new commits after creating the pull request. `TestPullRequest#diff` should find the last common hash of the source and target branch as the base hash. > > This patch fixes the bug and adds the test case. I don't know whether it is good to place the test case at the class `CSRBotTests`, but the method `CSRBotTests#testBackportCsr` indeed contains the situation of this bug. > > Thanks for taking the time to review. > > Best Regards, > -- Guoxiong This pull request has now been integrated. Changeset: cd765e93 Author: Guoxiong Li URL: https://git.openjdk.org/skara/commit/cd765e9302076b87c31d744b833c97c0cd2b80da Stats: 18 lines in 2 files changed: 10 ins; 3 del; 5 mod 1431: The method TestPullRequest#diff sometimes returns wrong information Reviewed-by: ihse, erikj ------------- PR: https://git.openjdk.org/skara/pull/1360 From erikj at openjdk.org Tue Aug 23 23:27:59 2022 From: erikj at openjdk.org (Erik Joelsson) Date: Tue, 23 Aug 2022 23:27:59 GMT Subject: RFR: 1552: Log relevant timer numbers Message-ID: This is a pretty big patch, but it shouldn't be too scary as it's only supposed to change logging, mainly by adding more logging (and also removing some now redundant log calls). The main focus here is to add structured timing data to some log messages. To do this, I'm using the optional `parameters` `Object[]` that you can supply to the logger from `java.util.logging`. If the LogstashHandler finds a `Duration` instance in that array, it will add it as a "duration" field in the logstash log document. In the BotRunner, we were already using parameters for a different purpose, so it got a little bit messy, but I think it's ok. There are two kinds of timings I'm adding logging for. The obvious one is timing external HTTP calls and running processes, as well as total running/pending/scheduled times for WorkItems and getPeriodicItems in the BotRunner. The other kind are more complex as I'm trying to measure the user perceived latency from when a PR or bug is updated to when the bot actually reacts with a response to that update. This is of course tricky to get right and needs to be carefully implemented for each instance, but I believe I have at least added some reasonably relevant approximations that will make sense to measure and track. To generate these, I had to introduce a new field in various WorkItems that stores the timestamp from the external change that triggered it (e.g. `prUpdatedAt`). In addition to this, there are a ton more regular log messages, mostly on info level, that detail actions performed by various WorkItems. This is something I have been missing a lot when browsing logs. ------------- Commit messages: - Better field name - Command name in latency log - Log latency when PR command successful - Adjusted CheckRun comments - SKARA-1552 Changes: https://git.openjdk.org/skara/pull/1361/files Webrev: https://webrevs.openjdk.org/?repo=skara&pr=1361&range=00 Issue: https://bugs.openjdk.org/browse/SKARA-1552 Stats: 286 lines in 28 files changed: 194 ins; 14 del; 78 mod Patch: https://git.openjdk.org/skara/pull/1361.diff Fetch: git fetch https://git.openjdk.org/skara pull/1361/head:pull/1361 PR: https://git.openjdk.org/skara/pull/1361 From erikj at openjdk.org Wed Aug 24 21:31:40 2022 From: erikj at openjdk.org (Erik Joelsson) Date: Wed, 24 Aug 2022 21:31:40 GMT Subject: RFR: 1560: Backport command fails to load census Message-ID: <2KQpfWa1hvdEgnA233-NJJY6CxCb4UiNAJKBoW3DnUs=.1a704edf-77b0-489f-8e2a-19b456ea375d@github.com> This patch attempts to fix the problem where a commit command (in our case a /backport command) fails to instantiate a CensusInstance, due to the configured jcheck "project", at the particular commit, not existing in the census in question. What happens then is that CensusInstance.create fails, which triggers an endless series of retries in the bot. Since none of the current commit commands have a need for the "project" part of the census, I decided to create a separate CensusInstance type `LimitedCensusInstance` without the "project", and explicitly use this in the commit commands. Tests are passing, and I could manually verify that this solves the problem we currently have with a bad /backport command in a private GitLab instance. ------------- Commit messages: - SKARA-1560 Changes: https://git.openjdk.org/skara/pull/1362/files Webrev: https://webrevs.openjdk.org/?repo=skara&pr=1362&range=00 Issue: https://bugs.openjdk.org/browse/SKARA-1560 Stats: 99 lines in 6 files changed: 55 ins; 22 del; 22 mod Patch: https://git.openjdk.org/skara/pull/1362.diff Fetch: git fetch https://git.openjdk.org/skara pull/1362/head:pull/1362 PR: https://git.openjdk.org/skara/pull/1362 From erikj at openjdk.org Wed Aug 24 21:48:38 2022 From: erikj at openjdk.org (Erik Joelsson) Date: Wed, 24 Aug 2022 21:48:38 GMT Subject: RFR: 1560: Backport command fails to load census [v2] In-Reply-To: <2KQpfWa1hvdEgnA233-NJJY6CxCb4UiNAJKBoW3DnUs=.1a704edf-77b0-489f-8e2a-19b456ea375d@github.com> References: <2KQpfWa1hvdEgnA233-NJJY6CxCb4UiNAJKBoW3DnUs=.1a704edf-77b0-489f-8e2a-19b456ea375d@github.com> Message-ID: <0tYxbd-SPgq3GUFIH_U6yWqdrCAKcQjdVK5za13aaWo=.a312d58f-2428-4950-92aa-69d89f781645@github.com> > This patch attempts to fix the problem where a commit command (in our case a /backport command) fails to instantiate a CensusInstance, due to the configured jcheck "project", at the particular commit, not existing in the census in question. What happens then is that CensusInstance.create fails, which triggers an endless series of retries in the bot. > > Since none of the current commit commands have a need for the "project" part of the census, I decided to create a separate CensusInstance type `LimitedCensusInstance` without the "project", and explicitly use this in the commit commands. > > Tests are passing, and I could manually verify that this solves the problem we currently have with a bad /backport command in a private GitLab instance. Erik Joelsson has updated the pull request incrementally with one additional commit since the last revision: Added comment ------------- Changes: - all: https://git.openjdk.org/skara/pull/1362/files - new: https://git.openjdk.org/skara/pull/1362/files/250f6f9f..1f4f2583 Webrevs: - full: https://webrevs.openjdk.org/?repo=skara&pr=1362&range=01 - incr: https://webrevs.openjdk.org/?repo=skara&pr=1362&range=00-01 Stats: 4 lines in 1 file changed: 4 ins; 0 del; 0 mod Patch: https://git.openjdk.org/skara/pull/1362.diff Fetch: git fetch https://git.openjdk.org/skara pull/1362/head:pull/1362 PR: https://git.openjdk.org/skara/pull/1362 From kcr at openjdk.org Wed Aug 24 22:56:32 2022 From: kcr at openjdk.org (Kevin Rushforth) Date: Wed, 24 Aug 2022 22:56:32 GMT Subject: RFR: 1560: Backport command fails to load census [v2] In-Reply-To: <0tYxbd-SPgq3GUFIH_U6yWqdrCAKcQjdVK5za13aaWo=.a312d58f-2428-4950-92aa-69d89f781645@github.com> References: <2KQpfWa1hvdEgnA233-NJJY6CxCb4UiNAJKBoW3DnUs=.1a704edf-77b0-489f-8e2a-19b456ea375d@github.com> <0tYxbd-SPgq3GUFIH_U6yWqdrCAKcQjdVK5za13aaWo=.a312d58f-2428-4950-92aa-69d89f781645@github.com> Message-ID: On Wed, 24 Aug 2022 21:48:38 GMT, Erik Joelsson wrote: >> This patch attempts to fix the problem where a commit command (in our case a /backport command) fails to instantiate a CensusInstance, due to the configured jcheck "project", at the particular commit, not existing in the census in question. What happens then is that CensusInstance.create fails, which triggers an endless series of retries in the bot. >> >> Since none of the current commit commands have a need for the "project" part of the census, I decided to create a separate CensusInstance type `LimitedCensusInstance` without the "project", and explicitly use this in the commit commands. >> >> Tests are passing, and I could manually verify that this solves the problem we currently have with a bad /backport command in a private GitLab instance. > > Erik Joelsson has updated the pull request incrementally with one additional commit since the last revision: > > Added comment Looks good. bots/pr/src/main/java/org/openjdk/skara/bots/pr/CensusInstance.java line 165: > 163: } > 164: > 165: class LimitedCensusInstance { Maybe move this to its own file? I try to avoid having more than one top level class in the same file, but do whatever makes the most sense (maybe Skara uses this in other places?). ------------- Marked as reviewed by kcr (Reviewer). PR: https://git.openjdk.org/skara/pull/1362 From erikj at openjdk.org Wed Aug 24 23:26:19 2022 From: erikj at openjdk.org (Erik Joelsson) Date: Wed, 24 Aug 2022 23:26:19 GMT Subject: RFR: 1560: Backport command fails to load census [v2] In-Reply-To: References: <2KQpfWa1hvdEgnA233-NJJY6CxCb4UiNAJKBoW3DnUs=.1a704edf-77b0-489f-8e2a-19b456ea375d@github.com> <0tYxbd-SPgq3GUFIH_U6yWqdrCAKcQjdVK5za13aaWo=.a312d58f-2428-4950-92aa-69d89f781645@github.com> Message-ID: On Wed, 24 Aug 2022 22:52:17 GMT, Kevin Rushforth wrote: >> Erik Joelsson has updated the pull request incrementally with one additional commit since the last revision: >> >> Added comment > > bots/pr/src/main/java/org/openjdk/skara/bots/pr/CensusInstance.java line 165: > >> 163: } >> 164: >> 165: class LimitedCensusInstance { > > Maybe move this to its own file? I try to avoid having more than one top level class in the same file, but do whatever makes the most sense (maybe Skara uses this in other places?). I will think about it. This is a rather common pattern in Skara, which is why I went for it. ------------- PR: https://git.openjdk.org/skara/pull/1362 From erikj at openjdk.org Wed Aug 24 23:42:58 2022 From: erikj at openjdk.org (Erik Joelsson) Date: Wed, 24 Aug 2022 23:42:58 GMT Subject: RFR: 1560: Backport command fails to load census [v3] In-Reply-To: <2KQpfWa1hvdEgnA233-NJJY6CxCb4UiNAJKBoW3DnUs=.1a704edf-77b0-489f-8e2a-19b456ea375d@github.com> References: <2KQpfWa1hvdEgnA233-NJJY6CxCb4UiNAJKBoW3DnUs=.1a704edf-77b0-489f-8e2a-19b456ea375d@github.com> Message-ID: > This patch attempts to fix the problem where a commit command (in our case a /backport command) fails to instantiate a CensusInstance, due to the configured jcheck "project", at the particular commit, not existing in the census in question. What happens then is that CensusInstance.create fails, which triggers an endless series of retries in the bot. > > Since none of the current commit commands have a need for the "project" part of the census, I decided to create a separate CensusInstance type `LimitedCensusInstance` without the "project", and explicitly use this in the commit commands. > > Tests are passing, and I could manually verify that this solves the problem we currently have with a bad /backport command in a private GitLab instance. Erik Joelsson has updated the pull request incrementally with one additional commit since the last revision: Move LimitedCensusInstance to its own file ------------- Changes: - all: https://git.openjdk.org/skara/pull/1362/files - new: https://git.openjdk.org/skara/pull/1362/files/1f4f2583..19d40ee2 Webrevs: - full: https://webrevs.openjdk.org/?repo=skara&pr=1362&range=02 - incr: https://webrevs.openjdk.org/?repo=skara&pr=1362&range=01-02 Stats: 211 lines in 5 files changed: 108 ins; 96 del; 7 mod Patch: https://git.openjdk.org/skara/pull/1362.diff Fetch: git fetch https://git.openjdk.org/skara pull/1362/head:pull/1362 PR: https://git.openjdk.org/skara/pull/1362 From kcr at openjdk.org Wed Aug 24 23:46:38 2022 From: kcr at openjdk.org (Kevin Rushforth) Date: Wed, 24 Aug 2022 23:46:38 GMT Subject: RFR: 1560: Backport command fails to load census [v3] In-Reply-To: References: <2KQpfWa1hvdEgnA233-NJJY6CxCb4UiNAJKBoW3DnUs=.1a704edf-77b0-489f-8e2a-19b456ea375d@github.com> Message-ID: On Wed, 24 Aug 2022 23:42:58 GMT, Erik Joelsson wrote: >> This patch attempts to fix the problem where a commit command (in our case a /backport command) fails to instantiate a CensusInstance, due to the configured jcheck "project", at the particular commit, not existing in the census in question. What happens then is that CensusInstance.create fails, which triggers an endless series of retries in the bot. >> >> Since none of the current commit commands have a need for the "project" part of the census, I decided to create a separate CensusInstance type `LimitedCensusInstance` without the "project", and explicitly use this in the commit commands. >> >> Tests are passing, and I could manually verify that this solves the problem we currently have with a bad /backport command in a private GitLab instance. > > Erik Joelsson has updated the pull request incrementally with one additional commit since the last revision: > > Move LimitedCensusInstance to its own file I think this is a cleaner separation. Looks good. ------------- Marked as reviewed by kcr (Reviewer). PR: https://git.openjdk.org/skara/pull/1362 From gli at openjdk.org Thu Aug 25 05:41:19 2022 From: gli at openjdk.org (Guoxiong Li) Date: Thu, 25 Aug 2022 05:41:19 GMT Subject: RFR: 1560: Backport command fails to load census [v3] In-Reply-To: References: <2KQpfWa1hvdEgnA233-NJJY6CxCb4UiNAJKBoW3DnUs=.1a704edf-77b0-489f-8e2a-19b456ea375d@github.com> Message-ID: <_ZMNPheBhsnPjSwO0eMJqV6QH1M1lRSQEqClUPVCXDc=.2583b7c8-ff7a-4c88-864c-f30e86bb1b78@github.com> On Wed, 24 Aug 2022 23:42:58 GMT, Erik Joelsson wrote: >> This patch attempts to fix the problem where a commit command (in our case a /backport command) fails to instantiate a CensusInstance, due to the configured jcheck "project", at the particular commit, not existing in the census in question. What happens then is that CensusInstance.create fails, which triggers an endless series of retries in the bot. >> >> Since none of the current commit commands have a need for the "project" part of the census, I decided to create a separate CensusInstance type `LimitedCensusInstance` without the "project", and explicitly use this in the commit commands. >> >> Tests are passing, and I could manually verify that this solves the problem we currently have with a bad /backport command in a private GitLab instance. > > Erik Joelsson has updated the pull request incrementally with one additional commit since the last revision: > > Move LimitedCensusInstance to its own file LGTM ------------- Marked as reviewed by gli (Committer). PR: https://git.openjdk.org/skara/pull/1362 From erikj at openjdk.org Thu Aug 25 17:45:49 2022 From: erikj at openjdk.org (Erik Joelsson) Date: Thu, 25 Aug 2022 17:45:49 GMT Subject: RFR: 1560: Backport command fails to load census [v3] In-Reply-To: References: <2KQpfWa1hvdEgnA233-NJJY6CxCb4UiNAJKBoW3DnUs=.1a704edf-77b0-489f-8e2a-19b456ea375d@github.com> Message-ID: On Wed, 24 Aug 2022 23:42:58 GMT, Erik Joelsson wrote: >> This patch attempts to fix the problem where a commit command (in our case a /backport command) fails to instantiate a CensusInstance, due to the configured jcheck "project", at the particular commit, not existing in the census in question. What happens then is that CensusInstance.create fails, which triggers an endless series of retries in the bot. >> >> Since none of the current commit commands have a need for the "project" part of the census, I decided to create a separate CensusInstance type `LimitedCensusInstance` without the "project", and explicitly use this in the commit commands. >> >> Tests are passing, and I could manually verify that this solves the problem we currently have with a bad /backport command in a private GitLab instance. > > Erik Joelsson has updated the pull request incrementally with one additional commit since the last revision: > > Move LimitedCensusInstance to its own file Thanks! ------------- PR: https://git.openjdk.org/skara/pull/1362 From erikj at openjdk.org Thu Aug 25 17:45:49 2022 From: erikj at openjdk.org (Erik Joelsson) Date: Thu, 25 Aug 2022 17:45:49 GMT Subject: Integrated: 1560: Backport command fails to load census In-Reply-To: <2KQpfWa1hvdEgnA233-NJJY6CxCb4UiNAJKBoW3DnUs=.1a704edf-77b0-489f-8e2a-19b456ea375d@github.com> References: <2KQpfWa1hvdEgnA233-NJJY6CxCb4UiNAJKBoW3DnUs=.1a704edf-77b0-489f-8e2a-19b456ea375d@github.com> Message-ID: On Wed, 24 Aug 2022 21:25:25 GMT, Erik Joelsson wrote: > This patch attempts to fix the problem where a commit command (in our case a /backport command) fails to instantiate a CensusInstance, due to the configured jcheck "project", at the particular commit, not existing in the census in question. What happens then is that CensusInstance.create fails, which triggers an endless series of retries in the bot. > > Since none of the current commit commands have a need for the "project" part of the census, I decided to create a separate CensusInstance type `LimitedCensusInstance` without the "project", and explicitly use this in the commit commands. > > Tests are passing, and I could manually verify that this solves the problem we currently have with a bad /backport command in a private GitLab instance. This pull request has now been integrated. Changeset: c7fc108a Author: Erik Joelsson URL: https://git.openjdk.org/skara/commit/c7fc108a271580fa7e6afecf290d5ba0fafeba05 Stats: 194 lines in 9 files changed: 113 ins; 64 del; 17 mod 1560: Backport command fails to load census Reviewed-by: kcr, gli ------------- PR: https://git.openjdk.org/skara/pull/1362 From erikj at openjdk.org Thu Aug 25 18:50:05 2022 From: erikj at openjdk.org (Erik Joelsson) Date: Thu, 25 Aug 2022 18:50:05 GMT Subject: RFR: 1555: Remove support for review by award emojis in GitLab Message-ID: This patch removes support for using award emojis for marking an MR in GitLab as reviewed. This was the original Skara implementation from back when GitLab did not have an `approve` button. In https://github.com/openjdk/skara/pull/939, support for the new approve button was introduced and the award emojis were deprecated. I think it's now time to remove them completely. The removed test had an assumeTrue that made it never run, as the TestForge had support for review body, so I chose to remove the whole test. ------------- Commit messages: - SKARA-1555 Changes: https://git.openjdk.org/skara/pull/1363/files Webrev: https://webrevs.openjdk.org/?repo=skara&pr=1363&range=00 Issue: https://bugs.openjdk.org/browse/SKARA-1555 Stats: 219 lines in 10 files changed: 0 ins; 214 del; 5 mod Patch: https://git.openjdk.org/skara/pull/1363.diff Fetch: git fetch https://git.openjdk.org/skara pull/1363/head:pull/1363 PR: https://git.openjdk.org/skara/pull/1363 From ihse at openjdk.org Tue Aug 30 10:26:48 2022 From: ihse at openjdk.org (Magnus Ihse Bursie) Date: Tue, 30 Aug 2022 10:26:48 GMT Subject: RFR: 1555: Remove support for review by award emojis in GitLab In-Reply-To: References: Message-ID: On Thu, 25 Aug 2022 18:45:31 GMT, Erik Joelsson wrote: > This patch removes support for using award emojis for marking an MR in GitLab as reviewed. This was the original Skara implementation from back when GitLab did not have an `approve` button. In https://github.com/openjdk/skara/pull/939, support for the new approve button was introduced and the award emojis were deprecated. I think it's now time to remove them completely. > > The removed test had an assumeTrue that made it never run, as the TestForge had support for review body, so I chose to remove the whole test. Good riddance! ------------- Marked as reviewed by ihse (Reviewer). PR: https://git.openjdk.org/skara/pull/1363 From ihse at openjdk.org Tue Aug 30 14:44:36 2022 From: ihse at openjdk.org (Magnus Ihse Bursie) Date: Tue, 30 Aug 2022 14:44:36 GMT Subject: RFR: 1552: Log relevant timer numbers In-Reply-To: References: Message-ID: On Tue, 23 Aug 2022 23:20:26 GMT, Erik Joelsson wrote: > This is a pretty big patch, but it shouldn't be too scary as it's only supposed to change logging, mainly by adding more logging (and also removing some now redundant log calls). > > The main focus here is to add structured timing data to some log messages. To do this, I'm using the optional `parameters` `Object[]` that you can supply to the logger from `java.util.logging`. If the LogstashHandler finds a `Duration` instance in that array, it will add it as a "duration" field in the logstash log document. In the BotRunner, we were already using parameters for a different purpose, so it got a little bit messy, but I think it's ok. > > There are two kinds of timings I'm adding logging for. The obvious one is timing external HTTP calls and running processes, as well as total running/pending/scheduled times for WorkItems and getPeriodicItems in the BotRunner. The other kind are more complex as I'm trying to measure the user perceived latency from when a PR or bug is updated to when the bot actually reacts with a response to that update. This is of course tricky to get right and needs to be carefully implemented for each instance, but I believe I have at least added some reasonably relevant approximations that will make sense to measure and track. To generate these, I had to introduce a new field in various WorkItems that stores the timestamp from the external change that triggered it (e.g. `prUpdatedAt`). > > In addition to this, there are a ton more regular log messages, mostly on info level, that detail actions performed by various WorkItems. This is something I have been missing a lot when browsing logs. bot/src/main/java/org/openjdk/skara/bot/BotRunner.java line 236: > 234: } finally { > 235: var duration = Duration.between(start, Instant.now()); > 236: log.log(Level.FINE, "Item " + item + " is now done " + duration, Suggestion: log.log(Level.FINE, "Item " + item + " is now done after " + duration, ------------- PR: https://git.openjdk.org/skara/pull/1361 From ihse at openjdk.org Tue Aug 30 14:52:01 2022 From: ihse at openjdk.org (Magnus Ihse Bursie) Date: Tue, 30 Aug 2022 14:52:01 GMT Subject: RFR: 1552: Log relevant timer numbers In-Reply-To: References: Message-ID: On Tue, 23 Aug 2022 23:20:26 GMT, Erik Joelsson wrote: > This is a pretty big patch, but it shouldn't be too scary as it's only supposed to change logging, mainly by adding more logging (and also removing some now redundant log calls). > > The main focus here is to add structured timing data to some log messages. To do this, I'm using the optional `parameters` `Object[]` that you can supply to the logger from `java.util.logging`. If the LogstashHandler finds a `Duration` instance in that array, it will add it as a "duration" field in the logstash log document. In the BotRunner, we were already using parameters for a different purpose, so it got a little bit messy, but I think it's ok. > > There are two kinds of timings I'm adding logging for. The obvious one is timing external HTTP calls and running processes, as well as total running/pending/scheduled times for WorkItems and getPeriodicItems in the BotRunner. The other kind are more complex as I'm trying to measure the user perceived latency from when a PR or bug is updated to when the bot actually reacts with a response to that update. This is of course tricky to get right and needs to be carefully implemented for each instance, but I believe I have at least added some reasonably relevant approximations that will make sense to measure and track. To generate these, I had to introduce a new field in various WorkItems that stores the timestamp from the external change that triggered it (e.g. `prUpdatedAt`). > > In addition to this, there are a ton more regular log messages, mostly on info level, that detail actions performed by various WorkItems. This is something I have been missing a lot when browsing logs. bots/pr/src/main/java/org/openjdk/skara/bots/pr/CheckWorkItem.java line 360: > 358: if (log.isLoggable(Level.INFO)) { > 359: var updatedPr = bot.repo().pullRequest(prId); > 360: logLatency("Time from PR updated to CheckRun done ", updatedPr.updatedAt(), log); logLatency only logs at INFO level anyway. I'm not sure why you put just this one instance inside the log.isLoggable guard. ------------- PR: https://git.openjdk.org/skara/pull/1361 From ihse at openjdk.org Tue Aug 30 14:56:31 2022 From: ihse at openjdk.org (Magnus Ihse Bursie) Date: Tue, 30 Aug 2022 14:56:31 GMT Subject: RFR: 1552: Log relevant timer numbers In-Reply-To: References: Message-ID: On Tue, 30 Aug 2022 14:48:48 GMT, Magnus Ihse Bursie wrote: >> This is a pretty big patch, but it shouldn't be too scary as it's only supposed to change logging, mainly by adding more logging (and also removing some now redundant log calls). >> >> The main focus here is to add structured timing data to some log messages. To do this, I'm using the optional `parameters` `Object[]` that you can supply to the logger from `java.util.logging`. If the LogstashHandler finds a `Duration` instance in that array, it will add it as a "duration" field in the logstash log document. In the BotRunner, we were already using parameters for a different purpose, so it got a little bit messy, but I think it's ok. >> >> There are two kinds of timings I'm adding logging for. The obvious one is timing external HTTP calls and running processes, as well as total running/pending/scheduled times for WorkItems and getPeriodicItems in the BotRunner. The other kind are more complex as I'm trying to measure the user perceived latency from when a PR or bug is updated to when the bot actually reacts with a response to that update. This is of course tricky to get right and needs to be carefully implemented for each instance, but I believe I have at least added some reasonably relevant approximations that will make sense to measure and track. To generate these, I had to introduce a new field in various WorkItems that stores the timestamp from the external change that triggered it (e.g. `prUpdatedAt`). >> >> In addition to this, there are a ton more regular log messages, mostly on info level, that detail actions performed by various WorkItems. This is something I have been missing a lot when browsing logs. > > bots/pr/src/main/java/org/openjdk/skara/bots/pr/CheckWorkItem.java line 360: > >> 358: if (log.isLoggable(Level.INFO)) { >> 359: var updatedPr = bot.repo().pullRequest(prId); >> 360: logLatency("Time from PR updated to CheckRun done ", updatedPr.updatedAt(), log); > > logLatency only logs at INFO level anyway. I'm not sure why you put just this one instance inside the log.isLoggable guard. Actually, I'm trying to understand why this is correct. `logLatency` looks like this: protected void logLatency(String message, ZonedDateTime endTime, Logger log) { var latency = Duration.between(prUpdatedAt, endTime); ... I would have expected the endTime to be the current time. But maybe I'm just getting lost here on the difference between `updatedPr.updatedAt()` and `prUpdatedAt`. ------------- PR: https://git.openjdk.org/skara/pull/1361 From ihse at openjdk.org Tue Aug 30 14:59:52 2022 From: ihse at openjdk.org (Magnus Ihse Bursie) Date: Tue, 30 Aug 2022 14:59:52 GMT Subject: RFR: 1552: Log relevant timer numbers In-Reply-To: References: Message-ID: On Tue, 23 Aug 2022 23:20:26 GMT, Erik Joelsson wrote: > This is a pretty big patch, but it shouldn't be too scary as it's only supposed to change logging, mainly by adding more logging (and also removing some now redundant log calls). > > The main focus here is to add structured timing data to some log messages. To do this, I'm using the optional `parameters` `Object[]` that you can supply to the logger from `java.util.logging`. If the LogstashHandler finds a `Duration` instance in that array, it will add it as a "duration" field in the logstash log document. In the BotRunner, we were already using parameters for a different purpose, so it got a little bit messy, but I think it's ok. > > There are two kinds of timings I'm adding logging for. The obvious one is timing external HTTP calls and running processes, as well as total running/pending/scheduled times for WorkItems and getPeriodicItems in the BotRunner. The other kind are more complex as I'm trying to measure the user perceived latency from when a PR or bug is updated to when the bot actually reacts with a response to that update. This is of course tricky to get right and needs to be carefully implemented for each instance, but I believe I have at least added some reasonably relevant approximations that will make sense to measure and track. To generate these, I had to introduce a new field in various WorkItems that stores the timestamp from the external change that triggered it (e.g. `prUpdatedAt`). > > In addition to this, there are a ton more regular log messages, mostly on info level, that detail actions performed by various WorkItems. This is something I have been missing a lot when browsing logs. Overall looks good to me. ------------- Marked as reviewed by ihse (Reviewer). PR: https://git.openjdk.org/skara/pull/1361 From erikj at openjdk.org Tue Aug 30 17:34:45 2022 From: erikj at openjdk.org (Erik Joelsson) Date: Tue, 30 Aug 2022 17:34:45 GMT Subject: RFR: 1552: Log relevant timer numbers In-Reply-To: References: Message-ID: On Tue, 30 Aug 2022 14:53:35 GMT, Magnus Ihse Bursie wrote: >> bots/pr/src/main/java/org/openjdk/skara/bots/pr/CheckWorkItem.java line 360: >> >>> 358: if (log.isLoggable(Level.INFO)) { >>> 359: var updatedPr = bot.repo().pullRequest(prId); >>> 360: logLatency("Time from PR updated to CheckRun done ", updatedPr.updatedAt(), log); >> >> logLatency only logs at INFO level anyway. I'm not sure why you put just this one instance inside the log.isLoggable guard. > > Actually, I'm trying to understand why this is correct. `logLatency` looks like this: > > protected void logLatency(String message, ZonedDateTime endTime, Logger log) { > var latency = Duration.between(prUpdatedAt, endTime); > ... > > > I would have expected the endTime to be the current time. But maybe I'm just getting lost here on the difference between `updatedPr.updatedAt()` and `prUpdatedAt`. We basically always log INFO so the guard isn't strictly necessary. Even so I tried to put guards whenever I needed to fetch data from a remote to generate the log message. In this case, it's refetching the PullRequest from the forge. I should add a comment about this to make it clearer. Whenever possible, I want the latency numbers to be calculated between timestamps from the same clock. In this case the clock is on the forge (e.g. GitHub). The start time is the updatedAt timestamp read from the PR when we triggered this WorkItem. The endTime is the updatedAt from the PR just now, after we updated it from this WorkItem (CheckRun.execute() will always update the `check` in the PR). ------------- PR: https://git.openjdk.org/skara/pull/1361 From erikj at openjdk.org Tue Aug 30 18:18:30 2022 From: erikj at openjdk.org (Erik Joelsson) Date: Tue, 30 Aug 2022 18:18:30 GMT Subject: RFR: 1552: Log relevant timer numbers [v2] In-Reply-To: References: Message-ID: > This is a pretty big patch, but it shouldn't be too scary as it's only supposed to change logging, mainly by adding more logging (and also removing some now redundant log calls). > > The main focus here is to add structured timing data to some log messages. To do this, I'm using the optional `parameters` `Object[]` that you can supply to the logger from `java.util.logging`. If the LogstashHandler finds a `Duration` instance in that array, it will add it as a "duration" field in the logstash log document. In the BotRunner, we were already using parameters for a different purpose, so it got a little bit messy, but I think it's ok. > > There are two kinds of timings I'm adding logging for. The obvious one is timing external HTTP calls and running processes, as well as total running/pending/scheduled times for WorkItems and getPeriodicItems in the BotRunner. The other kind are more complex as I'm trying to measure the user perceived latency from when a PR or bug is updated to when the bot actually reacts with a response to that update. This is of course tricky to get right and needs to be carefully implemented for each instance, but I believe I have at least added some reasonably relevant approximations that will make sense to measure and track. To generate these, I had to introduce a new field in various WorkItems that stores the timestamp from the external change that triggered it (e.g. `prUpdatedAt`). > > In addition to this, there are a ton more regular log messages, mostly on info level, that detail actions performed by various WorkItems. This is something I have been missing a lot when browsing logs. Erik Joelsson has updated the pull request incrementally with one additional commit since the last revision: Added comment, adjusted message, based on review ------------- Changes: - all: https://git.openjdk.org/skara/pull/1361/files - new: https://git.openjdk.org/skara/pull/1361/files/2cb0aee7..c22000d0 Webrevs: - full: https://webrevs.openjdk.org/?repo=skara&pr=1361&range=01 - incr: https://webrevs.openjdk.org/?repo=skara&pr=1361&range=00-01 Stats: 6 lines in 2 files changed: 5 ins; 0 del; 1 mod Patch: https://git.openjdk.org/skara/pull/1361.diff Fetch: git fetch https://git.openjdk.org/skara pull/1361/head:pull/1361 PR: https://git.openjdk.org/skara/pull/1361 From erikj at openjdk.org Tue Aug 30 18:18:30 2022 From: erikj at openjdk.org (Erik Joelsson) Date: Tue, 30 Aug 2022 18:18:30 GMT Subject: Integrated: 1552: Log relevant timer numbers In-Reply-To: References: Message-ID: On Tue, 23 Aug 2022 23:20:26 GMT, Erik Joelsson wrote: > This is a pretty big patch, but it shouldn't be too scary as it's only supposed to change logging, mainly by adding more logging (and also removing some now redundant log calls). > > The main focus here is to add structured timing data to some log messages. To do this, I'm using the optional `parameters` `Object[]` that you can supply to the logger from `java.util.logging`. If the LogstashHandler finds a `Duration` instance in that array, it will add it as a "duration" field in the logstash log document. In the BotRunner, we were already using parameters for a different purpose, so it got a little bit messy, but I think it's ok. > > There are two kinds of timings I'm adding logging for. The obvious one is timing external HTTP calls and running processes, as well as total running/pending/scheduled times for WorkItems and getPeriodicItems in the BotRunner. The other kind are more complex as I'm trying to measure the user perceived latency from when a PR or bug is updated to when the bot actually reacts with a response to that update. This is of course tricky to get right and needs to be carefully implemented for each instance, but I believe I have at least added some reasonably relevant approximations that will make sense to measure and track. To generate these, I had to introduce a new field in various WorkItems that stores the timestamp from the external change that triggered it (e.g. `prUpdatedAt`). > > In addition to this, there are a ton more regular log messages, mostly on info level, that detail actions performed by various WorkItems. This is something I have been missing a lot when browsing logs. This pull request has now been integrated. Changeset: 3b74c3b2 Author: Erik Joelsson URL: https://git.openjdk.org/skara/commit/3b74c3b2f4c1de7fd30f31c367a52cf6e3ee392f Stats: 290 lines in 28 files changed: 199 ins; 13 del; 78 mod 1552: Log relevant timer numbers Reviewed-by: ihse ------------- PR: https://git.openjdk.org/skara/pull/1361 From erikj at openjdk.org Tue Aug 30 18:22:08 2022 From: erikj at openjdk.org (Erik Joelsson) Date: Tue, 30 Aug 2022 18:22:08 GMT Subject: RFR: 1555: Remove support for review by award emojis in GitLab [v2] In-Reply-To: References: Message-ID: > This patch removes support for using award emojis for marking an MR in GitLab as reviewed. This was the original Skara implementation from back when GitLab did not have an `approve` button. In https://github.com/openjdk/skara/pull/939, support for the new approve button was introduced and the award emojis were deprecated. I think it's now time to remove them completely. > > The removed test had an assumeTrue that made it never run, as the TestForge had support for review body, so I chose to remove the whole test. Erik Joelsson has updated the pull request with a new target base due to a merge or a rebase. The pull request now contains two commits: - Merge branch 'master' into SKARA-1555-remove-emojis - SKARA-1555 ------------- Changes: https://git.openjdk.org/skara/pull/1363/files Webrev: https://webrevs.openjdk.org/?repo=skara&pr=1363&range=01 Stats: 220 lines in 10 files changed: 0 ins; 215 del; 5 mod Patch: https://git.openjdk.org/skara/pull/1363.diff Fetch: git fetch https://git.openjdk.org/skara pull/1363/head:pull/1363 PR: https://git.openjdk.org/skara/pull/1363 From erikj at openjdk.org Tue Aug 30 18:40:23 2022 From: erikj at openjdk.org (Erik Joelsson) Date: Tue, 30 Aug 2022 18:40:23 GMT Subject: Integrated: 1555: Remove support for review by award emojis in GitLab In-Reply-To: References: Message-ID: On Thu, 25 Aug 2022 18:45:31 GMT, Erik Joelsson wrote: > This patch removes support for using award emojis for marking an MR in GitLab as reviewed. This was the original Skara implementation from back when GitLab did not have an `approve` button. In https://github.com/openjdk/skara/pull/939, support for the new approve button was introduced and the award emojis were deprecated. I think it's now time to remove them completely. > > The removed test had an assumeTrue that made it never run, as the TestForge had support for review body, so I chose to remove the whole test. This pull request has now been integrated. Changeset: 92d5c034 Author: Erik Joelsson URL: https://git.openjdk.org/skara/commit/92d5c0343e81528c5d6bfd0deacfa192e3bbb8b6 Stats: 220 lines in 10 files changed: 0 ins; 215 del; 5 mod 1555: Remove support for review by award emojis in GitLab Reviewed-by: ihse ------------- PR: https://git.openjdk.org/skara/pull/1363 From gli at openjdk.org Wed Aug 31 11:31:44 2022 From: gli at openjdk.org (Guoxiong Li) Date: Wed, 31 Aug 2022 11:31:44 GMT Subject: RFR: 1199: Enforce maintainer approval in JBS before allowing to integrate backports into updates projects Message-ID: Hi all, This patch implements the feature we had discussed at the mail list and the issue SKARA-1199 [1], included mainly the proposal I submitted [2] and the suggestions and ideas other developers provided. The dev flow has been stated at my previous proposal [2]. Here, I mainly state the code implementation which are changed and newly added in this patch. ### The change in PullRequestBot - The config about approval and update change is added to the `PullRequestBot`. See below for the concrete config format. Please see the classes `PullRequestBotFactory`, `PullRequestBotBuilder` and `PullRequestBot` for the code. "repositories" : { "repo-name" : { // other fileds, ignored "approval" : [ { "branch" : "branch-pattern", "request-label" : "label-name1", "approval-label" : "label-name2", "disapproval-label" : "label-name3", "maintainers" : ["user1", "user2"] }, // other branch pattern and its information ] }, // other repo and its information } - A suggestion comment would be added to the update change pull request. Please see the method `CheckRun#addUpdateChangeSuggestionComment`. - Labels of the pull request and the issues would be updated according to the main issue state. Please see the method `CheckRun#updateLabelsForUpdatesChange` - Two commands `approval` and `request-approval` are added. Please see the classes `ApprovalCommand` and `RequestApprovalCommand`. The field `CommandExtractor#commandPattern` is also adjusted to identify the bar `-` in `request-approval`. - Some methods in class `CheckRun` are moved to class `PullRequestWorkItem` to let them reused in `PullRequestCommandWorkItem` and `CheckWorkItem`. - The `PullRequestWorkItem workItem` parameter is added to the method `CommandHandler#handle` so that the commands can use the work item and use the newly added and moved methods. And the `handle` method of the `CSRCommand` and `JEPCommand` need to adjust the parameter, too. - The field `CheckWorkItem#metadataComments` need to be adjusted to identify the tag `