From zsong at openjdk.org Mon Oct 2 18:11:03 2023 From: zsong at openjdk.org (Zhao Song) Date: Mon, 2 Oct 2023 18:11:03 GMT Subject: RFR: 2051: Use real target ref to determine whether this pr needs maintainer approval in CheckWorkItem Message-ID: <2R_zRzUQwy9ReH7UWbW-oneOUFDxuIAZJNFwaAq_YPw=.333afc8d-5212-4a11-a641-33e80f1bd557@github.com> After deployed [SKARA-2045](https://bugs.openjdk.org/browse/SKARA-2045) to production, I found that some dependent pull requests are still not updated. Then I realized that in CheckWorkItem, we also need to use real target ref to determine whether this pr needs maintainer approval. ------------- Commit messages: - SKARA-2051 Changes: https://git.openjdk.org/skara/pull/1566/files Webrev: https://webrevs.openjdk.org/?repo=skara&pr=1566&range=00 Issue: https://bugs.openjdk.org/browse/SKARA-2051 Stats: 1 line in 1 file changed: 0 ins; 0 del; 1 mod Patch: https://git.openjdk.org/skara/pull/1566.diff Fetch: git fetch https://git.openjdk.org/skara.git pull/1566/head:pull/1566 PR: https://git.openjdk.org/skara/pull/1566 From erikj at openjdk.org Mon Oct 2 18:45:51 2023 From: erikj at openjdk.org (Erik Joelsson) Date: Mon, 2 Oct 2023 18:45:51 GMT Subject: RFR: 2051: Use real target ref to determine whether this pr needs maintainer approval in CheckWorkItem In-Reply-To: <2R_zRzUQwy9ReH7UWbW-oneOUFDxuIAZJNFwaAq_YPw=.333afc8d-5212-4a11-a641-33e80f1bd557@github.com> References: <2R_zRzUQwy9ReH7UWbW-oneOUFDxuIAZJNFwaAq_YPw=.333afc8d-5212-4a11-a641-33e80f1bd557@github.com> Message-ID: On Mon, 2 Oct 2023 17:46:24 GMT, Zhao Song wrote: > After deployed [SKARA-2045](https://bugs.openjdk.org/browse/SKARA-2045) to production, I found that some dependent pull requests are still not updated. Then I realized that in CheckWorkItem, we also need to use real target ref to determine whether this pr needs maintainer approval. Marked as reviewed by erikj (Lead). ------------- PR Review: https://git.openjdk.org/skara/pull/1566#pullrequestreview-1653293437 From zsong at openjdk.org Mon Oct 2 18:50:41 2023 From: zsong at openjdk.org (Zhao Song) Date: Mon, 2 Oct 2023 18:50:41 GMT Subject: Integrated: 2051: Use real target ref to determine whether this pr needs maintainer approval in CheckWorkItem In-Reply-To: <2R_zRzUQwy9ReH7UWbW-oneOUFDxuIAZJNFwaAq_YPw=.333afc8d-5212-4a11-a641-33e80f1bd557@github.com> References: <2R_zRzUQwy9ReH7UWbW-oneOUFDxuIAZJNFwaAq_YPw=.333afc8d-5212-4a11-a641-33e80f1bd557@github.com> Message-ID: On Mon, 2 Oct 2023 17:46:24 GMT, Zhao Song wrote: > After deployed [SKARA-2045](https://bugs.openjdk.org/browse/SKARA-2045) to production, I found that some dependent pull requests are still not updated. Then I realized that in CheckWorkItem, we also need to use real target ref to determine whether this pr needs maintainer approval. This pull request has now been integrated. Changeset: a85bb60c Author: Zhao Song URL: https://git.openjdk.org/skara/commit/a85bb60c288e17e16ccbc5534e30dfc227f1cac0 Stats: 1 line in 1 file changed: 0 ins; 0 del; 1 mod 2051: Use real target ref to determine whether this pr needs maintainer approval in CheckWorkItem Reviewed-by: erikj ------------- PR: https://git.openjdk.org/skara/pull/1566 From sgehwolf at openjdk.org Tue Oct 3 08:49:54 2023 From: sgehwolf at openjdk.org (Severin Gehwolf) Date: Tue, 3 Oct 2023 08:49:54 GMT Subject: RFR: 2049: After enabled maintainer approval in a repo, some pull requests are not updated [v2] In-Reply-To: References: Message-ID: On Thu, 28 Sep 2023 18:51:20 GMT, Zhao Song wrote: >> Today, I enabled maintainer approval feature in openjdk/jdk8u-dev. >> >> After I enabled this feature, I expect to see the progress like "JDK-XXXXXXX needs maintainer approval" in the body of all open prs. >> >> However, this pr(https://github.com/openjdk/jdk8u-dev/pull/368) is not updated. >> >> The reason is that although after we enabled the maintainer approval feature, the issue labels will be calculated in the issueMetaData, there still exists some issues that doesn't contain any label. >> >> To solve this, I think we could add a string "approval" to issueMetaData after we enabled the maintainer approval. > > Zhao Song has updated the pull request incrementally with one additional commit since the last revision: > > Update bots/pr/src/main/java/org/openjdk/skara/bots/pr/CheckWorkItem.java > > Co-authored-by: Erik Joelsson <37597443+erikj79 at users.noreply.github.com> bots/pr/src/main/java/org/openjdk/skara/bots/pr/CheckWorkItem.java line 217: > 215: } > 216: if (bot.approval() != null && bot.approval().needsApproval(pr.targetRef())) { > 217: // Add a static sting to the metadata if the PR needs approval to force Typo: `string` instead of `sting`? ------------- PR Review Comment: https://git.openjdk.org/skara/pull/1563#discussion_r1343740419 From zsong at openjdk.org Tue Oct 3 16:00:26 2023 From: zsong at openjdk.org (Zhao Song) Date: Tue, 3 Oct 2023 16:00:26 GMT Subject: RFR: 2049: After enabled maintainer approval in a repo, some pull requests are not updated [v2] In-Reply-To: References: Message-ID: <5j8DH0x4KxjsihdyWcCNDix5_o8GLHTPcxbKSuFaryw=.3980d06a-029d-415c-bfd5-190ae94f5208@github.com> On Tue, 3 Oct 2023 08:47:44 GMT, Severin Gehwolf wrote: >> Zhao Song has updated the pull request incrementally with one additional commit since the last revision: >> >> Update bots/pr/src/main/java/org/openjdk/skara/bots/pr/CheckWorkItem.java >> >> Co-authored-by: Erik Joelsson <37597443+erikj79 at users.noreply.github.com> > > bots/pr/src/main/java/org/openjdk/skara/bots/pr/CheckWorkItem.java line 217: > >> 215: } >> 216: if (bot.approval() != null && bot.approval().needsApproval(pr.targetRef())) { >> 217: // Add a static sting to the metadata if the PR needs approval to force > > Typo: `string` instead of `sting`? Thanks! I will fix it later. ------------- PR Review Comment: https://git.openjdk.org/skara/pull/1563#discussion_r1344346997 From erikj at openjdk.org Tue Oct 10 18:57:30 2023 From: erikj at openjdk.org (Erik Joelsson) Date: Tue, 10 Oct 2023 18:57:30 GMT Subject: RFR: 2056: Improve logging of external commands Message-ID: In order to make it easier to analyze bot behavior, I would like to add the working directory to log messages when running external commands. Most external commands are git performing operations on a local repository and by including the directory in the log message, we can easily see which repository the git command is operating on. ------------- Commit messages: - SKARA-2056 Changes: https://git.openjdk.org/skara/pull/1567/files Webrev: https://webrevs.openjdk.org/?repo=skara&pr=1567&range=00 Issue: https://bugs.openjdk.org/browse/SKARA-2056 Stats: 3 lines in 1 file changed: 1 ins; 0 del; 2 mod Patch: https://git.openjdk.org/skara/pull/1567.diff Fetch: git fetch https://git.openjdk.org/skara.git pull/1567/head:pull/1567 PR: https://git.openjdk.org/skara/pull/1567 From zsong at openjdk.org Tue Oct 10 18:57:30 2023 From: zsong at openjdk.org (Zhao Song) Date: Tue, 10 Oct 2023 18:57:30 GMT Subject: RFR: 2056: Improve logging of external commands In-Reply-To: References: Message-ID: On Tue, 10 Oct 2023 18:45:21 GMT, Erik Joelsson wrote: > In order to make it easier to analyze bot behavior, I would like to add the working directory to log messages when running external commands. Most external commands are git performing operations on a local repository and by including the directory in the log message, we can easily see which repository the git command is operating on. Marked as reviewed by zsong (Reviewer). ------------- PR Review: https://git.openjdk.org/skara/pull/1567#pullrequestreview-1668617475 From ihse at openjdk.org Tue Oct 10 19:03:07 2023 From: ihse at openjdk.org (Magnus Ihse Bursie) Date: Tue, 10 Oct 2023 19:03:07 GMT Subject: RFR: 2056: Improve logging of external commands In-Reply-To: References: Message-ID: On Tue, 10 Oct 2023 18:45:21 GMT, Erik Joelsson wrote: > In order to make it easier to analyze bot behavior, I would like to add the working directory to log messages when running external commands. Most external commands are git performing operations on a local repository and by including the directory in the log message, we can easily see which repository the git command is operating on. Marked as reviewed by ihse (Reviewer). ------------- PR Review: https://git.openjdk.org/skara/pull/1567#pullrequestreview-1668670882 From zsong at openjdk.org Tue Oct 10 19:03:07 2023 From: zsong at openjdk.org (Zhao Song) Date: Tue, 10 Oct 2023 19:03:07 GMT Subject: RFR: 2056: Improve logging of external commands In-Reply-To: References: Message-ID: On Tue, 10 Oct 2023 18:45:21 GMT, Erik Joelsson wrote: > In order to make it easier to analyze bot behavior, I would like to add the working directory to log messages when running external commands. Most external commands are git performing operations on a local repository and by including the directory in the log message, we can easily see which repository the git command is operating on. I am surprised the GHA linux test failed, this patch only contains changes to log. ------------- PR Comment: https://git.openjdk.org/skara/pull/1567#issuecomment-1756057704 From erikj at openjdk.org Tue Oct 10 20:05:58 2023 From: erikj at openjdk.org (Erik Joelsson) Date: Tue, 10 Oct 2023 20:05:58 GMT Subject: RFR: 2056: Improve logging of external commands In-Reply-To: References: Message-ID: On Tue, 10 Oct 2023 18:59:08 GMT, Zhao Song wrote: > I am surprised the GHA linux test failed, this patch only contains changes to log. Running that particular test locally, I managed to get it to fail once just now. Not sure what's going wrong though. It did not take 10 seconds to run, so it's not a timeout. I'm guessing there is a race. ------------- PR Comment: https://git.openjdk.org/skara/pull/1567#issuecomment-1756141747 From zsong at openjdk.org Tue Oct 10 20:14:09 2023 From: zsong at openjdk.org (Zhao Song) Date: Tue, 10 Oct 2023 20:14:09 GMT Subject: RFR: 2056: Improve logging of external commands In-Reply-To: References: Message-ID: <0moNDrOfxBz3JP5psrkRDrYxuC4Gmkpt6TI6k0loyek=.fc4ed8f1-f524-4a94-a269-3422dc8b1f4d@github.com> On Tue, 10 Oct 2023 20:03:37 GMT, Erik Joelsson wrote: > > I am surprised the GHA linux test failed, this patch only contains changes to log. > > Running that particular test locally, I managed to get it to fail once just now. Not sure what's going wrong though. It did not take 10 seconds to run, so it's not a timeout. I'm guessing there is a race. Yes, this should be a bug since a long time ago, and it's surprising that it was not discovered until today. ------------- PR Comment: https://git.openjdk.org/skara/pull/1567#issuecomment-1756152868 From erikj at openjdk.org Tue Oct 10 20:40:33 2023 From: erikj at openjdk.org (Erik Joelsson) Date: Tue, 10 Oct 2023 20:40:33 GMT Subject: RFR: 2056: Improve logging of external commands In-Reply-To: <0moNDrOfxBz3JP5psrkRDrYxuC4Gmkpt6TI6k0loyek=.fc4ed8f1-f524-4a94-a269-3422dc8b1f4d@github.com> References: <0moNDrOfxBz3JP5psrkRDrYxuC4Gmkpt6TI6k0loyek=.fc4ed8f1-f524-4a94-a269-3422dc8b1f4d@github.com> Message-ID: On Tue, 10 Oct 2023 20:11:51 GMT, Zhao Song wrote: > > > I am surprised the GHA linux test failed, this patch only contains changes to log. > > > > > > Running that particular test locally, I managed to get it to fail once just now. Not sure what's going wrong though. It did not take 10 seconds to run, so it's not a timeout. I'm guessing there is a race. > > Yes, this should be a bug since a long time ago, and it's surprising that it was not discovered until today. Found the race and filed a bug. https://bugs.openjdk.org/browse/SKARA-2057 ------------- PR Comment: https://git.openjdk.org/skara/pull/1567#issuecomment-1756196432 From erikj at openjdk.org Tue Oct 10 20:40:33 2023 From: erikj at openjdk.org (Erik Joelsson) Date: Tue, 10 Oct 2023 20:40:33 GMT Subject: Integrated: 2056: Improve logging of external commands In-Reply-To: References: Message-ID: On Tue, 10 Oct 2023 18:45:21 GMT, Erik Joelsson wrote: > In order to make it easier to analyze bot behavior, I would like to add the working directory to log messages when running external commands. Most external commands are git performing operations on a local repository and by including the directory in the log message, we can easily see which repository the git command is operating on. This pull request has now been integrated. Changeset: 316c7e88 Author: Erik Joelsson URL: https://git.openjdk.org/skara/commit/316c7e88121822bd3e6a51fdb43568ba04c6faff Stats: 3 lines in 1 file changed: 1 ins; 0 del; 2 mod 2056: Improve logging of external commands Reviewed-by: zsong, ihse ------------- PR: https://git.openjdk.org/skara/pull/1567 From zsong at openjdk.org Thu Oct 12 14:44:32 2023 From: zsong at openjdk.org (Zhao Song) Date: Thu, 12 Oct 2023 14:44:32 GMT Subject: RFR: 2061: Checking for .jcheck/conf fails when pr/X branches are removed Message-ID: As Erik described in the issue, "The targetRefPRMap introduced in [SKARA-1937](https://bugs.openjdk.org/browse/SKARA-1937) isn't working well with the dependent PR feature. When a pr/X branch is removed, the map may still contains that branch which will cause getPeriodicItems to fail with exception when trying to get .jcheck/conf from the non existing branch.". To solve this issue, I think that every pr would only have one targetRef, so before the bot is trying to create an entry for the pr, the bot should delete any existing entry associated with that pr. Since the operations are all about HashMap, I believe they wouldn't significantly impact the bot's performance. ------------- Commit messages: - SKARA-2061 Changes: https://git.openjdk.org/skara/pull/1568/files Webrev: https://webrevs.openjdk.org/?repo=skara&pr=1568&range=00 Issue: https://bugs.openjdk.org/browse/SKARA-2061 Stats: 6 lines in 1 file changed: 6 ins; 0 del; 0 mod Patch: https://git.openjdk.org/skara/pull/1568.diff Fetch: git fetch https://git.openjdk.org/skara.git pull/1568/head:pull/1568 PR: https://git.openjdk.org/skara/pull/1568 From erikj at openjdk.org Thu Oct 12 15:01:18 2023 From: erikj at openjdk.org (Erik Joelsson) Date: Thu, 12 Oct 2023 15:01:18 GMT Subject: RFR: 2061: Checking for .jcheck/conf fails when pr/X branches are removed In-Reply-To: References: Message-ID: On Thu, 12 Oct 2023 14:16:22 GMT, Zhao Song wrote: > As Erik described in the issue, "The targetRefPRMap introduced in [SKARA-1937](https://bugs.openjdk.org/browse/SKARA-1937) isn't working well with the dependent PR feature. When a pr/X branch is removed, the map may still contains that branch which will cause getPeriodicItems to fail with exception when trying to get .jcheck/conf from the non existing branch.". > > To solve this issue, I think that every pr would only have one targetRef, so before the bot is trying to create an entry for the pr, the bot should delete any existing entry associated with that pr. > > Since the operations are all about HashMap, I believe they wouldn't significantly impact the bot's performance. bots/pr/src/main/java/org/openjdk/skara/bots/pr/PullRequestBot.java line 185: > 183: var prId = pr.id(); > 184: if (pr.isOpen()) { > 185: targetRefPRMap.keySet().forEach(key -> targetRefPRMap.get(key).remove(prId)); I think this could be simplified by just iterating over `targetRefPRMap.values()`. ------------- PR Review Comment: https://git.openjdk.org/skara/pull/1568#discussion_r1356953766 From zsong at openjdk.org Thu Oct 12 15:53:19 2023 From: zsong at openjdk.org (Zhao Song) Date: Thu, 12 Oct 2023 15:53:19 GMT Subject: RFR: 2061: Checking for .jcheck/conf fails when pr/X branches are removed [v2] In-Reply-To: References: Message-ID: > As Erik described in the issue, "The targetRefPRMap introduced in [SKARA-1937](https://bugs.openjdk.org/browse/SKARA-1937) isn't working well with the dependent PR feature. When a pr/X branch is removed, the map may still contains that branch which will cause getPeriodicItems to fail with exception when trying to get .jcheck/conf from the non existing branch.". > > To solve this issue, I think that every pr would only have one targetRef, so before the bot is trying to create an entry for the pr, the bot should delete any existing entry associated with that pr. > > Since the operations are all about HashMap, I believe they wouldn't significantly impact the bot's performance. Zhao Song has updated the pull request incrementally with one additional commit since the last revision: review comment ------------- Changes: - all: https://git.openjdk.org/skara/pull/1568/files - new: https://git.openjdk.org/skara/pull/1568/files/785adc64..d4f55dbb Webrevs: - full: https://webrevs.openjdk.org/?repo=skara&pr=1568&range=01 - incr: https://webrevs.openjdk.org/?repo=skara&pr=1568&range=00-01 Stats: 1 line in 1 file changed: 0 ins; 0 del; 1 mod Patch: https://git.openjdk.org/skara/pull/1568.diff Fetch: git fetch https://git.openjdk.org/skara.git pull/1568/head:pull/1568 PR: https://git.openjdk.org/skara/pull/1568 From zsong at openjdk.org Thu Oct 12 15:53:20 2023 From: zsong at openjdk.org (Zhao Song) Date: Thu, 12 Oct 2023 15:53:20 GMT Subject: RFR: 2061: Checking for .jcheck/conf fails when pr/X branches are removed [v2] In-Reply-To: References: Message-ID: On Thu, 12 Oct 2023 14:54:29 GMT, Erik Joelsson wrote: >> Zhao Song has updated the pull request incrementally with one additional commit since the last revision: >> >> review comment > > bots/pr/src/main/java/org/openjdk/skara/bots/pr/PullRequestBot.java line 185: > >> 183: var prId = pr.id(); >> 184: if (pr.isOpen()) { >> 185: targetRefPRMap.keySet().forEach(key -> targetRefPRMap.get(key).remove(prId)); > > I think this could be simplified by just iterating over `targetRefPRMap.values()`. Yes, it should be better. ------------- PR Review Comment: https://git.openjdk.org/skara/pull/1568#discussion_r1357027926 From erikj at openjdk.org Thu Oct 12 16:00:00 2023 From: erikj at openjdk.org (Erik Joelsson) Date: Thu, 12 Oct 2023 16:00:00 GMT Subject: RFR: 2061: Checking for .jcheck/conf fails when pr/X branches are removed [v2] In-Reply-To: References: Message-ID: On Thu, 12 Oct 2023 15:53:19 GMT, Zhao Song wrote: >> As Erik described in the issue, "The targetRefPRMap introduced in [SKARA-1937](https://bugs.openjdk.org/browse/SKARA-1937) isn't working well with the dependent PR feature. When a pr/X branch is removed, the map may still contains that branch which will cause getPeriodicItems to fail with exception when trying to get .jcheck/conf from the non existing branch.". >> >> To solve this issue, I think that every pr would only have one targetRef, so before the bot is trying to create an entry for the pr, the bot should delete any existing entry associated with that pr. >> >> Since the operations are all about HashMap, I believe they wouldn't significantly impact the bot's performance. > > Zhao Song has updated the pull request incrementally with one additional commit since the last revision: > > review comment Marked as reviewed by erikj (Lead). ------------- PR Review: https://git.openjdk.org/skara/pull/1568#pullrequestreview-1674553011 From zsong at openjdk.org Thu Oct 12 16:17:05 2023 From: zsong at openjdk.org (Zhao Song) Date: Thu, 12 Oct 2023 16:17:05 GMT Subject: RFR: 2061: Checking for .jcheck/conf fails when pr/X branches are removed [v2] In-Reply-To: References: Message-ID: On Thu, 12 Oct 2023 15:53:19 GMT, Zhao Song wrote: >> As Erik described in the issue, "The targetRefPRMap introduced in [SKARA-1937](https://bugs.openjdk.org/browse/SKARA-1937) isn't working well with the dependent PR feature. When a pr/X branch is removed, the map may still contains that branch which will cause getPeriodicItems to fail with exception when trying to get .jcheck/conf from the non existing branch.". >> >> To solve this issue, I think that every pr would only have one targetRef, so before the bot is trying to create an entry for the pr, the bot should delete any existing entry associated with that pr. >> >> Since the operations are all about HashMap, I believe they wouldn't significantly impact the bot's performance. > > Zhao Song has updated the pull request incrementally with one additional commit since the last revision: > > review comment Thanks for the review! ------------- PR Comment: https://git.openjdk.org/skara/pull/1568#issuecomment-1759938760 From zsong at openjdk.org Thu Oct 12 16:17:05 2023 From: zsong at openjdk.org (Zhao Song) Date: Thu, 12 Oct 2023 16:17:05 GMT Subject: Integrated: 2061: Checking for .jcheck/conf fails when pr/X branches are removed In-Reply-To: References: Message-ID: On Thu, 12 Oct 2023 14:16:22 GMT, Zhao Song wrote: > As Erik described in the issue, "The targetRefPRMap introduced in [SKARA-1937](https://bugs.openjdk.org/browse/SKARA-1937) isn't working well with the dependent PR feature. When a pr/X branch is removed, the map may still contains that branch which will cause getPeriodicItems to fail with exception when trying to get .jcheck/conf from the non existing branch.". > > To solve this issue, I think that every pr would only have one targetRef, so before the bot is trying to create an entry for the pr, the bot should delete any existing entry associated with that pr. > > Since the operations are all about HashMap, I believe they wouldn't significantly impact the bot's performance. This pull request has now been integrated. Changeset: 68e579f7 Author: Zhao Song URL: https://git.openjdk.org/skara/commit/68e579f723cefb9042e5266caf47cb42dc95e3e5 Stats: 6 lines in 1 file changed: 6 ins; 0 del; 0 mod 2061: Checking for .jcheck/conf fails when pr/X branches are removed Reviewed-by: erikj ------------- PR: https://git.openjdk.org/skara/pull/1568 From zsong at openjdk.org Thu Oct 12 21:21:12 2023 From: zsong at openjdk.org (Zhao Song) Date: Thu, 12 Oct 2023 21:21:12 GMT Subject: RFR: 2063: Reduce the health check times of local repo instance Message-ID: In the method HostedRepositoryPool#materializeClone, when the bot is trying to reuse a local repo instance, the bot will always check whether the repo is still good by processing command "git fsck --connectivity-only". Sometimes this command would be slow and Erik said that we should believe the bots are doing right things, so we should assume the local repos are good, so the health check is not always needed. But shutdown of the bot could make a local repo instance unhealthy, so we should at least do the health check once for each local repo instance after the bot is restarted. To solve this issue, Erik suggested to maintain a static map for keeping track of paths of known checked/good repositories. To leverage this patch to mlbridge bot, I also made some changes to WebrevStorage. ------------- Commit messages: - SKARA-2063 Changes: https://git.openjdk.org/skara/pull/1569/files Webrev: https://webrevs.openjdk.org/?repo=skara&pr=1569&range=00 Issue: https://bugs.openjdk.org/browse/SKARA-2063 Stats: 35 lines in 4 files changed: 22 ins; 4 del; 9 mod Patch: https://git.openjdk.org/skara/pull/1569.diff Fetch: git fetch https://git.openjdk.org/skara.git pull/1569/head:pull/1569 PR: https://git.openjdk.org/skara/pull/1569 From erikj at openjdk.org Thu Oct 12 22:08:34 2023 From: erikj at openjdk.org (Erik Joelsson) Date: Thu, 12 Oct 2023 22:08:34 GMT Subject: RFR: 2063: Reduce the health check times of local repo instance In-Reply-To: References: Message-ID: On Thu, 12 Oct 2023 20:50:17 GMT, Zhao Song wrote: > In the method HostedRepositoryPool#materializeClone, when the bot is trying to reuse a local repo instance, the bot will always check whether the repo is still good by processing command "git fsck --connectivity-only". > > Sometimes this command would be slow and Erik said that we should believe the bots are doing right things, so we should assume the local repos are good, so the health check is not always needed. But shutdown of the bot could make a local repo instance unhealthy, so we should at least do the health check once for each local repo instance after the bot is restarted. > > To solve this issue, Erik suggested to maintain a static map for keeping track of paths of known checked/good repositories. > > To leverage this patch to mlbridge bot, I also made some changes to WebrevStorage. bots/mlbridge/src/main/java/org/openjdk/skara/bots/mlbridge/WebrevStorage.java line 334: > 332: WebrevGenerator generator(PullRequest pr, Repository localRepository, Path scratchPath, HostedRepositoryPool hostedRepositoryPool) throws IOException { > 333: var jsonLocalStorage = jsonStorage == null ? null : hostedRepositoryPool.checkout(jsonStorage, storageRef, scratchPath); > 334: var htmlLocalStorage = htmlStorage == null ? null : hostedRepositoryPool.checkout(htmlStorage, storageRef, scratchPath); This will cause us to always materialize the repository, regardless of if webrevs will be generated. I still want it to be done lazily. You can add a method here that initializes the local storage fields and call it from the generate methods below. forge/src/main/java/org/openjdk/skara/forge/HostedRepositoryPool.java line 48: > 46: private final HostedRepository hostedRepository; > 47: private final Path seed; > 48: private static Set healthySet = new HashSet<>(); Should probably store `Path` in the set so we don't need to convert to string. forge/src/main/java/org/openjdk/skara/forge/HostedRepositoryPool.java line 153: > 151: if (!isHealthy(localRepoInstance, path)) { > 152: removeOldClone(path, "unhealthy"); > 153: return cloneSeeded(path, allowStale, bare); I think we should also add a path to the healthy set after a successful clone. Perhaps at the end of `cloneSeeded`. ------------- PR Review Comment: https://git.openjdk.org/skara/pull/1569#discussion_r1357470918 PR Review Comment: https://git.openjdk.org/skara/pull/1569#discussion_r1357460471 PR Review Comment: https://git.openjdk.org/skara/pull/1569#discussion_r1357464372 From zsong at openjdk.org Thu Oct 12 22:46:29 2023 From: zsong at openjdk.org (Zhao Song) Date: Thu, 12 Oct 2023 22:46:29 GMT Subject: RFR: 2063: Reduce the health check times of local repo instance In-Reply-To: References: Message-ID: On Thu, 12 Oct 2023 21:50:25 GMT, Erik Joelsson wrote: >> In the method HostedRepositoryPool#materializeClone, when the bot is trying to reuse a local repo instance, the bot will always check whether the repo is still good by processing command "git fsck --connectivity-only". >> >> Sometimes this command would be slow and Erik said that we should believe the bots are doing right things, so we should assume the local repos are good, so the health check is not always needed. But shutdown of the bot could make a local repo instance unhealthy, so we should at least do the health check once for each local repo instance after the bot is restarted. >> >> To solve this issue, Erik suggested to maintain a static map for keeping track of paths of known checked/good repositories. >> >> To leverage this patch to mlbridge bot, I also made some changes to WebrevStorage. > > forge/src/main/java/org/openjdk/skara/forge/HostedRepositoryPool.java line 153: > >> 151: if (!isHealthy(localRepoInstance, path)) { >> 152: removeOldClone(path, "unhealthy"); >> 153: return cloneSeeded(path, allowStale, bare); > > I think we should also add a path to the healthy set after a successful clone. Perhaps at the end of `cloneSeeded`. Yes, good idea. ------------- PR Review Comment: https://git.openjdk.org/skara/pull/1569#discussion_r1357514401 From zsong at openjdk.org Thu Oct 12 23:20:01 2023 From: zsong at openjdk.org (Zhao Song) Date: Thu, 12 Oct 2023 23:20:01 GMT Subject: RFR: 2063: Reduce the health check times of local repo instance [v2] In-Reply-To: References: Message-ID: > In the method HostedRepositoryPool#materializeClone, when the bot is trying to reuse a local repo instance, the bot will always check whether the repo is still good by processing command "git fsck --connectivity-only". > > Sometimes this command would be slow and Erik said that we should believe the bots are doing right things, so we should assume the local repos are good, so the health check is not always needed. But shutdown of the bot could make a local repo instance unhealthy, so we should at least do the health check once for each local repo instance after the bot is restarted. > > To solve this issue, Erik suggested to maintain a static map for keeping track of paths of known checked/good repositories. > > To leverage this patch to mlbridge bot, I also made some changes to WebrevStorage. Zhao Song has updated the pull request incrementally with one additional commit since the last revision: review comment ------------- Changes: - all: https://git.openjdk.org/skara/pull/1569/files - new: https://git.openjdk.org/skara/pull/1569/files/9345fbd2..e58482b0 Webrevs: - full: https://webrevs.openjdk.org/?repo=skara&pr=1569&range=01 - incr: https://webrevs.openjdk.org/?repo=skara&pr=1569&range=00-01 Stats: 30 lines in 2 files changed: 23 ins; 3 del; 4 mod Patch: https://git.openjdk.org/skara/pull/1569.diff Fetch: git fetch https://git.openjdk.org/skara.git pull/1569/head:pull/1569 PR: https://git.openjdk.org/skara/pull/1569 From zsong at openjdk.org Thu Oct 12 23:29:36 2023 From: zsong at openjdk.org (Zhao Song) Date: Thu, 12 Oct 2023 23:29:36 GMT Subject: RFR: 2063: Reduce the health check times of local repo instance [v3] In-Reply-To: References: Message-ID: > In the method HostedRepositoryPool#materializeClone, when the bot is trying to reuse a local repo instance, the bot will always check whether the repo is still good by processing command "git fsck --connectivity-only". > > Sometimes this command would be slow and Erik said that we should believe the bots are doing right things, so we should assume the local repos are good, so the health check is not always needed. But shutdown of the bot could make a local repo instance unhealthy, so we should at least do the health check once for each local repo instance after the bot is restarted. > > To solve this issue, Erik suggested to maintain a static map for keeping track of paths of known checked/good repositories. > > To leverage this patch to mlbridge bot, I also made some changes to WebrevStorage. Zhao Song has updated the pull request incrementally with one additional commit since the last revision: update ------------- Changes: - all: https://git.openjdk.org/skara/pull/1569/files - new: https://git.openjdk.org/skara/pull/1569/files/e58482b0..0722a701 Webrevs: - full: https://webrevs.openjdk.org/?repo=skara&pr=1569&range=02 - incr: https://webrevs.openjdk.org/?repo=skara&pr=1569&range=01-02 Stats: 3 lines in 1 file changed: 2 ins; 0 del; 1 mod Patch: https://git.openjdk.org/skara/pull/1569.diff Fetch: git fetch https://git.openjdk.org/skara.git pull/1569/head:pull/1569 PR: https://git.openjdk.org/skara/pull/1569 From erikj at openjdk.org Thu Oct 12 23:32:04 2023 From: erikj at openjdk.org (Erik Joelsson) Date: Thu, 12 Oct 2023 23:32:04 GMT Subject: RFR: 2063: Reduce the health check times of local repo instance [v3] In-Reply-To: References: Message-ID: On Thu, 12 Oct 2023 23:29:36 GMT, Zhao Song wrote: >> In the method HostedRepositoryPool#materializeClone, when the bot is trying to reuse a local repo instance, the bot will always check whether the repo is still good by processing command "git fsck --connectivity-only". >> >> Sometimes this command would be slow and Erik said that we should believe the bots are doing right things, so we should assume the local repos are good, so the health check is not always needed. But shutdown of the bot could make a local repo instance unhealthy, so we should at least do the health check once for each local repo instance after the bot is restarted. >> >> To solve this issue, Erik suggested to maintain a static map for keeping track of paths of known checked/good repositories. >> >> To leverage this patch to mlbridge bot, I also made some changes to WebrevStorage. > > Zhao Song has updated the pull request incrementally with one additional commit since the last revision: > > update bots/mlbridge/src/main/java/org/openjdk/skara/bots/mlbridge/WebrevStorage.java line 342: > 340: jsonLocalStorage = hostedRepositoryPool.checkout(jsonStorage, storageRef, scratchPath); > 341: } catch (IOException e) { > 342: throw new RuntimeException(e); This would throw `UncheckedIOException` before so should still do that. Can also consider just having one try-catch in the method, surrounding all of it, to avoid repeating the catch clause. ------------- PR Review Comment: https://git.openjdk.org/skara/pull/1569#discussion_r1357551428 From zsong at openjdk.org Thu Oct 12 23:43:07 2023 From: zsong at openjdk.org (Zhao Song) Date: Thu, 12 Oct 2023 23:43:07 GMT Subject: RFR: 2063: Reduce the health check times of local repo instance [v4] In-Reply-To: References: Message-ID: > In the method HostedRepositoryPool#materializeClone, when the bot is trying to reuse a local repo instance, the bot will always check whether the repo is still good by processing command "git fsck --connectivity-only". > > Sometimes this command would be slow and Erik said that we should believe the bots are doing right things, so we should assume the local repos are good, so the health check is not always needed. But shutdown of the bot could make a local repo instance unhealthy, so we should at least do the health check once for each local repo instance after the bot is restarted. > > To solve this issue, Erik suggested to maintain a static map for keeping track of paths of known checked/good repositories. > > To leverage this patch to mlbridge bot, I also made some changes to WebrevStorage. Zhao Song has updated the pull request incrementally with one additional commit since the last revision: fix an issue ------------- Changes: - all: https://git.openjdk.org/skara/pull/1569/files - new: https://git.openjdk.org/skara/pull/1569/files/0722a701..a66a3daa Webrevs: - full: https://webrevs.openjdk.org/?repo=skara&pr=1569&range=03 - incr: https://webrevs.openjdk.org/?repo=skara&pr=1569&range=02-03 Stats: 11 lines in 1 file changed: 2 ins; 6 del; 3 mod Patch: https://git.openjdk.org/skara/pull/1569.diff Fetch: git fetch https://git.openjdk.org/skara.git pull/1569/head:pull/1569 PR: https://git.openjdk.org/skara/pull/1569 From zsong at openjdk.org Thu Oct 12 23:43:07 2023 From: zsong at openjdk.org (Zhao Song) Date: Thu, 12 Oct 2023 23:43:07 GMT Subject: RFR: 2063: Reduce the health check times of local repo instance [v3] In-Reply-To: References: Message-ID: On Thu, 12 Oct 2023 23:28:00 GMT, Erik Joelsson wrote: >> Zhao Song has updated the pull request incrementally with one additional commit since the last revision: >> >> update > > bots/mlbridge/src/main/java/org/openjdk/skara/bots/mlbridge/WebrevStorage.java line 342: > >> 340: jsonLocalStorage = hostedRepositoryPool.checkout(jsonStorage, storageRef, scratchPath); >> 341: } catch (IOException e) { >> 342: throw new RuntimeException(e); > > This would throw `UncheckedIOException` before so should still do that. Can also consider just having one try-catch in the method, surrounding all of it, to avoid repeating the catch clause. You are right, thanks for catching it! ------------- PR Review Comment: https://git.openjdk.org/skara/pull/1569#discussion_r1357559329 From erikj at openjdk.org Fri Oct 13 13:52:53 2023 From: erikj at openjdk.org (Erik Joelsson) Date: Fri, 13 Oct 2023 13:52:53 GMT Subject: RFR: 2063: Reduce the health check times of local repo instance [v4] In-Reply-To: References: Message-ID: On Thu, 12 Oct 2023 23:43:07 GMT, Zhao Song wrote: >> In the method HostedRepositoryPool#materializeClone, when the bot is trying to reuse a local repo instance, the bot will always check whether the repo is still good by processing command "git fsck --connectivity-only". >> >> Sometimes this command would be slow and Erik said that we should believe the bots are doing right things, so we should assume the local repos are good, so the health check is not always needed. But shutdown of the bot could make a local repo instance unhealthy, so we should at least do the health check once for each local repo instance after the bot is restarted. >> >> To solve this issue, Erik suggested to maintain a static map for keeping track of paths of known checked/good repositories. >> >> To leverage this patch to mlbridge bot, I also made some changes to WebrevStorage. > > Zhao Song has updated the pull request incrementally with one additional commit since the last revision: > > fix an issue Marked as reviewed by erikj (Lead). ------------- PR Review: https://git.openjdk.org/skara/pull/1569#pullrequestreview-1676553050 From zsong at openjdk.org Fri Oct 13 16:09:23 2023 From: zsong at openjdk.org (Zhao Song) Date: Fri, 13 Oct 2023 16:09:23 GMT Subject: RFR: 2063: Reduce the health check times of local repo instance [v4] In-Reply-To: References: Message-ID: <10F515RUvx3jOOc9rO7RvOv6hMdyToLif41hfibnWY8=.360023c1-4ca7-4e92-ac26-641b29f296b2@github.com> On Thu, 12 Oct 2023 23:43:07 GMT, Zhao Song wrote: >> In the method HostedRepositoryPool#materializeClone, when the bot is trying to reuse a local repo instance, the bot will always check whether the repo is still good by processing command "git fsck --connectivity-only". >> >> Sometimes this command would be slow and Erik said that we should believe the bots are doing right things, so we should assume the local repos are good, so the health check is not always needed. But shutdown of the bot could make a local repo instance unhealthy, so we should at least do the health check once for each local repo instance after the bot is restarted. >> >> To solve this issue, Erik suggested to maintain a static map for keeping track of paths of known checked/good repositories. >> >> To leverage this patch to mlbridge bot, I also made some changes to WebrevStorage. > > Zhao Song has updated the pull request incrementally with one additional commit since the last revision: > > fix an issue Thanks for the review and suggestions! ------------- PR Comment: https://git.openjdk.org/skara/pull/1569#issuecomment-1761760324 From zsong at openjdk.org Fri Oct 13 16:09:23 2023 From: zsong at openjdk.org (Zhao Song) Date: Fri, 13 Oct 2023 16:09:23 GMT Subject: Integrated: 2063: Reduce the health check times of local repo instance In-Reply-To: References: Message-ID: On Thu, 12 Oct 2023 20:50:17 GMT, Zhao Song wrote: > In the method HostedRepositoryPool#materializeClone, when the bot is trying to reuse a local repo instance, the bot will always check whether the repo is still good by processing command "git fsck --connectivity-only". > > Sometimes this command would be slow and Erik said that we should believe the bots are doing right things, so we should assume the local repos are good, so the health check is not always needed. But shutdown of the bot could make a local repo instance unhealthy, so we should at least do the health check once for each local repo instance after the bot is restarted. > > To solve this issue, Erik suggested to maintain a static map for keeping track of paths of known checked/good repositories. > > To leverage this patch to mlbridge bot, I also made some changes to WebrevStorage. This pull request has now been integrated. Changeset: b2abbc89 Author: Zhao Song URL: https://git.openjdk.org/skara/commit/b2abbc896cf532420a90aebd2e5aaf8cae720738 Stats: 55 lines in 4 files changed: 41 ins; 5 del; 9 mod 2063: Reduce the health check times of local repo instance Reviewed-by: erikj ------------- PR: https://git.openjdk.org/skara/pull/1569 From zsong at openjdk.org Fri Oct 13 18:25:21 2023 From: zsong at openjdk.org (Zhao Song) Date: Fri, 13 Oct 2023 18:25:21 GMT Subject: RFR: 2064: Create jsonLocalStorage and htmlLocalStorage in separate directories Message-ID: <6dumCH4v05FbrlNapVG3Flh4JuOW1FlcL6yiTnEgmGE=.ce6db744-7d44-4a28-bf16-eb8180145872@github.com> In [SKARA-2063](https://bugs.openjdk.org/browse/SKARA-2063), I tried to let WebrevStorage to use HostedRepositoryPool, but I made a mistake. When the bot is trying to create a local clone, it will try to create jsonLocalStorage and htmlLocalStorage in the same directory. And previously, the bot will only create jsonLocalStorage when generateJson is true and create htmlLocalStorage when generateHtml is true. Now, the bot will create both clones when webrevGenerator.generate is called. To solve this problem, I think the bot should create jsonLocalStorage and htmlLocalStorage in separate directories and the bot should only create the local clone when it's necessary. ------------- Commit messages: - SKARA-2064 Changes: https://git.openjdk.org/skara/pull/1570/files Webrev: https://webrevs.openjdk.org/?repo=skara&pr=1570&range=00 Issue: https://bugs.openjdk.org/browse/SKARA-2064 Stats: 27 lines in 3 files changed: 8 ins; 1 del; 18 mod Patch: https://git.openjdk.org/skara/pull/1570.diff Fetch: git fetch https://git.openjdk.org/skara.git pull/1570/head:pull/1570 PR: https://git.openjdk.org/skara/pull/1570 From erikj at openjdk.org Fri Oct 13 19:43:28 2023 From: erikj at openjdk.org (Erik Joelsson) Date: Fri, 13 Oct 2023 19:43:28 GMT Subject: RFR: 2064: Create jsonLocalStorage and htmlLocalStorage in separate directories In-Reply-To: <6dumCH4v05FbrlNapVG3Flh4JuOW1FlcL6yiTnEgmGE=.ce6db744-7d44-4a28-bf16-eb8180145872@github.com> References: <6dumCH4v05FbrlNapVG3Flh4JuOW1FlcL6yiTnEgmGE=.ce6db744-7d44-4a28-bf16-eb8180145872@github.com> Message-ID: <07VcktsmXcjGxSk9LVoKwrW62mB8BZeUOJ43SQIlSPA=.890f9693-7445-4327-8e56-b348dea7db52@github.com> On Fri, 13 Oct 2023 18:20:30 GMT, Zhao Song wrote: > In [SKARA-2063](https://bugs.openjdk.org/browse/SKARA-2063), I tried to let WebrevStorage to use HostedRepositoryPool, but I made a mistake. When the bot is trying to create a local clone, it will try to create jsonLocalStorage and htmlLocalStorage in the same directory. And previously, the bot will only create jsonLocalStorage when generateJson is true and create htmlLocalStorage when generateHtml is true. Now, the bot will create both clones when webrevGenerator#generate is called. > > To solve this problem, I think the bot should create jsonLocalStorage and htmlLocalStorage in separate directories and the bot should only create the local clone when it's necessary. Marked as reviewed by erikj (Lead). ------------- PR Review: https://git.openjdk.org/skara/pull/1570#pullrequestreview-1677247704 From zsong at openjdk.org Fri Oct 13 19:46:48 2023 From: zsong at openjdk.org (Zhao Song) Date: Fri, 13 Oct 2023 19:46:48 GMT Subject: RFR: 2064: Create jsonLocalStorage and htmlLocalStorage in separate directories In-Reply-To: <6dumCH4v05FbrlNapVG3Flh4JuOW1FlcL6yiTnEgmGE=.ce6db744-7d44-4a28-bf16-eb8180145872@github.com> References: <6dumCH4v05FbrlNapVG3Flh4JuOW1FlcL6yiTnEgmGE=.ce6db744-7d44-4a28-bf16-eb8180145872@github.com> Message-ID: On Fri, 13 Oct 2023 18:20:30 GMT, Zhao Song wrote: > In [SKARA-2063](https://bugs.openjdk.org/browse/SKARA-2063), I tried to let WebrevStorage to use HostedRepositoryPool, but I made a mistake. When the bot is trying to create a local clone, it will try to create jsonLocalStorage and htmlLocalStorage in the same directory. And previously, the bot will only create jsonLocalStorage when generateJson is true and create htmlLocalStorage when generateHtml is true. Now, the bot will create both clones when webrevGenerator#generate is called. > > To solve this problem, I think the bot should create jsonLocalStorage and htmlLocalStorage in separate directories and the bot should only create the local clone when it's necessary. Thanks for the review! ------------- PR Comment: https://git.openjdk.org/skara/pull/1570#issuecomment-1762116202 From zsong at openjdk.org Fri Oct 13 19:46:48 2023 From: zsong at openjdk.org (Zhao Song) Date: Fri, 13 Oct 2023 19:46:48 GMT Subject: Integrated: 2064: Create jsonLocalStorage and htmlLocalStorage in separate directories In-Reply-To: <6dumCH4v05FbrlNapVG3Flh4JuOW1FlcL6yiTnEgmGE=.ce6db744-7d44-4a28-bf16-eb8180145872@github.com> References: <6dumCH4v05FbrlNapVG3Flh4JuOW1FlcL6yiTnEgmGE=.ce6db744-7d44-4a28-bf16-eb8180145872@github.com> Message-ID: <99YYpT24XtIdZV7dhh9iy9jLnIT2Kd-6MYvVzJMZRpI=.2c4e93c6-31f5-4f40-9b18-57242055d910@github.com> On Fri, 13 Oct 2023 18:20:30 GMT, Zhao Song wrote: > In [SKARA-2063](https://bugs.openjdk.org/browse/SKARA-2063), I tried to let WebrevStorage to use HostedRepositoryPool, but I made a mistake. When the bot is trying to create a local clone, it will try to create jsonLocalStorage and htmlLocalStorage in the same directory. And previously, the bot will only create jsonLocalStorage when generateJson is true and create htmlLocalStorage when generateHtml is true. Now, the bot will create both clones when webrevGenerator#generate is called. > > To solve this problem, I think the bot should create jsonLocalStorage and htmlLocalStorage in separate directories and the bot should only create the local clone when it's necessary. This pull request has now been integrated. Changeset: 1c9c6b39 Author: Zhao Song URL: https://git.openjdk.org/skara/commit/1c9c6b39cbcaaadf6b85d6c5650fd9433aa3e101 Stats: 27 lines in 3 files changed: 8 ins; 1 del; 18 mod 2064: Create jsonLocalStorage and htmlLocalStorage in separate directories Reviewed-by: erikj ------------- PR: https://git.openjdk.org/skara/pull/1570 From zsong at openjdk.org Wed Oct 18 21:49:24 2023 From: zsong at openjdk.org (Zhao Song) Date: Wed, 18 Oct 2023 21:49:24 GMT Subject: RFR: 2059: Improve error message when cli encounters 401 Message-ID: As Erik said in the issue "A user tried to run `git pr checkout` and got UncheckedRestException with 401. The repository in question had remotes pointing to a gitlab instance while the user was trying to get a PR from github. Not sure if that is relevant here. We should not fail with a stacktrace but should rather explain that we authorization failed and print which URL we tried to connect to." To not fail with a stack trace, in this patch, `UncheckedRestException` will be caught in the `GitSkara#main` and status code will be checked, if the status code is 401, the cli tool should print suitable error message to the user. ------------- Commit messages: - update - SKARA-2059 Changes: https://git.openjdk.org/skara/pull/1571/files Webrev: https://webrevs.openjdk.org/?repo=skara&pr=1571&range=00 Issue: https://bugs.openjdk.org/browse/SKARA-2059 Stats: 35 lines in 5 files changed: 22 ins; 0 del; 13 mod Patch: https://git.openjdk.org/skara/pull/1571.diff Fetch: git fetch https://git.openjdk.org/skara.git pull/1571/head:pull/1571 PR: https://git.openjdk.org/skara/pull/1571 From erikj at openjdk.org Thu Oct 19 14:09:09 2023 From: erikj at openjdk.org (Erik Joelsson) Date: Thu, 19 Oct 2023 14:09:09 GMT Subject: RFR: 2059: Improve error message when cli encounters 401 In-Reply-To: References: Message-ID: <1Q3Y-guJXNokszRP3MhBi3xhgM0dbYpBvszfUpflAug=.e6b9b425-3e63-4d6d-9b0d-2f58a44c0fc1@github.com> On Wed, 18 Oct 2023 21:25:13 GMT, Zhao Song wrote: > As Erik said in the issue "A user tried to run `git pr checkout` and got UncheckedRestException with 401. The repository in question had remotes pointing to a gitlab instance while the user was trying to get a PR from github. Not sure if that is relevant here. We should not fail with a stacktrace but should rather explain that we authorization failed and print which URL we tried to connect to." > > To not fail with a stack trace, in this patch, `UncheckedRestException` will be caught in the `GitSkara#main` and status code will be checked, if the status code is 401, the cli tool should print suitable error message to the user. Marked as reviewed by erikj (Lead). ------------- PR Review: https://git.openjdk.org/skara/pull/1571#pullrequestreview-1687846929 From zsong at openjdk.org Thu Oct 19 14:56:48 2023 From: zsong at openjdk.org (Zhao Song) Date: Thu, 19 Oct 2023 14:56:48 GMT Subject: RFR: 2059: Improve error message when cli encounters 401 In-Reply-To: References: Message-ID: On Wed, 18 Oct 2023 21:25:13 GMT, Zhao Song wrote: > As Erik said in the issue "A user tried to run `git pr checkout` and got UncheckedRestException with 401. The repository in question had remotes pointing to a gitlab instance while the user was trying to get a PR from github. Not sure if that is relevant here. We should not fail with a stacktrace but should rather explain that we authorization failed and print which URL we tried to connect to." > > To not fail with a stack trace, in this patch, `UncheckedRestException` will be caught in the `GitSkara#main` and status code will be checked, if the status code is 401, the cli tool should print suitable error message to the user. Thanks for the review! ------------- PR Comment: https://git.openjdk.org/skara/pull/1571#issuecomment-1771156626 From zsong at openjdk.org Thu Oct 19 15:01:24 2023 From: zsong at openjdk.org (Zhao Song) Date: Thu, 19 Oct 2023 15:01:24 GMT Subject: Integrated: 2059: Improve error message when cli encounters 401 In-Reply-To: References: Message-ID: On Wed, 18 Oct 2023 21:25:13 GMT, Zhao Song wrote: > As Erik said in the issue "A user tried to run `git pr checkout` and got UncheckedRestException with 401. The repository in question had remotes pointing to a gitlab instance while the user was trying to get a PR from github. Not sure if that is relevant here. We should not fail with a stacktrace but should rather explain that we authorization failed and print which URL we tried to connect to." > > To not fail with a stack trace, in this patch, `UncheckedRestException` will be caught in the `GitSkara#main` and status code will be checked, if the status code is 401, the cli tool should print suitable error message to the user. This pull request has now been integrated. Changeset: f3408c35 Author: Zhao Song URL: https://git.openjdk.org/skara/commit/f3408c35627b04ccf8638ced7be464ae542bbe10 Stats: 35 lines in 5 files changed: 22 ins; 0 del; 13 mod 2059: Improve error message when cli encounters 401 Reviewed-by: erikj ------------- PR: https://git.openjdk.org/skara/pull/1571 From zsong at openjdk.org Thu Oct 19 18:09:21 2023 From: zsong at openjdk.org (Zhao Song) Date: Thu, 19 Oct 2023 18:09:21 GMT Subject: RFR: 2068: Backport command should use default branch of repo as default Message-ID: When a user is using /backport command to create a backport and the user doesn't specify a target branch, skara bot would use "master" as a default target branch. However, in some repos, like lilliput-jdk17u and lilliput-jdk21u, the default branch is not "master". Therefore, it will be better to use the repository's default branch as the default target branch of /backport command. As Erik said, the value of default branch is already available in the repository JSON. So we just need to implement a `getDefaultBranchName` method. ------------- Commit messages: - SKARA-2068 Changes: https://git.openjdk.org/skara/pull/1572/files Webrev: https://webrevs.openjdk.org/?repo=skara&pr=1572&range=00 Issue: https://bugs.openjdk.org/browse/SKARA-2068 Stats: 46 lines in 9 files changed: 45 ins; 0 del; 1 mod Patch: https://git.openjdk.org/skara/pull/1572.diff Fetch: git fetch https://git.openjdk.org/skara.git pull/1572/head:pull/1572 PR: https://git.openjdk.org/skara/pull/1572 From erikj at openjdk.org Thu Oct 19 19:59:15 2023 From: erikj at openjdk.org (Erik Joelsson) Date: Thu, 19 Oct 2023 19:59:15 GMT Subject: RFR: 2068: Backport command should use default branch of repo as default In-Reply-To: References: Message-ID: <_g0RMpaj2TJYj6oKX_LR4enFY4vKDijMDg2cbhqOo8U=.edbf02c8-c95c-4eed-b91f-90696b370c94@github.com> On Thu, 19 Oct 2023 17:05:48 GMT, Zhao Song wrote: > When a user is using /backport command to create a backport and the user doesn't specify a target branch, skara bot would use "master" as a default target branch. However, in some repos, like lilliput-jdk17u and lilliput-jdk21u, the default branch is not "master". Therefore, it will be better to use the repository's default branch as the default target branch of /backport command. > > As Erik said, the value of default branch is already available in the repository JSON. So we just need to implement a `getDefaultBranchName` method. Marked as reviewed by erikj (Lead). ------------- PR Review: https://git.openjdk.org/skara/pull/1572#pullrequestreview-1688542589 From erikj at openjdk.org Thu Oct 19 21:23:35 2023 From: erikj at openjdk.org (Erik Joelsson) Date: Thu, 19 Oct 2023 21:23:35 GMT Subject: RFR: Draft: SKARA-2069 Message-ID: This patch adds some new methods on Forge and HostedRepository to interact with forge groups and repository collaborators respectively. It also adds a lot more possible customization to the test class `CensusBuilder` as well as adding some missing `final` declarations in test classes. All the new API calls have manual tests added. It's quite large for what it is, but isn't really affecting any current production code. ------------- Commit messages: - SKARA-2069 Changes: https://git.openjdk.org/skara/pull/1573/files Webrev: https://webrevs.openjdk.org/?repo=skara&pr=1573&range=00 Stats: 665 lines in 17 files changed: 466 ins; 117 del; 82 mod Patch: https://git.openjdk.org/skara/pull/1573.diff Fetch: git fetch https://git.openjdk.org/skara.git pull/1573/head:pull/1573 PR: https://git.openjdk.org/skara/pull/1573 From zsong at openjdk.org Thu Oct 19 20:20:36 2023 From: zsong at openjdk.org (Zhao Song) Date: Thu, 19 Oct 2023 20:20:36 GMT Subject: Integrated: 2068: Backport command should use default branch of repo as default In-Reply-To: References: Message-ID: On Thu, 19 Oct 2023 17:05:48 GMT, Zhao Song wrote: > When a user is using /backport command to create a backport and the user doesn't specify a target branch, skara bot would use "master" as a default target branch. However, in some repos, like lilliput-jdk17u and lilliput-jdk21u, the default branch is not "master". Therefore, it will be better to use the repository's default branch as the default target branch of /backport command. > > As Erik said, the value of default branch is already available in the repository JSON. So we just need to implement a `getDefaultBranchName` method. This pull request has now been integrated. Changeset: c8b57065 Author: Zhao Song URL: https://git.openjdk.org/skara/commit/c8b57065211143bf811dd26f2645c3f77986853d Stats: 46 lines in 9 files changed: 45 ins; 0 del; 1 mod 2068: Backport command should use default branch of repo as default Reviewed-by: erikj ------------- PR: https://git.openjdk.org/skara/pull/1572 From zsong at openjdk.org Thu Oct 19 20:20:36 2023 From: zsong at openjdk.org (Zhao Song) Date: Thu, 19 Oct 2023 20:20:36 GMT Subject: RFR: 2068: Backport command should use default branch of repo as default In-Reply-To: References: Message-ID: On Thu, 19 Oct 2023 17:05:48 GMT, Zhao Song wrote: > When a user is using /backport command to create a backport and the user doesn't specify a target branch, skara bot would use "master" as a default target branch. However, in some repos, like lilliput-jdk17u and lilliput-jdk21u, the default branch is not "master". Therefore, it will be better to use the repository's default branch as the default target branch of /backport command. > > As Erik said, the value of default branch is already available in the repository JSON. So we just need to implement a `getDefaultBranchName` method. Thanks for the review! ------------- PR Comment: https://git.openjdk.org/skara/pull/1572#issuecomment-1771645722 From zsong at openjdk.org Thu Oct 19 23:31:41 2023 From: zsong at openjdk.org (Zhao Song) Date: Thu, 19 Oct 2023 23:31:41 GMT Subject: RFR: 2069: Add group and collaborator methods to Forge and HostedRepository In-Reply-To: References: Message-ID: On Thu, 19 Oct 2023 21:19:35 GMT, Erik Joelsson wrote: > This patch adds some new methods on Forge and HostedRepository to interact with forge groups and repository collaborators respectively. It also adds a lot more possible customization to the test class `CensusBuilder` as well as adding some missing `final` declarations in test classes. All the new API calls have manual tests added. It's quite large for what it is, but isn't really affecting any current production code. I took a quick look, and it looks good, but we need to update the copyright and add copyright for new files. ------------- PR Comment: https://git.openjdk.org/skara/pull/1573#issuecomment-1771838008 From ihse at openjdk.org Fri Oct 20 13:07:54 2023 From: ihse at openjdk.org (Magnus Ihse Bursie) Date: Fri, 20 Oct 2023 13:07:54 GMT Subject: RFR: 2069: Add group and collaborator methods to Forge and HostedRepository In-Reply-To: References: Message-ID: On Thu, 19 Oct 2023 21:19:35 GMT, Erik Joelsson wrote: > This patch adds some new methods on Forge and HostedRepository to interact with forge groups and repository collaborators respectively. It also adds a lot more possible customization to the test class `CensusBuilder` as well as adding some missing `final` declarations in test classes. All the new API calls have manual tests added. It's quite large for what it is, but isn't really affecting any current production code. forge/src/test/java/org/openjdk/skara/forge/gitlab/GitLabRestApiTest.java line 69: > 67: > 68: @Test > 69: void testFilesUrl() throws IOException { Nice cleanup... ------------- PR Review Comment: https://git.openjdk.org/skara/pull/1573#discussion_r1366950434 From ihse at openjdk.org Fri Oct 20 13:12:10 2023 From: ihse at openjdk.org (Magnus Ihse Bursie) Date: Fri, 20 Oct 2023 13:12:10 GMT Subject: RFR: 2069: Add group and collaborator methods to Forge and HostedRepository In-Reply-To: References: Message-ID: On Thu, 19 Oct 2023 21:19:35 GMT, Erik Joelsson wrote: > This patch adds some new methods on Forge and HostedRepository to interact with forge groups and repository collaborators respectively. It also adds a lot more possible customization to the test class `CensusBuilder` as well as adding some missing `final` declarations in test classes. All the new API calls have manual tests added. It's quite large for what it is, but isn't really affecting any current production code. Looks good to me. ------------- Marked as reviewed by ihse (Reviewer). PR Review: https://git.openjdk.org/skara/pull/1573#pullrequestreview-1690007181 From erikj at openjdk.org Fri Oct 20 14:05:35 2023 From: erikj at openjdk.org (Erik Joelsson) Date: Fri, 20 Oct 2023 14:05:35 GMT Subject: RFR: 2069: Add group and collaborator methods to Forge and HostedRepository [v2] In-Reply-To: References: Message-ID: > This patch adds some new methods on Forge and HostedRepository to interact with forge groups and repository collaborators respectively. It also adds a lot more possible customization to the test class `CensusBuilder` as well as adding some missing `final` declarations in test classes. All the new API calls have manual tests added. It's quite large for what it is, but isn't really affecting any current production code. Erik Joelsson has updated the pull request incrementally with one additional commit since the last revision: copyright ------------- Changes: - all: https://git.openjdk.org/skara/pull/1573/files - new: https://git.openjdk.org/skara/pull/1573/files/17f3f8b4..0767eb2a Webrevs: - full: https://webrevs.openjdk.org/?repo=skara&pr=1573&range=01 - incr: https://webrevs.openjdk.org/?repo=skara&pr=1573&range=00-01 Stats: 48 lines in 6 files changed: 44 ins; 0 del; 4 mod Patch: https://git.openjdk.org/skara/pull/1573.diff Fetch: git fetch https://git.openjdk.org/skara.git pull/1573/head:pull/1573 PR: https://git.openjdk.org/skara/pull/1573 From zsong at openjdk.org Fri Oct 20 16:46:36 2023 From: zsong at openjdk.org (Zhao Song) Date: Fri, 20 Oct 2023 16:46:36 GMT Subject: RFR: 2069: Add group and collaborator methods to Forge and HostedRepository [v2] In-Reply-To: References: Message-ID: On Fri, 20 Oct 2023 14:05:35 GMT, Erik Joelsson wrote: >> This patch adds some new methods on Forge and HostedRepository to interact with forge groups and repository collaborators respectively. It also adds a lot more possible customization to the test class `CensusBuilder` as well as adding some missing `final` declarations in test classes. All the new API calls have manual tests added. It's quite large for what it is, but isn't really affecting any current production code. > > Erik Joelsson has updated the pull request incrementally with one additional commit since the last revision: > > copyright Looks very good! ------------- Marked as reviewed by zsong (Reviewer). PR Review: https://git.openjdk.org/skara/pull/1573#pullrequestreview-1690522014 From erikj at openjdk.org Fri Oct 20 17:31:17 2023 From: erikj at openjdk.org (Erik Joelsson) Date: Fri, 20 Oct 2023 17:31:17 GMT Subject: Integrated: 2069: Add group and collaborator methods to Forge and HostedRepository In-Reply-To: References: Message-ID: On Thu, 19 Oct 2023 21:19:35 GMT, Erik Joelsson wrote: > This patch adds some new methods on Forge and HostedRepository to interact with forge groups and repository collaborators respectively. It also adds a lot more possible customization to the test class `CensusBuilder` as well as adding some missing `final` declarations in test classes. All the new API calls have manual tests added. It's quite large for what it is, but isn't really affecting any current production code. This pull request has now been integrated. Changeset: 37c874ad Author: Erik Joelsson URL: https://git.openjdk.org/skara/commit/37c874ad9afaefefc7c613ebd6cd9d9c8bbcec76 Stats: 713 lines in 17 files changed: 510 ins; 117 del; 86 mod 2069: Add group and collaborator methods to Forge and HostedRepository Reviewed-by: ihse, zsong ------------- PR: https://git.openjdk.org/skara/pull/1573 From erikj at openjdk.org Fri Oct 20 21:26:16 2023 From: erikj at openjdk.org (Erik Joelsson) Date: Fri, 20 Oct 2023 21:26:16 GMT Subject: RFR: 2071: Track active status for users in IssueTracker Message-ID: This patch adds the concept of "active/inactive" users in a JiraHost and the ability to remove collaborators from a repository. I'm also adding test support where appropriate to be able to mock this in tests. ------------- Commit messages: - Add manual tests for removeCollaborator - Added manual jira test - Merge branch 'master' into SKARA-2052-inactive-users - WIP Changes: https://git.openjdk.org/skara/pull/1574/files Webrev: https://webrevs.openjdk.org/?repo=skara&pr=1574&range=00 Issue: https://bugs.openjdk.org/browse/SKARA-2071 Stats: 259 lines in 17 files changed: 214 ins; 20 del; 25 mod Patch: https://git.openjdk.org/skara/pull/1574.diff Fetch: git fetch https://git.openjdk.org/skara.git pull/1574/head:pull/1574 PR: https://git.openjdk.org/skara/pull/1574 From erikj at openjdk.org Fri Oct 20 22:38:42 2023 From: erikj at openjdk.org (Erik Joelsson) Date: Fri, 20 Oct 2023 22:38:42 GMT Subject: RFR: 2071: Track active status for users in IssueTracker [v2] In-Reply-To: References: Message-ID: <-Oc2xOQVTxoYRLIgm1GPuVhgmlw4AvvroL1yDJqo6Ow=.df657274-c5a3-4d92-950d-ff72d5acfeea@github.com> > This patch adds the concept of "active/inactive" users in a JiraHost and the ability to remove collaborators from a repository. I'm also adding test support where appropriate to be able to mock this in tests. Erik Joelsson has updated the pull request incrementally with one additional commit since the last revision: Copyright ------------- Changes: - all: https://git.openjdk.org/skara/pull/1574/files - new: https://git.openjdk.org/skara/pull/1574/files/81ad78b6..0387096f Webrevs: - full: https://webrevs.openjdk.org/?repo=skara&pr=1574&range=01 - incr: https://webrevs.openjdk.org/?repo=skara&pr=1574&range=00-01 Stats: 27 lines in 6 files changed: 22 ins; 0 del; 5 mod Patch: https://git.openjdk.org/skara/pull/1574.diff Fetch: git fetch https://git.openjdk.org/skara.git pull/1574/head:pull/1574 PR: https://git.openjdk.org/skara/pull/1574 From ihse at openjdk.org Mon Oct 23 11:55:46 2023 From: ihse at openjdk.org (Magnus Ihse Bursie) Date: Mon, 23 Oct 2023 11:55:46 GMT Subject: RFR: 2071: Track active status for users in IssueTracker [v2] In-Reply-To: <-Oc2xOQVTxoYRLIgm1GPuVhgmlw4AvvroL1yDJqo6Ow=.df657274-c5a3-4d92-950d-ff72d5acfeea@github.com> References: <-Oc2xOQVTxoYRLIgm1GPuVhgmlw4AvvroL1yDJqo6Ow=.df657274-c5a3-4d92-950d-ff72d5acfeea@github.com> Message-ID: On Fri, 20 Oct 2023 22:38:42 GMT, Erik Joelsson wrote: >> This patch adds the concept of "active/inactive" users in a JiraHost and the ability to remove collaborators from a repository. I'm also adding test support where appropriate to be able to mock this in tests. > > Erik Joelsson has updated the pull request incrementally with one additional commit since the last revision: > > Copyright Looks good. ------------- Marked as reviewed by ihse (Reviewer). PR Review: https://git.openjdk.org/skara/pull/1574#pullrequestreview-1692345640 From zsong at openjdk.org Mon Oct 23 16:37:15 2023 From: zsong at openjdk.org (Zhao Song) Date: Mon, 23 Oct 2023 16:37:15 GMT Subject: RFR: 2071: Track active status for users in IssueTracker [v2] In-Reply-To: <-Oc2xOQVTxoYRLIgm1GPuVhgmlw4AvvroL1yDJqo6Ow=.df657274-c5a3-4d92-950d-ff72d5acfeea@github.com> References: <-Oc2xOQVTxoYRLIgm1GPuVhgmlw4AvvroL1yDJqo6Ow=.df657274-c5a3-4d92-950d-ff72d5acfeea@github.com> Message-ID: <1j25rtmmxbY9MqPuBJXEt9LJbVuXcjInc_YYpjLMYGk=.4e617f1d-c0c0-4a16-b4cd-298eb4c1cbfe@github.com> On Fri, 20 Oct 2023 22:38:42 GMT, Erik Joelsson wrote: >> This patch adds the concept of "active/inactive" users in a JiraHost and the ability to remove collaborators from a repository. I'm also adding test support where appropriate to be able to mock this in tests. > > Erik Joelsson has updated the pull request incrementally with one additional commit since the last revision: > > Copyright LGTM ------------- Marked as reviewed by zsong (Reviewer). PR Review: https://git.openjdk.org/skara/pull/1574#pullrequestreview-1693014530 From zsong at openjdk.org Tue Oct 24 17:53:11 2023 From: zsong at openjdk.org (Zhao Song) Date: Tue, 24 Oct 2023 17:53:11 GMT Subject: RFR: 1962: JiraProject#queryIssues may return duplicate issues Message-ID: Since [SKARA-1912](https://bugs.openjdk.org/browse/SKARA-1912) deployed into prod, we have noticed occurrences of duplicate key exceptions in the IssueProjectPoller#updatedIssues. After investigation, we found that `JiraProject#queryIssues` may return duplicate issues and I think the pagination in this method is the root cause. The problem here is that, sometimes, issues will be updated during the pagination process. For example, the default page size is 50 and we assume that issue#50 is the last issue in the first page. And before querying the second page, another issue which not in the first page got updated, so issue#50 will also be the first issue in the second page. Therefore, the method may return duplicate issues. Erik's solution is right. We just need to order by updatedTime DESC and remove the duplicate issues. Ordering by updatedTime DESC would make sure that even some issues are updated during the pagination, the issues will be processed in the future round. ------------- Commit messages: - SKARA-1962 Changes: https://git.openjdk.org/skara/pull/1575/files Webrev: https://webrevs.openjdk.org/?repo=skara&pr=1575&range=00 Issue: https://bugs.openjdk.org/browse/SKARA-1962 Stats: 10 lines in 1 file changed: 3 ins; 0 del; 7 mod Patch: https://git.openjdk.org/skara/pull/1575.diff Fetch: git fetch https://git.openjdk.org/skara.git pull/1575/head:pull/1575 PR: https://git.openjdk.org/skara/pull/1575 From erikj at openjdk.org Tue Oct 24 21:33:31 2023 From: erikj at openjdk.org (Erik Joelsson) Date: Tue, 24 Oct 2023 21:33:31 GMT Subject: RFR: 1962: JiraProject#queryIssues may return duplicate issues In-Reply-To: References: Message-ID: On Tue, 24 Oct 2023 17:46:39 GMT, Zhao Song wrote: > Since [SKARA-1912](https://bugs.openjdk.org/browse/SKARA-1912) deployed into prod, we have noticed occurrences of duplicate key exceptions in the IssueProjectPoller#updatedIssues. > > After investigation, we found that `JiraProject#queryIssues` may return duplicate issues and I think the pagination in this method is the root cause. > > The problem here is that, sometimes, issues will be updated during the pagination process. For example, the default page size is 50 and we assume that issue#50 is the last issue in the first page. And before querying the second page, another issue which not in the first page got updated, so issue#50 will also be the first issue in the second page. Therefore, the method may return duplicate issues. > > Erik's solution is right. We just need to order by updatedTime DESC and remove the duplicate issues. Ordering by updatedTime DESC would make sure that even some issues are updated during the pagination, the issues will be processed in the future round. issuetracker/src/main/java/org/openjdk/skara/issuetracker/jira/JiraProject.java line 510: > 508: > 509: private List queryIssues(String jql) { > 510: var ret = new HashMap(); I think we should try to preserve the order we received issues. We can do that if we use a `LinkedHashMap` here. It would also make more sense to me to use ascending order. Then we will include any concurrently updated issue in this call instead of waiting for next call. I also think it makes sense for the bot to process issues in ascending order, so the first updated issue is processed first. ------------- PR Review Comment: https://git.openjdk.org/skara/pull/1575#discussion_r1370838440 From erikj at openjdk.org Tue Oct 24 22:53:02 2023 From: erikj at openjdk.org (Erik Joelsson) Date: Tue, 24 Oct 2023 22:53:02 GMT Subject: RFR: 1962: JiraProject#queryIssues may return duplicate issues In-Reply-To: References: Message-ID: On Tue, 24 Oct 2023 17:46:39 GMT, Zhao Song wrote: > Since [SKARA-1912](https://bugs.openjdk.org/browse/SKARA-1912) deployed into prod, we have noticed occurrences of duplicate key exceptions in the IssueProjectPoller#updatedIssues. > > After investigation, we found that `JiraProject#queryIssues` may return duplicate issues and I think the pagination in this method is the root cause. > > The problem here is that, sometimes, issues will be updated during the pagination process. For example, the default page size is 50 and we assume that issue#50 is the last issue in the first page. And before querying the second page, another issue which not in the first page got updated, so issue#50 will also be the first issue in the second page. Therefore, the method may return duplicate issues. > > Erik's solution is right. We just need to order by updatedTime DESC and remove the duplicate issues. Ordering by updatedTime DESC would make sure that even some issues are updated during the pagination, the issues will be processed in the future round. Marked as reviewed by erikj (Lead). ------------- PR Review: https://git.openjdk.org/skara/pull/1575#pullrequestreview-1696033425 From erikj at openjdk.org Tue Oct 24 22:53:02 2023 From: erikj at openjdk.org (Erik Joelsson) Date: Tue, 24 Oct 2023 22:53:02 GMT Subject: RFR: 1962: JiraProject#queryIssues may return duplicate issues In-Reply-To: References: Message-ID: On Tue, 24 Oct 2023 21:31:10 GMT, Erik Joelsson wrote: >> Since [SKARA-1912](https://bugs.openjdk.org/browse/SKARA-1912) deployed into prod, we have noticed occurrences of duplicate key exceptions in the IssueProjectPoller#updatedIssues. >> >> After investigation, we found that `JiraProject#queryIssues` may return duplicate issues and I think the pagination in this method is the root cause. >> >> The problem here is that, sometimes, issues will be updated during the pagination process. For example, the default page size is 50 and we assume that issue#50 is the last issue in the first page. And before querying the second page, another issue which not in the first page got updated, so issue#50 will also be the first issue in the second page. Therefore, the method may return duplicate issues. >> >> Erik's solution is right. We just need to order by updatedTime DESC and remove the duplicate issues. Ordering by updatedTime DESC would make sure that even some issues are updated during the pagination, the issues will be processed in the future round. > > issuetracker/src/main/java/org/openjdk/skara/issuetracker/jira/JiraProject.java line 510: > >> 508: >> 509: private List queryIssues(String jql) { >> 510: var ret = new HashMap(); > > I think we should try to preserve the order we received issues. We can do that if we use a `LinkedHashMap` here. It would also make more sense to me to use ascending order. Then we will include any concurrently updated issue in this call instead of waiting for next call. I also think it makes sense for the bot to process issues in ascending order, so the first updated issue is processed first. After offline discussion, I take this back, descending order is the correct solution. We could still preserve the order here, or maybe just sort the result on "updated" before returning it. ------------- PR Review Comment: https://git.openjdk.org/skara/pull/1575#discussion_r1370903482 From erikj at openjdk.org Tue Oct 24 22:53:02 2023 From: erikj at openjdk.org (Erik Joelsson) Date: Tue, 24 Oct 2023 22:53:02 GMT Subject: RFR: 1962: JiraProject#queryIssues may return duplicate issues In-Reply-To: References: Message-ID: <9C4zpIcMX96Oz8zxnf7vlZgCZBywMDCyl97ShK46ljY=.e2247a8b-0e05-422f-b0a3-4300833435bc@github.com> On Tue, 24 Oct 2023 22:49:43 GMT, Erik Joelsson wrote: >> issuetracker/src/main/java/org/openjdk/skara/issuetracker/jira/JiraProject.java line 510: >> >>> 508: >>> 509: private List queryIssues(String jql) { >>> 510: var ret = new HashMap(); >> >> I think we should try to preserve the order we received issues. We can do that if we use a `LinkedHashMap` here. It would also make more sense to me to use ascending order. Then we will include any concurrently updated issue in this call instead of waiting for next call. I also think it makes sense for the bot to process issues in ascending order, so the first updated issue is processed first. > > After offline discussion, I take this back, descending order is the correct solution. We could still preserve the order here, or maybe just sort the result on "updated" before returning it. Or just skip sorting. ------------- PR Review Comment: https://git.openjdk.org/skara/pull/1575#discussion_r1370903950 From erikj at openjdk.org Tue Oct 24 23:20:51 2023 From: erikj at openjdk.org (Erik Joelsson) Date: Tue, 24 Oct 2023 23:20:51 GMT Subject: Integrated: 2071: Track active status for users in IssueTracker In-Reply-To: References: Message-ID: On Fri, 20 Oct 2023 21:22:20 GMT, Erik Joelsson wrote: > This patch adds the concept of "active/inactive" users in a JiraHost and the ability to remove collaborators from a repository. I'm also adding test support where appropriate to be able to mock this in tests. This pull request has now been integrated. Changeset: 58ba50eb Author: Erik Joelsson URL: https://git.openjdk.org/skara/commit/58ba50eb98299042427bee9ff0b06737b9770cb8 Stats: 286 lines in 17 files changed: 236 ins; 20 del; 30 mod 2071: Track active status for users in IssueTracker Reviewed-by: ihse, zsong ------------- PR: https://git.openjdk.org/skara/pull/1574 From erikj at openjdk.org Wed Oct 25 13:09:15 2023 From: erikj at openjdk.org (Erik Joelsson) Date: Wed, 25 Oct 2023 13:09:15 GMT Subject: RFR: 1962: JiraProject#queryIssues may return duplicate issues In-Reply-To: References: Message-ID: On Tue, 24 Oct 2023 17:46:39 GMT, Zhao Song wrote: > Since [SKARA-1912](https://bugs.openjdk.org/browse/SKARA-1912) deployed into prod, we have noticed occurrences of duplicate key exceptions in the IssueProjectPoller#updatedIssues. > > After investigation, we found that `JiraProject#queryIssues` may return duplicate issues and I think the pagination in this method is the root cause. > > The problem here is that, sometimes, issues will be updated during the pagination process. For example, the default page size is 50 and we assume that issue#50 is the last issue in the first page. And before querying the second page, another issue which not in the first page got updated, so issue#50 will also be the first issue in the second page. Therefore, the method may return duplicate issues. > > Erik's solution is right. We just need to order by updatedTime DESC and remove the duplicate issues. Ordering by updatedTime DESC would make sure that even some issues are updated during the pagination, the issues will be processed in the future round. I think we need a comment explaining why we need the sorting order. ------------- PR Review: https://git.openjdk.org/skara/pull/1575#pullrequestreview-1697299952 From zsong at openjdk.org Wed Oct 25 15:53:32 2023 From: zsong at openjdk.org (Zhao Song) Date: Wed, 25 Oct 2023 15:53:32 GMT Subject: RFR: 1962: JiraProject#queryIssues may return duplicate issues In-Reply-To: References: Message-ID: On Wed, 25 Oct 2023 13:07:06 GMT, Erik Joelsson wrote: > I think we need a comment explaining why we need the sorting order. Sure ------------- PR Comment: https://git.openjdk.org/skara/pull/1575#issuecomment-1779580043 From zsong at openjdk.org Wed Oct 25 17:35:17 2023 From: zsong at openjdk.org (Zhao Song) Date: Wed, 25 Oct 2023 17:35:17 GMT Subject: RFR: 1962: JiraProject#queryIssues may return duplicate issues [v2] In-Reply-To: References: Message-ID: > Since [SKARA-1912](https://bugs.openjdk.org/browse/SKARA-1912) deployed into prod, we have noticed occurrences of duplicate key exceptions in the IssueProjectPoller#updatedIssues. > > After investigation, we found that `JiraProject#queryIssues` may return duplicate issues and I think the pagination in this method is the root cause. > > The problem here is that, sometimes, issues will be updated during the pagination process. For example, the default page size is 50 and we assume that issue#50 is the last issue in the first page. And before querying the second page, another issue which not in the first page got updated, so issue#50 will also be the first issue in the second page. Therefore, the method may return duplicate issues. > > Erik's solution is right. We just need to order by updatedTime DESC and remove the duplicate issues. Ordering by updatedTime DESC would make sure that even some issues are updated during the pagination, the issues will be processed in the future round. Zhao Song has updated the pull request incrementally with one additional commit since the last revision: add log ------------- Changes: - all: https://git.openjdk.org/skara/pull/1575/files - new: https://git.openjdk.org/skara/pull/1575/files/7f36bb16..2a2d70f2 Webrevs: - full: https://webrevs.openjdk.org/?repo=skara&pr=1575&range=01 - incr: https://webrevs.openjdk.org/?repo=skara&pr=1575&range=00-01 Stats: 2 lines in 1 file changed: 2 ins; 0 del; 0 mod Patch: https://git.openjdk.org/skara/pull/1575.diff Fetch: git fetch https://git.openjdk.org/skara.git pull/1575/head:pull/1575 PR: https://git.openjdk.org/skara/pull/1575 From erikj at openjdk.org Wed Oct 25 20:02:59 2023 From: erikj at openjdk.org (Erik Joelsson) Date: Wed, 25 Oct 2023 20:02:59 GMT Subject: RFR: 1962: JiraProject#queryIssues may return duplicate issues [v2] In-Reply-To: References: Message-ID: On Wed, 25 Oct 2023 17:35:17 GMT, Zhao Song wrote: >> Since [SKARA-1912](https://bugs.openjdk.org/browse/SKARA-1912) deployed into prod, we have noticed occurrences of duplicate key exceptions in the IssueProjectPoller#updatedIssues. >> >> After investigation, we found that `JiraProject#queryIssues` may return duplicate issues and I think the pagination in this method is the root cause. >> >> The problem here is that, sometimes, issues will be updated during the pagination process. For example, the default page size is 50 and we assume that issue#50 is the last issue in the first page. And before querying the second page, another issue which not in the first page got updated, so issue#50 will also be the first issue in the second page. Therefore, the method may return duplicate issues. >> >> Erik's solution is right. We just need to order by updatedTime DESC and remove the duplicate issues. Ordering by updatedTime DESC would make sure that even some issues are updated during the pagination, the issues will be processed in the future round. > > Zhao Song has updated the pull request incrementally with one additional commit since the last revision: > > add log Marked as reviewed by erikj (Lead). ------------- PR Review: https://git.openjdk.org/skara/pull/1575#pullrequestreview-1698157499 From zsong at openjdk.org Wed Oct 25 20:28:48 2023 From: zsong at openjdk.org (Zhao Song) Date: Wed, 25 Oct 2023 20:28:48 GMT Subject: RFR: 1962: JiraProject#queryIssues may return duplicate issues [v2] In-Reply-To: References: Message-ID: On Wed, 25 Oct 2023 17:35:17 GMT, Zhao Song wrote: >> Since [SKARA-1912](https://bugs.openjdk.org/browse/SKARA-1912) deployed into prod, we have noticed occurrences of duplicate key exceptions in the IssueProjectPoller#updatedIssues. >> >> After investigation, we found that `JiraProject#queryIssues` may return duplicate issues and I think the pagination in this method is the root cause. >> >> The problem here is that, sometimes, issues will be updated during the pagination process. For example, the default page size is 50 and we assume that issue#50 is the last issue in the first page. And before querying the second page, another issue which not in the first page got updated, so issue#50 will also be the first issue in the second page. Therefore, the method may return duplicate issues. >> >> Erik's solution is right. We just need to order by updatedTime DESC and remove the duplicate issues. Ordering by updatedTime DESC would make sure that even some issues are updated during the pagination, the issues will be processed in the future round. > > Zhao Song has updated the pull request incrementally with one additional commit since the last revision: > > add log Thanks for the review! ------------- PR Comment: https://git.openjdk.org/skara/pull/1575#issuecomment-1780002293 From zsong at openjdk.org Wed Oct 25 20:28:48 2023 From: zsong at openjdk.org (Zhao Song) Date: Wed, 25 Oct 2023 20:28:48 GMT Subject: Integrated: 1962: JiraProject#queryIssues may return duplicate issues In-Reply-To: References: Message-ID: On Tue, 24 Oct 2023 17:46:39 GMT, Zhao Song wrote: > Since [SKARA-1912](https://bugs.openjdk.org/browse/SKARA-1912) deployed into prod, we have noticed occurrences of duplicate key exceptions in the IssueProjectPoller#updatedIssues. > > After investigation, we found that `JiraProject#queryIssues` may return duplicate issues and I think the pagination in this method is the root cause. > > The problem here is that, sometimes, issues will be updated during the pagination process. For example, the default page size is 50 and we assume that issue#50 is the last issue in the first page. And before querying the second page, another issue which not in the first page got updated, so issue#50 will also be the first issue in the second page. Therefore, the method may return duplicate issues. > > Erik's solution is right. We just need to order by updatedTime DESC and remove the duplicate issues. Ordering by updatedTime DESC would make sure that even some issues are updated during the pagination, the issues will be processed in the future round. This pull request has now been integrated. Changeset: 4c3c83a6 Author: Zhao Song URL: https://git.openjdk.org/skara/commit/4c3c83a684f16b63a3a49f89177a6a34a30df2df Stats: 12 lines in 1 file changed: 5 ins; 0 del; 7 mod 1962: JiraProject#queryIssues may return duplicate issues Reviewed-by: erikj ------------- PR: https://git.openjdk.org/skara/pull/1575 From erikj at openjdk.org Wed Oct 25 22:20:10 2023 From: erikj at openjdk.org (Erik Joelsson) Date: Wed, 25 Oct 2023 22:20:10 GMT Subject: RFR: 2077: GitLabRepository.recentCommitComments may miss commits on non default branch Message-ID: The recentCommitComments method on GitLabRepository is a big hack to overcome shortcomings in the GitLab APIs that has caused trouble for us at several occasions. I have now discovered another flaw in the implementation. The sub method `getCommitTitleToCommitsMap` is used to build a map of commit messages to a list of commits. On first call, it builds the bulk of the map from a local repository. This is then kept up to date by calling the rest endpoint `GET /projects/:id/repository/commits` using the `since` argument to get all commits created after the newest commit already present in the map. The problem is that this API point will only return commits on the default branch. This means that after a bot restart, any new commits on other branches will not be found when looking for commit commands. To fix this, I found the parameter `all=true` for this [endpoint](https://docs.gitlab.com/ee/api/commits.html). According to my testing, this will include commits not on the default branch while still letting us filter using `since`. ------------- Commit messages: - SKARA-2077 Changes: https://git.openjdk.org/skara/pull/1576/files Webrev: https://webrevs.openjdk.org/?repo=skara&pr=1576&range=00 Issue: https://bugs.openjdk.org/browse/SKARA-2077 Stats: 1 line in 1 file changed: 1 ins; 0 del; 0 mod Patch: https://git.openjdk.org/skara/pull/1576.diff Fetch: git fetch https://git.openjdk.org/skara.git pull/1576/head:pull/1576 PR: https://git.openjdk.org/skara/pull/1576 From zsong at openjdk.org Wed Oct 25 22:31:17 2023 From: zsong at openjdk.org (Zhao Song) Date: Wed, 25 Oct 2023 22:31:17 GMT Subject: RFR: 2077: GitLabRepository.recentCommitComments may miss commits on non default branch In-Reply-To: References: Message-ID: <0j7HBkLo3zNsgfjYEPYKa2bOndaEc1Tvjai5-sTYzcE=.72b41ba0-72b8-401b-9a7c-1533a7fa550c@github.com> On Wed, 25 Oct 2023 22:00:31 GMT, Erik Joelsson wrote: > The recentCommitComments method on GitLabRepository is a big hack to overcome shortcomings in the GitLab APIs that has caused trouble for us at several occasions. I have now discovered another flaw in the implementation. > > The sub method `getCommitTitleToCommitsMap` is used to build a map of commit messages to a list of commits. On first call, it builds the bulk of the map from a local repository. This is then kept up to date by calling the rest endpoint `GET /projects/:id/repository/commits` using the `since` argument to get all commits created after the newest commit already present in the map. The problem is that this API point will only return commits on the default branch. This means that after a bot restart, any new commits on other branches will not be found when looking for commit commands. > > To fix this, I found the parameter `all=true` for this [endpoint](https://docs.gitlab.com/ee/api/commits.html). According to my testing, this will include commits not on the default branch while still letting us filter using `since`. Marked as reviewed by zsong (Reviewer). ------------- PR Review: https://git.openjdk.org/skara/pull/1576#pullrequestreview-1698343596 From erikj at openjdk.org Thu Oct 26 15:12:30 2023 From: erikj at openjdk.org (Erik Joelsson) Date: Thu, 26 Oct 2023 15:12:30 GMT Subject: Integrated: 2077: GitLabRepository.recentCommitComments may miss commits on non default branch In-Reply-To: References: Message-ID: On Wed, 25 Oct 2023 22:00:31 GMT, Erik Joelsson wrote: > The recentCommitComments method on GitLabRepository is a big hack to overcome shortcomings in the GitLab APIs that has caused trouble for us at several occasions. I have now discovered another flaw in the implementation. > > The sub method `getCommitTitleToCommitsMap` is used to build a map of commit messages to a list of commits. On first call, it builds the bulk of the map from a local repository. This is then kept up to date by calling the rest endpoint `GET /projects/:id/repository/commits` using the `since` argument to get all commits created after the newest commit already present in the map. The problem is that this API point will only return commits on the default branch. This means that after a bot restart, any new commits on other branches will not be found when looking for commit commands. > > To fix this, I found the parameter `all=true` for this [endpoint](https://docs.gitlab.com/ee/api/commits.html). According to my testing, this will include commits not on the default branch while still letting us filter using `since`. This pull request has now been integrated. Changeset: de78b014 Author: Erik Joelsson URL: https://git.openjdk.org/skara/commit/de78b014cd1f84a2baa6fd97818bcd8ecf047ce2 Stats: 1 line in 1 file changed: 1 ins; 0 del; 0 mod 2077: GitLabRepository.recentCommitComments may miss commits on non default branch Reviewed-by: zsong ------------- PR: https://git.openjdk.org/skara/pull/1576 From ehelin at openjdk.org Fri Oct 27 08:16:45 2023 From: ehelin at openjdk.org (Erik Duveblad) Date: Fri, 27 Oct 2023 08:16:45 GMT Subject: RFR: 2080: ini: allow values to be updated Message-ID: <5ubxaTsS4J8XQE02M-SlHzKQKscOaQjzg0AHHXdlaPM=.c069809d-5c71-401d-98b1-e6d4341d0c12@github.com> Hi all, please review this patch that makes the `INI` parser allow values to be updated _if_ they are set multiple times in a file. The below example would result in the key `name` having the value `jane`: name=john name=jane Values in sections and sub-sections can now also be updated, the below example would result in the key `age` in the subsection `[person "john"]` having the value `47`: [person] age=0 [person "john"] age=1 [person "john"] age=47 This is in line with how Git's INI parser is working, see the following example: $ printf '[test]\nfoo=bar\n' >> .git/config $ printf '[test]\nfoo=baz\n' >> .git/config $ cat .git/config [test] foo=bar [test] foo=baz $ git config test.foo baz Enabling keys to be updated makes modifying just a single key in a `.jcheck/conf` easier. Appending the following to any `.jcheck/conf` would make merge commits requiring reviews: [checks "reviewer"] merge=check This property will be used to simplify [AdditionalConfiguration](https://github.com/openjdk/skara/blob/master/bots/pr/src/main/java/org/openjdk/skara/bots/pr/AdditionalConfiguration.java) in an upcoming PR. Thanks, Erik ------------- Commit messages: - 2080 Changes: https://git.openjdk.org/skara/pull/1577/files Webrev: https://webrevs.openjdk.org/?repo=skara&pr=1577&range=00 Issue: https://bugs.openjdk.org/browse/SKARA-2080 Stats: 60 lines in 2 files changed: 44 ins; 9 del; 7 mod Patch: https://git.openjdk.org/skara/pull/1577.diff Fetch: git fetch https://git.openjdk.org/skara.git pull/1577/head:pull/1577 PR: https://git.openjdk.org/skara/pull/1577 From ehelin at openjdk.org Fri Oct 27 08:36:30 2023 From: ehelin at openjdk.org (Erik Duveblad) Date: Fri, 27 Oct 2023 08:36:30 GMT Subject: RFR: 2082: Use correct .jcheck/conf in AdditionalConfiguration.java Message-ID: Hi all, please review this patch that ensures that we pass the correct `.jcheck/conf` to [AdditionalConfiguration.java](https://github.com/openjdk/skara/blob/master/bots/pr/src/main/java/org/openjdk/skara/bots/pr/AdditionalConfiguration.java). The `.jcheck/conf` should either come from the "overriding" `.jcheck/conf` (if the PR bot is configured to use that) or the pull request's target branch. I have added that test that passes with this patch but fails without it. Thanks, Erik ------------- Commit messages: - 2082 Changes: https://git.openjdk.org/skara/pull/1578/files Webrev: https://webrevs.openjdk.org/?repo=skara&pr=1578&range=00 Issue: https://bugs.openjdk.org/browse/SKARA-2082 Stats: 78 lines in 5 files changed: 64 ins; 2 del; 12 mod Patch: https://git.openjdk.org/skara/pull/1578.diff Fetch: git fetch https://git.openjdk.org/skara.git pull/1578/head:pull/1578 PR: https://git.openjdk.org/skara/pull/1578 From erikj at openjdk.org Fri Oct 27 13:12:23 2023 From: erikj at openjdk.org (Erik Joelsson) Date: Fri, 27 Oct 2023 13:12:23 GMT Subject: RFR: 2080: ini: allow values to be updated In-Reply-To: <5ubxaTsS4J8XQE02M-SlHzKQKscOaQjzg0AHHXdlaPM=.c069809d-5c71-401d-98b1-e6d4341d0c12@github.com> References: <5ubxaTsS4J8XQE02M-SlHzKQKscOaQjzg0AHHXdlaPM=.c069809d-5c71-401d-98b1-e6d4341d0c12@github.com> Message-ID: On Fri, 27 Oct 2023 08:12:27 GMT, Erik Duveblad wrote: > Hi all, > > please review this patch that makes the `INI` parser allow values to be updated _if_ they are set multiple times in a file. The below example would result in the key `name` having the value `jane`: > > > name=john > name=jane > > > Values in sections and sub-sections can now also be updated, the below example would result in the key `age` in the subsection `[person "john"]` having the value `47`: > > > [person] > age=0 > > [person "john"] > age=1 > > [person "john"] > age=47 > > > This is in line with how Git's INI parser is working, see the following example: > > > $ printf '[test]\nfoo=bar\n' >> .git/config > $ printf '[test]\nfoo=baz\n' >> .git/config > $ cat .git/config > [test] > foo=bar > [test] > foo=baz > $ git config test.foo > baz > > > Enabling keys to be updated makes modifying just a single key in a `.jcheck/conf` easier. Appending the following to any `.jcheck/conf` would make merge commits requiring reviews: > > > [checks "reviewer"] > merge=check > > > This property will be used to simplify [AdditionalConfiguration](https://github.com/openjdk/skara/blob/master/bots/pr/src/main/java/org/openjdk/skara/bots/pr/AdditionalConfiguration.java) in an upcoming PR. > > Thanks, > Erik Marked as reviewed by erikj (Lead). ------------- PR Review: https://git.openjdk.org/skara/pull/1577#pullrequestreview-1701756532 From ehelin at openjdk.org Fri Oct 27 13:17:19 2023 From: ehelin at openjdk.org (Erik Duveblad) Date: Fri, 27 Oct 2023 13:17:19 GMT Subject: RFR: 2080: ini: allow values to be updated In-Reply-To: References: <5ubxaTsS4J8XQE02M-SlHzKQKscOaQjzg0AHHXdlaPM=.c069809d-5c71-401d-98b1-e6d4341d0c12@github.com> Message-ID: <0SVcb3Gy1nuSg2noFsYeta36xjhcNjzpCh7ckckic6c=.51e2fc02-e4bd-4cce-a716-fdfe626d294f@github.com> On Fri, 27 Oct 2023 13:10:07 GMT, Erik Joelsson wrote: >> Hi all, >> >> please review this patch that makes the `INI` parser allow values to be updated _if_ they are set multiple times in a file. The below example would result in the key `name` having the value `jane`: >> >> >> name=john >> name=jane >> >> >> Values in sections and sub-sections can now also be updated, the below example would result in the key `age` in the subsection `[person "john"]` having the value `47`: >> >> >> [person] >> age=0 >> >> [person "john"] >> age=1 >> >> [person "john"] >> age=47 >> >> >> This is in line with how Git's INI parser is working, see the following example: >> >> >> $ printf '[test]\nfoo=bar\n' >> .git/config >> $ printf '[test]\nfoo=baz\n' >> .git/config >> $ cat .git/config >> [test] >> foo=bar >> [test] >> foo=baz >> $ git config test.foo >> baz >> >> >> Enabling keys to be updated makes modifying just a single key in a `.jcheck/conf` easier. Appending the following to any `.jcheck/conf` would make merge commits requiring reviews: >> >> >> [checks "reviewer"] >> merge=check >> >> >> This property will be used to simplify [AdditionalConfiguration](https://github.com/openjdk/skara/blob/master/bots/pr/src/main/java/org/openjdk/skara/bots/pr/AdditionalConfiguration.java) in an upcoming PR. >> >> Thanks, >> Erik > > Marked as reviewed by erikj (Lead). Thanks for reviewing @erikj79 ! ------------- PR Comment: https://git.openjdk.org/skara/pull/1577#issuecomment-1782898174 From ehelin at openjdk.org Fri Oct 27 13:17:19 2023 From: ehelin at openjdk.org (Erik Duveblad) Date: Fri, 27 Oct 2023 13:17:19 GMT Subject: Integrated: 2080: ini: allow values to be updated In-Reply-To: <5ubxaTsS4J8XQE02M-SlHzKQKscOaQjzg0AHHXdlaPM=.c069809d-5c71-401d-98b1-e6d4341d0c12@github.com> References: <5ubxaTsS4J8XQE02M-SlHzKQKscOaQjzg0AHHXdlaPM=.c069809d-5c71-401d-98b1-e6d4341d0c12@github.com> Message-ID: On Fri, 27 Oct 2023 08:12:27 GMT, Erik Duveblad wrote: > Hi all, > > please review this patch that makes the `INI` parser allow values to be updated _if_ they are set multiple times in a file. The below example would result in the key `name` having the value `jane`: > > > name=john > name=jane > > > Values in sections and sub-sections can now also be updated, the below example would result in the key `age` in the subsection `[person "john"]` having the value `47`: > > > [person] > age=0 > > [person "john"] > age=1 > > [person "john"] > age=47 > > > This is in line with how Git's INI parser is working, see the following example: > > > $ printf '[test]\nfoo=bar\n' >> .git/config > $ printf '[test]\nfoo=baz\n' >> .git/config > $ cat .git/config > [test] > foo=bar > [test] > foo=baz > $ git config test.foo > baz > > > Enabling keys to be updated makes modifying just a single key in a `.jcheck/conf` easier. Appending the following to any `.jcheck/conf` would make merge commits requiring reviews: > > > [checks "reviewer"] > merge=check > > > This property will be used to simplify [AdditionalConfiguration](https://github.com/openjdk/skara/blob/master/bots/pr/src/main/java/org/openjdk/skara/bots/pr/AdditionalConfiguration.java) in an upcoming PR. > > Thanks, > Erik This pull request has now been integrated. Changeset: 0c638161 Author: Erik Duveblad URL: https://git.openjdk.org/skara/commit/0c63816196a00e157592b9d091a8c0c94912556f Stats: 60 lines in 2 files changed: 44 ins; 9 del; 7 mod 2080: ini: allow values to be updated Reviewed-by: erikj ------------- PR: https://git.openjdk.org/skara/pull/1577 From erikj at openjdk.org Fri Oct 27 13:31:28 2023 From: erikj at openjdk.org (Erik Joelsson) Date: Fri, 27 Oct 2023 13:31:28 GMT Subject: RFR: 2082: Use correct .jcheck/conf in AdditionalConfiguration.java In-Reply-To: References: Message-ID: On Fri, 27 Oct 2023 08:32:26 GMT, Erik Duveblad wrote: > Hi all, > > please review this patch that ensures that we pass the correct `.jcheck/conf` to [AdditionalConfiguration.java](https://github.com/openjdk/skara/blob/master/bots/pr/src/main/java/org/openjdk/skara/bots/pr/AdditionalConfiguration.java). The `.jcheck/conf` should either come from the "overriding" `.jcheck/conf` (if the PR bot is configured to use that) or the pull request's target branch. > > I have added that test that passes with this patch but fails without it. > > Thanks, > Erik bots/pr/src/main/java/org/openjdk/skara/bots/pr/AdditionalConfiguration.java line 34: > 32: > 33: public class AdditionalConfiguration { > 34: static List get(Optional conf, HostUser botUser, List comments, boolean reviewMerge) throws IOException { Using Optional as parameter isn't recommended, especially since we throw if it's empty. Is it truly optional here? I would think we should just `.orElseThrow()` at the parse call. bots/pr/src/main/java/org/openjdk/skara/bots/pr/CheckRun.java line 1259: > 1257: > 1258: Hash jcheckConfHash = checkablePullRequest.targetHash(); > 1259: PullRequestCheckIssueVisitor visitor = checkablePullRequest.createVisitor(jcheckConfHash); Any particular reason to spell out the types here? bots/pr/src/main/java/org/openjdk/skara/bots/pr/CheckRun.java line 1274: > 1272: pr.repository().forge().currentUser(), comments, reviewMerge); > 1273: checkablePullRequest.executeChecks(localHash, censusInstance, visitor, additionalConfiguration, jcheckConfHash); > 1274: // Don't need to run the second round if confOverride is set. Strictly speaking, should we regenerate the AdditionalConfiguration for the second run where we use the merged .jcheck/conf? ------------- PR Review Comment: https://git.openjdk.org/skara/pull/1578#discussion_r1374572674 PR Review Comment: https://git.openjdk.org/skara/pull/1578#discussion_r1374577254 PR Review Comment: https://git.openjdk.org/skara/pull/1578#discussion_r1374574088 From zsong at openjdk.org Fri Oct 27 17:49:29 2023 From: zsong at openjdk.org (Zhao Song) Date: Fri, 27 Oct 2023 17:49:29 GMT Subject: RFR: 2082: Use correct .jcheck/conf in AdditionalConfiguration.java In-Reply-To: References: Message-ID: <7uBITKBGAv_nxirANySKwwMkx_CZTQLRxSxQHVB1e5o=.39c43992-0106-4a77-97d4-b07eebf84081@github.com> On Fri, 27 Oct 2023 13:26:41 GMT, Erik Joelsson wrote: >> Hi all, >> >> please review this patch that ensures that we pass the correct `.jcheck/conf` to [AdditionalConfiguration.java](https://github.com/openjdk/skara/blob/master/bots/pr/src/main/java/org/openjdk/skara/bots/pr/AdditionalConfiguration.java). The `.jcheck/conf` should either come from the "overriding" `.jcheck/conf` (if the PR bot is configured to use that) or the pull request's target branch. >> >> I have added that test that passes with this patch but fails without it. >> >> Thanks, >> Erik > > bots/pr/src/main/java/org/openjdk/skara/bots/pr/CheckRun.java line 1274: > >> 1272: pr.repository().forge().currentUser(), comments, reviewMerge); >> 1273: checkablePullRequest.executeChecks(localHash, censusInstance, visitor, additionalConfiguration, jcheckConfHash); >> 1274: // Don't need to run the second round if confOverride is set. > > Strictly speaking, should we regenerate the AdditionalConfiguration for the second run where we use the merged .jcheck/conf? Oh, right, I overlooked it when I was implementing the second run of jcheck feature. ------------- PR Review Comment: https://git.openjdk.org/skara/pull/1578#discussion_r1374881522 From zsong at openjdk.org Mon Oct 30 21:58:01 2023 From: zsong at openjdk.org (Zhao Song) Date: Mon, 30 Oct 2023 21:58:01 GMT Subject: RFR: 2086: Use short Ids to get cached issue data in CheckWorkItem#getIssueMetadata Message-ID: A user reported that in a repo which is configured with maintainer approval feature, he added the approved label to the issue but skara bot didn't update the related PR. >From the log, I found skara bot said "[Issue]No activity since last check, not checking again." When investigating, I guess the scenario likes this. When the bot is evaluating a PR. The bot fetches the issue and the issue only contains label ?XXX-fix-request?. Then, the user add "XXX-fix-yes" to issue. So in the end of evaluation, the bot fetches the issue again and calculates the metadata again (at that time, the issue contains label ?XXX-fix-request, XXX-fix-yes?). So in the next round, the bot would think that new label ?XXX-fix-yes? is already handled, so it will not evaluate the PR again. But as Erik introduced a cache for IssueTrackerIssue in CheckWorkItem in [SKARA-1963](https://bugs.openjdk.org/browse/SKARA-1963). I think the bot shouldn't fetch the issue from JBS again in the end of checkRun. But the bot really did. Then I found that the key of the map(cache) are supposed to be short id of the JBS issue, But in CheckWorkItem#getIssueMetadata, the bot is trying to use the whole id of JBS issue to get the cached data, which will always miss and trigger the remote call. ------------- Commit messages: - SKARA-2086 Changes: https://git.openjdk.org/skara/pull/1579/files Webrev: https://webrevs.openjdk.org/?repo=skara&pr=1579&range=00 Issue: https://bugs.openjdk.org/browse/SKARA-2086 Stats: 4 lines in 2 files changed: 2 ins; 0 del; 2 mod Patch: https://git.openjdk.org/skara/pull/1579.diff Fetch: git fetch https://git.openjdk.org/skara.git pull/1579/head:pull/1579 PR: https://git.openjdk.org/skara/pull/1579 From erikj at openjdk.org Mon Oct 30 23:51:05 2023 From: erikj at openjdk.org (Erik Joelsson) Date: Mon, 30 Oct 2023 23:51:05 GMT Subject: RFR: 2086: Use short Ids to get cached issue data in CheckWorkItem#getIssueMetadata In-Reply-To: References: Message-ID: On Mon, 30 Oct 2023 21:22:29 GMT, Zhao Song wrote: > A user reported that in a repo which is configured with maintainer approval feature, he added the approved label to the issue but skara bot didn't update the related PR. > > From the log, I found skara bot said "[Issue]No activity since last check, not checking again." > > When investigating, I guess the scenario likes this. When the bot is evaluating a PR. The bot fetches the issue and the issue only contains label ?XXX-fix-request?. Then, the user add "XXX-fix-yes" to issue. So in the end of evaluation, the bot fetches the issue again and calculates the metadata again (at that time, the issue contains label ?XXX-fix-request, XXX-fix-yes?). So in the next round, the bot would think that new label ?XXX-fix-yes? is already handled, so it will not evaluate the PR again. > > But as Erik introduced a cache for IssueTrackerIssue in CheckWorkItem in [SKARA-1963](https://bugs.openjdk.org/browse/SKARA-1963). I think the bot shouldn't fetch the issue from JBS again in the end of checkRun. But the bot really did. Then I found that the key of the map(cache) are supposed to be short id of the JBS issue, But in CheckWorkItem#getIssueMetadata, the bot is trying to use the whole id of JBS issue to get the cached data, which will always miss and trigger the remote call. Can you add a comment on the `issueTrackerIssue` method that states that we expect the shortID, and perhaps also change the name of the parameter. ------------- PR Review: https://git.openjdk.org/skara/pull/1579#pullrequestreview-1705238418 From zsong at openjdk.org Tue Oct 31 17:09:13 2023 From: zsong at openjdk.org (Zhao Song) Date: Tue, 31 Oct 2023 17:09:13 GMT Subject: RFR: 2086: Use short Ids to get cached issue data in CheckWorkItem#getIssueMetadata In-Reply-To: References: Message-ID: On Mon, 30 Oct 2023 23:48:58 GMT, Erik Joelsson wrote: > Can you add a comment on the `issueTrackerIssue` method that states that we expect the shortID, and perhaps also change the name of the parameter. Sure. It's a good idea. ------------- PR Comment: https://git.openjdk.org/skara/pull/1579#issuecomment-1787631147 From zsong at openjdk.org Tue Oct 31 17:19:37 2023 From: zsong at openjdk.org (Zhao Song) Date: Tue, 31 Oct 2023 17:19:37 GMT Subject: RFR: 2086: Use short Ids to get cached issue data in CheckWorkItem#getIssueMetadata [v2] In-Reply-To: References: Message-ID: > A user reported that in a repo which is configured with maintainer approval feature, he added the approved label to the issue but skara bot didn't update the related PR. > > From the log, I found skara bot said "[Issue]No activity since last check, not checking again." > > When investigating, I guess the scenario likes this. When the bot is evaluating a PR. The bot fetches the issue and the issue only contains label ?XXX-fix-request?. Then, the user add "XXX-fix-yes" to issue. So in the end of evaluation, the bot fetches the issue again and calculates the metadata again (at that time, the issue contains label ?XXX-fix-request, XXX-fix-yes?). So in the next round, the bot would think that new label ?XXX-fix-yes? is already handled, so it will not evaluate the PR again. > > But as Erik introduced a cache for IssueTrackerIssue in CheckWorkItem in [SKARA-1963](https://bugs.openjdk.org/browse/SKARA-1963). I think the bot shouldn't fetch the issue from JBS again in the end of checkRun. But the bot really did. Then I found that the key of the map(cache) are supposed to be short id of the JBS issue, But in CheckWorkItem#getIssueMetadata, the bot is trying to use the whole id of JBS issue to get the cached data, which will always miss and trigger the remote call. Zhao Song has updated the pull request incrementally with one additional commit since the last revision: add a comment ------------- Changes: - all: https://git.openjdk.org/skara/pull/1579/files - new: https://git.openjdk.org/skara/pull/1579/files/788a049a..295206b0 Webrevs: - full: https://webrevs.openjdk.org/?repo=skara&pr=1579&range=01 - incr: https://webrevs.openjdk.org/?repo=skara&pr=1579&range=00-01 Stats: 5 lines in 1 file changed: 0 ins; 0 del; 5 mod Patch: https://git.openjdk.org/skara/pull/1579.diff Fetch: git fetch https://git.openjdk.org/skara.git pull/1579/head:pull/1579 PR: https://git.openjdk.org/skara/pull/1579 From erikj at openjdk.org Tue Oct 31 17:25:30 2023 From: erikj at openjdk.org (Erik Joelsson) Date: Tue, 31 Oct 2023 17:25:30 GMT Subject: RFR: 2086: Use short Ids to get cached issue data in CheckWorkItem#getIssueMetadata [v2] In-Reply-To: References: Message-ID: On Tue, 31 Oct 2023 17:19:37 GMT, Zhao Song wrote: >> A user reported that in a repo which is configured with maintainer approval feature, he added the approved label to the issue but skara bot didn't update the related PR. >> >> From the log, I found skara bot said "[Issue]No activity since last check, not checking again." >> >> When investigating, I guess the scenario likes this. When the bot is evaluating a PR. The bot fetches the issue and the issue only contains label ?XXX-fix-request?. Then, the user add "XXX-fix-yes" to issue. So in the end of evaluation, the bot fetches the issue again and calculates the metadata again (at that time, the issue contains label ?XXX-fix-request, XXX-fix-yes?). So in the next round, the bot would think that new label ?XXX-fix-yes? is already handled, so it will not evaluate the PR again. >> >> But as Erik introduced a cache for IssueTrackerIssue in CheckWorkItem in [SKARA-1963](https://bugs.openjdk.org/browse/SKARA-1963). I think the bot shouldn't fetch the issue from JBS again in the end of checkRun. But the bot really did. Then I found that the key of the map(cache) are supposed to be short id of the JBS issue, But in CheckWorkItem#getIssueMetadata, the bot is trying to use the whole id of JBS issue to get the cached data, which will always miss and trigger the remote call. > > Zhao Song has updated the pull request incrementally with one additional commit since the last revision: > > add a comment Marked as reviewed by erikj (Lead). ------------- PR Review: https://git.openjdk.org/skara/pull/1579#pullrequestreview-1706869280 From zsong at openjdk.org Tue Oct 31 19:53:58 2023 From: zsong at openjdk.org (Zhao Song) Date: Tue, 31 Oct 2023 19:53:58 GMT Subject: RFR: 2086: Use short Ids to get cached issue data in CheckWorkItem#getIssueMetadata [v2] In-Reply-To: References: Message-ID: <1qAMTAfSSI8zcbxPY_NeZpYMdijOHRdLv329eX-KQyw=.a9b5b8fa-577c-4bf2-ba6b-407da5455d11@github.com> On Tue, 31 Oct 2023 17:19:37 GMT, Zhao Song wrote: >> A user reported that in a repo which is configured with maintainer approval feature, he added the approved label to the issue but skara bot didn't update the related PR. >> >> From the log, I found skara bot said "[Issue]No activity since last check, not checking again." >> >> When investigating, I guess the scenario likes this. When the bot is evaluating a PR. The bot fetches the issue and the issue only contains label ?XXX-fix-request?. Then, the user add "XXX-fix-yes" to issue. So in the end of evaluation, the bot fetches the issue again and calculates the metadata again (at that time, the issue contains label ?XXX-fix-request, XXX-fix-yes?). So in the next round, the bot would think that new label ?XXX-fix-yes? is already handled, so it will not evaluate the PR again. >> >> But as Erik introduced a cache for IssueTrackerIssue in CheckWorkItem in [SKARA-1963](https://bugs.openjdk.org/browse/SKARA-1963). I think the bot shouldn't fetch the issue from JBS again in the end of checkRun. But the bot really did. Then I found that the key of the map(cache) are supposed to be short id of the JBS issue, But in CheckWorkItem#getIssueMetadata, the bot is trying to use the whole id of JBS issue to get the cached data, which will always miss and trigger the remote call. > > Zhao Song has updated the pull request incrementally with one additional commit since the last revision: > > add a comment Thanks for the review! ------------- PR Comment: https://git.openjdk.org/skara/pull/1579#issuecomment-1787937922 From zsong at openjdk.org Tue Oct 31 19:53:58 2023 From: zsong at openjdk.org (Zhao Song) Date: Tue, 31 Oct 2023 19:53:58 GMT Subject: Integrated: 2086: Use short Ids to get cached issue data in CheckWorkItem#getIssueMetadata In-Reply-To: References: Message-ID: On Mon, 30 Oct 2023 21:22:29 GMT, Zhao Song wrote: > A user reported that in a repo which is configured with maintainer approval feature, he added the approved label to the issue but skara bot didn't update the related PR. > > From the log, I found skara bot said "[Issue]No activity since last check, not checking again." > > When investigating, I guess the scenario likes this. When the bot is evaluating a PR. The bot fetches the issue and the issue only contains label ?XXX-fix-request?. Then, the user add "XXX-fix-yes" to issue. So in the end of evaluation, the bot fetches the issue again and calculates the metadata again (at that time, the issue contains label ?XXX-fix-request, XXX-fix-yes?). So in the next round, the bot would think that new label ?XXX-fix-yes? is already handled, so it will not evaluate the PR again. > > But as Erik introduced a cache for IssueTrackerIssue in CheckWorkItem in [SKARA-1963](https://bugs.openjdk.org/browse/SKARA-1963). I think the bot shouldn't fetch the issue from JBS again in the end of checkRun. But the bot really did. Then I found that the key of the map(cache) are supposed to be short id of the JBS issue, But in CheckWorkItem#getIssueMetadata, the bot is trying to use the whole id of JBS issue to get the cached data, which will always miss and trigger the remote call. This pull request has now been integrated. Changeset: c0c16a02 Author: Zhao Song URL: https://git.openjdk.org/skara/commit/c0c16a022cbdeaae19bba1784b3b7f6018e0732f Stats: 9 lines in 2 files changed: 2 ins; 0 del; 7 mod 2086: Use short Ids to get cached issue data in CheckWorkItem#getIssueMetadata Reviewed-by: erikj ------------- PR: https://git.openjdk.org/skara/pull/1579