From zsong at openjdk.org Thu Dec 1 00:04:51 2022 From: zsong at openjdk.org (Zhao Song) Date: Thu, 1 Dec 2022 00:04:51 GMT Subject: RFR: 1690: Make jcheck cli able to use conf from workspace or other commit [v5] In-Reply-To: References: Message-ID: > Currently, the command `git skara jcheck` can only use the `.jcheck/conf` configuration file in the current checking commit. > However, sometimes users want to run jcheck with different configurations to validate their commits. Therefore, we need to upgrade `git skara jcheck`. > > In this patch, `git skara jcheck` would be able to support following usecases. > > 1. Run jcheck on a commit or a series of commits using the .jcheck/conf in the same commit. (what we do by default today) > `git skara jcheck` > > 2. Run jcheck on a commit or a series of commits using the .jcheck/conf in a different specified commit. > `git skara jcheck --specified-conf-commit ` > > 3. Run jcheck on a commit or a series of commits using the .jcheck/conf in my workspace. > `git skara jcheck --workspace-conf` > > 4. Run jcheck on a commit or a series of commits using a config file that I point to directly, that may have any name. > `git skara jcheck --workspace-conf --conf-file ` > > 5. Run jcheck on the diff in my current workspace, either --staged or not using the .jcheck/conf in my workspace. > `git skara jcheck --workspace-diff` Zhao Song has updated the pull request incrementally with three additional commits since the last revision: - Update vcs/src/main/java/org/openjdk/skara/vcs/git/GitRepository.java Co-authored-by: Erik Joelsson <37597443+erikj79 at users.noreply.github.com> - Update vcs/src/main/java/org/openjdk/skara/vcs/git/GitRepository.java Co-authored-by: Erik Joelsson <37597443+erikj79 at users.noreply.github.com> - Update vcs/src/main/java/org/openjdk/skara/vcs/git/GitRepository.java Co-authored-by: Erik Joelsson <37597443+erikj79 at users.noreply.github.com> ------------- Changes: - all: https://git.openjdk.org/skara/pull/1428/files - new: https://git.openjdk.org/skara/pull/1428/files/3d2606a5..cf022be8 Webrevs: - full: https://webrevs.openjdk.org/?repo=skara&pr=1428&range=04 - incr: https://webrevs.openjdk.org/?repo=skara&pr=1428&range=03-04 Stats: 7 lines in 1 file changed: 4 ins; 0 del; 3 mod Patch: https://git.openjdk.org/skara/pull/1428.diff Fetch: git fetch https://git.openjdk.org/skara pull/1428/head:pull/1428 PR: https://git.openjdk.org/skara/pull/1428 From zsong at openjdk.org Thu Dec 1 00:20:56 2022 From: zsong at openjdk.org (Zhao Song) Date: Thu, 1 Dec 2022 00:20:56 GMT Subject: RFR: 1690: Make jcheck cli able to use conf from workspace or other commit [v3] In-Reply-To: References: Message-ID: On Wed, 30 Nov 2022 23:29:41 GMT, Erik Joelsson wrote: >> Zhao Song has updated the pull request incrementally with one additional commit since the last revision: >> >> implemented case 5 > > vcs/src/main/java/org/openjdk/skara/vcs/git/GitRepository.java line 1687: > >> 1685: @Override >> 1686: public Commit staged() throws IOException { >> 1687: var author = new Author("jcheck", "jcheck at none.none"); > > For the default user and email, there is already a `::userName` method. You can create one for email as well, and then use these values as fallbacks (`Optional.orElse`). > There are a lot of minor things, but all in all this turned out better than I had imagined. Looking forward to using it. > > Have you tried all the options against a repository with various degrees of local changes in it? Yes, I tried all the options and all the combination of options. ------------- PR: https://git.openjdk.org/skara/pull/1428 From zsong at openjdk.org Thu Dec 1 01:15:00 2022 From: zsong at openjdk.org (Zhao Song) Date: Thu, 1 Dec 2022 01:15:00 GMT Subject: RFR: 1690: Make jcheck cli able to use conf from workspace or other commit [v6] In-Reply-To: References: Message-ID: > Currently, the command `git skara jcheck` can only use the `.jcheck/conf` configuration file in the current checking commit. > However, sometimes users want to run jcheck with different configurations to validate their commits. Therefore, we need to upgrade `git skara jcheck`. > > In this patch, `git skara jcheck` would be able to support following usecases. > > 1. Run jcheck on a commit or a series of commits using the .jcheck/conf in the same commit. (what we do by default today) > `git skara jcheck` > > 2. Run jcheck on a commit or a series of commits using the .jcheck/conf in a different specified commit. > `git skara jcheck --specified-conf-commit ` > > 3. Run jcheck on a commit or a series of commits using the .jcheck/conf in my workspace. > `git skara jcheck --workspace-conf` > > 4. Run jcheck on a commit or a series of commits using a config file that I point to directly, that may have any name. > `git skara jcheck --workspace-conf --conf-file ` > > 5. Run jcheck on the diff in my current workspace, either --staged or not using the .jcheck/conf in my workspace. > `git skara jcheck --workspace-diff` Zhao Song has updated the pull request incrementally with two additional commits since the last revision: - fix some problems - fix some problems ------------- Changes: - all: https://git.openjdk.org/skara/pull/1428/files - new: https://git.openjdk.org/skara/pull/1428/files/cf022be8..37aca949 Webrevs: - full: https://webrevs.openjdk.org/?repo=skara&pr=1428&range=05 - incr: https://webrevs.openjdk.org/?repo=skara&pr=1428&range=04-05 Stats: 40 lines in 5 files changed: 25 ins; 0 del; 15 mod Patch: https://git.openjdk.org/skara/pull/1428.diff Fetch: git fetch https://git.openjdk.org/skara pull/1428/head:pull/1428 PR: https://git.openjdk.org/skara/pull/1428 From erikj at openjdk.org Thu Dec 1 14:12:06 2022 From: erikj at openjdk.org (Erik Joelsson) Date: Thu, 1 Dec 2022 14:12:06 GMT Subject: RFR: 1690: Make jcheck cli able to use conf from workspace or other commit [v3] In-Reply-To: References: Message-ID: On Wed, 30 Nov 2022 23:23:30 GMT, Erik Joelsson wrote: >> Zhao Song has updated the pull request incrementally with one additional commit since the last revision: >> >> implemented case 5 > > vcs/src/main/java/org/openjdk/skara/vcs/git/GitRepository.java line 1664: > >> 1662: >> 1663: @Override >> 1664: public Optional wholeHash(String rev) { > > Isn't this just `::resolve(String)`? What I mean is, you could use resolve instead of introducing this new method. ------------- PR: https://git.openjdk.org/skara/pull/1428 From erikj at openjdk.org Thu Dec 1 15:21:33 2022 From: erikj at openjdk.org (Erik Joelsson) Date: Thu, 1 Dec 2022 15:21:33 GMT Subject: RFR: 1701: Make use of Etag optional for MailingListArchiveReaderBot Message-ID: We are experiencing long delays from when a user posts an email to a PR thread to when the MailingListArchiveReaderBot posts it as a comment to the PR. I think this is caused by the service provider for the mailinglists archive not handling Etags correctly. While that is sorted out, we need to be able to turn the use of etags off. This patch makes it a configuration parameter. ------------- Commit messages: - SKARA-1701 Changes: https://git.openjdk.org/skara/pull/1430/files Webrev: https://webrevs.openjdk.org/?repo=skara&pr=1430&range=00 Issue: https://bugs.openjdk.org/browse/SKARA-1701 Stats: 21 lines in 5 files changed: 14 ins; 0 del; 7 mod Patch: https://git.openjdk.org/skara/pull/1430.diff Fetch: git fetch https://git.openjdk.org/skara pull/1430/head:pull/1430 PR: https://git.openjdk.org/skara/pull/1430 From zsong at openjdk.org Thu Dec 1 16:27:50 2022 From: zsong at openjdk.org (Zhao Song) Date: Thu, 1 Dec 2022 16:27:50 GMT Subject: RFR: 1690: Make jcheck cli able to use conf from workspace or other commit [v3] In-Reply-To: References: Message-ID: On Thu, 1 Dec 2022 14:08:52 GMT, Erik Joelsson wrote: >> vcs/src/main/java/org/openjdk/skara/vcs/git/GitRepository.java line 1664: >> >>> 1662: >>> 1663: @Override >>> 1664: public Optional wholeHash(String rev) { >> >> Isn't this just `::resolve(String)`? > > What I mean is, you could use resolve instead of introducing this new method. Oh, I don't know why I didn't see some of your review yesterday. Ah, right, I will use resolve and delete the new method ------------- PR: https://git.openjdk.org/skara/pull/1428 From ihse at openjdk.org Thu Dec 1 16:40:05 2022 From: ihse at openjdk.org (Magnus Ihse Bursie) Date: Thu, 1 Dec 2022 16:40:05 GMT Subject: RFR: 1701: Make use of Etag optional for MailingListArchiveReaderBot In-Reply-To: References: Message-ID: On Thu, 1 Dec 2022 15:15:46 GMT, Erik Joelsson wrote: > We are experiencing long delays from when a user posts an email to a PR thread to when the MailingListArchiveReaderBot posts it as a comment to the PR. I think this is caused by the service provider for the mailinglists archive not handling Etags correctly. While that is sorted out, we need to be able to turn the use of etags off. This patch makes it a configuration parameter. LGTM ------------- Marked as reviewed by ihse (Reviewer). PR: https://git.openjdk.org/skara/pull/1430 From ihse at openjdk.org Thu Dec 1 16:43:57 2022 From: ihse at openjdk.org (Magnus Ihse Bursie) Date: Thu, 1 Dec 2022 16:43:57 GMT Subject: RFR: 1687: Workaround bug in GitLab where merge request LIST API may serve stale data In-Reply-To: References: Message-ID: On Mon, 28 Nov 2022 21:43:24 GMT, Erik Joelsson wrote: > This patch provides a workaround for a GitLab bug that we are currently encountering. The issue has been fixed upstream, but until we have it deployed in our instance, we need to work around it. > > GitLab sometimes serves stale data for merge requests from list queries. For Skara this means that label changes aren't always picked up. I have verified that up-to-date data is always received when fetching single merge requests by ID, so the patch simply re-fetches MRs to make sure the data is always up to date. I'm also adding logging to track when stale data is detected. This will help us verify (at least empirically) when the GitLab issue resolved for us. LGTM forge/src/main/java/org/openjdk/skara/forge/gitlab/GitLabRepository.java line 201: > 199: var newData = request.get("merge_requests/" + id).execute(); > 200: // We can't compare the full json object returned from a list query > 201: // and get query call as they will always be different. The part we And you are sure the differences don't matter further down the processing chain? ------------- Marked as reviewed by ihse (Reviewer). PR: https://git.openjdk.org/skara/pull/1427 From erikj at openjdk.org Thu Dec 1 16:54:39 2022 From: erikj at openjdk.org (Erik Joelsson) Date: Thu, 1 Dec 2022 16:54:39 GMT Subject: RFR: 1687: Workaround bug in GitLab where merge request LIST API may serve stale data In-Reply-To: References: Message-ID: <8d9JTbXEeBiccFYYbV8tdm7-KYdqwB11CRO3GtGE6_g=.015a8a8f-7a1c-4a24-97f2-7f995e90557f@github.com> On Thu, 1 Dec 2022 16:41:24 GMT, Magnus Ihse Bursie wrote: >> This patch provides a workaround for a GitLab bug that we are currently encountering. The issue has been fixed upstream, but until we have it deployed in our instance, we need to work around it. >> >> GitLab sometimes serves stale data for merge requests from list queries. For Skara this means that label changes aren't always picked up. I have verified that up-to-date data is always received when fetching single merge requests by ID, so the patch simply re-fetches MRs to make sure the data is always up to date. I'm also adding logging to track when stale data is detected. This will help us verify (at least empirically) when the GitLab issue resolved for us. > > forge/src/main/java/org/openjdk/skara/forge/gitlab/GitLabRepository.java line 201: > >> 199: var newData = request.get("merge_requests/" + id).execute(); >> 200: // We can't compare the full json object returned from a list query >> 201: // and get query call as they will always be different. The part we > > And you are sure the differences don't matter further down the processing chain? They don't differ in any way that we care about when processing merge requests. We aren't looking at any of those fields. The only thing I can imagine this affecting is the PullRequestPoller, which may, at least in theory, detect some false positives due to this when detecting if a merge request has been updated or not. So that could potentially happen once 3 hours after a merge request has had its updated_at field updated, but only if the query was still including that 3 hour old result, which it normally shouldn't. So maybe possible, but rare enough so that I don't think we need to worry. ------------- PR: https://git.openjdk.org/skara/pull/1427 From zsong at openjdk.org Thu Dec 1 16:55:56 2022 From: zsong at openjdk.org (Zhao Song) Date: Thu, 1 Dec 2022 16:55:56 GMT Subject: RFR: 1690: Make jcheck cli able to use conf from workspace or other commit [v7] In-Reply-To: References: Message-ID: > Currently, the command `git skara jcheck` can only use the `.jcheck/conf` configuration file in the current checking commit. > However, sometimes users want to run jcheck with different configurations to validate their commits. Therefore, we need to upgrade `git skara jcheck`. > > In this patch, `git skara jcheck` would be able to support following usecases. > > 1. Run jcheck on a commit or a series of commits using the .jcheck/conf in the same commit. (what we do by default today) > `git skara jcheck` > > 2. Run jcheck on a commit or a series of commits using the .jcheck/conf in a different specified commit. > `git skara jcheck --specified-conf-commit ` > > 3. Run jcheck on a commit or a series of commits using the .jcheck/conf in my workspace. > `git skara jcheck --workspace-conf` > > 4. Run jcheck on a commit or a series of commits using a config file that I point to directly, that may have any name. > `git skara jcheck --workspace-conf --conf-file ` > > 5. Run jcheck on the diff in my current workspace, either --staged or not using the .jcheck/conf in my workspace. > `git skara jcheck --workspace-diff` Zhao Song has updated the pull request incrementally with one additional commit since the last revision: delete wholeHash() and update copyright headers ------------- Changes: - all: https://git.openjdk.org/skara/pull/1428/files - new: https://git.openjdk.org/skara/pull/1428/files/37aca949..abef3d5c Webrevs: - full: https://webrevs.openjdk.org/?repo=skara&pr=1428&range=06 - incr: https://webrevs.openjdk.org/?repo=skara&pr=1428&range=05-06 Stats: 33 lines in 7 files changed: 0 ins; 24 del; 9 mod Patch: https://git.openjdk.org/skara/pull/1428.diff Fetch: git fetch https://git.openjdk.org/skara pull/1428/head:pull/1428 PR: https://git.openjdk.org/skara/pull/1428 From erikj at openjdk.org Thu Dec 1 17:11:13 2022 From: erikj at openjdk.org (Erik Joelsson) Date: Thu, 1 Dec 2022 17:11:13 GMT Subject: RFR: 1690: Make jcheck cli able to use conf from workspace or other commit [v7] In-Reply-To: References: Message-ID: On Thu, 1 Dec 2022 16:55:56 GMT, Zhao Song wrote: >> Currently, the command `git skara jcheck` can only use the `.jcheck/conf` configuration file in the current checking commit. >> However, sometimes users want to run jcheck with different configurations to validate their commits. Therefore, we need to upgrade `git skara jcheck`. >> >> In this patch, `git skara jcheck` would be able to support following usecases. >> >> 1. Run jcheck on a commit or a series of commits using the .jcheck/conf in the same commit. (what we do by default today) >> `git skara jcheck` >> >> 2. Run jcheck on a commit or a series of commits using the .jcheck/conf in a different specified commit. >> `git skara jcheck --specified-conf-commit ` >> >> 3. Run jcheck on a commit or a series of commits using the .jcheck/conf in my workspace. >> `git skara jcheck --workspace-conf` >> >> 4. Run jcheck on a commit or a series of commits using a config file that I point to directly, that may have any name. >> `git skara jcheck --workspace-conf --conf-file ` >> >> 5. Run jcheck on the diff in my current workspace, either --staged or not using the .jcheck/conf in my workspace. >> `git skara jcheck --workspace-diff` > > Zhao Song has updated the pull request incrementally with one additional commit since the last revision: > > delete wholeHash() and update copyright headers Marked as reviewed by erikj (Lead). ------------- PR: https://git.openjdk.org/skara/pull/1428 From zsong at openjdk.org Thu Dec 1 17:59:38 2022 From: zsong at openjdk.org (Zhao Song) Date: Thu, 1 Dec 2022 17:59:38 GMT Subject: Integrated: 1690: Make jcheck cli able to use conf from workspace or other commit In-Reply-To: References: Message-ID: On Tue, 29 Nov 2022 18:24:50 GMT, Zhao Song wrote: > Currently, the command `git skara jcheck` can only use the `.jcheck/conf` configuration file in the current checking commit. > However, sometimes users want to run jcheck with different configurations to validate their commits. Therefore, we need to upgrade `git skara jcheck`. > > In this patch, `git skara jcheck` would be able to support following usecases. > > 1. Run jcheck on a commit or a series of commits using the .jcheck/conf in the same commit. (what we do by default today) > `git skara jcheck` > > 2. Run jcheck on a commit or a series of commits using the .jcheck/conf in a different specified commit. > `git skara jcheck --specified-conf-commit ` > > 3. Run jcheck on a commit or a series of commits using the .jcheck/conf in my workspace. > `git skara jcheck --workspace-conf` > > 4. Run jcheck on a commit or a series of commits using a config file that I point to directly, that may have any name. > `git skara jcheck --workspace-conf --conf-file ` > > 5. Run jcheck on the diff in my current workspace, either --staged or not using the .jcheck/conf in my workspace. > `git skara jcheck --workspace-diff` This pull request has now been integrated. Changeset: ef828913 Author: Zhao Song URL: https://git.openjdk.org/skara/commit/ef8289133213a045107a7a2e6508a03a67b03290 Stats: 266 lines in 8 files changed: 247 ins; 4 del; 15 mod 1690: Make jcheck cli able to use conf from workspace or other commit Reviewed-by: erikj ------------- PR: https://git.openjdk.org/skara/pull/1428 From erikj at openjdk.org Thu Dec 1 21:08:04 2022 From: erikj at openjdk.org (Erik Joelsson) Date: Thu, 1 Dec 2022 21:08:04 GMT Subject: Integrated: 1687: Workaround bug in GitLab where merge request LIST API may serve stale data In-Reply-To: References: Message-ID: On Mon, 28 Nov 2022 21:43:24 GMT, Erik Joelsson wrote: > This patch provides a workaround for a GitLab bug that we are currently encountering. The issue has been fixed upstream, but until we have it deployed in our instance, we need to work around it. > > GitLab sometimes serves stale data for merge requests from list queries. For Skara this means that label changes aren't always picked up. I have verified that up-to-date data is always received when fetching single merge requests by ID, so the patch simply re-fetches MRs to make sure the data is always up to date. I'm also adding logging to track when stale data is detected. This will help us verify (at least empirically) when the GitLab issue resolved for us. This pull request has now been integrated. Changeset: ba6e1ad2 Author: Erik Joelsson URL: https://git.openjdk.org/skara/commit/ba6e1ad2681041bcf92e34fbce1846e73f17946e Stats: 38 lines in 1 file changed: 38 ins; 0 del; 0 mod 1687: Workaround bug in GitLab where merge request LIST API may serve stale data Reviewed-by: zsong, ihse ------------- PR: https://git.openjdk.org/skara/pull/1427 From ihse at openjdk.org Thu Dec 1 23:35:20 2022 From: ihse at openjdk.org (Magnus Ihse Bursie) Date: Thu, 1 Dec 2022 23:35:20 GMT Subject: RFR: 1696: CSRBot uses wrong fixVersion when resolving CSR issues for PR In-Reply-To: References: Message-ID: On Tue, 29 Nov 2022 23:49:32 GMT, Erik Joelsson wrote: > I'm introducing a new module with this change where common bot logic can be housed, and put the updated version extraction logic there. This has been missing for a long time IMO and there is a lot of possible cleanup possible after introducing this module. Yay! ? ------------- PR: https://git.openjdk.org/skara/pull/1429 From ihse at openjdk.org Thu Dec 1 23:40:25 2022 From: ihse at openjdk.org (Magnus Ihse Bursie) Date: Thu, 1 Dec 2022 23:40:25 GMT Subject: RFR: 1696: CSRBot uses wrong fixVersion when resolving CSR issues for PR In-Reply-To: References: Message-ID: On Tue, 29 Nov 2022 23:49:32 GMT, Erik Joelsson wrote: > When CSRBot looks for a suitable issue in a set of Backports and main bug, it gets the fixVersion from the target branch of the PR. This is conceptually wrong, as the PR may be including a change to the fixVersion, and if that is the case, any associated bug should be resolved with fixVersion set to the value the PR changed it to. > > We currently have this PR where exactly that is happening: > https://github.com/openjdk/jdk/pull/10924 > > The associated bugs have fixVersion correctly set to 21, as this is the initial change for bumping the JDK version from 20 to 21. This is causing CSRBot to not be able to find the linked CSR issues. > > The same kind of logic is also present in PullRequestBot, and unfortunately the methods are copied. The reason for this is the lack of a common place to put application level shared logic for multiple bots. > > This patch changes the getVersion logic to first try to get it from the sourceHash of the PR and if that fails, fallback to the targetRef like before. I don't think this will fail much in practice, but there could be corner cases where it might be possible. The failure condition here is catching `UncheckedRestException` and checking the return code. This isn't ideal, and that particular exception is a bit of an implementation detail. Maybe we should consider providing a better API for getting file contents that may be missing? > > I'm introducing a new module with this change where common bot logic can be housed, and put the updated version extraction logic there. This has been missing for a long time IMO and there is a lot of possible cleanup possible after introducing this module. Several modules have similar classes doing almost the same thing. I also think that some of the "utils" logic in the forge module could find a better home here. Looks good to me. > the PR may be including a change to the fixVersion, and if that is the case, any associated bug should be resolved with fixVersion set to the value the PR changed it to. It took me a while to understand what you meant by this. The `fixVersion` threw me off. What you mean is that if the PR updates `version` field in `.jcheck/conf`, then this value that the PR changes it to should be used when determining the `fixVersion` for the corresponding JBS issue. I'm not nitpicking, it was just not clear to me, and I thought I should clarify for any other potential reviewers. ------------- Marked as reviewed by ihse (Reviewer). PR: https://git.openjdk.org/skara/pull/1429 From zsong at openjdk.org Thu Dec 1 23:40:29 2022 From: zsong at openjdk.org (Zhao Song) Date: Thu, 1 Dec 2022 23:40:29 GMT Subject: RFR: 1704: Add or update copyright header in some files Message-ID: The patch just updated copyright headers in some files. ------------- Commit messages: - SKARA-1704 Changes: https://git.openjdk.org/skara/pull/1431/files Webrev: https://webrevs.openjdk.org/?repo=skara&pr=1431&range=00 Issue: https://bugs.openjdk.org/browse/SKARA-1704 Stats: 1014 lines in 363 files changed: 682 ins; 0 del; 332 mod Patch: https://git.openjdk.org/skara/pull/1431.diff Fetch: git fetch https://git.openjdk.org/skara pull/1431/head:pull/1431 PR: https://git.openjdk.org/skara/pull/1431 From ihse at openjdk.org Thu Dec 1 23:42:07 2022 From: ihse at openjdk.org (Magnus Ihse Bursie) Date: Thu, 1 Dec 2022 23:42:07 GMT Subject: RFR: 1704: Add or update copyright header in some files In-Reply-To: References: Message-ID: On Thu, 1 Dec 2022 23:34:47 GMT, Zhao Song wrote: > The patch just updated copyright headers in some files. Have you used a script to get the last commit that touched the files to determine the updated copyright year? ------------- PR: https://git.openjdk.org/skara/pull/1431 From zsong at openjdk.org Thu Dec 1 23:48:40 2022 From: zsong at openjdk.org (Zhao Song) Date: Thu, 1 Dec 2022 23:48:40 GMT Subject: RFR: 1704: Add or update copyright header in some files In-Reply-To: References: Message-ID: On Thu, 1 Dec 2022 23:39:45 GMT, Magnus Ihse Bursie wrote: > Have you used a script to get the last commit that touched the files to determine the updated copyright year? No, I just used Intellij to see the git blame, and the last commit will have a '*'. ------------- PR: https://git.openjdk.org/skara/pull/1431 From ihse at openjdk.org Fri Dec 2 00:08:36 2022 From: ihse at openjdk.org (Magnus Ihse Bursie) Date: Fri, 2 Dec 2022 00:08:36 GMT Subject: RFR: 1704: Add or update copyright header in some files In-Reply-To: References: Message-ID: On Thu, 1 Dec 2022 23:34:47 GMT, Zhao Song wrote: > The patch just updated copyright headers in some files. ... but since you've gone through the effort, I'll rubber-stamp this PR for you. So you updated 350+ files manually? ? ? Apart from the fact that this must have been tedious work for you, I recommend in the future that you write a script to do stuff like this, since it is less likely to make a mistake. It is basically impossible for anyone to review if all these files have correct years, and you might have made a mistake when covering this many files. A script, on the other hand, can be presented to the reviewers, and then they can get additional confidence in that it has done the right thing. ------------- Marked as reviewed by ihse (Reviewer). PR: https://git.openjdk.org/skara/pull/1431 From zsong at openjdk.org Fri Dec 2 00:19:52 2022 From: zsong at openjdk.org (Zhao Song) Date: Fri, 2 Dec 2022 00:19:52 GMT Subject: RFR: 1704: Add or update copyright header in some files In-Reply-To: References: Message-ID: On Fri, 2 Dec 2022 00:05:14 GMT, Magnus Ihse Bursie wrote: > So you updated 350+ files manually? ? ? > > Apart from the fact that this must have been tedious work for you, I recommend in the future that you write a script to do stuff like this, since it is less likely to make a mistake. It is basically impossible for anyone to review if all these files have correct years, and you might have made a mistake when covering this many files. A script, on the other hand, can be presented to the reviewers, and then they can get additional confidence in that it has done the right thing. Ah, yes, I must find a better way to do this. Whatever, it's over for now. ? ------------- PR: https://git.openjdk.org/skara/pull/1431 From ihse at openjdk.org Fri Dec 2 00:30:22 2022 From: ihse at openjdk.org (Magnus Ihse Bursie) Date: Fri, 2 Dec 2022 00:30:22 GMT Subject: RFR: 1704: Add or update copyright header in some files In-Reply-To: References: Message-ID: <3PuMoOqZKCF4zNYvmTB8CzdVPAXQUaTMAF860Ojp2hg=.0e707378-f3e1-4029-a5ad-b5920ae5e557@github.com> On Thu, 1 Dec 2022 23:34:47 GMT, Zhao Song wrote: > The patch just updated copyright headers in some files. For example, here are some one-liners I use to mass-update copyright years: gt diff ${HASH}.. --name-only > ALLNAMES.txt sed -i -e 's/Copyright (c) ([0-9]*), Oracle and/or its affiliates. All rights reserved./Copyright (c) \1, 2022, Oracle and/or its affiliates. All rights reserved./g' $(cat ALLNAMES.txt ) sed -i -e 's/Copyright (c) ([0-9]*), [0-9]*, Oracle and/or its affiliates. All rights reserved./Copyright (c) \1, 2022, Oracle and/or its affiliates. All rights reserved./g' $(cat ALLNAMES.txt ) ------------- PR: https://git.openjdk.org/skara/pull/1431 From zsong at openjdk.org Fri Dec 2 00:30:22 2022 From: zsong at openjdk.org (Zhao Song) Date: Fri, 2 Dec 2022 00:30:22 GMT Subject: RFR: 1704: Add or update copyright header in some files In-Reply-To: <3PuMoOqZKCF4zNYvmTB8CzdVPAXQUaTMAF860Ojp2hg=.0e707378-f3e1-4029-a5ad-b5920ae5e557@github.com> References: <3PuMoOqZKCF4zNYvmTB8CzdVPAXQUaTMAF860Ojp2hg=.0e707378-f3e1-4029-a5ad-b5920ae5e557@github.com> Message-ID: On Fri, 2 Dec 2022 00:23:46 GMT, Magnus Ihse Bursie wrote: > For example, here are some one-liners I use to mass-update copyright years: > > ``` > gt diff ${HASH}.. --name-only > ALLNAMES.txt > > sed -i -e 's/Copyright (c) ([0-9]*), Oracle and/or its affiliates. All rights reserved./Copyright (c) \1, 2022, Oracle and/or its affiliates. All rights reserved./g' $(cat ALLNAMES.txt ) > > sed -i -e 's/Copyright (c) ([0-9]*), [0-9]*, Oracle and/or its affiliates. All rights reserved./Copyright (c) \1, 2022, Oracle and/or its affiliates. All rights reserved./g' $(cat ALLNAMES.txt ) > ``` Oh, thanks! Maybe I could run this script to check whether I made mistakes. ------------- PR: https://git.openjdk.org/skara/pull/1431 From ihse at openjdk.org Fri Dec 2 00:40:27 2022 From: ihse at openjdk.org (Magnus Ihse Bursie) Date: Fri, 2 Dec 2022 00:40:27 GMT Subject: RFR: 1704: Add or update copyright header in some files In-Reply-To: References: Message-ID: <3dqMfOXZAvyiJVGf_rFt3SXgM9UjNzeVX0yGvkga5TM=.6ddcbb2c-ad68-49a1-8237-c49bb0fe9357@github.com> On Thu, 1 Dec 2022 23:34:47 GMT, Zhao Song wrote: > The patch just updated copyright headers in some files. Sorry, that was just an example, to built upon. That particular sequence of command will find which files have been changed for a set of git commits, and will update the year to 2022. To get what you need for this PR is a bit more tricky. There is a script in the JDK, https://github.com/openjdk/jdk/blob/master/make/scripts/update_copyright_year.sh, which does something along these lines. I don't know if you can just download and run it in Skara, or if it needs adaptation. ------------- PR: https://git.openjdk.org/skara/pull/1431 From erikj at openjdk.org Fri Dec 2 13:32:44 2022 From: erikj at openjdk.org (Erik Joelsson) Date: Fri, 2 Dec 2022 13:32:44 GMT Subject: RFR: 1704: Add or update copyright header in some files In-Reply-To: References: Message-ID: On Thu, 1 Dec 2022 23:34:47 GMT, Zhao Song wrote: > The patch just updated copyright headers in some files. Marked as reviewed by erikj (Lead). ------------- PR: https://git.openjdk.org/skara/pull/1431 From erikj at openjdk.org Fri Dec 2 13:36:05 2022 From: erikj at openjdk.org (Erik Joelsson) Date: Fri, 2 Dec 2022 13:36:05 GMT Subject: RFR: 1704: Add or update copyright header in some files In-Reply-To: References: Message-ID: On Thu, 1 Dec 2022 23:34:47 GMT, Zhao Song wrote: > The patch just updated copyright headers in some files. As I said offline, I assumed you were just going to fix the files you had missed putting a header on recently, but this was a heroic effort, thanks for doing it! As Magnus says, it's probably better to try to automate this kind of change, but even if there is a mistake or two in here, I think we are much closer to being correct with this change than without it. We can always followup with an automated verification later. I have skimmed through and looked at the formatting. ------------- PR: https://git.openjdk.org/skara/pull/1431 From zsong at openjdk.org Fri Dec 2 20:49:00 2022 From: zsong at openjdk.org (Zhao Song) Date: Fri, 2 Dec 2022 20:49:00 GMT Subject: RFR: 1704: Add or update copyright header in some files In-Reply-To: References: Message-ID: On Fri, 2 Dec 2022 13:32:41 GMT, Erik Joelsson wrote: > As I said offline, I assumed you were just going to fix the files you had missed putting a header on recently, but this was a heroic effort, thanks for doing it! As Magnus says, it's probably better to try to automate this kind of change, but even if there is a mistake or two in here, I think we are much closer to being correct with this change than without it. We can always followup with an automated verification later. > > I have skimmed through and looked at the formatting. Thanks, Erik! ------------- PR: https://git.openjdk.org/skara/pull/1431 From zsong at openjdk.org Fri Dec 2 22:06:08 2022 From: zsong at openjdk.org (Zhao Song) Date: Fri, 2 Dec 2022 22:06:08 GMT Subject: RFR: 1708: Print more log to investigate SKARA-1707 Message-ID: Now, in method GitLabRepository#fileContents, first, it will try to read the content with once encoded filename, and if failed, it will try to read the content with double encoded filename. However, we don't print the log for the first try, so we couldn't know what happens in the first try. ------------- Commit messages: - SKARA-1708 Changes: https://git.openjdk.org/skara/pull/1432/files Webrev: https://webrevs.openjdk.org/?repo=skara&pr=1432&range=00 Issue: https://bugs.openjdk.org/browse/SKARA-1708 Stats: 2 lines in 1 file changed: 2 ins; 0 del; 0 mod Patch: https://git.openjdk.org/skara/pull/1432.diff Fetch: git fetch https://git.openjdk.org/skara pull/1432/head:pull/1432 PR: https://git.openjdk.org/skara/pull/1432 From erikj at openjdk.org Fri Dec 2 22:17:26 2022 From: erikj at openjdk.org (Erik Joelsson) Date: Fri, 2 Dec 2022 22:17:26 GMT Subject: RFR: 1708: Print more log to investigate SKARA-1707 In-Reply-To: References: Message-ID: On Fri, 2 Dec 2022 22:02:02 GMT, Zhao Song wrote: > Now, in method GitLabRepository#fileContents, first, it will try to read the content with once encoded filename, and if failed, it will try to read the content with double encoded filename. However, we don't print the log for the first try, so we couldn't know what happens in the first try. Marked as reviewed by erikj (Lead). ------------- PR: https://git.openjdk.org/skara/pull/1432 From zsong at openjdk.org Fri Dec 2 22:19:01 2022 From: zsong at openjdk.org (Zhao Song) Date: Fri, 2 Dec 2022 22:19:01 GMT Subject: Integrated: 1708: Print more log to investigate SKARA-1707 In-Reply-To: References: Message-ID: On Fri, 2 Dec 2022 22:02:02 GMT, Zhao Song wrote: > Now, in method GitLabRepository#fileContents, first, it will try to read the content with once encoded filename, and if failed, it will try to read the content with double encoded filename. However, we don't print the log for the first try, so we couldn't know what happens in the first try. This pull request has now been integrated. Changeset: 28e48507 Author: Zhao Song URL: https://git.openjdk.org/skara/commit/28e4850767a2f51c44b862f4e8423e7570bdf31a Stats: 2 lines in 1 file changed: 2 ins; 0 del; 0 mod 1708: Print more log to investigate SKARA-1707 Reviewed-by: erikj ------------- PR: https://git.openjdk.org/skara/pull/1432 From erikj at openjdk.org Mon Dec 5 15:16:48 2022 From: erikj at openjdk.org (Erik Joelsson) Date: Mon, 5 Dec 2022 15:16:48 GMT Subject: RFR: 1696: CSRBot uses wrong fixVersion when resolving CSR issues for PR [v2] In-Reply-To: References: Message-ID: > When CSRBot looks for a suitable issue in a set of Backports and main bug, it gets the fixVersion from the target branch of the PR. This is conceptually wrong, as the PR may be including a change to the fixVersion, and if that is the case, any associated bug should be resolved with fixVersion set to the value the PR changed it to. > > We currently have this PR where exactly that is happening: > https://github.com/openjdk/jdk/pull/10924 > > The associated bugs have fixVersion correctly set to 21, as this is the initial change for bumping the JDK version from 20 to 21. This is causing CSRBot to not be able to find the linked CSR issues. > > The same kind of logic is also present in PullRequestBot, and unfortunately the methods are copied. The reason for this is the lack of a common place to put application level shared logic for multiple bots. > > This patch changes the getVersion logic to first try to get it from the sourceHash of the PR and if that fails, fallback to the targetRef like before. I don't think this will fail much in practice, but there could be corner cases where it might be possible. The failure condition here is catching `UncheckedRestException` and checking the return code. This isn't ideal, and that particular exception is a bit of an implementation detail. Maybe we should consider providing a better API for getting file contents that may be missing? > > I'm introducing a new module with this change where common bot logic can be housed, and put the updated version extraction logic there. This has been missing for a long time IMO and there is a lot of possible cleanup possible after introducing this module. Several modules have similar classes doing almost the same thing. I also think that some of the "utils" logic in the forge module could find a better home here. Erik Joelsson has updated the pull request incrementally with one additional commit since the last revision: Fix another test ------------- Changes: - all: https://git.openjdk.org/skara/pull/1429/files - new: https://git.openjdk.org/skara/pull/1429/files/18a84ae5..e2c8ab4e Webrevs: - full: https://webrevs.openjdk.org/?repo=skara&pr=1429&range=01 - incr: https://webrevs.openjdk.org/?repo=skara&pr=1429&range=00-01 Stats: 8 lines in 1 file changed: 0 ins; 4 del; 4 mod Patch: https://git.openjdk.org/skara/pull/1429.diff Fetch: git fetch https://git.openjdk.org/skara pull/1429/head:pull/1429 PR: https://git.openjdk.org/skara/pull/1429 From erikj at openjdk.org Mon Dec 5 15:24:28 2022 From: erikj at openjdk.org (Erik Joelsson) Date: Mon, 5 Dec 2022 15:24:28 GMT Subject: RFR: 1701: Make use of Etag optional for MailingListArchiveReaderBot [v2] In-Reply-To: References: Message-ID: > We are experiencing long delays from when a user posts an email to a PR thread to when the MailingListArchiveReaderBot posts it as a comment to the PR. I think this is caused by the service provider for the mailinglists archive not handling Etags correctly. While that is sorted out, we need to be able to turn the use of etags off. This patch makes it a configuration parameter. Erik Joelsson has updated the pull request incrementally with one additional commit since the last revision: Fix test ------------- Changes: - all: https://git.openjdk.org/skara/pull/1430/files - new: https://git.openjdk.org/skara/pull/1430/files/fe4f7b79..0ab638b9 Webrevs: - full: https://webrevs.openjdk.org/?repo=skara&pr=1430&range=01 - incr: https://webrevs.openjdk.org/?repo=skara&pr=1430&range=00-01 Stats: 4 lines in 2 files changed: 0 ins; 0 del; 4 mod Patch: https://git.openjdk.org/skara/pull/1430.diff Fetch: git fetch https://git.openjdk.org/skara pull/1430/head:pull/1430 PR: https://git.openjdk.org/skara/pull/1430 From erikj at openjdk.org Mon Dec 5 17:56:05 2022 From: erikj at openjdk.org (Erik Joelsson) Date: Mon, 5 Dec 2022 17:56:05 GMT Subject: Integrated: 1696: CSRBot uses wrong fixVersion when resolving CSR issues for PR In-Reply-To: References: Message-ID: On Tue, 29 Nov 2022 23:49:32 GMT, Erik Joelsson wrote: > When CSRBot looks for a suitable issue in a set of Backports and main bug, it gets the fixVersion from the target branch of the PR. This is conceptually wrong, as the PR may be including a change to the fixVersion, and if that is the case, any associated bug should be resolved with fixVersion set to the value the PR changed it to. > > We currently have this PR where exactly that is happening: > https://github.com/openjdk/jdk/pull/10924 > > The associated bugs have fixVersion correctly set to 21, as this is the initial change for bumping the JDK version from 20 to 21. This is causing CSRBot to not be able to find the linked CSR issues. > > The same kind of logic is also present in PullRequestBot, and unfortunately the methods are copied. The reason for this is the lack of a common place to put application level shared logic for multiple bots. > > This patch changes the getVersion logic to first try to get it from the sourceHash of the PR and if that fails, fallback to the targetRef like before. I don't think this will fail much in practice, but there could be corner cases where it might be possible. The failure condition here is catching `UncheckedRestException` and checking the return code. This isn't ideal, and that particular exception is a bit of an implementation detail. Maybe we should consider providing a better API for getting file contents that may be missing? > > I'm introducing a new module with this change where common bot logic can be housed, and put the updated version extraction logic there. This has been missing for a long time IMO and there is a lot of possible cleanup possible after introducing this module. Several modules have similar classes doing almost the same thing. I also think that some of the "utils" logic in the forge module could find a better home here. This pull request has now been integrated. Changeset: e5add2dd Author: Erik Joelsson URL: https://git.openjdk.org/skara/commit/e5add2ddab6488e87d54a2ea26d89ddeeacf21d3 Stats: 209 lines in 14 files changed: 156 ins; 40 del; 13 mod 1696: CSRBot uses wrong fixVersion when resolving CSR issues for PR Reviewed-by: ihse ------------- PR: https://git.openjdk.org/skara/pull/1429 From erikj at openjdk.org Mon Dec 5 18:52:38 2022 From: erikj at openjdk.org (Erik Joelsson) Date: Mon, 5 Dec 2022 18:52:38 GMT Subject: Integrated: 1701: Make use of Etag optional for MailingListArchiveReaderBot In-Reply-To: References: Message-ID: <5xV295m7UvVTfmrvhgPh-vhN9IfWpN02c8xIhkdWSLA=.4f2973bc-7276-4de0-8dba-5f5c2d0239e7@github.com> On Thu, 1 Dec 2022 15:15:46 GMT, Erik Joelsson wrote: > We are experiencing long delays from when a user posts an email to a PR thread to when the MailingListArchiveReaderBot posts it as a comment to the PR. I think this is caused by the service provider for the mailinglists archive not handling Etags correctly. While that is sorted out, we need to be able to turn the use of etags off. This patch makes it a configuration parameter. This pull request has now been integrated. Changeset: 180efd2a Author: Erik Joelsson URL: https://git.openjdk.org/skara/commit/180efd2ab831e06493aea03f568d99c467ede4aa Stats: 25 lines in 7 files changed: 14 ins; 0 del; 11 mod 1701: Make use of Etag optional for MailingListArchiveReaderBot Reviewed-by: ihse ------------- PR: https://git.openjdk.org/skara/pull/1430 From erikj at openjdk.org Mon Dec 5 19:41:08 2022 From: erikj at openjdk.org (Erik Joelsson) Date: Mon, 5 Dec 2022 19:41:08 GMT Subject: RFR: 1713: Fix gradle deprecations Message-ID: This patch tries to fix the deprecation warnings currently printed by Gradle when building Skara. * enabled -> required: a simple rename of an attribute * TestReport was a little trickier, but as far as I can tell, this is how it's supposed to be done now, and it works just as well/bad as it did before this change. It's not clear to me what the testReport task is good for here, as it seems to me that it will only run if all tests are successful. * ModulePlugin.java: Was using some deprecated APIs. While at it I let Intellij fix some things that it was warning about. The null checks I removed were weird as the returned object was essentially an `Optional` (a different implementation of the same thing). Switched to using the modern instanceof assignment. * I updated gradle-wrapper.properties to match the version used by the Skara gradlew wrapper script. Intellij will by default pick the version from the properties file, so as long as we have that file around, we should make sure they match. I needed to do this to make the build succeed in Intellij as some of these changes didn't work in Gradle 7.2. ------------- Commit messages: - SKARA-1713 Changes: https://git.openjdk.org/skara/pull/1433/files Webrev: https://webrevs.openjdk.org/?repo=skara&pr=1433&range=00 Issue: https://bugs.openjdk.org/browse/SKARA-1713 Stats: 46 lines in 3 files changed: 11 ins; 15 del; 20 mod Patch: https://git.openjdk.org/skara/pull/1433.diff Fetch: git fetch https://git.openjdk.org/skara pull/1433/head:pull/1433 PR: https://git.openjdk.org/skara/pull/1433 From zsong at openjdk.org Mon Dec 5 20:01:02 2022 From: zsong at openjdk.org (Zhao Song) Date: Mon, 5 Dec 2022 20:01:02 GMT Subject: RFR: 1707: Warning of "No .jcheck/conf found" printed when there is no such problem actually Message-ID: In [SKARA-1393](https://bugs.openjdk.org/browse/SKARA-1393), I added some checks to ensure the target branch of a pr contains valid jcheck configuration. This change went live on November 28, however, some users found that the warning of "No .jcheck/conf found" printed after they integrated their pr. After more investigation, I am thinking it's maybe a orahub bug and filed an issue to orahub(https://jira-sd.mc1.oracleiaas.com/browse/ORAHUB-1638). Since when the issue happens, orahub will always return "Commit Not Found", so we could have a temporary workaround right now. For Orahub REST API: If commit not found, it will return "404 Commit Not Found" If file not found it will return "404 File Not Found" For Github REST API: If commit not found, it will return "No commit found for the ref " If file not found it will return "Not found" ------------- Commit messages: - SKARA-1707 Changes: https://git.openjdk.org/skara/pull/1434/files Webrev: https://webrevs.openjdk.org/?repo=skara&pr=1434&range=00 Issue: https://bugs.openjdk.org/browse/SKARA-1707 Stats: 24 lines in 5 files changed: 15 ins; 2 del; 7 mod Patch: https://git.openjdk.org/skara/pull/1434.diff Fetch: git fetch https://git.openjdk.org/skara pull/1434/head:pull/1434 PR: https://git.openjdk.org/skara/pull/1434 From ihse at openjdk.org Mon Dec 5 21:04:53 2022 From: ihse at openjdk.org (Magnus Ihse Bursie) Date: Mon, 5 Dec 2022 21:04:53 GMT Subject: RFR: 1713: Fix gradle deprecations In-Reply-To: References: Message-ID: <25_yUVEOsqoOYuOY6ucP-OL1AMl7yECfoZq86q_Rbsk=.d964f634-6347-4065-bf23-522f64f34c16@github.com> On Mon, 5 Dec 2022 19:34:07 GMT, Erik Joelsson wrote: > This patch tries to fix the deprecation warnings currently printed by Gradle when building Skara. > > * enabled -> required: a simple rename of an attribute > * TestReport was a little trickier, but as far as I can tell, this is how it's supposed to be done now, and it works just as well/bad as it did before this change. It's not clear to me what the testReport task is good for here, as it seems to me that it will only run if all tests are successful. > * ModulePlugin.java: Was using some deprecated APIs. While at it I let Intellij fix some things that it was warning about. The null checks I removed were weird as the returned object was essentially an `Optional` (a different implementation of the same thing). Switched to using the modern instanceof assignment. > * I updated gradle-wrapper.properties to match the version used by the Skara gradlew wrapper script. Intellij will by default pick the version from the properties file, so as long as we have that file around, we should make sure they match. I needed to do this to make the build succeed in Intellij as some of these changes didn't work in Gradle 7.2. LGTM. Thanks for doing the house cleaning stuff. ------------- Marked as reviewed by ihse (Reviewer). PR: https://git.openjdk.org/skara/pull/1433 From zsong at openjdk.org Mon Dec 5 21:42:59 2022 From: zsong at openjdk.org (Zhao Song) Date: Mon, 5 Dec 2022 21:42:59 GMT Subject: Withdrawn: 1707: Warning of "No .jcheck/conf found" printed when there is no such problem actually In-Reply-To: References: Message-ID: On Mon, 5 Dec 2022 19:56:46 GMT, Zhao Song wrote: > In [SKARA-1393](https://bugs.openjdk.org/browse/SKARA-1393), I added some checks to ensure the target branch of a pr contains valid jcheck configuration. This change went live on November 28, however, some users found that the warning of "No .jcheck/conf found" printed after they integrated their pr. After more investigation, I am thinking it's maybe a orahub bug and filed an issue to orahub(https://jira-sd.mc1.oracleiaas.com/browse/ORAHUB-1638). Since when the issue happens, orahub will always return "Commit Not Found", so we could have a temporary workaround right now. > > For Orahub REST API: > If commit not found, it will return "404 Commit Not Found" > > If file not found it will return "404 File Not Found" > > For Github REST API: > If commit not found, it will return "No commit found for the ref " > > If file not found it will return "Not found" This pull request has been closed without being integrated. ------------- PR: https://git.openjdk.org/skara/pull/1434 From erikj at openjdk.org Mon Dec 5 21:44:01 2022 From: erikj at openjdk.org (Erik Joelsson) Date: Mon, 5 Dec 2022 21:44:01 GMT Subject: Integrated: 1713: Fix gradle deprecations In-Reply-To: References: Message-ID: On Mon, 5 Dec 2022 19:34:07 GMT, Erik Joelsson wrote: > This patch tries to fix the deprecation warnings currently printed by Gradle when building Skara. > > * enabled -> required: a simple rename of an attribute > * TestReport was a little trickier, but as far as I can tell, this is how it's supposed to be done now, and it works just as well/bad as it did before this change. It's not clear to me what the testReport task is good for here, as it seems to me that it will only run if all tests are successful. > * ModulePlugin.java: Was using some deprecated APIs. While at it I let Intellij fix some things that it was warning about. The null checks I removed were weird as the returned object was essentially an `Optional` (a different implementation of the same thing). Switched to using the modern instanceof assignment. > * I updated gradle-wrapper.properties to match the version used by the Skara gradlew wrapper script. Intellij will by default pick the version from the properties file, so as long as we have that file around, we should make sure they match. I needed to do this to make the build succeed in Intellij as some of these changes didn't work in Gradle 7.2. This pull request has now been integrated. Changeset: 769c69b6 Author: Erik Joelsson URL: https://git.openjdk.org/skara/commit/769c69b6fb023bb06071a77275d5250042e22b21 Stats: 43 lines in 3 files changed: 11 ins; 15 del; 17 mod 1713: Fix gradle deprecations Reviewed-by: ihse ------------- PR: https://git.openjdk.org/skara/pull/1433 From zsong at openjdk.org Mon Dec 5 22:02:35 2022 From: zsong at openjdk.org (Zhao Song) Date: Mon, 5 Dec 2022 22:02:35 GMT Subject: RFR: 1707: Bot warns about "No .jcheck/conf found" when it should not Message-ID: In [SKARA-1393](https://bugs.openjdk.org/browse/SKARA-1393), I added some checks to ensure the target branch of a pr contains valid jcheck configuration. This change went live on November 28, however, some users found that the warning of "No .jcheck/conf found" printed after they integrated their pr. After more investigation, I am thinking it's maybe a GitLab bug. Since when the issue happens, GitLab will always return "Commit Not Found", so we could have a temporary workaround right now. For GitLab REST API: If commit not found, it will return "404 Commit Not Found" If file not found it will return "404 File Not Found" For Github REST API: If commit not found, it will return "No commit found for the ref " If file not found it will return "Not found" ------------- Commit messages: - SKARA-1707 Changes: https://git.openjdk.org/skara/pull/1435/files Webrev: https://webrevs.openjdk.org/?repo=skara&pr=1435&range=00 Issue: https://bugs.openjdk.org/browse/SKARA-1707 Stats: 24 lines in 5 files changed: 15 ins; 2 del; 7 mod Patch: https://git.openjdk.org/skara/pull/1435.diff Fetch: git fetch https://git.openjdk.org/skara pull/1435/head:pull/1435 PR: https://git.openjdk.org/skara/pull/1435 From erikj at openjdk.org Mon Dec 5 22:19:07 2022 From: erikj at openjdk.org (Erik Joelsson) Date: Mon, 5 Dec 2022 22:19:07 GMT Subject: RFR: 1707: Bot warns about "No .jcheck/conf found" when it should not In-Reply-To: References: Message-ID: On Mon, 5 Dec 2022 21:58:43 GMT, Zhao Song wrote: > In [SKARA-1393](https://bugs.openjdk.org/browse/SKARA-1393), I added some checks to ensure the target branch of a pr contains valid jcheck configuration. This change went live on November 28, however, some users found that the warning of "No .jcheck/conf found" printed after they integrated their pr. After more investigation, I am thinking it's maybe a GitLab bug. Since when the issue happens, GitLab will always return "Commit Not Found", so we could have a temporary workaround right now. > > For GitLab REST API: > If commit not found, it will return "404 Commit Not Found" > > If file not found it will return "404 File Not Found" > > For Github REST API: > If commit not found, it will return "No commit found for the ref " > > If file not found it will return "Not found" I think this is good enough for now, but as we talked about offline, we should follow up with changing the signature of `HostedRepository::fileContents` so that it returns an Optional instead. That Optional should only be empty on 404 File Not Found and any other error should throw an exception. bots/pr/src/main/java/org/openjdk/skara/bots/pr/LimitedCensusInstance.java line 77: > 75: } catch (UncheckedRestException e) { > 76: // Throw the exception if the error is not exactly "File not found" > 77: if (e.getStatusCode() != 404 || e.getBody().contains("Commit Not Found") || e.getBody().contains("No commit found")) { We should check for precisely "File Not Found" or "Not found". Suggestion: if (e.getStatusCode() != 404 || (!e.getBody().equals("File Not Found") && !e.getBody().equals("Not found"))) { ------------- PR: https://git.openjdk.org/skara/pull/1435 From zsong at openjdk.org Mon Dec 5 22:27:40 2022 From: zsong at openjdk.org (Zhao Song) Date: Mon, 5 Dec 2022 22:27:40 GMT Subject: RFR: 1707: Bot warns about "No .jcheck/conf found" when it should not [v2] In-Reply-To: References: Message-ID: > In [SKARA-1393](https://bugs.openjdk.org/browse/SKARA-1393), I added some checks to ensure the target branch of a pr contains valid jcheck configuration. This change went live on November 28, however, some users found that the warning of "No .jcheck/conf found" printed after they integrated their pr. After more investigation, I am thinking it's maybe a GitLab bug. Since when the issue happens, GitLab will always return "Commit Not Found", so we could have a temporary workaround right now. > > For GitLab REST API: > If commit not found, it will return "404 Commit Not Found" > > If file not found it will return "404 File Not Found" > > For Github REST API: > If commit not found, it will return "No commit found for the ref " > > If file not found it will return "Not found" Zhao Song has updated the pull request incrementally with one additional commit since the last revision: Update bots/pr/src/main/java/org/openjdk/skara/bots/pr/LimitedCensusInstance.java Co-authored-by: Erik Joelsson <37597443+erikj79 at users.noreply.github.com> ------------- Changes: - all: https://git.openjdk.org/skara/pull/1435/files - new: https://git.openjdk.org/skara/pull/1435/files/ad691448..f5c68229 Webrevs: - full: https://webrevs.openjdk.org/?repo=skara&pr=1435&range=01 - incr: https://webrevs.openjdk.org/?repo=skara&pr=1435&range=00-01 Stats: 1 line in 1 file changed: 0 ins; 0 del; 1 mod Patch: https://git.openjdk.org/skara/pull/1435.diff Fetch: git fetch https://git.openjdk.org/skara pull/1435/head:pull/1435 PR: https://git.openjdk.org/skara/pull/1435 From zsong at openjdk.org Mon Dec 5 22:27:41 2022 From: zsong at openjdk.org (Zhao Song) Date: Mon, 5 Dec 2022 22:27:41 GMT Subject: RFR: 1707: Bot warns about "No .jcheck/conf found" when it should not [v2] In-Reply-To: References: Message-ID: On Mon, 5 Dec 2022 22:12:23 GMT, Erik Joelsson wrote: >> Zhao Song has updated the pull request incrementally with one additional commit since the last revision: >> >> Update bots/pr/src/main/java/org/openjdk/skara/bots/pr/LimitedCensusInstance.java >> >> Co-authored-by: Erik Joelsson <37597443+erikj79 at users.noreply.github.com> > > bots/pr/src/main/java/org/openjdk/skara/bots/pr/LimitedCensusInstance.java line 77: > >> 75: } catch (UncheckedRestException e) { >> 76: // Throw the exception if the error is not exactly "File not found" >> 77: if (e.getStatusCode() != 404 || e.getBody().contains("Commit Not Found") || e.getBody().contains("No commit found")) { > > We should check for precisely "File Not Found" or "Not found". > > Suggestion: > > if (e.getStatusCode() != 404 || (!e.getBody().equals("File Not Found") && !e.getBody().equals("Not found"))) { Yes, it will be better. Thanks! ------------- PR: https://git.openjdk.org/skara/pull/1435 From erikj at openjdk.org Mon Dec 5 23:04:18 2022 From: erikj at openjdk.org (Erik Joelsson) Date: Mon, 5 Dec 2022 23:04:18 GMT Subject: RFR: 1707: Bot warns about "No .jcheck/conf found" when it should not [v2] In-Reply-To: References: Message-ID: <4Q65pBuaE5E56Wl3Y32OXIMk6Ls8SNBVFKl4A-TvDwI=.99130324-f112-4c46-820d-8f3f499e716c@github.com> On Mon, 5 Dec 2022 22:27:40 GMT, Zhao Song wrote: >> In [SKARA-1393](https://bugs.openjdk.org/browse/SKARA-1393), I added some checks to ensure the target branch of a pr contains valid jcheck configuration. This change went live on November 28, however, some users found that the warning of "No .jcheck/conf found" printed after they integrated their pr. After more investigation, I am thinking it's maybe a GitLab bug. Since when the issue happens, GitLab will always return "Commit Not Found", so we could have a temporary workaround right now. >> >> For GitLab REST API: >> If commit not found, it will return "404 Commit Not Found" >> >> If file not found it will return "404 File Not Found" >> >> For Github REST API: >> If commit not found, it will return "No commit found for the ref " >> >> If file not found it will return "Not found" > > Zhao Song has updated the pull request incrementally with one additional commit since the last revision: > > Update bots/pr/src/main/java/org/openjdk/skara/bots/pr/LimitedCensusInstance.java > > Co-authored-by: Erik Joelsson <37597443+erikj79 at users.noreply.github.com> Marked as reviewed by erikj (Lead). I see some tests are failing and it looks related. ------------- PR: https://git.openjdk.org/skara/pull/1435Changes requested by erikj (Lead). From zsong at openjdk.org Mon Dec 5 23:12:22 2022 From: zsong at openjdk.org (Zhao Song) Date: Mon, 5 Dec 2022 23:12:22 GMT Subject: RFR: 1707: Bot warns about "No .jcheck/conf found" when it should not [v2] In-Reply-To: <4Q65pBuaE5E56Wl3Y32OXIMk6Ls8SNBVFKl4A-TvDwI=.99130324-f112-4c46-820d-8f3f499e716c@github.com> References: <4Q65pBuaE5E56Wl3Y32OXIMk6Ls8SNBVFKl4A-TvDwI=.99130324-f112-4c46-820d-8f3f499e716c@github.com> Message-ID: <1zsN017d_Wi60rlaeXtl667xh3KpGZPbqW5oCett7z4=.6a5e2c14-d450-423f-b9eb-a058feb5659e@github.com> On Mon, 5 Dec 2022 23:00:40 GMT, Erik Joelsson wrote: > I see some tests are failing and it looks related. Yes, I will fix the tests. And I am also trying to make `HostedRepository::fileContents` returns `Optional`. Is it ok to modify this time? ------------- PR: https://git.openjdk.org/skara/pull/1435 From zsong at openjdk.org Mon Dec 5 23:49:09 2022 From: zsong at openjdk.org (Zhao Song) Date: Mon, 5 Dec 2022 23:49:09 GMT Subject: RFR: 1707: Bot warns about "No .jcheck/conf found" when it should not [v3] In-Reply-To: References: Message-ID: > In [SKARA-1393](https://bugs.openjdk.org/browse/SKARA-1393), I added some checks to ensure the target branch of a pr contains valid jcheck configuration. This change went live on November 28, however, some users found that the warning of "No .jcheck/conf found" printed after they integrated their pr. After more investigation, I am thinking it's maybe a GitLab bug. Since when the issue happens, GitLab will always return "Commit Not Found", so we could have a temporary workaround right now. > > For GitLab REST API: > If commit not found, it will return "404 Commit Not Found" > > If file not found it will return "404 File Not Found" > > For Github REST API: > If commit not found, it will return "No commit found for the ref " > > If file not found it will return "Not Found" Zhao Song has updated the pull request incrementally with one additional commit since the last revision: make HostedRepository::fileContents return Optional ------------- Changes: - all: https://git.openjdk.org/skara/pull/1435/files - new: https://git.openjdk.org/skara/pull/1435/files/f5c68229..df16b511 Webrevs: - full: https://webrevs.openjdk.org/?repo=skara&pr=1435&range=02 - incr: https://webrevs.openjdk.org/?repo=skara&pr=1435&range=01-02 Stats: 83 lines in 16 files changed: 26 ins; 13 del; 44 mod Patch: https://git.openjdk.org/skara/pull/1435.diff Fetch: git fetch https://git.openjdk.org/skara pull/1435/head:pull/1435 PR: https://git.openjdk.org/skara/pull/1435 From zsong at openjdk.org Mon Dec 5 23:55:09 2022 From: zsong at openjdk.org (Zhao Song) Date: Mon, 5 Dec 2022 23:55:09 GMT Subject: RFR: 1707: Bot warns about "No .jcheck/conf found" when it should not [v4] In-Reply-To: References: Message-ID: > In [SKARA-1393](https://bugs.openjdk.org/browse/SKARA-1393), I added some checks to ensure the target branch of a pr contains valid jcheck configuration. This change went live on November 28, however, some users found that the warning of "No .jcheck/conf found" printed after they integrated their pr. After more investigation, I am thinking it's maybe a GitLab bug. Since when the issue happens, GitLab will always return "Commit Not Found", so we could have a temporary workaround right now. > > For GitLab REST API: > If commit not found, it will return "404 Commit Not Found" > > If file not found it will return "404 File Not Found" > > For Github REST API: > If commit not found, it will return "No commit found for the ref " > > If file not found it will return "Not Found" Zhao Song has updated the pull request incrementally with one additional commit since the last revision: fix a problem ------------- Changes: - all: https://git.openjdk.org/skara/pull/1435/files - new: https://git.openjdk.org/skara/pull/1435/files/df16b511..d800c86c Webrevs: - full: https://webrevs.openjdk.org/?repo=skara&pr=1435&range=03 - incr: https://webrevs.openjdk.org/?repo=skara&pr=1435&range=02-03 Stats: 2 lines in 2 files changed: 0 ins; 2 del; 0 mod Patch: https://git.openjdk.org/skara/pull/1435.diff Fetch: git fetch https://git.openjdk.org/skara pull/1435/head:pull/1435 PR: https://git.openjdk.org/skara/pull/1435 From ihse at openjdk.org Tue Dec 6 10:40:15 2022 From: ihse at openjdk.org (Magnus Ihse Bursie) Date: Tue, 6 Dec 2022 10:40:15 GMT Subject: RFR: 1707: Bot warns about "No .jcheck/conf found" when it should not [v4] In-Reply-To: References: Message-ID: On Mon, 5 Dec 2022 23:55:09 GMT, Zhao Song wrote: >> In [SKARA-1393](https://bugs.openjdk.org/browse/SKARA-1393), I added some checks to ensure the target branch of a pr contains valid jcheck configuration. This change went live on November 28, however, some users found that the warning of "No .jcheck/conf found" printed after they integrated their pr. After more investigation, I am thinking it's maybe a GitLab bug. Since when the issue happens, GitLab will always return "Commit Not Found", so we could have a temporary workaround right now. >> >> For GitLab REST API: >> If commit not found, it will return "404 Commit Not Found" >> >> If file not found it will return "404 File Not Found" >> >> For Github REST API: >> If commit not found, it will return "No commit found for the ref " >> >> If file not found it will return "Not Found" > > Zhao Song has updated the pull request incrementally with one additional commit since the last revision: > > fix a problem Apart from the naming issue I think this looks good, but please await Erik's review as well. forge/src/main/java/org/openjdk/skara/forge/github/GitHubRepository.java line 265: > 263: @Override > 264: public Optional fileContents(String filename, String ref) { > 265: JSONValue conf; That's a bit odd name for the variable. I realize you did not come up with it, but I think it is a remnant from the time this function was only used to read the configuration file. I recommend changing it to `content`. forge/src/main/java/org/openjdk/skara/forge/gitlab/GitLabRepository.java line 291: > 289: public Optional fileContents(String filename, String ref) { > 290: var confName = URLEncoder.encode(filename, StandardCharsets.UTF_8); > 291: JSONValue conf = null; The same here. Also update `confName`, perhaps `encodedFileName`? ------------- Marked as reviewed by ihse (Reviewer). PR: https://git.openjdk.org/skara/pull/1435 From zsong at openjdk.org Tue Dec 6 16:56:59 2022 From: zsong at openjdk.org (Zhao Song) Date: Tue, 6 Dec 2022 16:56:59 GMT Subject: RFR: 1707: Bot warns about "No .jcheck/conf found" when it should not [v4] In-Reply-To: References: Message-ID: <3X_8chIbfj6dTAjE1NXwnr8FpE6_RG8K0hYn_a_oxS0=.5dbe32d2-718b-4fdb-9450-dd0a3dcc647b@github.com> On Tue, 6 Dec 2022 10:37:54 GMT, Magnus Ihse Bursie wrote: > Apart from the naming issue I think this looks good, but please await Erik's review as well. Thanks for the review! I will change the variable name. ------------- PR: https://git.openjdk.org/skara/pull/1435 From zsong at openjdk.org Tue Dec 6 18:10:54 2022 From: zsong at openjdk.org (Zhao Song) Date: Tue, 6 Dec 2022 18:10:54 GMT Subject: RFR: 1707: Bot warns about "No .jcheck/conf found" when it should not [v5] In-Reply-To: References: Message-ID: > In [SKARA-1393](https://bugs.openjdk.org/browse/SKARA-1393), I added some checks to ensure the target branch of a pr contains valid jcheck configuration. This change went live on November 28, however, some users found that the warning of "No .jcheck/conf found" printed after they integrated their pr. After more investigation, I am thinking it's maybe a GitLab bug. Since when the issue happens, GitLab will always return "Commit Not Found", so we could have a temporary workaround right now. > > For GitLab REST API: > If commit not found, it will return "404 Commit Not Found" > > If file not found it will return "404 File Not Found" > > For Github REST API: > If commit not found, it will return "No commit found for the ref " > > If file not found it will return "Not Found" Zhao Song has updated the pull request incrementally with one additional commit since the last revision: rename variables ------------- Changes: - all: https://git.openjdk.org/skara/pull/1435/files - new: https://git.openjdk.org/skara/pull/1435/files/d800c86c..2adf3643 Webrevs: - full: https://webrevs.openjdk.org/?repo=skara&pr=1435&range=04 - incr: https://webrevs.openjdk.org/?repo=skara&pr=1435&range=03-04 Stats: 10 lines in 2 files changed: 0 ins; 0 del; 10 mod Patch: https://git.openjdk.org/skara/pull/1435.diff Fetch: git fetch https://git.openjdk.org/skara pull/1435/head:pull/1435 PR: https://git.openjdk.org/skara/pull/1435 From erikj at openjdk.org Tue Dec 6 18:53:35 2022 From: erikj at openjdk.org (Erik Joelsson) Date: Tue, 6 Dec 2022 18:53:35 GMT Subject: RFR: 1707: Bot warns about "No .jcheck/conf found" when it should not [v5] In-Reply-To: References: Message-ID: On Tue, 6 Dec 2022 18:10:54 GMT, Zhao Song wrote: >> In [SKARA-1393](https://bugs.openjdk.org/browse/SKARA-1393), I added some checks to ensure the target branch of a pr contains valid jcheck configuration. This change went live on November 28, however, some users found that the warning of "No .jcheck/conf found" printed after they integrated their pr. After more investigation, I am thinking it's maybe a GitLab bug. Since when the issue happens, GitLab will always return "Commit Not Found", so we could have a temporary workaround right now. >> >> For GitLab REST API: >> If commit not found, it will return "404 Commit Not Found" >> >> If file not found it will return "404 File Not Found" >> >> For Github REST API: >> If commit not found, it will return "No commit found for the ref " >> >> If file not found it will return "Not Found" > > Zhao Song has updated the pull request incrementally with one additional commit since the last revision: > > rename variables bots/common/src/main/java/org/openjdk/skara/bots/common/BotUtils.java line 46: > 44: String confFile; > 45: confFile = pr.repository().fileContents(".jcheck/conf", pr.headHash().hex()) > 46: .orElse(pr.repository().fileContents(".jcheck/conf", pr.targetRef()).orElseThrow()); Could you add a message to the exception in orElseThrow describing what's happening. Something like () -> new RuntimeException("Missing .jcheck/conf in both src and target of PR " + pr) Also, no need to split declaration of confFile to a separate line anymore, nor declare the type. bots/pr/src/main/java/org/openjdk/skara/bots/pr/LimitedCensusInstance.java line 77: > 75: conf = Optional.of(Arrays.stream(contents.get().split("\n")).toList()); > 76: } > 77: return conf.map(JCheckConfiguration::parse); This seems a bit awkward. I think something like this should do it (untested and likely not completely right): return remoteRepo.fileContents(name, ref).map(contents -> JCheckConfiguration.parse(Arrays.stream(contents.split("\n")).toList())); forge/src/main/java/org/openjdk/skara/forge/github/GitHubRepository.java line 276: > 274: } > 275: throw e; > 276: } Instead of catching this exception, please use the onError method on the request to register an error handler. forge/src/main/java/org/openjdk/skara/forge/gitlab/GitLabRepository.java line 298: > 296: // Retry once with additional escaping of the path fragment > 297: // Only retry when the error is exactly "File Not Found" > 298: if (response.statusCode() == 404 && response.body().contains("File Not Found")) { The comment says 'exactly "File Not Found"' but the conditional uses `.contains`. I think we can switch to `.equals`. forge/src/main/java/org/openjdk/skara/forge/gitlab/GitLabRepository.java line 313: > 311: return Optional.empty(); > 312: } > 313: throw e; Instead of catching exception, I think it's better to register yet another `.onError` handler on the nested request object. You would then return a special JSONValue instance that you can check for and then the outer method can return `Optional.empty()`. There are other examples of this pattern in this file. network/src/main/java/org/openjdk/skara/network/UncheckedRestException.java line 13: > 11: String body; > 12: > 13: public UncheckedRestException(String message, int statusCode, String body) { Even if we don't end up using this extra body field on this exception type, I think it's good to have, so please leave it in. test/src/main/java/org/openjdk/skara/test/TestHostedRepository.java line 203: > 201: try { > 202: var lines = localRepository.lines(Path.of(filename), localRepository.resolve(ref).orElseThrow()); > 203: return Optional.of(String.join("\n", lines.orElseThrow())); This should be expressed as something like: return lines.map(content -> String.join("\n", content)); This makes us preserve an Optional.empty() returned from the localRepository. ------------- PR: https://git.openjdk.org/skara/pull/1435 From zsong at openjdk.org Tue Dec 6 19:03:56 2022 From: zsong at openjdk.org (Zhao Song) Date: Tue, 6 Dec 2022 19:03:56 GMT Subject: RFR: 1707: Bot warns about "No .jcheck/conf found" when it should not [v5] In-Reply-To: References: Message-ID: On Tue, 6 Dec 2022 18:25:35 GMT, Erik Joelsson wrote: >> Zhao Song has updated the pull request incrementally with one additional commit since the last revision: >> >> rename variables > > bots/common/src/main/java/org/openjdk/skara/bots/common/BotUtils.java line 46: > >> 44: String confFile; >> 45: confFile = pr.repository().fileContents(".jcheck/conf", pr.headHash().hex()) >> 46: .orElse(pr.repository().fileContents(".jcheck/conf", pr.targetRef()).orElseThrow()); > > Could you add a message to the exception in orElseThrow describing what's happening. Something like > > () -> new RuntimeException("Missing .jcheck/conf in both src and target of PR " + pr) > > > Also, no need to split declaration of confFile to a separate line anymore, nor declare the type. Sure, will fix it ------------- PR: https://git.openjdk.org/skara/pull/1435 From erikj at openjdk.org Tue Dec 6 19:09:51 2022 From: erikj at openjdk.org (Erik Joelsson) Date: Tue, 6 Dec 2022 19:09:51 GMT Subject: RFR: 1710: CommitCommandWorkItem always fetches Commit Message-ID: During bot restart, quite a few CommitCommandWorkItems are spawned. Most of them will discover that nothing needs to be done. One of the first things done in the run method is fetching the commit from the remote repository. The Commit instance will always be created with the full set of diffs for the commit. In certain cases, where commits are very large, this can take considerable time. Specifically in the jfx repos, there are examples where the REST request is split into 147 pages. (e.g. https://github.com/openjdk/jfx11u/commit/4bd6877dbd46e37f6774c50dcdc98a8a1ab41214) The way these WorkItems are spawned on startup, most of these gigantic commits will be included for a long time. The Commit instance isn't needed to figure out if a new command is present, just the comments are. This means we can move this query to after we have established that a command is to be executed. To further reduce overhead when fetching commits from remote repositories, I'm also making the inclusion of the "parentDiffs" optional. Of all the uses of `HostedRepository::commit`, there is currently only one that actually needs the diffs (BackportCommand). This will require the caller to be mindful of the state of the returned `Commit` object, especially when sending it other consumers, but I believe it's worth it to reduce unnecessary load on the system. This also affects `Forge::search`, where there is one other use where the diffs are used (when evaluating if a backport is clean). ------------- Commit messages: - SKARA-1710 Changes: https://git.openjdk.org/skara/pull/1436/files Webrev: https://webrevs.openjdk.org/?repo=skara&pr=1436&range=00 Issue: https://bugs.openjdk.org/browse/SKARA-1710 Stats: 82 lines in 12 files changed: 41 ins; 1 del; 40 mod Patch: https://git.openjdk.org/skara/pull/1436.diff Fetch: git fetch https://git.openjdk.org/skara pull/1436/head:pull/1436 PR: https://git.openjdk.org/skara/pull/1436 From zsong at openjdk.org Tue Dec 6 19:40:17 2022 From: zsong at openjdk.org (Zhao Song) Date: Tue, 6 Dec 2022 19:40:17 GMT Subject: RFR: 1707: Bot warns about "No .jcheck/conf found" when it should not [v5] In-Reply-To: References: Message-ID: On Tue, 6 Dec 2022 18:40:25 GMT, Erik Joelsson wrote: >> Zhao Song has updated the pull request incrementally with one additional commit since the last revision: >> >> rename variables > > forge/src/main/java/org/openjdk/skara/forge/github/GitHubRepository.java line 276: > >> 274: } >> 275: throw e; >> 276: } > > Instead of catching this exception, please use the onError method on the request to register an error handler. Sorry, I couldn't find a good way to use errorHandler with the current `RestRequest#transformBadResponse`. We will not be able to return `Optional.empty()` when the error is 404 Not Found. > forge/src/main/java/org/openjdk/skara/forge/gitlab/GitLabRepository.java line 298: > >> 296: // Retry once with additional escaping of the path fragment >> 297: // Only retry when the error is exactly "File Not Found" >> 298: if (response.statusCode() == 404 && response.body().contains("File Not Found")) { > > The comment says 'exactly "File Not Found"' but the conditional uses `.contains`. I think we can switch to `.equals`. I use contains because the real body is `{"message":"404 File Not Found"}`. I will use `response.body().equals("{"message":"404 File Not Found"}")` ------------- PR: https://git.openjdk.org/skara/pull/1435 From ihse at openjdk.org Tue Dec 6 19:48:44 2022 From: ihse at openjdk.org (Magnus Ihse Bursie) Date: Tue, 6 Dec 2022 19:48:44 GMT Subject: RFR: 1707: Bot warns about "No .jcheck/conf found" when it should not [v5] In-Reply-To: References: Message-ID: On Tue, 6 Dec 2022 19:00:05 GMT, Zhao Song wrote: >> bots/common/src/main/java/org/openjdk/skara/bots/common/BotUtils.java line 46: >> >>> 44: String confFile; >>> 45: confFile = pr.repository().fileContents(".jcheck/conf", pr.headHash().hex()) >>> 46: .orElse(pr.repository().fileContents(".jcheck/conf", pr.targetRef()).orElseThrow()); >> >> Could you add a message to the exception in orElseThrow describing what's happening. Something like >> >> () -> new RuntimeException("Missing .jcheck/conf in both src and target of PR " + pr) >> >> >> Also, no need to split declaration of confFile to a separate line anymore, nor declare the type. > > Sure, will fix it But Eriks suggested message sounds a bit contradictory. I suggest instead using: "Could not find .jcheck/conf in neither src or target of PR" ------------- PR: https://git.openjdk.org/skara/pull/1435 From zsong at openjdk.org Tue Dec 6 19:48:45 2022 From: zsong at openjdk.org (Zhao Song) Date: Tue, 6 Dec 2022 19:48:45 GMT Subject: RFR: 1707: Bot warns about "No .jcheck/conf found" when it should not [v5] In-Reply-To: References: Message-ID: On Tue, 6 Dec 2022 19:34:40 GMT, Zhao Song wrote: >> forge/src/main/java/org/openjdk/skara/forge/github/GitHubRepository.java line 276: >> >>> 274: } >>> 275: throw e; >>> 276: } >> >> Instead of catching this exception, please use the onError method on the request to register an error handler. > > Sorry, I couldn't find a good way to use errorHandler with the current `RestRequest#transformBadResponse`. > We will not be able to return `Optional.empty()` when the error is 404 Not Found. Ah, see your comments below. I know how to do it now. Thank you ------------- PR: https://git.openjdk.org/skara/pull/1435 From zsong at openjdk.org Tue Dec 6 20:30:53 2022 From: zsong at openjdk.org (Zhao Song) Date: Tue, 6 Dec 2022 20:30:53 GMT Subject: RFR: 1707: Bot warns about "No .jcheck/conf found" when it should not [v5] In-Reply-To: References: Message-ID: On Tue, 6 Dec 2022 18:36:56 GMT, Erik Joelsson wrote: >> Zhao Song has updated the pull request incrementally with one additional commit since the last revision: >> >> rename variables > > bots/pr/src/main/java/org/openjdk/skara/bots/pr/LimitedCensusInstance.java line 77: > >> 75: conf = Optional.of(Arrays.stream(contents.get().split("\n")).toList()); >> 76: } >> 77: return conf.map(JCheckConfiguration::parse); > > This seems a bit awkward. I think something like this should do it (untested and likely not completely right): > > return remoteRepo.fileContents(name, ref).map(contents -> JCheckConfiguration.parse(Arrays.stream(contents.split("\n")).toList())); I think it's right ------------- PR: https://git.openjdk.org/skara/pull/1435 From erikj at openjdk.org Tue Dec 6 20:43:09 2022 From: erikj at openjdk.org (Erik Joelsson) Date: Tue, 6 Dec 2022 20:43:09 GMT Subject: RFR: 1707: Bot warns about "No .jcheck/conf found" when it should not [v5] In-Reply-To: References: Message-ID: On Tue, 6 Dec 2022 19:36:27 GMT, Zhao Song wrote: >> forge/src/main/java/org/openjdk/skara/forge/gitlab/GitLabRepository.java line 298: >> >>> 296: // Retry once with additional escaping of the path fragment >>> 297: // Only retry when the error is exactly "File Not Found" >>> 298: if (response.statusCode() == 404 && response.body().contains("File Not Found")) { >> >> The comment says 'exactly "File Not Found"' but the conditional uses `.contains`. I think we can switch to `.equals`. > > I use contains because the real body is `{"message":"404 File Not Found"}`. I will use `response.body().equals("{"message":"404 File Not Found"}")` Hm, better parse the Json in that case. I don't think the string comparison should include any json code. Something like message.endsWith("File Not Found") seems good enough. ------------- PR: https://git.openjdk.org/skara/pull/1435 From zsong at openjdk.org Tue Dec 6 21:44:04 2022 From: zsong at openjdk.org (Zhao Song) Date: Tue, 6 Dec 2022 21:44:04 GMT Subject: RFR: 1707: Bot warns about "No .jcheck/conf found" when it should not [v6] In-Reply-To: References: Message-ID: <3wzN3S31uIykpXnSZQvg3AkV4X-eiEhFYkl5nMIGZEk=.32b1822b-9146-4e0d-bb05-91eb0325d441@github.com> > In [SKARA-1393](https://bugs.openjdk.org/browse/SKARA-1393), I added some checks to ensure the target branch of a pr contains valid jcheck configuration. This change went live on November 28, however, some users found that the warning of "No .jcheck/conf found" printed after they integrated their pr. After more investigation, I am thinking it's maybe a GitLab bug. Since when the issue happens, GitLab will always return "Commit Not Found", so we could have a temporary workaround right now. > > For GitLab REST API: > If commit not found, it will return "404 Commit Not Found" > > If file not found it will return "404 File Not Found" > > For Github REST API: > If commit not found, it will return "No commit found for the ref " > > If file not found it will return "Not Found" Zhao Song has updated the pull request incrementally with one additional commit since the last revision: fix problems ------------- Changes: - all: https://git.openjdk.org/skara/pull/1435/files - new: https://git.openjdk.org/skara/pull/1435/files/2adf3643..15bf695a Webrevs: - full: https://webrevs.openjdk.org/?repo=skara&pr=1435&range=05 - incr: https://webrevs.openjdk.org/?repo=skara&pr=1435&range=04-05 Stats: 67 lines in 12 files changed: 8 ins; 19 del; 40 mod Patch: https://git.openjdk.org/skara/pull/1435.diff Fetch: git fetch https://git.openjdk.org/skara pull/1435/head:pull/1435 PR: https://git.openjdk.org/skara/pull/1435 From zsong at openjdk.org Tue Dec 6 21:53:32 2022 From: zsong at openjdk.org (Zhao Song) Date: Tue, 6 Dec 2022 21:53:32 GMT Subject: RFR: 1707: Bot warns about "No .jcheck/conf found" when it should not [v7] In-Reply-To: References: Message-ID: > In [SKARA-1393](https://bugs.openjdk.org/browse/SKARA-1393), I added some checks to ensure the target branch of a pr contains valid jcheck configuration. This change went live on November 28, however, some users found that the warning of "No .jcheck/conf found" printed after they integrated their pr. After more investigation, I am thinking it's maybe a GitLab bug. Since when the issue happens, GitLab will always return "Commit Not Found", so we could have a temporary workaround right now. > > For GitLab REST API: > If commit not found, it will return "404 Commit Not Found" > > If file not found it will return "404 File Not Found" > > For Github REST API: > If commit not found, it will return "No commit found for the ref " > > If file not found it will return "Not Found" Zhao Song has updated the pull request incrementally with one additional commit since the last revision: fix problems ------------- Changes: - all: https://git.openjdk.org/skara/pull/1435/files - new: https://git.openjdk.org/skara/pull/1435/files/15bf695a..82cce5eb Webrevs: - full: https://webrevs.openjdk.org/?repo=skara&pr=1435&range=06 - incr: https://webrevs.openjdk.org/?repo=skara&pr=1435&range=05-06 Stats: 10 lines in 4 files changed: 6 ins; 1 del; 3 mod Patch: https://git.openjdk.org/skara/pull/1435.diff Fetch: git fetch https://git.openjdk.org/skara pull/1435/head:pull/1435 PR: https://git.openjdk.org/skara/pull/1435 From erikj at openjdk.org Tue Dec 6 23:42:39 2022 From: erikj at openjdk.org (Erik Joelsson) Date: Tue, 6 Dec 2022 23:42:39 GMT Subject: RFR: 1707: Bot warns about "No .jcheck/conf found" when it should not [v7] In-Reply-To: References: Message-ID: On Tue, 6 Dec 2022 21:53:32 GMT, Zhao Song wrote: >> In [SKARA-1393](https://bugs.openjdk.org/browse/SKARA-1393), I added some checks to ensure the target branch of a pr contains valid jcheck configuration. This change went live on November 28, however, some users found that the warning of "No .jcheck/conf found" printed after they integrated their pr. After more investigation, I am thinking it's maybe a GitLab bug. Since when the issue happens, GitLab will always return "Commit Not Found", so we could have a temporary workaround right now. >> >> For GitLab REST API: >> If commit not found, it will return "404 Commit Not Found" >> >> If file not found it will return "404 File Not Found" >> >> For Github REST API: >> If commit not found, it will return "No commit found for the ref " >> >> If file not found it will return "Not Found" > > Zhao Song has updated the pull request incrementally with one additional commit since the last revision: > > fix problems Marked as reviewed by erikj (Lead). ------------- PR: https://git.openjdk.org/skara/pull/1435 From zsong at openjdk.org Tue Dec 6 23:56:23 2022 From: zsong at openjdk.org (Zhao Song) Date: Tue, 6 Dec 2022 23:56:23 GMT Subject: Integrated: 1707: Bot warns about "No .jcheck/conf found" when it should not In-Reply-To: References: Message-ID: On Mon, 5 Dec 2022 21:58:43 GMT, Zhao Song wrote: > In [SKARA-1393](https://bugs.openjdk.org/browse/SKARA-1393), I added some checks to ensure the target branch of a pr contains valid jcheck configuration. This change went live on November 28, however, some users found that the warning of "No .jcheck/conf found" printed after they integrated their pr. After more investigation, I am thinking it's maybe a GitLab bug. Since when the issue happens, GitLab will always return "Commit Not Found", so we could have a temporary workaround right now. > > For GitLab REST API: > If commit not found, it will return "404 Commit Not Found" > > If file not found it will return "404 File Not Found" > > For Github REST API: > If commit not found, it will return "No commit found for the ref " > > If file not found it will return "Not Found" This pull request has now been integrated. Changeset: 1d7e355c Author: Zhao Song URL: https://git.openjdk.org/skara/commit/1d7e355c863768214389f64ecfb47193c78964ef Stats: 95 lines in 16 files changed: 36 ins; 18 del; 41 mod 1707: Bot warns about "No .jcheck/conf found" when it should not Reviewed-by: erikj, ihse ------------- PR: https://git.openjdk.org/skara/pull/1435 From erikj at openjdk.org Wed Dec 7 00:01:52 2022 From: erikj at openjdk.org (Erik Joelsson) Date: Wed, 7 Dec 2022 00:01:52 GMT Subject: RFR: 1709: Only fetch comments once in PullRequestWorkItem in pr bot Message-ID: This patch provides a minor optimization for `PullRequestWorkItem` subclasses in the pr bot by avoiding re-fetching of pr.comments(). I initially started this patch with a much bigger optimization in mind, but it turned out to not work. I still think it worth applying the remaining patch since I've gone through the trouble of implementing it. ------------- Commit messages: - SKARA-1709 Changes: https://git.openjdk.org/skara/pull/1437/files Webrev: https://webrevs.openjdk.org/?repo=skara&pr=1437&range=00 Issue: https://bugs.openjdk.org/browse/SKARA-1709 Stats: 25 lines in 5 files changed: 15 ins; 6 del; 4 mod Patch: https://git.openjdk.org/skara/pull/1437.diff Fetch: git fetch https://git.openjdk.org/skara pull/1437/head:pull/1437 PR: https://git.openjdk.org/skara/pull/1437 From zsong at openjdk.org Wed Dec 7 00:08:51 2022 From: zsong at openjdk.org (Zhao Song) Date: Wed, 7 Dec 2022 00:08:51 GMT Subject: RFR: 1716: Revert SKARA-1655 Message-ID: In [SKARA-1655](https://bugs.openjdk.org/browse/SKARA-1655), I implemented a commit check named jCheckConfCheck. It was supposed to check if .jcheck/conf is valid in user's commit. But now, we have a better idea([SKARA-1691](https://bugs.openjdk.org/browse/SKARA-1691)) to check and we no longer need this check any more. Also, this check is not using by any repo right now. It doesn't hurt to revert it. ------------- Commit messages: - SKARA-1716 Changes: https://git.openjdk.org/skara/pull/1438/files Webrev: https://webrevs.openjdk.org/?repo=skara&pr=1438&range=00 Issue: https://bugs.openjdk.org/browse/SKARA-1716 Stats: 365 lines in 9 files changed: 0 ins; 364 del; 1 mod Patch: https://git.openjdk.org/skara/pull/1438.diff Fetch: git fetch https://git.openjdk.org/skara pull/1438/head:pull/1438 PR: https://git.openjdk.org/skara/pull/1438 From zsong at openjdk.org Wed Dec 7 00:55:59 2022 From: zsong at openjdk.org (Zhao Song) Date: Wed, 7 Dec 2022 00:55:59 GMT Subject: RFR: 1710: CommitCommandWorkItem always fetches Commit In-Reply-To: References: Message-ID: <6bZI-oKqhobPs5xI1_2T0HdmRku5-Rd4DX3AoNGC9no=.67595261-8cb4-434b-8bb1-354af869b93c@github.com> On Tue, 6 Dec 2022 19:05:34 GMT, Erik Joelsson wrote: > During bot restart, quite a few CommitCommandWorkItems are spawned. Most of them will discover that nothing needs to be done. One of the first things done in the run method is fetching the commit from the remote repository. The Commit instance will always be created with the full set of diffs for the commit. In certain cases, where commits are very large, this can take considerable time. Specifically in the jfx repos, there are examples where the REST request is split into 147 pages. (e.g. https://github.com/openjdk/jfx11u/commit/4bd6877dbd46e37f6774c50dcdc98a8a1ab41214) The way these WorkItems are spawned on startup, most of these gigantic commits will be included for a long time. The Commit instance isn't needed to figure out if a new command is present, just the comments are. This means we can move this query to after we have established that a command is to be executed. > > To further reduce overhead when fetching commits from remote repositories, I'm also making the inclusion of the "parentDiffs" optional. Of all the uses of `HostedRepository::commit`, there is currently only one that actually needs the diffs (BackportCommand). This will require the caller to be mindful of the state of the returned `Commit` object, especially when sending it other consumers, but I believe it's worth it to reduce unnecessary load on the system. > > This also affects `Forge::search`, where there is one other use where the diffs are used (when evaluating if a backport is clean). LGTM! ------------- Marked as reviewed by zsong (Committer). PR: https://git.openjdk.org/skara/pull/1436 From erikj at openjdk.org Wed Dec 7 15:22:29 2022 From: erikj at openjdk.org (Erik Joelsson) Date: Wed, 7 Dec 2022 15:22:29 GMT Subject: RFR: 1716: Revert SKARA-1655 In-Reply-To: References: Message-ID: On Wed, 7 Dec 2022 00:04:41 GMT, Zhao Song wrote: > In [SKARA-1655](https://bugs.openjdk.org/browse/SKARA-1655), I implemented a commit check named jCheckConfCheck. It was supposed to check if .jcheck/conf is valid in user's commit. But now, we have a better idea([SKARA-1691](https://bugs.openjdk.org/browse/SKARA-1691)) to check and we no longer need this check any more. Also, this check is not using by any repo right now. It doesn't hurt to revert it. Marked as reviewed by erikj (Lead). ------------- PR: https://git.openjdk.org/skara/pull/1438 From zsong at openjdk.org Wed Dec 7 17:02:49 2022 From: zsong at openjdk.org (Zhao Song) Date: Wed, 7 Dec 2022 17:02:49 GMT Subject: Integrated: 1716: Revert SKARA-1655 In-Reply-To: References: Message-ID: On Wed, 7 Dec 2022 00:04:41 GMT, Zhao Song wrote: > In [SKARA-1655](https://bugs.openjdk.org/browse/SKARA-1655), I implemented a commit check named jCheckConfCheck. It was supposed to check if .jcheck/conf is valid in user's commit. But now, we have a better idea([SKARA-1691](https://bugs.openjdk.org/browse/SKARA-1691)) to check and we no longer need this check any more. Also, this check is not using by any repo right now. It doesn't hurt to revert it. This pull request has now been integrated. Changeset: b0723850 Author: Zhao Song URL: https://git.openjdk.org/skara/commit/b072385020718181d9ef04811f9a937d5b894d8a Stats: 365 lines in 9 files changed: 0 ins; 364 del; 1 mod 1716: Revert SKARA-1655 Reviewed-by: erikj ------------- PR: https://git.openjdk.org/skara/pull/1438 From zsong at openjdk.org Wed Dec 7 19:49:04 2022 From: zsong at openjdk.org (Zhao Song) Date: Wed, 7 Dec 2022 19:49:04 GMT Subject: RFR: 1691: Run Jcheck twice in CheckRun if .jcheck/conf has changed Message-ID: When a pr is created, pr bot will run jcheck to check whether this commit is good. However, currently, we only run jcheck with the configuration from the target branch. Sometimes, user wants to modify jcheck configuration and accidentally messes up the format. In this case, pr bot would not be able to find this problem. After this commit with badly formatted jcheck configuration integrated, all the jcheck will fail and our bot gets stuck into a retry loop. In this patch, if jcheck configuration file is updated, the bot will run jcheck twice, once with the configuration in the target and once with the new one in the source. So if the new configuration has any problem, it will became an integration blocker. ------------- Commit messages: - SKARA-1691 - Merge branch 'master' into SKARA-1691 - continue working after merge master Changes: https://git.openjdk.org/skara/pull/1439/files Webrev: https://webrevs.openjdk.org/?repo=skara&pr=1439&range=00 Issue: https://bugs.openjdk.org/browse/SKARA-1691 Stats: 185 lines in 9 files changed: 181 ins; 0 del; 4 mod Patch: https://git.openjdk.org/skara/pull/1439.diff Fetch: git fetch https://git.openjdk.org/skara pull/1439/head:pull/1439 PR: https://git.openjdk.org/skara/pull/1439 From zsong at openjdk.org Wed Dec 7 21:38:05 2022 From: zsong at openjdk.org (Zhao Song) Date: Wed, 7 Dec 2022 21:38:05 GMT Subject: RFR: 1709: Only fetch comments once in PullRequestWorkItem in pr bot In-Reply-To: References: Message-ID: On Tue, 6 Dec 2022 23:57:39 GMT, Erik Joelsson wrote: > This patch provides a minor optimization for `PullRequestWorkItem` subclasses in the pr bot by avoiding re-fetching of pr.comments(). I initially started this patch with a much bigger optimization in mind, but it turned out to not work. I still think it worth applying the remaining patch since I've gone through the trouble of implementing it. LGTM ------------- Marked as reviewed by zsong (Committer). PR: https://git.openjdk.org/skara/pull/1437 From erikj at openjdk.org Wed Dec 7 21:43:41 2022 From: erikj at openjdk.org (Erik Joelsson) Date: Wed, 7 Dec 2022 21:43:41 GMT Subject: RFR: 1691: Run Jcheck twice in CheckRun if .jcheck/conf has changed In-Reply-To: References: Message-ID: On Wed, 7 Dec 2022 19:44:52 GMT, Zhao Song wrote: > When a pr is created, pr bot will run jcheck to check whether this commit is good. However, currently, we only run jcheck with the configuration from the target branch. Sometimes, user wants to modify jcheck configuration and accidentally messes up the format. In this case, pr bot would not be able to find this problem. After this commit with badly formatted jcheck configuration integrated, all the jcheck will fail and our bot gets stuck into a retry loop. > > In this patch, if jcheck configuration file is updated, the bot will run jcheck twice, once with the configuration in the target and once with the new one in the source. So if the new configuration has any problem, it will became an integration blocker. bots/pr/src/main/java/org/openjdk/skara/bots/pr/CheckRun.java line 1083: > 1081: var additionalConfiguration = AdditionalConfiguration.get(localRepo, localHash, pr.repository().forge().currentUser(), comments); > 1082: checkablePullRequest.executeChecks(localHash, censusInstance, visitor, additionalConfiguration, true); > 1083: if (localRepo.isFileUpdated(".jcheck/conf")) { We probably don't need to run the second round if confOverride is set. bots/pr/src/main/java/org/openjdk/skara/bots/pr/CheckRun.java line 1094: > 1092: .toList()); > 1093: } catch (Exception e) { > 1094: secondJCheckMessage.add(e.getMessage() + "(exception thrown when running jcheck with updated jcheck configuration)"); Can we log this exception? bots/pr/src/main/java/org/openjdk/skara/bots/pr/CheckablePullRequest.java line 209: > 207: > 208: void executeChecks(Hash localHash, CensusInstance censusInstance, PullRequestCheckIssueVisitor visitor, > 209: List additionalConfiguration, boolean useTargetConf) throws IOException { Instead of a boolean `useTargetConf`, I think it's better if we just supply the hash we want to use (target or src, if confOverride is set, then this method will handle it). bots/pr/src/test/java/org/openjdk/skara/bots/pr/CheckTests.java line 2729: > 2727: > 2728: // pr body should have the integrationBlocker for whitespace and reviewer check > 2729: assertTrue(pr.store().body().contains("Whitespace errors(failed with the updated jcheck configuration)")); Please add a space before the `(` for all of these. bots/pr/src/test/java/org/openjdk/skara/bots/pr/CheckTests.java line 2786: > 2784: try (var output = new FileWriter(checkConf.toFile(), true)) { > 2785: output.append("\nRandomCharacters"); > 2786: } This could be done with Files.writeString(checkConf, "text...", StandardOpenOption.APPEND) vcs/src/main/java/org/openjdk/skara/vcs/git/GitRepository.java line 1725: > 1723: > 1724: @Override > 1725: public boolean isFileUpdated(String filename) throws IOException { This method seems a bit too specialized to me. It should at least take a Hash argument. I think it would be even better to use the existing `commits` method from which you can get to the Diff and Patch objects. That way we have properly parsed and typed data to find the filename in. ------------- PR: https://git.openjdk.org/skara/pull/1439 From zsong at openjdk.org Wed Dec 7 22:22:01 2022 From: zsong at openjdk.org (Zhao Song) Date: Wed, 7 Dec 2022 22:22:01 GMT Subject: RFR: 1691: Run Jcheck twice in CheckRun if .jcheck/conf has changed In-Reply-To: References: Message-ID: On Wed, 7 Dec 2022 21:18:17 GMT, Erik Joelsson wrote: >> When a pr is created, pr bot will run jcheck to check whether this commit is good. However, currently, we only run jcheck with the configuration from the target branch. Sometimes, user wants to modify jcheck configuration and accidentally messes up the format. In this case, pr bot would not be able to find this problem. After this commit with badly formatted jcheck configuration integrated, all the jcheck will fail and our bot gets stuck into a retry loop. >> >> In this patch, if jcheck configuration file is updated, the bot will run jcheck twice, once with the configuration in the target and once with the new one in the source. So if the new configuration has any problem, it will became an integration blocker. > > bots/pr/src/main/java/org/openjdk/skara/bots/pr/CheckRun.java line 1083: > >> 1081: var additionalConfiguration = AdditionalConfiguration.get(localRepo, localHash, pr.repository().forge().currentUser(), comments); >> 1082: checkablePullRequest.executeChecks(localHash, censusInstance, visitor, additionalConfiguration, true); >> 1083: if (localRepo.isFileUpdated(".jcheck/conf")) { > > We probably don't need to run the second round if confOverride is set. Good point! ------------- PR: https://git.openjdk.org/skara/pull/1439 From zsong at openjdk.org Wed Dec 7 22:33:34 2022 From: zsong at openjdk.org (Zhao Song) Date: Wed, 7 Dec 2022 22:33:34 GMT Subject: RFR: 1691: Run Jcheck twice in CheckRun if .jcheck/conf has changed In-Reply-To: References: Message-ID: On Wed, 7 Dec 2022 21:19:35 GMT, Erik Joelsson wrote: >> When a pr is created, pr bot will run jcheck to check whether this commit is good. However, currently, we only run jcheck with the configuration from the target branch. Sometimes, user wants to modify jcheck configuration and accidentally messes up the format. In this case, pr bot would not be able to find this problem. After this commit with badly formatted jcheck configuration integrated, all the jcheck will fail and our bot gets stuck into a retry loop. >> >> In this patch, if jcheck configuration file is updated, the bot will run jcheck twice, once with the configuration in the target and once with the new one in the source. So if the new configuration has any problem, it will became an integration blocker. > > bots/pr/src/main/java/org/openjdk/skara/bots/pr/CheckablePullRequest.java line 209: > >> 207: >> 208: void executeChecks(Hash localHash, CensusInstance censusInstance, PullRequestCheckIssueVisitor visitor, >> 209: List additionalConfiguration, boolean useTargetConf) throws IOException { > > Instead of a boolean `useTargetConf`, I think it's better if we just supply the hash we want to use (target or src, if confOverride is set, then this method will handle it). Yeah, pass hash is better. But even if we pass hash here, it won't affect the logic of using confOverride because confOverride is set in the constructor of `CheckablePullRequest` ------------- PR: https://git.openjdk.org/skara/pull/1439 From zsong at openjdk.org Wed Dec 7 22:41:01 2022 From: zsong at openjdk.org (Zhao Song) Date: Wed, 7 Dec 2022 22:41:01 GMT Subject: RFR: 1691: Run Jcheck twice in CheckRun if .jcheck/conf has changed In-Reply-To: References: Message-ID: On Wed, 7 Dec 2022 21:38:52 GMT, Erik Joelsson wrote: >> When a pr is created, pr bot will run jcheck to check whether this commit is good. However, currently, we only run jcheck with the configuration from the target branch. Sometimes, user wants to modify jcheck configuration and accidentally messes up the format. In this case, pr bot would not be able to find this problem. After this commit with badly formatted jcheck configuration integrated, all the jcheck will fail and our bot gets stuck into a retry loop. >> >> In this patch, if jcheck configuration file is updated, the bot will run jcheck twice, once with the configuration in the target and once with the new one in the source. So if the new configuration has any problem, it will became an integration blocker. > > bots/pr/src/test/java/org/openjdk/skara/bots/pr/CheckTests.java line 2786: > >> 2784: try (var output = new FileWriter(checkConf.toFile(), true)) { >> 2785: output.append("\nRandomCharacters"); >> 2786: } > > This could be done with > > Files.writeString(checkConf, "text...", StandardOpenOption.APPEND) It's better! Thx ------------- PR: https://git.openjdk.org/skara/pull/1439 From zsong at openjdk.org Wed Dec 7 23:28:32 2022 From: zsong at openjdk.org (Zhao Song) Date: Wed, 7 Dec 2022 23:28:32 GMT Subject: RFR: 1691: Run Jcheck twice in CheckRun if .jcheck/conf has changed In-Reply-To: References: Message-ID: On Wed, 7 Dec 2022 22:30:04 GMT, Zhao Song wrote: >> bots/pr/src/main/java/org/openjdk/skara/bots/pr/CheckablePullRequest.java line 209: >> >>> 207: >>> 208: void executeChecks(Hash localHash, CensusInstance censusInstance, PullRequestCheckIssueVisitor visitor, >>> 209: List additionalConfiguration, boolean useTargetConf) throws IOException { >> >> Instead of a boolean `useTargetConf`, I think it's better if we just supply the hash we want to use (target or src, if confOverride is set, then this method will handle it). > > Yeah, pass hash is better. But even if we pass hash here, it won't affect the logic of using confOverride because confOverride is set in the constructor of `CheckablePullRequest` Now, I prefer not to change this method because I think ` CheckablePullRequest#targetHash` can help us reuse the target hash. If we add hash as an argument, we will need to call `PullRequestUtils.targetHash(localRepo)` more than once. ------------- PR: https://git.openjdk.org/skara/pull/1439 From zsong at openjdk.org Wed Dec 7 23:33:05 2022 From: zsong at openjdk.org (Zhao Song) Date: Wed, 7 Dec 2022 23:33:05 GMT Subject: RFR: 1691: Run Jcheck twice in CheckRun if .jcheck/conf has changed [v2] In-Reply-To: References: Message-ID: > When a pr is created, pr bot will run jcheck to check whether this commit is good. However, currently, we only run jcheck with the configuration from the target branch. Sometimes, user wants to modify jcheck configuration and accidentally messes up the format. In this case, pr bot would not be able to find this problem. After this commit with badly formatted jcheck configuration integrated, all the jcheck will fail and our bot gets stuck into a retry loop. > > In this patch, if jcheck configuration file is updated, the bot will run jcheck twice, once with the configuration in the target and once with the new one in the source. So if the new configuration has any problem, it will became an integration blocker. Zhao Song has updated the pull request incrementally with one additional commit since the last revision: fix some problems ------------- Changes: - all: https://git.openjdk.org/skara/pull/1439/files - new: https://git.openjdk.org/skara/pull/1439/files/a3018141..62b3ecd2 Webrevs: - full: https://webrevs.openjdk.org/?repo=skara&pr=1439&range=01 - incr: https://webrevs.openjdk.org/?repo=skara&pr=1439&range=00-01 Stats: 44 lines in 6 files changed: 3 ins; 14 del; 27 mod Patch: https://git.openjdk.org/skara/pull/1439.diff Fetch: git fetch https://git.openjdk.org/skara pull/1439/head:pull/1439 PR: https://git.openjdk.org/skara/pull/1439 From zsong at openjdk.org Wed Dec 7 23:33:05 2022 From: zsong at openjdk.org (Zhao Song) Date: Wed, 7 Dec 2022 23:33:05 GMT Subject: RFR: 1691: Run Jcheck twice in CheckRun if .jcheck/conf has changed [v2] In-Reply-To: References: Message-ID: On Wed, 7 Dec 2022 23:29:31 GMT, Zhao Song wrote: >> When a pr is created, pr bot will run jcheck to check whether this commit is good. However, currently, we only run jcheck with the configuration from the target branch. Sometimes, user wants to modify jcheck configuration and accidentally messes up the format. In this case, pr bot would not be able to find this problem. After this commit with badly formatted jcheck configuration integrated, all the jcheck will fail and our bot gets stuck into a retry loop. >> >> In this patch, if jcheck configuration file is updated, the bot will run jcheck twice, once with the configuration in the target and once with the new one in the source. So if the new configuration has any problem, it will became an integration blocker. > > Zhao Song has updated the pull request incrementally with one additional commit since the last revision: > > fix some problems vcs/src/main/java/org/openjdk/skara/vcs/git/GitRepository.java line 1731: > 1729: .anyMatch(diff -> diff.patches().stream() > 1730: .anyMatch(patch -> patch.toString().contains(filename)))); > 1731: } @erikj79 I am not sure I get you correctly. Correct me if I misunderstand what you mean. ------------- PR: https://git.openjdk.org/skara/pull/1439 From erikj at openjdk.org Thu Dec 8 15:30:52 2022 From: erikj at openjdk.org (Erik Joelsson) Date: Thu, 8 Dec 2022 15:30:52 GMT Subject: RFR: 1691: Run Jcheck twice in CheckRun if .jcheck/conf has changed [v2] In-Reply-To: References: Message-ID: On Wed, 7 Dec 2022 23:29:30 GMT, Zhao Song wrote: >> Zhao Song has updated the pull request incrementally with one additional commit since the last revision: >> >> fix some problems > > vcs/src/main/java/org/openjdk/skara/vcs/git/GitRepository.java line 1731: > >> 1729: .anyMatch(diff -> diff.patches().stream() >> 1730: .anyMatch(patch -> patch.toString().contains(filename)))); >> 1731: } > > @erikj79 I am not sure I get you correctly. Correct me if I misunderstand what you mean. I wasn't very clear. I don't think we need this method on GitRepository. I figured we could just inline this at the single call site in CheckRun, but since the expression is quite long, then probably a private method in CheckRun would be best. You can use `commits(String range)` to just get the single commit we are interested in. ------------- PR: https://git.openjdk.org/skara/pull/1439 From erikj at openjdk.org Thu Dec 8 15:34:21 2022 From: erikj at openjdk.org (Erik Joelsson) Date: Thu, 8 Dec 2022 15:34:21 GMT Subject: RFR: 1691: Run Jcheck twice in CheckRun if .jcheck/conf has changed [v2] In-Reply-To: References: Message-ID: On Wed, 7 Dec 2022 23:25:22 GMT, Zhao Song wrote: >> Yeah, pass hash is better. But even if we pass hash here, it won't affect the logic of using confOverride because confOverride is set in the constructor of `CheckablePullRequest` > > Now, I prefer not to change this method because I think ` CheckablePullRequest#targetHash` can help us reuse the target hash. If we add hash as an argument, we will need to call `PullRequestUtils.targetHash(localRepo)` more than once. `CheckablePullRequest#targetHash` is public so you can call it from `CheckRun`. ------------- PR: https://git.openjdk.org/skara/pull/1439 From zsong at openjdk.org Thu Dec 8 17:47:47 2022 From: zsong at openjdk.org (Zhao Song) Date: Thu, 8 Dec 2022 17:47:47 GMT Subject: RFR: 1691: Run Jcheck twice in CheckRun if .jcheck/conf has changed [v2] In-Reply-To: References: Message-ID: On Thu, 8 Dec 2022 15:30:51 GMT, Erik Joelsson wrote: >> Now, I prefer not to change this method because I think ` CheckablePullRequest#targetHash` can help us reuse the target hash. If we add hash as an argument, we will need to call `PullRequestUtils.targetHash(localRepo)` more than once. > > `CheckablePullRequest#targetHash` is public so you can call it from `CheckRun`. Oh, then I will change the method. Thanks. ------------- PR: https://git.openjdk.org/skara/pull/1439 From zsong at openjdk.org Thu Dec 8 18:13:39 2022 From: zsong at openjdk.org (Zhao Song) Date: Thu, 8 Dec 2022 18:13:39 GMT Subject: RFR: 1691: Run Jcheck twice in CheckRun if .jcheck/conf has changed [v3] In-Reply-To: References: Message-ID: <5LlHHcc11mwV8K0DyyYLYem28cf0Tg-43u28CedBvWI=.a2496622-27de-4680-8861-b88e7df731ee@github.com> > When a pr is created, pr bot will run jcheck to check whether this commit is good. However, currently, we only run jcheck with the configuration from the target branch. Sometimes, user wants to modify jcheck configuration and accidentally messes up the format. In this case, pr bot would not be able to find this problem. After this commit with badly formatted jcheck configuration integrated, all the jcheck will fail and our bot gets stuck into a retry loop. > > In this patch, if jcheck configuration file is updated, the bot will run jcheck twice, once with the configuration in the target and once with the new one in the source. So if the new configuration has any problem, it will became an integration blocker. Zhao Song has updated the pull request incrementally with one additional commit since the last revision: fix problems ------------- Changes: - all: https://git.openjdk.org/skara/pull/1439/files - new: https://git.openjdk.org/skara/pull/1439/files/62b3ecd2..396dba51 Webrevs: - full: https://webrevs.openjdk.org/?repo=skara&pr=1439&range=02 - incr: https://webrevs.openjdk.org/?repo=skara&pr=1439&range=01-02 Stats: 37 lines in 7 files changed: 7 ins; 24 del; 6 mod Patch: https://git.openjdk.org/skara/pull/1439.diff Fetch: git fetch https://git.openjdk.org/skara pull/1439/head:pull/1439 PR: https://git.openjdk.org/skara/pull/1439 From ihse at openjdk.org Thu Dec 8 18:36:35 2022 From: ihse at openjdk.org (Magnus Ihse Bursie) Date: Thu, 8 Dec 2022 18:36:35 GMT Subject: RFR: 1691: Run Jcheck twice in CheckRun if .jcheck/conf has changed [v3] In-Reply-To: <5LlHHcc11mwV8K0DyyYLYem28cf0Tg-43u28CedBvWI=.a2496622-27de-4680-8861-b88e7df731ee@github.com> References: <5LlHHcc11mwV8K0DyyYLYem28cf0Tg-43u28CedBvWI=.a2496622-27de-4680-8861-b88e7df731ee@github.com> Message-ID: <3ruU4R_gJDrVI6vnnJz1Q89mvGauaCrffN1PlFrkKBc=.b454c144-2f9c-408f-b2e8-f3a877143544@github.com> On Thu, 8 Dec 2022 18:13:39 GMT, Zhao Song wrote: >> When a pr is created, pr bot will run jcheck to check whether this commit is good. However, currently, we only run jcheck with the configuration from the target branch. Sometimes, user wants to modify jcheck configuration and accidentally messes up the format. In this case, pr bot would not be able to find this problem. After this commit with badly formatted jcheck configuration integrated, all the jcheck will fail and our bot gets stuck into a retry loop. >> >> In this patch, if jcheck configuration file is updated, the bot will run jcheck twice, once with the configuration in the target and once with the new one in the source. So if the new configuration has any problem, it will became an integration blocker. > > Zhao Song has updated the pull request incrementally with one additional commit since the last revision: > > fix problems Are we sure we are not painting ourselves in a corner were a change in .jcheck/conf is impossible to do since it will fail either before or after applying? I can't really think of such example, but maybe I'm just not trying hard enough. ------------- PR: https://git.openjdk.org/skara/pull/1439 From zsong at openjdk.org Thu Dec 8 18:57:32 2022 From: zsong at openjdk.org (Zhao Song) Date: Thu, 8 Dec 2022 18:57:32 GMT Subject: RFR: 1691: Run Jcheck twice in CheckRun if .jcheck/conf has changed [v3] In-Reply-To: <3ruU4R_gJDrVI6vnnJz1Q89mvGauaCrffN1PlFrkKBc=.b454c144-2f9c-408f-b2e8-f3a877143544@github.com> References: <5LlHHcc11mwV8K0DyyYLYem28cf0Tg-43u28CedBvWI=.a2496622-27de-4680-8861-b88e7df731ee@github.com> <3ruU4R_gJDrVI6vnnJz1Q89mvGauaCrffN1PlFrkKBc=.b454c144-2f9c-408f-b2e8-f3a877143544@github.com> Message-ID: <2zMaAVdjLVZ40kzilNp4llYZyhfVIVyLO-pvIz250JI=.49942690-161b-4fd9-9570-d76f04b3ad21@github.com> On Thu, 8 Dec 2022 18:34:19 GMT, Magnus Ihse Bursie wrote: > Are we sure we are not painting ourselves in a corner were a change in .jcheck/conf is impossible to do since it will fail either before or after applying? I can't really think of such example, but maybe I'm just not trying hard enough. I am not sure but I couldn't think of such example too. ------------- PR: https://git.openjdk.org/skara/pull/1439 From erikj at openjdk.org Thu Dec 8 18:57:32 2022 From: erikj at openjdk.org (Erik Joelsson) Date: Thu, 8 Dec 2022 18:57:32 GMT Subject: RFR: 1691: Run Jcheck twice in CheckRun if .jcheck/conf has changed [v3] In-Reply-To: <2zMaAVdjLVZ40kzilNp4llYZyhfVIVyLO-pvIz250JI=.49942690-161b-4fd9-9570-d76f04b3ad21@github.com> References: <5LlHHcc11mwV8K0DyyYLYem28cf0Tg-43u28CedBvWI=.a2496622-27de-4680-8861-b88e7df731ee@github.com> <3ruU4R_gJDrVI6vnnJz1Q89mvGauaCrffN1PlFrkKBc=.b454c144-2f9c-408f-b2e8-f3a877143544@github.com> <2zMaAVdjLVZ40kzilNp4llYZyhfVIVyLO-pvIz250JI=.49942690-161b-4fd9-9570-d76f04b3ad21@github.com> Message-ID: On Thu, 8 Dec 2022 18:53:54 GMT, Zhao Song wrote: > Are we sure we are not painting ourselves in a corner were a change in .jcheck/conf is impossible to do since it will fail either before or after applying? I can't really think of such example, but maybe I'm just not trying hard enough. It's possible, but in that case the contributor will have to split the change into two separate changes. First relax existing requirements and then add the new requirement. ------------- PR: https://git.openjdk.org/skara/pull/1439 From zsong at openjdk.org Thu Dec 8 19:38:13 2022 From: zsong at openjdk.org (Zhao Song) Date: Thu, 8 Dec 2022 19:38:13 GMT Subject: RFR: 1691: Run Jcheck twice in CheckRun if .jcheck/conf has changed [v4] In-Reply-To: References: Message-ID: > When a pr is created, pr bot will run jcheck to check whether this commit is good. However, currently, we only run jcheck with the configuration from the target branch. Sometimes, user wants to modify jcheck configuration and accidentally messes up the format. In this case, pr bot would not be able to find this problem. After this commit with badly formatted jcheck configuration integrated, all the jcheck will fail and our bot gets stuck into a retry loop. > > In this patch, if jcheck configuration file is updated, the bot will run jcheck twice, once with the configuration in the target and once with the new one in the source. So if the new configuration has any problem, it will became an integration blocker. Zhao Song has updated the pull request incrementally with one additional commit since the last revision: fix a problem ------------- Changes: - all: https://git.openjdk.org/skara/pull/1439/files - new: https://git.openjdk.org/skara/pull/1439/files/396dba51..4737447d Webrevs: - full: https://webrevs.openjdk.org/?repo=skara&pr=1439&range=03 - incr: https://webrevs.openjdk.org/?repo=skara&pr=1439&range=02-03 Stats: 1 line in 1 file changed: 0 ins; 0 del; 1 mod Patch: https://git.openjdk.org/skara/pull/1439.diff Fetch: git fetch https://git.openjdk.org/skara pull/1439/head:pull/1439 PR: https://git.openjdk.org/skara/pull/1439 From zsong at openjdk.org Thu Dec 8 21:00:40 2022 From: zsong at openjdk.org (Zhao Song) Date: Thu, 8 Dec 2022 21:00:40 GMT Subject: RFR: 1691: Run Jcheck twice in CheckRun if .jcheck/conf has changed [v5] In-Reply-To: References: Message-ID: > When a pr is created, pr bot will run jcheck to check whether this commit is good. However, currently, we only run jcheck with the configuration from the target branch. Sometimes, user wants to modify jcheck configuration and accidentally messes up the format. In this case, pr bot would not be able to find this problem. After this commit with badly formatted jcheck configuration integrated, all the jcheck will fail and our bot gets stuck into a retry loop. > > In this patch, if jcheck configuration file is updated, the bot will run jcheck twice, once with the configuration in the target and once with the new one in the source. So if the new configuration has any problem, it will became an integration blocker. Zhao Song has updated the pull request incrementally with one additional commit since the last revision: fix problems ------------- Changes: - all: https://git.openjdk.org/skara/pull/1439/files - new: https://git.openjdk.org/skara/pull/1439/files/4737447d..dfe06f6c Webrevs: - full: https://webrevs.openjdk.org/?repo=skara&pr=1439&range=04 - incr: https://webrevs.openjdk.org/?repo=skara&pr=1439&range=03-04 Stats: 4 lines in 2 files changed: 1 ins; 1 del; 2 mod Patch: https://git.openjdk.org/skara/pull/1439.diff Fetch: git fetch https://git.openjdk.org/skara pull/1439/head:pull/1439 PR: https://git.openjdk.org/skara/pull/1439 From erikj at openjdk.org Thu Dec 8 21:00:41 2022 From: erikj at openjdk.org (Erik Joelsson) Date: Thu, 8 Dec 2022 21:00:41 GMT Subject: RFR: 1691: Run Jcheck twice in CheckRun if .jcheck/conf has changed [v3] In-Reply-To: <5LlHHcc11mwV8K0DyyYLYem28cf0Tg-43u28CedBvWI=.a2496622-27de-4680-8861-b88e7df731ee@github.com> References: <5LlHHcc11mwV8K0DyyYLYem28cf0Tg-43u28CedBvWI=.a2496622-27de-4680-8861-b88e7df731ee@github.com> Message-ID: On Thu, 8 Dec 2022 18:13:39 GMT, Zhao Song wrote: >> When a pr is created, pr bot will run jcheck to check whether this commit is good. However, currently, we only run jcheck with the configuration from the target branch. Sometimes, user wants to modify jcheck configuration and accidentally messes up the format. In this case, pr bot would not be able to find this problem. After this commit with badly formatted jcheck configuration integrated, all the jcheck will fail and our bot gets stuck into a retry loop. >> >> In this patch, if jcheck configuration file is updated, the bot will run jcheck twice, once with the configuration in the target and once with the new one in the source. So if the new configuration has any problem, it will became an integration blocker. > > Zhao Song has updated the pull request incrementally with one additional commit since the last revision: > > fix problems bots/pr/src/main/java/org/openjdk/skara/bots/pr/CheckRun.java line 1202: > 1200: > 1201: private boolean isFileUpdated(String filename, Hash hash) throws IOException { > 1202: return localRepo.commits(hash.hex()).asList().stream() Suggestion: return localRepo.commits(hash.hex()).stream() bots/pr/src/main/java/org/openjdk/skara/bots/pr/CheckRun.java line 1205: > 1203: .anyMatch(commit -> commit.parentDiffs().stream() > 1204: .anyMatch(diff -> diff.patches().stream() > 1205: .anyMatch(patch -> patch.toString().contains(filename)))); Suggestion: .anyMatch(patch -> patch.source().path().toString().equals(filename) || patch.target().path().toString().equals(filename)))); forge/src/main/java/org/openjdk/skara/forge/PullRequestUtils.java line 168: > 166: var localRepo = hostedRepositoryPool.checkout(pr.repository(), pr.headHash().hex(), path); > 167: localRepo.fetch(pr.repository().url(), "+" + pr.targetRef() + ":prutils_targetref", false); > 168: localRepo.fetch(pr.repository().url(), "+" + pr.sourceRef() + ":prutils_sourceref", false); I'm curious as to why this is needed. Line 166 should make sure we are fetching the source/head of the pr. I don't see any use of the local branch `prutils_sourceref`. ------------- PR: https://git.openjdk.org/skara/pull/1439 From zsong at openjdk.org Thu Dec 8 21:00:41 2022 From: zsong at openjdk.org (Zhao Song) Date: Thu, 8 Dec 2022 21:00:41 GMT Subject: RFR: 1691: Run Jcheck twice in CheckRun if .jcheck/conf has changed [v3] In-Reply-To: References: <5LlHHcc11mwV8K0DyyYLYem28cf0Tg-43u28CedBvWI=.a2496622-27de-4680-8861-b88e7df731ee@github.com> Message-ID: <0SKGlDK9h6cr_BCED6LuYBqguoG8LLd8Eu6vcraDWjw=.e6f388eb-b46a-45f8-ae95-0ce0c6b174f3@github.com> On Thu, 8 Dec 2022 19:36:53 GMT, Erik Joelsson wrote: >> Zhao Song has updated the pull request incrementally with one additional commit since the last revision: >> >> fix problems > > forge/src/main/java/org/openjdk/skara/forge/PullRequestUtils.java line 168: > >> 166: var localRepo = hostedRepositoryPool.checkout(pr.repository(), pr.headHash().hex(), path); >> 167: localRepo.fetch(pr.repository().url(), "+" + pr.targetRef() + ":prutils_targetref", false); >> 168: localRepo.fetch(pr.repository().url(), "+" + pr.sourceRef() + ":prutils_sourceref", false); > > I'm curious as to why this is needed. Line 166 should make sure we are fetching the source/head of the pr. I don't see any use of the local branch `prutils_sourceref`. Sorry, I forget to delete it. I was thinking we need this and then I found it is useless, so I didn't use it any more. But I forget to delete it later. ------------- PR: https://git.openjdk.org/skara/pull/1439 From erikj at openjdk.org Thu Dec 8 21:54:41 2022 From: erikj at openjdk.org (Erik Joelsson) Date: Thu, 8 Dec 2022 21:54:41 GMT Subject: RFR: 1691: Run Jcheck twice in CheckRun if .jcheck/conf has changed [v5] In-Reply-To: References: Message-ID: On Thu, 8 Dec 2022 21:00:40 GMT, Zhao Song wrote: >> When a pr is created, pr bot will run jcheck to check whether this commit is good. However, currently, we only run jcheck with the configuration from the target branch. Sometimes, user wants to modify jcheck configuration and accidentally messes up the format. In this case, pr bot would not be able to find this problem. After this commit with badly formatted jcheck configuration integrated, all the jcheck will fail and our bot gets stuck into a retry loop. >> >> In this patch, if jcheck configuration file is updated, the bot will run jcheck twice, once with the configuration in the target and once with the new one in the source. So if the new configuration has any problem, it will became an integration blocker. > > Zhao Song has updated the pull request incrementally with one additional commit since the last revision: > > fix problems Marked as reviewed by erikj (Lead). ------------- PR: https://git.openjdk.org/skara/pull/1439 From zsong at openjdk.org Thu Dec 8 22:15:45 2022 From: zsong at openjdk.org (Zhao Song) Date: Thu, 8 Dec 2022 22:15:45 GMT Subject: RFR: 1691: Run Jcheck twice in CheckRun if .jcheck/conf has changed [v5] In-Reply-To: References: Message-ID: <4NLl2OR_QDPWKnjCK4S0cYF55IyMyOjD5GNSbBFLvrQ=.c1b2c73d-5ce4-4b51-94b2-a4f36ed28e4e@github.com> On Thu, 8 Dec 2022 21:00:40 GMT, Zhao Song wrote: >> When a pr is created, pr bot will run jcheck to check whether this commit is good. However, currently, we only run jcheck with the configuration from the target branch. Sometimes, user wants to modify jcheck configuration and accidentally messes up the format. In this case, pr bot would not be able to find this problem. After this commit with badly formatted jcheck configuration integrated, all the jcheck will fail and our bot gets stuck into a retry loop. >> >> In this patch, if jcheck configuration file is updated, the bot will run jcheck twice, once with the configuration in the target and once with the new one in the source. So if the new configuration has any problem, it will became an integration blocker. > > Zhao Song has updated the pull request incrementally with one additional commit since the last revision: > > fix problems Thanks for review! ------------- PR: https://git.openjdk.org/skara/pull/1439 From zsong at openjdk.org Thu Dec 8 22:15:46 2022 From: zsong at openjdk.org (Zhao Song) Date: Thu, 8 Dec 2022 22:15:46 GMT Subject: Integrated: 1691: Run Jcheck twice in CheckRun if .jcheck/conf has changed In-Reply-To: References: Message-ID: On Wed, 7 Dec 2022 19:44:52 GMT, Zhao Song wrote: > When a pr is created, pr bot will run jcheck to check whether this commit is good. However, currently, we only run jcheck with the configuration from the target branch. Sometimes, user wants to modify jcheck configuration and accidentally messes up the format. In this case, pr bot would not be able to find this problem. After this commit with badly formatted jcheck configuration integrated, all the jcheck will fail and our bot gets stuck into a retry loop. > > In this patch, if jcheck configuration file is updated, the bot will run jcheck twice, once with the configuration in the target and once with the new one in the source. So if the new configuration has any problem, it will became an integration blocker. This pull request has now been integrated. Changeset: 69da5904 Author: Zhao Song URL: https://git.openjdk.org/skara/commit/69da5904fab8a3e238a91a373df3fa47e7e0d0c2 Stats: 181 lines in 4 files changed: 161 ins; 8 del; 12 mod 1691: Run Jcheck twice in CheckRun if .jcheck/conf has changed Reviewed-by: erikj ------------- PR: https://git.openjdk.org/skara/pull/1439 From zsong at openjdk.org Fri Dec 9 01:00:44 2022 From: zsong at openjdk.org (Zhao Song) Date: Fri, 9 Dec 2022 01:00:44 GMT Subject: RFR: 1695: Set GitHub API date version header Message-ID: <0X9_uJ68j207qDw5DlEnKmAl212f23LA_nAeBXMbVDw=.b3da7e54-bf1a-4511-a847-1003138e1af5@github.com> In this patch, added new Header `X-GitHub-Api-Version` to all GitHub API calls. ------------- Commit messages: - SKARA-1695 Changes: https://git.openjdk.org/skara/pull/1440/files Webrev: https://webrevs.openjdk.org/?repo=skara&pr=1440&range=00 Issue: https://bugs.openjdk.org/browse/SKARA-1695 Stats: 2 lines in 1 file changed: 1 ins; 0 del; 1 mod Patch: https://git.openjdk.org/skara/pull/1440.diff Fetch: git fetch https://git.openjdk.org/skara pull/1440/head:pull/1440 PR: https://git.openjdk.org/skara/pull/1440 From erikj at openjdk.org Fri Dec 9 14:40:40 2022 From: erikj at openjdk.org (Erik Joelsson) Date: Fri, 9 Dec 2022 14:40:40 GMT Subject: RFR: 1695: Set GitHub API date version header In-Reply-To: <0X9_uJ68j207qDw5DlEnKmAl212f23LA_nAeBXMbVDw=.b3da7e54-bf1a-4511-a847-1003138e1af5@github.com> References: <0X9_uJ68j207qDw5DlEnKmAl212f23LA_nAeBXMbVDw=.b3da7e54-bf1a-4511-a847-1003138e1af5@github.com> Message-ID: On Fri, 9 Dec 2022 00:56:13 GMT, Zhao Song wrote: > In this patch, added new Header `X-GitHub-Api-Version` to all GitHub API calls. Marked as reviewed by erikj (Lead). ------------- PR: https://git.openjdk.org/skara/pull/1440 From zsong at openjdk.org Fri Dec 9 16:01:45 2022 From: zsong at openjdk.org (Zhao Song) Date: Fri, 9 Dec 2022 16:01:45 GMT Subject: Integrated: 1695: Set GitHub API date version header In-Reply-To: <0X9_uJ68j207qDw5DlEnKmAl212f23LA_nAeBXMbVDw=.b3da7e54-bf1a-4511-a847-1003138e1af5@github.com> References: <0X9_uJ68j207qDw5DlEnKmAl212f23LA_nAeBXMbVDw=.b3da7e54-bf1a-4511-a847-1003138e1af5@github.com> Message-ID: On Fri, 9 Dec 2022 00:56:13 GMT, Zhao Song wrote: > In this patch, added new Header `X-GitHub-Api-Version` to all GitHub API calls. This pull request has now been integrated. Changeset: edfd3e88 Author: Zhao Song URL: https://git.openjdk.org/skara/commit/edfd3e8860ac871a5f030e9cd480e57a39a2a74e Stats: 2 lines in 1 file changed: 1 ins; 0 del; 1 mod 1695: Set GitHub API date version header Reviewed-by: erikj ------------- PR: https://git.openjdk.org/skara/pull/1440 From zsong at openjdk.org Fri Dec 9 16:21:15 2022 From: zsong at openjdk.org (Zhao Song) Date: Fri, 9 Dec 2022 16:21:15 GMT Subject: RFR: 1698: Warning not to force push/rebase can be made clearer Message-ID: <-2dDpCD-z1JVpq1w_aLL37Pfuh2pCgL3GcSYLvOv2hs=.dc090c80-3fba-44ff-a82a-3d418314fef5@github.com> Rephrased the second sentence of the warning not to force push/rebase. ------------- Commit messages: - SKARA-1698 Changes: https://git.openjdk.org/skara/pull/1441/files Webrev: https://webrevs.openjdk.org/?repo=skara&pr=1441&range=00 Issue: https://bugs.openjdk.org/browse/SKARA-1698 Stats: 1 line in 1 file changed: 0 ins; 0 del; 1 mod Patch: https://git.openjdk.org/skara/pull/1441.diff Fetch: git fetch https://git.openjdk.org/skara pull/1441/head:pull/1441 PR: https://git.openjdk.org/skara/pull/1441 From darcy at openjdk.org Fri Dec 9 17:37:49 2022 From: darcy at openjdk.org (Joe Darcy) Date: Fri, 9 Dec 2022 17:37:49 GMT Subject: RFR: 1698: Warning not to force push/rebase can be made clearer In-Reply-To: <-2dDpCD-z1JVpq1w_aLL37Pfuh2pCgL3GcSYLvOv2hs=.dc090c80-3fba-44ff-a82a-3d418314fef5@github.com> References: <-2dDpCD-z1JVpq1w_aLL37Pfuh2pCgL3GcSYLvOv2hs=.dc090c80-3fba-44ff-a82a-3d418314fef5@github.com> Message-ID: On Fri, 9 Dec 2022 16:17:04 GMT, Zhao Song wrote: > Rephrased the second sentence of the warning not to force push/rebase. Looks fine. Regarding the word "always", I assume there are no situations where the squash doesn't occur? ------------- Marked as reviewed by darcy (Reviewer). PR: https://git.openjdk.org/skara/pull/1441 From zsong at openjdk.org Fri Dec 9 17:45:00 2022 From: zsong at openjdk.org (Zhao Song) Date: Fri, 9 Dec 2022 17:45:00 GMT Subject: RFR: 1698: Warning not to force push/rebase can be made clearer In-Reply-To: References: <-2dDpCD-z1JVpq1w_aLL37Pfuh2pCgL3GcSYLvOv2hs=.dc090c80-3fba-44ff-a82a-3d418314fef5@github.com> Message-ID: On Fri, 9 Dec 2022 17:34:15 GMT, Joe Darcy wrote: > Looks fine. > > Regarding the word "always", I assume there are no situations where the squash doesn't occur? AFAIK, the squash always occurs. Thanks for rephrasing the sentence! ------------- PR: https://git.openjdk.org/skara/pull/1441 From erikj at openjdk.org Fri Dec 9 18:37:47 2022 From: erikj at openjdk.org (Erik Joelsson) Date: Fri, 9 Dec 2022 18:37:47 GMT Subject: RFR: 1698: Warning not to force push/rebase can be made clearer In-Reply-To: References: <-2dDpCD-z1JVpq1w_aLL37Pfuh2pCgL3GcSYLvOv2hs=.dc090c80-3fba-44ff-a82a-3d418314fef5@github.com> Message-ID: <-pkZ_GYZEOFIuKR1yhF1qWDs5e9F-dXM0tc8tSFWIJw=.d64a47cf-a231-4bf3-8d11-dc9879d759bc@github.com> On Fri, 9 Dec 2022 17:42:12 GMT, Zhao Song wrote: > Regarding the word "always", I assume there are no situations where the squash doesn't occur? In a merge style PR, there is no squashing. Those are typically only performed by integrators. ------------- PR: https://git.openjdk.org/skara/pull/1441 From zsong at openjdk.org Fri Dec 9 22:29:55 2022 From: zsong at openjdk.org (Zhao Song) Date: Fri, 9 Dec 2022 22:29:55 GMT Subject: Integrated: 1698: Warning not to force push/rebase can be made clearer In-Reply-To: <-2dDpCD-z1JVpq1w_aLL37Pfuh2pCgL3GcSYLvOv2hs=.dc090c80-3fba-44ff-a82a-3d418314fef5@github.com> References: <-2dDpCD-z1JVpq1w_aLL37Pfuh2pCgL3GcSYLvOv2hs=.dc090c80-3fba-44ff-a82a-3d418314fef5@github.com> Message-ID: <5LucpFDJFdwT6RmxfHuFvWA2YL53rupL8XhO1_oKzs4=.47f0ad7a-83be-4f2d-b513-c23935ace116@github.com> On Fri, 9 Dec 2022 16:17:04 GMT, Zhao Song wrote: > Rephrased the second sentence of the warning not to force push/rebase. This pull request has now been integrated. Changeset: ca31af39 Author: Zhao Song URL: https://git.openjdk.org/skara/commit/ca31af3943aefe9be0202089b3df4605a0135d08 Stats: 1 line in 1 file changed: 0 ins; 0 del; 1 mod 1698: Warning not to force push/rebase can be made clearer Reviewed-by: darcy ------------- PR: https://git.openjdk.org/skara/pull/1441 From erikj at openjdk.org Mon Dec 12 23:56:56 2022 From: erikj at openjdk.org (Erik Joelsson) Date: Mon, 12 Dec 2022 23:56:56 GMT Subject: RFR: 1598: GitRepository.isHealthy takes a long time in ArchiveWorkItem Message-ID: In the mlbridge bot, the `ArchiveWorkItem` is spending an inordinate amount of time running `git fsck --connectivity-only`. This is part of a rather sane strategy of verifying a repository before using it. This is done when a repository is materialized using the `HostedRepositoryPool`, if a previous local copy is found and reused, it's first checked using `Repository::isHealthy`, which runs the fsck. In the `ArchiveWorkItem` case, the repository in question is the archive repository, where all sent emails are stored for reference (and to prevent resending duplicate messages). The repository is materialized first in every `ArchiveWorkItem::run` call as it needs to read the current archive for the PR. Especially on bot restart, this triggers a lot of extra work. This repository is growing quite fast as it contains every PR for every repository we have Skara running against on GitHub. Having every ArchiveWorkItem verifying all this contents just doesn't scale well, when each work item will only ever read and write to a single file in the repository. To eliminate this inefficiency, I'm changing `ArchiveWorkItem` to read and write the single file over REST API instead. This does introduce another level of inefficiency, where the complete file must be transferred to and from the server each time, instead of just diffs, but I think given the size of the repo, and the rate at which it is growing (not to mention the fsck checks), this is the better tradeoff. The patch adds a new method `HostedRepository::writeFileContents` (that pairs with the existing `::fileContents`). I've added manual tests that verifies these two methods together for both GitHub and GitLab. This prompted rewriting of `GitHubRepository::fileContents` to use "raw" mode for retrieving file contents. This in turn made me discover that we could remove all the preview feature `Accept` headers, as all those GitHub APIs are stable by now. (There is currently a bug in `RestRequest` causing only the last header with the same key to be used, so only the last of the list of preview headers was ever actually in effect. This bug was also preventing me from using raw mode, as that requires another `Accept` header. Given this, and that the mockingbird API is verified to be stable now, I'm confident in removing all of them. I will file a separate issue for the bug in `RestRequest`.) To use "raw" mode, I needed to use `RestRequest::executeUnparsed` and discovered that it wasn't behaving quite the same as the normal `::execute`. I changed it to throw `UncheckedRestException` like its sibling. I inspected all uses of the method and made adjustments where needed. `TestHostedRepository::fileContents` didn't return the raw contents of files. Specifically it didn't preserve trailing newlines. To fix this, I changed it to no longer convert to lines and then joining them again. ------------- Commit messages: - SKARA-1598 Changes: https://git.openjdk.org/skara/pull/1442/files Webrev: https://webrevs.openjdk.org/?repo=skara&pr=1442&range=00 Issue: https://bugs.openjdk.org/browse/SKARA-1598 Stats: 240 lines in 10 files changed: 209 ins; 11 del; 20 mod Patch: https://git.openjdk.org/skara/pull/1442.diff Fetch: git fetch https://git.openjdk.org/skara pull/1442/head:pull/1442 PR: https://git.openjdk.org/skara/pull/1442 From ihse at openjdk.org Wed Dec 14 12:13:49 2022 From: ihse at openjdk.org (Magnus Ihse Bursie) Date: Wed, 14 Dec 2022 12:13:49 GMT Subject: RFR: 1598: GitRepository.isHealthy takes a long time in ArchiveWorkItem In-Reply-To: References: Message-ID: On Mon, 12 Dec 2022 23:52:50 GMT, Erik Joelsson wrote: > In the mlbridge bot, the `ArchiveWorkItem` is spending an inordinate amount of time running `git fsck --connectivity-only`. This is part of a rather sane strategy of verifying a repository before using it. This is done when a repository is materialized using the `HostedRepositoryPool`, if a previous local copy is found and reused, it's first checked using `Repository::isHealthy`, which runs the fsck. In the `ArchiveWorkItem` case, the repository in question is the archive repository, where all sent emails are stored for reference (and to prevent resending duplicate messages). The repository is materialized first in every `ArchiveWorkItem::run` call as it needs to read the current archive for the PR. Especially on bot restart, this triggers a lot of extra work. > > This repository is growing quite fast as it contains every PR for every repository we have Skara running against on GitHub. Having every ArchiveWorkItem verifying all this contents just doesn't scale well, when each work item will only ever read and write to a single file in the repository. > > To eliminate this inefficiency, I'm changing `ArchiveWorkItem` to read and write the single file over REST API instead. This does introduce another level of inefficiency, where the complete file must be transferred to and from the server each time, instead of just diffs, but I think given the size of the repo, and the rate at which it is growing (not to mention the fsck checks), this is the better tradeoff. > > The patch adds a new method `HostedRepository::writeFileContents` (that pairs with the existing `::fileContents`). I've added manual tests that verifies these two methods together for both GitHub and GitLab. This prompted rewriting of `GitHubRepository::fileContents` to use "raw" mode for retrieving file contents. This in turn made me discover that we could remove all the preview feature `Accept` headers, as all those GitHub APIs are stable by now. (There is currently a bug in `RestRequest` causing only the last header with the same key to be used, so only the last of the list of preview headers was ever actually in effect. This bug was also preventing me from using raw mode, as that requires another `Accept` header. Given this, and that the mockingbird API is verified to be stable now, I'm confident in removing all of them. I will file a separate issue for the bug in `RestRequest`.) > > To use "raw" mode, I needed to use `RestRequest::executeUnparsed` and discovered that it wasn't behaving quite the same as the normal `::execute`. I changed it to throw `UncheckedRestException` like its sibling. I inspected all uses of the method and made adjustments where needed. > > `TestHostedRepository::fileContents` didn't return the raw contents of files. Specifically it didn't preserve trailing newlines. To fix this, I changed it to no longer convert to lines and then joining them again. bots/mlbridge/src/main/java/org/openjdk/skara/bots/mlbridge/ArchiveWorkItem.java line 252: > 250: var sentMails = new ArrayList(); > 251: var archiveContents = bot.archiveRepo().fileContents(mboxFile(), bot.archiveRef()); > 252: archiveContents.ifPresent(s -> sentMails.addAll(Mbox.splitMbox(s, bot.emailAddress()))); So this adds the old contents to sentMails, and if there is none, we just ignore adding it? I needed some time to figure out why it was safe doing nothing if archiveContents was empty. Perhaps a comment about this? forge/src/main/java/org/openjdk/skara/forge/HostedRepository.java line 100: > 98: * @param authorEmail Email of author and committer for commit > 99: */ > 100: void writeFileContents(String content, String filename, Branch branch, String message, String authorName, String authorEmail); Maybe not a big deal, but I thought the symmetry with `fileContents` was a bit lost when `String filename` was not the first argument to this method. forge/src/main/java/org/openjdk/skara/forge/github/GitHubRepository.java line 272: > 270: return Optional.of(content); > 271: } catch (UncheckedRestException e) { > 272: // The onError handler is not used with executeParsed, so have to Suggestion: // The onError handler is not used with executeUnparsed, so have to forge/src/main/java/org/openjdk/skara/forge/github/GitHubRepository.java line 292: > 290: .put("name", authorName) > 291: .put("email", authorEmail)) > 292: .put("content", new String(Base64.getEncoder().encode(content.getBytes(StandardCharsets.UTF_8)), StandardCharsets.UTF_8)); Are we talking about 10's of MB here as well? We don't risk running into limits? ------------- PR: https://git.openjdk.org/skara/pull/1442 From ihse at openjdk.org Wed Dec 14 12:20:44 2022 From: ihse at openjdk.org (Magnus Ihse Bursie) Date: Wed, 14 Dec 2022 12:20:44 GMT Subject: RFR: 1598: GitRepository.isHealthy takes a long time in ArchiveWorkItem In-Reply-To: References: Message-ID: On Mon, 12 Dec 2022 23:52:50 GMT, Erik Joelsson wrote: > In the mlbridge bot, the `ArchiveWorkItem` is spending an inordinate amount of time running `git fsck --connectivity-only`. This is part of a rather sane strategy of verifying a repository before using it. This is done when a repository is materialized using the `HostedRepositoryPool`, if a previous local copy is found and reused, it's first checked using `Repository::isHealthy`, which runs the fsck. In the `ArchiveWorkItem` case, the repository in question is the archive repository, where all sent emails are stored for reference (and to prevent resending duplicate messages). The repository is materialized first in every `ArchiveWorkItem::run` call as it needs to read the current archive for the PR. Especially on bot restart, this triggers a lot of extra work. > > This repository is growing quite fast as it contains every PR for every repository we have Skara running against on GitHub. Having every ArchiveWorkItem verifying all this contents just doesn't scale well, when each work item will only ever read and write to a single file in the repository. > > To eliminate this inefficiency, I'm changing `ArchiveWorkItem` to read and write the single file over REST API instead. This does introduce another level of inefficiency, where the complete file must be transferred to and from the server each time, instead of just diffs, but I think given the size of the repo, and the rate at which it is growing (not to mention the fsck checks), this is the better tradeoff. > > The patch adds a new method `HostedRepository::writeFileContents` (that pairs with the existing `::fileContents`). I've added manual tests that verifies these two methods together for both GitHub and GitLab. This prompted rewriting of `GitHubRepository::fileContents` to use "raw" mode for retrieving file contents. This in turn made me discover that we could remove all the preview feature `Accept` headers, as all those GitHub APIs are stable by now. (There is currently a bug in `RestRequest` causing only the last header with the same key to be used, so only the last of the list of preview headers was ever actually in effect. This bug was also preventing me from using raw mode, as that requires another `Accept` header. Given this, and that the mockingbird API is verified to be stable now, I'm confident in removing all of them. I will file a separate issue for the bug in `RestRequest`.) > > To use "raw" mode, I needed to use `RestRequest::executeUnparsed` and discovered that it wasn't behaving quite the same as the normal `::execute`. I changed it to throw `UncheckedRestException` like its sibling. I inspected all uses of the method and made adjustments where needed. > > `TestHostedRepository::fileContents` didn't return the raw contents of files. Specifically it didn't preserve trailing newlines. To fix this, I changed it to no longer convert to lines and then joining them again. Marked as reviewed by ihse (Reviewer). forge/src/main/java/org/openjdk/skara/forge/gitlab/GitLabRepository.java line 334: > 332: return Optional.of(request.post("repository/files/" + encodedFileName) > 333: .body(body) > 334: .execute()); You don't need an `onError` to return `Optional.empty()` for the POST case? Or is it the default if you don't specify an error handler? ------------- PR: https://git.openjdk.org/skara/pull/1442 From ihse at openjdk.org Wed Dec 14 12:54:48 2022 From: ihse at openjdk.org (Magnus Ihse Bursie) Date: Wed, 14 Dec 2022 12:54:48 GMT Subject: RFR: 1709: Only fetch comments once in PullRequestWorkItem in pr bot In-Reply-To: References: Message-ID: On Tue, 6 Dec 2022 23:57:39 GMT, Erik Joelsson wrote: > This patch provides a minor optimization for `PullRequestWorkItem` subclasses in the pr bot by avoiding re-fetching of pr.comments(). I initially started this patch with a much bigger optimization in mind, but it turned out to not work. I still think it worth applying the remaining patch since I've gone through the trouble of implementing it. LGTM ------------- Marked as reviewed by ihse (Reviewer). PR: https://git.openjdk.org/skara/pull/1437 From ihse at openjdk.org Wed Dec 14 13:03:37 2022 From: ihse at openjdk.org (Magnus Ihse Bursie) Date: Wed, 14 Dec 2022 13:03:37 GMT Subject: RFR: 1710: CommitCommandWorkItem always fetches Commit In-Reply-To: References: Message-ID: On Tue, 6 Dec 2022 19:05:34 GMT, Erik Joelsson wrote: > During bot restart, quite a few CommitCommandWorkItems are spawned. Most of them will discover that nothing needs to be done. One of the first things done in the run method is fetching the commit from the remote repository. The Commit instance will always be created with the full set of diffs for the commit. In certain cases, where commits are very large, this can take considerable time. Specifically in the jfx repos, there are examples where the REST request is split into 147 pages. (e.g. https://github.com/openjdk/jfx11u/commit/4bd6877dbd46e37f6774c50dcdc98a8a1ab41214) The way these WorkItems are spawned on startup, most of these gigantic commits will be included for a long time. The Commit instance isn't needed to figure out if a new command is present, just the comments are. This means we can move this query to after we have established that a command is to be executed. > > To further reduce overhead when fetching commits from remote repositories, I'm also making the inclusion of the "parentDiffs" optional. Of all the uses of `HostedRepository::commit`, there is currently only one that actually needs the diffs (BackportCommand). This will require the caller to be mindful of the state of the returned `Commit` object, especially when sending it other consumers, but I believe it's worth it to reduce unnecessary load on the system. > > This also affects `Forge::search`, where there is one other use where the diffs are used (when evaluating if a backport is clean). LGTM ------------- Marked as reviewed by ihse (Reviewer). PR: https://git.openjdk.org/skara/pull/1436 From erikj at openjdk.org Wed Dec 14 14:26:38 2022 From: erikj at openjdk.org (Erik Joelsson) Date: Wed, 14 Dec 2022 14:26:38 GMT Subject: RFR: 1598: GitRepository.isHealthy takes a long time in ArchiveWorkItem [v2] In-Reply-To: References: Message-ID: > In the mlbridge bot, the `ArchiveWorkItem` is spending an inordinate amount of time running `git fsck --connectivity-only`. This is part of a rather sane strategy of verifying a repository before using it. This is done when a repository is materialized using the `HostedRepositoryPool`, if a previous local copy is found and reused, it's first checked using `Repository::isHealthy`, which runs the fsck. In the `ArchiveWorkItem` case, the repository in question is the archive repository, where all sent emails are stored for reference (and to prevent resending duplicate messages). The repository is materialized first in every `ArchiveWorkItem::run` call as it needs to read the current archive for the PR. Especially on bot restart, this triggers a lot of extra work. > > This repository is growing quite fast as it contains every PR for every repository we have Skara running against on GitHub. Having every ArchiveWorkItem verifying all this contents just doesn't scale well, when each work item will only ever read and write to a single file in the repository. > > To eliminate this inefficiency, I'm changing `ArchiveWorkItem` to read and write the single file over REST API instead. This does introduce another level of inefficiency, where the complete file must be transferred to and from the server each time, instead of just diffs, but I think given the size of the repo, and the rate at which it is growing (not to mention the fsck checks), this is the better tradeoff. > > The patch adds a new method `HostedRepository::writeFileContents` (that pairs with the existing `::fileContents`). I've added manual tests that verifies these two methods together for both GitHub and GitLab. This prompted rewriting of `GitHubRepository::fileContents` to use "raw" mode for retrieving file contents. This in turn made me discover that we could remove all the preview feature `Accept` headers, as all those GitHub APIs are stable by now. (There is currently a bug in `RestRequest` causing only the last header with the same key to be used, so only the last of the list of preview headers was ever actually in effect. This bug was also preventing me from using raw mode, as that requires another `Accept` header. Given this, and that the mockingbird API is verified to be stable now, I'm confident in removing all of them. I will file a separate issue for the bug in `RestRequest`.) > > To use "raw" mode, I needed to use `RestRequest::executeUnparsed` and discovered that it wasn't behaving quite the same as the normal `::execute`. I changed it to throw `UncheckedRestException` like its sibling. I inspected all uses of the method and made adjustments where needed. > > `TestHostedRepository::fileContents` didn't return the raw contents of files. Specifically it didn't preserve trailing newlines. To fix this, I changed it to no longer convert to lines and then joining them again. Erik Joelsson has updated the pull request incrementally with one additional commit since the last revision: Update forge/src/main/java/org/openjdk/skara/forge/github/GitHubRepository.java Co-authored-by: Magnus Ihse Bursie ------------- Changes: - all: https://git.openjdk.org/skara/pull/1442/files - new: https://git.openjdk.org/skara/pull/1442/files/de5bf32b..eaab258f Webrevs: - full: https://webrevs.openjdk.org/?repo=skara&pr=1442&range=01 - incr: https://webrevs.openjdk.org/?repo=skara&pr=1442&range=00-01 Stats: 1 line in 1 file changed: 0 ins; 0 del; 1 mod Patch: https://git.openjdk.org/skara/pull/1442.diff Fetch: git fetch https://git.openjdk.org/skara pull/1442/head:pull/1442 PR: https://git.openjdk.org/skara/pull/1442 From erikj at openjdk.org Wed Dec 14 14:46:23 2022 From: erikj at openjdk.org (Erik Joelsson) Date: Wed, 14 Dec 2022 14:46:23 GMT Subject: RFR: 1598: GitRepository.isHealthy takes a long time in ArchiveWorkItem [v3] In-Reply-To: References: Message-ID: > In the mlbridge bot, the `ArchiveWorkItem` is spending an inordinate amount of time running `git fsck --connectivity-only`. This is part of a rather sane strategy of verifying a repository before using it. This is done when a repository is materialized using the `HostedRepositoryPool`, if a previous local copy is found and reused, it's first checked using `Repository::isHealthy`, which runs the fsck. In the `ArchiveWorkItem` case, the repository in question is the archive repository, where all sent emails are stored for reference (and to prevent resending duplicate messages). The repository is materialized first in every `ArchiveWorkItem::run` call as it needs to read the current archive for the PR. Especially on bot restart, this triggers a lot of extra work. > > This repository is growing quite fast as it contains every PR for every repository we have Skara running against on GitHub. Having every ArchiveWorkItem verifying all this contents just doesn't scale well, when each work item will only ever read and write to a single file in the repository. > > To eliminate this inefficiency, I'm changing `ArchiveWorkItem` to read and write the single file over REST API instead. This does introduce another level of inefficiency, where the complete file must be transferred to and from the server each time, instead of just diffs, but I think given the size of the repo, and the rate at which it is growing (not to mention the fsck checks), this is the better tradeoff. > > The patch adds a new method `HostedRepository::writeFileContents` (that pairs with the existing `::fileContents`). I've added manual tests that verifies these two methods together for both GitHub and GitLab. This prompted rewriting of `GitHubRepository::fileContents` to use "raw" mode for retrieving file contents. This in turn made me discover that we could remove all the preview feature `Accept` headers, as all those GitHub APIs are stable by now. (There is currently a bug in `RestRequest` causing only the last header with the same key to be used, so only the last of the list of preview headers was ever actually in effect. This bug was also preventing me from using raw mode, as that requires another `Accept` header. Given this, and that the mockingbird API is verified to be stable now, I'm confident in removing all of them. I will file a separate issue for the bug in `RestRequest`.) > > To use "raw" mode, I needed to use `RestRequest::executeUnparsed` and discovered that it wasn't behaving quite the same as the normal `::execute`. I changed it to throw `UncheckedRestException` like its sibling. I inspected all uses of the method and made adjustments where needed. > > `TestHostedRepository::fileContents` didn't return the raw contents of files. Specifically it didn't preserve trailing newlines. To fix this, I changed it to no longer convert to lines and then joining them again. Erik Joelsson has updated the pull request incrementally with one additional commit since the last revision: Review comments ------------- Changes: - all: https://git.openjdk.org/skara/pull/1442/files - new: https://git.openjdk.org/skara/pull/1442/files/eaab258f..2263ed0e Webrevs: - full: https://webrevs.openjdk.org/?repo=skara&pr=1442&range=02 - incr: https://webrevs.openjdk.org/?repo=skara&pr=1442&range=01-02 Stats: 21 lines in 8 files changed: 2 ins; 1 del; 18 mod Patch: https://git.openjdk.org/skara/pull/1442.diff Fetch: git fetch https://git.openjdk.org/skara pull/1442/head:pull/1442 PR: https://git.openjdk.org/skara/pull/1442 From erikj at openjdk.org Wed Dec 14 14:46:25 2022 From: erikj at openjdk.org (Erik Joelsson) Date: Wed, 14 Dec 2022 14:46:25 GMT Subject: RFR: 1598: GitRepository.isHealthy takes a long time in ArchiveWorkItem [v3] In-Reply-To: References: Message-ID: <4SNbegjkET7f8fRp1exhp8QpSc-OPE5LXx6ZybkWW5U=.331a51c2-b3cf-401d-9728-373b283ad5a6@github.com> On Wed, 14 Dec 2022 12:03:41 GMT, Magnus Ihse Bursie wrote: >> Erik Joelsson has updated the pull request incrementally with one additional commit since the last revision: >> >> Review comments > > bots/mlbridge/src/main/java/org/openjdk/skara/bots/mlbridge/ArchiveWorkItem.java line 252: > >> 250: var sentMails = new ArrayList(); >> 251: var archiveContents = bot.archiveRepo().fileContents(mboxFile(), bot.archiveRef()); >> 252: archiveContents.ifPresent(s -> sentMails.addAll(Mbox.splitMbox(s, bot.emailAddress()))); > > So this adds the old contents to sentMails, and if there is none, we just ignore adding it? I needed some time to figure out why it was safe doing nothing if archiveContents was empty. Perhaps a comment about this? I didn't reflect on that as it's the same logic as before, only before it was handled with the ignored `IOException`, but I can add a comment. > forge/src/main/java/org/openjdk/skara/forge/HostedRepository.java line 100: > >> 98: * @param authorEmail Email of author and committer for commit >> 99: */ >> 100: void writeFileContents(String content, String filename, Branch branch, String message, String authorName, String authorEmail); > > Maybe not a big deal, but I thought the symmetry with `fileContents` was a bit lost when `String filename` was not the first argument to this method. Sure, makes sense. > forge/src/main/java/org/openjdk/skara/forge/github/GitHubRepository.java line 292: > >> 290: .put("name", authorName) >> 291: .put("email", authorEmail)) >> 292: .put("content", new String(Base64.getEncoder().encode(content.getBytes(StandardCharsets.UTF_8)), StandardCharsets.UTF_8)); > > Are we talking about 10's of MB here as well? We don't risk running into limits? Hm, I just realized the comment on fileContents is wrong, with raw we get up to 100MB, so I will fix that. I can't find any documentation regarding a limit when writing file contents. The manual test I have tests up to 10MB, which was fine. The biggest file in our current archive is just over 1MB. The current archive is however hosted on GitLab and not GitHub, so the GitHub limits aren't really relevant at this time. > forge/src/main/java/org/openjdk/skara/forge/gitlab/GitLabRepository.java line 334: > >> 332: return Optional.of(request.post("repository/files/" + encodedFileName) >> 333: .body(body) >> 334: .execute()); > > You don't need an `onError` to return `Optional.empty()` for the POST case? Or is it the default if you don't specify an error handler? The default for `execute()` is to throw `UncheckedRestException` if we get any kind of error response from the server. Here I made the assumption that replacing files is more common than creating new files, and simply first try PUT to replace a file and if that fails with the documented error code 400, we retry with POST to create a new file. If the POST fails, then we just let it fail. Another way of doing it would have been to first call GET to see if the file exists and then pick either PUT or POST depending on the result. I just tried to keep the number of calls down for the current common case. ------------- PR: https://git.openjdk.org/skara/pull/1442 From erikj at openjdk.org Wed Dec 14 14:53:10 2022 From: erikj at openjdk.org (Erik Joelsson) Date: Wed, 14 Dec 2022 14:53:10 GMT Subject: Integrated: 1709: Only fetch comments once in PullRequestWorkItem in pr bot In-Reply-To: References: Message-ID: On Tue, 6 Dec 2022 23:57:39 GMT, Erik Joelsson wrote: > This patch provides a minor optimization for `PullRequestWorkItem` subclasses in the pr bot by avoiding re-fetching of pr.comments(). I initially started this patch with a much bigger optimization in mind, but it turned out to not work. I still think it worth applying the remaining patch since I've gone through the trouble of implementing it. This pull request has now been integrated. Changeset: 531bf966 Author: Erik Joelsson URL: https://git.openjdk.org/skara/commit/531bf9661ff9f061761aa6c027b47cf0cf0f06ed Stats: 25 lines in 5 files changed: 15 ins; 6 del; 4 mod 1709: Only fetch comments once in PullRequestWorkItem in pr bot Reviewed-by: zsong, ihse ------------- PR: https://git.openjdk.org/skara/pull/1437 From erikj at openjdk.org Wed Dec 14 14:53:33 2022 From: erikj at openjdk.org (Erik Joelsson) Date: Wed, 14 Dec 2022 14:53:33 GMT Subject: Integrated: 1710: CommitCommandWorkItem always fetches Commit In-Reply-To: References: Message-ID: On Tue, 6 Dec 2022 19:05:34 GMT, Erik Joelsson wrote: > During bot restart, quite a few CommitCommandWorkItems are spawned. Most of them will discover that nothing needs to be done. One of the first things done in the run method is fetching the commit from the remote repository. The Commit instance will always be created with the full set of diffs for the commit. In certain cases, where commits are very large, this can take considerable time. Specifically in the jfx repos, there are examples where the REST request is split into 147 pages. (e.g. https://github.com/openjdk/jfx11u/commit/4bd6877dbd46e37f6774c50dcdc98a8a1ab41214) The way these WorkItems are spawned on startup, most of these gigantic commits will be included for a long time. The Commit instance isn't needed to figure out if a new command is present, just the comments are. This means we can move this query to after we have established that a command is to be executed. > > To further reduce overhead when fetching commits from remote repositories, I'm also making the inclusion of the "parentDiffs" optional. Of all the uses of `HostedRepository::commit`, there is currently only one that actually needs the diffs (BackportCommand). This will require the caller to be mindful of the state of the returned `Commit` object, especially when sending it other consumers, but I believe it's worth it to reduce unnecessary load on the system. > > This also affects `Forge::search`, where there is one other use where the diffs are used (when evaluating if a backport is clean). This pull request has now been integrated. Changeset: 28fe36f3 Author: Erik Joelsson URL: https://git.openjdk.org/skara/commit/28fe36f3f5ba0998c5de3430d64856285604a4e6 Stats: 82 lines in 12 files changed: 41 ins; 1 del; 40 mod 1710: CommitCommandWorkItem always fetches Commit Reviewed-by: zsong, ihse ------------- PR: https://git.openjdk.org/skara/pull/1436 From erikj at openjdk.org Wed Dec 14 15:03:01 2022 From: erikj at openjdk.org (Erik Joelsson) Date: Wed, 14 Dec 2022 15:03:01 GMT Subject: RFR: 1722: Add duration logging for RestRequestCache lock Message-ID: This patch adds additional logging of durations for things that take time. In this case it's measuring how long it takes to get the lock before calling a remote REST API. ------------- Commit messages: - SKARA-1722 Changes: https://git.openjdk.org/skara/pull/1444/files Webrev: https://webrevs.openjdk.org/?repo=skara&pr=1444&range=00 Issue: https://bugs.openjdk.org/browse/SKARA-1722 Stats: 12 lines in 1 file changed: 6 ins; 0 del; 6 mod Patch: https://git.openjdk.org/skara/pull/1444.diff Fetch: git fetch https://git.openjdk.org/skara pull/1444/head:pull/1444 PR: https://git.openjdk.org/skara/pull/1444 From ihse at openjdk.org Wed Dec 14 16:37:18 2022 From: ihse at openjdk.org (Magnus Ihse Bursie) Date: Wed, 14 Dec 2022 16:37:18 GMT Subject: RFR: 1598: GitRepository.isHealthy takes a long time in ArchiveWorkItem [v3] In-Reply-To: References: Message-ID: On Wed, 14 Dec 2022 14:46:23 GMT, Erik Joelsson wrote: >> In the mlbridge bot, the `ArchiveWorkItem` is spending an inordinate amount of time running `git fsck --connectivity-only`. This is part of a rather sane strategy of verifying a repository before using it. This is done when a repository is materialized using the `HostedRepositoryPool`, if a previous local copy is found and reused, it's first checked using `Repository::isHealthy`, which runs the fsck. In the `ArchiveWorkItem` case, the repository in question is the archive repository, where all sent emails are stored for reference (and to prevent resending duplicate messages). The repository is materialized first in every `ArchiveWorkItem::run` call as it needs to read the current archive for the PR. Especially on bot restart, this triggers a lot of extra work. >> >> This repository is growing quite fast as it contains every PR for every repository we have Skara running against on GitHub. Having every ArchiveWorkItem verifying all this contents just doesn't scale well, when each work item will only ever read and write to a single file in the repository. >> >> To eliminate this inefficiency, I'm changing `ArchiveWorkItem` to read and write the single file over REST API instead. This does introduce another level of inefficiency, where the complete file must be transferred to and from the server each time, instead of just diffs, but I think given the size of the repo, and the rate at which it is growing (not to mention the fsck checks), this is the better tradeoff. >> >> The patch adds a new method `HostedRepository::writeFileContents` (that pairs with the existing `::fileContents`). I've added manual tests that verifies these two methods together for both GitHub and GitLab. This prompted rewriting of `GitHubRepository::fileContents` to use "raw" mode for retrieving file contents. This in turn made me discover that we could remove all the preview feature `Accept` headers, as all those GitHub APIs are stable by now. (There is currently a bug in `RestRequest` causing only the last header with the same key to be used, so only the last of the list of preview headers was ever actually in effect. This bug was also preventing me from using raw mode, as that requires another `Accept` header. Given this, and that the mockingbird API is verified to be stable now, I'm confident in removing all of them. I will file a separate issue for the bug in `RestRequest`.) >> >> To use "raw" mode, I needed to use `RestRequest::executeUnparsed` and discovered that it wasn't behaving quite the same as the normal `::execute`. I changed it to throw `UncheckedRestException` like its sibling. I inspected all uses of the method and made adjustments where needed. >> >> `TestHostedRepository::fileContents` didn't return the raw contents of files. Specifically it didn't preserve trailing newlines. To fix this, I changed it to no longer convert to lines and then joining them again. > > Erik Joelsson has updated the pull request incrementally with one additional commit since the last revision: > > Review comments Marked as reviewed by ihse (Reviewer). ------------- PR: https://git.openjdk.org/skara/pull/1442 From ihse at openjdk.org Wed Dec 14 16:38:44 2022 From: ihse at openjdk.org (Magnus Ihse Bursie) Date: Wed, 14 Dec 2022 16:38:44 GMT Subject: RFR: 1722: Add duration logging for RestRequestCache lock In-Reply-To: References: Message-ID: On Wed, 14 Dec 2022 14:59:05 GMT, Erik Joelsson wrote: > This patch adds additional logging of durations for things that take time. In this case it's measuring how long it takes to get the lock before calling a remote REST API. network/src/main/java/org/openjdk/skara/network/RestRequestCache.java line 228: > 226: log.finer("Not using response cache for " + finalRequest + " (" + authId + ")"); > 227: Instant lastUpdate; > 228: var beforeLock = Instant.now(); Should this not be accompanied with any logging? ------------- PR: https://git.openjdk.org/skara/pull/1444 From ihse at openjdk.org Wed Dec 14 16:38:45 2022 From: ihse at openjdk.org (Magnus Ihse Bursie) Date: Wed, 14 Dec 2022 16:38:45 GMT Subject: RFR: 1722: Add duration logging for RestRequestCache lock In-Reply-To: References: Message-ID: On Wed, 14 Dec 2022 16:35:12 GMT, Magnus Ihse Bursie wrote: >> This patch adds additional logging of durations for things that take time. In this case it's measuring how long it takes to get the lock before calling a remote REST API. > > network/src/main/java/org/openjdk/skara/network/RestRequestCache.java line 228: > >> 226: log.finer("Not using response cache for " + finalRequest + " (" + authId + ")"); >> 227: Instant lastUpdate; >> 228: var beforeLock = Instant.now(); > > Should this not be accompanied with any logging? Ah, I see, you measure the time across two locks. Nevermind. ------------- PR: https://git.openjdk.org/skara/pull/1444 From ihse at openjdk.org Wed Dec 14 16:44:34 2022 From: ihse at openjdk.org (Magnus Ihse Bursie) Date: Wed, 14 Dec 2022 16:44:34 GMT Subject: RFR: 1722: Add duration logging for RestRequestCache lock In-Reply-To: References: Message-ID: On Wed, 14 Dec 2022 14:59:05 GMT, Erik Joelsson wrote: > This patch adds additional logging of durations for things that take time. In this case it's measuring how long it takes to get the lock before calling a remote REST API. Marked as reviewed by ihse (Reviewer). ------------- PR: https://git.openjdk.org/skara/pull/1444 From erikj at openjdk.org Wed Dec 14 17:05:16 2022 From: erikj at openjdk.org (Erik Joelsson) Date: Wed, 14 Dec 2022 17:05:16 GMT Subject: git: openjdk/skara: 1598: GitRepository.isHealthy takes a long time in ArchiveWorkItem Message-ID: Changeset: affcd47d Author: Erik Joelsson Date: 2022-12-14 17:03:29 +0000 URL: https://git.openjdk.org/skara/commit/affcd47dcc5c996fd2c1bcfcf13c7a2b904688a4 1598: GitRepository.isHealthy takes a long time in ArchiveWorkItem Reviewed-by: ihse ! bots/mlbridge/src/main/java/org/openjdk/skara/bots/mlbridge/ArchiveWorkItem.java ! bots/tester/src/test/java/org/openjdk/skara/bots/tester/InMemoryHostedRepository.java ! forge/src/main/java/org/openjdk/skara/forge/HostedRepository.java ! forge/src/main/java/org/openjdk/skara/forge/github/GitHubHost.java ! forge/src/main/java/org/openjdk/skara/forge/github/GitHubRepository.java ! forge/src/main/java/org/openjdk/skara/forge/gitlab/GitLabRepository.java ! forge/src/test/java/org/openjdk/skara/forge/github/GitHubRestApiTests.java ! forge/src/test/java/org/openjdk/skara/forge/gitlab/GitLabRestApiTest.java ! network/src/main/java/org/openjdk/skara/network/RestRequest.java ! test/src/main/java/org/openjdk/skara/test/TestHostedRepository.java From erikj at openjdk.org Wed Dec 14 17:06:44 2022 From: erikj at openjdk.org (Erik Joelsson) Date: Wed, 14 Dec 2022 17:06:44 GMT Subject: Integrated: 1598: GitRepository.isHealthy takes a long time in ArchiveWorkItem In-Reply-To: References: Message-ID: On Mon, 12 Dec 2022 23:52:50 GMT, Erik Joelsson wrote: > In the mlbridge bot, the `ArchiveWorkItem` is spending an inordinate amount of time running `git fsck --connectivity-only`. This is part of a rather sane strategy of verifying a repository before using it. This is done when a repository is materialized using the `HostedRepositoryPool`, if a previous local copy is found and reused, it's first checked using `Repository::isHealthy`, which runs the fsck. In the `ArchiveWorkItem` case, the repository in question is the archive repository, where all sent emails are stored for reference (and to prevent resending duplicate messages). The repository is materialized first in every `ArchiveWorkItem::run` call as it needs to read the current archive for the PR. Especially on bot restart, this triggers a lot of extra work. > > This repository is growing quite fast as it contains every PR for every repository we have Skara running against on GitHub. Having every ArchiveWorkItem verifying all this contents just doesn't scale well, when each work item will only ever read and write to a single file in the repository. > > To eliminate this inefficiency, I'm changing `ArchiveWorkItem` to read and write the single file over REST API instead. This does introduce another level of inefficiency, where the complete file must be transferred to and from the server each time, instead of just diffs, but I think given the size of the repo, and the rate at which it is growing (not to mention the fsck checks), this is the better tradeoff. > > The patch adds a new method `HostedRepository::writeFileContents` (that pairs with the existing `::fileContents`). I've added manual tests that verifies these two methods together for both GitHub and GitLab. This prompted rewriting of `GitHubRepository::fileContents` to use "raw" mode for retrieving file contents. This in turn made me discover that we could remove all the preview feature `Accept` headers, as all those GitHub APIs are stable by now. (There is currently a bug in `RestRequest` causing only the last header with the same key to be used, so only the last of the list of preview headers was ever actually in effect. This bug was also preventing me from using raw mode, as that requires another `Accept` header. Given this, and that the mockingbird API is verified to be stable now, I'm confident in removing all of them. I will file a separate issue for the bug in `RestRequest`.) > > To use "raw" mode, I needed to use `RestRequest::executeUnparsed` and discovered that it wasn't behaving quite the same as the normal `::execute`. I changed it to throw `UncheckedRestException` like its sibling. I inspected all uses of the method and made adjustments where needed. > > `TestHostedRepository::fileContents` didn't return the raw contents of files. Specifically it didn't preserve trailing newlines. To fix this, I changed it to no longer convert to lines and then joining them again. This pull request has now been integrated. Changeset: affcd47d Author: Erik Joelsson URL: https://git.openjdk.org/skara/commit/affcd47dcc5c996fd2c1bcfcf13c7a2b904688a4 Stats: 242 lines in 10 files changed: 210 ins; 11 del; 21 mod 1598: GitRepository.isHealthy takes a long time in ArchiveWorkItem Reviewed-by: ihse ------------- PR: https://git.openjdk.org/skara/pull/1442 From erikj at openjdk.org Wed Dec 14 17:07:11 2022 From: erikj at openjdk.org (Erik Joelsson) Date: Wed, 14 Dec 2022 17:07:11 GMT Subject: Integrated: 1722: Add duration logging for RestRequestCache lock In-Reply-To: References: Message-ID: On Wed, 14 Dec 2022 14:59:05 GMT, Erik Joelsson wrote: > This patch adds additional logging of durations for things that take time. In this case it's measuring how long it takes to get the lock before calling a remote REST API. This pull request has now been integrated. Changeset: 3620b0e0 Author: Erik Joelsson URL: https://git.openjdk.org/skara/commit/3620b0e0e5022ce5f5a6596d26fc866d33bec0f8 Stats: 12 lines in 1 file changed: 6 ins; 0 del; 6 mod 1722: Add duration logging for RestRequestCache lock Reviewed-by: ihse ------------- PR: https://git.openjdk.org/skara/pull/1444 From zsong at openjdk.org Wed Dec 14 19:51:07 2022 From: zsong at openjdk.org (Zhao Song) Date: Wed, 14 Dec 2022 19:51:07 GMT Subject: RFR: 1714: CSR detection only happens on the title bug of a PR Message-ID: <_q1X42GM_kNlhHEYoSYfZr5D3w_puyzAER5DR7RX0sM=.c2a9fbea-99b4-426b-93c5-9f590180c1db@github.com> Currently, our bots only detect CSR issues that are directly linked to the main issue. With this patch, our bots will be able to detect CSR issues that are linked to all of the issues that this PR solves. Additionally, all detected CSR issues will affect the integration blockers. The `/csr needed` command will allow you to create a CSR issue that is linked to any of the issues solved by this PR. The `/csr unneeded` command will be blocked if any of the issues associated with this PR have an active CSR issue. ------------- Commit messages: - SKARA-1714 Changes: https://git.openjdk.org/skara/pull/1443/files Webrev: https://webrevs.openjdk.org/?repo=skara&pr=1443&range=00 Issue: https://bugs.openjdk.org/browse/SKARA-1714 Stats: 563 lines in 9 files changed: 321 ins; 52 del; 190 mod Patch: https://git.openjdk.org/skara/pull/1443.diff Fetch: git fetch https://git.openjdk.org/skara pull/1443/head:pull/1443 PR: https://git.openjdk.org/skara/pull/1443 From zsong at openjdk.org Wed Dec 14 21:20:24 2022 From: zsong at openjdk.org (Zhao Song) Date: Wed, 14 Dec 2022 21:20:24 GMT Subject: RFR: 1704: Add or update copyright header in some files [v2] In-Reply-To: References: Message-ID: > The patch just updated copyright headers in some files. Zhao Song has updated the pull request incrementally with one additional commit since the last revision: run script to update copyright year ------------- Changes: - all: https://git.openjdk.org/skara/pull/1431/files - new: https://git.openjdk.org/skara/pull/1431/files/e69e110d..a55ef1d7 Webrevs: - full: https://webrevs.openjdk.org/?repo=skara&pr=1431&range=01 - incr: https://webrevs.openjdk.org/?repo=skara&pr=1431&range=00-01 Stats: 236 lines in 236 files changed: 0 ins; 0 del; 236 mod Patch: https://git.openjdk.org/skara/pull/1431.diff Fetch: git fetch https://git.openjdk.org/skara pull/1431/head:pull/1431 PR: https://git.openjdk.org/skara/pull/1431 From zsong at openjdk.org Wed Dec 14 21:20:25 2022 From: zsong at openjdk.org (Zhao Song) Date: Wed, 14 Dec 2022 21:20:25 GMT Subject: RFR: 1704: Add or update copyright header in some files In-Reply-To: <3dqMfOXZAvyiJVGf_rFt3SXgM9UjNzeVX0yGvkga5TM=.6ddcbb2c-ad68-49a1-8237-c49bb0fe9357@github.com> References: <3dqMfOXZAvyiJVGf_rFt3SXgM9UjNzeVX0yGvkga5TM=.6ddcbb2c-ad68-49a1-8237-c49bb0fe9357@github.com> Message-ID: On Fri, 2 Dec 2022 00:36:50 GMT, Magnus Ihse Bursie wrote: > Sorry, that was just an example, to built upon. That particular sequence of command will find which files have been changed for a set of git commits, and will update the year to 2022. To get what you need for this PR is a bit more tricky. > > There is a script in the JDK, https://github.com/openjdk/jdk/blob/master/make/scripts/update_copyright_year.sh, which does something along these lines. I don't know if you can just download and run it in Skara, or if it needs adaptation. I ran the script in Skara and it worked perfectly! Thank you! ------------- PR: https://git.openjdk.org/skara/pull/1431 From zsong at openjdk.org Thu Dec 15 00:51:06 2022 From: zsong at openjdk.org (Zhao Song) Date: Thu, 15 Dec 2022 00:51:06 GMT Subject: Integrated: 1704: Add or update copyright header in some files In-Reply-To: References: Message-ID: On Thu, 1 Dec 2022 23:34:47 GMT, Zhao Song wrote: > The patch just updated copyright headers in some files. This pull request has now been integrated. Changeset: 1fd55208 Author: Zhao Song URL: https://git.openjdk.org/skara/commit/1fd5520828be551f18bc684d1f53fe8b1f464be8 Stats: 1022 lines in 371 files changed: 682 ins; 0 del; 340 mod 1704: Add or update copyright header in some files Reviewed-by: ihse, erikj ------------- PR: https://git.openjdk.org/skara/pull/1431 From erikj at openjdk.org Thu Dec 15 17:53:49 2022 From: erikj at openjdk.org (Erik Joelsson) Date: Thu, 15 Dec 2022 17:53:49 GMT Subject: RFR: 1714: CSR detection only happens on the title bug of a PR In-Reply-To: <_q1X42GM_kNlhHEYoSYfZr5D3w_puyzAER5DR7RX0sM=.c2a9fbea-99b4-426b-93c5-9f590180c1db@github.com> References: <_q1X42GM_kNlhHEYoSYfZr5D3w_puyzAER5DR7RX0sM=.c2a9fbea-99b4-426b-93c5-9f590180c1db@github.com> Message-ID: On Wed, 14 Dec 2022 01:08:08 GMT, Zhao Song wrote: > Currently, our bots only detect CSR issues that are directly linked to the main issue. > > With this patch, our bots will be able to detect CSR issues that are linked to all of the issues that this PR solves. Additionally, all detected CSR issues will affect the integration blockers. > > The `/csr needed` command will allow you to create a CSR issue that is linked to any of the issues solved by this PR. The `/csr unneeded` command will be blocked if any of the issues associated with this PR have an active CSR issue. I've made a first pass over this patch. It's a rather complicated change which introduces a lot of corner cases that need careful reasoning and testing. I found some issues that I've pointed out, but there could definitely be more. We also know that the current CSR implementation has issues, which may get amplified by this change. I'm still not sure if we should risk going ahead with this before we figure out a better solution to the whole CSR functionality. bots/csr/src/main/java/org/openjdk/skara/bots/csr/PullRequestWorkItem.java line 96: > 94: var statusMessage = getStatusMessage(pr); > 95: return hasCsrIssue(statusMessage, csr) && > 96: (statusMessage.contains("- [ ] Change requires a CSR request (" + csr.id() + ") to be approved") || Please remove the 'a' and parentheses when the CSR issue number is known. Suggestion: (statusMessage.contains("- [ ] Change requires CSR request " + csr.id() + " to be approved") || bots/csr/src/main/java/org/openjdk/skara/bots/csr/PullRequestWorkItem.java line 136: > 134: var issues = new ArrayList(); > 135: mainIssue.ifPresent(issues::add); > 136: issues.addAll(SolvesTracker.currentSolved(pr.repository().forge().currentUser(), pr.comments())); The `SolvesTracker` in the pr bot shouldn't be reachable from this module. If you need it in multiple bots, then please move it to bots.utils. bots/csr/src/main/java/org/openjdk/skara/bots/csr/PullRequestWorkItem.java line 154: > 152: var jbsIssueOpt = project.issue(issue.shortId()); > 153: if (jbsIssueOpt.isEmpty()) { > 154: // One issue could not be found in JBS, so the csr label cannot be removed until the problem is solved Suggestion: // An issue could not be found, so the csr label cannot be removed bots/csr/src/main/java/org/openjdk/skara/bots/csr/PullRequestWorkItem.java line 157: > 155: allCSRApproved = false; > 156: var issueId = issue.project().isEmpty() ? (project.name() + "-" + issue.id()) : issue.id(); > 157: log.info("No issue found in JBS related with this issue " + issueId + "for " + describe(pr)); Suggestion: log.info("issueId + " for " + describe(pr) + " not found"); bots/csr/src/main/java/org/openjdk/skara/bots/csr/PullRequestWorkItem.java line 158: > 156: var issueId = issue.project().isEmpty() ? (project.name() + "-" + issue.id()) : issue.id(); > 157: log.info("No issue found in JBS related with this issue " + issueId + "for " + describe(pr)); > 158: // allCSRApproved is false now, so we could break now Suggestion: // allCSRApproved is now false, so there is no point in continuing bots/csr/src/main/java/org/openjdk/skara/bots/csr/PullRequestWorkItem.java line 165: > 163: if (csrOptional.isEmpty()) { > 164: log.info("No CSR found for issue " + jbsIssueOpt.get().id() + " for " + describe(pr)); > 165: continue; What happens if no issue has a CSR but the label was added using the CSR command? In that case we should not remove the label. bots/csr/src/main/java/org/openjdk/skara/bots/csr/PullRequestWorkItem.java line 175: > 173: log.info("The PR body doesn't have the CSR issue or progress, adding the csr update marker for this csr issue" > 174: + csr.id() + " for " + describe(pr)); > 175: addUpdateMarker(pr); `addUpdateMarker` isn't suitable to be called multiple times. Each call will modify the body again. bots/pr/src/main/java/org/openjdk/skara/bots/pr/CSRCommand.java line 61: > 59: writer.println("@" + pr.author().username() + " please create a [CSR](https://wiki.openjdk.org/display/csr/Main) " + > 60: "request for any issue associated with this pr with the correct fix version " + > 61: "This pull request cannot be integrated until the CSR request is approved."); This message doesn't really make sense. If there is more than one issue associated with the pr, then I don't think there is any point printing this at all. We can't know which issue that needs a CSR, so the person issuing the command has to be responsible for clarifying that. The main point of this message is to link to the issue. So, if there is only one issue (which will be the case in 99% of cases involving CSRs), print the old message, otherwise don't call this method, or have it do nothing. bots/pr/src/main/java/org/openjdk/skara/bots/pr/CSRCommand.java line 136: > 134: // The issue has a non-withdrawn csr issue, the bot should direct the user to withdraw the csr firstly. > 135: reply.println("The CSR requirement cannot be removed as there is already a CSR associated with the issue [" + > 136: jbsIssue.id() + "](" + jbsIssue.webUrl() + ") of this pull request. Please withdraw the CSR [" Suggestion: jbsIssue.id() + "](" + jbsIssue.webUrl() + "). Please withdraw the CSR [" bots/pr/src/main/java/org/openjdk/skara/bots/pr/CSRCommand.java line 142: > 140: } > 141: } > 142: // All the issue associated with this pr don't have csr issue or the csr issue has already been withdrawn, Suggestion: // All the issues associated with this pr either don't have csr issue or the csr issue has been withdrawn, bots/pr/src/main/java/org/openjdk/skara/bots/pr/CSRCommand.java line 193: > 191: continue; > 192: } > 193: // Could find a csr issue for one of the issues associated with this pr Suggestion: // Found a csr issue for one of the issues associated with this pr bots/pr/src/main/java/org/openjdk/skara/bots/pr/CSRCommand.java line 217: > 215: pr.addLabel(CSR_LABEL); > 216: } > 217: return; I'm not sure about this behavior. We loop until we find an existing CSR, then print a reply based on the state of that CSR. bots/pr/src/main/java/org/openjdk/skara/bots/pr/CSRCommand.java line 219: > 217: return; > 218: } > 219: // All the issues associated with pr don't have csr issue or the csr issue has already been withdrawn Suggestion: // All the issues associated with pr either don't have csr issue or the csr issue has already been withdrawn bots/pr/src/main/java/org/openjdk/skara/bots/pr/CheckRun.java line 268: > 266: .toList(); > 267: if (csrIssues.isEmpty() && pr.labelNames().contains("csr")) { > 268: ret.put("Change requires a CSR request (need to be created) to be approved", false); Suggestion: ret.put("Change requires a CSR request (needs to be created) to be approved", false); bots/pr/src/main/java/org/openjdk/skara/bots/pr/CheckRun.java line 272: > 270: for (var csrIssue : csrIssues) { > 271: if (!csrIssue.isClosed()) { > 272: ret.put("Change requires a CSR request (" + csrIssue.id() + ") to be approved", false); I think we can make the issue IDs links here. bots/pr/src/main/java/org/openjdk/skara/bots/pr/CheckRun.java line 661: > 659: if (issueType != null && "CSR".equals(issueType.asString())) { > 660: progressBody.append(" (**CSR**)"); > 661: if(isWithdrawnCSR(iss.get())){ Suggestion: if (isWithdrawnCSR(iss.get())) { ------------- PR: https://git.openjdk.org/skara/pull/1443 From zsong at openjdk.org Thu Dec 15 18:18:12 2022 From: zsong at openjdk.org (Zhao Song) Date: Thu, 15 Dec 2022 18:18:12 GMT Subject: RFR: 1714: CSR detection only happens on the title bug of a PR In-Reply-To: References: <_q1X42GM_kNlhHEYoSYfZr5D3w_puyzAER5DR7RX0sM=.c2a9fbea-99b4-426b-93c5-9f590180c1db@github.com> Message-ID: On Thu, 15 Dec 2022 17:00:22 GMT, Erik Joelsson wrote: >> Currently, our bots only detect CSR issues that are directly linked to the main issue. >> >> With this patch, our bots will be able to detect CSR issues that are linked to all of the issues that this PR solves. Additionally, all detected CSR issues will affect the integration blockers. >> >> The `/csr needed` command will allow you to create a CSR issue that is linked to any of the issues solved by this PR. The `/csr unneeded` command will be blocked if any of the issues associated with this PR have an active CSR issue. > > bots/csr/src/main/java/org/openjdk/skara/bots/csr/PullRequestWorkItem.java line 175: > >> 173: log.info("The PR body doesn't have the CSR issue or progress, adding the csr update marker for this csr issue" >> 174: + csr.id() + " for " + describe(pr)); >> 175: addUpdateMarker(pr); > > `addUpdateMarker` isn't suitable to be called multiple times. Each call will modify the body again. Right, I will change it, thx! ------------- PR: https://git.openjdk.org/skara/pull/1443 From zsong at openjdk.org Thu Dec 15 18:27:42 2022 From: zsong at openjdk.org (Zhao Song) Date: Thu, 15 Dec 2022 18:27:42 GMT Subject: RFR: 1714: CSR detection only happens on the title bug of a PR In-Reply-To: References: <_q1X42GM_kNlhHEYoSYfZr5D3w_puyzAER5DR7RX0sM=.c2a9fbea-99b4-426b-93c5-9f590180c1db@github.com> Message-ID: On Thu, 15 Dec 2022 17:02:22 GMT, Erik Joelsson wrote: >> Currently, our bots only detect CSR issues that are directly linked to the main issue. >> >> With this patch, our bots will be able to detect CSR issues that are linked to all of the issues that this PR solves. Additionally, all detected CSR issues will affect the integration blockers. >> >> The `/csr needed` command will allow you to create a CSR issue that is linked to any of the issues solved by this PR. The `/csr unneeded` command will be blocked if any of the issues associated with this PR have an active CSR issue. > > bots/csr/src/main/java/org/openjdk/skara/bots/csr/PullRequestWorkItem.java line 165: > >> 163: if (csrOptional.isEmpty()) { >> 164: log.info("No CSR found for issue " + jbsIssueOpt.get().id() + " for " + describe(pr)); >> 165: continue; > > What happens if no issue has a CSR but the label was added using the CSR command? In that case we should not remove the label. I overlooked this case. Will fix it. ------------- PR: https://git.openjdk.org/skara/pull/1443 From zsong at openjdk.org Thu Dec 15 18:40:45 2022 From: zsong at openjdk.org (Zhao Song) Date: Thu, 15 Dec 2022 18:40:45 GMT Subject: RFR: 1714: CSR detection only happens on the title bug of a PR In-Reply-To: References: <_q1X42GM_kNlhHEYoSYfZr5D3w_puyzAER5DR7RX0sM=.c2a9fbea-99b4-426b-93c5-9f590180c1db@github.com> Message-ID: On Thu, 15 Dec 2022 17:33:58 GMT, Erik Joelsson wrote: >> Currently, our bots only detect CSR issues that are directly linked to the main issue. >> >> With this patch, our bots will be able to detect CSR issues that are linked to all of the issues that this PR solves. Additionally, all detected CSR issues will affect the integration blockers. >> >> The `/csr needed` command will allow you to create a CSR issue that is linked to any of the issues solved by this PR. The `/csr unneeded` command will be blocked if any of the issues associated with this PR have an active CSR issue. > > bots/pr/src/main/java/org/openjdk/skara/bots/pr/CSRCommand.java line 217: > >> 215: pr.addLabel(CSR_LABEL); >> 216: } >> 217: return; > > I'm not sure about this behavior. We loop until we find an existing CSR, then print a reply based on the state of that CSR. Yes, I think this behavior is right because if we could reach the return statement, it means we already found a non-withdrawn CSR. ------------- PR: https://git.openjdk.org/skara/pull/1443 From zsong at openjdk.org Thu Dec 15 18:44:02 2022 From: zsong at openjdk.org (Zhao Song) Date: Thu, 15 Dec 2022 18:44:02 GMT Subject: RFR: 1714: CSR detection only happens on the title bug of a PR In-Reply-To: References: <_q1X42GM_kNlhHEYoSYfZr5D3w_puyzAER5DR7RX0sM=.c2a9fbea-99b4-426b-93c5-9f590180c1db@github.com> Message-ID: On Thu, 15 Dec 2022 17:44:49 GMT, Erik Joelsson wrote: >> Currently, our bots only detect CSR issues that are directly linked to the main issue. >> >> With this patch, our bots will be able to detect CSR issues that are linked to all of the issues that this PR solves. Additionally, all detected CSR issues will affect the integration blockers. >> >> The `/csr needed` command will allow you to create a CSR issue that is linked to any of the issues solved by this PR. The `/csr unneeded` command will be blocked if any of the issues associated with this PR have an active CSR issue. > > bots/pr/src/main/java/org/openjdk/skara/bots/pr/CheckRun.java line 272: > >> 270: for (var csrIssue : csrIssues) { >> 271: if (!csrIssue.isClosed()) { >> 272: ret.put("Change requires a CSR request (" + csrIssue.id() + ") to be approved", false); > > I think we can make the issue IDs links here. I thought about this too. But then I realized the links can be found under the 'Issues' part in PR body. Do you think we still need the links here? If so, I will add the links. ------------- PR: https://git.openjdk.org/skara/pull/1443 From zsong at openjdk.org Thu Dec 15 18:50:35 2022 From: zsong at openjdk.org (Zhao Song) Date: Thu, 15 Dec 2022 18:50:35 GMT Subject: RFR: 1714: CSR detection only happens on the title bug of a PR In-Reply-To: References: <_q1X42GM_kNlhHEYoSYfZr5D3w_puyzAER5DR7RX0sM=.c2a9fbea-99b4-426b-93c5-9f590180c1db@github.com> Message-ID: On Thu, 15 Dec 2022 17:48:06 GMT, Erik Joelsson wrote: >> Currently, our bots only detect CSR issues that are directly linked to the main issue. >> >> With this patch, our bots will be able to detect CSR issues that are linked to all of the issues that this PR solves. Additionally, all detected CSR issues will affect the integration blockers. >> >> The `/csr needed` command will allow you to create a CSR issue that is linked to any of the issues solved by this PR. The `/csr unneeded` command will be blocked if any of the issues associated with this PR have an active CSR issue. > > bots/csr/src/main/java/org/openjdk/skara/bots/csr/PullRequestWorkItem.java line 136: > >> 134: var issues = new ArrayList(); >> 135: mainIssue.ifPresent(issues::add); >> 136: issues.addAll(SolvesTracker.currentSolved(pr.repository().forge().currentUser(), pr.comments())); > > The `SolvesTracker` in the pr bot shouldn't be reachable from this module. If you need it in multiple bots, then please move it to bots.common. Thanks, fixed it. ------------- PR: https://git.openjdk.org/skara/pull/1443 From zsong at openjdk.org Thu Dec 15 19:29:38 2022 From: zsong at openjdk.org (Zhao Song) Date: Thu, 15 Dec 2022 19:29:38 GMT Subject: RFR: 1714: CSR detection only happens on the title bug of a PR [v2] In-Reply-To: <_q1X42GM_kNlhHEYoSYfZr5D3w_puyzAER5DR7RX0sM=.c2a9fbea-99b4-426b-93c5-9f590180c1db@github.com> References: <_q1X42GM_kNlhHEYoSYfZr5D3w_puyzAER5DR7RX0sM=.c2a9fbea-99b4-426b-93c5-9f590180c1db@github.com> Message-ID: > Currently, our bots only detect CSR issues that are directly linked to the main issue. > > With this patch, our bots will be able to detect CSR issues that are linked to all of the issues that this PR solves. Additionally, all detected CSR issues will affect the integration blockers. > > The `/csr needed` command will allow you to create a CSR issue that is linked to any of the issues solved by this PR. The `/csr unneeded` command will be blocked if any of the issues associated with this PR have an active CSR issue. Zhao Song has updated the pull request incrementally with one additional commit since the last revision: fix problems ------------- Changes: - all: https://git.openjdk.org/skara/pull/1443/files - new: https://git.openjdk.org/skara/pull/1443/files/31c75211..f2f88456 Webrevs: - full: https://webrevs.openjdk.org/?repo=skara&pr=1443&range=01 - incr: https://webrevs.openjdk.org/?repo=skara&pr=1443&range=00-01 Stats: 237 lines in 12 files changed: 81 ins; 70 del; 86 mod Patch: https://git.openjdk.org/skara/pull/1443.diff Fetch: git fetch https://git.openjdk.org/skara pull/1443/head:pull/1443 PR: https://git.openjdk.org/skara/pull/1443 From zsong at openjdk.org Thu Dec 15 19:37:08 2022 From: zsong at openjdk.org (Zhao Song) Date: Thu, 15 Dec 2022 19:37:08 GMT Subject: RFR: 1714: CSR detection only happens on the title bug of a PR [v2] In-Reply-To: References: <_q1X42GM_kNlhHEYoSYfZr5D3w_puyzAER5DR7RX0sM=.c2a9fbea-99b4-426b-93c5-9f590180c1db@github.com> Message-ID: <7Fe49mCm7C23qjZHkaFucbxMN6wZT19U8n_zkw09xaY=.df61673d-6e33-4d52-822a-8079fb6d6e96@github.com> On Thu, 15 Dec 2022 17:51:37 GMT, Erik Joelsson wrote: > I've made a first pass over this patch. It's a rather complicated change which introduces a lot of corner cases that need careful reasoning and testing. I found some issues that I've pointed out, but there could definitely be more. We also know that the current CSR implementation has issues, which may get amplified by this change. I'm still not sure if we should risk going ahead with this before we figure out a better solution to the whole CSR functionality. I fixed some of the problems you mentioned. Before trying to fix SKARA-1714, I didn't realize it's so complicated. So I can totally understand your concerns. There must be some corner cases we haven't covered. I think I should do more tests and also think of how to move the functions from CSR bots to PR bots later. ------------- PR: https://git.openjdk.org/skara/pull/1443 From erikj at openjdk.org Thu Dec 15 22:16:39 2022 From: erikj at openjdk.org (Erik Joelsson) Date: Thu, 15 Dec 2022 22:16:39 GMT Subject: RFR: 1715: Configure custom default security level for new backport issues in notifier Message-ID: This patch makes it possible to configure a default security level to use for any newly created backport in the IssueNotifier. The configuration is done per repository and branch, with a regex for matching branch names. This is what it would look like in the bot configuration. "issue": { "project": "bugs/JDK", "fixversions": {}, "buildname": "master", "headversion": true, "defaultsecurity": { // <- new "jdk.*": "10000", } } If the parent bug already has a security level, then that is inherited, just like today, otherwise this new default value is used. ------------- Commit messages: - SKARA-1715 Changes: https://git.openjdk.org/skara/pull/1445/files Webrev: https://webrevs.openjdk.org/?repo=skara&pr=1445&range=00 Issue: https://bugs.openjdk.org/browse/SKARA-1715 Stats: 136 lines in 6 files changed: 125 ins; 0 del; 11 mod Patch: https://git.openjdk.org/skara/pull/1445.diff Fetch: git fetch https://git.openjdk.org/skara pull/1445/head:pull/1445 PR: https://git.openjdk.org/skara/pull/1445 From erikj at openjdk.org Thu Dec 15 22:19:07 2022 From: erikj at openjdk.org (Erik Joelsson) Date: Thu, 15 Dec 2022 22:19:07 GMT Subject: RFR: 1714: CSR detection only happens on the title bug of a PR [v2] In-Reply-To: References: <_q1X42GM_kNlhHEYoSYfZr5D3w_puyzAER5DR7RX0sM=.c2a9fbea-99b4-426b-93c5-9f590180c1db@github.com> Message-ID: On Thu, 15 Dec 2022 18:38:30 GMT, Zhao Song wrote: >> bots/pr/src/main/java/org/openjdk/skara/bots/pr/CSRCommand.java line 217: >> >>> 215: pr.addLabel(CSR_LABEL); >>> 216: } >>> 217: return; >> >> I'm not sure about this behavior. We loop until we find an existing CSR, then print a reply based on the state of that CSR. > > Yes, I think this behavior is right because if we could reach the return statement, it means we already found a non-withdrawn CSR. But what if there are more? If we find one approved, there may also be another that is withdrawn. >> bots/pr/src/main/java/org/openjdk/skara/bots/pr/CheckRun.java line 272: >> >>> 270: for (var csrIssue : csrIssues) { >>> 271: if (!csrIssue.isClosed()) { >>> 272: ret.put("Change requires a CSR request (" + csrIssue.id() + ") to be approved", false); >> >> I think we can make the issue IDs links here. > > I thought about this too. But then I realized the links can be found under the 'Issues' part in PR body. Do you think we still need the links here? If so, I will add the links. I think it would help. ------------- PR: https://git.openjdk.org/skara/pull/1443 From zsong at openjdk.org Thu Dec 15 22:31:29 2022 From: zsong at openjdk.org (Zhao Song) Date: Thu, 15 Dec 2022 22:31:29 GMT Subject: RFR: 1714: CSR detection only happens on the title bug of a PR [v2] In-Reply-To: References: <_q1X42GM_kNlhHEYoSYfZr5D3w_puyzAER5DR7RX0sM=.c2a9fbea-99b4-426b-93c5-9f590180c1db@github.com> Message-ID: On Thu, 15 Dec 2022 22:14:20 GMT, Erik Joelsson wrote: >> Yes, I think this behavior is right because if we could reach the return statement, it means we already found a non-withdrawn CSR. > > But what if there are more? If we find one approved, there may also be another that is withdrawn. If we found one approved, we will reply some messages to user and return. What do you think should we do if there is another that is withdrawn? I don't think we need to take any action. >> I thought about this too. But then I realized the links can be found under the 'Issues' part in PR body. Do you think we still need the links here? If so, I will add the links. > > I think it would help. Ok, I will add the links ------------- PR: https://git.openjdk.org/skara/pull/1443 From zsong at openjdk.org Thu Dec 15 23:07:55 2022 From: zsong at openjdk.org (Zhao Song) Date: Thu, 15 Dec 2022 23:07:55 GMT Subject: RFR: 1714: CSR detection only happens on the title bug of a PR [v3] In-Reply-To: <_q1X42GM_kNlhHEYoSYfZr5D3w_puyzAER5DR7RX0sM=.c2a9fbea-99b4-426b-93c5-9f590180c1db@github.com> References: <_q1X42GM_kNlhHEYoSYfZr5D3w_puyzAER5DR7RX0sM=.c2a9fbea-99b4-426b-93c5-9f590180c1db@github.com> Message-ID: <5ynjqomwiITd-Aoo1--MZIPay4WL2qwUUea2Lctssew=.69ffd021-d9aa-43ba-a0cd-c5e1e8f9c4b6@github.com> > Currently, our bots only detect CSR issues that are directly linked to the main issue. > > With this patch, our bots will be able to detect CSR issues that are linked to all of the issues that this PR solves. Additionally, all detected CSR issues will affect the integration blockers. > > The `/csr needed` command will allow you to create a CSR issue that is linked to any of the issues solved by this PR. The `/csr unneeded` command will be blocked if any of the issues associated with this PR have an active CSR issue. Zhao Song has updated the pull request incrementally with one additional commit since the last revision: fix a problem ------------- Changes: - all: https://git.openjdk.org/skara/pull/1443/files - new: https://git.openjdk.org/skara/pull/1443/files/f2f88456..989937b2 Webrevs: - full: https://webrevs.openjdk.org/?repo=skara&pr=1443&range=02 - incr: https://webrevs.openjdk.org/?repo=skara&pr=1443&range=01-02 Stats: 42 lines in 4 files changed: 17 ins; 0 del; 25 mod Patch: https://git.openjdk.org/skara/pull/1443.diff Fetch: git fetch https://git.openjdk.org/skara pull/1443/head:pull/1443 PR: https://git.openjdk.org/skara/pull/1443 From erikj at openjdk.org Fri Dec 16 13:39:34 2022 From: erikj at openjdk.org (Erik Joelsson) Date: Fri, 16 Dec 2022 13:39:34 GMT Subject: RFR: 1714: CSR detection only happens on the title bug of a PR [v3] In-Reply-To: References: <_q1X42GM_kNlhHEYoSYfZr5D3w_puyzAER5DR7RX0sM=.c2a9fbea-99b4-426b-93c5-9f590180c1db@github.com> Message-ID: <739Y2MeQLvvjk07nEl6G8EvmmtqDx887vb7Kbza5iA4=.9a8da8c2-ce5f-4d60-a022-05d98fd27132@github.com> On Thu, 15 Dec 2022 22:29:12 GMT, Zhao Song wrote: >> But what if there are more? If we find one approved, there may also be another that is withdrawn. > > If we found one approved, we will reply some messages to user and return. What do you think should we do if there is another that is withdrawn? I don't think we need to take any action. Say there are two CSRs present already, one approved and one open. If the approved one is found first, then we get the needed marker but no label. If the open one is found first, then we also get the label. This is inconsistent behavior. What I think should happen is. If there are already CSRs present, then 1. if all are approved, just add the needed marker 2. If at least one is open, add both needed marker and label 3. If all are withdrawn, add both needed marker and label If there are no CSRs present, then 4. Add both needed marker and label To achieve this you will have to always iterate over all of them and set booleans for each condition, then decide the correct action based on the complete state. ------------- PR: https://git.openjdk.org/skara/pull/1443 From zsong at openjdk.org Fri Dec 16 17:27:23 2022 From: zsong at openjdk.org (Zhao Song) Date: Fri, 16 Dec 2022 17:27:23 GMT Subject: RFR: 1714: CSR detection only happens on the title bug of a PR [v3] In-Reply-To: <739Y2MeQLvvjk07nEl6G8EvmmtqDx887vb7Kbza5iA4=.9a8da8c2-ce5f-4d60-a022-05d98fd27132@github.com> References: <_q1X42GM_kNlhHEYoSYfZr5D3w_puyzAER5DR7RX0sM=.c2a9fbea-99b4-426b-93c5-9f590180c1db@github.com> <739Y2MeQLvvjk07nEl6G8EvmmtqDx887vb7Kbza5iA4=.9a8da8c2-ce5f-4d60-a022-05d98fd27132@github.com> Message-ID: On Fri, 16 Dec 2022 13:36:04 GMT, Erik Joelsson wrote: >> If we found one approved, we will reply some messages to user and return. What do you think should we do if there is another that is withdrawn? I don't think we need to take any action. > > Say there are two CSRs present already, one approved and one open. If the approved one is found first, then we get the needed marker but no label. If the open one is found first, then we also get the label. This is inconsistent behavior. > > What I think should happen is. > > If there are already CSRs present, then > 1. if all are approved, just add the needed marker, message could list them and say they are already approved. > 2. If at least one is open, add both needed marker and label, message should list it and say it needs to be approved. Could also list the approved ones. > 3. If all are withdrawn, add both needed marker and label, message doesn't need to list them. > > If there are no CSRs present, then > 4. Add both needed marker and label > > To achieve this you will have to always iterate over all of them and set booleans for each condition, then decide the correct action based on the complete state. Yes, you are right. I will fix it. ------------- PR: https://git.openjdk.org/skara/pull/1443 From zsong at openjdk.org Fri Dec 16 19:03:29 2022 From: zsong at openjdk.org (Zhao Song) Date: Fri, 16 Dec 2022 19:03:29 GMT Subject: RFR: 1714: CSR detection only happens on the title bug of a PR [v4] In-Reply-To: <_q1X42GM_kNlhHEYoSYfZr5D3w_puyzAER5DR7RX0sM=.c2a9fbea-99b4-426b-93c5-9f590180c1db@github.com> References: <_q1X42GM_kNlhHEYoSYfZr5D3w_puyzAER5DR7RX0sM=.c2a9fbea-99b4-426b-93c5-9f590180c1db@github.com> Message-ID: > Currently, our bots only detect CSR issues that are directly linked to the main issue. > > With this patch, our bots will be able to detect CSR issues that are linked to all of the issues that this PR solves. Additionally, all detected CSR issues will affect the integration blockers. > > The `/csr needed` command will allow you to create a CSR issue that is linked to any of the issues solved by this PR. The `/csr unneeded` command will be blocked if any of the issues associated with this PR have an active CSR issue. Zhao Song has updated the pull request incrementally with one additional commit since the last revision: fix a problem ------------- Changes: - all: https://git.openjdk.org/skara/pull/1443/files - new: https://git.openjdk.org/skara/pull/1443/files/989937b2..5966f674 Webrevs: - full: https://webrevs.openjdk.org/?repo=skara&pr=1443&range=03 - incr: https://webrevs.openjdk.org/?repo=skara&pr=1443&range=02-03 Stats: 144 lines in 2 files changed: 130 ins; 4 del; 10 mod Patch: https://git.openjdk.org/skara/pull/1443.diff Fetch: git fetch https://git.openjdk.org/skara pull/1443/head:pull/1443 PR: https://git.openjdk.org/skara/pull/1443 From zsong at openjdk.org Fri Dec 16 19:08:40 2022 From: zsong at openjdk.org (Zhao Song) Date: Fri, 16 Dec 2022 19:08:40 GMT Subject: RFR: 1714: CSR detection only happens on the title bug of a PR [v5] In-Reply-To: <_q1X42GM_kNlhHEYoSYfZr5D3w_puyzAER5DR7RX0sM=.c2a9fbea-99b4-426b-93c5-9f590180c1db@github.com> References: <_q1X42GM_kNlhHEYoSYfZr5D3w_puyzAER5DR7RX0sM=.c2a9fbea-99b4-426b-93c5-9f590180c1db@github.com> Message-ID: > Currently, our bots only detect CSR issues that are directly linked to the main issue. > > With this patch, our bots will be able to detect CSR issues that are linked to all of the issues that this PR solves. Additionally, all detected CSR issues will affect the integration blockers. > > The `/csr needed` command will allow you to create a CSR issue that is linked to any of the issues solved by this PR. The `/csr unneeded` command will be blocked if any of the issues associated with this PR have an active CSR issue. Zhao Song has updated the pull request with a new target base due to a merge or a rebase. The incremental webrev excludes the unrelated changes brought in by the merge/rebase. The pull request contains five additional commits since the last revision: - Merge branch 'master' into SKARA-1714 - fix a problem - fix a problem - fix problems - SKARA-1714 ------------- Changes: - all: https://git.openjdk.org/skara/pull/1443/files - new: https://git.openjdk.org/skara/pull/1443/files/5966f674..2280b0a1 Webrevs: - full: https://webrevs.openjdk.org/?repo=skara&pr=1443&range=04 - incr: https://webrevs.openjdk.org/?repo=skara&pr=1443&range=03-04 Stats: 1383 lines in 383 files changed: 954 ins; 17 del; 412 mod Patch: https://git.openjdk.org/skara/pull/1443.diff Fetch: git fetch https://git.openjdk.org/skara pull/1443/head:pull/1443 PR: https://git.openjdk.org/skara/pull/1443 From zsong at openjdk.org Fri Dec 16 21:10:52 2022 From: zsong at openjdk.org (Zhao Song) Date: Fri, 16 Dec 2022 21:10:52 GMT Subject: RFR: 1715: Configure custom default security level for new backport issues in notifier In-Reply-To: References: Message-ID: On Thu, 15 Dec 2022 22:12:29 GMT, Erik Joelsson wrote: > This patch makes it possible to configure a default security level to use for any newly created backport in the IssueNotifier. The configuration is done per repository and branch, with a regex for matching branch names. This is what it would look like in the bot configuration. > > > "issue": { > "project": "bugs/JDK", > "fixversions": {}, > "buildname": "master", > "headversion": true, > "defaultsecurity": { // <- new > "jdk.*": "10000", > } > } > > > If the parent bug already has a security level, then that is inherited, just like today, otherwise this new default value is used. LGTM! The test is very clear and well-organized. ------------- Marked as reviewed by zsong (Committer). PR: https://git.openjdk.org/skara/pull/1445 From erikj at openjdk.org Tue Dec 20 21:19:20 2022 From: erikj at openjdk.org (Erik Joelsson) Date: Tue, 20 Dec 2022 21:19:20 GMT Subject: RFR: 1699: CommitCommentsWorkItem does excessive unnecessary work Message-ID: The `CommitCommentsWorkItem` gets scheduled once for each repository by `PullRequestBot::getPeriodicItems`. It's responsible for querying the repository for new comments on commits, and if any are found, spawn `CommitCommandWorkItem` for them. It's unfortunately not very straight forward to query forges for new commit comments. For GitHub we have a rather complicated GraphQL that does the trick, but for GitLab, we need to jump through some rather nasty hoops. The main issue is that the data we get from the "events" query from GitLab contains notes posted to commits, but without the commit hash. We only get the commit "title" to identify which commit was commented on. To solve this, we have a rather elaborate logic that first builds a complete map from title to set of hashes for every commit in the repo. This is done preemptively on a local clone of the repo. Using this map and some clever comparisons, we can figure out which commit each comment belongs to. I don't have data on how long it takes to build this map for a big repository, like the JDK, but it's likely not trivially short. In addition to this, before using any local clone of a repository, we always run the git fsck check (see [SKARA-1598](https://bugs.openjdk.org/browse/SKARA-1598)), which can take around 20s for the JDK repo. The local clone is unfortunately also used to check if the commits for the found comments belong to a valid branch. However, both of these operations are read only, so we shouldn't really need a clone for this. We should be fine just using the seed repository directly from the `HostedRepositoryPool`. This patch tries to make this situation better in several ways. 1. Use the seed repository instead of a clone in `CommitCommentsWorkItem`. (Removes ~20s fsck time every time this is scheduled for a jdk repository, which is in the order of once a minute times the numer of JDK repo clones that Skara operates on) 2. Refactored `HostedRepository::recentCommitComments` to take a `ReadOnlyRepository` instead of the pre calculated map. `GitLabRepository` is then responsible for using this repository to build the map itself. 3. `HostedRepository::recentCommitComments` now takes an `updatedAfter` arg to limit the number of comments that get returned. In `GitLabRepository` this was already limited to last 4 days, but in `GitHubRepository`, we just got the last 100 comments, regardless of age (due to a limitation in GraphQL). We can still filter the returned comments on updatedAt so that we get less comments to process in many cases. This should make a big impact on bot restarts. 4. `CommitCommentsWorkItem` has a configuration option to ignore comments from certain users (typically meant for bot users, e.g. the notifier who posts a comment on every commit). This configuration wasn't working due to a bug in the factory. (Together with 3, should bring down the number of scheduled `CommitCommandWorkItem` on bot restart from 100 per repo to a handful and often 0.) 5. `GitLabRepository` keeps a cached copy of the commitTitleToHash map. It will also only build or update the map if any new notes are actually found. (This won't have a huge impact, but should speed things up a little bit more) ------------- Commit messages: - Cache titleToCommit map between calls - SKARA-1699 Changes: https://git.openjdk.org/skara/pull/1446/files Webrev: https://webrevs.openjdk.org/?repo=skara&pr=1446&range=00 Issue: https://bugs.openjdk.org/browse/SKARA-1699 Stats: 117 lines in 7 files changed: 77 ins; 21 del; 19 mod Patch: https://git.openjdk.org/skara/pull/1446.diff Fetch: git fetch https://git.openjdk.org/skara pull/1446/head:pull/1446 PR: https://git.openjdk.org/skara/pull/1446 From erikj at openjdk.org Tue Dec 20 22:12:40 2022 From: erikj at openjdk.org (Erik Joelsson) Date: Tue, 20 Dec 2022 22:12:40 GMT Subject: RFR: 1714: CSR detection only happens on the title bug of a PR [v5] In-Reply-To: References: <_q1X42GM_kNlhHEYoSYfZr5D3w_puyzAER5DR7RX0sM=.c2a9fbea-99b4-426b-93c5-9f590180c1db@github.com> Message-ID: On Fri, 16 Dec 2022 19:08:40 GMT, Zhao Song wrote: >> Currently, our bots only detect CSR issues that are directly linked to the main issue. >> >> With this patch, our bots will be able to detect CSR issues that are linked to all of the issues that this PR solves. Additionally, all detected CSR issues will affect the integration blockers. >> >> The `/csr needed` command will allow you to create a CSR issue that is linked to any of the issues solved by this PR. The `/csr unneeded` command will be blocked if any of the issues associated with this PR have an active CSR issue. > > Zhao Song has updated the pull request with a new target base due to a merge or a rebase. The incremental webrev excludes the unrelated changes brought in by the merge/rebase. The pull request contains five additional commits since the last revision: > > - Merge branch 'master' into SKARA-1714 > - fix a problem > - fix a problem > - fix problems > - SKARA-1714 I think this is ok now, but would be good if you could try it in staging on the playground repo with bugs-stage and go through some scenarios. ------------- Marked as reviewed by erikj (Lead). PR: https://git.openjdk.org/skara/pull/1443 From zsong at openjdk.org Tue Dec 20 22:20:59 2022 From: zsong at openjdk.org (Zhao Song) Date: Tue, 20 Dec 2022 22:20:59 GMT Subject: RFR: 1714: CSR detection only happens on the title bug of a PR [v5] In-Reply-To: References: <_q1X42GM_kNlhHEYoSYfZr5D3w_puyzAER5DR7RX0sM=.c2a9fbea-99b4-426b-93c5-9f590180c1db@github.com> Message-ID: <5j1Z197UkzRjBekLsKzTHSAtP_--HmkBF-SWoEtoKjQ=.85cdbf1a-45c4-4b5b-a2f7-4894bab56efd@github.com> On Tue, 20 Dec 2022 22:10:31 GMT, Erik Joelsson wrote: > I think this is ok now, but would be good if you could try it in staging on the playground repo with bugs-stage and go through some scenarios. Thanks for the review! Sure, I would do some tests in staging later. ------------- PR: https://git.openjdk.org/skara/pull/1443 From zsong at openjdk.org Wed Dec 21 00:40:19 2022 From: zsong at openjdk.org (Zhao Song) Date: Wed, 21 Dec 2022 00:40:19 GMT Subject: RFR: 1699: CommitCommentsWorkItem does excessive unnecessary work In-Reply-To: References: Message-ID: On Tue, 20 Dec 2022 21:14:41 GMT, Erik Joelsson wrote: > The `CommitCommentsWorkItem` gets scheduled once for each repository by `PullRequestBot::getPeriodicItems`. It's responsible for querying the repository for new comments on commits, and if any are found, spawn `CommitCommandWorkItem` for them. It's unfortunately not very straight forward to query forges for new commit comments. For GitHub we have a rather complicated GraphQL that does the trick, but for GitLab, we need to jump through some rather nasty hoops. > > The main issue is that the data we get from the "events" query from GitLab contains notes posted to commits, but without the commit hash. We only get the commit "title" to identify which commit was commented on. To solve this, we have a rather elaborate logic that first builds a complete map from title to set of hashes for every commit in the repo. This is done preemptively on a local clone of the repo. Using this map and some clever comparisons, we can figure out which commit each comment belongs to. > > I don't have data on how long it takes to build this map for a big repository, like the JDK, but it's likely not trivially short. In addition to this, before using any local clone of a repository, we always run the git fsck check (see [SKARA-1598](https://bugs.openjdk.org/browse/SKARA-1598)), which can take around 20s for the JDK repo. The local clone is unfortunately also used to check if the commits for the found comments belong to a valid branch. However, both of these operations are read only, so we shouldn't really need a clone for this. We should be fine just using the seed repository directly from the `HostedRepositoryPool`. > > This patch tries to make this situation better in several ways. > > 1. Use the seed repository instead of a clone in `CommitCommentsWorkItem`. (Removes ~20s fsck time every time this is scheduled for a jdk repository, which is in the order of once a minute times the numer of JDK repo clones that Skara operates on) > 2. Refactored `HostedRepository::recentCommitComments` to take a `ReadOnlyRepository` instead of the pre calculated map. `GitLabRepository` is then responsible for using this repository to build the map itself. > 3. `HostedRepository::recentCommitComments` now takes an `updatedAfter` arg to limit the number of comments that get returned. In `GitLabRepository` this was already limited to last 4 days, but in `GitHubRepository`, we just got the last 100 comments, regardless of age (due to a limitation in GraphQL). We can still filter the returned comments on updatedAt so that we get less comments to process in many cases. This should make a big impact on bot restarts. > 4. `CommitCommentsWorkItem` has a configuration option to ignore comments from certain users (typically meant for bot users, e.g. the notifier who posts a comment on every commit). This configuration wasn't working due to a bug in the factory. (Together with 3, should bring down the number of scheduled `CommitCommandWorkItem` on bot restart from 100 per repo to a handful and often 0.) > 5. `GitLabRepository` keeps a cached copy of the commitTitleToHash map. It will also only build or update the map if any new notes are actually found. (This won't have a huge impact, but should speed things up a little bit more) LGTM! I believe this patch will bring us some performance improvements. I want to verify my understanding of the` HostedRepositoryPool#materialize` and `HostedRepositoryPool#seedRepository` methods. As I understand it, the first method creates a new directory separate from the seed directory and clones the hosted repository into it, allowing read and write operations to be performed on the cloned repository. The second method simply returns the updated seed repository, which does not allow write operations and also does not require a health check before being used. ------------- Marked as reviewed by zsong (Committer). PR: https://git.openjdk.org/skara/pull/1446 From erikj at openjdk.org Wed Dec 21 14:07:25 2022 From: erikj at openjdk.org (Erik Joelsson) Date: Wed, 21 Dec 2022 14:07:25 GMT Subject: RFR: 1699: CommitCommentsWorkItem does excessive unnecessary work In-Reply-To: References: Message-ID: <145Yg9gTASHBM5mFhClhGnBrCQKcP8zVpQvKvVtpsDE=.c93aaa04-5d91-46a7-ba1d-1cff30c84198@github.com> On Wed, 21 Dec 2022 00:36:20 GMT, Zhao Song wrote: > I want to verify my understanding of the` HostedRepositoryPool#materialize` and `HostedRepositoryPool#seedRepository` methods. As I understand it, the first method creates a new directory separate from the seed directory and clones the hosted repository into it, allowing read and write operations to be performed on the cloned repository. The second method simply returns the updated seed repository, which does not allow write operations and also does not require a health check before being used. That is how I see it. However, the `seedRepository` method doesn't return a `ReadOnlyRepository` (which would have been nicer), and I couldn't make it do so, because the pr branch notifier is currently using the seed repository for creating the pr branch. I don't think this is good behavior. That bot should probably materialize a clone instead. One could also argue that we really shouldn't be touching the seed repo at all. Maybe we could have two levels of materialize, something like `materializeReadOnly` that would return a `ReadOnlyRepository` and that method wouldn't need to perform as much health checking. But for now, I think this is a reasonable change for this bot. ------------- PR: https://git.openjdk.org/skara/pull/1446 From zsong at openjdk.org Wed Dec 21 18:04:31 2022 From: zsong at openjdk.org (Zhao Song) Date: Wed, 21 Dec 2022 18:04:31 GMT Subject: RFR: 1699: CommitCommentsWorkItem does excessive unnecessary work In-Reply-To: <145Yg9gTASHBM5mFhClhGnBrCQKcP8zVpQvKvVtpsDE=.c93aaa04-5d91-46a7-ba1d-1cff30c84198@github.com> References: <145Yg9gTASHBM5mFhClhGnBrCQKcP8zVpQvKvVtpsDE=.c93aaa04-5d91-46a7-ba1d-1cff30c84198@github.com> Message-ID: On Wed, 21 Dec 2022 14:05:18 GMT, Erik Joelsson wrote: > > I want to verify my understanding of the` HostedRepositoryPool#materialize` and `HostedRepositoryPool#seedRepository` methods. As I understand it, the first method creates a new directory separate from the seed directory and clones the hosted repository into it, allowing read and write operations to be performed on the cloned repository. The second method simply returns the updated seed repository, which does not allow write operations and also does not require a health check before being used. > > That is how I see it. However, the `seedRepository` method doesn't return a `ReadOnlyRepository` (which would have been nicer), and I couldn't make it do so, because the pr branch notifier is currently using the seed repository for creating the pr branch. I don't think this is good behavior. That bot should probably materialize a clone instead. > > One could also argue that we really shouldn't be touching the seed repo at all. Maybe we could have two levels of materialize, something like `materializeReadOnly` that would return a `ReadOnlyRepository` and that method wouldn't need to perform as much health checking. > > But for now, I think this is a reasonable change for this bot. Agree with you! ------------- PR: https://git.openjdk.org/skara/pull/1446 From zsong at openjdk.org Wed Dec 21 18:59:11 2022 From: zsong at openjdk.org (Zhao Song) Date: Wed, 21 Dec 2022 18:59:11 GMT Subject: Integrated: 1714: CSR detection only happens on the title bug of a PR In-Reply-To: <_q1X42GM_kNlhHEYoSYfZr5D3w_puyzAER5DR7RX0sM=.c2a9fbea-99b4-426b-93c5-9f590180c1db@github.com> References: <_q1X42GM_kNlhHEYoSYfZr5D3w_puyzAER5DR7RX0sM=.c2a9fbea-99b4-426b-93c5-9f590180c1db@github.com> Message-ID: On Wed, 14 Dec 2022 01:08:08 GMT, Zhao Song wrote: > Currently, our bots only detect CSR issues that are directly linked to the main issue. > > With this patch, our bots will be able to detect CSR issues that are linked to all of the issues that this PR solves. Additionally, all detected CSR issues will affect the integration blockers. > > The `/csr needed` command will allow you to create a CSR issue that is linked to any of the issues solved by this PR. The `/csr unneeded` command will be blocked if any of the issues associated with this PR have an active CSR issue. This pull request has now been integrated. Changeset: 98f997da Author: Zhao Song URL: https://git.openjdk.org/skara/commit/98f997dafdbba591e6ffbb7f75425f113e73734a Stats: 844 lines in 11 files changed: 552 ins; 129 del; 163 mod 1714: CSR detection only happens on the title bug of a PR Reviewed-by: erikj ------------- PR: https://git.openjdk.org/skara/pull/1443 From zsong at openjdk.org Thu Dec 22 01:09:24 2022 From: zsong at openjdk.org (Zhao Song) Date: Thu, 22 Dec 2022 01:09:24 GMT Subject: RFR: 1726: PROGRESS_MARKER should be added before the CSR bot add CSR_UPDATE_MARKER Message-ID: While testing [SKARA-1714](https://bugs.openjdk.org/browse/SKARA-1714) in this pull request (https://github.com/openjdk/playground/pull/112), we discovered that if a CSR issue already exists for a main issue and a pull request is then created for the main issue, the CSR and PR bots become stuck in an endless loop and are unable to update the pull request body. After further investigation, we determined that the cause of this issue is that the CSR bot checks for the presence of the CSR_UPDATE_MARKER in the pull request body before adding it, but first checks for the PROGRESS_MARKER. However, when the CSR bot's pullRequestWorkItem runs for the first time, it is likely that the PR bot's CheckWorkItem has not yet run, resulting in the absence of the PROGRESS_MARKER in the pull request body. In this case, the CSR bot continues to add the CSR_UPDATE_MARKER, while the PR bot attempts to update the pull request body but finds that it has been recently updated and therefore gives up on the update. The correct fix is to only add CSR_UPDATE_MARKER if we find a PROGRESS_MARKER. ------------- Commit messages: - fix test - SKARA-1726 Changes: https://git.openjdk.org/skara/pull/1447/files Webrev: https://webrevs.openjdk.org/?repo=skara&pr=1447&range=00 Issue: https://bugs.openjdk.org/browse/SKARA-1726 Stats: 5 lines in 2 files changed: 2 ins; 0 del; 3 mod Patch: https://git.openjdk.org/skara/pull/1447.diff Fetch: git fetch https://git.openjdk.org/skara pull/1447/head:pull/1447 PR: https://git.openjdk.org/skara/pull/1447 From jwaters at openjdk.org Thu Dec 22 06:35:28 2022 From: jwaters at openjdk.org (Julian Waters) Date: Thu, 22 Dec 2022 06:35:28 GMT Subject: RFR: 1663: Mark integrated Pull Requests as properly merged in their repositories [v40] In-Reply-To: References: Message-ID: On Sat, 26 Nov 2022 16:10:58 GMT, Julian Waters wrote: >> Skara currently closes integrated Pull Requests, since the actual integration is done internally and then pushed to the repository separately. This makes every integrated request always look like it was closed to an outside observer, forcing the use of a special integrated flag to distinguish between integrated and closed, or rejected and closed. On top of that, it just looks awful in the interface all around (and is also not included in the accepted Pull Request count by GitHub for the committer, which is frustrating for newer contributors to a surprising extent). In the [willful absence of a reply from the GitHub team to allow marking a pull request as merged through a flag](https://github.com/orgs/community/discussions/12437), and [GitLab being unlikely to make a similar change either](https://gitlab.com/gitlab-org/gitlab/-/issues/16701), we can look at implementing this as an integration feature in Skara itself instead. >> >> Every pull has a corresponding pre-integration branch created at the time it was made, and this special branch is marked for deletion when the pull is integrated, and initially it was thought to merge the Pull Request into these corresponding pr/ branches just before their deletion but that proved to present too many challenges to be viable. Instead a related approach of enhancing the Pull Request bots with their own special integration branches can be taken, as well as providing a flexible utility method for extracting the actual branch the Pull Request was integrated into, to address the concern of losing information on what the actual Pull Request target branch was. > > Julian Waters has updated the pull request incrementally with one additional commit since the last revision: > > Just stick to the old method signature An early look at how integrated Pull Requests look like after this change: https://github.com/Elshout/Sandbox/pull/2 I will definitely need more time to work on this to make it run better though, currently it's a bit of a mess ------------- PR: https://git.openjdk.org/skara/pull/1409 From jwaters at openjdk.org Thu Dec 22 08:23:14 2022 From: jwaters at openjdk.org (Julian Waters) Date: Thu, 22 Dec 2022 08:23:14 GMT Subject: RFR: 1663: Mark integrated Pull Requests as properly merged in their repositories [v41] In-Reply-To: References: Message-ID: <8DX3djuRmYyymzAwYfZXLX8SixmgyVS-xcpwimK_vug=.74b4a8be-651c-43c1-b8d4-22e5810a3ec0@github.com> > Skara currently closes integrated Pull Requests, since the actual integration is done internally and then pushed to the repository separately. This makes every integrated request always look like it was closed to an outside observer, forcing the use of a special integrated flag to distinguish between integrated and closed, or rejected and closed. On top of that, it just looks awful in the interface all around (and is also not included in the accepted Pull Request count by GitHub for the committer, which is frustrating for newer contributors to a surprising extent). In the [willful absence of a reply from the GitHub team to allow marking a pull request as merged through a flag](https://github.com/orgs/community/discussions/12437), and [GitLab being unlikely to make a similar change either](https://gitlab.com/gitlab-org/gitlab/-/issues/16701), we can look at implementing this as an integration feature in Skara itself instead. > > Every pull has a corresponding pre-integration branch created at the time it was made, and this special branch is marked for deletion when the pull is integrated, and initially it was thought to merge the Pull Request into these corresponding pr/ branches just before their deletion but that proved to present too many challenges to be viable. Instead a related approach of enhancing the Pull Request bots with their own special integration branches can be taken, as well as providing a flexible utility method for extracting the actual branch the Pull Request was integrated into, to address the concern of losing information on what the actual Pull Request target branch was. Julian Waters has updated the pull request with a new target base due to a merge or a rebase. The pull request now contains 45 commits: - Merge remote-tracking branch 'upstream/master' into integration - Special Case in targetRef() - Just stick to the old method signature - Leave merging functionality for another enhancement - Merge remote-tracking branch 'upstream/master' into integration - Cleanup - GitLab should mirror the GitHub status getter - Field merged_at still exists even in unmerged Pull Requests - Minor commit comment change - Revert to more general repository - ... and 35 more: https://git.openjdk.org/skara/compare/98f997da...f8c8d446 ------------- Changes: https://git.openjdk.org/skara/pull/1409/files Webrev: https://webrevs.openjdk.org/?repo=skara&pr=1409&range=40 Stats: 143 lines in 6 files changed: 88 ins; 20 del; 35 mod Patch: https://git.openjdk.org/skara/pull/1409.diff Fetch: git fetch https://git.openjdk.org/skara pull/1409/head:pull/1409 PR: https://git.openjdk.org/skara/pull/1409 From jwaters at openjdk.org Thu Dec 22 08:28:53 2022 From: jwaters at openjdk.org (Julian Waters) Date: Thu, 22 Dec 2022 08:28:53 GMT Subject: RFR: 1663: Mark integrated Pull Requests as properly merged in their repositories [v42] In-Reply-To: References: Message-ID: > Skara currently closes integrated Pull Requests, since the actual integration is done internally and then pushed to the repository separately. This makes every integrated request always look like it was closed to an outside observer, forcing the use of a special integrated flag to distinguish between integrated and closed, or rejected and closed. On top of that, it just looks awful in the interface all around (and is also not included in the accepted Pull Request count by GitHub for the committer, which is frustrating for newer contributors to a surprising extent). In the [willful absence of a reply from the GitHub team to allow marking a pull request as merged through a flag](https://github.com/orgs/community/discussions/12437), and [GitLab being unlikely to make a similar change either](https://gitlab.com/gitlab-org/gitlab/-/issues/16701), we can look at implementing this as an integration feature in Skara itself instead. > > Every pull has a corresponding pre-integration branch created at the time it was made, and this special branch is marked for deletion when the pull is integrated, and initially it was thought to merge the Pull Request into these corresponding pr/ branches just before their deletion but that proved to present too many challenges to be viable. Instead a related approach of enhancing the Pull Request bots with their own special integration branches can be taken, as well as providing a flexible utility method for extracting the actual branch the Pull Request was integrated into, to address the concern of losing information on what the actual Pull Request target branch was. Julian Waters has updated the pull request incrementally with one additional commit since the last revision: Whitespace ------------- Changes: - all: https://git.openjdk.org/skara/pull/1409/files - new: https://git.openjdk.org/skara/pull/1409/files/f8c8d446..cd02b297 Webrevs: - full: https://webrevs.openjdk.org/?repo=skara&pr=1409&range=41 - incr: https://webrevs.openjdk.org/?repo=skara&pr=1409&range=40-41 Stats: 3 lines in 1 file changed: 0 ins; 0 del; 3 mod Patch: https://git.openjdk.org/skara/pull/1409.diff Fetch: git fetch https://git.openjdk.org/skara pull/1409/head:pull/1409 PR: https://git.openjdk.org/skara/pull/1409 From jwaters at openjdk.org Thu Dec 22 09:13:13 2022 From: jwaters at openjdk.org (Julian Waters) Date: Thu, 22 Dec 2022 09:13:13 GMT Subject: RFR: 1663: Mark integrated Pull Requests as properly merged in their repositories [v43] In-Reply-To: References: Message-ID: <_3RkBfiAmgFiRpdiOaVjYTP8_5NZBat4SsUQmx0Dbv4=.4e5e8bd4-ddbe-44b1-aa1d-033fcd607224@github.com> > Skara currently closes integrated Pull Requests, since the actual integration is done internally and then pushed to the repository separately. This makes every integrated request always look like it was closed to an outside observer, forcing the use of a special integrated flag to distinguish between integrated and closed, or rejected and closed. On top of that, it just looks awful in the interface all around (and is also not included in the accepted Pull Request count by GitHub for the committer, which is frustrating for newer contributors to a surprising extent). In the [willful absence of a reply from the GitHub team to allow marking a pull request as merged through a flag](https://github.com/orgs/community/discussions/12437), and [GitLab being unlikely to make a similar change either](https://gitlab.com/gitlab-org/gitlab/-/issues/16701), we can look at implementing this as an integration feature in Skara itself instead. > > Every pull has a corresponding pre-integration branch created at the time it was made, and this special branch is marked for deletion when the pull is integrated, and initially it was thought to merge the Pull Request into these corresponding pr/ branches just before their deletion but that proved to present too many challenges to be viable. Instead a related approach of enhancing the Pull Request bots with their own special integration branches can be taken, as well as providing a flexible utility method for extracting the actual branch the Pull Request was integrated into, to address the concern of losing information on what the actual Pull Request target branch was. Julian Waters has updated the pull request incrementally with one additional commit since the last revision: Message Change ------------- Changes: - all: https://git.openjdk.org/skara/pull/1409/files - new: https://git.openjdk.org/skara/pull/1409/files/cd02b297..307f02d7 Webrevs: - full: https://webrevs.openjdk.org/?repo=skara&pr=1409&range=42 - incr: https://webrevs.openjdk.org/?repo=skara&pr=1409&range=41-42 Stats: 2 lines in 1 file changed: 0 ins; 0 del; 2 mod Patch: https://git.openjdk.org/skara/pull/1409.diff Fetch: git fetch https://git.openjdk.org/skara pull/1409/head:pull/1409 PR: https://git.openjdk.org/skara/pull/1409 From jwaters at openjdk.org Thu Dec 22 09:21:34 2022 From: jwaters at openjdk.org (Julian Waters) Date: Thu, 22 Dec 2022 09:21:34 GMT Subject: RFR: 1663: Mark integrated Pull Requests as properly merged in their repositories [v43] In-Reply-To: <_3RkBfiAmgFiRpdiOaVjYTP8_5NZBat4SsUQmx0Dbv4=.4e5e8bd4-ddbe-44b1-aa1d-033fcd607224@github.com> References: <_3RkBfiAmgFiRpdiOaVjYTP8_5NZBat4SsUQmx0Dbv4=.4e5e8bd4-ddbe-44b1-aa1d-033fcd607224@github.com> Message-ID: On Thu, 22 Dec 2022 09:13:13 GMT, Julian Waters wrote: >> Skara currently closes integrated Pull Requests, since the actual integration is done internally and then pushed to the repository separately. This makes every integrated request always look like it was closed to an outside observer, forcing the use of a special integrated flag to distinguish between integrated and closed, or rejected and closed. On top of that, it just looks awful in the interface all around (and is also not included in the accepted Pull Request count by GitHub for the committer, which is frustrating for newer contributors to a surprising extent). In the [willful absence of a reply from the GitHub team to allow marking a pull request as merged through a flag](https://github.com/orgs/community/discussions/12437), and [GitLab being unlikely to make a similar change either](https://gitlab.com/gitlab-org/gitlab/-/issues/16701), we can look at implementing this as an integration feature in Skara itself instead. >> >> Every pull has a corresponding pre-integration branch created at the time it was made, and this special branch is marked for deletion when the pull is integrated, and initially it was thought to merge the Pull Request into these corresponding pr/ branches just before their deletion but that proved to present too many challenges to be viable. Instead a related approach of enhancing the Pull Request bots with their own special integration branches can be taken, as well as providing a flexible utility method for extracting the actual branch the Pull Request was integrated into, to address the concern of losing information on what the actual Pull Request target branch was. > > Julian Waters has updated the pull request incrementally with one additional commit since the last revision: > > Message Change Improved Version: https://github.com/Elshout/Sandbox/pull/1 ------------- PR: https://git.openjdk.org/skara/pull/1409 From jwaters at openjdk.org Thu Dec 22 11:34:00 2022 From: jwaters at openjdk.org (Julian Waters) Date: Thu, 22 Dec 2022 11:34:00 GMT Subject: RFR: 1663: Mark integrated Pull Requests as properly merged in their repositories [v44] In-Reply-To: References: Message-ID: > Skara currently closes integrated Pull Requests, since the actual integration is done internally and then pushed to the repository separately. This makes every integrated request always look like it was closed to an outside observer, forcing the use of a special integrated flag to distinguish between integrated and closed, or rejected and closed. On top of that, it just looks awful in the interface all around (and is also not included in the accepted Pull Request count by GitHub for the committer, which is frustrating for newer contributors to a surprising extent). In the [willful absence of a reply from the GitHub team to allow marking a pull request as merged through a flag](https://github.com/orgs/community/discussions/12437), and [GitLab being unlikely to make a similar change either](https://gitlab.com/gitlab-org/gitlab/-/issues/16701), we can look at implementing this as an integration feature in Skara itself instead. > > Every pull has a corresponding pre-integration branch created at the time it was made, and this special branch is marked for deletion when the pull is integrated, and initially it was thought to merge the Pull Request into these corresponding pr/ branches just before their deletion but that proved to present too many challenges to be viable. Instead a related approach of enhancing the Pull Request bots with their own special integration branches can be taken, as well as providing a flexible utility method for extracting the actual branch the Pull Request was integrated into, to address the concern of losing information on what the actual Pull Request target branch was. Julian Waters has updated the pull request incrementally with two additional commits since the last revision: - Whitespace - Make integration marker checking more robust ------------- Changes: - all: https://git.openjdk.org/skara/pull/1409/files - new: https://git.openjdk.org/skara/pull/1409/files/307f02d7..dd8b110c Webrevs: - full: https://webrevs.openjdk.org/?repo=skara&pr=1409&range=43 - incr: https://webrevs.openjdk.org/?repo=skara&pr=1409&range=42-43 Stats: 11 lines in 3 files changed: 2 ins; 0 del; 9 mod Patch: https://git.openjdk.org/skara/pull/1409.diff Fetch: git fetch https://git.openjdk.org/skara pull/1409/head:pull/1409 PR: https://git.openjdk.org/skara/pull/1409 From erikj at openjdk.org Thu Dec 22 13:57:21 2022 From: erikj at openjdk.org (Erik Joelsson) Date: Thu, 22 Dec 2022 13:57:21 GMT Subject: RFR: 1726: PROGRESS_MARKER should be added before the CSR bot add CSR_UPDATE_MARKER In-Reply-To: References: Message-ID: <-OvQCh-ApXFLm9bUnI3Vx1ImWpnppZmNHPhBn58PjPY=.2e5a9ae5-34cd-4521-a537-063146d51cec@github.com> On Thu, 22 Dec 2022 00:05:41 GMT, Zhao Song wrote: > While testing [SKARA-1714](https://bugs.openjdk.org/browse/SKARA-1714) in this pull request (https://github.com/openjdk/playground/pull/112), we discovered that if a CSR issue already exists for a main issue and a pull request is then created for the main issue, the CSR and PR bots become stuck in an endless loop and are unable to update the pull request body. > > After further investigation, we determined that the cause of this issue is that the CSR bot checks for the presence of the CSR_UPDATE_MARKER in the pull request body before adding it, but first checks for the PROGRESS_MARKER. However, when the CSR bot's pullRequestWorkItem runs for the first time, it is likely that the PR bot's CheckWorkItem has not yet run, resulting in the absence of the PROGRESS_MARKER in the pull request body. In this case, the CSR bot continues to add the CSR_UPDATE_MARKER, while the PR bot attempts to update the pull request body but finds that it has been recently updated and therefore gives up on the update. > > The correct fix is to only add CSR_UPDATE_MARKER if we find a PROGRESS_MARKER. Marked as reviewed by erikj (Lead). ------------- PR: https://git.openjdk.org/skara/pull/1447 From zsong at openjdk.org Thu Dec 22 17:07:25 2022 From: zsong at openjdk.org (Zhao Song) Date: Thu, 22 Dec 2022 17:07:25 GMT Subject: Integrated: 1726: PROGRESS_MARKER should be added before the CSR bot add CSR_UPDATE_MARKER In-Reply-To: References: Message-ID: <-7n41gkKLbJusWip09kuwDC21VQ4V-vHRc00u6cCklY=.c4641753-9a03-4812-b51a-34f7c56b3b21@github.com> On Thu, 22 Dec 2022 00:05:41 GMT, Zhao Song wrote: > While testing [SKARA-1714](https://bugs.openjdk.org/browse/SKARA-1714) in this pull request (https://github.com/openjdk/playground/pull/112), we discovered that if a CSR issue already exists for a main issue and a pull request is then created for the main issue, the CSR and PR bots become stuck in an endless loop and are unable to update the pull request body. > > After further investigation, we determined that the cause of this issue is that the CSR bot checks for the presence of the CSR_UPDATE_MARKER in the pull request body before adding it, but first checks for the PROGRESS_MARKER. However, when the CSR bot's pullRequestWorkItem runs for the first time, it is likely that the PR bot's CheckWorkItem has not yet run, resulting in the absence of the PROGRESS_MARKER in the pull request body. In this case, the CSR bot continues to add the CSR_UPDATE_MARKER, while the PR bot attempts to update the pull request body but finds that it has been recently updated and therefore gives up on the update. > > The correct fix is to only add CSR_UPDATE_MARKER if we find a PROGRESS_MARKER. This pull request has now been integrated. Changeset: 7a6ddeec Author: Zhao Song URL: https://git.openjdk.org/skara/commit/7a6ddeec05069bbcd0cc56075613591a4b141942 Stats: 5 lines in 2 files changed: 2 ins; 0 del; 3 mod 1726: PROGRESS_MARKER should be added before the CSR bot add CSR_UPDATE_MARKER Reviewed-by: erikj ------------- PR: https://git.openjdk.org/skara/pull/1447 From jwaters at openjdk.org Thu Dec 22 18:06:26 2022 From: jwaters at openjdk.org (Julian Waters) Date: Thu, 22 Dec 2022 18:06:26 GMT Subject: RFR: Comment detailing refspec format in GitRepository#push Message-ID: Small change to document the git refspec field in GitRepository#push in a comment ------------- Commit messages: - Documentation Changes: https://git.openjdk.org/skara/pull/1448/files Webrev: https://webrevs.openjdk.org/?repo=skara&pr=1448&range=00 Stats: 7 lines in 1 file changed: 7 ins; 0 del; 0 mod Patch: https://git.openjdk.org/skara/pull/1448.diff Fetch: git fetch https://git.openjdk.org/skara pull/1448/head:pull/1448 PR: https://git.openjdk.org/skara/pull/1448 From jwaters at openjdk.org Thu Dec 22 18:21:57 2022 From: jwaters at openjdk.org (Julian Waters) Date: Thu, 22 Dec 2022 18:21:57 GMT Subject: RFR: 1663: Mark integrated Pull Requests as properly merged in their repositories [v45] In-Reply-To: References: Message-ID: > Skara currently closes integrated Pull Requests, since the actual integration is done internally and then pushed to the repository separately. This makes every integrated request always look like it was closed to an outside observer, forcing the use of a special integrated flag to distinguish between integrated and closed, or rejected and closed. On top of that, it just looks awful in the interface all around (and is also not included in the accepted Pull Request count by GitHub for the committer, which is frustrating for newer contributors to a surprising extent). In the [willful absence of a reply from the GitHub team to allow marking a pull request as merged through a flag](https://github.com/orgs/community/discussions/12437), and [GitLab being unlikely to make a similar change either](https://gitlab.com/gitlab-org/gitlab/-/issues/16701), we can look at implementing this as an integration feature in Skara itself instead. > > Every pull has a corresponding pre-integration branch created at the time it was made, and this special branch is marked for deletion when the pull is integrated, and initially it was thought to merge the Pull Request into these corresponding pr/ branches just before their deletion but that proved to present too many challenges to be viable. Instead a related approach of enhancing the Pull Request bots with their own special integration branches can be taken, as well as providing a flexible utility method for extracting the actual branch the Pull Request was integrated into, to address the concern of losing information on what the actual Pull Request target branch was. Julian Waters has updated the pull request incrementally with one additional commit since the last revision: Remove Comment ------------- Changes: - all: https://git.openjdk.org/skara/pull/1409/files - new: https://git.openjdk.org/skara/pull/1409/files/dd8b110c..211f0f16 Webrevs: - full: https://webrevs.openjdk.org/?repo=skara&pr=1409&range=44 - incr: https://webrevs.openjdk.org/?repo=skara&pr=1409&range=43-44 Stats: 7 lines in 1 file changed: 0 ins; 7 del; 0 mod Patch: https://git.openjdk.org/skara/pull/1409.diff Fetch: git fetch https://git.openjdk.org/skara pull/1409/head:pull/1409 PR: https://git.openjdk.org/skara/pull/1409 From zsong at openjdk.org Thu Dec 22 19:21:35 2022 From: zsong at openjdk.org (Zhao Song) Date: Thu, 22 Dec 2022 19:21:35 GMT Subject: RFR: 1728: Should iterate over all issues in PullRequestWorkItem in CSR bot Message-ID: While testing [SKARA-1714](https://bugs.openjdk.org/browse/SKARA-1714) on the staging environment, I discovered a bug in the PullRequestWorkItem in the CSR bot. The issue is that the for loop does not always iterate over all CSR issues for a pull request. For example, if we have two active CSR issues (csr1 and csr2) for one pull request, and we withdraw csr2, the loop will exit early after iterating over csr1 and CSR_UPDATE_MARKER will not be added, which means the PR body will not be updated and the status change of csr2 will not be displayed. Also, if we find a withdrawn pr, we need to add CSR_UPDATE_MARKER to the pr body. ------------- Commit messages: - SKARA-1728 Changes: https://git.openjdk.org/skara/pull/1449/files Webrev: https://webrevs.openjdk.org/?repo=skara&pr=1449&range=00 Issue: https://bugs.openjdk.org/browse/SKARA-1728 Stats: 34 lines in 2 files changed: 6 ins; 8 del; 20 mod Patch: https://git.openjdk.org/skara/pull/1449.diff Fetch: git fetch https://git.openjdk.org/skara pull/1449/head:pull/1449 PR: https://git.openjdk.org/skara/pull/1449 From jwaters at openjdk.org Thu Dec 22 19:31:34 2022 From: jwaters at openjdk.org (Julian Waters) Date: Thu, 22 Dec 2022 19:31:34 GMT Subject: RFR: 1663: Mark integrated Pull Requests as properly merged in their repositories [v46] In-Reply-To: References: Message-ID: > Skara currently closes integrated Pull Requests, since the actual integration is done internally and then pushed to the repository separately. This makes every integrated request always look like it was closed to an outside observer, forcing the use of a special integrated flag to distinguish between integrated and closed, or rejected and closed. On top of that, it just looks awful in the interface all around (and is also not included in the accepted Pull Request count by GitHub for the committer, which is frustrating for newer contributors to a surprising extent). In the [willful absence of a reply from the GitHub team to allow marking a pull request as merged through a flag](https://github.com/orgs/community/discussions/12437), and [GitLab being unlikely to make a similar change either](https://gitlab.com/gitlab-org/gitlab/-/issues/16701), we can look at implementing this as an integration feature in Skara itself instead. > > Every pull has a corresponding pre-integration branch created at the time it was made, and this special branch is marked for deletion when the pull is integrated, and initially it was thought to merge the Pull Request into these corresponding pr/ branches just before their deletion but that proved to present too many challenges to be viable. Instead a related approach of enhancing the Pull Request bots with their own special integration branches can be taken, as well as providing a flexible utility method for extracting the actual branch the Pull Request was integrated into, to address the concern of losing information on what the actual Pull Request target branch was. Julian Waters has updated the pull request incrementally with one additional commit since the last revision: Integration branches should always be identified by marker and not pull request state ------------- Changes: - all: https://git.openjdk.org/skara/pull/1409/files - new: https://git.openjdk.org/skara/pull/1409/files/211f0f16..4c9cb948 Webrevs: - full: https://webrevs.openjdk.org/?repo=skara&pr=1409&range=45 - incr: https://webrevs.openjdk.org/?repo=skara&pr=1409&range=44-45 Stats: 14 lines in 2 files changed: 0 ins; 10 del; 4 mod Patch: https://git.openjdk.org/skara/pull/1409.diff Fetch: git fetch https://git.openjdk.org/skara pull/1409/head:pull/1409 PR: https://git.openjdk.org/skara/pull/1409 From erikj at openjdk.org Thu Dec 22 19:36:34 2022 From: erikj at openjdk.org (Erik Joelsson) Date: Thu, 22 Dec 2022 19:36:34 GMT Subject: RFR: 1728: Should iterate over all issues in PullRequestWorkItem in CSR bot In-Reply-To: References: Message-ID: On Thu, 22 Dec 2022 19:04:34 GMT, Zhao Song wrote: > While testing [SKARA-1714](https://bugs.openjdk.org/browse/SKARA-1714) on the staging environment, I discovered a bug in the PullRequestWorkItem in the CSR bot. The issue is that the for loop does not always iterate over all CSR issues for a pull request. > > For example, if we have two active CSR issues (csr1 and csr2) for one pull request, and we withdraw csr2, the loop will exit early after iterating over csr1 and CSR_UPDATE_MARKER will not be added, which means the PR body will not be updated and the status change of csr2 will not be displayed. > > Also, if we find a withdrawn pr, we need to add CSR_UPDATE_MARKER to the pr body. bots/csr/src/test/java/org/openjdk/skara/bots/csr/CSRBotTests.java line 43: > 41: class CSRBotTests { > 42: private final static String CSR_UPDATE_MARKER = ""; > 43: private static final String PROGRESS_MARKER = ""; I didn't notice this before, but these markers shouldn't be redefined here in the test. Make them package protected or even public in `PullRequestWorkItem` instead. ------------- PR: https://git.openjdk.org/skara/pull/1449 From zsong at openjdk.org Thu Dec 22 19:49:29 2022 From: zsong at openjdk.org (Zhao Song) Date: Thu, 22 Dec 2022 19:49:29 GMT Subject: RFR: 1728: Should iterate over all issues in PullRequestWorkItem in CSR bot In-Reply-To: References: Message-ID: On Thu, 22 Dec 2022 19:34:06 GMT, Erik Joelsson wrote: >> While testing [SKARA-1714](https://bugs.openjdk.org/browse/SKARA-1714) on the staging environment, I discovered a bug in the PullRequestWorkItem in the CSR bot. The issue is that the for loop does not always iterate over all CSR issues for a pull request. >> >> For example, if we have two active CSR issues (csr1 and csr2) for one pull request, and we withdraw csr2, the loop will exit early after iterating over csr1 and CSR_UPDATE_MARKER will not be added, which means the PR body will not be updated and the status change of csr2 will not be displayed. >> >> Also, if we find a withdrawn pr, we need to add CSR_UPDATE_MARKER to the pr body. > > bots/csr/src/test/java/org/openjdk/skara/bots/csr/CSRBotTests.java line 43: > >> 41: class CSRBotTests { >> 42: private final static String CSR_UPDATE_MARKER = ""; >> 43: private static final String PROGRESS_MARKER = ""; > > I didn't notice this before, but these markers shouldn't be redefined here in the test. Make them package protected or even public in `PullRequestWorkItem` instead. > Sure, will fix it. Thx! ------------- PR: https://git.openjdk.org/skara/pull/1449 From zsong at openjdk.org Thu Dec 22 20:24:42 2022 From: zsong at openjdk.org (Zhao Song) Date: Thu, 22 Dec 2022 20:24:42 GMT Subject: RFR: 1728: Should iterate over all issues in PullRequestWorkItem in CSR bot [v2] In-Reply-To: References: Message-ID: > While testing [SKARA-1714](https://bugs.openjdk.org/browse/SKARA-1714) on the staging environment, I discovered a bug in the PullRequestWorkItem in the CSR bot. The issue is that the for loop does not always iterate over all CSR issues for a pull request. > > For example, if we have two active CSR issues (csr1 and csr2) for one pull request, and we withdraw csr2, the loop will exit early after iterating over csr1 and CSR_UPDATE_MARKER will not be added, which means the PR body will not be updated and the status change of csr2 will not be displayed. > > Also, if we find a withdrawn pr, we need to add CSR_UPDATE_MARKER to the pr body. Zhao Song has updated the pull request incrementally with one additional commit since the last revision: fix a problem ------------- Changes: - all: https://git.openjdk.org/skara/pull/1449/files - new: https://git.openjdk.org/skara/pull/1449/files/fcf90d95..58a9f6ec Webrevs: - full: https://webrevs.openjdk.org/?repo=skara&pr=1449&range=01 - incr: https://webrevs.openjdk.org/?repo=skara&pr=1449&range=00-01 Stats: 7 lines in 2 files changed: 2 ins; 3 del; 2 mod Patch: https://git.openjdk.org/skara/pull/1449.diff Fetch: git fetch https://git.openjdk.org/skara pull/1449/head:pull/1449 PR: https://git.openjdk.org/skara/pull/1449 From erikj at openjdk.org Thu Dec 22 20:47:36 2022 From: erikj at openjdk.org (Erik Joelsson) Date: Thu, 22 Dec 2022 20:47:36 GMT Subject: RFR: 1728: Should iterate over all issues in PullRequestWorkItem in CSR bot [v2] In-Reply-To: References: Message-ID: On Thu, 22 Dec 2022 20:24:42 GMT, Zhao Song wrote: >> While testing [SKARA-1714](https://bugs.openjdk.org/browse/SKARA-1714) on the staging environment, I discovered a bug in the PullRequestWorkItem in the CSR bot. The issue is that the for loop does not always iterate over all CSR issues for a pull request. >> >> For example, if we have two active CSR issues (csr1 and csr2) for one pull request, and we withdraw csr2, the loop will exit early after iterating over csr1 and CSR_UPDATE_MARKER will not be added, which means the PR body will not be updated and the status change of csr2 will not be displayed. >> >> Also, if we find a withdrawn pr, we need to add CSR_UPDATE_MARKER to the pr body. > > Zhao Song has updated the pull request incrementally with one additional commit since the last revision: > > fix a problem bots/csr/src/main/java/org/openjdk/skara/bots/csr/PullRequestWorkItem.java line 53: > 51: private final static String CSR_LABEL = "csr"; > 52: protected final static String CSR_UPDATE_MARKER = ""; > 53: protected static final String PROGRESS_MARKER = ""; Does this work? I thought you needed to use the default package protected (by removing any `private` or `protected` prefix). ------------- PR: https://git.openjdk.org/skara/pull/1449 From zsong at openjdk.org Thu Dec 22 20:53:50 2022 From: zsong at openjdk.org (Zhao Song) Date: Thu, 22 Dec 2022 20:53:50 GMT Subject: RFR: 1728: Should iterate over all issues in PullRequestWorkItem in CSR bot [v3] In-Reply-To: References: Message-ID: <7jZMg4TtKsF0Rv5axQtlXFokHZQqZ6b-h6nEBvtmkX0=.a17aa82e-d3a6-4786-8785-215eda946956@github.com> > While testing [SKARA-1714](https://bugs.openjdk.org/browse/SKARA-1714) on the staging environment, I discovered a bug in the PullRequestWorkItem in the CSR bot. The issue is that the for loop does not always iterate over all CSR issues for a pull request. > > For example, if we have two active CSR issues (csr1 and csr2) for one pull request, and we withdraw csr2, the loop will exit early after iterating over csr1 and CSR_UPDATE_MARKER will not be added, which means the PR body will not be updated and the status change of csr2 will not be displayed. > > Also, if we find a withdrawn pr, we need to add CSR_UPDATE_MARKER to the pr body. Zhao Song has updated the pull request incrementally with one additional commit since the last revision: fix a problem ------------- Changes: - all: https://git.openjdk.org/skara/pull/1449/files - new: https://git.openjdk.org/skara/pull/1449/files/58a9f6ec..6e616ace Webrevs: - full: https://webrevs.openjdk.org/?repo=skara&pr=1449&range=02 - incr: https://webrevs.openjdk.org/?repo=skara&pr=1449&range=01-02 Stats: 2 lines in 1 file changed: 0 ins; 0 del; 2 mod Patch: https://git.openjdk.org/skara/pull/1449.diff Fetch: git fetch https://git.openjdk.org/skara pull/1449/head:pull/1449 PR: https://git.openjdk.org/skara/pull/1449 From zsong at openjdk.org Thu Dec 22 20:53:52 2022 From: zsong at openjdk.org (Zhao Song) Date: Thu, 22 Dec 2022 20:53:52 GMT Subject: RFR: 1728: Should iterate over all issues in PullRequestWorkItem in CSR bot [v2] In-Reply-To: References: Message-ID: <2PJD3IME7Rm3MuDPa0u8hzP6IHgaHpiaoDW67NDeozg=.c03bbe06-6cdb-4fd2-9b92-5c79b468dd01@github.com> On Thu, 22 Dec 2022 20:45:29 GMT, Erik Joelsson wrote: >> Zhao Song has updated the pull request incrementally with one additional commit since the last revision: >> >> fix a problem > > bots/csr/src/main/java/org/openjdk/skara/bots/csr/PullRequestWorkItem.java line 53: > >> 51: private final static String CSR_LABEL = "csr"; >> 52: protected final static String CSR_UPDATE_MARKER = ""; >> 53: protected static final String PROGRESS_MARKER = ""; > > Does this work? I thought you needed to use the default package protected (by removing any `private` or `protected` prefix). Yes, this works. Ok, I will remove the prefix. ------------- PR: https://git.openjdk.org/skara/pull/1449 From erikj at openjdk.org Thu Dec 22 20:58:47 2022 From: erikj at openjdk.org (Erik Joelsson) Date: Thu, 22 Dec 2022 20:58:47 GMT Subject: RFR: 1728: Should iterate over all issues in PullRequestWorkItem in CSR bot [v3] In-Reply-To: <7jZMg4TtKsF0Rv5axQtlXFokHZQqZ6b-h6nEBvtmkX0=.a17aa82e-d3a6-4786-8785-215eda946956@github.com> References: <7jZMg4TtKsF0Rv5axQtlXFokHZQqZ6b-h6nEBvtmkX0=.a17aa82e-d3a6-4786-8785-215eda946956@github.com> Message-ID: On Thu, 22 Dec 2022 20:53:50 GMT, Zhao Song wrote: >> While testing [SKARA-1714](https://bugs.openjdk.org/browse/SKARA-1714) on the staging environment, I discovered a bug in the PullRequestWorkItem in the CSR bot. The issue is that the for loop does not always iterate over all CSR issues for a pull request. >> >> For example, if we have two active CSR issues (csr1 and csr2) for one pull request, and we withdraw csr2, the loop will exit early after iterating over csr1 and CSR_UPDATE_MARKER will not be added, which means the PR body will not be updated and the status change of csr2 will not be displayed. >> >> Also, if we find a withdrawn pr, we need to add CSR_UPDATE_MARKER to the pr body. > > Zhao Song has updated the pull request incrementally with one additional commit since the last revision: > > fix a problem Marked as reviewed by erikj (Lead). ------------- PR: https://git.openjdk.org/skara/pull/1449 From zsong at openjdk.org Thu Dec 22 21:17:12 2022 From: zsong at openjdk.org (Zhao Song) Date: Thu, 22 Dec 2022 21:17:12 GMT Subject: Integrated: 1728: Should iterate over all issues in PullRequestWorkItem in CSR bot In-Reply-To: References: Message-ID: On Thu, 22 Dec 2022 19:04:34 GMT, Zhao Song wrote: > While testing [SKARA-1714](https://bugs.openjdk.org/browse/SKARA-1714) on the staging environment, I discovered a bug in the PullRequestWorkItem in the CSR bot. The issue is that the for loop does not always iterate over all CSR issues for a pull request. > > For example, if we have two active CSR issues (csr1 and csr2) for one pull request, and we withdraw csr2, the loop will exit early after iterating over csr1 and CSR_UPDATE_MARKER will not be added, which means the PR body will not be updated and the status change of csr2 will not be displayed. > > Also, if we find a withdrawn pr, we need to add CSR_UPDATE_MARKER to the pr body. This pull request has now been integrated. Changeset: e66ba112 Author: Zhao Song URL: https://git.openjdk.org/skara/commit/e66ba11213957b2235e623d94de3137cef8824cc Stats: 35 lines in 2 files changed: 5 ins; 8 del; 22 mod 1728: Should iterate over all issues in PullRequestWorkItem in CSR bot Reviewed-by: erikj ------------- PR: https://git.openjdk.org/skara/pull/1449 From jwaters at openjdk.org Sun Dec 25 11:52:23 2022 From: jwaters at openjdk.org (Julian Waters) Date: Sun, 25 Dec 2022 11:52:23 GMT Subject: RFR: 1663: Mark integrated Pull Requests as properly merged in their repositories [v47] In-Reply-To: References: Message-ID: > Skara currently closes integrated Pull Requests, since the actual integration is done internally and then pushed to the repository separately. This makes every integrated request always look like it was closed to an outside observer, forcing the use of a special integrated flag to distinguish between integrated and closed, or rejected and closed. On top of that, it just looks awful in the interface all around (and is also not included in the accepted Pull Request count by GitHub for the committer, which is frustrating for newer contributors to a surprising extent). In the [willful absence of a reply from the GitHub team to allow marking a pull request as merged through a flag](https://github.com/orgs/community/discussions/12437), and [GitLab being unlikely to make a similar change either](https://gitlab.com/gitlab-org/gitlab/-/issues/16701), we can look at implementing this as an integration feature in Skara itself instead. > > Every pull has a corresponding pre-integration branch created at the time it was made, and this special branch is marked for deletion when the pull is integrated, and initially it was thought to merge the Pull Request into these corresponding pr/ branches just before their deletion but that proved to present too many challenges to be viable. Instead a related approach of enhancing the Pull Request bots with their own special integration branches can be taken, as well as providing a flexible utility method for extracting the actual branch the Pull Request was integrated into, to address the concern of losing information on what the actual Pull Request target branch was. > > > https://gitlab.com/gitlab-org/gitlab/-/blob/master/app/models/merge_request.rb#L172 Julian Waters has updated the pull request incrementally with one additional commit since the last revision: Different approach ------------- Changes: - all: https://git.openjdk.org/skara/pull/1409/files - new: https://git.openjdk.org/skara/pull/1409/files/4c9cb948..96c3f471 Webrevs: - full: https://webrevs.openjdk.org/?repo=skara&pr=1409&range=46 - incr: https://webrevs.openjdk.org/?repo=skara&pr=1409&range=45-46 Stats: 32 lines in 3 files changed: 16 ins; 8 del; 8 mod Patch: https://git.openjdk.org/skara/pull/1409.diff Fetch: git fetch https://git.openjdk.org/skara pull/1409/head:pull/1409 PR: https://git.openjdk.org/skara/pull/1409 From jwaters at openjdk.org Sun Dec 25 15:24:49 2022 From: jwaters at openjdk.org (Julian Waters) Date: Sun, 25 Dec 2022 15:24:49 GMT Subject: RFR: 1663: Mark integrated Pull Requests as properly merged in their repositories [v48] In-Reply-To: References: Message-ID: > Skara currently closes integrated Pull Requests, since the actual integration is done internally and then pushed to the repository separately. This makes every integrated request always look like it was closed to an outside observer, forcing the use of a special integrated flag to distinguish between integrated and closed, or rejected and closed. On top of that, it just looks awful in the interface all around (and is also not included in the accepted Pull Request count by GitHub for the committer, which is frustrating for newer contributors to a surprising extent). In the [willful absence of a reply from the GitHub team to allow marking a pull request as merged through a flag](https://github.com/orgs/community/discussions/12437), and [GitLab being unlikely to make a similar change either](https://gitlab.com/gitlab-org/gitlab/-/issues/16701), we can look at implementing this as an integration feature in Skara itself instead. > > Every pull has a corresponding pre-integration branch created at the time it was made, and this special branch is marked for deletion when the pull is integrated, and initially it was thought to merge the Pull Request into these corresponding pr/ branches just before their deletion but that proved to present too many challenges to be viable. Instead a related approach of enhancing the Pull Request bots with their own special integration branches can be taken, as well as providing a flexible utility method for extracting the actual branch the Pull Request was integrated into, to address the concern of losing information on what the actual Pull Request target branch was. > > > https://gitlab.com/gitlab-org/gitlab/-/blob/master/app/models/merge_request.rb#L172 Julian Waters has updated the pull request incrementally with one additional commit since the last revision: Logging ------------- Changes: - all: https://git.openjdk.org/skara/pull/1409/files - new: https://git.openjdk.org/skara/pull/1409/files/96c3f471..4ac3afc4 Webrevs: - full: https://webrevs.openjdk.org/?repo=skara&pr=1409&range=47 - incr: https://webrevs.openjdk.org/?repo=skara&pr=1409&range=46-47 Stats: 15 lines in 1 file changed: 1 ins; 4 del; 10 mod Patch: https://git.openjdk.org/skara/pull/1409.diff Fetch: git fetch https://git.openjdk.org/skara pull/1409/head:pull/1409 PR: https://git.openjdk.org/skara/pull/1409 From jwaters at openjdk.org Sun Dec 25 17:39:48 2022 From: jwaters at openjdk.org (Julian Waters) Date: Sun, 25 Dec 2022 17:39:48 GMT Subject: RFR: 1663: Mark integrated Pull Requests as properly merged in their repositories [v49] In-Reply-To: References: Message-ID: > Skara currently closes integrated Pull Requests, since the actual integration is done internally and then pushed to the repository separately. This makes every integrated request always look like it was closed to an outside observer, forcing the use of a special integrated flag to distinguish between integrated and closed, or rejected and closed. On top of that, it just looks awful in the interface all around (and is also not included in the accepted Pull Request count by GitHub for the committer, which is frustrating for newer contributors to a surprising extent). In the [willful absence of a reply from the GitHub team to allow marking a pull request as merged through a flag](https://github.com/orgs/community/discussions/12437), and [GitLab being unlikely to make a similar change either](https://gitlab.com/gitlab-org/gitlab/-/issues/16701), we can look at implementing this as an integration feature in Skara itself instead. > > Every pull has a corresponding pre-integration branch created at the time it was made, and this special branch is marked for deletion when the pull is integrated, and initially it was thought to merge the Pull Request into these corresponding pr/ branches just before their deletion but that proved to present too many challenges to be viable. Instead a related approach of enhancing the Pull Request bots with their own special integration branches can be taken, as well as providing a flexible utility method for extracting the actual branch the Pull Request was integrated into, to address the concern of losing information on what the actual Pull Request target branch was. > > > https://gitlab.com/gitlab-org/gitlab/-/blob/master/app/models/merge_request.rb#L172 Julian Waters has updated the pull request incrementally with one additional commit since the last revision: Checkout Prior ------------- Changes: - all: https://git.openjdk.org/skara/pull/1409/files - new: https://git.openjdk.org/skara/pull/1409/files/4ac3afc4..9a4f3881 Webrevs: - full: https://webrevs.openjdk.org/?repo=skara&pr=1409&range=48 - incr: https://webrevs.openjdk.org/?repo=skara&pr=1409&range=47-48 Stats: 8 lines in 1 file changed: 1 ins; 0 del; 7 mod Patch: https://git.openjdk.org/skara/pull/1409.diff Fetch: git fetch https://git.openjdk.org/skara pull/1409/head:pull/1409 PR: https://git.openjdk.org/skara/pull/1409 From jwaters at openjdk.org Sun Dec 25 19:39:10 2022 From: jwaters at openjdk.org (Julian Waters) Date: Sun, 25 Dec 2022 19:39:10 GMT Subject: RFR: 1663: Mark integrated Pull Requests as properly merged in their repositories [v50] In-Reply-To: References: Message-ID: > Skara currently closes integrated Pull Requests, since the actual integration is done internally and then pushed to the repository separately. This makes every integrated request always look like it was closed to an outside observer, forcing the use of a special integrated flag to distinguish between integrated and closed, or rejected and closed. On top of that, it just looks awful in the interface all around (and is also not included in the accepted Pull Request count by GitHub for the committer, which is frustrating for newer contributors to a surprising extent). In the [willful absence of a reply from the GitHub team to allow marking a pull request as merged through a flag](https://github.com/orgs/community/discussions/12437), and [GitLab being unlikely to make a similar change either](https://gitlab.com/gitlab-org/gitlab/-/issues/16701), we can look at implementing this as an integration feature in Skara itself instead. > > Every pull has a corresponding pre-integration branch created at the time it was made, and this special branch is marked for deletion when the pull is integrated, and initially it was thought to merge the Pull Request into these corresponding pr/ branches just before their deletion but that proved to present too many challenges to be viable. Instead a related approach of enhancing the Pull Request bots with their own special integration branches can be taken, as well as providing a flexible utility method for extracting the actual branch the Pull Request was integrated into, to address the concern of losing information on what the actual Pull Request target branch was. > > > https://gitlab.com/gitlab-org/gitlab/-/blob/master/app/models/merge_request.rb#L172 Julian Waters has updated the pull request incrementally with two additional commits since the last revision: - GitRepository#revert is faulty - Copy PullRequestUtils ------------- Changes: - all: https://git.openjdk.org/skara/pull/1409/files - new: https://git.openjdk.org/skara/pull/1409/files/9a4f3881..94772742 Webrevs: - full: https://webrevs.openjdk.org/?repo=skara&pr=1409&range=49 - incr: https://webrevs.openjdk.org/?repo=skara&pr=1409&range=48-49 Stats: 3 lines in 2 files changed: 1 ins; 0 del; 2 mod Patch: https://git.openjdk.org/skara/pull/1409.diff Fetch: git fetch https://git.openjdk.org/skara pull/1409/head:pull/1409 PR: https://git.openjdk.org/skara/pull/1409 From jwaters at openjdk.org Sun Dec 25 19:52:34 2022 From: jwaters at openjdk.org (Julian Waters) Date: Sun, 25 Dec 2022 19:52:34 GMT Subject: RFR: GitRepository#revert does not work as intended Message-ID: <0zkAqsBtkhRC-887Vz7beYeclV8sDu5GHebMLgKVbMk=.176419bd-927e-492e-aff2-a9f20f270466@github.com> The `Repository#revert` interface is meant to allow for rolling the Repository back to a specific commit, but the Git implementation in `GitRepository#revert` implements this via a checkout, which can cause issues, especially with newer versions of Git where it doesn't have any effect at all. More appropriate would be using `git restore --source` to implement the GitRepository variant instead, which is actually the encouraged way to do so nowadays ------------- Commit messages: - GitRepository#revert does not work as intended Changes: https://git.openjdk.org/skara/pull/1450/files Webrev: https://webrevs.openjdk.org/?repo=skara&pr=1450&range=00 Stats: 1 line in 1 file changed: 0 ins; 0 del; 1 mod Patch: https://git.openjdk.org/skara/pull/1450.diff Fetch: git fetch https://git.openjdk.org/skara pull/1450/head:pull/1450 PR: https://git.openjdk.org/skara/pull/1450 From jwaters at openjdk.org Sun Dec 25 19:54:51 2022 From: jwaters at openjdk.org (Julian Waters) Date: Sun, 25 Dec 2022 19:54:51 GMT Subject: RFR: 1663: Mark integrated Pull Requests as properly merged in their repositories [v51] In-Reply-To: References: Message-ID: <300jNG7askrCPp2173HOblF6t5AQlK5Dig22LE5zlUU=.c1406fa5-313f-4a29-b346-5caf51b04a2e@github.com> > Skara currently closes integrated Pull Requests, since the actual integration is done internally and then pushed to the repository separately. This makes every integrated request always look like it was closed to an outside observer, forcing the use of a special integrated flag to distinguish between integrated and closed, or rejected and closed. On top of that, it just looks awful in the interface all around (and is also not included in the accepted Pull Request count by GitHub for the committer, which is frustrating for newer contributors to a surprising extent). In the [willful absence of a reply from the GitHub team to allow marking a pull request as merged through a flag](https://github.com/orgs/community/discussions/12437), and [GitLab being unlikely to make a similar change either](https://gitlab.com/gitlab-org/gitlab/-/issues/16701), we can look at implementing this as an integration feature in Skara itself instead. > > Every pull has a corresponding pre-integration branch created at the time it was made, and this special branch is marked for deletion when the pull is integrated, and initially it was thought to merge the Pull Request into these corresponding pr/ branches just before their deletion but that proved to present too many challenges to be viable. Instead a related approach of enhancing the Pull Request bots with their own special integration branches can be taken, as well as providing a flexible utility method for extracting the actual branch the Pull Request was integrated into, to address the concern of losing information on what the actual Pull Request target branch was. > > > https://gitlab.com/gitlab-org/gitlab/-/blob/master/app/models/merge_request.rb#L172 Julian Waters has updated the pull request incrementally with one additional commit since the last revision: Missing addremove call ------------- Changes: - all: https://git.openjdk.org/skara/pull/1409/files - new: https://git.openjdk.org/skara/pull/1409/files/94772742..076b3489 Webrevs: - full: https://webrevs.openjdk.org/?repo=skara&pr=1409&range=50 - incr: https://webrevs.openjdk.org/?repo=skara&pr=1409&range=49-50 Stats: 1 line in 1 file changed: 1 ins; 0 del; 0 mod Patch: https://git.openjdk.org/skara/pull/1409.diff Fetch: git fetch https://git.openjdk.org/skara pull/1409/head:pull/1409 PR: https://git.openjdk.org/skara/pull/1409