From ehelin at openjdk.org Thu Feb 1 10:05:11 2024 From: ehelin at openjdk.org (Erik Duveblad) Date: Thu, 1 Feb 2024 10:05:11 GMT Subject: RFR: 2163: Add API to Repository for working "git notes" [v2] In-Reply-To: <7qBpGyt5oaRZri0mXTEt6N0oMbnBDwbVRjHNs8duzzo=.44140011-6630-4772-bdaf-4bd94ed419b0@github.com> References: <91jqh9fj5GU7K6-hwqlq5alhz5MDdznFLTXuv98Ap6Y=.6f6e0686-2876-48b1-852f-59d915c87c47@github.com> <7qBpGyt5oaRZri0mXTEt6N0oMbnBDwbVRjHNs8duzzo=.44140011-6630-4772-bdaf-4bd94ed419b0@github.com> Message-ID: On Tue, 30 Jan 2024 17:24:48 GMT, Zhao Song wrote: >> Erik Duveblad has updated the pull request incrementally with one additional commit since the last revision: >> >> note -> lines > > Marked as reviewed by zsong (Reviewer). Thanks @zhaosongzs and @erikj79 for reviewing! ------------- PR Comment: https://git.openjdk.org/skara/pull/1607#issuecomment-1920952340 From ehelin at openjdk.org Thu Feb 1 10:05:11 2024 From: ehelin at openjdk.org (Erik Duveblad) Date: Thu, 1 Feb 2024 10:05:11 GMT Subject: Integrated: 2163: Add API to Repository for working "git notes" In-Reply-To: <91jqh9fj5GU7K6-hwqlq5alhz5MDdznFLTXuv98Ap6Y=.6f6e0686-2876-48b1-852f-59d915c87c47@github.com> References: <91jqh9fj5GU7K6-hwqlq5alhz5MDdznFLTXuv98Ap6Y=.6f6e0686-2876-48b1-852f-59d915c87c47@github.com> Message-ID: On Mon, 29 Jan 2024 14:28:06 GMT, Erik Duveblad wrote: > Hi all, > > please review this patch that adds the methods `addNote`. `notes` and `pushNotes` to `Repository.java` to allow Skara code to work with [git notes](https://git-scm.com/docs/git-notes). This will be used in another patch to implement a "git note"-notifier (see https://bugs.openjdk.org/browse/SKARA-2164). > > I opted for a fairly minimal API here even though the `git notes` command exposes many more features. I also decided to add the `pushNotes` method although callers could use the `push(String refspec, URI uri)` variant because I don't like the notes refspec to "leak" out into other parts of the code. > > Also added a couple of unit tests. > > Thanks, > Erik This pull request has now been integrated. Changeset: 21e12f6d Author: Erik Duveblad URL: https://git.openjdk.org/skara/commit/21e12f6db868a9f1d840cdc0623b456e05ab4f67 Stats: 121 lines in 4 files changed: 121 ins; 0 del; 0 mod 2163: Add API to Repository for working "git notes" Reviewed-by: erikj, zsong ------------- PR: https://git.openjdk.org/skara/pull/1607 From ihse at openjdk.org Thu Feb 1 13:06:12 2024 From: ihse at openjdk.org (Magnus Ihse Bursie) Date: Thu, 1 Feb 2024 13:06:12 GMT Subject: Withdrawn: 2101: Change confusing requirement text In-Reply-To: <_9TiK5X6jvRi72rOXir9voOHGXx8wpSRUcis4Z76jpM=.1405bc8b-c635-4164-a97b-0a32ebe89036@github.com> References: <_9TiK5X6jvRi72rOXir9voOHGXx8wpSRUcis4Z76jpM=.1405bc8b-c635-4164-a97b-0a32ebe89036@github.com> Message-ID: On Wed, 8 Nov 2023 17:36:17 GMT, Magnus Ihse Bursie wrote: > The checkbox list in "Progress" in the PRs body says: > > "Commit message must refer to an issue" > > but it is not really talking about the commit message, but the PR title. The "refer to an issue" is also vague; especially since the only time you will be hit by this problem is when you are a newcomer, and do not understand what is expected of you. > > The text should be rephrased to be more helpful to newcomers. This pull request has been closed without being integrated. ------------- PR: https://git.openjdk.org/skara/pull/1584 From lujaniuk at openjdk.org Thu Feb 8 14:03:21 2024 From: lujaniuk at openjdk.org (Ludvig Janiuk) Date: Thu, 8 Feb 2024 14:03:21 GMT Subject: RFR: make Backport case insensitive Message-ID: There's no reason why this should validate case, and has caused unnecessary friction in the past. ------------- Commit messages: - make Backport case insensitive Changes: https://git.openjdk.org/skara/pull/1611/files Webrev: https://webrevs.openjdk.org/?repo=skara&pr=1611&range=00 Stats: 2 lines in 1 file changed: 0 ins; 0 del; 2 mod Patch: https://git.openjdk.org/skara/pull/1611.diff Fetch: git fetch https://git.openjdk.org/skara.git pull/1611/head:pull/1611 PR: https://git.openjdk.org/skara/pull/1611 From erikj at openjdk.org Thu Feb 8 14:16:19 2024 From: erikj at openjdk.org (Erik Joelsson) Date: Thu, 8 Feb 2024 14:16:19 GMT Subject: RFR: make Backport case insensitive In-Reply-To: References: Message-ID: On Thu, 8 Feb 2024 13:59:48 GMT, Ludvig Janiuk wrote: > There's no reason why this should validate case, and has caused unnecessary friction in the past. Please file a bug for this and associate with the PR. You can also enable "actions" for your fork to automatically run the tests. ------------- PR Comment: https://git.openjdk.org/skara/pull/1611#issuecomment-1934208939 From lujaniuk at openjdk.org Thu Feb 8 16:27:10 2024 From: lujaniuk at openjdk.org (Ludvig Janiuk) Date: Thu, 8 Feb 2024 16:27:10 GMT Subject: RFR: 2169 pr-bot should ignore case in "Backport " PR titles In-Reply-To: References: Message-ID: On Thu, 8 Feb 2024 13:59:48 GMT, Ludvig Janiuk wrote: > There's no reason why this should validate case, and has caused unnecessary friction in the past. `sh gradlew build` passes with the changes. ------------- PR Comment: https://git.openjdk.org/skara/pull/1611#issuecomment-1934486580 From lujaniuk at openjdk.org Fri Feb 9 19:24:06 2024 From: lujaniuk at openjdk.org (Ludvig Janiuk) Date: Fri, 9 Feb 2024 19:24:06 GMT Subject: RFR: 2169 pr-bot should ignore case in "Backport " PR titles [v2] In-Reply-To: References: Message-ID: <-7DK8kqUo_lNYnACp-fkf0qMjybTXNM9Ba6U0edsIfU=.6b43e2ed-f25a-46a7-acee-3eebaa69492b@github.com> > There's no reason why this should validate case, and has caused unnecessary friction in the past. Ludvig Janiuk has updated the pull request incrementally with two additional commits since the last revision: - noop - noop ------------- Changes: - all: https://git.openjdk.org/skara/pull/1611/files - new: https://git.openjdk.org/skara/pull/1611/files/36d0b672..f6e0794b Webrevs: - full: https://webrevs.openjdk.org/?repo=skara&pr=1611&range=01 - incr: https://webrevs.openjdk.org/?repo=skara&pr=1611&range=00-01 Stats: 0 lines in 0 files changed: 0 ins; 0 del; 0 mod Patch: https://git.openjdk.org/skara/pull/1611.diff Fetch: git fetch https://git.openjdk.org/skara.git pull/1611/head:pull/1611 PR: https://git.openjdk.org/skara/pull/1611 From gli at openjdk.org Sun Feb 11 23:56:19 2024 From: gli at openjdk.org (Guoxiong Li) Date: Sun, 11 Feb 2024 23:56:19 GMT Subject: RFR: 2169 pr-bot should ignore case in "Backport " PR titles [v2] In-Reply-To: <-7DK8kqUo_lNYnACp-fkf0qMjybTXNM9Ba6U0edsIfU=.6b43e2ed-f25a-46a7-acee-3eebaa69492b@github.com> References: <-7DK8kqUo_lNYnACp-fkf0qMjybTXNM9Ba6U0edsIfU=.6b43e2ed-f25a-46a7-acee-3eebaa69492b@github.com> Message-ID: On Fri, 9 Feb 2024 19:24:06 GMT, Ludvig Janiuk wrote: >> There's no reason why this should validate case, and has caused unnecessary friction in the past. > > Ludvig Janiuk has updated the pull request incrementally with two additional commits since the last revision: > > - noop > - noop It is good to add some test cases to verify this changement. ------------- PR Comment: https://git.openjdk.org/skara/pull/1611#issuecomment-1937918137 From ehelin at openjdk.org Mon Feb 12 15:02:45 2024 From: ehelin at openjdk.org (Erik Duveblad) Date: Mon, 12 Feb 2024 15:02:45 GMT Subject: RFR: 2167: Unify search methods in IssueProject [v2] In-Reply-To: References: Message-ID: > Hi all, > > this patch unifies the various ways of searching that exist within `JiraProejct`. I also chose to expose the `search` method in the `IssueProject` since I will make use of it in later patches. This is the first patch of _many_ that will extract the JBS specific parts from `JiraProject` so that `JiraProject` will be about JIRA and JBS specific functionality will be moved elsewhere. This refactoring is needed to allow other parts of the Skara to use JBS functionality with only an `IssueProject` instance (without turning `JiraProject` into effectively `JbsProject`). > > Thanks, > Erik Erik Duveblad has updated the pull request incrementally with two additional commits since the last revision: - update JiraProject::search - Do not expose the "search" method ------------- Changes: - all: https://git.openjdk.org/skara/pull/1610/files - new: https://git.openjdk.org/skara/pull/1610/files/7216ea96..ef17ef1f Webrevs: - full: https://webrevs.openjdk.org/?repo=skara&pr=1610&range=01 - incr: https://webrevs.openjdk.org/?repo=skara&pr=1610&range=00-01 Stats: 40 lines in 3 files changed: 18 ins; 12 del; 10 mod Patch: https://git.openjdk.org/skara/pull/1610.diff Fetch: git fetch https://git.openjdk.org/skara.git pull/1610/head:pull/1610 PR: https://git.openjdk.org/skara/pull/1610 From erikj at openjdk.org Mon Feb 12 18:26:00 2024 From: erikj at openjdk.org (Erik Joelsson) Date: Mon, 12 Feb 2024 18:26:00 GMT Subject: RFR: 2167: Unify search methods in IssueProject [v2] In-Reply-To: References: Message-ID: <13aQfdqYn9YrTh_vbx8s7BH62VcW5mYahH5309E65uk=.f332723c-dd4e-461b-94d8-2023dcdcedea@github.com> On Mon, 12 Feb 2024 15:02:45 GMT, Erik Duveblad wrote: >> Hi all, >> >> this patch unifies the various ways of searching that exist within `JiraProejct`. I also chose to expose the `search` method in the `IssueProject` since I will make use of it in later patches. This is the first patch of _many_ that will extract the JBS specific parts from `JiraProject` so that `JiraProject` will be about JIRA and JBS specific functionality will be moved elsewhere. This refactoring is needed to allow other parts of the Skara to use JBS functionality with only an `IssueProject` instance (without turning `JiraProject` into effectively `JbsProject`). >> >> Thanks, >> Erik > > Erik Duveblad has updated the pull request incrementally with two additional commits since the last revision: > > - update JiraProject::search > - Do not expose the "search" method issuetracker/src/main/java/org/openjdk/skara/issuetracker/jira/JiraProject.java line 522: > 520: } > 521: > 522: if (count < issues.get("total").asInt() && count < limit) { If `limit == -1` then we will never try to request more pages. Is that intended? ------------- PR Review Comment: https://git.openjdk.org/skara/pull/1610#discussion_r1486595794 From lujaniuk at openjdk.org Tue Feb 13 11:29:15 2024 From: lujaniuk at openjdk.org (Ludvig Janiuk) Date: Tue, 13 Feb 2024 11:29:15 GMT Subject: RFR: 2169 pr-bot should ignore case in "Backport " PR titles [v3] In-Reply-To: References: Message-ID: <_ibim6YovVT53LTjbviMusrhFasIzw6SwItY5ed-G5w=.c492e1ae-1a13-475a-987c-fd33dc138b27@github.com> > There's no reason why this should validate case, and has caused unnecessary friction in the past. Ludvig Janiuk has updated the pull request incrementally with one additional commit since the last revision: Tests ------------- Changes: - all: https://git.openjdk.org/skara/pull/1611/files - new: https://git.openjdk.org/skara/pull/1611/files/f6e0794b..9022ee2f Webrevs: - full: https://webrevs.openjdk.org/?repo=skara&pr=1611&range=02 - incr: https://webrevs.openjdk.org/?repo=skara&pr=1611&range=01-02 Stats: 67 lines in 1 file changed: 67 ins; 0 del; 0 mod Patch: https://git.openjdk.org/skara/pull/1611.diff Fetch: git fetch https://git.openjdk.org/skara.git pull/1611/head:pull/1611 PR: https://git.openjdk.org/skara/pull/1611 From lujaniuk at openjdk.org Tue Feb 13 11:29:15 2024 From: lujaniuk at openjdk.org (Ludvig Janiuk) Date: Tue, 13 Feb 2024 11:29:15 GMT Subject: RFR: 2169 pr-bot should ignore case in "Backport " PR titles [v2] In-Reply-To: References: <-7DK8kqUo_lNYnACp-fkf0qMjybTXNM9Ba6U0edsIfU=.6b43e2ed-f25a-46a7-acee-3eebaa69492b@github.com> Message-ID: On Sun, 11 Feb 2024 23:54:08 GMT, Guoxiong Li wrote: >> Ludvig Janiuk has updated the pull request incrementally with two additional commits since the last revision: >> >> - noop >> - noop > > It is good to add some test cases to verify this changement. @lgxbslgx You're right. I've added tests to check both changes. ------------- PR Comment: https://git.openjdk.org/skara/pull/1611#issuecomment-1941279801 From ehelin at openjdk.org Tue Feb 13 13:51:33 2024 From: ehelin at openjdk.org (Erik Duveblad) Date: Tue, 13 Feb 2024 13:51:33 GMT Subject: RFR: 2167: Unify search methods in IssueProject [v3] In-Reply-To: References: Message-ID: > Hi all, > > this patch unifies the various ways of searching that exist within `JiraProejct`. I also chose to expose the `search` method in the `IssueProject` since I will make use of it in later patches. This is the first patch of _many_ that will extract the JBS specific parts from `JiraProject` so that `JiraProject` will be about JIRA and JBS specific functionality will be moved elsewhere. This refactoring is needed to allow other parts of the Skara to use JBS functionality with only an `IssueProject` instance (without turning `JiraProject` into effectively `JbsProject`). > > Thanks, > Erik Erik Duveblad has updated the pull request incrementally with one additional commit since the last revision: Fix limit ------------- Changes: - all: https://git.openjdk.org/skara/pull/1610/files - new: https://git.openjdk.org/skara/pull/1610/files/ef17ef1f..6abc5a55 Webrevs: - full: https://webrevs.openjdk.org/?repo=skara&pr=1610&range=02 - incr: https://webrevs.openjdk.org/?repo=skara&pr=1610&range=01-02 Stats: 3 lines in 1 file changed: 2 ins; 0 del; 1 mod Patch: https://git.openjdk.org/skara/pull/1610.diff Fetch: git fetch https://git.openjdk.org/skara.git pull/1610/head:pull/1610 PR: https://git.openjdk.org/skara/pull/1610 From erikj at openjdk.org Tue Feb 13 18:46:52 2024 From: erikj at openjdk.org (Erik Joelsson) Date: Tue, 13 Feb 2024 18:46:52 GMT Subject: RFR: 2169 pr-bot should ignore case in "Backport " PR titles [v3] In-Reply-To: <_ibim6YovVT53LTjbviMusrhFasIzw6SwItY5ed-G5w=.c492e1ae-1a13-475a-987c-fd33dc138b27@github.com> References: <_ibim6YovVT53LTjbviMusrhFasIzw6SwItY5ed-G5w=.c492e1ae-1a13-475a-987c-fd33dc138b27@github.com> Message-ID: On Tue, 13 Feb 2024 11:29:15 GMT, Ludvig Janiuk wrote: >> There's no reason why this should validate case, and has caused unnecessary friction in the past. > > Ludvig Janiuk has updated the pull request incrementally with one additional commit since the last revision: > > Tests Marked as reviewed by erikj (Lead). bots/pr/src/test/java/org/openjdk/skara/bots/pr/BackportTests.java line 1187: > 1185: var editHash = localRepo.commit("Backport", "duke", "duke at openjdk.org"); > 1186: localRepo.push(editHash, author.authenticatedUrl(), "refs/heads/edit", true); > 1187: var pr = credentials.createPullRequest(author, "master", "edit", "bacKporT" + releaseHash.hex()); I hadn't realized this, but we actually don't require space between `backport` and the hash. That's probably fine and not something we need to change here. ------------- PR Review: https://git.openjdk.org/skara/pull/1611#pullrequestreview-1878689272 PR Review Comment: https://git.openjdk.org/skara/pull/1611#discussion_r1488407041 From lujaniuk at openjdk.org Tue Feb 13 18:46:52 2024 From: lujaniuk at openjdk.org (Ludvig Janiuk) Date: Tue, 13 Feb 2024 18:46:52 GMT Subject: Integrated: 2169 pr-bot should ignore case in "Backport " PR titles In-Reply-To: References: Message-ID: On Thu, 8 Feb 2024 13:59:48 GMT, Ludvig Janiuk wrote: > There's no reason why this should validate case, and has caused unnecessary friction in the past. This pull request has now been integrated. Changeset: b8e2fce1 Author: Ludvig Janiuk Committer: Erik Joelsson URL: https://git.openjdk.org/skara/commit/b8e2fce10b97de230b7c67bb195b91e6c81147e7 Stats: 69 lines in 2 files changed: 67 ins; 0 del; 2 mod 2169: pr-bot should ignore case in "Backport " PR titles Reviewed-by: erikj ------------- PR: https://git.openjdk.org/skara/pull/1611 From erikj at openjdk.org Tue Feb 13 18:56:51 2024 From: erikj at openjdk.org (Erik Joelsson) Date: Tue, 13 Feb 2024 18:56:51 GMT Subject: RFR: 2167: Unify search methods in IssueProject [v3] In-Reply-To: References: Message-ID: On Tue, 13 Feb 2024 13:51:33 GMT, Erik Duveblad wrote: >> Hi all, >> >> this patch unifies the various ways of searching that exist within `JiraProejct`. I also chose to expose the `search` method in the `IssueProject` since I will make use of it in later patches. This is the first patch of _many_ that will extract the JBS specific parts from `JiraProject` so that `JiraProject` will be about JIRA and JBS specific functionality will be moved elsewhere. This refactoring is needed to allow other parts of the Skara to use JBS functionality with only an `IssueProject` instance (without turning `JiraProject` into effectively `JbsProject`). >> >> Thanks, >> Erik > > Erik Duveblad has updated the pull request incrementally with one additional commit since the last revision: > > Fix limit This looks like it's doing the right thing. It would be good if you could add a manual test that verifies that it at least doesn't blow up, similar to existing manual tests for API calls. ------------- PR Review: https://git.openjdk.org/skara/pull/1610#pullrequestreview-1878710103 From lujaniuk at openjdk.org Wed Feb 14 08:49:28 2024 From: lujaniuk at openjdk.org (Ludvig Janiuk) Date: Wed, 14 Feb 2024 08:49:28 GMT Subject: RFR: 2169 pr-bot should ignore case in "Backport " PR titles [v3] In-Reply-To: References: <_ibim6YovVT53LTjbviMusrhFasIzw6SwItY5ed-G5w=.c492e1ae-1a13-475a-987c-fd33dc138b27@github.com> Message-ID: <1CHeEdXHqtHai4Ghe2zKDO0Ihl3wFo4P57k0JU9iiCk=.780e8e59-dd82-4ce7-ad0a-ab944a4e16ba@github.com> On Tue, 13 Feb 2024 18:42:25 GMT, Erik Joelsson wrote: >> Ludvig Janiuk has updated the pull request incrementally with one additional commit since the last revision: >> >> Tests > > bots/pr/src/test/java/org/openjdk/skara/bots/pr/BackportTests.java line 1187: > >> 1185: var editHash = localRepo.commit("Backport", "duke", "duke at openjdk.org"); >> 1186: localRepo.push(editHash, author.authenticatedUrl(), "refs/heads/edit", true); >> 1187: var pr = credentials.createPullRequest(author, "master", "edit", "bacKporT" + releaseHash.hex()); > > I hadn't realized this, but we actually don't require space between `backport` and the hash. That's probably fine and not something we need to change here. Yep ------------- PR Review Comment: https://git.openjdk.org/skara/pull/1611#discussion_r1489113128 From ehelin at openjdk.org Thu Feb 15 13:42:33 2024 From: ehelin at openjdk.org (Erik Duveblad) Date: Thu, 15 Feb 2024 13:42:33 GMT Subject: RFR: 2164: Add git notes notifier [v2] In-Reply-To: <3pd0BnGk8UTr4mAKxa6J4_7auq4uib1doqLyKLQmrg8=.44bbc5e0-dad4-4979-9c45-2982cec86d88@github.com> References: <3pd0BnGk8UTr4mAKxa6J4_7auq4uib1doqLyKLQmrg8=.44bbc5e0-dad4-4979-9c45-2982cec86d88@github.com> Message-ID: > Hi all, > > please review this patch that adds a new notifier that records the link to a commit, the link to the review and ev. links to issues in a [git note](https://git-scm.com/docs/git-notes). This will make it possible to view this information using e.g. `git log` or `git show` _without_ having these URLs in the commit message itself. Git notes are also modifiable (they are just blobs referred to by a ref), so they can be updated if ever change the scheme or domain for these links. > > I also added two unit tests for testing the new notifier. > > Thanks, > Erik Erik Duveblad has updated the pull request with a new target base due to a merge or a rebase. The incremental webrev excludes the unrelated changes brought in by the merge/rebase. The pull request contains two additional commits since the last revision: - Merge branch 'master' into note-notifier - 2164 ------------- Changes: - all: https://git.openjdk.org/skara/pull/1608/files - new: https://git.openjdk.org/skara/pull/1608/files/ca5af0bd..3cbff60f Webrevs: - full: https://webrevs.openjdk.org/?repo=skara&pr=1608&range=01 - incr: https://webrevs.openjdk.org/?repo=skara&pr=1608&range=00-01 Stats: 241 lines in 18 files changed: 189 ins; 17 del; 35 mod Patch: https://git.openjdk.org/skara/pull/1608.diff Fetch: git fetch https://git.openjdk.org/skara.git pull/1608/head:pull/1608 PR: https://git.openjdk.org/skara/pull/1608 From ehelin at openjdk.org Mon Feb 19 14:04:20 2024 From: ehelin at openjdk.org (Erik Duveblad) Date: Mon, 19 Feb 2024 14:04:20 GMT Subject: RFR: 2176: Repository.fetch should return an Optional Message-ID: Hi all, please review this patch that makes `Repository.fetch` return an `Optional` instead of just a `Hash` (since a `fetch` operation might now always fetch a commit, for example if code is trying to fetch a ref that does not exist in the remote repo). A fairly large diff, but I just moved the `.orElseThrow` one frame up in the call chain, so now the caller decides whether it should do `.orElseThrow` (instread of the callee, in this case `fetch`). Thanks, Erik ------------- Commit messages: - 2176 Changes: https://git.openjdk.org/skara/pull/1613/files Webrev: https://webrevs.openjdk.org/?repo=skara&pr=1613&range=00 Issue: https://bugs.openjdk.org/browse/SKARA-2176 Stats: 85 lines in 37 files changed: 0 ins; 3 del; 82 mod Patch: https://git.openjdk.org/skara/pull/1613.diff Fetch: git fetch https://git.openjdk.org/skara.git pull/1613/head:pull/1613 PR: https://git.openjdk.org/skara/pull/1613 From erikj at openjdk.org Tue Feb 20 15:41:10 2024 From: erikj at openjdk.org (Erik Joelsson) Date: Tue, 20 Feb 2024 15:41:10 GMT Subject: RFR: 2176: Repository.fetch should return an Optional In-Reply-To: References: Message-ID: <81okKxLckGdbwR_zggQJ2y3pnyKqKYuQ5A9hQt1GQYo=.1075b350-b5f9-44b0-bd2f-dd4c1c3f6ec3@github.com> On Mon, 19 Feb 2024 14:01:33 GMT, Erik Duveblad wrote: > Hi all, > > please review this patch that makes `Repository.fetch` return an `Optional` instead of just a `Hash` (since a `fetch` operation might now always fetch a commit, for example if code is trying to fetch a ref that does not exist in the remote repo). A fairly large diff, but I just moved the `.orElseThrow` one frame up in the call chain, so now the caller decides whether it should do `.orElseThrow` (instread of the callee, in this case `fetch`). > > Thanks, > Erik Marked as reviewed by erikj (Lead). ------------- PR Review: https://git.openjdk.org/skara/pull/1613#pullrequestreview-1890779650 From ehelin at openjdk.org Tue Feb 20 17:07:11 2024 From: ehelin at openjdk.org (Erik Duveblad) Date: Tue, 20 Feb 2024 17:07:11 GMT Subject: RFR: 2176: Repository.fetch should return an Optional In-Reply-To: <81okKxLckGdbwR_zggQJ2y3pnyKqKYuQ5A9hQt1GQYo=.1075b350-b5f9-44b0-bd2f-dd4c1c3f6ec3@github.com> References: <81okKxLckGdbwR_zggQJ2y3pnyKqKYuQ5A9hQt1GQYo=.1075b350-b5f9-44b0-bd2f-dd4c1c3f6ec3@github.com> Message-ID: On Tue, 20 Feb 2024 15:39:06 GMT, Erik Joelsson wrote: >> Hi all, >> >> please review this patch that makes `Repository.fetch` return an `Optional` instead of just a `Hash` (since a `fetch` operation might now always fetch a commit, for example if code is trying to fetch a ref that does not exist in the remote repo). A fairly large diff, but I just moved the `.orElseThrow` one frame up in the call chain, so now the caller decides whether it should do `.orElseThrow` (instread of the callee, in this case `fetch`). >> >> Thanks, >> Erik > > Marked as reviewed by erikj (Lead). Thanks @erikj79 for reviewing! ------------- PR Comment: https://git.openjdk.org/skara/pull/1613#issuecomment-1954658461 From ehelin at openjdk.org Tue Feb 20 17:07:11 2024 From: ehelin at openjdk.org (Erik Duveblad) Date: Tue, 20 Feb 2024 17:07:11 GMT Subject: Integrated: 2176: Repository.fetch should return an Optional In-Reply-To: References: Message-ID: On Mon, 19 Feb 2024 14:01:33 GMT, Erik Duveblad wrote: > Hi all, > > please review this patch that makes `Repository.fetch` return an `Optional` instead of just a `Hash` (since a `fetch` operation might now always fetch a commit, for example if code is trying to fetch a ref that does not exist in the remote repo). A fairly large diff, but I just moved the `.orElseThrow` one frame up in the call chain, so now the caller decides whether it should do `.orElseThrow` (instread of the callee, in this case `fetch`). > > Thanks, > Erik This pull request has now been integrated. Changeset: 301d61f0 Author: Erik Duveblad URL: https://git.openjdk.org/skara/commit/301d61f075b2f8e6d97f0eb43c62ba7fe2459739 Stats: 85 lines in 37 files changed: 0 ins; 3 del; 82 mod 2176: Repository.fetch should return an Optional Reviewed-by: erikj ------------- PR: https://git.openjdk.org/skara/pull/1613 From ehelin at openjdk.org Tue Feb 20 17:16:23 2024 From: ehelin at openjdk.org (Erik Duveblad) Date: Tue, 20 Feb 2024 17:16:23 GMT Subject: RFR: 2164: Add git notes notifier [v3] In-Reply-To: <3pd0BnGk8UTr4mAKxa6J4_7auq4uib1doqLyKLQmrg8=.44bbc5e0-dad4-4979-9c45-2982cec86d88@github.com> References: <3pd0BnGk8UTr4mAKxa6J4_7auq4uib1doqLyKLQmrg8=.44bbc5e0-dad4-4979-9c45-2982cec86d88@github.com> Message-ID: <3zkFN5UQPkJYfG-veInEKgBXwfWbp7yqgqn5dd9NaoQ=.01f752a4-72d5-4690-bf80-ae2240cb8d48@github.com> > Hi all, > > please review this patch that adds a new notifier that records the link to a commit, the link to the review and ev. links to issues in a [git note](https://git-scm.com/docs/git-notes). This will make it possible to view this information using e.g. `git log` or `git show` _without_ having these URLs in the commit message itself. Git notes are also modifiable (they are just blobs referred to by a ref), so they can be updated if ever change the scheme or domain for these links. > > I also added two unit tests for testing the new notifier. > > Thanks, > Erik Erik Duveblad has updated the pull request with a new target base due to a merge or a rebase. The incremental webrev excludes the unrelated changes brought in by the merge/rebase. The pull request contains four additional commits since the last revision: - Merge branch 'master' into note-notifier - Add configuration test - Merge branch 'master' into note-notifier - 2164 ------------- Changes: - all: https://git.openjdk.org/skara/pull/1608/files - new: https://git.openjdk.org/skara/pull/1608/files/3cbff60f..495efc2b Webrevs: - full: https://webrevs.openjdk.org/?repo=skara&pr=1608&range=02 - incr: https://webrevs.openjdk.org/?repo=skara&pr=1608&range=01-02 Stats: 161 lines in 40 files changed: 73 ins; 3 del; 85 mod Patch: https://git.openjdk.org/skara/pull/1608.diff Fetch: git fetch https://git.openjdk.org/skara.git pull/1608/head:pull/1608 PR: https://git.openjdk.org/skara/pull/1608 From erikj at openjdk.org Tue Feb 20 18:08:06 2024 From: erikj at openjdk.org (Erik Joelsson) Date: Tue, 20 Feb 2024 18:08:06 GMT Subject: RFR: 2164: Add git notes notifier [v3] In-Reply-To: <3zkFN5UQPkJYfG-veInEKgBXwfWbp7yqgqn5dd9NaoQ=.01f752a4-72d5-4690-bf80-ae2240cb8d48@github.com> References: <3pd0BnGk8UTr4mAKxa6J4_7auq4uib1doqLyKLQmrg8=.44bbc5e0-dad4-4979-9c45-2982cec86d88@github.com> <3zkFN5UQPkJYfG-veInEKgBXwfWbp7yqgqn5dd9NaoQ=.01f752a4-72d5-4690-bf80-ae2240cb8d48@github.com> Message-ID: On Tue, 20 Feb 2024 17:16:23 GMT, Erik Duveblad wrote: >> Hi all, >> >> please review this patch that adds a new notifier that records the link to a commit, the link to the review and ev. links to issues in a [git note](https://git-scm.com/docs/git-notes). This will make it possible to view this information using e.g. `git log` or `git show` _without_ having these URLs in the commit message itself. Git notes are also modifiable (they are just blobs referred to by a ref), so they can be updated if ever change the scheme or domain for these links. >> >> I also added two unit tests for testing the new notifier. >> >> Thanks, >> Erik > > Erik Duveblad has updated the pull request with a new target base due to a merge or a rebase. The incremental webrev excludes the unrelated changes brought in by the merge/rebase. The pull request contains four additional commits since the last revision: > > - Merge branch 'master' into note-notifier > - Add configuration test > - Merge branch 'master' into note-notifier > - 2164 Marked as reviewed by erikj (Lead). ------------- PR Review: https://git.openjdk.org/skara/pull/1608#pullrequestreview-1891130895 From ehelin at openjdk.org Wed Feb 21 15:16:16 2024 From: ehelin at openjdk.org (Erik Duveblad) Date: Wed, 21 Feb 2024 15:16:16 GMT Subject: RFR: 2164: Add git notes notifier [v3] In-Reply-To: References: <3pd0BnGk8UTr4mAKxa6J4_7auq4uib1doqLyKLQmrg8=.44bbc5e0-dad4-4979-9c45-2982cec86d88@github.com> <3zkFN5UQPkJYfG-veInEKgBXwfWbp7yqgqn5dd9NaoQ=.01f752a4-72d5-4690-bf80-ae2240cb8d48@github.com> Message-ID: On Tue, 20 Feb 2024 18:05:50 GMT, Erik Joelsson wrote: >> Erik Duveblad has updated the pull request with a new target base due to a merge or a rebase. The incremental webrev excludes the unrelated changes brought in by the merge/rebase. The pull request contains four additional commits since the last revision: >> >> - Merge branch 'master' into note-notifier >> - Add configuration test >> - Merge branch 'master' into note-notifier >> - 2164 > > Marked as reviewed by erikj (Lead). Thanks @erikj79 for reviewing! ------------- PR Comment: https://git.openjdk.org/skara/pull/1608#issuecomment-1956894611 From ehelin at openjdk.org Wed Feb 21 15:16:17 2024 From: ehelin at openjdk.org (Erik Duveblad) Date: Wed, 21 Feb 2024 15:16:17 GMT Subject: Integrated: 2164: Add git notes notifier In-Reply-To: <3pd0BnGk8UTr4mAKxa6J4_7auq4uib1doqLyKLQmrg8=.44bbc5e0-dad4-4979-9c45-2982cec86d88@github.com> References: <3pd0BnGk8UTr4mAKxa6J4_7auq4uib1doqLyKLQmrg8=.44bbc5e0-dad4-4979-9c45-2982cec86d88@github.com> Message-ID: On Mon, 29 Jan 2024 14:35:37 GMT, Erik Duveblad wrote: > Hi all, > > please review this patch that adds a new notifier that records the link to a commit, the link to the review and ev. links to issues in a [git note](https://git-scm.com/docs/git-notes). This will make it possible to view this information using e.g. `git log` or `git show` _without_ having these URLs in the commit message itself. Git notes are also modifiable (they are just blobs referred to by a ref), so they can be updated if ever change the scheme or domain for these links. > > I also added two unit tests for testing the new notifier. > > Thanks, > Erik This pull request has now been integrated. Changeset: 1c5d7e3a Author: Erik Duveblad URL: https://git.openjdk.org/skara/commit/1c5d7e3aa122fa5feba2f9d3bcf3a67c70ccea95 Stats: 319 lines in 8 files changed: 316 ins; 0 del; 3 mod 2164: Add git notes notifier Reviewed-by: erikj ------------- PR: https://git.openjdk.org/skara/pull/1608 From duke at openjdk.org Wed Feb 21 20:48:52 2024 From: duke at openjdk.org (duke) Date: Wed, 21 Feb 2024 20:48:52 GMT Subject: Withdrawn: 2137: Add integrity check to integrate and sponsor commands In-Reply-To: References: Message-ID: On Wed, 10 Jan 2024 09:12:50 GMT, Erik Duveblad wrote: > Hi all, > > please review this patch that adds that makes it possible to enable an "integrity check" for the `/integrate` and `/sponsor` commands. The integrity check will be run before a commit is pushed and ensures that the most recent commit on the target branch was integrated by the bots. > > The integrity check is implemented by utilizing an additional repository that for each repository and branch combination for a forge stores the two most recent commits pushed by the bots. This is information is stored in a text file called `heads.txt`, the first list line being the full hash of the most recent commit and the second line being the full hash of the next to most recent commit. The hashes of the two most recent commits are needed for the scenario when a bot throws an exception _after_ the file `heads.txt` in the integrity repo has been updated but _before_ the actual commit has been pushed to the target branch of the pull request. The `IntegrityVerifier` will recognize this case the next time a commit is verified (via a call to `verifyBranch`) and automatically correct the data in the integrity repository. > > I also added a couple of new unit tests as well as two larger unit tests for the `/integrate` and `/sponsor` commands. > > Thanks, > Erik This pull request has been closed without being integrated. ------------- PR: https://git.openjdk.org/skara/pull/1596 From erikj at openjdk.org Fri Feb 23 21:50:52 2024 From: erikj at openjdk.org (Erik Joelsson) Date: Fri, 23 Feb 2024 21:50:52 GMT Subject: RFR: 2182: Throw UncheckedRestException on empty GET responses Message-ID: <-XJ9xMwxbiYTbXJULp89k5FYnrQqjIkhRNI_4Rr6AXw=.20f81c5a-d196-4a3d-b10f-0342c208752c@github.com> A very common type of SEVERE logging message, which we get notifications for as admins, is this: Exception during periodic items checking: Cannot invoke "org.openjdk.skara.json.JSONArray.stream()" because the return value of "org.openjdk.skara.json.JSONValue.asArray()" is null There are some other kinds of instances where the NPE isn't providing any information, but that are likely caused by the same thing. From what I can tell, what's happening is that a REST GET call (typically to GitHub) returns an empty string in the response body. Our JSON parser translates this to a JSONNull and we subsequently try to treat it as a JSONArray and get the NPE when trying to call `stream()` on it. This typically happens a few times per day across all the bots. Ultimately I think this is a bug with GitHub, but we need to deal with it in a better way. My proposed solution is to add an option to RestRequest, `failOnEmptyResponse`, that defaults to true for GET requests but otherwise false. If set and the response body is empty, an UncheckedRestException is thrown. This exception is already handled as just a warning and we are collecting metrics for it to sound alarms if we get a lot of them. ------------- Commit messages: - SKARA-2182 Changes: https://git.openjdk.org/skara/pull/1614/files Webrev: https://webrevs.openjdk.org/?repo=skara&pr=1614&range=00 Issue: https://bugs.openjdk.org/browse/SKARA-2182 Stats: 16 lines in 2 files changed: 14 ins; 0 del; 2 mod Patch: https://git.openjdk.org/skara/pull/1614.diff Fetch: git fetch https://git.openjdk.org/skara.git pull/1614/head:pull/1614 PR: https://git.openjdk.org/skara/pull/1614 From zsong at openjdk.org Mon Feb 26 23:25:06 2024 From: zsong at openjdk.org (Zhao Song) Date: Mon, 26 Feb 2024 23:25:06 GMT Subject: RFR: 2168: Don't preserve invalid local clone Message-ID: When skara bot is running, it frequently clones a lot of repos to the local machine and sometimes, the clone would be invalid due to various reasons. Currently, if the skara bot found an invalid local clone, it will preserve the invalid clone by renaming it. ErikJ said the original authors might want to do some investigations on the invalid clones, so they chose to preserve the invalid clone. But now, we rarely investigate the clones. And as time goes by, the local machine running skara will have more and more invalid clones and the disk usage will be very high. And skara admin will need to clean the invalid repos periodically. To fix this issue, we should just clear the invalid repo in HostedRepositoryPool#removeOldClone. ------------- Commit messages: - SKARA-2168 Changes: https://git.openjdk.org/skara/pull/1615/files Webrev: https://webrevs.openjdk.org/?repo=skara&pr=1615&range=00 Issue: https://bugs.openjdk.org/browse/SKARA-2168 Stats: 13 lines in 1 file changed: 0 ins; 10 del; 3 mod Patch: https://git.openjdk.org/skara/pull/1615.diff Fetch: git fetch https://git.openjdk.org/skara.git pull/1615/head:pull/1615 PR: https://git.openjdk.org/skara/pull/1615 From erikj at openjdk.org Mon Feb 26 23:40:50 2024 From: erikj at openjdk.org (Erik Joelsson) Date: Mon, 26 Feb 2024 23:40:50 GMT Subject: RFR: 2168: Don't preserve invalid local clone In-Reply-To: References: Message-ID: On Mon, 26 Feb 2024 23:08:24 GMT, Zhao Song wrote: > When skara bot is running, it frequently clones a lot of repos to the local machine and sometimes, the clone would be invalid due to various reasons. > > Currently, if the skara bot found an invalid local clone, it will preserve the invalid clone by renaming it. ErikJ said the original authors might want to do some investigations on the invalid clones, so they chose to preserve the invalid clone. > > But now, we rarely investigate the clones. And as time goes by, the local machine running skara will have more and more invalid clones and the disk usage will be very high. And skara admin will need to clean the invalid repos periodically. > > To fix this issue, we should just clear the invalid repo in HostedRepositoryPool#removeOldClone. Marked as reviewed by erikj (Lead). ------------- PR Review: https://git.openjdk.org/skara/pull/1615#pullrequestreview-1902244013 From zsong at openjdk.org Tue Feb 27 14:51:15 2024 From: zsong at openjdk.org (Zhao Song) Date: Tue, 27 Feb 2024 14:51:15 GMT Subject: RFR: 2168: Don't preserve invalid local clone In-Reply-To: References: Message-ID: On Mon, 26 Feb 2024 23:08:24 GMT, Zhao Song wrote: > When skara bot is running, it frequently clones a lot of repos to the local machine and sometimes, the clone would be invalid due to various reasons. > > Currently, if the skara bot found an invalid local clone, it will preserve the invalid clone by renaming it. ErikJ said the original authors might want to do some investigations on the invalid clones, so they chose to preserve the invalid clone. > > But now, we rarely investigate the clones. And as time goes by, the local machine running skara will have more and more invalid clones and the disk usage will be very high. And skara admin will need to clean the invalid repos periodically. > > To fix this issue, we should just clear the invalid repo in HostedRepositoryPool#removeOldClone. Thanks for the review! ------------- PR Comment: https://git.openjdk.org/skara/pull/1615#issuecomment-1966718665 From zsong at openjdk.org Tue Feb 27 14:51:15 2024 From: zsong at openjdk.org (Zhao Song) Date: Tue, 27 Feb 2024 14:51:15 GMT Subject: Integrated: 2168: Don't preserve invalid local clone In-Reply-To: References: Message-ID: On Mon, 26 Feb 2024 23:08:24 GMT, Zhao Song wrote: > When skara bot is running, it frequently clones a lot of repos to the local machine and sometimes, the clone would be invalid due to various reasons. > > Currently, if the skara bot found an invalid local clone, it will preserve the invalid clone by renaming it. ErikJ said the original authors might want to do some investigations on the invalid clones, so they chose to preserve the invalid clone. > > But now, we rarely investigate the clones. And as time goes by, the local machine running skara will have more and more invalid clones and the disk usage will be very high. And skara admin will need to clean the invalid repos periodically. > > To fix this issue, we should just clear the invalid repo in HostedRepositoryPool#removeOldClone. This pull request has now been integrated. Changeset: f385a0c0 Author: Zhao Song URL: https://git.openjdk.org/skara/commit/f385a0c09d8a8cad10a1362abfd9d152c1a348f2 Stats: 13 lines in 1 file changed: 0 ins; 10 del; 3 mod 2168: Don't preserve invalid local clone Reviewed-by: erikj ------------- PR: https://git.openjdk.org/skara/pull/1615 From erikj at openjdk.org Wed Feb 28 19:06:02 2024 From: erikj at openjdk.org (Erik Joelsson) Date: Wed, 28 Feb 2024 19:06:02 GMT Subject: RFR: 2182: Throw UncheckedRestException on empty GET responses [v2] In-Reply-To: <-XJ9xMwxbiYTbXJULp89k5FYnrQqjIkhRNI_4Rr6AXw=.20f81c5a-d196-4a3d-b10f-0342c208752c@github.com> References: <-XJ9xMwxbiYTbXJULp89k5FYnrQqjIkhRNI_4Rr6AXw=.20f81c5a-d196-4a3d-b10f-0342c208752c@github.com> Message-ID: > A very common type of SEVERE logging message, which we get notifications for as admins, is this: > > Exception during periodic items checking: Cannot invoke "org.openjdk.skara.json.JSONArray.stream()" because the return value of "org.openjdk.skara.json.JSONValue.asArray()" is null > > There are some other kinds of instances where the NPE isn't providing any information, but that are likely caused by the same thing. From what I can tell, what's happening is that a REST GET call (typically to GitHub) returns an empty string in the response body. Our JSON parser translates this to a JSONNull and we subsequently try to treat it as a JSONArray and get the NPE when trying to call `stream()` on it. This typically happens a few times per day across all the bots. Ultimately I think this is a bug with GitHub, but we need to deal with it in a better way. > > My proposed solution is to add an option to RestRequest, `failOnEmptyResponse`, that defaults to true for GET requests but otherwise false. If set and the response body is empty, an UncheckedRestException is thrown. This exception is already handled as just a warning and we are collecting metrics for it to sound alarms if we get a lot of them. Erik Joelsson has updated the pull request incrementally with three additional commits since the last revision: - Only include the redirect in the RestException message if Location header is present - Include Location in RestException message - Improve logging of RestException ------------- Changes: - all: https://git.openjdk.org/skara/pull/1614/files - new: https://git.openjdk.org/skara/pull/1614/files/a1527449..e72f3612 Webrevs: - full: https://webrevs.openjdk.org/?repo=skara&pr=1614&range=01 - incr: https://webrevs.openjdk.org/?repo=skara&pr=1614&range=00-01 Stats: 6 lines in 2 files changed: 3 ins; 0 del; 3 mod Patch: https://git.openjdk.org/skara/pull/1614.diff Fetch: git fetch https://git.openjdk.org/skara.git pull/1614/head:pull/1614 PR: https://git.openjdk.org/skara/pull/1614 From erikj at openjdk.org Wed Feb 28 19:08:38 2024 From: erikj at openjdk.org (Erik Joelsson) Date: Wed, 28 Feb 2024 19:08:38 GMT Subject: RFR: 2182: Throw UncheckedRestException on empty GET responses [v2] In-Reply-To: References: <-XJ9xMwxbiYTbXJULp89k5FYnrQqjIkhRNI_4Rr6AXw=.20f81c5a-d196-4a3d-b10f-0342c208752c@github.com> Message-ID: <9nygP-xA3sC0kryz9Tv22rOxpQkL9Pl8tGTspcGiaSs=.0b0ceb41-dc0c-450b-b28b-309943e16401@github.com> On Wed, 28 Feb 2024 19:06:02 GMT, Erik Joelsson wrote: >> A very common type of SEVERE logging message, which we get notifications for as admins, is this: >> >> Exception during periodic items checking: Cannot invoke "org.openjdk.skara.json.JSONArray.stream()" because the return value of "org.openjdk.skara.json.JSONValue.asArray()" is null >> >> There are some other kinds of instances where the NPE isn't providing any information, but that are likely caused by the same thing. From what I can tell, what's happening is that a REST GET call (typically to GitHub) returns an empty string in the response body. Our JSON parser translates this to a JSONNull and we subsequently try to treat it as a JSONArray and get the NPE when trying to call `stream()` on it. This typically happens a few times per day across all the bots. Ultimately I think this is a bug with GitHub, but we need to deal with it in a better way. >> >> My proposed solution is to add an option to RestRequest, `failOnEmptyResponse`, that defaults to true for GET requests but otherwise false. If set and the response body is empty, an UncheckedRestException is thrown. This exception is already handled as just a warning and we are collecting metrics for it to sound alarms if we get a lot of them. >> >> Investigation and experimentation with this patch in our staging environment showed that these are (at least primarily) a mix of 302 and 304 responses. The 302 from JBS with a redirect to a maintenance page and 304 from GitHub for an unknown reason. We could have tried to handle every response code, or made >=300 errors, but for now I think the current approach is more straightforward and more likely to catch the problems we are specifically seeing. We are still logging the response codes so if we see other patterns in the future, we can adapt. > > Erik Joelsson has updated the pull request incrementally with three additional commits since the last revision: > > - Only include the redirect in the RestException message if Location header is present > - Include Location in RestException message > - Improve logging of RestException Investigation and experimentation with this patch in our staging environment showed that these are (at least primarily) a mix of 302 and 304 responses. The 302 from JBS with a redirect to a maintenance page and 304 from GitHub for an unknown reason. We could have tried to handle every response code, or made >=300 errors, but for now I think the current approach is more straightforward and more likely to catch the problems we are specifically seeing. We are still logging the response codes so if we see other patterns in the future, we can adapt. ------------- PR Comment: https://git.openjdk.org/skara/pull/1614#issuecomment-1969664628 From zsong at openjdk.org Thu Feb 29 21:20:14 2024 From: zsong at openjdk.org (Zhao Song) Date: Thu, 29 Feb 2024 21:20:14 GMT Subject: RFR: 2182: Throw UncheckedRestException on empty GET responses [v2] In-Reply-To: References: <-XJ9xMwxbiYTbXJULp89k5FYnrQqjIkhRNI_4Rr6AXw=.20f81c5a-d196-4a3d-b10f-0342c208752c@github.com> Message-ID: <-wWqS2QBNOg6Zj76qfSjzOGez2JXnl2qbEDTk-WlIt4=.42607453-bf44-4332-a40b-9bc87c51f46b@github.com> On Wed, 28 Feb 2024 19:06:02 GMT, Erik Joelsson wrote: >> A very common type of SEVERE logging message, which we get notifications for as admins, is this: >> >> Exception during periodic items checking: Cannot invoke "org.openjdk.skara.json.JSONArray.stream()" because the return value of "org.openjdk.skara.json.JSONValue.asArray()" is null >> >> There are some other kinds of instances where the NPE isn't providing any information, but that are likely caused by the same thing. From what I can tell, what's happening is that a REST GET call (typically to GitHub) returns an empty string in the response body. Our JSON parser translates this to a JSONNull and we subsequently try to treat it as a JSONArray and get the NPE when trying to call `stream()` on it. This typically happens a few times per day across all the bots. Ultimately I think this is a bug with GitHub, but we need to deal with it in a better way. >> >> My proposed solution is to add an option to RestRequest, `failOnEmptyResponse`, that defaults to true for GET requests but otherwise false. If set and the response body is empty, an UncheckedRestException is thrown. This exception is already handled as just a warning and we are collecting metrics for it to sound alarms if we get a lot of them. >> >> Investigation and experimentation with this patch in our staging environment showed that these are (at least primarily) a mix of 302 and 304 responses. The 302 from JBS with a redirect to a maintenance page and 304 from GitHub for an unknown reason. We could have tried to handle every response code, or made >=300 errors, but for now I think the current approach is more straightforward and more likely to catch the problems we are specifically seeing. We are still logging the response codes so if we see other patterns in the future, we can adapt. > > Erik Joelsson has updated the pull request incrementally with three additional commits since the last revision: > > - Only include the redirect in the RestException message if Location header is present > - Include Location in RestException message > - Improve logging of RestException Looks Good! ------------- Marked as reviewed by zsong (Reviewer). PR Review: https://git.openjdk.org/skara/pull/1614#pullrequestreview-1909891035 From erikj at openjdk.org Thu Feb 29 22:03:13 2024 From: erikj at openjdk.org (Erik Joelsson) Date: Thu, 29 Feb 2024 22:03:13 GMT Subject: RFR: 2182: Throw UncheckedRestException on empty GET responses [v2] In-Reply-To: References: <-XJ9xMwxbiYTbXJULp89k5FYnrQqjIkhRNI_4Rr6AXw=.20f81c5a-d196-4a3d-b10f-0342c208752c@github.com> Message-ID: <2v9M7qe7JpBhzZ6I12OUh0A3n6Bq-s9fM8eiktZpfWY=.9c1c0715-273f-4e2a-93eb-4422a0a0f5ec@github.com> On Wed, 28 Feb 2024 19:06:02 GMT, Erik Joelsson wrote: >> A very common type of SEVERE logging message, which we get notifications for as admins, is this: >> >> Exception during periodic items checking: Cannot invoke "org.openjdk.skara.json.JSONArray.stream()" because the return value of "org.openjdk.skara.json.JSONValue.asArray()" is null >> >> There are some other kinds of instances where the NPE isn't providing any information, but that are likely caused by the same thing. From what I can tell, what's happening is that a REST GET call (typically to GitHub) returns an empty string in the response body. Our JSON parser translates this to a JSONNull and we subsequently try to treat it as a JSONArray and get the NPE when trying to call `stream()` on it. This typically happens a few times per day across all the bots. Ultimately I think this is a bug with GitHub, but we need to deal with it in a better way. >> >> My proposed solution is to add an option to RestRequest, `failOnEmptyResponse`, that defaults to true for GET requests but otherwise false. If set and the response body is empty, an UncheckedRestException is thrown. This exception is already handled as just a warning and we are collecting metrics for it to sound alarms if we get a lot of them. >> >> Investigation and experimentation with this patch in our staging environment showed that these are (at least primarily) a mix of 302 and 304 responses. The 302 from JBS with a redirect to a maintenance page and 304 from GitHub for an unknown reason. We could have tried to handle every response code, or made >=300 errors, but for now I think the current approach is more straightforward and more likely to catch the problems we are specifically seeing. We are still logging the response codes so if we see other patterns in the future, we can adapt. > > Erik Joelsson has updated the pull request incrementally with three additional commits since the last revision: > > - Only include the redirect in the RestException message if Location header is present > - Include Location in RestException message > - Improve logging of RestException Thanks for review! ------------- PR Comment: https://git.openjdk.org/skara/pull/1614#issuecomment-1972042482 From erikj at openjdk.org Thu Feb 29 22:03:13 2024 From: erikj at openjdk.org (Erik Joelsson) Date: Thu, 29 Feb 2024 22:03:13 GMT Subject: Integrated: 2182: Throw UncheckedRestException on empty GET responses In-Reply-To: <-XJ9xMwxbiYTbXJULp89k5FYnrQqjIkhRNI_4Rr6AXw=.20f81c5a-d196-4a3d-b10f-0342c208752c@github.com> References: <-XJ9xMwxbiYTbXJULp89k5FYnrQqjIkhRNI_4Rr6AXw=.20f81c5a-d196-4a3d-b10f-0342c208752c@github.com> Message-ID: On Fri, 23 Feb 2024 21:47:07 GMT, Erik Joelsson wrote: > A very common type of SEVERE logging message, which we get notifications for as admins, is this: > > Exception during periodic items checking: Cannot invoke "org.openjdk.skara.json.JSONArray.stream()" because the return value of "org.openjdk.skara.json.JSONValue.asArray()" is null > > There are some other kinds of instances where the NPE isn't providing any information, but that are likely caused by the same thing. From what I can tell, what's happening is that a REST GET call (typically to GitHub) returns an empty string in the response body. Our JSON parser translates this to a JSONNull and we subsequently try to treat it as a JSONArray and get the NPE when trying to call `stream()` on it. This typically happens a few times per day across all the bots. Ultimately I think this is a bug with GitHub, but we need to deal with it in a better way. > > My proposed solution is to add an option to RestRequest, `failOnEmptyResponse`, that defaults to true for GET requests but otherwise false. If set and the response body is empty, an UncheckedRestException is thrown. This exception is already handled as just a warning and we are collecting metrics for it to sound alarms if we get a lot of them. > > Investigation and experimentation with this patch in our staging environment showed that these are (at least primarily) a mix of 302 and 304 responses. The 302 from JBS with a redirect to a maintenance page and 304 from GitHub for an unknown reason. We could have tried to handle every response code, or made >=300 errors, but for now I think the current approach is more straightforward and more likely to catch the problems we are specifically seeing. We are still logging the response codes so if we see other patterns in the future, we can adapt. This pull request has now been integrated. Changeset: dd8223ba Author: Erik Joelsson URL: https://git.openjdk.org/skara/commit/dd8223ba7577356bc3de0ae98d6843d071962cfa Stats: 21 lines in 3 files changed: 17 ins; 0 del; 4 mod 2182: Throw UncheckedRestException on empty GET responses Reviewed-by: zsong ------------- PR: https://git.openjdk.org/skara/pull/1614