From zsong at openjdk.org Mon Nov 18 22:02:57 2024 From: zsong at openjdk.org (Zhao Song) Date: Mon, 18 Nov 2024 22:02:57 GMT Subject: RFR: 2400: Avoid overwriting mbox archive Message-ID: Sometimes, the ArchiveWorkItem fails to retrieve the current contents of an mbox file while processing a PR. When this happens, it will resend all emails. Erik pointed out that in GitLab, creating a new file uses a POST request, while updating an existing file uses PUT. Similarly, GitHub requires the current "SHA" to update a file. The ArchiveWorkItem can determine whether it found existing content in the mbox archive and choose the right method accordingly. If it incorrectly fails to retrieve the existing content from GitLab or GitHub, it will attempt to create a new file. This attempt should fail, triggering a retry of the WorkItem, which would ideally succeed in retrieving the existing content. ------------- Commit messages: - update - fix typo - 2400 Changes: https://git.openjdk.org/skara/pull/1696/files Webrev: https://webrevs.openjdk.org/?repo=skara&pr=1696&range=00 Issue: https://bugs.openjdk.org/browse/SKARA-2400 Stats: 34 lines in 9 files changed: 2 ins; 1 del; 31 mod Patch: https://git.openjdk.org/skara/pull/1696.diff Fetch: git fetch https://git.openjdk.org/skara.git pull/1696/head:pull/1696 PR: https://git.openjdk.org/skara/pull/1696 From erikj at openjdk.org Mon Nov 18 23:01:41 2024 From: erikj at openjdk.org (Erik Joelsson) Date: Mon, 18 Nov 2024 23:01:41 GMT Subject: RFR: 2400: Avoid overwriting mbox archive In-Reply-To: References: Message-ID: <3--eEu4zbhRBOnYaA29rFe6jX0dAtjCVv8pxAZdIywQ=.1f78c88c-edde-4ca0-99de-dc41a6a49422@github.com> On Mon, 18 Nov 2024 21:44:55 GMT, Zhao Song wrote: > Sometimes, the ArchiveWorkItem fails to retrieve the current contents of an mbox file while processing a PR. When this happens, it will resend all emails. Erik pointed out that in GitLab, creating a new file uses a POST request, while updating an existing file uses PUT. Similarly, GitHub requires the current "SHA" to update a file. > > The ArchiveWorkItem can determine whether it found existing content in the mbox archive and choose the right method accordingly. If it incorrectly fails to retrieve the existing content from GitLab or GitHub, it will attempt to create a new file. This attempt should fail, triggering a retry of the WorkItem, which would ideally succeed in retrieving the existing content. forge/src/main/java/org/openjdk/skara/forge/HostedRepository.java line 111: > 109: * @param authorEmail Email of author and committer for commit > 110: */ > 111: void writeFileContents(String filename, String content, Branch branch, String message, String authorName, String authorEmail, boolean createNewFile); Please update the javadoc with the new parameter and make it clear what should happen when set to true or false. If set to true, the call will fail if the file exists. If set to false, the call will fail if the file does not exist. forge/src/main/java/org/openjdk/skara/forge/github/GitHubRepository.java line 294: > 292: > 293: @Override > 294: public void writeFileContents(String filename, String content, Branch branch, String message, String authorName, String authorEmail, boolean createNewEmail) { This variable name doesn't make sense here. test/src/main/java/org/openjdk/skara/test/TestHostedRepository.java line 232: > 230: > 231: @Override > 232: public void writeFileContents(String filename, String content, Branch branch, String message, String authorName, String authorEmail, boolean createNewFile) { This implementation should also enforce the new behavior so we catch any mistakes in testing. ------------- PR Review Comment: https://git.openjdk.org/skara/pull/1696#discussion_r1847386217 PR Review Comment: https://git.openjdk.org/skara/pull/1696#discussion_r1847387062 PR Review Comment: https://git.openjdk.org/skara/pull/1696#discussion_r1847389068 From zsong at openjdk.org Mon Nov 18 23:48:26 2024 From: zsong at openjdk.org (Zhao Song) Date: Mon, 18 Nov 2024 23:48:26 GMT Subject: RFR: 2400: Avoid overwriting mbox archive [v2] In-Reply-To: References: Message-ID: > Sometimes, the ArchiveWorkItem fails to retrieve the current contents of an mbox file while processing a PR. When this happens, it will resend all emails. Erik pointed out that in GitLab, creating a new file uses a POST request, while updating an existing file uses PUT. Similarly, GitHub requires the current "SHA" to update a file. > > The ArchiveWorkItem can determine whether it found existing content in the mbox archive and choose the right method accordingly. If it incorrectly fails to retrieve the existing content from GitLab or GitHub, it will attempt to create a new file. This attempt should fail, triggering a retry of the WorkItem, which would ideally succeed in retrieving the existing content. Zhao Song has updated the pull request incrementally with one additional commit since the last revision: review comment ------------- Changes: - all: https://git.openjdk.org/skara/pull/1696/files - new: https://git.openjdk.org/skara/pull/1696/files/9bc5ad18..26998838 Webrevs: - full: https://webrevs.openjdk.org/?repo=skara&pr=1696&range=01 - incr: https://webrevs.openjdk.org/?repo=skara&pr=1696&range=00-01 Stats: 25 lines in 3 files changed: 19 ins; 1 del; 5 mod Patch: https://git.openjdk.org/skara/pull/1696.diff Fetch: git fetch https://git.openjdk.org/skara.git pull/1696/head:pull/1696 PR: https://git.openjdk.org/skara/pull/1696 From zsong at openjdk.org Mon Nov 18 23:48:26 2024 From: zsong at openjdk.org (Zhao Song) Date: Mon, 18 Nov 2024 23:48:26 GMT Subject: RFR: 2400: Avoid overwriting mbox archive [v2] In-Reply-To: <3--eEu4zbhRBOnYaA29rFe6jX0dAtjCVv8pxAZdIywQ=.1f78c88c-edde-4ca0-99de-dc41a6a49422@github.com> References: <3--eEu4zbhRBOnYaA29rFe6jX0dAtjCVv8pxAZdIywQ=.1f78c88c-edde-4ca0-99de-dc41a6a49422@github.com> Message-ID: On Mon, 18 Nov 2024 22:55:41 GMT, Erik Joelsson wrote: >> Zhao Song has updated the pull request incrementally with one additional commit since the last revision: >> >> review comment > > forge/src/main/java/org/openjdk/skara/forge/HostedRepository.java line 111: > >> 109: * @param authorEmail Email of author and committer for commit >> 110: */ >> 111: void writeFileContents(String filename, String content, Branch branch, String message, String authorName, String authorEmail, boolean createNewFile); > > Please update the javadoc with the new parameter and make it clear what should happen when set to true or false. If set to true, the call will fail if the file exists. If set to false, the call will fail if the file does not exist. Will do > forge/src/main/java/org/openjdk/skara/forge/github/GitHubRepository.java line 294: > >> 292: >> 293: @Override >> 294: public void writeFileContents(String filename, String content, Branch branch, String message, String authorName, String authorEmail, boolean createNewEmail) { > > This variable name doesn't make sense here. Thanks for catching it! I was in a hurry when working on it > test/src/main/java/org/openjdk/skara/test/TestHostedRepository.java line 232: > >> 230: >> 231: @Override >> 232: public void writeFileContents(String filename, String content, Branch branch, String message, String authorName, String authorEmail, boolean createNewFile) { > > This implementation should also enforce the new behavior so we catch any mistakes in testing. Will fix it ------------- PR Review Comment: https://git.openjdk.org/skara/pull/1696#discussion_r1847419833 PR Review Comment: https://git.openjdk.org/skara/pull/1696#discussion_r1847420953 PR Review Comment: https://git.openjdk.org/skara/pull/1696#discussion_r1847421339 From erikj at openjdk.org Tue Nov 19 00:04:58 2024 From: erikj at openjdk.org (Erik Joelsson) Date: Tue, 19 Nov 2024 00:04:58 GMT Subject: RFR: 2400: Avoid overwriting mbox archive [v2] In-Reply-To: References: Message-ID: On Mon, 18 Nov 2024 23:48:26 GMT, Zhao Song wrote: >> Sometimes, the ArchiveWorkItem fails to retrieve the current contents of an mbox file while processing a PR. When this happens, it will resend all emails. Erik pointed out that in GitLab, creating a new file uses a POST request, while updating an existing file uses PUT. Similarly, GitHub requires the current "SHA" to update a file. >> >> The ArchiveWorkItem can determine whether it found existing content in the mbox archive and choose the right method accordingly. If it incorrectly fails to retrieve the existing content from GitLab or GitHub, it will attempt to create a new file. This attempt should fail, triggering a retry of the WorkItem, which would ideally succeed in retrieving the existing content. > > Zhao Song has updated the pull request incrementally with one additional commit since the last revision: > > review comment Marked as reviewed by erikj (Lead). ------------- PR Review: https://git.openjdk.org/skara/pull/1696#pullrequestreview-2444023831 From zsong at openjdk.org Tue Nov 19 18:29:17 2024 From: zsong at openjdk.org (Zhao Song) Date: Tue, 19 Nov 2024 18:29:17 GMT Subject: RFR: 2400: Avoid overwriting mbox archive [v2] In-Reply-To: References: Message-ID: On Mon, 18 Nov 2024 23:48:26 GMT, Zhao Song wrote: >> Sometimes, the ArchiveWorkItem fails to retrieve the current contents of an mbox file while processing a PR. When this happens, it will resend all emails. Erik pointed out that in GitLab, creating a new file uses a POST request, while updating an existing file uses PUT. Similarly, GitHub requires the current "SHA" to update a file. >> >> The ArchiveWorkItem can determine whether it found existing content in the mbox archive and choose the right method accordingly. If it incorrectly fails to retrieve the existing content from GitLab or GitHub, it will attempt to create a new file. This attempt should fail, triggering a retry of the WorkItem, which would ideally succeed in retrieving the existing content. > > Zhao Song has updated the pull request incrementally with one additional commit since the last revision: > > review comment Thank you for the review! ------------- PR Comment: https://git.openjdk.org/skara/pull/1696#issuecomment-2486439750 From zsong at openjdk.org Tue Nov 19 18:29:17 2024 From: zsong at openjdk.org (Zhao Song) Date: Tue, 19 Nov 2024 18:29:17 GMT Subject: Integrated: 2400: Avoid overwriting mbox archive In-Reply-To: References: Message-ID: On Mon, 18 Nov 2024 21:44:55 GMT, Zhao Song wrote: > Sometimes, the ArchiveWorkItem fails to retrieve the current contents of an mbox file while processing a PR. When this happens, it will resend all emails. Erik pointed out that in GitLab, creating a new file uses a POST request, while updating an existing file uses PUT. Similarly, GitHub requires the current "SHA" to update a file. > > The ArchiveWorkItem can determine whether it found existing content in the mbox archive and choose the right method accordingly. If it incorrectly fails to retrieve the existing content from GitLab or GitHub, it will attempt to create a new file. This attempt should fail, triggering a retry of the WorkItem, which would ideally succeed in retrieving the existing content. This pull request has now been integrated. Changeset: 4666739a Author: Zhao Song URL: https://git.openjdk.org/skara/commit/4666739a6cc42616b440ceeafb67c9e992cb1483 Stats: 57 lines in 9 files changed: 21 ins; 2 del; 34 mod 2400: Avoid overwriting mbox archive Reviewed-by: erikj ------------- PR: https://git.openjdk.org/skara/pull/1696 From zsong at openjdk.org Tue Nov 19 18:59:42 2024 From: zsong at openjdk.org (Zhao Song) Date: Tue, 19 Nov 2024 18:59:42 GMT Subject: RFR: 2410: Introduce process killing finally block in GitCommits Message-ID: Recently, I found that for pr-external container, in GitCommits, the exit of the git process could time out and then the process will be stuck in sleeping state and eat memory. With Pavel's suggestion, I think we should introduce the process killing finally block. ------------- Commit messages: - SKARA-2410 Changes: https://git.openjdk.org/skara/pull/1697/files Webrev: https://webrevs.openjdk.org/?repo=skara&pr=1697&range=00 Issue: https://bugs.openjdk.org/browse/SKARA-2410 Stats: 32 lines in 1 file changed: 22 ins; 2 del; 8 mod Patch: https://git.openjdk.org/skara/pull/1697.diff Fetch: git fetch https://git.openjdk.org/skara.git pull/1697/head:pull/1697 PR: https://git.openjdk.org/skara/pull/1697 From zsong at openjdk.org Tue Nov 19 18:59:42 2024 From: zsong at openjdk.org (Zhao Song) Date: Tue, 19 Nov 2024 18:59:42 GMT Subject: RFR: 2410: Introduce process killing finally block in GitCommits In-Reply-To: References: Message-ID: <_2GCBbBfwLVKdx14eoTx0Yc1aDUghk5aMO21kQYl_Mo=.1af86989-090c-4a08-b668-4534eaee8a54@github.com> On Tue, 19 Nov 2024 18:39:41 GMT, Zhao Song wrote: > Recently, I found that for pr-external container, in GitCommits, the exit of the git process could time out and then the process will be stuck in sleeping state and eat memory. With Pavel's suggestion, I think we should introduce the process killing finally block. Since most of the code are from Pavel. I am going to set him as the author ------------- PR Comment: https://git.openjdk.org/skara/pull/1697#issuecomment-2486477771 From erikj at openjdk.org Tue Nov 19 21:27:09 2024 From: erikj at openjdk.org (Erik Joelsson) Date: Tue, 19 Nov 2024 21:27:09 GMT Subject: RFR: 2410: Introduce process killing finally block in GitCommits In-Reply-To: References: Message-ID: On Tue, 19 Nov 2024 18:39:41 GMT, Zhao Song wrote: > Recently, I found that for pr-external container, in GitCommits, the exit of the git process could time out and then the process will be stuck in sleeping state and eat memory. With Pavel's suggestion, I think we should introduce the process killing finally block. Marked as reviewed by erikj (Lead). ------------- PR Review: https://git.openjdk.org/skara/pull/1697#pullrequestreview-2446684579 From zsong at openjdk.org Tue Nov 19 21:35:56 2024 From: zsong at openjdk.org (Zhao Song) Date: Tue, 19 Nov 2024 21:35:56 GMT Subject: RFR: 2410: Introduce process killing finally block in GitCommits In-Reply-To: References: Message-ID: On Tue, 19 Nov 2024 18:39:41 GMT, Zhao Song wrote: > Recently, I found that for pr-external container, in GitCommits, the exit of the git process could time out and then the process will be stuck in sleeping state and eat memory. With Pavel's suggestion, I think we should introduce the process killing finally block. Thank you for the review! ------------- PR Comment: https://git.openjdk.org/skara/pull/1697#issuecomment-2486797604 From zsong at openjdk.org Tue Nov 19 21:35:56 2024 From: zsong at openjdk.org (Zhao Song) Date: Tue, 19 Nov 2024 21:35:56 GMT Subject: Integrated: 2410: Introduce process killing finally block in GitCommits In-Reply-To: References: Message-ID: On Tue, 19 Nov 2024 18:39:41 GMT, Zhao Song wrote: > Recently, I found that for pr-external container, in GitCommits, the exit of the git process could time out and then the process will be stuck in sleeping state and eat memory. With Pavel's suggestion, I think we should introduce the process killing finally block. This pull request has now been integrated. Changeset: c5882168 Author: Pavel Rappo Committer: Zhao Song URL: https://git.openjdk.org/skara/commit/c5882168725100819d2847ae0fdb0bbffb1d5b6e Stats: 32 lines in 1 file changed: 22 ins; 2 del; 8 mod 2410: Introduce process killing finally block in GitCommits Reviewed-by: erikj ------------- PR: https://git.openjdk.org/skara/pull/1697 From prappo at openjdk.org Tue Nov 19 22:42:18 2024 From: prappo at openjdk.org (Pavel Rappo) Date: Tue, 19 Nov 2024 22:42:18 GMT Subject: RFR: 2410: Introduce process killing finally block in GitCommits In-Reply-To: References: Message-ID: On Tue, 19 Nov 2024 18:39:41 GMT, Zhao Song wrote: > Recently, I found that for pr-external container, in GitCommits, the exit of the git process could time out and then the process will be stuck in sleeping state and eat memory. With Pavel's suggestion, I think we should introduce the process killing finally block. vcs/src/main/java/org/openjdk/skara/vcs/git/GitCommits.java line 170: > 168: throw new IOException("'" + String.join(" ", command) + "' was interrupted", e); > 169: } finally { > 170: if (p.isAlive()) { For the record, this might be insufficient because `GitCommits` might have created multiple processes and this destroys only one of them. In general, Skara needs better tools to deal with processes. ------------- PR Review Comment: https://git.openjdk.org/skara/pull/1697#discussion_r1849143881 From erikj at openjdk.org Tue Nov 19 23:41:17 2024 From: erikj at openjdk.org (Erik Joelsson) Date: Tue, 19 Nov 2024 23:41:17 GMT Subject: RFR: 2410: Introduce process killing finally block in GitCommits In-Reply-To: References: Message-ID: On Tue, 19 Nov 2024 22:39:42 GMT, Pavel Rappo wrote: >> Recently, I found that for pr-external container, in GitCommits, the exit of the git process could time out and then the process will be stuck in sleeping state and eat memory. With Pavel's suggestion, I think we should introduce the process killing finally block. > > vcs/src/main/java/org/openjdk/skara/vcs/git/GitCommits.java line 170: > >> 168: throw new IOException("'" + String.join(" ", command) + "' was interrupted", e); >> 169: } finally { >> 170: if (p.isAlive()) { > > For the record, this might be insufficient because `GitCommits` might have created multiple processes and this destroys only one of them. > > In general, Skara needs better tools to deal with processes. Oh, you are right. I didn't look at the whole class. ------------- PR Review Comment: https://git.openjdk.org/skara/pull/1697#discussion_r1849303669 From zsong at openjdk.org Tue Nov 19 23:44:54 2024 From: zsong at openjdk.org (Zhao Song) Date: Tue, 19 Nov 2024 23:44:54 GMT Subject: RFR: 2410: Introduce process killing finally block in GitCommits In-Reply-To: References: Message-ID: On Tue, 19 Nov 2024 23:39:04 GMT, Erik Joelsson wrote: >> vcs/src/main/java/org/openjdk/skara/vcs/git/GitCommits.java line 170: >> >>> 168: throw new IOException("'" + String.join(" ", command) + "' was interrupted", e); >>> 169: } finally { >>> 170: if (p.isAlive()) { >> >> For the record, this might be insufficient because `GitCommits` might have created multiple processes and this destroys only one of them. >> >> In general, Skara needs better tools to deal with processes. > > Oh, you are right. I didn't look at the whole class. Do you mean we need to iterate all the processes and destroy them in the finally block? ------------- PR Review Comment: https://git.openjdk.org/skara/pull/1697#discussion_r1849306154 From erikj at openjdk.org Wed Nov 20 13:40:35 2024 From: erikj at openjdk.org (Erik Joelsson) Date: Wed, 20 Nov 2024 13:40:35 GMT Subject: RFR: 2410: Introduce process killing finally block in GitCommits In-Reply-To: References: Message-ID: <-eb0d6S8qhL33hJfCdLGt_N0q8EK5_4ERihhzl3690s=.0f9fc827-0adb-4afb-9715-3b02fbc8192b@github.com> On Tue, 19 Nov 2024 23:42:42 GMT, Zhao Song wrote: >> Oh, you are right. I didn't look at the whole class. > > Do you mean we need to iterate all the processes and destroy them in the finally block? I think the loop needs to make sure the inner close method is called on each process by catching exceptions and saving one of them, probably the first one, and then rethrow it once all processes have been dealt with. In practice I doubt there will ever be multiple processes though. That would only happen if `::iterator` is called multiple times. ------------- PR Review Comment: https://git.openjdk.org/skara/pull/1697#discussion_r1850333202 From erikj at openjdk.org Wed Nov 20 15:17:26 2024 From: erikj at openjdk.org (Erik Joelsson) Date: Wed, 20 Nov 2024 15:17:26 GMT Subject: RFR: 2411: Host::user should throw when failing Message-ID: The implementations of Host::user are all currently returning Optional::empty on any error return code. This means a that a bot checking for the existence of a user encounters an error with a host, e.g. JBS, it will think the user doesn't exist and take actions based on that assumption. This would be correct if the server response was 404, but in all other cases, it would be better if the WorkItem failed to be automatically retried later. This patch changes the onError lambda for these methods so that it only handles the 404 case, and in all other cases lets the RestException be thrown. This matches the behavior of most other methods in these classes. ------------- Commit messages: - SKARA-2411 Changes: https://git.openjdk.org/skara/pull/1698/files Webrev: https://webrevs.openjdk.org/?repo=skara&pr=1698&range=00 Issue: https://bugs.openjdk.org/browse/SKARA-2411 Stats: 5 lines in 3 files changed: 0 ins; 0 del; 5 mod Patch: https://git.openjdk.org/skara/pull/1698.diff Fetch: git fetch https://git.openjdk.org/skara.git pull/1698/head:pull/1698 PR: https://git.openjdk.org/skara/pull/1698 From kcr at openjdk.org Wed Nov 20 15:21:49 2024 From: kcr at openjdk.org (Kevin Rushforth) Date: Wed, 20 Nov 2024 15:21:49 GMT Subject: RFR: 2411: Host::user should throw when failing In-Reply-To: References: Message-ID: On Wed, 20 Nov 2024 15:12:10 GMT, Erik Joelsson wrote: > The implementations of Host::user are all currently returning Optional::empty on any error return code. This means a that a bot checking for the existence of a user encounters an error with a host, e.g. JBS, it will think the user doesn't exist and take actions based on that assumption. This would be correct if the server response was 404, but in all other cases, it would be better if the WorkItem failed to be automatically retried later. > > This patch changes the onError lambda for these methods so that it only handles the 404 case, and in all other cases lets the RestException be thrown. This matches the behavior of most other methods in these classes. This looks good. to me. One question about the Description: > The implementations of Host::user are all currently returning Optional::empty on any error return code I think you mean they are all currently returning `Optional.of(JSON.of())` right? And that after this change you will continue to return that for 404, and return `Optional.empty()` (which will throw) for all other cases. ------------- Marked as reviewed by kcr (Reviewer). PR Review: https://git.openjdk.org/skara/pull/1698#pullrequestreview-2448939410 From prappo at openjdk.org Wed Nov 20 15:30:23 2024 From: prappo at openjdk.org (Pavel Rappo) Date: Wed, 20 Nov 2024 15:30:23 GMT Subject: RFR: 2410: Introduce process killing finally block in GitCommits In-Reply-To: <-eb0d6S8qhL33hJfCdLGt_N0q8EK5_4ERihhzl3690s=.0f9fc827-0adb-4afb-9715-3b02fbc8192b@github.com> References: <-eb0d6S8qhL33hJfCdLGt_N0q8EK5_4ERihhzl3690s=.0f9fc827-0adb-4afb-9715-3b02fbc8192b@github.com> Message-ID: On Wed, 20 Nov 2024 13:38:07 GMT, Erik Joelsson wrote: > I think the loop needs to make sure the inner close method is called on each process by catching exceptions Correct. > and saving one of them, probably the first one, and then rethrow it once all processes have been dealt with. You don't need to figure out which exception you need to save. Save all of them. For that, use java.lang.Throwable::addSuppressed. > In practice I doubt there will ever be multiple processes though. That would only happen if `::iterator` is called multiple times. Also true. I only pointed out that the `close` method is not adequate for the current state of the class. Either `close` should account for multiple iterators, or it should throw an exception if more than one iterator is created. I guess `GitCommits` is itera_BLE_ and not itera_TOR_ only to be able to be used with the enhanced `for` statement. Tradeoffs were made. ------------- PR Review Comment: https://git.openjdk.org/skara/pull/1697#discussion_r1850531104 From erikj at openjdk.org Wed Nov 20 15:32:07 2024 From: erikj at openjdk.org (Erik Joelsson) Date: Wed, 20 Nov 2024 15:32:07 GMT Subject: RFR: 2411: Host::user should throw when failing In-Reply-To: References: Message-ID: On Wed, 20 Nov 2024 15:19:40 GMT, Kevin Rushforth wrote: > One question about the Description: > > > The implementations of Host::user are all currently returning Optional::empty on any error return code > > I think you mean they are all currently returning `Optional.of(JSON.of())` right? And that after this change you will continue to return that for 404, and return `Optional.empty()` (which will throw) for all other cases. No, the description is about the `Host::user` method, which in turn uses `RestRequest`, providing a lambda for `onError`. What you are describing is what that lambda does, which the `Host::user` methods will translate to `Optional` that either contains an instance of HostUser, or is empty. ------------- PR Comment: https://git.openjdk.org/skara/pull/1698#issuecomment-2488894530 From kcr at openjdk.org Wed Nov 20 15:52:26 2024 From: kcr at openjdk.org (Kevin Rushforth) Date: Wed, 20 Nov 2024 15:52:26 GMT Subject: RFR: 2411: Host::user should throw when failing In-Reply-To: References: Message-ID: <21H3OJsK8Ejx9JvChnwVkyIyUzax0TqrknRgvVX8a_E=.6d00412b-781f-4530-b297-89fc82e330fe@github.com> On Wed, 20 Nov 2024 15:12:10 GMT, Erik Joelsson wrote: > The implementations of Host::user are all currently returning Optional::empty on any error return code. This means a that a bot checking for the existence of a user encounters an error with a host, e.g. JBS, it will think the user doesn't exist and take actions based on that assumption. This would be correct if the server response was 404, but in all other cases, it would be better if the WorkItem failed to be automatically retried later. > > This patch changes the onError lambda for these methods so that it only handles the 404 case, and in all other cases lets the RestException be thrown. This matches the behavior of most other methods in these classes. Thanks for the clarification. ------------- PR Comment: https://git.openjdk.org/skara/pull/1698#issuecomment-2488947498 From zsong at openjdk.org Thu Nov 21 17:46:12 2024 From: zsong at openjdk.org (Zhao Song) Date: Thu, 21 Nov 2024 17:46:12 GMT Subject: RFR: 2411: Host::user should throw when failing In-Reply-To: References: Message-ID: On Wed, 20 Nov 2024 15:12:10 GMT, Erik Joelsson wrote: > The implementations of Host::user are all currently returning Optional::empty on any error return code. This means a that a bot checking for the existence of a user encounters an error with a host, e.g. JBS, it will think the user doesn't exist and take actions based on that assumption. This would be correct if the server response was 404, but in all other cases, it would be better if the WorkItem failed to be automatically retried later. > > This patch changes the onError lambda for these methods so that it only handles the 404 case, and in all other cases lets the RestException be thrown. This matches the behavior of most other methods in these classes. Marked as reviewed by zsong (Reviewer). ------------- PR Review: https://git.openjdk.org/skara/pull/1698#pullrequestreview-2452219663 From zsong at openjdk.org Thu Nov 21 18:41:00 2024 From: zsong at openjdk.org (Zhao Song) Date: Thu, 21 Nov 2024 18:41:00 GMT Subject: RFR: 2415: Update GitCommits::close to make sure it terminates all processes when an exception is thrown Message-ID: As Pavel pointed out in the pull request of [SKARA-2410](https://bugs.openjdk.org/browse/SKARA-2410)(https://github.com/openjdk/skara/pull/1697), with current implementation , GitCommits::close may have created multiple processes and only destroy one of them. As Erik and Pavel suggested, we should make sure close() is called on each process. ------------- Commit messages: - SKARA-2415 Changes: https://git.openjdk.org/skara/pull/1699/files Webrev: https://webrevs.openjdk.org/?repo=skara&pr=1699&range=00 Issue: https://bugs.openjdk.org/browse/SKARA-2415 Stats: 15 lines in 1 file changed: 14 ins; 0 del; 1 mod Patch: https://git.openjdk.org/skara/pull/1699.diff Fetch: git fetch https://git.openjdk.org/skara.git pull/1699/head:pull/1699 PR: https://git.openjdk.org/skara/pull/1699 From erikj at openjdk.org Thu Nov 21 21:23:20 2024 From: erikj at openjdk.org (Erik Joelsson) Date: Thu, 21 Nov 2024 21:23:20 GMT Subject: RFR: 2411: Host::user should throw when failing In-Reply-To: References: Message-ID: On Wed, 20 Nov 2024 15:12:10 GMT, Erik Joelsson wrote: > The implementations of Host::user are all currently returning Optional::empty on any error return code. This means a that a bot checking for the existence of a user encounters an error with a host, e.g. JBS, it will think the user doesn't exist and take actions based on that assumption. This would be correct if the server response was 404, but in all other cases, it would be better if the WorkItem failed to be automatically retried later. > > This patch changes the onError lambda for these methods so that it only handles the 404 case, and in all other cases lets the RestException be thrown. This matches the behavior of most other methods in these classes. Thanks! ------------- PR Comment: https://git.openjdk.org/skara/pull/1698#issuecomment-2492361305 From erikj at openjdk.org Thu Nov 21 21:23:20 2024 From: erikj at openjdk.org (Erik Joelsson) Date: Thu, 21 Nov 2024 21:23:20 GMT Subject: Integrated: 2411: Host::user should throw when failing In-Reply-To: References: Message-ID: On Wed, 20 Nov 2024 15:12:10 GMT, Erik Joelsson wrote: > The implementations of Host::user are all currently returning Optional::empty on any error return code. This means a that a bot checking for the existence of a user encounters an error with a host, e.g. JBS, it will think the user doesn't exist and take actions based on that assumption. This would be correct if the server response was 404, but in all other cases, it would be better if the WorkItem failed to be automatically retried later. > > This patch changes the onError lambda for these methods so that it only handles the 404 case, and in all other cases lets the RestException be thrown. This matches the behavior of most other methods in these classes. This pull request has now been integrated. Changeset: 0a02ca10 Author: Erik Joelsson URL: https://git.openjdk.org/skara/commit/0a02ca10e68c966a9ae1d385ed22fea00d90ad79 Stats: 5 lines in 3 files changed: 0 ins; 0 del; 5 mod 2411: Host::user should throw when failing Reviewed-by: kcr, zsong ------------- PR: https://git.openjdk.org/skara/pull/1698 From erikj at openjdk.org Thu Nov 21 21:44:26 2024 From: erikj at openjdk.org (Erik Joelsson) Date: Thu, 21 Nov 2024 21:44:26 GMT Subject: RFR: 2415: Update GitCommits::close to make sure it terminates all processes when an exception is thrown In-Reply-To: References: Message-ID: On Thu, 21 Nov 2024 18:16:45 GMT, Zhao Song wrote: > As Pavel pointed out in the pull request of [SKARA-2410](https://bugs.openjdk.org/browse/SKARA-2410)(https://github.com/openjdk/skara/pull/1697), with current implementation , GitCommits::close may have created multiple processes and only destroy one of them. > > As Erik and Pavel suggested, we should make sure close() is called on each process. vcs/src/main/java/org/openjdk/skara/vcs/git/GitCommits.java line 147: > 145: } catch (Exception e) { > 146: if (runtimeException == null) { > 147: runtimeException = new RuntimeException(e); This converts an IOException to a RuntimeException, which I don't think is ideal. I would suggest just saving the first exception, which should be either a RuntimeException or IOException, and then add any additional exceptions to the suppressed set on the saved exception. ------------- PR Review Comment: https://git.openjdk.org/skara/pull/1699#discussion_r1852944281 From zsong at openjdk.org Thu Nov 21 22:03:41 2024 From: zsong at openjdk.org (Zhao Song) Date: Thu, 21 Nov 2024 22:03:41 GMT Subject: RFR: 2415: Update GitCommits::close to make sure it terminates all processes when an exception is thrown [v2] In-Reply-To: References: Message-ID: > As Pavel pointed out in the pull request of [SKARA-2410](https://bugs.openjdk.org/browse/SKARA-2410)(https://github.com/openjdk/skara/pull/1697), with current implementation , GitCommits::close may have created multiple processes and only destroy one of them. > > As Erik and Pavel suggested, we should make sure close() is called on each process. Zhao Song has updated the pull request incrementally with one additional commit since the last revision: review comment ------------- Changes: - all: https://git.openjdk.org/skara/pull/1699/files - new: https://git.openjdk.org/skara/pull/1699/files/f1d39f11..2a9da277 Webrevs: - full: https://webrevs.openjdk.org/?repo=skara&pr=1699&range=01 - incr: https://webrevs.openjdk.org/?repo=skara&pr=1699&range=00-01 Stats: 11 lines in 1 file changed: 4 ins; 0 del; 7 mod Patch: https://git.openjdk.org/skara/pull/1699.diff Fetch: git fetch https://git.openjdk.org/skara.git pull/1699/head:pull/1699 PR: https://git.openjdk.org/skara/pull/1699 From erikj at openjdk.org Fri Nov 22 00:20:39 2024 From: erikj at openjdk.org (Erik Joelsson) Date: Fri, 22 Nov 2024 00:20:39 GMT Subject: RFR: 2415: Update GitCommits::close to make sure it terminates all processes when an exception is thrown [v2] In-Reply-To: References: Message-ID: On Thu, 21 Nov 2024 22:03:41 GMT, Zhao Song wrote: >> As Pavel pointed out in the pull request of [SKARA-2410](https://bugs.openjdk.org/browse/SKARA-2410)(https://github.com/openjdk/skara/pull/1697), with current implementation , GitCommits::close may have created multiple processes and only destroy one of them. >> >> As Erik and Pavel suggested, we should make sure close() is called on each process. > > Zhao Song has updated the pull request incrementally with one additional commit since the last revision: > > review comment Marked as reviewed by erikj (Lead). ------------- PR Review: https://git.openjdk.org/skara/pull/1699#pullrequestreview-2453049482 From prappo at openjdk.org Fri Nov 22 12:06:45 2024 From: prappo at openjdk.org (Pavel Rappo) Date: Fri, 22 Nov 2024 12:06:45 GMT Subject: RFR: 2415: Update GitCommits::close to make sure it terminates all processes when an exception is thrown [v2] In-Reply-To: References: Message-ID: On Thu, 21 Nov 2024 22:03:41 GMT, Zhao Song wrote: >> As Pavel pointed out in the pull request of [SKARA-2410](https://bugs.openjdk.org/browse/SKARA-2410)(https://github.com/openjdk/skara/pull/1697), with current implementation , GitCommits::close may have created multiple processes and only destroy one of them. >> >> As Erik and Pavel suggested, we should make sure close() is called on each process. > > Zhao Song has updated the pull request incrementally with one additional commit since the last revision: > > review comment vcs/src/main/java/org/openjdk/skara/vcs/git/GitCommits.java line 145: > 143: try { > 144: close(p, command); > 145: } catch (IOException | RuntimeException e) { What kind of runtime exception are you expecting here? ------------- PR Review Comment: https://git.openjdk.org/skara/pull/1699#discussion_r1853813660 From erikj at openjdk.org Fri Nov 22 14:01:40 2024 From: erikj at openjdk.org (Erik Joelsson) Date: Fri, 22 Nov 2024 14:01:40 GMT Subject: RFR: 2415: Update GitCommits::close to make sure it terminates all processes when an exception is thrown [v2] In-Reply-To: References: Message-ID: <70Ms53KQ9sHpbDW9KepV9dyRrBJOd9fm8hYwBgtwg1o=.cc1a545b-bdef-46b8-80ad-65609e6ff487@github.com> On Fri, 22 Nov 2024 12:04:36 GMT, Pavel Rappo wrote: >> Zhao Song has updated the pull request incrementally with one additional commit since the last revision: >> >> review comment > > vcs/src/main/java/org/openjdk/skara/vcs/git/GitCommits.java line 145: > >> 143: try { >> 144: close(p, command); >> 145: } catch (IOException | RuntimeException e) { > > What kind of runtime exception are you expecting here? I suggested catching it mostly to be defensive. The Skara code base uses RuntimeExceptions quite a lot. In this case, there is no obvious place where one would be thrown though ------------- PR Review Comment: https://git.openjdk.org/skara/pull/1699#discussion_r1853959016 From prappo at openjdk.org Fri Nov 22 14:55:46 2024 From: prappo at openjdk.org (Pavel Rappo) Date: Fri, 22 Nov 2024 14:55:46 GMT Subject: RFR: 2415: Update GitCommits::close to make sure it terminates all processes when an exception is thrown [v2] In-Reply-To: <70Ms53KQ9sHpbDW9KepV9dyRrBJOd9fm8hYwBgtwg1o=.cc1a545b-bdef-46b8-80ad-65609e6ff487@github.com> References: <70Ms53KQ9sHpbDW9KepV9dyRrBJOd9fm8hYwBgtwg1o=.cc1a545b-bdef-46b8-80ad-65609e6ff487@github.com> Message-ID: On Fri, 22 Nov 2024 13:59:30 GMT, Erik Joelsson wrote: > I suggested catching it mostly to be defensive. The Skara code base uses RuntimeExceptions quite a lot. In this case, there is no obvious place where one would be thrown though I've had a look at those; I see what you mean. A separate nit for later is calling those exceptions _errors_; this is not conventional in Java: class GitHubApplicationError extends RuntimeException "Error" strongly suggests a subclass of `java.lang.Error`. If you need some other word, but "Exception" is undesirable, you may want to call it "Failure". ------------- PR Review Comment: https://git.openjdk.org/skara/pull/1699#discussion_r1854036589 From zsong at openjdk.org Fri Nov 22 18:27:47 2024 From: zsong at openjdk.org (Zhao Song) Date: Fri, 22 Nov 2024 18:27:47 GMT Subject: RFR: 2415: Update GitCommits::close to make sure it terminates all processes when an exception is thrown [v2] In-Reply-To: References: Message-ID: <8yJXtnAUd2KOdC0dTsgLu8L12uOLHEz9xUY5dboe8qE=.071de502-0483-4c56-b67f-45051a0da8ae@github.com> On Thu, 21 Nov 2024 22:03:41 GMT, Zhao Song wrote: >> As Pavel pointed out in the pull request of [SKARA-2410](https://bugs.openjdk.org/browse/SKARA-2410)(https://github.com/openjdk/skara/pull/1697), with current implementation , GitCommits::close may have created multiple processes and only destroy one of them. >> >> As Erik and Pavel suggested, we should make sure close() is called on each process. > > Zhao Song has updated the pull request incrementally with one additional commit since the last revision: > > review comment Thank you for the review! ------------- PR Comment: https://git.openjdk.org/skara/pull/1699#issuecomment-2494490619 From zsong at openjdk.org Fri Nov 22 18:27:47 2024 From: zsong at openjdk.org (Zhao Song) Date: Fri, 22 Nov 2024 18:27:47 GMT Subject: RFR: 2415: Update GitCommits::close to make sure it terminates all processes when an exception is thrown [v2] In-Reply-To: References: <70Ms53KQ9sHpbDW9KepV9dyRrBJOd9fm8hYwBgtwg1o=.cc1a545b-bdef-46b8-80ad-65609e6ff487@github.com> Message-ID: On Fri, 22 Nov 2024 14:53:37 GMT, Pavel Rappo wrote: >> I suggested catching it mostly to be defensive. The Skara code base uses RuntimeExceptions quite a lot. In this case, there is no obvious place where one would be thrown though > >> I suggested catching it mostly to be defensive. The Skara code base uses RuntimeExceptions quite a lot. In this case, there is no obvious place where one would be thrown though > > I've had a look at those; I see what you mean. > > A separate nit for later is calling those exceptions _errors_; this is not conventional in Java: > > class GitHubApplicationError extends RuntimeException > > "Error" strongly suggests a subclass of `java.lang.Error`. If you need some other word, but "Exception" is undesirable, you may want to call it "Failure". I think we can improve it later, but not in this pr. ------------- PR Review Comment: https://git.openjdk.org/skara/pull/1699#discussion_r1854464062 From zsong at openjdk.org Fri Nov 22 18:27:47 2024 From: zsong at openjdk.org (Zhao Song) Date: Fri, 22 Nov 2024 18:27:47 GMT Subject: Integrated: 2415: Update GitCommits::close to make sure it terminates all processes when an exception is thrown In-Reply-To: References: Message-ID: On Thu, 21 Nov 2024 18:16:45 GMT, Zhao Song wrote: > As Pavel pointed out in the pull request of [SKARA-2410](https://bugs.openjdk.org/browse/SKARA-2410)(https://github.com/openjdk/skara/pull/1697), with current implementation , GitCommits::close may have created multiple processes and only destroy one of them. > > As Erik and Pavel suggested, we should make sure close() is called on each process. This pull request has now been integrated. Changeset: a42517c6 Author: Zhao Song URL: https://git.openjdk.org/skara/commit/a42517c63e3a7489629a81aafd96e0cfe290ea9a Stats: 19 lines in 1 file changed: 18 ins; 0 del; 1 mod 2415: Update GitCommits::close to make sure it terminates all processes when an exception is thrown Reviewed-by: erikj ------------- PR: https://git.openjdk.org/skara/pull/1699 From prappo at openjdk.org Fri Nov 22 19:29:24 2024 From: prappo at openjdk.org (Pavel Rappo) Date: Fri, 22 Nov 2024 19:29:24 GMT Subject: RFR: 2415: Update GitCommits::close to make sure it terminates all processes when an exception is thrown [v2] In-Reply-To: References: <70Ms53KQ9sHpbDW9KepV9dyRrBJOd9fm8hYwBgtwg1o=.cc1a545b-bdef-46b8-80ad-65609e6ff487@github.com> Message-ID: On Fri, 22 Nov 2024 18:25:26 GMT, Zhao Song wrote: >>> I suggested catching it mostly to be defensive. The Skara code base uses RuntimeExceptions quite a lot. In this case, there is no obvious place where one would be thrown though >> >> I've had a look at those; I see what you mean. >> >> A separate nit for later is calling those exceptions _errors_; this is not conventional in Java: >> >> class GitHubApplicationError extends RuntimeException >> >> "Error" strongly suggests a subclass of `java.lang.Error`. If you need some other word, but "Exception" is undesirable, you may want to call it "Failure". > > I think we can improve it later, but not in this pr. I see you have already integrated the PR; okay. Erik, I couldn't see where those Skara-specific runtime exceptions could be thrown from in this particular case, but maybe you see it as some sort of future proofing? ------------- PR Review Comment: https://git.openjdk.org/skara/pull/1699#discussion_r1854545624 From erikj at openjdk.org Fri Nov 22 21:58:44 2024 From: erikj at openjdk.org (Erik Joelsson) Date: Fri, 22 Nov 2024 21:58:44 GMT Subject: RFR: 2415: Update GitCommits::close to make sure it terminates all processes when an exception is thrown [v2] In-Reply-To: References: <70Ms53KQ9sHpbDW9KepV9dyRrBJOd9fm8hYwBgtwg1o=.cc1a545b-bdef-46b8-80ad-65609e6ff487@github.com> Message-ID: On Fri, 22 Nov 2024 19:27:04 GMT, Pavel Rappo wrote: >> I think we can improve it later, but not in this pr. > > I see you have already integrated the PR; okay. > > Erik, I couldn't see where those Skara-specific runtime exceptions could be thrown from in this particular case, but maybe you see it as some sort of future proofing? > Erik, I couldn't see where those Skara-specific runtime exceptions could be thrown from in this particular case, but maybe you see it as some sort of future proofing? > In this case, there is no obvious place where one would be thrown though I meant that there is no explicit throwing of RuntimeException in this code, but if we would ever get an NPE or similar, I think we should still make sure all processes are killed. ------------- PR Review Comment: https://git.openjdk.org/skara/pull/1699#discussion_r1854773270