From inakonechnyy at openjdk.org Tue Nov 1 22:03:59 2022 From: inakonechnyy at openjdk.org (Ilarion Nakonechnyy) Date: Tue, 1 Nov 2022 22:03:59 GMT Subject: [crac] RFR: Draft: Report checkpoint processing to jcmd Message-ID: Continuation of PR#10 ------------- Commit messages: - Merge branch 'crac' into crac-jcmd - Rename stream to jcmd as the only possible - Fix another whitespace problem - fixup! Fix style a bit - Print whole exception to jcmd, as for API - fixup! Refactor a bit around linux attach - Fix assert on checkpointRestore API call - fixup! Fix style a bit - Refactor a bit around linux attach op - Fix style a bit - ... and 40 more: https://git.openjdk.org/crac/compare/217d5bc6...f11cd65a Changes: https://git.openjdk.org/crac/pull/33/files Webrev: https://webrevs.openjdk.org/?repo=crac&pr=33&range=00 Stats: 411 lines in 11 files changed: 257 ins; 96 del; 58 mod Patch: https://git.openjdk.org/crac/pull/33.diff Fetch: git fetch https://git.openjdk.org/crac pull/33/head:pull/33 PR: https://git.openjdk.org/crac/pull/33 From inakonechnyy at openjdk.org Wed Nov 2 11:35:14 2022 From: inakonechnyy at openjdk.org (Ilarion Nakonechnyy) Date: Wed, 2 Nov 2022 11:35:14 GMT Subject: [crac] RFR: Report checkpoint processing to jcmd [v26] In-Reply-To: References: Message-ID: > pass output stream from diagnosticCommand.cpp through java code into os_linux.cpp::VM_crac::doit() Ilarion Nakonechnyy has updated the pull request with a new target base due to a merge or a rebase. The incremental webrev excludes the unrelated changes brought in by the merge/rebase. The pull request contains 50 additional commits since the last revision: - Merge branch 'pr-10' into crac - Rename stream to jcmd as the only possible - Fix another whitespace problem - fixup! Fix style a bit - Print whole exception to jcmd, as for API - fixup! Refactor a bit around linux attach - Fix assert on checkpointRestore API call - fixup! Fix style a bit - Refactor a bit around linux attach op * narrow the interface of LinuxAttachListener and make it more consistent * only LinuxAttachOp can float on Linux * rename to current_op - Fix style a bit - ... and 40 more: https://git.openjdk.org/crac/compare/217d5bc6...72b8cdb2 ------------- Changes: - all: https://git.openjdk.org/crac/pull/10/files - new: https://git.openjdk.org/crac/pull/10/files/2dc03844..72b8cdb2 Webrevs: - full: https://webrevs.openjdk.org/?repo=crac&pr=10&range=25 - incr: https://webrevs.openjdk.org/?repo=crac&pr=10&range=24-25 Stats: 2956 lines in 68 files changed: 2581 ins; 186 del; 189 mod Patch: https://git.openjdk.org/crac/pull/10.diff Fetch: git fetch https://git.openjdk.org/crac pull/10/head:pull/10 PR: https://git.openjdk.org/crac/pull/10 From akozlov at azul.com Wed Nov 2 15:28:03 2022 From: akozlov at azul.com (Anton Kozlov) Date: Wed, 2 Nov 2022 17:28:03 +0200 Subject: CRac example usage In-Reply-To: References: Message-ID: <9b70abc4-0e0e-28b3-a2da-a5e3b8d3c344@azul.com> Hi Gabriele, On 10/10/22 16:29, Gabriele Cardosi wrote: > I'm trying to run the spring-boot example from the github repo. > I tried with the JDK 17 build, but it did not work due to the "java.base does not 'opens java.lang'" issue. In general, this mail list is not a good place for this discussion, github issues would be a better place. > Then I tried with the JDK 14 build, but this time I have the "jdk.crac.impl.CheckpointOpenFileException: /var/lib/sss/mc/passwd" exception. > I also tried with?the workaround explained here (https://mail.openjdk.org/pipermail/crac-dev/2022-January/000079.html ) but it did not work on my machine (RHEL 8.6). Could you provide more details about what you did and how does the output look? Because I'd expect the workaround to work. Thanks, Anton From gcardosi at redhat.com Wed Nov 2 15:39:33 2022 From: gcardosi at redhat.com (Gabriele Cardosi) Date: Wed, 2 Nov 2022 16:39:33 +0100 Subject: CRac example usage In-Reply-To: <9b70abc4-0e0e-28b3-a2da-a5e3b8d3c344@azul.com> References: <9b70abc4-0e0e-28b3-a2da-a5e3b8d3c344@azul.com> Message-ID: Hi Anton, here's the log jdk.crac.impl.CheckpointOpenFileException: /var/lib/sss/mc/passwd at java.base/jdk.crac.Core.translateJVMExceptions(Core.java:76) at java.base/jdk.crac.Core.checkpointRestore1(Core.java:137) at java.base/jdk.crac.Core.checkpointRestore(Core.java:177) at java.base/jdk.crac.Core.lambda$checkpointRestoreInternal$0(Core.java:194) at java.base/java.lang.Thread.run(Thread.java:832) By the way, I talked of this with the original proposer of the workaround, and he also confirmed the same issue on rhel 8.6 There should be something slightly different, but I could not understand exactly what. Best Gabriele On Wed, Nov 2, 2022 at 4:28 PM Anton Kozlov wrote: > Hi Gabriele, > > On 10/10/22 16:29, Gabriele Cardosi wrote: > > I'm trying to run the spring-boot example from the github repo. > > I tried with the JDK 17 build, but it did not work due to the "java.base > does not 'opens java.lang'" issue. > > In general, this mail list is not a good place for this discussion, github > issues would be a better place. > > > Then I tried with the JDK 14 build, but this time I have the > "jdk.crac.impl.CheckpointOpenFileException: /var/lib/sss/mc/passwd" > exception. > > I also tried with the workaround explained here ( > https://mail.openjdk.org/pipermail/crac-dev/2022-January/000079.html < > https://mail.openjdk.org/pipermail/crac-dev/2022-January/000079.html>) > but it did not work on my machine (RHEL 8.6). > > Could you provide more details about what you did and how does the output > look? Because I'd expect the workaround to work. > > Thanks, > Anton > > -- GABRIELE CARDOSI SENIOR SOFTWARE ENGINEERS, MW Red Hat Ltd gcardosi at redhat.com M: +39-3461717132 -------------- next part -------------- An HTML attachment was scrubbed... URL: From jkratochvil at azul.com Wed Nov 2 19:25:01 2022 From: jkratochvil at azul.com (Jan Kratochvil) Date: Wed, 2 Nov 2022 20:25:01 +0100 Subject: CRac example usage In-Reply-To: References: <9b70abc4-0e0e-28b3-a2da-a5e3b8d3c344@azul.com> Message-ID: On Wed, 02 Nov 2022 16:39:33 +0100, Gabriele Cardosi wrote: > There should be something slightly different, but I could not understand > exactly what. The original workaround https://mail.openjdk.org/pipermail/crac-dev/2022-January/000079.html passwd: sss files systemd group: sss files systemd -> passwd: files sss systemd group: files sss systemd was running CRaC under root: $ sudo ./build/linux-x86_64-server-slowdebug/images/jdk/bin/java ... #6 0x00007ffff5e960eb in get_user_name (uid=0) at and therefore the root account was found in local /etc/passwd file. If you run it without sudo and your non-root user is only available by SSS (NIS?) the workaround will not work. Jan From akozlov at azul.com Thu Nov 3 06:36:22 2022 From: akozlov at azul.com (Anton Kozlov) Date: Thu, 3 Nov 2022 08:36:22 +0200 Subject: CRac example usage In-Reply-To: References: <9b70abc4-0e0e-28b3-a2da-a5e3b8d3c344@azul.com> Message-ID: <5e9a53dc-1d4e-a894-6270-e449eab68a05@azul.com> On 11/2/22 21:25, Jan Kratochvil wrote: > https://mail.openjdk.org/pipermail/crac-dev/2022-January/000079.html > passwd: sss files systemd > group: sss files systemd > -> > passwd: files sss systemd > group: files sss systemd Thanks for the link. Does it help to remove "sss" from the list? Thanks, Anton From gcardosi at redhat.com Thu Nov 3 10:25:16 2022 From: gcardosi at redhat.com (Gabriele Cardosi) Date: Thu, 3 Nov 2022 11:25:16 +0100 Subject: CRac example usage In-Reply-To: <5e9a53dc-1d4e-a894-6270-e449eab68a05@azul.com> References: <9b70abc4-0e0e-28b3-a2da-a5e3b8d3c344@azul.com> <5e9a53dc-1d4e-a894-6270-e449eab68a05@azul.com> Message-ID: Hi guys, I confirm that it works with `*sudo*`, but (at least for rhel 8.6) this is what I had to do 1) create a "custom" authselect profile (I called it "crac" - see https://access.redhat.com/documentation/en-us/red_hat_enterprise_linux/8/html/configuring_authentication_and_authorization_in_rhel/configuring-user-authentication-using-authselect_configuring-authentication-and-authorization-in-rhel#creating-and-deploying-your-own-authselect-profile_configuring-user-authentication-using-authselect ) 2) load it with "*sudo authselect select custom/crac*" 3) start springboot application with "sudo" There is still something not clear to me, i.e. on rhel 8.6 the default authprofile is "sssd", so probably the following is true *"If you run it without sudo and your non-root user is only available by SSS(NIS?) the workaround will not work."* so, even creating a "custom" profile, I have to run as "sudo". Anyway, many thanks for the help, now I can keep going on with it! Bye for now On Thu, Nov 3, 2022 at 7:36 AM Anton Kozlov wrote: > On 11/2/22 21:25, Jan Kratochvil wrote: > > https://mail.openjdk.org/pipermail/crac-dev/2022-January/000079.html > > passwd: sss files systemd > > group: sss files systemd > > -> > > passwd: files sss systemd > > group: files sss systemd > Thanks for the link. > > Does it help to remove "sss" from the list? > > Thanks, > Anton > > -- GABRIELE CARDOSI SENIOR SOFTWARE ENGINEERS, MW Red Hat Ltd gcardosi at redhat.com M: +39-3461717132 -------------- next part -------------- An HTML attachment was scrubbed... URL: From inakonechnyy at openjdk.org Thu Nov 3 14:22:00 2022 From: inakonechnyy at openjdk.org (Ilarion Nakonechnyy) Date: Thu, 3 Nov 2022 14:22:00 GMT Subject: [crac] RFR: Report checkpoint processing to jcmd [v27] In-Reply-To: References: Message-ID: <54DxHIGCQP_3SVaEMW7qYX6mcqX_PBqjaRi2zximGMQ=.1b56655c-7c8e-43ca-b631-b76d9063d454@github.com> > pass output stream from diagnosticCommand.cpp through java code into os_linux.cpp::VM_crac::doit() Ilarion Nakonechnyy has updated the pull request incrementally with one additional commit since the last revision: Print exceptions to tty if jcmd reporting is 'completed' ------------- Changes: - all: https://git.openjdk.org/crac/pull/10/files - new: https://git.openjdk.org/crac/pull/10/files/72b8cdb2..29365804 Webrevs: - full: https://webrevs.openjdk.org/?repo=crac&pr=10&range=26 - incr: https://webrevs.openjdk.org/?repo=crac&pr=10&range=25-26 Stats: 13 lines in 3 files changed: 9 ins; 0 del; 4 mod Patch: https://git.openjdk.org/crac/pull/10.diff Fetch: git fetch https://git.openjdk.org/crac pull/10/head:pull/10 PR: https://git.openjdk.org/crac/pull/10 From inakonechnyy at openjdk.org Thu Nov 3 14:40:15 2022 From: inakonechnyy at openjdk.org (Ilarion Nakonechnyy) Date: Thu, 3 Nov 2022 14:40:15 GMT Subject: [crac] RFR: Report checkpoint processing to jcmd [v21] In-Reply-To: References: Message-ID: On Mon, 15 Aug 2022 11:31:29 GMT, Anton Kozlov wrote: >> A straightforward implementation of this assertion leads to triggering it at restore. >> Seems like `LinuxAttachListener::write_fully()` doesnt reset the buffer_pos in `bufferedStream`, so the `bufferedStream->size() `is non-zero. >> >> Probably, it worth to add `bufferedStream->reset()` to `LinuxAttachOperation::write_operation_result()` > > I believe the data there is the text I observe with the CRAllowToSkipCheckpoint. So the root casue should be fixed (the text that won't be shown). Looks like the issue with this assertion was fixed with the commit 29365804b4be9d0cf41a4feaf5ac46fe823d4fba ------------- PR: https://git.openjdk.org/crac/pull/10 From inakonechnyy at openjdk.org Thu Nov 3 14:52:26 2022 From: inakonechnyy at openjdk.org (Ilarion Nakonechnyy) Date: Thu, 3 Nov 2022 14:52:26 GMT Subject: [crac] RFR: Report checkpoint processing to jcmd [v28] In-Reply-To: References: Message-ID: > pass output stream from diagnosticCommand.cpp through java code into os_linux.cpp::VM_crac::doit() Ilarion Nakonechnyy has updated the pull request incrementally with one additional commit since the last revision: Correct magic number ------------- Changes: - all: https://git.openjdk.org/crac/pull/10/files - new: https://git.openjdk.org/crac/pull/10/files/29365804..f6a2dbd1 Webrevs: - full: https://webrevs.openjdk.org/?repo=crac&pr=10&range=27 - incr: https://webrevs.openjdk.org/?repo=crac&pr=10&range=26-27 Stats: 1 line in 1 file changed: 0 ins; 0 del; 1 mod Patch: https://git.openjdk.org/crac/pull/10.diff Fetch: git fetch https://git.openjdk.org/crac pull/10/head:pull/10 PR: https://git.openjdk.org/crac/pull/10 From inakonechnyy at openjdk.org Thu Nov 3 15:21:36 2022 From: inakonechnyy at openjdk.org (Ilarion Nakonechnyy) Date: Thu, 3 Nov 2022 15:21:36 GMT Subject: [crac] RFR: Report checkpoint processing to jcmd [v29] In-Reply-To: References: Message-ID: > pass output stream from diagnosticCommand.cpp through java code into os_linux.cpp::VM_crac::doit() Ilarion Nakonechnyy has updated the pull request incrementally with one additional commit since the last revision: Use stringStream for printing ------------- Changes: - all: https://git.openjdk.org/crac/pull/10/files - new: https://git.openjdk.org/crac/pull/10/files/f6a2dbd1..9e165dd6 Webrevs: - full: https://webrevs.openjdk.org/?repo=crac&pr=10&range=28 - incr: https://webrevs.openjdk.org/?repo=crac&pr=10&range=27-28 Stats: 6 lines in 1 file changed: 2 ins; 2 del; 2 mod Patch: https://git.openjdk.org/crac/pull/10.diff Fetch: git fetch https://git.openjdk.org/crac pull/10/head:pull/10 PR: https://git.openjdk.org/crac/pull/10 From inakonechnyy at openjdk.org Thu Nov 3 15:35:35 2022 From: inakonechnyy at openjdk.org (Ilarion Nakonechnyy) Date: Thu, 3 Nov 2022 15:35:35 GMT Subject: [crac] RFR: Report checkpoint processing to jcmd [v30] In-Reply-To: References: Message-ID: > pass output stream from diagnosticCommand.cpp through java code into os_linux.cpp::VM_crac::doit() Ilarion Nakonechnyy has updated the pull request incrementally with one additional commit since the last revision: getter renamed ------------- Changes: - all: https://git.openjdk.org/crac/pull/10/files - new: https://git.openjdk.org/crac/pull/10/files/9e165dd6..640671e7 Webrevs: - full: https://webrevs.openjdk.org/?repo=crac&pr=10&range=29 - incr: https://webrevs.openjdk.org/?repo=crac&pr=10&range=28-29 Stats: 2 lines in 2 files changed: 0 ins; 0 del; 2 mod Patch: https://git.openjdk.org/crac/pull/10.diff Fetch: git fetch https://git.openjdk.org/crac pull/10/head:pull/10 PR: https://git.openjdk.org/crac/pull/10 From inakonechnyy at openjdk.org Thu Nov 3 15:52:26 2022 From: inakonechnyy at openjdk.org (Ilarion Nakonechnyy) Date: Thu, 3 Nov 2022 15:52:26 GMT Subject: [crac] RFR: Report checkpoint processing to jcmd [v31] In-Reply-To: References: Message-ID: > pass output stream from diagnosticCommand.cpp through java code into os_linux.cpp::VM_crac::doit() Ilarion Nakonechnyy has updated the pull request incrementally with one additional commit since the last revision: Style fix ------------- Changes: - all: https://git.openjdk.org/crac/pull/10/files - new: https://git.openjdk.org/crac/pull/10/files/640671e7..6a56411a Webrevs: - full: https://webrevs.openjdk.org/?repo=crac&pr=10&range=30 - incr: https://webrevs.openjdk.org/?repo=crac&pr=10&range=29-30 Stats: 7 lines in 1 file changed: 2 ins; 5 del; 0 mod Patch: https://git.openjdk.org/crac/pull/10.diff Fetch: git fetch https://git.openjdk.org/crac pull/10/head:pull/10 PR: https://git.openjdk.org/crac/pull/10 From inakonechnyy at openjdk.org Thu Nov 3 15:57:52 2022 From: inakonechnyy at openjdk.org (Ilarion Nakonechnyy) Date: Thu, 3 Nov 2022 15:57:52 GMT Subject: [crac] Withdrawn: Draft: Report checkpoint processing to jcmd In-Reply-To: References: Message-ID: On Tue, 1 Nov 2022 21:56:52 GMT, Ilarion Nakonechnyy wrote: > Continuation of PR#10 This pull request has been closed without being integrated. ------------- PR: https://git.openjdk.org/crac/pull/33 From akozlov at openjdk.org Fri Nov 4 09:32:12 2022 From: akozlov at openjdk.org (Anton Kozlov) Date: Fri, 4 Nov 2022 09:32:12 GMT Subject: [crac] RFR: Report checkpoint processing to jcmd [v31] In-Reply-To: References: Message-ID: On Thu, 3 Nov 2022 15:52:26 GMT, Ilarion Nakonechnyy wrote: >> pass output stream from diagnosticCommand.cpp through java code into os_linux.cpp::VM_crac::doit() > > Ilarion Nakonechnyy has updated the pull request incrementally with one additional commit since the last revision: > > Style fix Marked as reviewed by akozlov (Lead). src/hotspot/share/services/diagnosticCommand.cpp line 1062: > 1060: tty->print("%s", output_buf.as_string()); > 1061: } else { > 1062: output()->print("%s", output_buf.as_string()); Nit-picking: it looks like you may avoid the intermediate buffer by choosing a stream in advance outputStream stream = ... ? output() : tty stream->println("An exception during a checkpoint operation:"); stream->println("%s", out); ------------- PR: https://git.openjdk.org/crac/pull/10 From akozlov at openjdk.org Fri Nov 4 09:37:53 2022 From: akozlov at openjdk.org (Anton Kozlov) Date: Fri, 4 Nov 2022 09:37:53 GMT Subject: [crac] RFR: Report checkpoint processing to jcmd [v31] In-Reply-To: References: Message-ID: On Thu, 3 Nov 2022 15:52:26 GMT, Ilarion Nakonechnyy wrote: >> pass output stream from diagnosticCommand.cpp through java code into os_linux.cpp::VM_crac::doit() > > Ilarion Nakonechnyy has updated the pull request incrementally with one additional commit since the last revision: > > Style fix I've played with the most recent version of this change and it works as expected ? Interesting that VM messages go to the jcmd stream, and probably we may want Resource to be able to log to that stream as well. It may be a follow-up for this change. BTW, have you found the reason the submit test has failed? ------------- PR: https://git.openjdk.org/crac/pull/10 From inakonechnyy at openjdk.org Mon Nov 7 12:17:15 2022 From: inakonechnyy at openjdk.org (Ilarion Nakonechnyy) Date: Mon, 7 Nov 2022 12:17:15 GMT Subject: [crac] RFR: Report checkpoint processing to jcmd [v31] In-Reply-To: References: Message-ID: <1irt7mY3DLZ85rwsFoYJZF4VnQhZEfZtAUV3KJdWSCQ=.94f9eba4-d21d-491f-bca8-4f92042df2c0@github.com> On Thu, 3 Nov 2022 15:52:26 GMT, Ilarion Nakonechnyy wrote: >> pass output stream from diagnosticCommand.cpp through java code into os_linux.cpp::VM_crac::doit() > > Ilarion Nakonechnyy has updated the pull request incrementally with one additional commit since the last revision: > > Style fix No, I didn't find the reason yet Referred to this link ( that passed ok )it looks like tests hs/tier1_serviceability is unstable for some reason https://github.com/Larry-N/crac/actions/runs/3387045467 ------------- PR: https://git.openjdk.org/crac/pull/10 From akozlov at openjdk.org Mon Nov 7 14:41:05 2022 From: akozlov at openjdk.org (Anton Kozlov) Date: Mon, 7 Nov 2022 14:41:05 GMT Subject: [crac] RFR: Report checkpoint processing to jcmd [v31] In-Reply-To: References: Message-ID: On Thu, 3 Nov 2022 15:52:26 GMT, Ilarion Nakonechnyy wrote: >> pass output stream from diagnosticCommand.cpp through java code into os_linux.cpp::VM_crac::doit() > > Ilarion Nakonechnyy has updated the pull request incrementally with one additional commit since the last revision: > > Style fix OK, I see it has passed. ------------- PR: https://git.openjdk.org/crac/pull/10 From inakonechnyy at openjdk.org Mon Nov 7 15:12:23 2022 From: inakonechnyy at openjdk.org (Ilarion Nakonechnyy) Date: Mon, 7 Nov 2022 15:12:23 GMT Subject: [crac] RFR: Report checkpoint processing to jcmd [v32] In-Reply-To: References: Message-ID: > pass output stream from diagnosticCommand.cpp through java code into os_linux.cpp::VM_crac::doit() Ilarion Nakonechnyy has updated the pull request incrementally with one additional commit since the last revision: Replace buffer with stream ------------- Changes: - all: https://git.openjdk.org/crac/pull/10/files - new: https://git.openjdk.org/crac/pull/10/files/6a56411a..94567c51 Webrevs: - full: https://webrevs.openjdk.org/?repo=crac&pr=10&range=31 - incr: https://webrevs.openjdk.org/?repo=crac&pr=10&range=30-31 Stats: 7 lines in 1 file changed: 0 ins; 4 del; 3 mod Patch: https://git.openjdk.org/crac/pull/10.diff Fetch: git fetch https://git.openjdk.org/crac pull/10/head:pull/10 PR: https://git.openjdk.org/crac/pull/10 From akozlov at openjdk.org Tue Nov 8 01:34:09 2022 From: akozlov at openjdk.org (Anton Kozlov) Date: Tue, 8 Nov 2022 01:34:09 GMT Subject: [crac] RFR: Report checkpoint processing to jcmd [v32] In-Reply-To: References: Message-ID: <8fO5M75UxpkfNDd_MFAi-TPwPR_lVkLRN0_fq4y81co=.fece51d4-7628-4b1e-987c-557e5d283da7@github.com> On Mon, 7 Nov 2022 15:12:23 GMT, Ilarion Nakonechnyy wrote: >> pass output stream from diagnosticCommand.cpp through java code into os_linux.cpp::VM_crac::doit() > > Ilarion Nakonechnyy has updated the pull request incrementally with one additional commit since the last revision: > > Replace buffer with stream Still looks fine. ------------- Marked as reviewed by akozlov (Lead). PR: https://git.openjdk.org/crac/pull/10 From akozlov at openjdk.org Thu Nov 10 15:40:10 2022 From: akozlov at openjdk.org (Anton Kozlov) Date: Thu, 10 Nov 2022 15:40:10 GMT Subject: [crac] RFR: Update Reference Handling for CRaC [v3] In-Reply-To: References: Message-ID: On Mon, 25 Apr 2022 17:59:57 GMT, Anton Kozlov wrote: >> This change updates Reference Handling after Alan's comments [1]. >> * The new API is moved to the CRaC-related classes to avoid polluting or changing standard classes. This should make EA builds more attractive. >> * The method for waiting threads now accepts timeout, as turns to be needed by CleanerImpl.beforeCheckpoint(). >> * Now reference handling outside of the JDK code is supported, see the supplied test update. The handing does not depend on the first-level reference processing thread's beforeCheckpoint is called first. After that, it was possible to remove the resource and the corresponding REFERENCE_HANDLER priority. >> >> The methods for waiting threads cannot guarantee that a reference handling is complete for a particular queue and a set of threads, as nothing prevents an another concurrent thread to change the reachability of a random object that will end up in the queue after the method returns. The method is designed to synchronize reference handling and checkpoint. That is, to ensure that objects that are on the way to be enqueued are indeed enqueued and corresponding clean-up is performed, as demonstrated by the test. >> >> [1] https://github.com/openjdk/crac/pull/13#issuecomment-1028024855 > > Anton Kozlov has updated the pull request incrementally with one additional commit since the last revision: > > Make jdk.crac.Misc final I'm going to close this with without intergration, see #34 ------------- PR: https://git.openjdk.org/crac/pull/22 From akozlov at openjdk.org Thu Nov 10 15:42:31 2022 From: akozlov at openjdk.org (Anton Kozlov) Date: Thu, 10 Nov 2022 15:42:31 GMT Subject: [crac] RFR: Backout new API to sync with Reference Handler Message-ID: This reverts commit 9cf1995693eead85d3807fb4c83ab38c14e27042 and makes #22 obsolete. The API introduced in 9cf1995693eead85d3807fb4c83ab38c14e27042 (waitForWaiters) and changed in #22 waits for the state when all discovered references are processed. So WaitForWaiters is used to implement predictable Reference Handling, ensuring that clean-up actions have fired for an object after it becomes unreachable. I think that API was a mistake and should be reverted. In general, the problem of predictable Reference Handling is independent of CRaC. So I thought about extracting that out of CRaC and found a few issues with the approach. A user needs to know what RefQueue gets References after an object becomes unreachable, to call waitForWaiters on that queue. The queue is not necessarily evident, so a deep understanding of refs and queues in an application is required to select the proper queue to wait on, and to build the right order of them to wait on. Also, it's required somehow to know the number of threads servicing a queue. And there are situations when waitForWaiters may report that all refs are processed, but some of them are not -- consider a thread that is polling a queue and gets refs to be processed but then buffers them in another queue for later, in this example waitForWaiters does not provide the guarantee that corresponding clean-up actions were performed. The common and more straightforward way to have predictable clean-up is to call an explicit method like close()/release()/cleanup() that performs object-specific clean-up actions predictably. ------------- Commit messages: - Backout new API to sync with Reference Handler Changes: https://git.openjdk.org/crac/pull/34/files Webrev: https://webrevs.openjdk.org/?repo=crac&pr=34&range=00 Stats: 130 lines in 5 files changed: 0 ins; 128 del; 2 mod Patch: https://git.openjdk.org/crac/pull/34.diff Fetch: git fetch https://git.openjdk.org/crac pull/34/head:pull/34 PR: https://git.openjdk.org/crac/pull/34 From akozlov at openjdk.org Thu Nov 10 16:00:05 2022 From: akozlov at openjdk.org (Anton Kozlov) Date: Thu, 10 Nov 2022 16:00:05 GMT Subject: [crac] Withdrawn: Update Reference Handling for CRaC In-Reply-To: References: Message-ID: On Mon, 18 Apr 2022 18:15:13 GMT, Anton Kozlov wrote: > This change updates Reference Handling after Alan's comments [1]. > * The new API is moved to the CRaC-related classes to avoid polluting or changing standard classes. This should make EA builds more attractive. > * The method for waiting threads now accepts timeout, as turns to be needed by CleanerImpl.beforeCheckpoint(). > * Now reference handling outside of the JDK code is supported, see the supplied test update. The handing does not depend on the first-level reference processing thread's beforeCheckpoint is called first. After that, it was possible to remove the resource and the corresponding REFERENCE_HANDLER priority. > > The methods for waiting threads cannot guarantee that a reference handling is complete for a particular queue and a set of threads, as nothing prevents an another concurrent thread to change the reachability of a random object that will end up in the queue after the method returns. The method is designed to synchronize reference handling and checkpoint. That is, to ensure that objects that are on the way to be enqueued are indeed enqueued and corresponding clean-up is performed, as demonstrated by the test. > > [1] https://github.com/openjdk/crac/pull/13#issuecomment-1028024855 This pull request has been closed without being integrated. ------------- PR: https://git.openjdk.org/crac/pull/22 From inakonechnyy at openjdk.org Thu Nov 10 16:46:59 2022 From: inakonechnyy at openjdk.org (Ilarion Nakonechnyy) Date: Thu, 10 Nov 2022 16:46:59 GMT Subject: [crac] Integrated: Report checkpoint processing to jcmd In-Reply-To: References: Message-ID: On Tue, 25 Jan 2022 15:07:55 GMT, Ilarion Nakonechnyy wrote: > pass output stream from diagnosticCommand.cpp through java code into os_linux.cpp::VM_crac::doit() This pull request has now been integrated. Changeset: fdf4eb8d Author: Ilarion Nakonechnyy Committer: Anton Kozlov URL: https://git.openjdk.org/crac/commit/fdf4eb8dd5c2a11176eb1ec8f40b3f4133a872aa Stats: 420 lines in 11 files changed: 263 ins; 100 del; 57 mod Report checkpoint processing to jcmd Reviewed-by: akozlov ------------- PR: https://git.openjdk.org/crac/pull/10 From inakonechnyy at openjdk.org Fri Nov 11 14:38:33 2022 From: inakonechnyy at openjdk.org (Ilarion Nakonechnyy) Date: Fri, 11 Nov 2022 14:38:33 GMT Subject: [crac] RFR: Force closing a "redirected to the file" standard io descriptor [v2] In-Reply-To: References: Message-ID: > CRIU restore fail after Checkpoint stdout to file. > Redirecting a stdout to the pipeline doesn't break the restore. > > A proposed approach forces the close of stdout, stdin, and stderr file descriptors if they are redirected to the files, on checkpoint resources verification. Ilarion Nakonechnyy has updated the pull request with a new target base due to a merge or a rebase. The incremental webrev excludes the unrelated changes brought in by the merge/rebase. The pull request contains six additional commits since the last revision: - Different approach, if std descriptors bind to a file, replace is with binded to pseudo terminal - Merge branch 'crac' into stdout-restore - jchek corrections - corrections - corrections - Close stdin, stderr, stdout fd before checkpoint if they are redirected to a file. ------------- Changes: - all: https://git.openjdk.org/crac/pull/24/files - new: https://git.openjdk.org/crac/pull/24/files/6cf8cb99..c3347ad3 Webrevs: - full: https://webrevs.openjdk.org/?repo=crac&pr=24&range=01 - incr: https://webrevs.openjdk.org/?repo=crac&pr=24&range=00-01 Stats: 1290 lines in 30 files changed: 961 ins; 184 del; 145 mod Patch: https://git.openjdk.org/crac/pull/24.diff Fetch: git fetch https://git.openjdk.org/crac pull/24/head:pull/24 PR: https://git.openjdk.org/crac/pull/24 From grcevski at gmail.com Mon Nov 28 22:04:42 2022 From: grcevski at gmail.com (Nikola Grcevski) Date: Mon, 28 Nov 2022 17:04:42 -0500 Subject: Unable to create snapshots with Java modules Message-ID: Hello crac-dev, I've been experimenting with the latest CRaC JDK17 build in the hopes that I can get it to run with Elasticsearch, at least a very early checkpoint before we open any sockets or files. However, I noticed that modules loaded remain open at runtime and the snapshot creation always fails. I created a small Java example to try it out: public class SimpleModule { public static void main(String[] args) throws RestoreException, CheckpointException { Core.checkpointRestore(); System.out.println("Hello modules!"); } } module simplemodule { exports co.elastic.simple; } When I run snapshot create and then restore with Java classpath, everything works as expected: ~/work/simplemodule$ //openjdk-17-crac+3_linux-x64/bin/java -cp lib/simplemodule-1.0-SNAPSHOT.jar -XX:CRaCCheckpointTo=/tmp/simple co.elastic.simple.SimpleModule CR: Checkpoint ... Killed ~/work/simplemodule$ //openjdk-17-crac+3_linux-x64/bin/java -cp lib/simplemodule-1.0-SNAPSHOT.jar -XX:CRaCRestoreFrom=/tmp/simple Hello modules! However, if I try the same thing with the modules usage, the snapshot creation fails: ~/work/simplemodule$ //openjdk-17-crac+3_linux-x64/bin/java -XX:CRaCCheckpointTo=/tmp/simple --module-path lib --module simplemodule/co.elastic.simple.SimpleModule Exception in thread "main" jdk.crac.CheckpointException at java.base/jdk.crac.Core.checkpointRestore1(Core.java:138) at java.base/jdk.crac.Core.checkpointRestore(Core.java:237) at simplemodule/co.elastic.simple.SimpleModule.main(SimpleModule.java:9) Suppressed: jdk.crac.impl.CheckpointOpenFileException: //work/simplemodule/target/simplemodule-1.0-SNAPSHOT.jar at java.base/jdk.crac.Core.translateJVMExceptions(Core.java:84) at java.base/jdk.crac.Core.checkpointRestore1(Core.java:142) ... 2 more If I check the open file descriptors, the simplemodule-1.0-SNAPSHOT.jar is open regardless of modules or not. I'd appreciate any pointers or suggestions where to look next or if there's a workaround I can use. Thanks, Nikola From akozlov at azul.com Tue Nov 29 16:11:11 2022 From: akozlov at azul.com (Anton Kozlov) Date: Tue, 29 Nov 2022 18:11:11 +0200 Subject: Unable to create snapshots with Java modules In-Reply-To: References: Message-ID: On 11/29/22 00:04, Nikola Grcevski wrote: > However, if I try the same thing with the modules usage, the snapshot > creation fails: > ~/work/simplemodule$ //openjdk-17-crac+3_linux-x64/bin/java > -XX:CRaCCheckpointTo=/tmp/simple --module-path lib --module > simplemodule/co.elastic.simple.SimpleModule Hi Nikola, I've reproduced the problem. It seems JDK misses a piece of code for modules that is there for class loaders. It looks easy to fix, I'm looking at this. Meanwhile, while you're experimenting, you may disable CheckpointException by -XX:+UnlockExperimentalVMOptions -XX:-CRDoThrowCheckpointException. Of course, it's not recommended for anything beyond experiments :) Thanks, Anton From grcevski at gmail.com Tue Nov 29 16:24:31 2022 From: grcevski at gmail.com (Nikola Grcevski) Date: Tue, 29 Nov 2022 11:24:31 -0500 Subject: Unable to create snapshots with Java modules In-Reply-To: References: Message-ID: Great, thanks! I started looking at it as well, I added arguments.cpp/hpp _java_module_path = new SystemProperty("jdk.module.path", "", true); and in os_linux.cpp do_classpaths(mark_classpath_entry, &fds, Arguments::get_appmodulepath()); I was able to make things work if I put the expanded module path directly e.g: --module-path lib/simplemodule-1.0-SNAPSHOT.jar I'm not sure if there's a helper to expand the module path fully with directories, when I tried to create a mark function that handles both directories and files, the directory scan and mark never matches on my app lib directory on this check: if (dent->d_ino == fstat->st_ino) in os_linux::static void mark_all_in(FdsInfo *fds, char* dirpath). Thanks so much for your help. Nikola On Tue, Nov 29, 2022 at 11:11 AM Anton Kozlov wrote: > > On 11/29/22 00:04, Nikola Grcevski wrote: > > However, if I try the same thing with the modules usage, the snapshot > > creation fails: > > ~/work/simplemodule$ //openjdk-17-crac+3_linux-x64/bin/java > > -XX:CRaCCheckpointTo=/tmp/simple --module-path lib --module > > simplemodule/co.elastic.simple.SimpleModule > > Hi Nikola, > > I've reproduced the problem. It seems JDK misses a piece of code for modules that is there for class loaders. It looks easy to fix, I'm looking at this. > > Meanwhile, while you're experimenting, you may disable CheckpointException by -XX:+UnlockExperimentalVMOptions -XX:-CRDoThrowCheckpointException. Of course, it's not recommended for anything beyond experiments :) > > Thanks, > Anton From grcevski at gmail.com Tue Nov 29 16:36:32 2022 From: grcevski at gmail.com (Nikola Grcevski) Date: Tue, 29 Nov 2022 11:36:32 -0500 Subject: Unable to create snapshots with Java modules In-Reply-To: References: Message-ID: I found out the issue I had with my directory, I was using a symlinked file, so the parent directory doesn't match the st_ino of the file. Perhaps there's a much better way, but I managed to get things to work with the following addition (no symlink support): static void mark_classpath_or_dir(FdsInfo *fds, char* cp) { DIR *dir = os::opendir(cp); if (!dir) { mark_classpath_entry(fds, cp); return; } os::closedir(dir); mark_all_in(fds, cp); } ... do_classpaths(mark_classpath_or_dir, &fds, Arguments::get_appmodulepath()); I also had to make a small change in do_classpaths, at the end instead of calling mark_classpath_entry, I changed it to call fn(fds, cp). On Tue, Nov 29, 2022 at 11:24 AM Nikola Grcevski wrote: > > Great, thanks! > > I started looking at it as well, I added > > arguments.cpp/hpp > _java_module_path = new SystemProperty("jdk.module.path", "", true); > > and in os_linux.cpp > do_classpaths(mark_classpath_entry, &fds, Arguments::get_appmodulepath()); > > I was able to make things work if I put the expanded module path directly e.g: > --module-path lib/simplemodule-1.0-SNAPSHOT.jar > > I'm not sure if there's a helper to expand the module path fully with > directories, when I tried > to create a mark function that handles both directories and files, the > directory scan > and mark never matches on my app lib directory on this check: > > if (dent->d_ino == fstat->st_ino) > > in os_linux::static void mark_all_in(FdsInfo *fds, char* dirpath). > > Thanks so much for your help. > Nikola > > On Tue, Nov 29, 2022 at 11:11 AM Anton Kozlov wrote: > > > > On 11/29/22 00:04, Nikola Grcevski wrote: > > > However, if I try the same thing with the modules usage, the snapshot > > > creation fails: > > > ~/work/simplemodule$ //openjdk-17-crac+3_linux-x64/bin/java > > > -XX:CRaCCheckpointTo=/tmp/simple --module-path lib --module > > > simplemodule/co.elastic.simple.SimpleModule > > > > Hi Nikola, > > > > I've reproduced the problem. It seems JDK misses a piece of code for modules that is there for class loaders. It looks easy to fix, I'm looking at this. > > > > Meanwhile, while you're experimenting, you may disable CheckpointException by -XX:+UnlockExperimentalVMOptions -XX:-CRDoThrowCheckpointException. Of course, it's not recommended for anything beyond experiments :) > > > > Thanks, > > Anton From akozlov at azul.com Tue Nov 29 18:08:17 2022 From: akozlov at azul.com (Anton Kozlov) Date: Tue, 29 Nov 2022 20:08:17 +0200 Subject: Unable to create snapshots with Java modules In-Reply-To: References: Message-ID: Hmm, I came up with the pure Java solution [1]. I'm not sure those mark_classpath functions are still actual or required. [1] https://github.com/openjdk/crac/pull/35 Thanks, Anton On 11/29/22 18:36, Nikola Grcevski wrote: > I found out the issue I had with my directory, I was using a symlinked file, so > the parent directory doesn't match the st_ino of the file. Perhaps > there's a much > better way, but I managed to get things to work with the following addition > (no symlink support): > > static void mark_classpath_or_dir(FdsInfo *fds, char* cp) { > DIR *dir = os::opendir(cp); > if (!dir) { > mark_classpath_entry(fds, cp); > return; > } > os::closedir(dir); > mark_all_in(fds, cp); > } > > ... > do_classpaths(mark_classpath_or_dir, &fds, Arguments::get_appmodulepath()); > > I also had to make a small change in do_classpaths, at the end instead > of calling > mark_classpath_entry, I changed it to call fn(fds, cp). > > On Tue, Nov 29, 2022 at 11:24 AM Nikola Grcevski wrote: >> >> Great, thanks! >> >> I started looking at it as well, I added >> >> arguments.cpp/hpp >> _java_module_path = new SystemProperty("jdk.module.path", "", true); >> >> and in os_linux.cpp >> do_classpaths(mark_classpath_entry, &fds, Arguments::get_appmodulepath()); >> >> I was able to make things work if I put the expanded module path directly e.g: >> --module-path lib/simplemodule-1.0-SNAPSHOT.jar >> >> I'm not sure if there's a helper to expand the module path fully with >> directories, when I tried >> to create a mark function that handles both directories and files, the >> directory scan >> and mark never matches on my app lib directory on this check: >> >> if (dent->d_ino == fstat->st_ino) >> >> in os_linux::static void mark_all_in(FdsInfo *fds, char* dirpath). >> >> Thanks so much for your help. >> Nikola >> >> On Tue, Nov 29, 2022 at 11:11 AM Anton Kozlov wrote: >>> >>> On 11/29/22 00:04, Nikola Grcevski wrote: >>>> However, if I try the same thing with the modules usage, the snapshot >>>> creation fails: >>>> ~/work/simplemodule$ //openjdk-17-crac+3_linux-x64/bin/java >>>> -XX:CRaCCheckpointTo=/tmp/simple --module-path lib --module >>>> simplemodule/co.elastic.simple.SimpleModule >>> >>> Hi Nikola, >>> >>> I've reproduced the problem. It seems JDK misses a piece of code for modules that is there for class loaders. It looks easy to fix, I'm looking at this. >>> >>> Meanwhile, while you're experimenting, you may disable CheckpointException by -XX:+UnlockExperimentalVMOptions -XX:-CRDoThrowCheckpointException. Of course, it's not recommended for anything beyond experiments :) >>> >>> Thanks, >>> Anton From akozlov at openjdk.org Tue Nov 29 18:14:54 2022 From: akozlov at openjdk.org (Anton Kozlov) Date: Tue, 29 Nov 2022 18:14:54 GMT Subject: [crac] RFR: Assume module jars persistent Message-ID: Jar files opened by URLClassLoaders are assumed persistent. We need to handle module Jars the same way to prevent CheckpointException as reported by [1] [1] https://mail.openjdk.org/pipermail/crac-dev/2022-November/000380.html ------------- Commit messages: - Assume module jars persistent Changes: https://git.openjdk.org/crac/pull/35/files Webrev: https://webrevs.openjdk.org/?repo=crac&pr=35&range=00 Stats: 85 lines in 3 files changed: 76 ins; 7 del; 2 mod Patch: https://git.openjdk.org/crac/pull/35.diff Fetch: git fetch https://git.openjdk.org/crac pull/35/head:pull/35 PR: https://git.openjdk.org/crac/pull/35 From grcevski at gmail.com Tue Nov 29 18:51:33 2022 From: grcevski at gmail.com (Nikola Grcevski) Date: Tue, 29 Nov 2022 13:51:33 -0500 Subject: Unable to create snapshots with Java modules In-Reply-To: References: Message-ID: Thanks again Anton! I confirm that your patch works and it is much simpler than the hacks I did in os_linux: https://github.com/grcevski/crac/pull/1 I'm guessing since the jars are recorded in URLClassPath and in ModuleReferences now, those mark_classpaths are probably not needed. I did a quick experiment by commenting out: do_classpaths(mark_classpath_entry, &fds, Arguments::get_appclasspath()); I didn't see any issue while running by using the class path approach. On Tue, Nov 29, 2022 at 1:08 PM Anton Kozlov wrote: > > Hmm, I came up with the pure Java solution [1]. I'm not sure those mark_classpath functions are still actual or required. > > [1] https://github.com/openjdk/crac/pull/35 > > Thanks, > Anton > > On 11/29/22 18:36, Nikola Grcevski wrote: > > I found out the issue I had with my directory, I was using a symlinked file, so > > the parent directory doesn't match the st_ino of the file. Perhaps > > there's a much > > better way, but I managed to get things to work with the following addition > > (no symlink support): > > > > static void mark_classpath_or_dir(FdsInfo *fds, char* cp) { > > DIR *dir = os::opendir(cp); > > if (!dir) { > > mark_classpath_entry(fds, cp); > > return; > > } > > os::closedir(dir); > > mark_all_in(fds, cp); > > } > > > > ... > > do_classpaths(mark_classpath_or_dir, &fds, Arguments::get_appmodulepath()); > > > > I also had to make a small change in do_classpaths, at the end instead > > of calling > > mark_classpath_entry, I changed it to call fn(fds, cp). > > > > On Tue, Nov 29, 2022 at 11:24 AM Nikola Grcevski wrote: > >> > >> Great, thanks! > >> > >> I started looking at it as well, I added > >> > >> arguments.cpp/hpp > >> _java_module_path = new SystemProperty("jdk.module.path", "", true); > >> > >> and in os_linux.cpp > >> do_classpaths(mark_classpath_entry, &fds, Arguments::get_appmodulepath()); > >> > >> I was able to make things work if I put the expanded module path directly e.g: > >> --module-path lib/simplemodule-1.0-SNAPSHOT.jar > >> > >> I'm not sure if there's a helper to expand the module path fully with > >> directories, when I tried > >> to create a mark function that handles both directories and files, the > >> directory scan > >> and mark never matches on my app lib directory on this check: > >> > >> if (dent->d_ino == fstat->st_ino) > >> > >> in os_linux::static void mark_all_in(FdsInfo *fds, char* dirpath). > >> > >> Thanks so much for your help. > >> Nikola > >> > >> On Tue, Nov 29, 2022 at 11:11 AM Anton Kozlov wrote: > >>> > >>> On 11/29/22 00:04, Nikola Grcevski wrote: > >>>> However, if I try the same thing with the modules usage, the snapshot > >>>> creation fails: > >>>> ~/work/simplemodule$ //openjdk-17-crac+3_linux-x64/bin/java > >>>> -XX:CRaCCheckpointTo=/tmp/simple --module-path lib --module > >>>> simplemodule/co.elastic.simple.SimpleModule > >>> > >>> Hi Nikola, > >>> > >>> I've reproduced the problem. It seems JDK misses a piece of code for modules that is there for class loaders. It looks easy to fix, I'm looking at this. > >>> > >>> Meanwhile, while you're experimenting, you may disable CheckpointException by -XX:+UnlockExperimentalVMOptions -XX:-CRDoThrowCheckpointException. Of course, it's not recommended for anything beyond experiments :) > >>> > >>> Thanks, > >>> Anton From heidinga at openjdk.org Tue Nov 29 18:57:19 2022 From: heidinga at openjdk.org (Dan Heidinga) Date: Tue, 29 Nov 2022 18:57:19 GMT Subject: [crac] RFR: Assume module jars persistent In-Reply-To: References: Message-ID: On Tue, 29 Nov 2022 18:06:30 GMT, Anton Kozlov wrote: > Jar files opened by URLClassLoaders are assumed persistent. We need to handle module Jars the same way to prevent CheckpointException as reported by [1] > > [1] https://mail.openjdk.org/pipermail/crac-dev/2022-November/000380.html src/java.base/share/classes/jdk/internal/module/ModuleReferences.java line 247: > 245: this.jf = newJarFile(path); > 246: this.uri = uri; > 247: this.resource = new JDKResource() { Would it be cleaner to implement this JDKResource subclass as a named class in `JarFileCRaCSupport` and wrap this logic so we have something like: this.resource = JarFileCRaCSupport.wrapAndRegisterJarFile(this.jf); It centralizes more of the Jar file support in one place.... ------------- PR: https://git.openjdk.org/crac/pull/35 From akozlov at azul.com Wed Nov 30 14:18:30 2022 From: akozlov at azul.com (Anton Kozlov) Date: Wed, 30 Nov 2022 16:18:30 +0200 Subject: Unable to create snapshots with Java modules In-Reply-To: References: Message-ID: On 11/29/22 20:51, Nikola Grcevski wrote: > Thanks again Anton! I confirm that your patch works and it > is much simpler than the hacks I did in os_linux: > https://github.com/grcevski/crac/pull/1 Thanks for confirming! > I'm guessing since the jars are recorded in URLClassPath > and in ModuleReferences now, those mark_classpaths are > probably not needed. I did a quick experiment by commenting > out It seems the VM code around that needs to be cleaned up. I will happily review a PR for that. Thanks, Anton From akozlov at openjdk.org Wed Nov 30 15:04:51 2022 From: akozlov at openjdk.org (Anton Kozlov) Date: Wed, 30 Nov 2022 15:04:51 GMT Subject: [crac] RFR: Assume module jars persistent [v2] In-Reply-To: References: Message-ID: > Jar files opened by URLClassLoaders are assumed persistent. We need to handle module Jars the same way to prevent CheckpointException as reported by [1] > > [1] https://mail.openjdk.org/pipermail/crac-dev/2022-November/000380.html Anton Kozlov has updated the pull request incrementally with one additional commit since the last revision: Move everything to the separate class ------------- Changes: - all: https://git.openjdk.org/crac/pull/35/files - new: https://git.openjdk.org/crac/pull/35/files/a30d1609..3f4bc13e Webrevs: - full: https://webrevs.openjdk.org/?repo=crac&pr=35&range=01 - incr: https://webrevs.openjdk.org/?repo=crac&pr=35&range=00-01 Stats: 181 lines in 4 files changed: 69 ins; 107 del; 5 mod Patch: https://git.openjdk.org/crac/pull/35.diff Fetch: git fetch https://git.openjdk.org/crac pull/35/head:pull/35 PR: https://git.openjdk.org/crac/pull/35 From akozlov at openjdk.org Wed Nov 30 15:04:53 2022 From: akozlov at openjdk.org (Anton Kozlov) Date: Wed, 30 Nov 2022 15:04:53 GMT Subject: [crac] RFR: Assume module jars persistent [v2] In-Reply-To: References: Message-ID: <_ACKQBP6u4-6C2ZX492Y-sU_vMte1gzxvt2dhA0qSHo=.5f2c1095-8ad7-4495-ba0a-c2ce24b408c7@github.com> On Tue, 29 Nov 2022 18:55:01 GMT, Dan Heidinga wrote: >> Anton Kozlov has updated the pull request incrementally with one additional commit since the last revision: >> >> Move everything to the separate class > > src/java.base/share/classes/jdk/internal/module/ModuleReferences.java line 247: > >> 245: this.jf = newJarFile(path); >> 246: this.uri = uri; >> 247: this.resource = new JDKResource() { > > Would it be cleaner to implement this JDKResource subclass as a named class in `JarFileCRaCSupport` and wrap this logic so we have something like: > > this.resource = JarFileCRaCSupport.wrapAndRegisterJarFile(this.jf); > > > It centralizes more of the Jar file support in one place.... Indeed, I've created ClassLoaderJarFile for that purpose. Moving the whole class to be used by ModuleReferences. ------------- PR: https://git.openjdk.org/crac/pull/35 From heidinga at openjdk.org Wed Nov 30 15:35:05 2022 From: heidinga at openjdk.org (Dan Heidinga) Date: Wed, 30 Nov 2022 15:35:05 GMT Subject: [crac] RFR: Assume module jars persistent [v2] In-Reply-To: References: Message-ID: On Wed, 30 Nov 2022 15:04:51 GMT, Anton Kozlov wrote: >> Jar files opened by URLClassLoaders are assumed persistent. We need to handle module Jars the same way to prevent CheckpointException as reported by [1] >> >> [1] https://mail.openjdk.org/pipermail/crac-dev/2022-November/000380.html > > Anton Kozlov has updated the pull request incrementally with one additional commit since the last revision: > > Move everything to the separate class Marked as reviewed by heidinga (Committer). src/java.base/share/classes/jdk/internal/util/jar/PersistentJarFile.java line 56: > 54: zipBeforeCheckpoint.setAccessible(true); > 55: return null; > 56: }}); Something to think about that doesn't need to be addressed now: We may want to lazily cache the accessible reflect Method in a static field to avoid looking it up for each PersistentJarFile. The value of doing that depends somewhat on how many jars we expect here and how much we care about time spent in checkpoint hooks. ------------- PR: https://git.openjdk.org/crac/pull/35 From akozlov at openjdk.org Wed Nov 30 17:36:48 2022 From: akozlov at openjdk.org (Anton Kozlov) Date: Wed, 30 Nov 2022 17:36:48 GMT Subject: [crac] RFR: Assume module jars persistent [v3] In-Reply-To: References: Message-ID: > Jar files opened by URLClassLoaders are assumed persistent. We need to handle module Jars the same way to prevent CheckpointException as reported by [1] > > [1] https://mail.openjdk.org/pipermail/crac-dev/2022-November/000380.html Anton Kozlov has updated the pull request incrementally with one additional commit since the last revision: Use SharedSecrets instead of reflection ------------- Changes: - all: https://git.openjdk.org/crac/pull/35/files - new: https://git.openjdk.org/crac/pull/35/files/3f4bc13e..03bd36f3 Webrevs: - full: https://webrevs.openjdk.org/?repo=crac&pr=35&range=02 - incr: https://webrevs.openjdk.org/?repo=crac&pr=35&range=01-02 Stats: 20 lines in 3 files changed: 5 ins; 12 del; 3 mod Patch: https://git.openjdk.org/crac/pull/35.diff Fetch: git fetch https://git.openjdk.org/crac pull/35/head:pull/35 PR: https://git.openjdk.org/crac/pull/35 From akozlov at openjdk.org Wed Nov 30 17:40:38 2022 From: akozlov at openjdk.org (Anton Kozlov) Date: Wed, 30 Nov 2022 17:40:38 GMT Subject: [crac] RFR: Assume module jars persistent [v2] In-Reply-To: References: Message-ID: On Wed, 30 Nov 2022 15:32:27 GMT, Dan Heidinga wrote: >> Anton Kozlov has updated the pull request incrementally with one additional commit since the last revision: >> >> Move everything to the separate class > > src/java.base/share/classes/jdk/internal/util/jar/PersistentJarFile.java line 56: > >> 54: zipBeforeCheckpoint.setAccessible(true); >> 55: return null; >> 56: }}); > > Something to think about that doesn't need to be addressed now: > > We may want to lazily cache the accessible reflect Method in a static field to avoid looking it up for each PersistentJarFile. The value of doing that depends somewhat on how many jars we expect here and how much we care about time spent in checkpoint hooks. Oh, you're right. Not sure when it will be an another chance to improve, so better be done now :) This implementation with reflection predated the SharedSecrets, which cleaner, safer, faster -- so I just used that. ------------- PR: https://git.openjdk.org/crac/pull/35 From heidinga at openjdk.org Wed Nov 30 18:45:58 2022 From: heidinga at openjdk.org (Dan Heidinga) Date: Wed, 30 Nov 2022 18:45:58 GMT Subject: [crac] RFR: Assume module jars persistent [v3] In-Reply-To: References: Message-ID: On Wed, 30 Nov 2022 17:36:48 GMT, Anton Kozlov wrote: >> Jar files opened by URLClassLoaders are assumed persistent. We need to handle module Jars the same way to prevent CheckpointException as reported by [1] >> >> [1] https://mail.openjdk.org/pipermail/crac-dev/2022-November/000380.html > > Anton Kozlov has updated the pull request incrementally with one additional commit since the last revision: > > Use SharedSecrets instead of reflection Agreed. The SharedSecrets approach is even better ------------- Marked as reviewed by heidinga (Committer). PR: https://git.openjdk.org/crac/pull/35