From duke at openjdk.org Mon Sep 12 07:37:50 2022 From: duke at openjdk.org (Kuznetsov Ilya Alexandrovich) Date: Mon, 12 Sep 2022 07:37:50 GMT Subject: [crac] RFR: X11 CRaC reinitializing on CheckpointRestore [v2] In-Reply-To: References: Message-ID: > Allows CRaC to perform a CheckpointRestore operation for applications using GUI (Swing, AWT) and X11 connection. > > Resources are registered only if the application uses the GUI. The order in which resources are reinitialized matters: Toolkit should be cleared before reference handling for a proper garbage collection, and GraphicsEnvironment after handling for a correct X11 disconnection. Some resources restore lazily. > > The `beforeCheckpoint()` operation dispose necessary toolkit and connection resources and disconnects from X11. This allows CRaC to perform a Checkpoint since there is no external connection. > The `afterRestore()` operations reconnect to X11 and then restore necessary connection and toolkit resources. > > Thus, after the Restore operation, we have a clean X11 connection. It is ready to restore the original GUI state. Kuznetsov Ilya Alexandrovich has updated the pull request with a new target base due to a merge or a rebase. The incremental webrev excludes the unrelated changes brought in by the merge/rebase. The pull request contains 74 additional commits since the last revision: - Minimize changes - Merge remote-tracking branch 'origin/x11-dev' into remote-x11-dev - Reduce public interfaces using AWTAccessor and package-private, made some code and docs improvements Signed-off-by: kznts9v-1lya - Change AWT part of reinitialization using AWTAccessor Signed-off-by: kznts9v-1lya - Change Disposer reinitialization using AWTAccessor Signed-off-by: kznts9v-1lya - Disposer reinitialization using SharedSecrets Signed-off-by: kznts9v-1lya - Fix in tests Signed-off-by: kznts9v-1lya - Made some fixes Signed-off-by: kznts9v-1lya - Merge remote-tracking branch 'upstream2/refqueue-2' into x11-dev # Conflicts: # src/java.base/share/classes/jdk/internal/crac/JDKResource.java - Make jdk.crac.Misc final - ... and 64 more: https://git.openjdk.org/crac/compare/aa7fa48d...2f8751d1 ------------- Changes: - all: https://git.openjdk.org/crac/pull/19/files - new: https://git.openjdk.org/crac/pull/19/files/494e482d..2f8751d1 Webrevs: - full: https://webrevs.openjdk.org/?repo=crac&pr=19&range=01 - incr: https://webrevs.openjdk.org/?repo=crac&pr=19&range=00-01 Stats: 1145 lines in 28 files changed: 588 ins; 412 del; 145 mod Patch: https://git.openjdk.org/crac/pull/19.diff Fetch: git fetch https://git.openjdk.org/crac pull/19/head:pull/19 PR: https://git.openjdk.org/crac/pull/19 From duke at openjdk.org Mon Sep 12 07:48:23 2022 From: duke at openjdk.org (Kuznetsov Ilya Alexandrovich) Date: Mon, 12 Sep 2022 07:48:23 GMT Subject: [crac] RFR: X11 CRaC reinitializing on CheckpointRestore [v3] In-Reply-To: References: Message-ID: > Allows CRaC to perform a CheckpointRestore operation for applications using GUI (Swing, AWT) and X11 connection. > > Resources are registered only if the application uses the GUI. The order in which resources are reinitialized matters: Toolkit should be cleared before reference handling for a proper garbage collection, and GraphicsEnvironment after handling for a correct X11 disconnection. Some resources restore lazily. > > The `beforeCheckpoint()` operation dispose necessary toolkit and connection resources and disconnects from X11. This allows CRaC to perform a Checkpoint since there is no external connection. > The `afterRestore()` operations reconnect to X11 and then restore necessary connection and toolkit resources. > > Thus, after the Restore operation, we have a clean X11 connection. It is ready to restore the original GUI state. Kuznetsov Ilya Alexandrovich has updated the pull request incrementally with one additional commit since the last revision: Update CI ------------- Changes: - all: https://git.openjdk.org/crac/pull/19/files - new: https://git.openjdk.org/crac/pull/19/files/2f8751d1..aa940d8f Webrevs: - full: https://webrevs.openjdk.org/?repo=crac&pr=19&range=02 - incr: https://webrevs.openjdk.org/?repo=crac&pr=19&range=01-02 Stats: 6 lines in 1 file changed: 0 ins; 0 del; 6 mod Patch: https://git.openjdk.org/crac/pull/19.diff Fetch: git fetch https://git.openjdk.org/crac pull/19/head:pull/19 PR: https://git.openjdk.org/crac/pull/19 From duke at openjdk.org Mon Sep 12 07:50:57 2022 From: duke at openjdk.org (Kuznetsov Ilya Alexandrovich) Date: Mon, 12 Sep 2022 07:50:57 GMT Subject: [crac] RFR: X11 CRaC reinitializing on CheckpointRestore [v4] In-Reply-To: References: Message-ID: > Allows CRaC to perform a CheckpointRestore operation for applications using GUI (Swing, AWT) and X11 connection. > > Resources are registered only if the application uses the GUI. The order in which resources are reinitialized matters: Toolkit should be cleared before reference handling for a proper garbage collection, and GraphicsEnvironment after handling for a correct X11 disconnection. Some resources restore lazily. > > The `beforeCheckpoint()` operation dispose necessary toolkit and connection resources and disconnects from X11. This allows CRaC to perform a Checkpoint since there is no external connection. > The `afterRestore()` operations reconnect to X11 and then restore necessary connection and toolkit resources. > > Thus, after the Restore operation, we have a clean X11 connection. It is ready to restore the original GUI state. Kuznetsov Ilya Alexandrovich has updated the pull request incrementally with one additional commit since the last revision: Fix newline ------------- Changes: - all: https://git.openjdk.org/crac/pull/19/files - new: https://git.openjdk.org/crac/pull/19/files/aa940d8f..6aa77a49 Webrevs: - full: https://webrevs.openjdk.org/?repo=crac&pr=19&range=03 - incr: https://webrevs.openjdk.org/?repo=crac&pr=19&range=02-03 Stats: 1 line in 1 file changed: 0 ins; 0 del; 1 mod Patch: https://git.openjdk.org/crac/pull/19.diff Fetch: git fetch https://git.openjdk.org/crac pull/19/head:pull/19 PR: https://git.openjdk.org/crac/pull/19 From akozlov at openjdk.org Mon Sep 12 19:59:52 2022 From: akozlov at openjdk.org (Anton Kozlov) Date: Mon, 12 Sep 2022 19:59:52 GMT Subject: [crac] RFR: API for restore time Message-ID: This PR adds an ability to get restore time, similar to RuntimeMXBean.getUptime() [1] and getStartTime(). Before I used to print timestamps and parse console output, which was not very convenient. [1] https://docs.oracle.com/en/java/javase/17/docs/api/java.management/java/lang/management/RuntimeMXBean.html#getUptime() ------------- Commit messages: - Cleanup - Highlight global variables - Add new pacakge description - Update interface - Fallback to the restore complete time - Add missing CRaCMXBean - Pass restore time/counter - Extract read_from - Add CRaCMXBean - Prepare CracRestoreParameters for extension Changes: https://git.openjdk.org/crac/pull/29/files Webrev: https://webrevs.openjdk.org/?repo=crac&pr=29&range=00 Stats: 561 lines in 13 files changed: 427 ins; 69 del; 65 mod Patch: https://git.openjdk.org/crac/pull/29.diff Fetch: git fetch https://git.openjdk.org/crac pull/29/head:pull/29 PR: https://git.openjdk.org/crac/pull/29 From snazarki at openjdk.org Wed Sep 14 21:07:28 2022 From: snazarki at openjdk.org (Sergey Nazarkin) Date: Wed, 14 Sep 2022 21:07:28 GMT Subject: [crac] RFR: API for restore time In-Reply-To: References: Message-ID: On Mon, 12 Sep 2022 19:54:41 GMT, Anton Kozlov wrote: > This PR adds an ability to get restore time, similar to RuntimeMXBean.getUptime() [1] and getStartTime(). Before I used to print timestamps and parse console output, which was not very convenient. > > [1] https://docs.oracle.com/en/java/javase/17/docs/api/java.management/java/lang/management/RuntimeMXBean.html#getUptime() src/hotspot/os/linux/os_linux.cpp line 354: > 352: char prop[4096]; > 353: int len = snprintf(prop, sizeof(prop), "%s=%s", p->key(), p->value()); > 354: guarantee((unsigned)len < sizeof(prop), "property does not fit temp buffer"); And len > 0 ------------- PR: https://git.openjdk.org/crac/pull/29 From snazarki at openjdk.org Thu Sep 15 09:26:13 2022 From: snazarki at openjdk.org (Sergey Nazarkin) Date: Thu, 15 Sep 2022 09:26:13 GMT Subject: [crac] RFR: API for restore time In-Reply-To: References: Message-ID: On Mon, 12 Sep 2022 19:54:41 GMT, Anton Kozlov wrote: > This PR adds an ability to get restore time, similar to RuntimeMXBean.getUptime() [1] and getStartTime(). Before I used to print timestamps and parse console output, which was not very convenient. > > [1] https://docs.oracle.com/en/java/javase/17/docs/api/java.management/java/lang/management/RuntimeMXBean.html#getUptime() Marked as reviewed by snazarki (no project role). ------------- PR: https://git.openjdk.org/crac/pull/29 From akozlov at openjdk.org Fri Sep 16 19:07:54 2022 From: akozlov at openjdk.org (Anton Kozlov) Date: Fri, 16 Sep 2022 19:07:54 GMT Subject: [crac] RFR: API for restore time [v2] In-Reply-To: References: Message-ID: > This PR adds an ability to get restore time, similar to RuntimeMXBean.getUptime() [1] and getStartTime(). Before I used to print timestamps and parse console output, which was not very convenient. > > [1] https://docs.oracle.com/en/java/javase/17/docs/api/java.management/java/lang/management/RuntimeMXBean.html#getUptime() Anton Kozlov has updated the pull request incrementally with one additional commit since the last revision: Guarantee len ------------- Changes: - all: https://git.openjdk.org/crac/pull/29/files - new: https://git.openjdk.org/crac/pull/29/files/a57caf3c..54512e3d Webrevs: - full: https://webrevs.openjdk.org/?repo=crac&pr=29&range=01 - incr: https://webrevs.openjdk.org/?repo=crac&pr=29&range=00-01 Stats: 1 line in 1 file changed: 0 ins; 0 del; 1 mod Patch: https://git.openjdk.org/crac/pull/29.diff Fetch: git fetch https://git.openjdk.org/crac pull/29/head:pull/29 PR: https://git.openjdk.org/crac/pull/29 From akozlov at openjdk.org Tue Sep 27 15:55:05 2022 From: akozlov at openjdk.org (Anton Kozlov) Date: Tue, 27 Sep 2022 15:55:05 GMT Subject: [crac] RFR: API for restore time [v2] In-Reply-To: References: Message-ID: <2uK8y2dhvmO-Bsbh2Hc741BrkdnuDvahosVe_Gg85Ec=.769092e5-ec69-4790-a63d-44dbe3322a59@github.com> On Fri, 16 Sep 2022 19:07:54 GMT, Anton Kozlov wrote: >> This PR adds an ability to get restore time, similar to RuntimeMXBean.getUptime() [1] and getStartTime(). Before I used to print timestamps and parse console output, which was not very convenient. >> >> [1] https://docs.oracle.com/en/java/javase/17/docs/api/java.management/java/lang/management/RuntimeMXBean.html#getUptime() > > Anton Kozlov has updated the pull request incrementally with one additional commit since the last revision: > > Guarantee len OK, I think this was hanging long enough already :) It blocks further works, so I'm going to integrate this. ------------- PR: https://git.openjdk.org/crac/pull/29 From akozlov at openjdk.org Wed Sep 28 13:50:35 2022 From: akozlov at openjdk.org (Anton Kozlov) Date: Wed, 28 Sep 2022 13:50:35 GMT Subject: [crac] RFR: API for restore time [v2] In-Reply-To: References: Message-ID: On Thu, 15 Sep 2022 09:22:43 GMT, Sergey Nazarkin wrote: >> Anton Kozlov has updated the pull request incrementally with one additional commit since the last revision: >> >> Guarantee len > > Marked as reviewed by snazarki (no project role). @snazarkin thanks for the review! ------------- PR: https://git.openjdk.org/crac/pull/29 From akozlov at openjdk.org Wed Sep 28 13:50:36 2022 From: akozlov at openjdk.org (Anton Kozlov) Date: Wed, 28 Sep 2022 13:50:36 GMT Subject: [crac] Integrated: API for restore time In-Reply-To: References: Message-ID: On Mon, 12 Sep 2022 19:54:41 GMT, Anton Kozlov wrote: > This PR adds an ability to get restore time, similar to RuntimeMXBean.getUptime() [1] and getStartTime(). Before I used to print timestamps and parse console output, which was not very convenient. > > [1] https://docs.oracle.com/en/java/javase/17/docs/api/java.management/java/lang/management/RuntimeMXBean.html#getUptime() This pull request has now been integrated. Changeset: 9ed96110 Author: Anton Kozlov URL: https://git.openjdk.org/crac/commit/9ed961106a255145274de777e151577863b013ea Stats: 561 lines in 13 files changed: 427 ins; 69 del; 65 mod API for restore time Reviewed-by: snazarki ------------- PR: https://git.openjdk.org/crac/pull/29 From duke at openjdk.org Thu Sep 29 15:45:01 2022 From: duke at openjdk.org (Roman Marchenko) Date: Thu, 29 Sep 2022 15:45:01 GMT Subject: [crac] RFR: Environment vars propagation into restored process Message-ID: This PR provides functionality to propagate actual environment variables to a restored process, as well as the test for this functionality. Env propagation is done in few steps: - Store the actual environment before restoring - After restoring, replace the restored `environ` with a new one. - On `afterRestore` event, propagate the new environment into a restored process via `ProcessEnvironment`. ------------- Commit messages: - Merge branch 'crac' into env_propagation - Some minor changes - Used putenv once rather that setenv in a loop - Some style fixes - Propagating actual environment vars into the restored process. - Cleanup - Highlight global variables - Add new pacakge description - Update interface - Fallback to the restore complete time - ... and 5 more: https://git.openjdk.org/crac/compare/9ed96110...0270dc4b Changes: https://git.openjdk.org/crac/pull/30/files Webrev: https://webrevs.openjdk.org/?repo=crac&pr=30&range=00 Stats: 178 lines in 4 files changed: 174 ins; 0 del; 4 mod Patch: https://git.openjdk.org/crac/pull/30.diff Fetch: git fetch https://git.openjdk.org/crac pull/30/head:pull/30 PR: https://git.openjdk.org/crac/pull/30 From akozlov at openjdk.org Fri Sep 30 13:52:42 2022 From: akozlov at openjdk.org (Anton Kozlov) Date: Fri, 30 Sep 2022 13:52:42 GMT Subject: [crac] RFR: Environment vars propagation into restored process In-Reply-To: References: Message-ID: On Thu, 29 Sep 2022 15:38:52 GMT, Roman Marchenko wrote: > This PR provides functionality to propagate actual environment variables to a restored process, as well as the test for this functionality. > > Env propagation is done in few steps: > - Store the actual environment before restoring > - After restoring, replace the restored `environ` with a new one. > - On `afterRestore` event, propagate the new environment into a restored process via `ProcessEnvironment`. LGTM, thanks! Just few minor questionable style issues. src/hotspot/os/linux/os_linux.cpp line 373: > 371: // Write env vars > 372: for (char** env = environ; *env; ++env) > 373: { Suggestion: for (char** env = environ; *env; ++env) { src/hotspot/os/linux/os_linux.cpp line 6463: > 6461: > 6462: { > 6463: char* env_mem = NEW_C_HEAP_ARRAY(char, hdr->_env_memory_size, mtArguments); // left this pointer unowned, it is freed when process dies Suggestion: char* env_mem = NEW_C_HEAP_ARRAY(char, hdr->_env_memory_size, mtArguments); // left this pointer unowned, it is freed when process dies Here and below, we don't hesitate to introduce few local variables without creating a new syntax block. Please note the change once applied makes the code uncompilable. ------------- Marked as reviewed by akozlov (Lead). PR: https://git.openjdk.org/crac/pull/30 From heidinga at openjdk.org Fri Sep 30 14:39:00 2022 From: heidinga at openjdk.org (Dan Heidinga) Date: Fri, 30 Sep 2022 14:39:00 GMT Subject: [crac] RFR: Environment vars propagation into restored process In-Reply-To: References: Message-ID: <4bABOyN_ecZeOziPY8Wsmfbco0VPwgYWNiD4IZFSntA=.3ca718e6-f1db-469e-a466-b55fe9ebde17@github.com> On Thu, 29 Sep 2022 15:38:52 GMT, Roman Marchenko wrote: > This PR provides functionality to propagate actual environment variables to a restored process, as well as the test for this functionality. > > Env propagation is done in few steps: > - Store the actual environment before restoring > - After restoring, replace the restored `environ` with a new one. > - On `afterRestore` event, propagate the new environment into a restored process via `ProcessEnvironment`. One concern with this approach - it means that environment variables will change values after a restore. It seems odd to say this is a concern when it's the intended behaviour of this PR but it is a concern. Users typically cache environment variables in static fields or use them to make a one time decision. They don't expect them (at least at the Java layer) to change value throughout a run of the same process. This change means two reads of the same env var can give different results at different times which may put unsuspecting applications into inconsistent states if two locations read the env var before vs after a restore. That's going to be a hard to debug issue. The VM may also read env vars and bind tightly to the value. Native code after a restore will still have the original env while java code the modified env. Do we foresee any issues there? ------------- PR: https://git.openjdk.org/crac/pull/30