From jespersm at openjdk.java.net Wed Dec 1 00:20:17 2021 From: jespersm at openjdk.java.net (Jesper Steen =?UTF-8?B?TcO4bGxlcg==?=) Date: Wed, 1 Dec 2021 00:20:17 GMT Subject: [lworld] RFR: 8274800: [lworld] Primitive classes can't be retransformed Message-ID: Fixes JVMTI reconstituter to work with lworld bytecodes and flags ------------- Commit messages: - 8274800: [lworld] Primitive classes can't be retransformed Changes: https://git.openjdk.java.net/valhalla/pull/586/files Webrev: https://webrevs.openjdk.java.net/?repo=valhalla&pr=586&range=00 Issue: https://bugs.openjdk.java.net/browse/JDK-8274800 Stats: 131 lines in 3 files changed: 131 ins; 0 del; 0 mod Patch: https://git.openjdk.java.net/valhalla/pull/586.diff Fetch: git fetch https://git.openjdk.java.net/valhalla pull/586/head:pull/586 PR: https://git.openjdk.java.net/valhalla/pull/586 From jespersm at openjdk.java.net Thu Dec 2 12:47:37 2021 From: jespersm at openjdk.java.net (Jesper Steen =?UTF-8?B?TcO4bGxlcg==?=) Date: Thu, 2 Dec 2021 12:47:37 GMT Subject: [lworld] RFR: 8274800: [lworld] Primitive classes can't be retransformed In-Reply-To: References: Message-ID: On Wed, 1 Dec 2021 00:15:07 GMT, Jesper Steen M?ller wrote: > Fixes JVMTI reconstituter to work with lworld bytecodes and flags Closed as requested by Alex Menkov. ------------- PR: https://git.openjdk.java.net/valhalla/pull/586 From jespersm at openjdk.java.net Thu Dec 2 12:47:37 2021 From: jespersm at openjdk.java.net (Jesper Steen =?UTF-8?B?TcO4bGxlcg==?=) Date: Thu, 2 Dec 2021 12:47:37 GMT Subject: [lworld] Withdrawn: 8274800: [lworld] Primitive classes can't be retransformed In-Reply-To: References: Message-ID: <-7-AkGsz7LJKE7PaS_mHicb2dTAoQ44R13oUqZ5KY_E=.dde7de8d-a5ba-440a-a255-a470b522dbcf@github.com> On Wed, 1 Dec 2021 00:15:07 GMT, Jesper Steen M?ller wrote: > Fixes JVMTI reconstituter to work with lworld bytecodes and flags This pull request has been closed without being integrated. ------------- PR: https://git.openjdk.java.net/valhalla/pull/586 From dsimms at openjdk.java.net Fri Dec 3 17:35:20 2021 From: dsimms at openjdk.java.net (David Simms) Date: Fri, 3 Dec 2021 17:35:20 GMT Subject: [lworld] RFR: Merge jdk Message-ID: Merge tag 'jdk-18+25' into lworld_merge_jdk_nov Missed a compile issue for non-local compile ------------- Commit messages: - Missed a compile issue for non-local compile - Merge tag 'jdk-18+25' into lworld_merge_jdk_nov - 8274685: Documentation suggests there are ArbitrarilyJumpableGenerator when none - 8273792: JumpableGenerator.rngs() documentation refers to wrong method - 8277428: G1: Move and inline G1STWIsAliveClosure::do_object_b - 8266593: vmTestbase/nsk/jvmti/PopFrame/popframe011 fails with "assert(java_thread == _state->get_thread()) failed: Must be" - 8277534: Remove unused ReferenceProcessor::has_discovered_references - 8277385: Zero: Enable CompactStrings support - 8275448: [REDO] AArch64: Implement string_compare intrinsic in SVE - 8224922: Access JavaFileObject from Element(s) - ... and 589 more: https://git.openjdk.java.net/valhalla/compare/b9afd2a0...e4d4a19a The webrevs contain the adjustments done while merging with regards to each parent branch: - lworld: https://webrevs.openjdk.java.net/?repo=valhalla&pr=587&range=00.0 - jdk: https://webrevs.openjdk.java.net/?repo=valhalla&pr=587&range=00.1 Changes: https://git.openjdk.java.net/valhalla/pull/587/files Stats: 926182 lines in 2332 files changed: 482553 ins; 431095 del; 12534 mod Patch: https://git.openjdk.java.net/valhalla/pull/587.diff Fetch: git fetch https://git.openjdk.java.net/valhalla pull/587/head:pull/587 PR: https://git.openjdk.java.net/valhalla/pull/587 From dsimms at openjdk.java.net Fri Dec 3 17:36:16 2021 From: dsimms at openjdk.java.net (David Simms) Date: Fri, 3 Dec 2021 17:36:16 GMT Subject: [lworld] Integrated: Merge jdk In-Reply-To: References: Message-ID: On Fri, 3 Dec 2021 17:26:46 GMT, David Simms wrote: > Merge tag 'jdk-18+25' into lworld_merge_jdk_nov > Missed a compile issue for non-local compile This pull request has now been integrated. Changeset: caf5e5a9 Author: David Simms URL: https://git.openjdk.java.net/valhalla/commit/caf5e5a9daedebcb35a913cad218672f065393a7 Stats: 926182 lines in 2332 files changed: 482553 ins; 431095 del; 12534 mod Merge jdk Merge tag 'jdk-18+25' ------------- PR: https://git.openjdk.java.net/valhalla/pull/587 From amenkov at openjdk.java.net Sat Dec 4 00:53:01 2021 From: amenkov at openjdk.java.net (Alex Menkov) Date: Sat, 4 Dec 2021 00:53:01 GMT Subject: [lworld] RFR: 8274800: [lworld] Primitive classes can't be retransformed Message-ID: <9xCk7tXonaf_XjWjc4XgjWtwOtXiajDOLRTvM9rTZWA=.82a1bcf2-d76b-48ae-81ac-f8dcb981aeb8@github.com> Fix for java.lang.instrument.Instrumentation.retransformClasses() Cases for retransformation primitive to identity (and identity to primitive) classes and changing primitive class fields (so default_value becomes invalid) are handled by existing code and spec changes are not required. Also added serviceability test to hotspot_valhalla group ------------- Commit messages: - fixed tabs - retransformClasses fix Changes: https://git.openjdk.java.net/valhalla/pull/588/files Webrev: https://webrevs.openjdk.java.net/?repo=valhalla&pr=588&range=00 Issue: https://bugs.openjdk.java.net/browse/JDK-8274800 Stats: 522 lines in 6 files changed: 518 ins; 1 del; 3 mod Patch: https://git.openjdk.java.net/valhalla/pull/588.diff Fetch: git fetch https://git.openjdk.java.net/valhalla pull/588/head:pull/588 PR: https://git.openjdk.java.net/valhalla/pull/588 From amenkov at openjdk.java.net Thu Dec 9 22:12:21 2021 From: amenkov at openjdk.java.net (Alex Menkov) Date: Thu, 9 Dec 2021 22:12:21 GMT Subject: [lworld] RFR: 8274800: [lworld] Primitive classes can't be retransformed [v2] In-Reply-To: <9xCk7tXonaf_XjWjc4XgjWtwOtXiajDOLRTvM9rTZWA=.82a1bcf2-d76b-48ae-81ac-f8dcb981aeb8@github.com> References: <9xCk7tXonaf_XjWjc4XgjWtwOtXiajDOLRTvM9rTZWA=.82a1bcf2-d76b-48ae-81ac-f8dcb981aeb8@github.com> Message-ID: > Fix for java.lang.instrument.Instrumentation.retransformClasses() > > Cases for retransformation primitive to identity (and identity to primitive) classes and changing primitive class fields (so default_value becomes invalid) are handled by existing code and spec changes are not required. > > Also added serviceability test to hotspot_valhalla group Alex Menkov has updated the pull request incrementally with two additional commits since the last revision: - tabs to spaces in TEST.group - fixed CP mapping for defaultvalue bytecode ------------- Changes: - all: https://git.openjdk.java.net/valhalla/pull/588/files - new: https://git.openjdk.java.net/valhalla/pull/588/files/b26c171e..4a82c8d0 Webrevs: - full: https://webrevs.openjdk.java.net/?repo=valhalla&pr=588&range=01 - incr: https://webrevs.openjdk.java.net/?repo=valhalla&pr=588&range=00-01 Stats: 25 lines in 3 files changed: 24 ins; 0 del; 1 mod Patch: https://git.openjdk.java.net/valhalla/pull/588.diff Fetch: git fetch https://git.openjdk.java.net/valhalla pull/588/head:pull/588 PR: https://git.openjdk.java.net/valhalla/pull/588 From amenkov at openjdk.java.net Fri Dec 10 02:42:40 2021 From: amenkov at openjdk.java.net (Alex Menkov) Date: Fri, 10 Dec 2021 02:42:40 GMT Subject: [lworld] RFR: 8274800: [lworld] Primitive classes can't be retransformed In-Reply-To: References: <9xCk7tXonaf_XjWjc4XgjWtwOtXiajDOLRTvM9rTZWA=.82a1bcf2-d76b-48ae-81ac-f8dcb981aeb8@github.com> Message-ID: On Thu, 9 Dec 2021 00:44:40 GMT, Alex Menkov wrote: > I clicked on approved a bit too quickly. In jvmtiRedefineClass.cpp, the withfield bytecode has been added, but not the defaultvalue bytecode. > Fixed and testcase added. ------------- PR: https://git.openjdk.java.net/valhalla/pull/588 From amenkov at openjdk.java.net Fri Dec 10 22:37:28 2021 From: amenkov at openjdk.java.net (Alex Menkov) Date: Fri, 10 Dec 2021 22:37:28 GMT Subject: [lworld] Integrated: 8274800: [lworld] Primitive classes can't be retransformed In-Reply-To: <9xCk7tXonaf_XjWjc4XgjWtwOtXiajDOLRTvM9rTZWA=.82a1bcf2-d76b-48ae-81ac-f8dcb981aeb8@github.com> References: <9xCk7tXonaf_XjWjc4XgjWtwOtXiajDOLRTvM9rTZWA=.82a1bcf2-d76b-48ae-81ac-f8dcb981aeb8@github.com> Message-ID: <11VRsICL9xV-cCzJ1uSyvymPzcwwRq-SL7QYLrGBFNQ=.43831258-5503-410a-9994-722c87c20eb6@github.com> On Sat, 4 Dec 2021 00:30:11 GMT, Alex Menkov wrote: > Fix for java.lang.instrument.Instrumentation.retransformClasses() > > Cases for retransformation primitive to identity (and identity to primitive) classes and changing primitive class fields (so default_value becomes invalid) are handled by existing code and spec changes are not required. > > Also added serviceability test to hotspot_valhalla group This pull request has now been integrated. Changeset: 5ac15a15 Author: Alex Menkov URL: https://git.openjdk.java.net/valhalla/commit/5ac15a15e265e4770c8e13c4dfe7ddf760dd0a9f Stats: 546 lines in 6 files changed: 542 ins; 1 del; 3 mod 8274800: [lworld] Primitive classes can't be retransformed Reviewed-by: fparain, sspitsyn ------------- PR: https://git.openjdk.java.net/valhalla/pull/588 From fparain at openjdk.java.net Tue Dec 7 19:47:29 2021 From: fparain at openjdk.java.net (Frederic Parain) Date: Tue, 7 Dec 2021 19:47:29 GMT Subject: [lworld] RFR: 8274800: [lworld] Primitive classes can't be retransformed In-Reply-To: <9xCk7tXonaf_XjWjc4XgjWtwOtXiajDOLRTvM9rTZWA=.82a1bcf2-d76b-48ae-81ac-f8dcb981aeb8@github.com> References: <9xCk7tXonaf_XjWjc4XgjWtwOtXiajDOLRTvM9rTZWA=.82a1bcf2-d76b-48ae-81ac-f8dcb981aeb8@github.com> Message-ID: On Sat, 4 Dec 2021 00:30:11 GMT, Alex Menkov wrote: > Fix for java.lang.instrument.Instrumentation.retransformClasses() > > Cases for retransformation primitive to identity (and identity to primitive) classes and changing primitive class fields (so default_value becomes invalid) are handled by existing code and spec changes are not required. > > Also added serviceability test to hotspot_valhalla group I clicked on approved a bit too quickly. In jvmtiRedefineClass.cpp, the withfield bytecode has been added, but not the defaultvalue bytecode. Fred ------------- PR: https://git.openjdk.java.net/valhalla/pull/588 From fparain at openjdk.java.net Wed Dec 8 01:23:33 2021 From: fparain at openjdk.java.net (Frederic Parain) Date: Wed, 8 Dec 2021 01:23:33 GMT Subject: [lworld] RFR: 8274800: [lworld] Primitive classes can't be retransformed In-Reply-To: References: <9xCk7tXonaf_XjWjc4XgjWtwOtXiajDOLRTvM9rTZWA=.82a1bcf2-d76b-48ae-81ac-f8dcb981aeb8@github.com> Message-ID: On Wed, 8 Dec 2021 00:49:49 GMT, Serguei Spitsyn wrote: >> Fix for java.lang.instrument.Instrumentation.retransformClasses() >> >> Cases for retransformation primitive to identity (and identity to primitive) classes and changing primitive class fields (so default_value becomes invalid) are handled by existing code and spec changes are not required. >> >> Also added serviceability test to hotspot_valhalla group > > src/hotspot/share/prims/jvmtiClassFileReconstituter.cpp line 886: > >> 884: >> 885: // JVMSpec| u2 access_flags; >> 886: write_u2(ik()->access_flags().get_flags() & (JVM_RECOGNIZED_CLASS_MODIFIERS | JVM_ACC_INLINE)); > > Would it better to consider adding JVM_ACC_INLINE to JVM_RECOGNIZED_CLASS_MODIFIERS? > Also, I see other definitions of the JVM_RECOGNIZED_CLASS_MODIFIERS in files: > > src/hotspot/share/include/jvm_constants.h > src/jdk.hotspot.agent/share/classes/sun/jvm/hotspot/runtime/ClassConstants.java > > Do we want this modifier added there as well? The problem is that JVM_RECOGNIZED_CLASS_MODIFIERS contains all modifiers defined for JDK8 and earlier, but the validity of new modifiers depends on class file version (see classFileParser.cpp where JVM_RECOGNIZED_CLASS_MODIFIERS is also used). ------------- PR: https://git.openjdk.java.net/valhalla/pull/588 From amenkov at openjdk.java.net Thu Dec 9 00:48:16 2021 From: amenkov at openjdk.java.net (Alex Menkov) Date: Thu, 9 Dec 2021 00:48:16 GMT Subject: [lworld] RFR: 8274800: [lworld] Primitive classes can't be retransformed In-Reply-To: References: <9xCk7tXonaf_XjWjc4XgjWtwOtXiajDOLRTvM9rTZWA=.82a1bcf2-d76b-48ae-81ac-f8dcb981aeb8@github.com> Message-ID: On Tue, 7 Dec 2021 19:44:17 GMT, Frederic Parain wrote: > > I clicked on approved a bit too quickly. In jvmtiRedefineClass.cpp, the withfield bytecode has been added, but not the defaultvalue bytecode. > I didn't get any failures for defaultvalue. I suppose index in CP is always the same. But I think I need to add it too. Will try to develop testcase for this ------------- PR: https://git.openjdk.java.net/valhalla/pull/588 From thartmann at openjdk.java.net Wed Dec 8 09:36:47 2021 From: thartmann at openjdk.java.net (Tobias Hartmann) Date: Wed, 8 Dec 2021 09:36:47 GMT Subject: [lworld] RFR: 8278350: [lworld] Compiler test failures after merge with mainline Message-ID: Added two missing `@ForceInline` annotations and increased warmup iterations for a test now that [JDK-8276546](https://bugs.openjdk.java.net/browse/JDK-8276546) was merged in and IR verification is enabled with `-XX:CICompileThreshold=100`. Thanks, Tobias ------------- Commit messages: - 8278350: [lworld] Compiler test failures after merge with mainline Changes: https://git.openjdk.java.net/valhalla/pull/589/files Webrev: https://webrevs.openjdk.java.net/?repo=valhalla&pr=589&range=00 Issue: https://bugs.openjdk.java.net/browse/JDK-8278350 Stats: 3 lines in 2 files changed: 3 ins; 0 del; 0 mod Patch: https://git.openjdk.java.net/valhalla/pull/589.diff Fetch: git fetch https://git.openjdk.java.net/valhalla pull/589/head:pull/589 PR: https://git.openjdk.java.net/valhalla/pull/589 From thartmann at openjdk.java.net Wed Dec 8 14:42:31 2021 From: thartmann at openjdk.java.net (Tobias Hartmann) Date: Wed, 8 Dec 2021 14:42:31 GMT Subject: [lworld] Integrated: 8278352: [lworld] nmethod assembly snippet in hs_err file misses method name and parameters In-Reply-To: References: Message-ID: On Wed, 8 Dec 2021 13:54:36 GMT, Tobias Hartmann wrote: > Some users of `nmethod::maybe_print_nmethod` do not set the `_nmethod_to_print` field and as a result, the method name and arguments are not printed. Now that the `ProtectionDomainSet_lock` has been removed by [JDK-8259242](https://bugs.openjdk.java.net/browse/JDK-8259242) we can also remove the workaround code and move the computation of the calling convention into the print method. > > In addition, the regex in the MachCodeFramesInErrorFile test does not properly handle the labels for multiple entry points (for details, see bug report). I adjusted it accordingly. > > Best regards, > Tobias This pull request has now been integrated. Changeset: e49b8508 Author: Tobias Hartmann URL: https://git.openjdk.java.net/valhalla/commit/e49b8508781b3f34e5a62141575dad0ca3798491 Stats: 28 lines in 2 files changed: 1 ins; 18 del; 9 mod 8278352: [lworld] nmethod assembly snippet in hs_err file misses method name and parameters ------------- PR: https://git.openjdk.java.net/valhalla/pull/590 From fparain at openjdk.java.net Tue Dec 7 19:44:30 2021 From: fparain at openjdk.java.net (Frederic Parain) Date: Tue, 7 Dec 2021 19:44:30 GMT Subject: [lworld] RFR: 8274800: [lworld] Primitive classes can't be retransformed In-Reply-To: <9xCk7tXonaf_XjWjc4XgjWtwOtXiajDOLRTvM9rTZWA=.82a1bcf2-d76b-48ae-81ac-f8dcb981aeb8@github.com> References: <9xCk7tXonaf_XjWjc4XgjWtwOtXiajDOLRTvM9rTZWA=.82a1bcf2-d76b-48ae-81ac-f8dcb981aeb8@github.com> Message-ID: On Sat, 4 Dec 2021 00:30:11 GMT, Alex Menkov wrote: > Fix for java.lang.instrument.Instrumentation.retransformClasses() > > Cases for retransformation primitive to identity (and identity to primitive) classes and changing primitive class fields (so default_value becomes invalid) are handled by existing code and spec changes are not required. > > Also added serviceability test to hotspot_valhalla group Looks good to me. Thank you for writing the test. Fred ------------- Marked as reviewed by fparain (Committer). PR: https://git.openjdk.java.net/valhalla/pull/588 From thartmann at openjdk.java.net Wed Dec 8 14:00:55 2021 From: thartmann at openjdk.java.net (Tobias Hartmann) Date: Wed, 8 Dec 2021 14:00:55 GMT Subject: [lworld] RFR: 8278352: [lworld] nmethod assembly snippet in hs_err file misses method name and parameters Message-ID: Some users of `nmethod::maybe_print_nmethod` do not set the `_nmethod_to_print` field and as a result, the method name and arguments are not printed. Now that the `ProtectionDomainSet_lock` has been removed by [JDK-8259242](https://bugs.openjdk.java.net/browse/JDK-8259242) we can also remove the workaround code and move the computation of the calling convention into the print method. In addition, the regex in the MachCodeFramesInErrorFile test does not properly handle the labels for multiple entry points (for details, see bug report). I adjusted it accordingly. Best regards, Tobias ------------- Commit messages: - 8278352: [lworld] nmethod assembly snippet in hs_err file misses method name and parameters Changes: https://git.openjdk.java.net/valhalla/pull/590/files Webrev: https://webrevs.openjdk.java.net/?repo=valhalla&pr=590&range=00 Issue: https://bugs.openjdk.java.net/browse/JDK-8278352 Stats: 28 lines in 2 files changed: 1 ins; 18 del; 9 mod Patch: https://git.openjdk.java.net/valhalla/pull/590.diff Fetch: git fetch https://git.openjdk.java.net/valhalla pull/590/head:pull/590 PR: https://git.openjdk.java.net/valhalla/pull/590 From sspitsyn at openjdk.java.net Wed Dec 8 00:56:44 2021 From: sspitsyn at openjdk.java.net (Serguei Spitsyn) Date: Wed, 8 Dec 2021 00:56:44 GMT Subject: [lworld] RFR: 8274800: [lworld] Primitive classes can't be retransformed In-Reply-To: <9xCk7tXonaf_XjWjc4XgjWtwOtXiajDOLRTvM9rTZWA=.82a1bcf2-d76b-48ae-81ac-f8dcb981aeb8@github.com> References: <9xCk7tXonaf_XjWjc4XgjWtwOtXiajDOLRTvM9rTZWA=.82a1bcf2-d76b-48ae-81ac-f8dcb981aeb8@github.com> Message-ID: On Sat, 4 Dec 2021 00:30:11 GMT, Alex Menkov wrote: > Fix for java.lang.instrument.Instrumentation.retransformClasses() > > Cases for retransformation primitive to identity (and identity to primitive) classes and changing primitive class fields (so default_value becomes invalid) are handled by existing code and spec changes are not required. > > Also added serviceability test to hotspot_valhalla group Marked as reviewed by sspitsyn (no project role). src/hotspot/share/prims/jvmtiClassFileReconstituter.cpp line 886: > 884: > 885: // JVMSpec| u2 access_flags; > 886: write_u2(ik()->access_flags().get_flags() & (JVM_RECOGNIZED_CLASS_MODIFIERS | JVM_ACC_INLINE)); Would it better to consider adding JVM_ACC_INLINE to JVM_RECOGNIZED_CLASS_MODIFIERS? Also, I see other definitions of the JVM_RECOGNIZED_CLASS_MODIFIERS in files: src/hotspot/share/include/jvm_constants.h src/jdk.hotspot.agent/share/classes/sun/jvm/hotspot/runtime/ClassConstants.java Do we want this modifier added there as well? test/hotspot/jtreg/TEST.groups line 58: > 56: runtime/valhalla \ > 57: compiler/valhalla \ > 58: serviceability/jvmti/Valhalla For consistency, should the we rename the test sub-folder from `Valhalla` to `valhalla` ? This comment is not about this current fix though. ------------- PR: https://git.openjdk.java.net/valhalla/pull/588 From sspitsyn at openjdk.java.net Wed Dec 8 13:44:40 2021 From: sspitsyn at openjdk.java.net (Serguei Spitsyn) Date: Wed, 8 Dec 2021 13:44:40 GMT Subject: [lworld] RFR: 8274800: [lworld] Primitive classes can't be retransformed In-Reply-To: References: <9xCk7tXonaf_XjWjc4XgjWtwOtXiajDOLRTvM9rTZWA=.82a1bcf2-d76b-48ae-81ac-f8dcb981aeb8@github.com> Message-ID: On Wed, 8 Dec 2021 01:20:43 GMT, Frederic Parain wrote: >> src/hotspot/share/prims/jvmtiClassFileReconstituter.cpp line 886: >> >>> 884: >>> 885: // JVMSpec| u2 access_flags; >>> 886: write_u2(ik()->access_flags().get_flags() & (JVM_RECOGNIZED_CLASS_MODIFIERS | JVM_ACC_INLINE)); >> >> Would it better to consider adding JVM_ACC_INLINE to JVM_RECOGNIZED_CLASS_MODIFIERS? >> Also, I see other definitions of the JVM_RECOGNIZED_CLASS_MODIFIERS in files: >> >> src/hotspot/share/include/jvm_constants.h >> src/jdk.hotspot.agent/share/classes/sun/jvm/hotspot/runtime/ClassConstants.java >> >> Do we want this modifier added there as well? > > The problem is that JVM_RECOGNIZED_CLASS_MODIFIERS contains all modifiers defined for JDK8 and earlier, but the validity of new modifiers depends on class file version (see classFileParser.cpp where JVM_RECOGNIZED_CLASS_MODIFIERS is also used). Thanks, Frederic. Of course, it is always possible to add this modifier conditionally depending on the release. But it is not that elegant. ------------- PR: https://git.openjdk.java.net/valhalla/pull/588 From thartmann at openjdk.java.net Wed Dec 8 10:40:29 2021 From: thartmann at openjdk.java.net (Tobias Hartmann) Date: Wed, 8 Dec 2021 10:40:29 GMT Subject: [lworld] Integrated: 8278350: [lworld] Compiler test failures after merge with mainline In-Reply-To: References: Message-ID: On Wed, 8 Dec 2021 09:31:19 GMT, Tobias Hartmann wrote: > Added two missing `@ForceInline` annotations and increased warmup iterations for a test now that [JDK-8276546](https://bugs.openjdk.java.net/browse/JDK-8276546) was merged in and IR verification is enabled with `-XX:CICompileThreshold=100`. > > Thanks, > Tobias This pull request has now been integrated. Changeset: 4bbbdaa8 Author: Tobias Hartmann URL: https://git.openjdk.java.net/valhalla/commit/4bbbdaa8da4d4f92a7a63594b8f66c95420a6b84 Stats: 3 lines in 2 files changed: 3 ins; 0 del; 0 mod 8278350: [lworld] Compiler test failures after merge with mainline ------------- PR: https://git.openjdk.java.net/valhalla/pull/589 From john.r.rose at oracle.com Sun Dec 12 00:42:23 2021 From: john.r.rose at oracle.com (John Rose) Date: Sat, 11 Dec 2021 16:42:23 -0800 Subject: a Saturday puzzler: streaming over variable-length data Message-ID: <8C508557-8DD8-43A8-9638-1DD8C80D6127@oracle.com> Here?s a puzzler (actually a family of puzzlers) that occurred to me. Suppose I, as a Java and Panama programmer, need to communicate arrays of strings with native code. To be specific, let?s talk only about null-terminated UTF8 strings (on the native side). (To be very very general, all of the rest of this discusses uses UTF8 strings as a ?for instance? example, and in fact any kind of self-delimiting variable length data would be about as interesting and informative. For example, var-ints in the classic form of ?bit seven means read more bytes after this one?. Moreover, if the data is self-synchronizing, as with strings and var-ints, you can write a spliterator over it for parallel stream processing.) So, how do I make a stream over a memory segment of type `char*` that consists of a series of zero-terminated UTF8 strings, back to back? The answers differ a little depending on loop termination: 0. start with a predefined count of the strings to decode, or 1. keep going right up to the upper bound of the segment, or 2. stop when an empty string (a pair of null bytes) is found. Bonus points for avoiding double scanning of the strings. This means `MS::getUtf8String` is not necessarily the best tool. (But what is, then?) Second puzzler: How to do the whole thing backwards? That is, convert a stream of Java strings back into a memory segment containing the UTF8 string bodies concatenated with trailing nulls. The result is disposed of one of these ways: A. Allocate a fresh native MS in a given session. B. Allocate a fresh heap MS in the global session. C. Store the data into a given MS at a given offset, returning the new offset, and indicating if there is more that didn?t fit. (And allowing restart at that offset, for recovery code.) The number of converted strings is also reported, corresponding to the above options: 0. return the count of strings encoded and do nothing more, or 1. return a segment whose upper bound is after the last string?s null, or 2. encode an extra empty string (a pair of null bytes) at the end. And two more puzzlers pop into mind, for an argv/envp array, of type `char**`. Here, I suppose that the stream that reads the things will walk over a pointer to the `char**` array and read each `char*` item, decoding as it goes. Again the number of items to decode can be determined 0. with a predefined array-length for the strings to decode, or 1. read to the upper bound of the segment holding the array, or 2. stop when a `NULL` array element is found. For the reverse encoding there are a bunch of options: A. Allocate a *single* fresh native MS in a given session. B. Allocate a *pair* of MS?s, with the string bodies in a native MS in a given session. C. Store the data into one or two given MS?s at one or two a given offsets, etc. And the count can similarly be represented: 0. return the count of strings encoded and do nothing more, or 1. return a segment whose upper bound is after the last array element, or 2. encode an extra empty `NULL` element at the end of the output array The options C above are tricky but some applications may choose to embrace the complexity in order to reduce end-to-end copying. That in turn suggests that maybe someone should create a library to manage blocks of working storage that accumulate data structures for eventual posting to other native code. A very highly developed framework might also emphasize pointer-free and/or position-independent data structures. The various structured packet frameworks do this, one way or another. My favorite is https://capnproto.org ! From john.r.rose at oracle.com Sun Dec 12 00:58:01 2021 From: john.r.rose at oracle.com (John Rose) Date: Sat, 11 Dec 2021 16:58:01 -0800 Subject: a Saturday puzzler: streaming over variable-length data In-Reply-To: <8C508557-8DD8-43A8-9638-1DD8C80D6127@oracle.com> References: <8C508557-8DD8-43A8-9638-1DD8C80D6127@oracle.com> Message-ID: <87ABBA89-F27A-457A-ABFF-DC922AF172F8@oracle.com> Apologies: That was meant for panama-dev. Please disregard. On 11 Dec 2021, at 16:42, John Rose wrote: > Here?s a puzzler (actually a family of puzzlers) that occurred to > me. > > Suppose I, as a Java and Panama programmer, need to communicate arrays > of strings with native code. > > To be specific, let?s talk only about null-terminated UTF8 strings > (on the native side). > > (To be very very general, all of the rest of this discusses uses UTF8 > strings as a ?for instance? example, and in fact any kind of > self-delimiting variable length data would be about as interesting and > informative. For example, var-ints in the classic form of ?bit > seven means read more bytes after this one?. Moreover, if the data > is self-synchronizing, as with strings and var-ints, you can write a > spliterator over it for parallel stream processing.) > > So, how do I make a stream over a memory segment of type `char*` that > consists of a series of zero-terminated UTF8 strings, back to back? > > The answers differ a little depending on loop termination: > > 0. start with a predefined count of the strings to decode, or > 1. keep going right up to the upper bound of the segment, or > 2. stop when an empty string (a pair of null bytes) is found. > > Bonus points for avoiding double scanning of the strings. This means > `MS::getUtf8String` is not necessarily the best tool. (But what is, > then?) > > Second puzzler: How to do the whole thing backwards? That is, > convert a stream of Java strings back into a memory segment containing > the UTF8 string bodies concatenated with trailing nulls. > > The result is disposed of one of these ways: > > A. Allocate a fresh native MS in a given session. > B. Allocate a fresh heap MS in the global session. > C. Store the data into a given MS at a given offset, returning the new > offset, and indicating if there is more that didn?t fit. (And > allowing restart at that offset, for recovery code.) > > The number of converted strings is also reported, corresponding to the > above options: > > 0. return the count of strings encoded and do nothing more, or > 1. return a segment whose upper bound is after the last string?s > null, or > 2. encode an extra empty string (a pair of null bytes) at the end. > > And two more puzzlers pop into mind, for an argv/envp array, of type > `char**`. Here, I suppose that the stream that reads the things will > walk over a pointer to the `char**` array and read each `char*` item, > decoding as it goes. > > Again the number of items to decode can be determined > > 0. with a predefined array-length for the strings to decode, or > 1. read to the upper bound of the segment holding the array, or > 2. stop when a `NULL` array element is found. > > For the reverse encoding there are a bunch of options: > > A. Allocate a *single* fresh native MS in a given session. > B. Allocate a *pair* of MS?s, with the string bodies in a native MS > in a given session. > C. Store the data into one or two given MS?s at one or two a given > offsets, etc. > > And the count can similarly be represented: > > 0. return the count of strings encoded and do nothing more, or > 1. return a segment whose upper bound is after the last array element, > or > 2. encode an extra empty `NULL` element at the end of the output array > > The options C above are tricky but some applications may choose to > embrace the complexity in order to reduce end-to-end copying. That in > turn suggests that maybe someone should create a library to manage > blocks of working storage that accumulate data structures for eventual > posting to other native code. > > A very highly developed framework might also emphasize pointer-free > and/or position-independent data structures. The various structured > packet frameworks do this, one way or another. My favorite is > https://capnproto.org ! From stigdoessing at gmail.com Thu Dec 23 22:13:03 2021 From: stigdoessing at gmail.com (=?UTF-8?Q?Stig_Rohde_D=C3=B8ssing?=) Date: Thu, 23 Dec 2021 23:13:03 +0100 Subject: Slight inconsistency between state of valhalla and the value objects draft JEP Message-ID: Hi, The updated state of valhalla at https://openjdk.java.net/projects/valhalla/design-notes/state-of-valhalla/03-vm-model#flattening-and-representation says that final fields containing value objects can regularly be flattened, even when using the L descriptor. It also says that value objects might not be flattenable if referenced through a non-sealed interface, which I understand to mean that if it is referenced through a sealed interface, we might not have to pay the polymorphism tax, and might still be able to flatten? The draft JEP at https://openjdk.java.net/jeps/8277163 is more conservative, and says that value objects in fields and value objects viewed as an instance of a supertype will be heap allocated. Is the JEP just being a bit too conservative? It sounds great if flattening is still possible as long as the fields are final or the superinterface is sealed. I think this would make a big difference for Scala's Option type, which is a sealed interface. Completely unrelated, I noticed in https://openjdk.java.net/projects/valhalla/design-notes/state-of-valhalla/02-object-model#identity-sensitive-operations, the System::IdentitiyHashCode section references primitive objects, which are introduced later in the document. I'm guessing it is supposed to reference value objects instead? From forax at univ-mlv.fr Thu Dec 23 23:28:29 2021 From: forax at univ-mlv.fr (Remi Forax) Date: Fri, 24 Dec 2021 00:28:29 +0100 (CET) Subject: Slight inconsistency between state of valhalla and the value objects draft JEP In-Reply-To: References: Message-ID: <1325146545.1294177.1640302109196.JavaMail.zimbra@u-pem.fr> ----- Original Message ----- > From: "Stig Rohde D?ssing" > To: "valhalla-dev" > Sent: Thursday, December 23, 2021 11:13:03 PM > Subject: Slight inconsistency between state of valhalla and the value objects draft JEP > Hi, > > The updated state of valhalla at > https://openjdk.java.net/projects/valhalla/design-notes/state-of-valhalla/03-vm-model#flattening-and-representation > says that final fields containing value objects can regularly be flattened, > even when using the L descriptor. It also says that value objects might not > be flattenable if referenced through a non-sealed interface, which I > understand to mean that if it is referenced through a sealed interface, we > might not have to pay the polymorphism tax, and might still be able to > flatten? > > The draft JEP at https://openjdk.java.net/jeps/8277163 is more > conservative, and says that value objects in fields and value objects > viewed as an instance of a supertype will be heap allocated. > > Is the JEP just being a bit too conservative? It sounds great if flattening > is still possible as long as the fields are final or the superinterface is > sealed. I think this would make a big difference for Scala's Option type, > which is a sealed interface. To be able to flatten on heap, you need - the type to be loaded eagerly (a Q-type or a L-type in the Preload table) - I see no issue to store a sealed interface in the Preload table - if it's a primitive type, it should not be too big - if it's a value type, it should be <= 64 bits knowing that you need a bit for nullability (in practice a byte) - if it's a sealed interface, you need several bits to store the index of the permitted subtypes (in practice a byte but you may reuse the same byte as above (255 or 128 permitted subtypes seems a lot)) So for Option, if the VM is using compressed oops (so no ZGC !), it should be okay. Now, does this heroic bits packing stuff will be implemented one day, maybe, the VM already does bits packing in the object header after all, but don't expect this to be implemented when Valhalla will be first released. R?mi From stigdoessing at gmail.com Fri Dec 24 08:51:32 2021 From: stigdoessing at gmail.com (=?UTF-8?Q?Stig_Rohde_D=C3=B8ssing?=) Date: Fri, 24 Dec 2021 09:51:32 +0100 Subject: Slight inconsistency between state of valhalla and the value objects draft JEP In-Reply-To: <1325146545.1294177.1640302109196.JavaMail.zimbra@u-pem.fr> References: <1325146545.1294177.1640302109196.JavaMail.zimbra@u-pem.fr> Message-ID: Thanks R?mi, that sounds great. Being able to flatten in fields (even if only sometimes) is very nice. I was just concerned to read that viewing a value object as an instance of a supertype would disable flattening for parameters/return values, but if this can be avoided when the supertype is sealed, that will work well enough for Option I think. Den fre. 24. dec. 2021 kl. 00.28 skrev Remi Forax : > ----- Original Message ----- > > From: "Stig Rohde D?ssing" > > To: "valhalla-dev" > > Sent: Thursday, December 23, 2021 11:13:03 PM > > Subject: Slight inconsistency between state of valhalla and the value > objects draft JEP > > > Hi, > > > > The updated state of valhalla at > > > https://openjdk.java.net/projects/valhalla/design-notes/state-of-valhalla/03-vm-model#flattening-and-representation > > says that final fields containing value objects can regularly be > flattened, > > even when using the L descriptor. It also says that value objects might > not > > be flattenable if referenced through a non-sealed interface, which I > > understand to mean that if it is referenced through a sealed interface, > we > > might not have to pay the polymorphism tax, and might still be able to > > flatten? > > > > The draft JEP at https://openjdk.java.net/jeps/8277163 is more > > conservative, and says that value objects in fields and value objects > > viewed as an instance of a supertype will be heap allocated. > > > > Is the JEP just being a bit too conservative? It sounds great if > flattening > > is still possible as long as the fields are final or the superinterface > is > > sealed. I think this would make a big difference for Scala's Option type, > > which is a sealed interface. > > To be able to flatten on heap, you need > - the type to be loaded eagerly (a Q-type or a L-type in the Preload table) > - I see no issue to store a sealed interface in the Preload table > - if it's a primitive type, it should not be too big > - if it's a value type, it should be <= 64 bits knowing that you need a > bit for nullability > (in practice a byte) > - if it's a sealed interface, you need several bits to store the index of > the permitted subtypes > (in practice a byte but you may reuse the same byte as above (255 or 128 > permitted subtypes seems a lot)) > > So for Option, if the VM is using compressed oops (so no ZGC !), it should > be okay. > > Now, does this heroic bits packing stuff will be implemented one day, > maybe, the VM already does bits packing in the object header after all, > but don't expect this to be implemented when Valhalla will be first > released. > > R?mi > From john.r.rose at oracle.com Fri Dec 24 22:31:36 2021 From: john.r.rose at oracle.com (John Rose) Date: Fri, 24 Dec 2021 22:31:36 +0000 Subject: Slight inconsistency between state of valhalla and the value objects draft JEP In-Reply-To: <1325146545.1294177.1640302109196.JavaMail.zimbra@u-pem.fr> References: <1325146545.1294177.1640302109196.JavaMail.zimbra@u-pem.fr> Message-ID: <7D2E740D-6DB9-4DDD-9B29-99C8239AE454@oracle.com> You can use sentinel values as well as tag bytes in some cases like Optional, which amounts to a fractional bit of overhead instead of one or eight bits. That should work for Optional and many other one-field types. None of these tactics are guaranteed by the JVMS. It?s to be expected that different docs will set slightly different expectations for optimizations as long as they agree on the required contracts. One should expect that the performance model will evolve over time (within a stable JVMS) even though we are saying now what we think what will be true. > On Dec 23, 2021, at 3:28 PM, Remi Forax wrote: > > - if it's a sealed interface, you need several bits to store the index of the permitted subtypes > (in practice a byte but you may reuse the same byte as above (255 or 128 permitted subtypes seems a lot))