From mcimadamore at openjdk.java.net Mon Nov 1 12:05:32 2021 From: mcimadamore at openjdk.java.net (Maurizio Cimadamore) Date: Mon, 1 Nov 2021 12:05:32 GMT Subject: RFR: 8275063: Implementation of Foreign Function & Memory API (Second incubator) [v10] In-Reply-To: References: Message-ID: > This PR contains the API and implementation changes for JEP-419 [1]. A more detailed description of such changes, to avoid repetitions during the review process, is included as a separate comment. > > [1] - https://openjdk.java.net/jeps/419 Maurizio Cimadamore has updated the pull request with a new target base due to a merge or a rebase. The pull request now contains 17 commits: - Add cache for memory address var handles - Merge branch 'master' into JEP-419 - Fix regression in VaList treatment on AArch64 (contributed by @nick-arm) - Merge branch 'master' into JEP-419 - Fix copyright header in TestArrayCopy - Fix failing microbenchmarks. Contributed by @FrauBoes (thanks!) - * use `invokeWithArguments` to simplify new test - Add test for liveness check with high-aririty downcalls (make sure that if an exception occurs in a downcall because of liveness, ref count of other resources are left intact). - * Fix javadoc issue in VaList * Fix bug in concurrent logic for shared scope acquire - Address review comments - ... and 7 more: https://git.openjdk.java.net/jdk/compare/5bb1992b...9b519343 ------------- Changes: https://git.openjdk.java.net/jdk/pull/5907/files Webrev: https://webrevs.openjdk.java.net/?repo=jdk&pr=5907&range=09 Stats: 14497 lines in 189 files changed: 6773 ins; 5149 del; 2575 mod Patch: https://git.openjdk.java.net/jdk/pull/5907.diff Fetch: git fetch https://git.openjdk.java.net/jdk pull/5907/head:pull/5907 PR: https://git.openjdk.java.net/jdk/pull/5907 From mcimadamore at openjdk.java.net Mon Nov 1 17:15:55 2021 From: mcimadamore at openjdk.java.net (Maurizio Cimadamore) Date: Mon, 1 Nov 2021 17:15:55 GMT Subject: RFR: 8275063: Implementation of Foreign Function & Memory API (Second incubator) [v11] In-Reply-To: References: Message-ID: <8DLqVOZo6ZXYqntQe91nI4wIKu0_gn0DY-l8MA2rznM=.fdab6f3c-119e-492d-b61c-6314d51cdd58@github.com> > This PR contains the API and implementation changes for JEP-419 [1]. A more detailed description of such changes, to avoid repetitions during the review process, is included as a separate comment. > > [1] - https://openjdk.java.net/jeps/419 Maurizio Cimadamore has updated the pull request incrementally with one additional commit since the last revision: Fix liveness issue with loader lookups ------------- Changes: - all: https://git.openjdk.java.net/jdk/pull/5907/files - new: https://git.openjdk.java.net/jdk/pull/5907/files/9b519343..17f45861 Webrevs: - full: https://webrevs.openjdk.java.net/?repo=jdk&pr=5907&range=10 - incr: https://webrevs.openjdk.java.net/?repo=jdk&pr=5907&range=09-10 Stats: 191 lines in 6 files changed: 187 ins; 0 del; 4 mod Patch: https://git.openjdk.java.net/jdk/pull/5907.diff Fetch: git fetch https://git.openjdk.java.net/jdk pull/5907/head:pull/5907 PR: https://git.openjdk.java.net/jdk/pull/5907 From mcimadamore at openjdk.java.net Mon Nov 1 22:36:40 2021 From: mcimadamore at openjdk.java.net (Maurizio Cimadamore) Date: Mon, 1 Nov 2021 22:36:40 GMT Subject: RFR: 8275063: Implementation of Foreign Function & Memory API (Second incubator) [v12] In-Reply-To: References: Message-ID: > This PR contains the API and implementation changes for JEP-419 [1]. A more detailed description of such changes, to avoid repetitions during the review process, is included as a separate comment. > > [1] - https://openjdk.java.net/jeps/419 Maurizio Cimadamore has updated the pull request incrementally with one additional commit since the last revision: Tweak javadoc of loaderLookup ------------- Changes: - all: https://git.openjdk.java.net/jdk/pull/5907/files - new: https://git.openjdk.java.net/jdk/pull/5907/files/17f45861..7cf4fcd9 Webrevs: - full: https://webrevs.openjdk.java.net/?repo=jdk&pr=5907&range=11 - incr: https://webrevs.openjdk.java.net/?repo=jdk&pr=5907&range=10-11 Stats: 2 lines in 1 file changed: 0 ins; 0 del; 2 mod Patch: https://git.openjdk.java.net/jdk/pull/5907.diff Fetch: git fetch https://git.openjdk.java.net/jdk pull/5907/head:pull/5907 PR: https://git.openjdk.java.net/jdk/pull/5907 From psandoz at openjdk.java.net Tue Nov 2 00:27:23 2021 From: psandoz at openjdk.java.net (Paul Sandoz) Date: Tue, 2 Nov 2021 00:27:23 GMT Subject: RFR: 8275063: Implementation of Foreign Function & Memory API (Second incubator) [v10] In-Reply-To: References: Message-ID: On Mon, 1 Nov 2021 12:05:32 GMT, Maurizio Cimadamore wrote: >> This PR contains the API and implementation changes for JEP-419 [1]. A more detailed description of such changes, to avoid repetitions during the review process, is included as a separate comment. >> >> [1] - https://openjdk.java.net/jeps/419 > > Maurizio Cimadamore has updated the pull request with a new target base due to a merge or a rebase. The pull request now contains 17 commits: > > - Add cache for memory address var handles > - Merge branch 'master' into JEP-419 > - Fix regression in VaList treatment on AArch64 (contributed by @nick-arm) > - Merge branch 'master' into JEP-419 > - Fix copyright header in TestArrayCopy > - Fix failing microbenchmarks. Contributed by @FrauBoes (thanks!) > - * use `invokeWithArguments` to simplify new test > - Add test for liveness check with high-aririty downcalls > (make sure that if an exception occurs in a downcall because of liveness, > ref count of other resources are left intact). > - * Fix javadoc issue in VaList > * Fix bug in concurrent logic for shared scope acquire > - Address review comments > - ... and 7 more: https://git.openjdk.java.net/jdk/compare/5bb1992b...9b519343 src/jdk.incubator.foreign/share/classes/jdk/internal/foreign/Utils.java line 111: > 109: class VarHandleCache { > 110: private static final Map handleMap = new ConcurrentHashMap<>(); > 111: private static final Map handleMapNoAlignCheck = new ConcurrentHashMap<>(); Something to consider later if this is an issue. Since the number of `ValueLayout` instances is fixed, carrier x order = 18, we can use stable arrays with ordinals on the instances. ------------- PR: https://git.openjdk.java.net/jdk/pull/5907 From ysatowse at openjdk.java.net Tue Nov 2 01:26:09 2021 From: ysatowse at openjdk.java.net (Yoshiki Sato) Date: Tue, 2 Nov 2021 01:26:09 GMT Subject: RFR: 8275766: (tz) Update Timezone Data to 2021e In-Reply-To: References: Message-ID: On Thu, 28 Oct 2021 01:02:27 GMT, Yoshiki Sato wrote: > Please review the integration of tzdata2021e (including tzdata2021d) to the JDK. > The fix has passed all relevant JTREG regression tests and JCK tests. > > 8275754: (tz) Update Timezone Data to 2021d > 8275849: TestZoneInfo310.java fails with tzdata2021e @coffeys please complete your review on this PR. ------------- PR: https://git.openjdk.java.net/jdk/pull/6144 From coffeys at openjdk.java.net Tue Nov 2 09:26:09 2021 From: coffeys at openjdk.java.net (Sean Coffey) Date: Tue, 2 Nov 2021 09:26:09 GMT Subject: RFR: 8275766: (tz) Update Timezone Data to 2021e In-Reply-To: References: Message-ID: <-bYG3g0xtNPZ3g5S_XUAjUwoui2kNWfOlGq4yLTMvMg=.90b063de-5c3b-41fd-b1c9-d1e10fb2a8bf@github.com> On Thu, 28 Oct 2021 01:02:27 GMT, Yoshiki Sato wrote: > Please review the integration of tzdata2021e (including tzdata2021d) to the JDK. > The fix has passed all relevant JTREG regression tests and JCK tests. > > 8275754: (tz) Update Timezone Data to 2021d > 8275849: TestZoneInfo310.java fails with tzdata2021e Marked as reviewed by coffeys (Reviewer). I think you've enough Reviewers already but yes, this looks fine. ------------- PR: https://git.openjdk.java.net/jdk/pull/6144 From mcimadamore at openjdk.java.net Tue Nov 2 10:34:18 2021 From: mcimadamore at openjdk.java.net (Maurizio Cimadamore) Date: Tue, 2 Nov 2021 10:34:18 GMT Subject: RFR: 8275063: Implementation of Foreign Function & Memory API (Second incubator) [v10] In-Reply-To: References: Message-ID: On Tue, 2 Nov 2021 00:24:12 GMT, Paul Sandoz wrote: >> Maurizio Cimadamore has updated the pull request with a new target base due to a merge or a rebase. The pull request now contains 17 commits: >> >> - Add cache for memory address var handles >> - Merge branch 'master' into JEP-419 >> - Fix regression in VaList treatment on AArch64 (contributed by @nick-arm) >> - Merge branch 'master' into JEP-419 >> - Fix copyright header in TestArrayCopy >> - Fix failing microbenchmarks. Contributed by @FrauBoes (thanks!) >> - * use `invokeWithArguments` to simplify new test >> - Add test for liveness check with high-aririty downcalls >> (make sure that if an exception occurs in a downcall because of liveness, >> ref count of other resources are left intact). >> - * Fix javadoc issue in VaList >> * Fix bug in concurrent logic for shared scope acquire >> - Address review comments >> - ... and 7 more: https://git.openjdk.java.net/jdk/compare/5bb1992b...9b519343 > > src/jdk.incubator.foreign/share/classes/jdk/internal/foreign/Utils.java line 111: > >> 109: class VarHandleCache { >> 110: private static final Map handleMap = new ConcurrentHashMap<>(); >> 111: private static final Map handleMapNoAlignCheck = new ConcurrentHashMap<>(); > > Something to consider later if this is an issue. Since the number of `ValueLayout` instances is fixed, carrier x order = 18, we can use stable arrays with ordinals on the instances. What about alignment? ------------- PR: https://git.openjdk.java.net/jdk/pull/5907 From hannesw at openjdk.java.net Tue Nov 2 12:14:17 2021 From: hannesw at openjdk.java.net (Hannes =?UTF-8?B?V2FsbG7DtmZlcg==?=) Date: Tue, 2 Nov 2021 12:14:17 GMT Subject: Integrated: JDK-8275406: Add copy-to-clipboard feature to snippet UI In-Reply-To: References: Message-ID: <4W2lEuwCfbaGZv07v957hcAY_sYPk-4MVIq5hbsStzw=.b8dc72fa-afa0-4953-9d8e-2d56e946d6d5@github.com> On Tue, 19 Oct 2021 13:51:03 GMT, Hannes Walln?fer wrote: > Please review a change to add a copy-to-clipboard feature to snippets. I took special care to make the feature usable on mobile devices. Sample output can be viewed and tested here: > > http://cr.openjdk.java.net/~hannesw/8275406/api.00/java.base/java/lang/Throwable.html#printStackTrace() This pull request has now been integrated. Changeset: 8630f55e Author: Hannes Walln?fer URL: https://git.openjdk.java.net/jdk/commit/8630f55ed7a0483ec5dcb13a7f53b52bc4ab6fc6 Stats: 180 lines in 14 files changed: 166 ins; 0 del; 14 mod 8275406: Add copy-to-clipboard feature to snippet UI Reviewed-by: erikj, jjg ------------- PR: https://git.openjdk.java.net/jdk/pull/6011 From ysatowse at openjdk.java.net Tue Nov 2 13:06:20 2021 From: ysatowse at openjdk.java.net (Yoshiki Sato) Date: Tue, 2 Nov 2021 13:06:20 GMT Subject: Integrated: 8275766: (tz) Update Timezone Data to 2021e In-Reply-To: References: Message-ID: On Thu, 28 Oct 2021 01:02:27 GMT, Yoshiki Sato wrote: > Please review the integration of tzdata2021e (including tzdata2021d) to the JDK. > The fix has passed all relevant JTREG regression tests and JCK tests. > > 8275754: (tz) Update Timezone Data to 2021d > 8275849: TestZoneInfo310.java fails with tzdata2021e This pull request has now been integrated. Changeset: 5b4e3986 Author: Yoshiki Sato Committer: Sean Coffey URL: https://git.openjdk.java.net/jdk/commit/5b4e39863dbc0d61e91675261dd6887f704ab868 Stats: 52 lines in 6 files changed: 29 ins; 6 del; 17 mod 8275766: (tz) Update Timezone Data to 2021e 8275849: TestZoneInfo310.java fails with tzdata2021e Reviewed-by: naoto, iris, erikj, coffeys ------------- PR: https://git.openjdk.java.net/jdk/pull/6144 From psandoz at openjdk.java.net Tue Nov 2 15:44:13 2021 From: psandoz at openjdk.java.net (Paul Sandoz) Date: Tue, 2 Nov 2021 15:44:13 GMT Subject: RFR: 8275063: Implementation of Foreign Function & Memory API (Second incubator) [v10] In-Reply-To: References: Message-ID: <5onID0SnzIoPH5_Le4f71eC5ll_zGn0DQecQVpL1jDM=.43d7b2af-185a-4251-828f-058da6a69115@github.com> On Tue, 2 Nov 2021 10:30:42 GMT, Maurizio Cimadamore wrote: >> src/jdk.incubator.foreign/share/classes/jdk/internal/foreign/Utils.java line 111: >> >>> 109: class VarHandleCache { >>> 110: private static final Map handleMap = new ConcurrentHashMap<>(); >>> 111: private static final Map handleMapNoAlignCheck = new ConcurrentHashMap<>(); >> >> Something to consider later if this is an issue. Since the number of `ValueLayout` instances is fixed, carrier x order = 18, we can use stable arrays with ordinals on the instances. > > What about alignment? Drat, `skipAlignmentCheck` misled me but perhaps there is still benefit for common constants with 8 bit and size alignment and fallback otherwise. ------------- PR: https://git.openjdk.java.net/jdk/pull/5907 From psandoz at openjdk.java.net Tue Nov 2 15:44:12 2021 From: psandoz at openjdk.java.net (Paul Sandoz) Date: Tue, 2 Nov 2021 15:44:12 GMT Subject: RFR: 8275063: Implementation of Foreign Function & Memory API (Second incubator) [v12] In-Reply-To: References: Message-ID: On Mon, 1 Nov 2021 22:36:40 GMT, Maurizio Cimadamore wrote: >> This PR contains the API and implementation changes for JEP-419 [1]. A more detailed description of such changes, to avoid repetitions during the review process, is included as a separate comment. >> >> [1] - https://openjdk.java.net/jeps/419 > > Maurizio Cimadamore has updated the pull request incrementally with one additional commit since the last revision: > > Tweak javadoc of loaderLookup Marked as reviewed by psandoz (Reviewer). ------------- PR: https://git.openjdk.java.net/jdk/pull/5907 From jvernee at openjdk.java.net Tue Nov 2 17:32:17 2021 From: jvernee at openjdk.java.net (Jorn Vernee) Date: Tue, 2 Nov 2021 17:32:17 GMT Subject: RFR: 8275063: Implementation of Foreign Function & Memory API (Second incubator) [v12] In-Reply-To: References: Message-ID: On Mon, 1 Nov 2021 22:36:40 GMT, Maurizio Cimadamore wrote: >> This PR contains the API and implementation changes for JEP-419 [1]. A more detailed description of such changes, to avoid repetitions during the review process, is included as a separate comment. >> >> [1] - https://openjdk.java.net/jeps/419 > > Maurizio Cimadamore has updated the pull request incrementally with one additional commit since the last revision: > > Tweak javadoc of loaderLookup Mostly some minor javadoc comments. src/java.base/share/classes/java/lang/Module.java line 32: > 30: import java.lang.annotation.Annotation; > 31: import java.lang.invoke.MethodHandle; > 32: import java.lang.invoke.VarHandle; These imports seem spurious now. src/jdk.incubator.foreign/share/classes/jdk/incubator/foreign/ValueLayout.java line 177: > 175: } > 176: if (carrier.isPrimitive() && Wrapper.forPrimitiveType(carrier).bitWidth() != size && > 177: carrier != boolean.class && size != 8) { I find this condition hard to parse, I'd suggest re-writing it as: if (carrier.isPrimitive()) { long expectedSize = carrier == boolean.class ? 8 : Wrapper.forPrimitiveType(carrier).bitWidth(); if (size != expectedSize) { throw ... } } (Maybe even change the `if` to an `else` and combine it with the above if). src/jdk.incubator.foreign/share/classes/jdk/incubator/foreign/ValueLayout.java line 484: > 482: public static final class OfAddress extends ValueLayout { > 483: OfAddress(ByteOrder order) { > 484: super(MemoryAddress.class, order, Unsafe.ADDRESS_SIZE * 8); I see `Unsafe.ADDRESS_SIZE` used in several places, suggest to maybe add an `ADDRESS_SIZE_BITS` constants somewhere (it's a bit more readable). src/jdk.incubator.foreign/share/classes/jdk/internal/foreign/ArenaAllocator.java line 42: > 40: final long blockSize; > 41: final long arenaSize; > 42: final ResourceScope scope; Could these field be made private? src/jdk.incubator.foreign/share/classes/jdk/internal/foreign/ArenaAllocator.java line 88: > 86: if (size > arenaSize) { > 87: throw new OutOfMemoryError(); > 88: } Isn't this already covered by the `finally` block? Also, this seems to be checking the unaltered `size`, which I think should have been already done at the end of the previous `allocate` call right? src/jdk.incubator.foreign/share/classes/jdk/internal/foreign/ResourceScopeImpl.java line 122: > 120: ResourceScopeImpl targetImpl = (ResourceScopeImpl)target; > 121: targetImpl.acquire0(); > 122: addCloseAction(targetImpl::release0); Maybe this should explicitly check if target is `null` (though the call to `acquire0` would also produce an NPE, the stack trace having Objects::requireNonNull in there would make the error more obvious I think). Suggestion: public void keepAlive(ResourceScope target) { Objects.requireNonNull(target); if (target == this) { throw new IllegalArgumentException("Invalid target scope."); } ResourceScopeImpl targetImpl = (ResourceScopeImpl)target; targetImpl.acquire0(); addCloseAction(targetImpl::release0); src/jdk.incubator.foreign/share/classes/jdk/internal/foreign/SharedScope.java line 101: > 99: int value; > 100: do { > 101: value = (int) STATE.getVolatile(jdk.internal.foreign.SharedScope.this); Doesn't need to be fully qualified I think? Suggestion: value = (int) STATE.getVolatile(this); src/jdk.incubator.foreign/share/classes/jdk/internal/foreign/SharedScope.java line 106: > 104: throw new IllegalStateException("Already closed"); > 105: } > 106: } while (!STATE.compareAndSet(jdk.internal.foreign.SharedScope.this, value, value - 1)); Same here Suggestion: } while (!STATE.compareAndSet(this, value, value - 1)); ------------- Marked as reviewed by jvernee (Reviewer). PR: https://git.openjdk.java.net/jdk/pull/5907 From jvernee at openjdk.java.net Tue Nov 2 17:32:24 2021 From: jvernee at openjdk.java.net (Jorn Vernee) Date: Tue, 2 Nov 2021 17:32:24 GMT Subject: RFR: 8275063: Implementation of Foreign Function & Memory API (Second incubator) [v10] In-Reply-To: References: Message-ID: On Mon, 1 Nov 2021 12:05:32 GMT, Maurizio Cimadamore wrote: >> This PR contains the API and implementation changes for JEP-419 [1]. A more detailed description of such changes, to avoid repetitions during the review process, is included as a separate comment. >> >> [1] - https://openjdk.java.net/jeps/419 > > Maurizio Cimadamore has updated the pull request with a new target base due to a merge or a rebase. The pull request now contains 17 commits: > > - Add cache for memory address var handles > - Merge branch 'master' into JEP-419 > - Fix regression in VaList treatment on AArch64 (contributed by @nick-arm) > - Merge branch 'master' into JEP-419 > - Fix copyright header in TestArrayCopy > - Fix failing microbenchmarks. Contributed by @FrauBoes (thanks!) > - * use `invokeWithArguments` to simplify new test > - Add test for liveness check with high-aririty downcalls > (make sure that if an exception occurs in a downcall because of liveness, > ref count of other resources are left intact). > - * Fix javadoc issue in VaList > * Fix bug in concurrent logic for shared scope acquire > - Address review comments > - ... and 7 more: https://git.openjdk.java.net/jdk/compare/5bb1992b...9b519343 src/java.base/share/classes/java/lang/invoke/MethodHandleImpl.java line 1586: > 1584: public void ensureCustomized(MethodHandle mh) { > 1585: mh.customize(); > 1586: } This is no longer needed, but it probably got picked up in the merge. src/java.base/share/classes/jdk/internal/access/JavaLangInvokeAccess.java line 144: > 142: * @param mh the method handle > 143: */ > 144: void ensureCustomized(MethodHandle mh); Same here, no longer needed. (it was used by now removed upcall handler code. See https://github.com/openjdk/panama-foreign/pull/553) src/jdk.incubator.foreign/share/classes/jdk/incubator/foreign/MemoryAddress.java line 107: > 105: * > 106: * @param offset offset in bytes (relative to this address). The final address of this read operation can be expressed as {@code toRowLongValue() + offset}. > 107: * @return a Java UTF-8 string containing all the bytes read from the given starting address ({@code toRowLongValue() + offset}) (see also comment on MemorySegment.getUtf8String) Suggestion: * @return a Java string constructed from the bytes read from the given starting address ({@code toRowLongValue() + offset}) src/jdk.incubator.foreign/share/classes/jdk/incubator/foreign/MemorySegment.java line 387: > 385: > 386: /** > 387: * Performs an element-wise bulk copy from given source segment to this segment. More specifically, the bytes at Suggestion: * Performs a byte-wise bulk copy from given source segment to this segment. More specifically, the bytes at src/jdk.incubator.foreign/share/classes/jdk/incubator/foreign/MemorySegment.java line 400: > 398: * a multiple of the source element layout size, if the source segment is incompatible with the alignment constraints > 399: * in the source element layout, or if this segment is incompatible with the alignment constraints > 400: * in the destination element layout. This speaks about element layouts, but I don't see any element layouts in the method implementation. src/jdk.incubator.foreign/share/classes/jdk/incubator/foreign/MemorySegment.java line 633: > 631: * java.nio.charset.CharsetDecoder} class should be used when more control > 632: * over the decoding process is required. > 633: * @param offset offset in bytes (relative to this segment). For instance, if this segment is a {@link #isNative()} segment, Suggestion: * @param offset offset in bytes (relative to this segment). For instance, if this segment is a {@link #isNative() native} segment, src/jdk.incubator.foreign/share/classes/jdk/incubator/foreign/MemorySegment.java line 636: > 634: * the final address of this read operation can be expressed as {@code address().toRowLongValue() + offset}. > 635: * @return a Java UTF-8 string containing all the bytes read from the given starting address up to (but not including) > 636: * the first {@code '\0'} terminator character (assuming one is found). The phrase "a Java UTF-8 string" sounds strange to me, as Java Strings are not encoded in UTF-8. The string that is read is UTF-8 encoded, but then it is converted from UTF-8 to Java internal String encoding (UTF-16 or Latin1). I'd suggest just dropping the 'UTF-8', and changing 'containing all' to 'constructed from'. Suggestion: * @return a Java string constructed from the bytes read from the given starting address up to (but not including) * the first {@code '\0'} terminator character (assuming one is found). src/jdk.incubator.foreign/share/classes/jdk/incubator/foreign/MemorySegment.java line 652: > 650: * java.nio.charset.CharsetDecoder} class should be used when more control > 651: * over the decoding process is required. > 652: * @param offset offset in bytes (relative to this segment). For instance, if this segment is a {@link #isNative()} segment, Suggestion: * @param offset offset in bytes (relative to this segment). For instance, if this segment is a {@link #isNative() native} segment, src/jdk.incubator.foreign/share/classes/jdk/incubator/foreign/MemorySegment.java line 762: > 760: > 761: /** > 762: * Creates a new native memory segment with given size and resource scope, and whose base address is this address. Suggestion: * Creates a new native memory segment with given size and resource scope, and whose base address is the given address. src/jdk.incubator.foreign/share/classes/jdk/incubator/foreign/MemorySegment.java line 769: > 767: * provided resource scope. > 768: *

> 769: * Clients should ensure that the address and bounds refers to a valid region of memory that is accessible for reading and, Suggestion: * Clients should ensure that the address and bounds refer to a valid region of memory that is accessible for reading and, src/jdk.incubator.foreign/share/classes/jdk/incubator/foreign/MemorySegment.java line 1035: > 1033: * > 1034: * @param layout the layout of the memory region to be read. > 1035: * @param offset offset in bytes (relative to this segment). For instance, if this segment is a {@link #isNative()} segment, Suggestion: * @param offset offset in bytes (relative to this segment). For instance, if this segment is a {@link #isNative() native} segment, src/jdk.incubator.foreign/share/classes/jdk/incubator/foreign/MemorySegment.java line 1549: > 1547: * @param index index (relative to this segment). For instance, if this segment is a {@link #isNative()} segment, > 1548: * the final address of this write operation can be expressed as {@code address().toRowLongValue() + (index * layout.byteSize())}. > 1549: * @param value the byte value to be written. Suggestion: * @param value the address value to be written. src/jdk.incubator.foreign/share/classes/jdk/incubator/foreign/MemorySegment.java line 1563: > 1561: * Copies a number of elements from a source segment to a destination array, > 1562: * starting at a given segment offset (expressed in bytes), and a given array index, using the given source element layout. > 1563: * Supported array types are {@code byte[]}, {@code char[]},{@code short[]},{@code int[]},{@code float[]},{@code long[]} and {@code double[]}. Suggestion: * Supported array types are {@code byte[]}, {@code char[]}, {@code short[]}, {@code int[]}, {@code float[]}, {@code long[]} and {@code double[]}. src/jdk.incubator.foreign/share/classes/jdk/incubator/foreign/MemorySegment.java line 1604: > 1602: * Copies a number of elements from a source array to a destination segment, > 1603: * starting at a given array index, and a given segment offset (expressed in bytes), using the given destination element layout. > 1604: * Supported array types are {@code byte[]}, {@code char[]},{@code short[]},{@code int[]},{@code float[]},{@code long[]} and {@code double[]}. Suggestion: * Supported array types are {@code byte[]}, {@code char[]}, {@code short[]}, {@code int[]}, {@code float[]}, {@code long[]} and {@code double[]}. src/jdk.incubator.foreign/share/classes/jdk/incubator/foreign/ResourceScope.java line 208: > 206: */ > 207: static ResourceScope newConfinedScope() { > 208: return ResourceScopeImpl.createConfined( Thread.currentThread(), null); Suggestion: return ResourceScopeImpl.createConfined(Thread.currentThread(), null); src/jdk.incubator.foreign/share/classes/jdk/incubator/foreign/VaList.java line 132: > 130: /** > 131: * Copies this variable argument list at its current position into a new variable argument list associated > 132: * with the same scope as this variable argument list. using the segment provided allocator. Copying is useful to I think ". using the segment provided allocator" can be removed. Seems like a leftover from when we had an overload that took an allocator. Suggestion: * with the same scope as this variable argument list. Copying is useful to ------------- PR: https://git.openjdk.java.net/jdk/pull/5907 From jvernee at openjdk.java.net Tue Nov 2 17:32:25 2021 From: jvernee at openjdk.java.net (Jorn Vernee) Date: Tue, 2 Nov 2021 17:32:25 GMT Subject: RFR: 8275063: Implementation of Foreign Function & Memory API (Second incubator) [v10] In-Reply-To: References: Message-ID: On Mon, 1 Nov 2021 15:38:18 GMT, Jorn Vernee wrote: >> Maurizio Cimadamore has updated the pull request with a new target base due to a merge or a rebase. The pull request now contains 17 commits: >> >> - Add cache for memory address var handles >> - Merge branch 'master' into JEP-419 >> - Fix regression in VaList treatment on AArch64 (contributed by @nick-arm) >> - Merge branch 'master' into JEP-419 >> - Fix copyright header in TestArrayCopy >> - Fix failing microbenchmarks. Contributed by @FrauBoes (thanks!) >> - * use `invokeWithArguments` to simplify new test >> - Add test for liveness check with high-aririty downcalls >> (make sure that if an exception occurs in a downcall because of liveness, >> ref count of other resources are left intact). >> - * Fix javadoc issue in VaList >> * Fix bug in concurrent logic for shared scope acquire >> - Address review comments >> - ... and 7 more: https://git.openjdk.java.net/jdk/compare/5bb1992b...9b519343 > > src/jdk.incubator.foreign/share/classes/jdk/incubator/foreign/MemorySegment.java line 1035: > >> 1033: * >> 1034: * @param layout the layout of the memory region to be read. >> 1035: * @param offset offset in bytes (relative to this segment). For instance, if this segment is a {@link #isNative()} segment, > > Suggestion: > > * @param offset offset in bytes (relative to this segment). For instance, if this segment is a {@link #isNative() native} segment, Same suggestion with all the other getters/setters below (I assume you wanted to add text to the link here?) > src/jdk.incubator.foreign/share/classes/jdk/incubator/foreign/MemorySegment.java line 1549: > >> 1547: * @param index index (relative to this segment). For instance, if this segment is a {@link #isNative()} segment, >> 1548: * the final address of this write operation can be expressed as {@code address().toRowLongValue() + (index * layout.byteSize())}. >> 1549: * @param value the byte value to be written. > > Suggestion: > > * @param value the address value to be written. I think all the setters have this problem. ------------- PR: https://git.openjdk.java.net/jdk/pull/5907 From mcimadamore at openjdk.java.net Tue Nov 2 18:52:21 2021 From: mcimadamore at openjdk.java.net (Maurizio Cimadamore) Date: Tue, 2 Nov 2021 18:52:21 GMT Subject: RFR: 8275063: Implementation of Foreign Function & Memory API (Second incubator) [v12] In-Reply-To: References: Message-ID: On Tue, 2 Nov 2021 16:51:06 GMT, Jorn Vernee wrote: >> Maurizio Cimadamore has updated the pull request incrementally with one additional commit since the last revision: >> >> Tweak javadoc of loaderLookup > > src/jdk.incubator.foreign/share/classes/jdk/internal/foreign/ArenaAllocator.java line 88: > >> 86: if (size > arenaSize) { >> 87: throw new OutOfMemoryError(); >> 88: } > > Isn't this already covered by the `finally` block? Also, this seems to be checking the unaltered `size`, which I think should have been already done at the end of the previous `allocate` call right? I'll have to think some more about this. I don't think this is covered inside the block - that is, the block tries to allocate, and then in the finally we throw if we realized we've allocated too much. ------------- PR: https://git.openjdk.java.net/jdk/pull/5907 From mcimadamore at openjdk.java.net Tue Nov 2 19:49:14 2021 From: mcimadamore at openjdk.java.net (Maurizio Cimadamore) Date: Tue, 2 Nov 2021 19:49:14 GMT Subject: RFR: 8275063: Implementation of Foreign Function & Memory API (Second incubator) [v13] In-Reply-To: References: Message-ID: <1DhHETKpULKzqGU-0EU7qcdSWDngTBO1UMQ39E8qzBw=.ad279b49-57fb-4026-9049-862b4aef2ada@github.com> > This PR contains the API and implementation changes for JEP-419 [1]. A more detailed description of such changes, to avoid repetitions during the review process, is included as a separate comment. > > [1] - https://openjdk.java.net/jeps/419 Maurizio Cimadamore has updated the pull request incrementally with two additional commits since the last revision: - Address impl review comments - Address API review comments ------------- Changes: - all: https://git.openjdk.java.net/jdk/pull/5907/files - new: https://git.openjdk.java.net/jdk/pull/5907/files/7cf4fcd9..1126133a Webrevs: - full: https://webrevs.openjdk.java.net/?repo=jdk&pr=5907&range=12 - incr: https://webrevs.openjdk.java.net/?repo=jdk&pr=5907&range=11-12 Stats: 103 lines in 11 files changed: 8 ins; 23 del; 72 mod Patch: https://git.openjdk.java.net/jdk/pull/5907.diff Fetch: git fetch https://git.openjdk.java.net/jdk pull/5907/head:pull/5907 PR: https://git.openjdk.java.net/jdk/pull/5907 From alanb at openjdk.java.net Tue Nov 2 19:49:15 2021 From: alanb at openjdk.java.net (Alan Bateman) Date: Tue, 2 Nov 2021 19:49:15 GMT Subject: RFR: 8275063: Implementation of Foreign Function & Memory API (Second incubator) [v13] In-Reply-To: <1DhHETKpULKzqGU-0EU7qcdSWDngTBO1UMQ39E8qzBw=.ad279b49-57fb-4026-9049-862b4aef2ada@github.com> References: <1DhHETKpULKzqGU-0EU7qcdSWDngTBO1UMQ39E8qzBw=.ad279b49-57fb-4026-9049-862b4aef2ada@github.com> Message-ID: On Tue, 2 Nov 2021 19:35:29 GMT, Maurizio Cimadamore wrote: >> This PR contains the API and implementation changes for JEP-419 [1]. A more detailed description of such changes, to avoid repetitions during the review process, is included as a separate comment. >> >> [1] - https://openjdk.java.net/jeps/419 > > Maurizio Cimadamore has updated the pull request incrementally with two additional commits since the last revision: > > - Address impl review comments > - Address API review comments src/java.base/share/classes/java/lang/Module.java line 114: > 112: > 113: // true, if this module allows restricted native access; @Stable makes sure that modules that allow native > 114: // access capture this property as a constant. Do you mind fixing this comment to avoid the really long line, it sticks out compare to everything else around it. src/java.base/share/classes/sun/nio/ch/IOUtil.java line 478: > 476: private static final JavaNioAccess NIO_ACCESS = SharedSecrets.getJavaNioAccess(); > 477: > 478: static Runnable acquireScope(ByteBuffer bb, boolean async) { At some point (not this PR) we should move the "async" out of this file, IOUtil was for synchronous I/O. ------------- PR: https://git.openjdk.java.net/jdk/pull/5907 From mcimadamore at openjdk.java.net Tue Nov 2 19:49:16 2021 From: mcimadamore at openjdk.java.net (Maurizio Cimadamore) Date: Tue, 2 Nov 2021 19:49:16 GMT Subject: RFR: 8275063: Implementation of Foreign Function & Memory API (Second incubator) [v12] In-Reply-To: References: Message-ID: On Tue, 2 Nov 2021 18:48:57 GMT, Maurizio Cimadamore wrote: >> src/jdk.incubator.foreign/share/classes/jdk/internal/foreign/ArenaAllocator.java line 88: >> >>> 86: if (size > arenaSize) { >>> 87: throw new OutOfMemoryError(); >>> 88: } >> >> Isn't this already covered by the `finally` block? Also, this seems to be checking the unaltered `size`, which I think should have been already done at the end of the previous `allocate` call right? > > I'll have to think some more about this. I don't think this is covered inside the block - that is, the block tries to allocate, and then in the finally we throw if we realized we've allocated too much. What is missing, I think, is a check (size > arenaSize) at the beginning of the method (we only check this in one of the paths). But we need to check before and after, I think, as it is possible to allocate a segment and then realize that we ended up overflowing the arena size. ------------- PR: https://git.openjdk.java.net/jdk/pull/5907 From mcimadamore at openjdk.java.net Tue Nov 2 19:49:16 2021 From: mcimadamore at openjdk.java.net (Maurizio Cimadamore) Date: Tue, 2 Nov 2021 19:49:16 GMT Subject: RFR: 8275063: Implementation of Foreign Function & Memory API (Second incubator) [v12] In-Reply-To: References: Message-ID: On Tue, 2 Nov 2021 18:55:47 GMT, Maurizio Cimadamore wrote: >> I'll have to think some more about this. I don't think this is covered inside the block - that is, the block tries to allocate, and then in the finally we throw if we realized we've allocated too much. > > What is missing, I think, is a check (size > arenaSize) at the beginning of the method (we only check this in one of the paths). But we need to check before and after, I think, as it is possible to allocate a segment and then realize that we ended up overflowing the arena size. While what I said above correctly reflects what the implementation does, I think a broader issue is that the arena allocator implementation is allocating sometimes more native memory than what its contract specifies. While in some cases we can prevent that, I think in the general case (e.g. where we allocate a new block) we cannot, unless we add extra API guarantees - e.g. that the arena size should be a multiple of the block size (but then we'd have to special case `Long.MAX_VALUE`, or maybe pick a "big enough" power of two instead) ------------- PR: https://git.openjdk.java.net/jdk/pull/5907 From jvernee at openjdk.java.net Tue Nov 2 19:49:16 2021 From: jvernee at openjdk.java.net (Jorn Vernee) Date: Tue, 2 Nov 2021 19:49:16 GMT Subject: RFR: 8275063: Implementation of Foreign Function & Memory API (Second incubator) [v12] In-Reply-To: References: Message-ID: On Tue, 2 Nov 2021 19:02:51 GMT, Maurizio Cimadamore wrote: >> What is missing, I think, is a check (size > arenaSize) at the beginning of the method (we only check this in one of the paths). But we need to check before and after, I think, as it is possible to allocate a segment and then realize that we ended up overflowing the arena size. > > While what I said above correctly reflects what the implementation does, I think a broader issue is that the arena allocator implementation is allocating sometimes more native memory than what its contract specifies. While in some cases we can prevent that, I think in the general case (e.g. where we allocate a new block) we cannot, unless we add extra API guarantees - e.g. that the arena size should be a multiple of the block size (but then we'd have to special case `Long.MAX_VALUE`, or maybe pick a "big enough" power of two instead) Maybe we should not support block size in the case of a bounded arena. i.e. just allocate the whole thing upfront, and have 3 APIs: 1. arena with no bounds and default block size. 2. arena with no bounds and custom block size. 3. arena with bounds, that has no blocks size but allocates the whole thing in one go (could be modeled as block size = arena size). Right now we have 1. and 2., but instead of 3. we have a variant that allows setting both the arena size and block size. If we want to keep what we currently have, I'd suggest changing the arena size to a block count for the variant that takes both the arena size and the block size (I think in that case `Long.MAX_VALUE` should still work?). Any ways, that seems like something that could be addressed in 19 as well. ------------- PR: https://git.openjdk.java.net/jdk/pull/5907 From mcimadamore at openjdk.java.net Tue Nov 2 21:33:46 2021 From: mcimadamore at openjdk.java.net (Maurizio Cimadamore) Date: Tue, 2 Nov 2021 21:33:46 GMT Subject: RFR: 8275063: Implementation of Foreign Function & Memory API (Second incubator) [v14] In-Reply-To: References: Message-ID: > This PR contains the API and implementation changes for JEP-419 [1]. A more detailed description of such changes, to avoid repetitions during the review process, is included as a separate comment. > > [1] - https://openjdk.java.net/jeps/419 Maurizio Cimadamore has updated the pull request incrementally with one additional commit since the last revision: Fix long comment line in Module.java ------------- Changes: - all: https://git.openjdk.java.net/jdk/pull/5907/files - new: https://git.openjdk.java.net/jdk/pull/5907/files/1126133a..c219ae12 Webrevs: - full: https://webrevs.openjdk.java.net/?repo=jdk&pr=5907&range=13 - incr: https://webrevs.openjdk.java.net/?repo=jdk&pr=5907&range=12-13 Stats: 2 lines in 1 file changed: 0 ins; 1 del; 1 mod Patch: https://git.openjdk.java.net/jdk/pull/5907.diff Fetch: git fetch https://git.openjdk.java.net/jdk pull/5907/head:pull/5907 PR: https://git.openjdk.java.net/jdk/pull/5907 From mcimadamore at openjdk.java.net Wed Nov 3 11:32:50 2021 From: mcimadamore at openjdk.java.net (Maurizio Cimadamore) Date: Wed, 3 Nov 2021 11:32:50 GMT Subject: RFR: 8275063: Implementation of Foreign Function & Memory API (Second incubator) [v15] In-Reply-To: References: Message-ID: <3l5SgC7qqzs4wj1leQ3TKp4gqDMXozx6W6bUxO1wlTA=.5db656f9-98d2-474a-918a-f076e63be127@github.com> > This PR contains the API and implementation changes for JEP-419 [1]. A more detailed description of such changes, to avoid repetitions during the review process, is included as a separate comment. > > [1] - https://openjdk.java.net/jeps/419 Maurizio Cimadamore has updated the pull request incrementally with one additional commit since the last revision: Simplify ArenaAllocator impl. The arena should respect its boundaries and never allocate more memory than its size specifies. ------------- Changes: - all: https://git.openjdk.java.net/jdk/pull/5907/files - new: https://git.openjdk.java.net/jdk/pull/5907/files/c219ae12..7f847271 Webrevs: - full: https://webrevs.openjdk.java.net/?repo=jdk&pr=5907&range=14 - incr: https://webrevs.openjdk.java.net/?repo=jdk&pr=5907&range=13-14 Stats: 40 lines in 1 file changed: 8 ins; 15 del; 17 mod Patch: https://git.openjdk.java.net/jdk/pull/5907.diff Fetch: git fetch https://git.openjdk.java.net/jdk pull/5907/head:pull/5907 PR: https://git.openjdk.java.net/jdk/pull/5907 From shade at openjdk.java.net Wed Nov 3 12:02:22 2021 From: shade at openjdk.java.net (Aleksey Shipilev) Date: Wed, 3 Nov 2021 12:02:22 GMT Subject: RFR: 8276550: Use SHA256 hash in build.tools.depend.Depend Message-ID: JDK-8182285 added the incremental build capabilities for modules, by hashing the APIs of each module. The original change uses MD5, which is quite weak, and JDK-8214483 allows `MessageDigest` to have no MD5 implementation. This is the cause of some build failures when using a FIPS-compliant boot JDK that has no MD5 implementation. I suggest we switch to the latest available hash instead. Additional testing: - [x] Linux x86_64 fastdebug build completes - [x] Linux x86_64 fastdebug build times do not regress ------------- Commit messages: - Fix Changes: https://git.openjdk.java.net/jdk/pull/6231/files Webrev: https://webrevs.openjdk.java.net/?repo=jdk&pr=6231&range=00 Issue: https://bugs.openjdk.java.net/browse/JDK-8276550 Stats: 1 line in 1 file changed: 0 ins; 0 del; 1 mod Patch: https://git.openjdk.java.net/jdk/pull/6231.diff Fetch: git fetch https://git.openjdk.java.net/jdk pull/6231/head:pull/6231 PR: https://git.openjdk.java.net/jdk/pull/6231 From mcimadamore at openjdk.java.net Wed Nov 3 13:08:55 2021 From: mcimadamore at openjdk.java.net (Maurizio Cimadamore) Date: Wed, 3 Nov 2021 13:08:55 GMT Subject: RFR: 8275063: Implementation of Foreign Function & Memory API (Second incubator) [v16] In-Reply-To: References: Message-ID: > This PR contains the API and implementation changes for JEP-419 [1]. A more detailed description of such changes, to avoid repetitions during the review process, is included as a separate comment. > > [1] - https://openjdk.java.net/jeps/419 Maurizio Cimadamore has updated the pull request incrementally with one additional commit since the last revision: Make ArenaAllocator impl more flexible in the face of OOME An ArenaAllocator should remain open for business, even if OOME is thrown in case other allocations can fit the arena size. ------------- Changes: - all: https://git.openjdk.java.net/jdk/pull/5907/files - new: https://git.openjdk.java.net/jdk/pull/5907/files/7f847271..9fafb2a6 Webrevs: - full: https://webrevs.openjdk.java.net/?repo=jdk&pr=5907&range=15 - incr: https://webrevs.openjdk.java.net/?repo=jdk&pr=5907&range=14-15 Stats: 13 lines in 2 files changed: 3 ins; 6 del; 4 mod Patch: https://git.openjdk.java.net/jdk/pull/5907.diff Fetch: git fetch https://git.openjdk.java.net/jdk pull/5907/head:pull/5907 PR: https://git.openjdk.java.net/jdk/pull/5907 From jvernee at openjdk.java.net Wed Nov 3 13:40:16 2021 From: jvernee at openjdk.java.net (Jorn Vernee) Date: Wed, 3 Nov 2021 13:40:16 GMT Subject: RFR: 8275063: Implementation of Foreign Function & Memory API (Second incubator) [v16] In-Reply-To: References: Message-ID: On Wed, 3 Nov 2021 13:08:55 GMT, Maurizio Cimadamore wrote: >> This PR contains the API and implementation changes for JEP-419 [1]. A more detailed description of such changes, to avoid repetitions during the review process, is included as a separate comment. >> >> [1] - https://openjdk.java.net/jeps/419 > > Maurizio Cimadamore has updated the pull request incrementally with one additional commit since the last revision: > > Make ArenaAllocator impl more flexible in the face of OOME > An ArenaAllocator should remain open for business, even if OOME is thrown in case other allocations can fit the arena size. Marked as reviewed by jvernee (Reviewer). ------------- PR: https://git.openjdk.java.net/jdk/pull/5907 From adinn at openjdk.java.net Wed Nov 3 14:13:18 2021 From: adinn at openjdk.java.net (Andrew Dinn) Date: Wed, 3 Nov 2021 14:13:18 GMT Subject: RFR: 8276550: Use SHA256 hash in build.tools.depend.Depend In-Reply-To: References: Message-ID: On Wed, 3 Nov 2021 11:54:39 GMT, Aleksey Shipilev wrote: > JDK-8182285 added the incremental build capabilities for modules, by hashing the APIs of each module. > > The original change uses MD5, which is quite weak, and JDK-8214483 allows `MessageDigest` to have no MD5 implementation. This is the cause of some build failures when using a FIPS-compliant boot JDK that has no MD5 implementation. I suggest we switch to the latest available hash instead. > > Additional testing: > - [x] Linux x86_64 fastdebug build completes > - [x] Linux x86_64 fastdebug build times do not regress Looks good. ------------- Marked as reviewed by adinn (Reviewer). PR: https://git.openjdk.java.net/jdk/pull/6231 From aph at openjdk.java.net Wed Nov 3 14:37:13 2021 From: aph at openjdk.java.net (Andrew Haley) Date: Wed, 3 Nov 2021 14:37:13 GMT Subject: RFR: 8276550: Use SHA256 hash in build.tools.depend.Depend In-Reply-To: References: Message-ID: On Wed, 3 Nov 2021 11:54:39 GMT, Aleksey Shipilev wrote: > [JDK-8182285](https://bugs.openjdk.java.net/browse/JDK-8182285) added the incremental build capabilities for modules, by hashing the APIs of each module. > > The original change uses MD5, which is quite weak, and [JDK-8214483](https://bugs.openjdk.java.net/browse/JDK-8214483) allows `MessageDigest` to have no MD5 implementation. This is the cause of some build failures when using a FIPS-compliant boot JDK that has no MD5 implementation. I suggest we switch to the latest available hash instead. > > Additional testing: > - [x] Linux x86_64 fastdebug build completes > - [x] Linux x86_64 fastdebug build times do not regress Marked as reviewed by aph (Reviewer). ------------- PR: https://git.openjdk.java.net/jdk/pull/6231 From s.chaliasos21 at imperial.ac.uk Wed Nov 3 15:12:28 2021 From: s.chaliasos21 at imperial.ac.uk (Chaliasos, Stefanos) Date: Wed, 3 Nov 2021 15:12:28 +0000 Subject: Running jdk's tests to produce coverage report Message-ID: Hello, I'm trying to compute code coverage for langtools in the JDK repo on a Ubuntu 18.04 machine using JDK 18 for the compilation. I have run the following commands: ``` cd /home/user git clone https://github.com/openjdk/jdk.git cd jdk && bash configure && make jdk && cd ../ git clone https://github.com/openjdk/jtreg.git cd jtreg bash make/build.sh --jdk /home/user/jdk/build/linux-x86_64-server-release/jdk/ cd ../jdk bash configure --with-jtreg=/home/user/jtreg/build/images/jtreg --with-jcov=/home/user/jtreg/build/deps/jcov/ make jcov-image ``` In the last command, I get errors with the following message: ``` Exception details: Unsupported class file major version 62 ... ``` and then the message: ``` SEVERE : wrong command for jlink mods: .... ``` I tried to compile JCOV with JDK 18 but without success. Does anyone know if it is possible to compute code coverage on the master branch? Kind regards, Stefanos Chaliasos From tim.bell at oracle.com Wed Nov 3 16:12:30 2021 From: tim.bell at oracle.com (tim.bell at oracle.com) Date: Wed, 3 Nov 2021 09:12:30 -0700 Subject: Running jdk's tests to produce coverage report In-Reply-To: References: Message-ID: Hello Stefanos On 11/3/21 08:12, Chaliasos, Stefanos wrote: > Hello, > > I'm trying to compute code coverage for langtools in the JDK repo on a Ubuntu 18.04 machine using JDK 18 for the compilation. I have run the following commands: > > ``` > cd /home/user > git clonehttps://github.com/openjdk/jdk.git > cd jdk && bash configure && make jdk && cd ../ Does your JDK 18 build work in general? (EG: running 'java -version' or compiling and running a simple HelloWorld.java program)? If not, fix that first. > git clonehttps://github.com/openjdk/jtreg.git > cd jtreg > bash make/build.sh --jdk /home/user/jdk/build/linux-x86_64-server-release/jdk/ > cd ../jdk > bash configure --with-jtreg=/home/user/jtreg/build/images/jtreg --with-jcov=/home/user/jtreg/build/deps/jcov/ > make jcov-image > ``` > > In the last command, I get errors with the following message: > > ``` > Exception details: Unsupported class file major version 62 JDK 18 was advanced to classfile version 62 in this changeset: https://github.com/openjdk/jdk/commit/b018c450e5e4737ccd08ed505fd06cee16c42648 This is a good question for the jcov-dev list at: https://mail.openjdk.java.net/mailman/listinfo/jcov-dev Please subscribe to jcov-dev before sending to the list. See also this jcov bug report: https://bugs.openjdk.java.net/browse/CODETOOLS-7902988 I hope this helps- Tim > ... > ``` > > and then the message: > > ``` > SEVERE : wrong command for jlink mods: > .... > ``` > > I tried to compile JCOV with JDK 18 but without success. Does anyone know if it is possible to compute code coverage on the master branch? > > Kind regards, > > Stefanos Chaliasos From Alan.Bateman at oracle.com Wed Nov 3 16:25:27 2021 From: Alan.Bateman at oracle.com (Alan Bateman) Date: Wed, 3 Nov 2021 16:25:27 +0000 Subject: Running jdk's tests to produce coverage report In-Reply-To: References: Message-ID: <774b4d92-dd82-31be-1a73-2320df613367@oracle.com> On 03/11/2021 15:12, Chaliasos, Stefanos wrote: > Hello, > > I'm trying to compute code coverage for langtools in the JDK repo on a Ubuntu 18.04 machine using JDK 18 for the compilation. I have run the following commands: > > ``` > cd /home/user > git clone https://github.com/openjdk/jdk.git > cd jdk && bash configure && make jdk && cd ../ > git clone https://github.com/openjdk/jtreg.git > cd jtreg > bash make/build.sh --jdk /home/user/jdk/build/linux-x86_64-server-release/jdk/ > cd ../jdk > bash configure --with-jtreg=/home/user/jtreg/build/images/jtreg --with-jcov=/home/user/jtreg/build/deps/jcov/ > make jcov-image > ``` > > In the last command, I get errors with the following message: > > ``` > Exception details: Unsupported class file major version 62 > ... > Which version of jcov is this? I did a jcov test on my local build today using what claims to be "3.0-9-jdk-asm+1.0" and it worked okay. -Alan From s.chaliasos21 at imperial.ac.uk Wed Nov 3 16:34:04 2021 From: s.chaliasos21 at imperial.ac.uk (Chaliasos, Stefanos) Date: Wed, 3 Nov 2021 16:34:04 +0000 Subject: Running jdk's tests to produce coverage report In-Reply-To: <774b4d92-dd82-31be-1a73-2320df613367@oracle.com> References: <774b4d92-dd82-31be-1a73-2320df613367@oracle.com> Message-ID: Thanks Alan, I used the one that jtreg uses. The complete configuration of JCOV for jtreg is: ``` DEFAULT_JCOV_SRC_TAG=jcov3.0-b07 DEFAULT_JCOV_SRC_ARCHIVE_CHECKSUM=c5c26085750628d58de275b3f50a7409300c0497 DEFAULT_ANT_VERSION=1.10.8 DEFAULT_ANT_ARCHIVE_CHECKSUM=dbe187ce2963f9df8a67de8aaff3b0a437d06978 DEFAULT_ASM_VERSION=8.0 DEFAULT_ASM_JAR_CHECKSUM=d1a17d07c60e9e82c8b31b1d8f9ca98726418db4 DEFAULT_ASM_TREE_JAR_CHECKSUM=7b31ca94da9f57334a5aed79b40f2b88c5ee9f4f DEFAULT_ASM_UTIL_JAR_CHECKSUM=b21996293fd49851ed9017cfde3191e49f77fbd0 DEFAULT_JTHARNESS_SRC_TAG=jt6.0-b13 DEFAULT_JTHARNESS_SRC_ARCHIVE_CHECKSUM=43936b2616476fcac8ee4bd0132e73c015119337 ``` Could you please share from where I can download the JCOV version that you are referring to? Cheers, Stefanos ________________________________ From: Alan Bateman Sent: 03 November 2021 16:25 To: Chaliasos, Stefanos ; build-dev at openjdk.java.net Subject: Re: Running jdk's tests to produce coverage report ******************* This email originates from outside Imperial. Do not click on links and attachments unless you recognise the sender. If you trust the sender, add them to your safe senders list https://spam.ic.ac.uk/SpamConsole/Senders.aspx to disable email stamping for this address. ******************* On 03/11/2021 15:12, Chaliasos, Stefanos wrote: > Hello, > > I'm trying to compute code coverage for langtools in the JDK repo on a Ubuntu 18.04 machine using JDK 18 for the compilation. I have run the following commands: > > ``` > cd /home/user > git clone https://github.com/openjdk/jdk.git > cd jdk && bash configure && make jdk && cd ../ > git clone https://github.com/openjdk/jtreg.git > cd jtreg > bash make/build.sh --jdk /home/user/jdk/build/linux-x86_64-server-release/jdk/ > cd ../jdk > bash configure --with-jtreg=/home/user/jtreg/build/images/jtreg --with-jcov=/home/user/jtreg/build/deps/jcov/ > make jcov-image > ``` > > In the last command, I get errors with the following message: > > ``` > Exception details: Unsupported class file major version 62 > ... > Which version of jcov is this? I did a jcov test on my local build today using what claims to be "3.0-9-jdk-asm+1.0" and it worked okay. -Alan From Alan.Bateman at oracle.com Wed Nov 3 16:39:09 2021 From: Alan.Bateman at oracle.com (Alan Bateman) Date: Wed, 3 Nov 2021 16:39:09 +0000 Subject: Running jdk's tests to produce coverage report In-Reply-To: References: <774b4d92-dd82-31be-1a73-2320df613367@oracle.com> Message-ID: <45abf5ba-60ae-bcca-9681-cef94fbfa658@oracle.com> On 03/11/2021 16:34, Chaliasos, Stefanos wrote: > Thanks Alan, > > I used the one that jtreg uses. The complete configuration of JCOV for > jtreg is: > > ``` > DEFAULT_JCOV_SRC_TAG=jcov3.0-b07 > DEFAULT_JCOV_SRC_ARCHIVE_CHECKSUM=c5c26085750628d58de275b3f50a7409300c0497 > > DEFAULT_ANT_VERSION=1.10.8 > DEFAULT_ANT_ARCHIVE_CHECKSUM=dbe187ce2963f9df8a67de8aaff3b0a437d06978 > DEFAULT_ASM_VERSION=8.0 > DEFAULT_ASM_JAR_CHECKSUM=d1a17d07c60e9e82c8b31b1d8f9ca98726418db4 > DEFAULT_ASM_TREE_JAR_CHECKSUM=7b31ca94da9f57334a5aed79b40f2b88c5ee9f4f > DEFAULT_ASM_UTIL_JAR_CHECKSUM=b21996293fd49851ed9017cfde3191e49f77fbd0 > DEFAULT_JTHARNESS_SRC_TAG=jt6.0-b13 > DEFAULT_JTHARNESS_SRC_ARCHIVE_CHECKSUM=43936b2616476fcac8ee4bd0132e73c015119337 > > ``` > > Could you please share from where I can download the JCOV version that > you are referring?to? > Here's the JBS issue that upgraded jcov to support v62 class files. So I think you need jcov 3.0.9 https://bugs.openjdk.java.net/browse/CODETOOLS-7902988 -Alan From s.chaliasos21 at imperial.ac.uk Wed Nov 3 16:55:07 2021 From: s.chaliasos21 at imperial.ac.uk (Chaliasos, Stefanos) Date: Wed, 3 Nov 2021 16:55:07 +0000 Subject: Running jdk's tests to produce coverage report In-Reply-To: <45abf5ba-60ae-bcca-9681-cef94fbfa658@oracle.com> References: <774b4d92-dd82-31be-1a73-2320df613367@oracle.com> <45abf5ba-60ae-bcca-9681-cef94fbfa658@oracle.com> Message-ID: Do you know where I can download this version? In GitHub there are releases until jcov3.0-b07 Stefanos ________________________________ From: Alan Bateman Sent: 03 November 2021 16:39 To: Chaliasos, Stefanos ; build-dev at openjdk.java.net Subject: Re: Running jdk's tests to produce coverage report This email from Alan.Bateman at oracle.com originates from outside Imperial. Do not click on links and attachments unless you recognise the sender. If you trust the sender, add them to your safe senders list to disable email stamping for this address. On 03/11/2021 16:34, Chaliasos, Stefanos wrote: Thanks Alan, I used the one that jtreg uses. The complete configuration of JCOV for jtreg is: ``` DEFAULT_JCOV_SRC_TAG=jcov3.0-b07 DEFAULT_JCOV_SRC_ARCHIVE_CHECKSUM=c5c26085750628d58de275b3f50a7409300c0497 DEFAULT_ANT_VERSION=1.10.8 DEFAULT_ANT_ARCHIVE_CHECKSUM=dbe187ce2963f9df8a67de8aaff3b0a437d06978 DEFAULT_ASM_VERSION=8.0 DEFAULT_ASM_JAR_CHECKSUM=d1a17d07c60e9e82c8b31b1d8f9ca98726418db4 DEFAULT_ASM_TREE_JAR_CHECKSUM=7b31ca94da9f57334a5aed79b40f2b88c5ee9f4f DEFAULT_ASM_UTIL_JAR_CHECKSUM=b21996293fd49851ed9017cfde3191e49f77fbd0 DEFAULT_JTHARNESS_SRC_TAG=jt6.0-b13 DEFAULT_JTHARNESS_SRC_ARCHIVE_CHECKSUM=43936b2616476fcac8ee4bd0132e73c015119337 ``` Could you please share from where I can download the JCOV version that you are referring to? Here's the JBS issue that upgraded jcov to support v62 class files. So I think you need jcov 3.0.9 https://bugs.openjdk.java.net/browse/CODETOOLS-7902988 -Alan From mcimadamore at openjdk.java.net Wed Nov 3 17:40:56 2021 From: mcimadamore at openjdk.java.net (Maurizio Cimadamore) Date: Wed, 3 Nov 2021 17:40:56 GMT Subject: RFR: 8275063: Implementation of Foreign Function & Memory API (Second incubator) [v17] In-Reply-To: References: Message-ID: > This PR contains the API and implementation changes for JEP-419 [1]. A more detailed description of such changes, to avoid repetitions during the review process, is included as a separate comment. > > [1] - https://openjdk.java.net/jeps/419 Maurizio Cimadamore has updated the pull request incrementally with one additional commit since the last revision: Fix TestUpcall * reverse() has a bug, as it doesn't tweak parameter types * reverse() is applied to the wrong MH ------------- Changes: - all: https://git.openjdk.java.net/jdk/pull/5907/files - new: https://git.openjdk.java.net/jdk/pull/5907/files/9fafb2a6..b9432473 Webrevs: - full: https://webrevs.openjdk.java.net/?repo=jdk&pr=5907&range=16 - incr: https://webrevs.openjdk.java.net/?repo=jdk&pr=5907&range=15-16 Stats: 3 lines in 1 file changed: 0 ins; 0 del; 3 mod Patch: https://git.openjdk.java.net/jdk/pull/5907.diff Fetch: git fetch https://git.openjdk.java.net/jdk pull/5907/head:pull/5907 PR: https://git.openjdk.java.net/jdk/pull/5907 From andrew at openjdk.java.net Wed Nov 3 21:03:13 2021 From: andrew at openjdk.java.net (Andrew John Hughes) Date: Wed, 3 Nov 2021 21:03:13 GMT Subject: RFR: 8276550: Use SHA256 hash in build.tools.depend.Depend In-Reply-To: References: Message-ID: On Wed, 3 Nov 2021 11:54:39 GMT, Aleksey Shipilev wrote: > [JDK-8182285](https://bugs.openjdk.java.net/browse/JDK-8182285) added the incremental build capabilities for modules, by hashing the APIs of each module. > > The original change uses MD5, which is quite weak, and [JDK-8214483](https://bugs.openjdk.java.net/browse/JDK-8214483) allows `MessageDigest` to have no MD5 implementation. This is the cause of some build failures when using a FIPS-compliant boot JDK that has no MD5 implementation. I suggest we switch to the latest available hash instead. > > Additional testing: > - [x] Linux x86_64 fastdebug build completes > - [x] Linux x86_64 fastdebug build times do not regress SHA-256 is the right choice here from the small list of required algorithms in the boot JDK. MD5 has already been removed from the list of required algorithms and SHA-1 deprecation for JARs is planned: https://java.com/en/jre-jdk-cryptoroadmap.html ------------- Marked as reviewed by andrew (Reviewer). PR: https://git.openjdk.java.net/jdk/pull/6231 From shade at openjdk.java.net Thu Nov 4 13:16:20 2021 From: shade at openjdk.java.net (Aleksey Shipilev) Date: Thu, 4 Nov 2021 13:16:20 GMT Subject: RFR: 8276550: Use SHA256 hash in build.tools.depend.Depend In-Reply-To: References: Message-ID: On Wed, 3 Nov 2021 11:54:39 GMT, Aleksey Shipilev wrote: > [JDK-8182285](https://bugs.openjdk.java.net/browse/JDK-8182285) added the incremental build capabilities for modules, by hashing the APIs of each module. > > The original change uses MD5, which is quite weak, and [JDK-8214483](https://bugs.openjdk.java.net/browse/JDK-8214483) allows `MessageDigest` to have no MD5 implementation. This is the cause of some build failures when using a FIPS-compliant boot JDK that has no MD5 implementation. I suggest we switch to the latest available hash instead. > > Additional testing: > - [x] Linux x86_64 fastdebug build completes > - [x] Linux x86_64 fastdebug build times do not regress @magicus, are you good with this patch? ------------- PR: https://git.openjdk.java.net/jdk/pull/6231 From ihse at openjdk.java.net Thu Nov 4 13:20:12 2021 From: ihse at openjdk.java.net (Magnus Ihse Bursie) Date: Thu, 4 Nov 2021 13:20:12 GMT Subject: RFR: 8276550: Use SHA256 hash in build.tools.depend.Depend In-Reply-To: References: Message-ID: On Wed, 3 Nov 2021 11:54:39 GMT, Aleksey Shipilev wrote: > [JDK-8182285](https://bugs.openjdk.java.net/browse/JDK-8182285) added the incremental build capabilities for modules, by hashing the APIs of each module. > > The original change uses MD5, which is quite weak, and [JDK-8214483](https://bugs.openjdk.java.net/browse/JDK-8214483) allows `MessageDigest` to have no MD5 implementation. This is the cause of some build failures when using a FIPS-compliant boot JDK that has no MD5 implementation. I suggest we switch to the latest available hash instead. > > Additional testing: > - [x] Linux x86_64 fastdebug build completes > - [x] Linux x86_64 fastdebug build times do not regress Shoot! ------------- PR: https://git.openjdk.java.net/jdk/pull/6231 From shade at openjdk.java.net Thu Nov 4 15:35:11 2021 From: shade at openjdk.java.net (Aleksey Shipilev) Date: Thu, 4 Nov 2021 15:35:11 GMT Subject: RFR: 8276550: Use SHA256 hash in build.tools.depend.Depend In-Reply-To: References: Message-ID: <8dU5ePyO8YqN5kZDJrm6hTHfAagQZHMtrYL5Y8-X8nU=.1c8c93a4-7f18-429c-89ba-b7a6df30a9e1@github.com> On Thu, 4 Nov 2021 13:17:13 GMT, Magnus Ihse Bursie wrote: > Shoot! ...as in "Approved"? :) ------------- PR: https://git.openjdk.java.net/jdk/pull/6231 From ihse at openjdk.java.net Thu Nov 4 15:44:17 2021 From: ihse at openjdk.java.net (Magnus Ihse Bursie) Date: Thu, 4 Nov 2021 15:44:17 GMT Subject: RFR: 8276550: Use SHA256 hash in build.tools.depend.Depend In-Reply-To: References: Message-ID: On Wed, 3 Nov 2021 11:54:39 GMT, Aleksey Shipilev wrote: > [JDK-8182285](https://bugs.openjdk.java.net/browse/JDK-8182285) added the incremental build capabilities for modules, by hashing the APIs of each module. > > The original change uses MD5, which is quite weak, and [JDK-8214483](https://bugs.openjdk.java.net/browse/JDK-8214483) allows `MessageDigest` to have no MD5 implementation. This is the cause of some build failures when using a FIPS-compliant boot JDK that has no MD5 implementation. I suggest we switch to the latest available hash instead. > > Additional testing: > - [x] Linux x86_64 fastdebug build completes > - [x] Linux x86_64 fastdebug build times do not regress Marked as reviewed by ihse (Reviewer). Yes, sorry, I was only on the phone and the github app didn't let me approve the PR, or I was too stupid to figure out how to correctly fight their UI. This is kind of border-line build functionality, so I didn't know my approval was needed. :) But now you have it, anyway. :) ------------- PR: https://git.openjdk.java.net/jdk/pull/6231 From shade at openjdk.java.net Thu Nov 4 15:44:17 2021 From: shade at openjdk.java.net (Aleksey Shipilev) Date: Thu, 4 Nov 2021 15:44:17 GMT Subject: RFR: 8276550: Use SHA256 hash in build.tools.depend.Depend In-Reply-To: References: Message-ID: <-yKDa1gUy36WcvNLlEG4uxDy0aTX-2S_3tYB4tf63Sg=.d9a050f1-46d9-49bd-8a6e-ab29b961ae92@github.com> On Wed, 3 Nov 2021 11:54:39 GMT, Aleksey Shipilev wrote: > [JDK-8182285](https://bugs.openjdk.java.net/browse/JDK-8182285) added the incremental build capabilities for modules, by hashing the APIs of each module. > > The original change uses MD5, which is quite weak, and [JDK-8214483](https://bugs.openjdk.java.net/browse/JDK-8214483) allows `MessageDigest` to have no MD5 implementation. This is the cause of some build failures when using a FIPS-compliant boot JDK that has no MD5 implementation. I suggest we switch to the latest available hash instead. > > Additional testing: > - [x] Linux x86_64 fastdebug build completes > - [x] Linux x86_64 fastdebug build times do not regress Good, thanks! ------------- PR: https://git.openjdk.java.net/jdk/pull/6231 From shade at openjdk.java.net Thu Nov 4 15:44:18 2021 From: shade at openjdk.java.net (Aleksey Shipilev) Date: Thu, 4 Nov 2021 15:44:18 GMT Subject: Integrated: 8276550: Use SHA256 hash in build.tools.depend.Depend In-Reply-To: References: Message-ID: <5ZJwEiPjea4jhp2KOVlEjmvXNxC688zQoljf1XCicck=.9b72de1f-8fff-4dd6-8c29-b8ff284e6f98@github.com> On Wed, 3 Nov 2021 11:54:39 GMT, Aleksey Shipilev wrote: > [JDK-8182285](https://bugs.openjdk.java.net/browse/JDK-8182285) added the incremental build capabilities for modules, by hashing the APIs of each module. > > The original change uses MD5, which is quite weak, and [JDK-8214483](https://bugs.openjdk.java.net/browse/JDK-8214483) allows `MessageDigest` to have no MD5 implementation. This is the cause of some build failures when using a FIPS-compliant boot JDK that has no MD5 implementation. I suggest we switch to the latest available hash instead. > > Additional testing: > - [x] Linux x86_64 fastdebug build completes > - [x] Linux x86_64 fastdebug build times do not regress This pull request has now been integrated. Changeset: afb502e2 Author: Aleksey Shipilev URL: https://git.openjdk.java.net/jdk/commit/afb502e28a81183cef76a7e60fc052e8cad3afe3 Stats: 1 line in 1 file changed: 0 ins; 0 del; 1 mod 8276550: Use SHA256 hash in build.tools.depend.Depend Reviewed-by: adinn, aph, andrew, ihse ------------- PR: https://git.openjdk.java.net/jdk/pull/6231 From sviswanathan at openjdk.java.net Thu Nov 4 18:01:33 2021 From: sviswanathan at openjdk.java.net (Sandhya Viswanathan) Date: Thu, 4 Nov 2021 18:01:33 GMT Subject: RFR: 8276025: Hotspot's libsvml.so may conflict with user dependency Message-ID: This patch removes conflicts with libsvml.so distributed with Intel's MKL library: Renames exported symbols from __svml to __jsvml. Renames library from libsvml.so to libjsvml.so. Updates the stubGenerator_x86_64.cpp accordingly to load libjsvml.so and the renamed symbols. Updates tests to look for the new library. Please review. Best Regards, Sandhya ------------- Commit messages: - load svml test fixes - update tests - 8276025: Hotspot's libsvml.so may conflict with user dependency Changes: https://git.openjdk.java.net/jdk/pull/6265/files Webrev: https://webrevs.openjdk.java.net/?repo=jdk&pr=6265&range=00 Issue: https://bugs.openjdk.java.net/browse/JDK-8276025 Stats: 391989 lines in 123 files changed: 192755 ins; 192755 del; 6479 mod Patch: https://git.openjdk.java.net/jdk/pull/6265.diff Fetch: git fetch https://git.openjdk.java.net/jdk pull/6265/head:pull/6265 PR: https://git.openjdk.java.net/jdk/pull/6265 From kvn at openjdk.java.net Thu Nov 4 18:15:11 2021 From: kvn at openjdk.java.net (Vladimir Kozlov) Date: Thu, 4 Nov 2021 18:15:11 GMT Subject: RFR: 8276025: Hotspot's libsvml.so may conflict with user dependency In-Reply-To: References: Message-ID: On Thu, 4 Nov 2021 17:48:56 GMT, Sandhya Viswanathan wrote: > This patch removes conflicts with libsvml.so distributed with Intel's MKL library: > Renames exported symbols from __svml to __jsvml. > Renames library from libsvml.so to libjsvml.so. > Updates the stubGenerator_x86_64.cpp accordingly to load libjsvml.so and the renamed symbols. > Updates tests to look for the new library. > > Please review. > > Best Regards, > Sandhya Looks good. I will run tests. ------------- Marked as reviewed by kvn (Reviewer). PR: https://git.openjdk.java.net/jdk/pull/6265 From sviswanathan at openjdk.java.net Thu Nov 4 18:18:09 2021 From: sviswanathan at openjdk.java.net (Sandhya Viswanathan) Date: Thu, 4 Nov 2021 18:18:09 GMT Subject: RFR: 8276025: Hotspot's libsvml.so may conflict with user dependency In-Reply-To: References: Message-ID: On Thu, 4 Nov 2021 18:11:41 GMT, Vladimir Kozlov wrote: >> This patch removes conflicts with libsvml.so distributed with Intel's MKL library: >> Renames exported symbols from __svml to __jsvml. >> Renames library from libsvml.so to libjsvml.so. >> Updates the stubGenerator_x86_64.cpp accordingly to load libjsvml.so and the renamed symbols. >> Updates tests to look for the new library. >> >> Please review. >> >> Best Regards, >> Sandhya > > Looks good. I will run tests. Thanks a lot @vnkozlov. ------------- PR: https://git.openjdk.java.net/jdk/pull/6265 From kvn at openjdk.java.net Thu Nov 4 18:24:12 2021 From: kvn at openjdk.java.net (Vladimir Kozlov) Date: Thu, 4 Nov 2021 18:24:12 GMT Subject: RFR: 8276025: Hotspot's libsvml.so may conflict with user dependency In-Reply-To: References: Message-ID: On Thu, 4 Nov 2021 17:48:56 GMT, Sandhya Viswanathan wrote: > This patch removes conflicts with libsvml.so distributed with Intel's MKL library: > Renames exported symbols from __svml to __jsvml. > Renames library from libsvml.so to libjsvml.so. > Updates the stubGenerator_x86_64.cpp accordingly to load libjsvml.so and the renamed symbols. > Updates tests to look for the new library. > > Please review. > > Best Regards, > Sandhya For completeness may be rename files too: jsvml_*.S and jsvml_*.S.inc ------------- PR: https://git.openjdk.java.net/jdk/pull/6265 From sviswanathan at openjdk.java.net Thu Nov 4 19:49:08 2021 From: sviswanathan at openjdk.java.net (Sandhya Viswanathan) Date: Thu, 4 Nov 2021 19:49:08 GMT Subject: RFR: 8276025: Hotspot's libsvml.so may conflict with user dependency [v2] In-Reply-To: References: Message-ID: > This patch removes conflicts with libsvml.so distributed with Intel's MKL library: > Renames exported symbols from __svml to __jsvml. > Renames library from libsvml.so to libjsvml.so. > Updates the stubGenerator_x86_64.cpp accordingly to load libjsvml.so and the renamed symbols. > Updates tests to look for the new library. > > Please review. > > Best Regards, > Sandhya Sandhya Viswanathan has updated the pull request incrementally with one additional commit since the last revision: change filename to jsvml* ------------- Changes: - all: https://git.openjdk.java.net/jdk/pull/6265/files - new: https://git.openjdk.java.net/jdk/pull/6265/files/7c488c10..70d962ae Webrevs: - full: https://webrevs.openjdk.java.net/?repo=jdk&pr=6265&range=01 - incr: https://webrevs.openjdk.java.net/?repo=jdk&pr=6265&range=00-01 Stats: 0 lines in 72 files changed: 0 ins; 0 del; 0 mod Patch: https://git.openjdk.java.net/jdk/pull/6265.diff Fetch: git fetch https://git.openjdk.java.net/jdk pull/6265/head:pull/6265 PR: https://git.openjdk.java.net/jdk/pull/6265 From kvn at openjdk.java.net Thu Nov 4 20:08:12 2021 From: kvn at openjdk.java.net (Vladimir Kozlov) Date: Thu, 4 Nov 2021 20:08:12 GMT Subject: RFR: 8276025: Hotspot's libsvml.so may conflict with user dependency [v2] In-Reply-To: References: Message-ID: On Thu, 4 Nov 2021 19:49:08 GMT, Sandhya Viswanathan wrote: >> This patch removes conflicts with libsvml.so distributed with Intel's MKL library: >> Renames exported symbols from __svml to __jsvml. >> Renames library from libsvml.so to libjsvml.so. >> Updates the stubGenerator_x86_64.cpp accordingly to load libjsvml.so and the renamed symbols. >> Updates tests to look for the new library. >> >> Please review. >> >> Best Regards, >> Sandhya > > Sandhya Viswanathan has updated the pull request incrementally with one additional commit since the last revision: > > change filename to jsvml* Good. ------------- Marked as reviewed by kvn (Reviewer). PR: https://git.openjdk.java.net/jdk/pull/6265 From erikj at openjdk.java.net Thu Nov 4 20:13:10 2021 From: erikj at openjdk.java.net (Erik Joelsson) Date: Thu, 4 Nov 2021 20:13:10 GMT Subject: RFR: 8276025: Hotspot's libsvml.so may conflict with user dependency [v2] In-Reply-To: References: Message-ID: On Thu, 4 Nov 2021 19:49:08 GMT, Sandhya Viswanathan wrote: >> This patch removes conflicts with libsvml.so distributed with Intel's MKL library: >> Renames exported symbols from __svml to __jsvml. >> Renames library from libsvml.so to libjsvml.so. >> Updates the stubGenerator_x86_64.cpp accordingly to load libjsvml.so and the renamed symbols. >> Updates tests to look for the new library. >> >> Please review. >> >> Best Regards, >> Sandhya > > Sandhya Viswanathan has updated the pull request incrementally with one additional commit since the last revision: > > change filename to jsvml* Build changes look good. ------------- Marked as reviewed by erikj (Reviewer). PR: https://git.openjdk.java.net/jdk/pull/6265 From kvn at openjdk.java.net Thu Nov 4 20:48:12 2021 From: kvn at openjdk.java.net (Vladimir Kozlov) Date: Thu, 4 Nov 2021 20:48:12 GMT Subject: RFR: 8276025: Hotspot's libsvml.so may conflict with user dependency [v2] In-Reply-To: References: Message-ID: On Thu, 4 Nov 2021 19:49:08 GMT, Sandhya Viswanathan wrote: >> This patch removes conflicts with libsvml.so distributed with Intel's MKL library: >> Renames exported symbols from __svml to __jsvml. >> Renames library from libsvml.so to libjsvml.so. >> Updates the stubGenerator_x86_64.cpp accordingly to load libjsvml.so and the renamed symbols. >> Updates tests to look for the new library. >> >> Please review. >> >> Best Regards, >> Sandhya > > Sandhya Viswanathan has updated the pull request incrementally with one additional commit since the last revision: > > change filename to jsvml* There is also need to update one closed test we have before integration. ------------- PR: https://git.openjdk.java.net/jdk/pull/6265 From psandoz at openjdk.java.net Thu Nov 4 21:05:10 2021 From: psandoz at openjdk.java.net (Paul Sandoz) Date: Thu, 4 Nov 2021 21:05:10 GMT Subject: RFR: 8276025: Hotspot's libsvml.so may conflict with user dependency [v2] In-Reply-To: References: Message-ID: On Thu, 4 Nov 2021 19:49:08 GMT, Sandhya Viswanathan wrote: >> This patch removes conflicts with libsvml.so distributed with Intel's MKL library: >> Renames exported symbols from __svml to __jsvml. >> Renames library from libsvml.so to libjsvml.so. >> Updates the stubGenerator_x86_64.cpp accordingly to load libjsvml.so and the renamed symbols. >> Updates tests to look for the new library. >> >> Please review. >> >> Best Regards, >> Sandhya > > Sandhya Viswanathan has updated the pull request incrementally with one additional commit since the last revision: > > change filename to jsvml* I did a case insensitive regex search `[^j]svml` over the checked out PR and did not find anything relevant that was missed. ------------- Marked as reviewed by psandoz (Reviewer). PR: https://git.openjdk.java.net/jdk/pull/6265 From aleonard at openjdk.java.net Thu Nov 4 21:16:31 2021 From: aleonard at openjdk.java.net (Andrew Leonard) Date: Thu, 4 Nov 2021 21:16:31 GMT Subject: RFR: 8276400: openjdk image Jars, Zips and Jmods built from the same source are not reproducible Message-ID: <5UDNFhfUqwOsOgcV2wEBWHZKU84d5w6Cc5JZQ-XEITk=.e4431826-178a-4c33-8cf4-b5a6e83016e5@github.com> This PR enables reproducible Jars, Jmods and openjdk image zip files (eg.src.zip). It provides support for SOURCE_DATE_EPOCH for Jar, Jmod and underlying ZipOutputStream's. It fixes the following keys issues relating to reproducibility: - Jar and ZipOutputStream are not SOURCE_DATE_EPOCH aware - Jar and ZipOutputStream now detect SOURCE_DATE_EPOCH environment setting - Jar and Jmod file content ordering was non-determinsitic - Fixes to Jar and Jmod main's to ensure sorted classes content ordering - openjdk image zip file generation used "zip" which is non-determinsitic - New openjdk build tool "GenerateZip" which produces the final determinsitic zip files as part of the build and also detects SOURCE_DATE_EPOCH Signed-off-by: Andrew Leonard ------------- Commit messages: - 8276400: openjdk image Jars, Zips and Jmods built from the same source are not reproducible Changes: https://git.openjdk.java.net/jdk/pull/6268/files Webrev: https://webrevs.openjdk.java.net/?repo=jdk&pr=6268&range=00 Issue: https://bugs.openjdk.java.net/browse/JDK-8276400 Stats: 818 lines in 11 files changed: 797 ins; 3 del; 18 mod Patch: https://git.openjdk.java.net/jdk/pull/6268.diff Fetch: git fetch https://git.openjdk.java.net/jdk pull/6268/head:pull/6268 PR: https://git.openjdk.java.net/jdk/pull/6268 From aleonard at openjdk.java.net Thu Nov 4 21:16:32 2021 From: aleonard at openjdk.java.net (Andrew Leonard) Date: Thu, 4 Nov 2021 21:16:32 GMT Subject: RFR: 8276400: openjdk image Jars, Zips and Jmods built from the same source are not reproducible In-Reply-To: <5UDNFhfUqwOsOgcV2wEBWHZKU84d5w6Cc5JZQ-XEITk=.e4431826-178a-4c33-8cf4-b5a6e83016e5@github.com> References: <5UDNFhfUqwOsOgcV2wEBWHZKU84d5w6Cc5JZQ-XEITk=.e4431826-178a-4c33-8cf4-b5a6e83016e5@github.com> Message-ID: <7f3EDTxxB759aaBolckEm4yIHTGgkqWou-BHq6ghMQg=.6979cc3d-1043-4ae6-9695-c293c3528c7c@github.com> On Thu, 4 Nov 2021 20:56:45 GMT, Andrew Leonard wrote: > This PR enables reproducible Jars, Jmods and openjdk image zip files (eg.src.zip). > It provides support for SOURCE_DATE_EPOCH for Jar, Jmod and underlying ZipOutputStream's. > It fixes the following keys issues relating to reproducibility: > - Jar and ZipOutputStream are not SOURCE_DATE_EPOCH aware > - Jar and ZipOutputStream now detect SOURCE_DATE_EPOCH environment setting > - Jar and Jmod file content ordering was non-determinsitic > - Fixes to Jar and Jmod main's to ensure sorted classes content ordering > - openjdk image zip file generation used "zip" which is non-determinsitic > - New openjdk build tool "GenerateZip" which produces the final determinsitic zip files as part of the build and also detects SOURCE_DATE_EPOCH > > Signed-off-by: Andrew Leonard @magicus fyi, please review if you have a moment, thanks ------------- PR: https://git.openjdk.java.net/jdk/pull/6268 From darcy at openjdk.java.net Thu Nov 4 21:55:12 2021 From: darcy at openjdk.java.net (Joe Darcy) Date: Thu, 4 Nov 2021 21:55:12 GMT Subject: RFR: 8276400: openjdk image Jars, Zips and Jmods built from the same source are not reproducible In-Reply-To: <5UDNFhfUqwOsOgcV2wEBWHZKU84d5w6Cc5JZQ-XEITk=.e4431826-178a-4c33-8cf4-b5a6e83016e5@github.com> References: <5UDNFhfUqwOsOgcV2wEBWHZKU84d5w6Cc5JZQ-XEITk=.e4431826-178a-4c33-8cf4-b5a6e83016e5@github.com> Message-ID: On Thu, 4 Nov 2021 20:56:45 GMT, Andrew Leonard wrote: > This PR enables reproducible Jars, Jmods and openjdk image zip files (eg.src.zip). > It provides support for SOURCE_DATE_EPOCH for Jar, Jmod and underlying ZipOutputStream's. > It fixes the following keys issues relating to reproducibility: > - Jar and ZipOutputStream are not SOURCE_DATE_EPOCH aware > - Jar and ZipOutputStream now detect SOURCE_DATE_EPOCH environment setting > - Jar and Jmod file content ordering was non-determinsitic > - Fixes to Jar and Jmod main's to ensure sorted classes content ordering > - openjdk image zip file generation used "zip" which is non-determinsitic > - New openjdk build tool "GenerateZip" which produces the final determinsitic zip files as part of the build and also detects SOURCE_DATE_EPOCH > > Signed-off-by: Andrew Leonard Please file a CSR for the behavioral change of recognizing SOURCE_DATE_EPOCH, etc. Thanks. ------------- PR: https://git.openjdk.java.net/jdk/pull/6268 From ihse at openjdk.java.net Thu Nov 4 22:07:11 2021 From: ihse at openjdk.java.net (Magnus Ihse Bursie) Date: Thu, 4 Nov 2021 22:07:11 GMT Subject: RFR: 8276025: Hotspot's libsvml.so may conflict with user dependency [v2] In-Reply-To: References: Message-ID: On Thu, 4 Nov 2021 19:49:08 GMT, Sandhya Viswanathan wrote: >> This patch removes conflicts with libsvml.so distributed with Intel's MKL library: >> Renames exported symbols from __svml to __jsvml. >> Renames library from libsvml.so to libjsvml.so. >> Updates the stubGenerator_x86_64.cpp accordingly to load libjsvml.so and the renamed symbols. >> Updates tests to look for the new library. >> >> Please review. >> >> Best Regards, >> Sandhya > > Sandhya Viswanathan has updated the pull request incrementally with one additional commit since the last revision: > > change filename to jsvml* Marked as reviewed by ihse (Reviewer). ------------- PR: https://git.openjdk.java.net/jdk/pull/6265 From aleonard at openjdk.java.net Thu Nov 4 22:11:10 2021 From: aleonard at openjdk.java.net (Andrew Leonard) Date: Thu, 4 Nov 2021 22:11:10 GMT Subject: RFR: 8276400: openjdk image Jars, Zips and Jmods built from the same source are not reproducible In-Reply-To: References: <5UDNFhfUqwOsOgcV2wEBWHZKU84d5w6Cc5JZQ-XEITk=.e4431826-178a-4c33-8cf4-b5a6e83016e5@github.com> Message-ID: On Thu, 4 Nov 2021 21:51:46 GMT, Joe Darcy wrote: >> This PR enables reproducible Jars, Jmods and openjdk image zip files (eg.src.zip). >> It provides support for SOURCE_DATE_EPOCH for Jar, Jmod and underlying ZipOutputStream's. >> It fixes the following keys issues relating to reproducibility: >> - Jar and ZipOutputStream are not SOURCE_DATE_EPOCH aware >> - Jar and ZipOutputStream now detect SOURCE_DATE_EPOCH environment setting >> - Jar and Jmod file content ordering was non-determinsitic >> - Fixes to Jar and Jmod main's to ensure sorted classes content ordering >> - openjdk image zip file generation used "zip" which is non-determinsitic >> - New openjdk build tool "GenerateZip" which produces the final determinsitic zip files as part of the build and also detects SOURCE_DATE_EPOCH >> >> Signed-off-by: Andrew Leonard > > Please file a CSR for the behavioral change of recognizing SOURCE_DATE_EPOCH, etc. Thanks. @jddarcy thanks, I did wonder that, i'll start a CSR, thanks ------------- PR: https://git.openjdk.java.net/jdk/pull/6268 From aleonard at openjdk.java.net Thu Nov 4 22:43:07 2021 From: aleonard at openjdk.java.net (Andrew Leonard) Date: Thu, 4 Nov 2021 22:43:07 GMT Subject: RFR: 8276400: openjdk image Jars, Zips and Jmods built from the same source are not reproducible In-Reply-To: References: <5UDNFhfUqwOsOgcV2wEBWHZKU84d5w6Cc5JZQ-XEITk=.e4431826-178a-4c33-8cf4-b5a6e83016e5@github.com> Message-ID: <_3m1Ajg1RnTqOz28MYiVIPW2CR6NqJWk8AWHKi-o22A=.b25869fd-f9ca-4a33-a510-509d88c6da10@github.com> On Thu, 4 Nov 2021 21:51:46 GMT, Joe Darcy wrote: >> This PR enables reproducible Jars, Jmods and openjdk image zip files (eg.src.zip). >> It provides support for SOURCE_DATE_EPOCH for Jar, Jmod and underlying ZipOutputStream's. >> It fixes the following keys issues relating to reproducibility: >> - Jar and ZipOutputStream are not SOURCE_DATE_EPOCH aware >> - Jar and ZipOutputStream now detect SOURCE_DATE_EPOCH environment setting >> - Jar and Jmod file content ordering was non-determinsitic >> - Fixes to Jar and Jmod main's to ensure sorted classes content ordering >> - openjdk image zip file generation used "zip" which is non-determinsitic >> - New openjdk build tool "GenerateZip" which produces the final determinsitic zip files as part of the build and also detects SOURCE_DATE_EPOCH >> >> Signed-off-by: Andrew Leonard > > Please file a CSR for the behavioral change of recognizing SOURCE_DATE_EPOCH, etc. Thanks. @jddarcy CSR created: https://bugs.openjdk.java.net/browse/JDK-8276667 ------------- PR: https://git.openjdk.java.net/jdk/pull/6268 From jonathan.gibbons at oracle.com Thu Nov 4 23:50:42 2021 From: jonathan.gibbons at oracle.com (Jonathan Gibbons) Date: Thu, 4 Nov 2021 16:50:42 -0700 Subject: Running jdk's tests to produce coverage report In-Reply-To: References: <774b4d92-dd82-31be-1a73-2320df613367@oracle.com> <45abf5ba-60ae-bcca-9681-cef94fbfa658@oracle.com> Message-ID: <0eb4f99c-ae1c-ba42-9233-133fb1937049@oracle.com> I'll update the jtreg build script, and then you'll be able to build jtreg yourself, and get the jcov from there. -- Jon On 11/3/21 9:55 AM, Chaliasos, Stefanos wrote: > Do you know where I can download this version? > > In GitHub there are releases until jcov3.0-b07 > > Stefanos > ________________________________ > From: Alan Bateman > Sent: 03 November 2021 16:39 > To: Chaliasos, Stefanos ; build-dev at openjdk.java.net > Subject: Re: Running jdk's tests to produce coverage report > > > This email from Alan.Bateman at oracle.com originates from outside Imperial. Do not click on links and attachments unless you recognise the sender. If you trust the sender, add them to your safe senders list to disable email stamping for this address. > > > > > On 03/11/2021 16:34, Chaliasos, Stefanos wrote: > Thanks Alan, > > I used the one that jtreg uses. The complete configuration of JCOV for jtreg is: > > ``` > DEFAULT_JCOV_SRC_TAG=jcov3.0-b07 > DEFAULT_JCOV_SRC_ARCHIVE_CHECKSUM=c5c26085750628d58de275b3f50a7409300c0497 > > DEFAULT_ANT_VERSION=1.10.8 > DEFAULT_ANT_ARCHIVE_CHECKSUM=dbe187ce2963f9df8a67de8aaff3b0a437d06978 > > DEFAULT_ASM_VERSION=8.0 > DEFAULT_ASM_JAR_CHECKSUM=d1a17d07c60e9e82c8b31b1d8f9ca98726418db4 > DEFAULT_ASM_TREE_JAR_CHECKSUM=7b31ca94da9f57334a5aed79b40f2b88c5ee9f4f > DEFAULT_ASM_UTIL_JAR_CHECKSUM=b21996293fd49851ed9017cfde3191e49f77fbd0 > > DEFAULT_JTHARNESS_SRC_TAG=jt6.0-b13 > DEFAULT_JTHARNESS_SRC_ARCHIVE_CHECKSUM=43936b2616476fcac8ee4bd0132e73c015119337 > ``` > > Could you please share from where I can download the JCOV version that you are referring to? > > Here's the JBS issue that upgraded jcov to support v62 class files. So I think you need jcov 3.0.9 > > https://bugs.openjdk.java.net/browse/CODETOOLS-7902988 > > -Alan From kvn at openjdk.java.net Fri Nov 5 00:59:09 2021 From: kvn at openjdk.java.net (Vladimir Kozlov) Date: Fri, 5 Nov 2021 00:59:09 GMT Subject: RFR: 8276025: Hotspot's libsvml.so may conflict with user dependency [v2] In-Reply-To: References: Message-ID: On Thu, 4 Nov 2021 18:15:08 GMT, Sandhya Viswanathan wrote: >> Looks good. I will run tests. > > Thanks a lot @vnkozlov. @sviswa7 testing passed you can integrate. ------------- PR: https://git.openjdk.java.net/jdk/pull/6265 From duke at openjdk.java.net Fri Nov 5 01:46:10 2021 From: duke at openjdk.java.net (Michael Bien) Date: Fri, 5 Nov 2021 01:46:10 GMT Subject: RFR: 8276400: openjdk image Jars, Zips and Jmods built from the same source are not reproducible In-Reply-To: <5UDNFhfUqwOsOgcV2wEBWHZKU84d5w6Cc5JZQ-XEITk=.e4431826-178a-4c33-8cf4-b5a6e83016e5@github.com> References: <5UDNFhfUqwOsOgcV2wEBWHZKU84d5w6Cc5JZQ-XEITk=.e4431826-178a-4c33-8cf4-b5a6e83016e5@github.com> Message-ID: On Thu, 4 Nov 2021 20:56:45 GMT, Andrew Leonard wrote: > This PR enables reproducible Jars, Jmods and openjdk image zip files (eg.src.zip). > It provides support for SOURCE_DATE_EPOCH for Jar, Jmod and underlying ZipOutputStream's. > It fixes the following keys issues relating to reproducibility: > - Jar and ZipOutputStream are not SOURCE_DATE_EPOCH aware > - Jar and ZipOutputStream now detect SOURCE_DATE_EPOCH environment setting > - Jar and Jmod file content ordering was non-determinsitic > - Fixes to Jar and Jmod main's to ensure sorted classes content ordering > - openjdk image zip file generation used "zip" which is non-determinsitic > - New openjdk build tool "GenerateZip" which produces the final determinsitic zip files as part of the build and also detects SOURCE_DATE_EPOCH > > Signed-off-by: Andrew Leonard a few minor comments make/jdk/src/classes/build/tools/generatezip/GenerateZip.java line 96: > 94: ok = createOk; > 95: } > 96: out.close(); could be a try-with-resource block make/jdk/src/classes/build/tools/generatezip/GenerateZip.java line 191: > 189: Iterator> itr = filesToProcess.entrySet().iterator(); > 190: while(itr.hasNext()) { > 191: Map.Entry entry = itr.next(); could be `for (Map.Entry entry : filesToProcess.entrySet())` make/jdk/src/classes/build/tools/generatezip/GenerateZip.java line 262: > 260: zos.write(buf, 0, len); > 261: } > 262: is.close(); try-with-resource candidate + in.transferTo(zos) make/jdk/src/classes/build/tools/generatezip/GenerateZip.java line 291: > 289: newArgs.add(arg); > 290: } > 291: return newArgs.toArray(new String[newArgs.size()]); something might be missing here. It just adds args to a list and makes it an array again + If the language level allows it toArray(String[]::new) could be used. src/java.base/share/classes/java/util/zip/ZipOutputStream.java line 106: > 104: value = -1; > 105: } > 106: return new Long(value); value can be returned here directly which will use auto-boxing. (new Long(long) is deprecated). src/jdk.jlink/share/classes/jdk/tools/jmod/JmodTask.java line 799: > 797: Iterator> itr = filesToProcess.entrySet().iterator(); > 798: while(itr.hasNext()) { > 799: Map.Entry entry = itr.next(); another for-each candidate test/jdk/java/util/zip/ZipSourceDateEpoch.java line 71: > 69: zos.close(); > 70: os.close(); > 71: } try-with-resource test/jdk/java/util/zip/ZipSourceDateEpoch.java line 97: > 95: zis.close(); > 96: fis.close(); > 97: } try-with-resource ------------- PR: https://git.openjdk.java.net/jdk/pull/6268 From ysuenaga at openjdk.java.net Fri Nov 5 03:10:22 2021 From: ysuenaga at openjdk.java.net (Yasumasa Suenaga) Date: Fri, 5 Nov 2021 03:10:22 GMT Subject: RFR: 8276672: Cannot build hsdis on WSL Message-ID: <4lLdxiPsFsB1GfI7iWSgLOXQ-Q_WU4ZZdOXHkRuO0kc=.45eea799-3e3d-4b1c-90dc-a6fa5e3d5b74@github.com> JDK-8275128 introduced new Makefile for hsdis to integrate it to normal build system, however it does not work on WSL 1 Ubuntu 20.04 and MinGW from Ubuntu. Hsdis.gmk has two problems: 1. MinGW version is fixed to "9.2.0" 2. Assumes sysroot for MinGW is located at /usr/$MINGW_BASE/sys-root I tested this change on WSL only, so it is appreciate to review on Cygwin. ------------- Commit messages: - 8276672: Cannot build hsdis on WSL Changes: https://git.openjdk.java.net/jdk/pull/6269/files Webrev: https://webrevs.openjdk.java.net/?repo=jdk&pr=6269&range=00 Issue: https://bugs.openjdk.java.net/browse/JDK-8276672 Stats: 13 lines in 1 file changed: 8 ins; 0 del; 5 mod Patch: https://git.openjdk.java.net/jdk/pull/6269.diff Fetch: git fetch https://git.openjdk.java.net/jdk pull/6269/head:pull/6269 PR: https://git.openjdk.java.net/jdk/pull/6269 From sviswanathan at openjdk.java.net Fri Nov 5 03:34:18 2021 From: sviswanathan at openjdk.java.net (Sandhya Viswanathan) Date: Fri, 5 Nov 2021 03:34:18 GMT Subject: RFR: 8276025: Hotspot's libsvml.so may conflict with user dependency [v2] In-Reply-To: References: Message-ID: <-gwQAH9K9_dyJZ6cjH_wbyvEP_VEyUVnXcq4cBjsqQQ=.00c317de-b82b-4db7-a44f-da0b71e4aae3@github.com> On Fri, 5 Nov 2021 00:56:05 GMT, Vladimir Kozlov wrote: >> Thanks a lot @vnkozlov. > > @sviswa7 testing passed you can integrate. Thanks a lot @vnkozlov for testing and review. Thanks @erikj79 @PaulSandoz @magicus for the review. ------------- PR: https://git.openjdk.java.net/jdk/pull/6265 From sviswanathan at openjdk.java.net Fri Nov 5 03:34:20 2021 From: sviswanathan at openjdk.java.net (Sandhya Viswanathan) Date: Fri, 5 Nov 2021 03:34:20 GMT Subject: Integrated: 8276025: Hotspot's libsvml.so may conflict with user dependency In-Reply-To: References: Message-ID: On Thu, 4 Nov 2021 17:48:56 GMT, Sandhya Viswanathan wrote: > This patch removes conflicts with libsvml.so distributed with Intel's MKL library: > Renames exported symbols from __svml to __jsvml. > Renames library from libsvml.so to libjsvml.so. > Updates the stubGenerator_x86_64.cpp accordingly to load libjsvml.so and the renamed symbols. > Updates tests to look for the new library. > > Please review. > > Best Regards, > Sandhya This pull request has now been integrated. Changeset: 9ad4d3d0 Author: Sandhya Viswanathan URL: https://git.openjdk.java.net/jdk/commit/9ad4d3d06bb356436d69af07726ef6727c500f59 Stats: 391989 lines in 123 files changed: 192755 ins; 192755 del; 6479 mod 8276025: Hotspot's libsvml.so may conflict with user dependency Reviewed-by: kvn, erikj, psandoz, ihse ------------- PR: https://git.openjdk.java.net/jdk/pull/6265 From jpai at openjdk.java.net Fri Nov 5 06:40:12 2021 From: jpai at openjdk.java.net (Jaikiran Pai) Date: Fri, 5 Nov 2021 06:40:12 GMT Subject: RFR: 8276400: openjdk image Jars, Zips and Jmods built from the same source are not reproducible In-Reply-To: <5UDNFhfUqwOsOgcV2wEBWHZKU84d5w6Cc5JZQ-XEITk=.e4431826-178a-4c33-8cf4-b5a6e83016e5@github.com> References: <5UDNFhfUqwOsOgcV2wEBWHZKU84d5w6Cc5JZQ-XEITk=.e4431826-178a-4c33-8cf4-b5a6e83016e5@github.com> Message-ID: <3j98nPAewSZPIOKWUZoA8s0WcSmn2Jo7bnvjSFp7hcQ=.9e5f2017-f750-4fa7-94a4-e6f100a3c673@github.com> On Thu, 4 Nov 2021 20:56:45 GMT, Andrew Leonard wrote: > This PR enables reproducible Jars, Jmods and openjdk image zip files (eg.src.zip). > It provides support for SOURCE_DATE_EPOCH for Jar, Jmod and underlying ZipOutputStream's. > It fixes the following keys issues relating to reproducibility: > - Jar and ZipOutputStream are not SOURCE_DATE_EPOCH aware > - Jar and ZipOutputStream now detect SOURCE_DATE_EPOCH environment setting > - Jar and Jmod file content ordering was non-determinsitic > - Fixes to Jar and Jmod main's to ensure sorted classes content ordering > - openjdk image zip file generation used "zip" which is non-determinsitic > - New openjdk build tool "GenerateZip" which produces the final determinsitic zip files as part of the build and also detects SOURCE_DATE_EPOCH > > Signed-off-by: Andrew Leonard make/jdk/src/classes/build/tools/generatezip/GenerateZip.java line 2: > 1: /* > 2: * Copyright (c) 2021, 2021, Oracle and/or its affiliates. All rights reserved. Hello @andrew-m-leonard, for a new file, the year should just be stated once. So just `2021,` ------------- PR: https://git.openjdk.java.net/jdk/pull/6268 From Alan.Bateman at oracle.com Fri Nov 5 07:23:40 2021 From: Alan.Bateman at oracle.com (Alan Bateman) Date: Fri, 5 Nov 2021 07:23:40 +0000 Subject: RFR: 8276400: openjdk image Jars, Zips and Jmods built from the same source are not reproducible In-Reply-To: <5UDNFhfUqwOsOgcV2wEBWHZKU84d5w6Cc5JZQ-XEITk=.e4431826-178a-4c33-8cf4-b5a6e83016e5@github.com> References: <5UDNFhfUqwOsOgcV2wEBWHZKU84d5w6Cc5JZQ-XEITk=.e4431826-178a-4c33-8cf4-b5a6e83016e5@github.com> Message-ID: On 04/11/2021 21:16, Andrew Leonard wrote: > This PR enables reproducible Jars, Jmods and openjdk image zip files (eg.src.zip). > It provides support for SOURCE_DATE_EPOCH for Jar, Jmod and underlying ZipOutputStream's. > It fixes the following keys issues relating to reproducibility: > - Jar and ZipOutputStream are not SOURCE_DATE_EPOCH aware > - Jar and ZipOutputStream now detect SOURCE_DATE_EPOCH environment setting > - Jar and Jmod file content ordering was non-determinsitic > - Fixes to Jar and Jmod main's to ensure sorted classes content ordering > - openjdk image zip file generation used "zip" which is non-determinsitic > - New openjdk build tool "GenerateZip" which produces the final determinsitic zip files as part of the build and also detects SOURCE_DATE_EPOCH > I think this is going to require discussion, a PR may be too premature. Is your goal to get the JDK build and run-time images creates with jlink to use the SOURCE_DATE_EPOCH? Are you looking for project builds that use the jar/jmod/etc. tools to use it? Or are you looking to have every library and application that uses the java.util.zip APIs to read it? If it includes the latter, and the patch in the PR suggests you are, then it will require analysis to work through the API spec changes. One suggestion is to look at JDK-8231640 and PR 5372 [1]. The original proposal was to use SOURCE_DATE_EPOCH. After a lengthy discussion we converged on the system property java.properties.date rather than the env variable. I suspect that much of that discussion applies here. -Alan [1] https://github.com/openjdk/jdk/pull/5372 From mcimadamore at openjdk.java.net Fri Nov 5 11:06:53 2021 From: mcimadamore at openjdk.java.net (Maurizio Cimadamore) Date: Fri, 5 Nov 2021 11:06:53 GMT Subject: RFR: 8275063: Implementation of Foreign Function & Memory API (Second incubator) [v18] In-Reply-To: References: Message-ID: <4iHYQMJoHZwfRJCHV9tYB_5t92pjEsgISw_d9_Nt6H8=.1fb75c40-a6ee-498a-9d4e-cf1b6d11b583@github.com> > This PR contains the API and implementation changes for JEP-419 [1]. A more detailed description of such changes, to avoid repetitions during the review process, is included as a separate comment. > > [1] - https://openjdk.java.net/jeps/419 Maurizio Cimadamore has updated the pull request incrementally with one additional commit since the last revision: * Add two new CLinker static methods to compute upcall/downcall method types * Clarify section on CLinker downcall type * Add section on CLinker safety guarantees ------------- Changes: - all: https://git.openjdk.java.net/jdk/pull/5907/files - new: https://git.openjdk.java.net/jdk/pull/5907/files/b9432473..ce561e1f Webrevs: - full: https://webrevs.openjdk.java.net/?repo=jdk&pr=5907&range=17 - incr: https://webrevs.openjdk.java.net/?repo=jdk&pr=5907&range=16-17 Stats: 79 lines in 3 files changed: 47 ins; 17 del; 15 mod Patch: https://git.openjdk.java.net/jdk/pull/5907.diff Fetch: git fetch https://git.openjdk.java.net/jdk pull/5907/head:pull/5907 PR: https://git.openjdk.java.net/jdk/pull/5907 From lancea at openjdk.java.net Fri Nov 5 11:30:19 2021 From: lancea at openjdk.java.net (Lance Andersen) Date: Fri, 5 Nov 2021 11:30:19 GMT Subject: RFR: 8276400: openjdk image Jars, Zips and Jmods built from the same source are not reproducible In-Reply-To: <5UDNFhfUqwOsOgcV2wEBWHZKU84d5w6Cc5JZQ-XEITk=.e4431826-178a-4c33-8cf4-b5a6e83016e5@github.com> References: <5UDNFhfUqwOsOgcV2wEBWHZKU84d5w6Cc5JZQ-XEITk=.e4431826-178a-4c33-8cf4-b5a6e83016e5@github.com> Message-ID: On Thu, 4 Nov 2021 20:56:45 GMT, Andrew Leonard wrote: > This PR enables reproducible Jars, Jmods and openjdk image zip files (eg.src.zip). > It provides support for SOURCE_DATE_EPOCH for Jar, Jmod and underlying ZipOutputStream's. > It fixes the following keys issues relating to reproducibility: > - Jar and ZipOutputStream are not SOURCE_DATE_EPOCH aware > - Jar and ZipOutputStream now detect SOURCE_DATE_EPOCH environment setting > - Jar and Jmod file content ordering was non-determinsitic > - Fixes to Jar and Jmod main's to ensure sorted classes content ordering > - openjdk image zip file generation used "zip" which is non-determinsitic > - New openjdk build tool "GenerateZip" which produces the final determinsitic zip files as part of the build and also detects SOURCE_DATE_EPOCH > > Signed-off-by: Andrew Leonard Thank you for your contribution Andrew. Thank you for your efforts here Andrew. A few comments in addition to what have already been mentioned by others on a very quick scan. For new tests, please consider using TestNG. src/jdk.jartool/share/classes/sun/tools/jar/Main.java line 795: > 793: // Ensure files list is sorted for reproducible jar content > 794: Arrays.sort(files); > 795: Have you had a chance to measure the performance with a large number of Zip entries with this change? test/jdk/java/util/zip/TestZipSourceDateEpoch.sh line 1: > 1: #!/bin/sh Unless there is a specific reason to use a shell script, it would be better to convert this to java. We have been trying to reduce the usage of shell scripts ------------- PR: https://git.openjdk.java.net/jdk/pull/6268 From mcimadamore at openjdk.java.net Fri Nov 5 11:30:59 2021 From: mcimadamore at openjdk.java.net (Maurizio Cimadamore) Date: Fri, 5 Nov 2021 11:30:59 GMT Subject: RFR: 8275063: Implementation of Foreign Function & Memory API (Second incubator) [v19] In-Reply-To: References: Message-ID: > This PR contains the API and implementation changes for JEP-419 [1]. A more detailed description of such changes, to avoid repetitions during the review process, is included as a separate comment. > > [1] - https://openjdk.java.net/jeps/419 Maurizio Cimadamore has updated the pull request incrementally with one additional commit since the last revision: Streamline javadoc for package-info ------------- Changes: - all: https://git.openjdk.java.net/jdk/pull/5907/files - new: https://git.openjdk.java.net/jdk/pull/5907/files/ce561e1f..350f1f07 Webrevs: - full: https://webrevs.openjdk.java.net/?repo=jdk&pr=5907&range=18 - incr: https://webrevs.openjdk.java.net/?repo=jdk&pr=5907&range=17-18 Stats: 37 lines in 1 file changed: 9 ins; 3 del; 25 mod Patch: https://git.openjdk.java.net/jdk/pull/5907.diff Fetch: git fetch https://git.openjdk.java.net/jdk pull/5907/head:pull/5907 PR: https://git.openjdk.java.net/jdk/pull/5907 From aleonard at openjdk.java.net Fri Nov 5 11:34:13 2021 From: aleonard at openjdk.java.net (Andrew Leonard) Date: Fri, 5 Nov 2021 11:34:13 GMT Subject: RFR: 8276400: openjdk image Jars, Zips and Jmods built from the same source are not reproducible In-Reply-To: <5UDNFhfUqwOsOgcV2wEBWHZKU84d5w6Cc5JZQ-XEITk=.e4431826-178a-4c33-8cf4-b5a6e83016e5@github.com> References: <5UDNFhfUqwOsOgcV2wEBWHZKU84d5w6Cc5JZQ-XEITk=.e4431826-178a-4c33-8cf4-b5a6e83016e5@github.com> Message-ID: On Thu, 4 Nov 2021 20:56:45 GMT, Andrew Leonard wrote: > This PR enables reproducible Jars, Jmods and openjdk image zip files (eg.src.zip). > It provides support for SOURCE_DATE_EPOCH for Jar, Jmod and underlying ZipOutputStream's. > It fixes the following keys issues relating to reproducibility: > - Jar and ZipOutputStream are not SOURCE_DATE_EPOCH aware > - Jar and ZipOutputStream now detect SOURCE_DATE_EPOCH environment setting > - Jar and Jmod file content ordering was non-determinsitic > - Fixes to Jar and Jmod main's to ensure sorted classes content ordering > - openjdk image zip file generation used "zip" which is non-determinsitic > - New openjdk build tool "GenerateZip" which produces the final determinsitic zip files as part of the build and also detects SOURCE_DATE_EPOCH > > Signed-off-by: Andrew Leonard > I think this is going to require discussion, a PR may be too premature. Is your goal to get the JDK build and run-time images creates with jlink to use the SOURCE_DATE_EPOCH? Are you looking for project builds that use the jar/jmod/etc. tools to use it? Or are you looking to have every library and application that uses the java.util.zip APIs to read it? If it includes the latter, and the patch in the PR suggests you are, then it will require analysis to work through the API spec changes. > > One suggestion is to look at JDK-8231640 and PR 5372 [1]. The original proposal was to use SOURCE_DATE_EPOCH. After a lengthy discussion we converged on the system property java.properties.date rather than the env variable. I suspect that much of that discussion applies here. > > -Alan > > [1] https://github.com/[/pull/5372](https://github.com/openjdk/jdk/pull/5372) @AlanBateman Hi Alan, thank you for the comments. So yes, totally agree this has a way to go, I thought creating a PR would kick things off...! So just to clarify the initial objectives which you raise above. My current aim is actually purely an OpenJDK JDK image perspective, building on the work @magicus has been doing with SOURCE_DATE_EPOCH in the build, hence my natural approach to this PR. So from that perspective, it's purely a "build" change. However, the openjdk build obviously uses openjdk Jar, Jmod tooling to build itself, hence the direction I took in this PR. Magnus's PR #5372 is useful thank you, I had not discovered that change, as you say the direction there is similar to mine, and I understand the potential pitfalls of adding doPrivileged's etc. So that resolved to a new system property java.properties.date, which makes sense. So with that in mind, and given the objective of JDK image reproducibility, this would imply presumably an approach of a new cmd line option to Jar and Jmod tools (eg.--source-date ), since these tools are cmds? One of the key changes in this PR is to ZipOutputStream where it sets the ZipEntry.xdostime if not set, to the current time. Now I am thinking given the "objective", if we fix all upstream tooling to ensure they set ZipEntry times, using "--source-date" (or whatever mechanism), then we can then avoid changing ZipOutputStream I believe. JDK tools like jar, jmod generate/update extra files like Manifests, and in fact I think the current jmod implementation writes all its content using the current time. So something along these lines: jar -cf myjar.jar --source-date "" * jmod create --source-date "" ... my.jmod The jar, jmod,.. tooling then ensure all ZipEntry times are set to "source-date". I chose "source-date" name based on being synonymous with SOURCE_DATE_EPOCH, implying a similar "use case". So taking this approach for JDK tooling should achieve the reproducible JDK image objective, without a java API behavior change I am hoping. Thoughts? Regards Andrew ------------- PR: https://git.openjdk.java.net/jdk/pull/6268 From mcimadamore at openjdk.java.net Fri Nov 5 11:37:57 2021 From: mcimadamore at openjdk.java.net (Maurizio Cimadamore) Date: Fri, 5 Nov 2021 11:37:57 GMT Subject: RFR: 8275063: Implementation of Foreign Function & Memory API (Second incubator) [v20] In-Reply-To: References: Message-ID: <1EDavlhSqnzIbpu1uQArxPknmjIMaeQEoPV8W1T3UjE=.9a5dcd88-0cc7-4965-b2b7-3cccaf70b50e@github.com> > This PR contains the API and implementation changes for JEP-419 [1]. A more detailed description of such changes, to avoid repetitions during the review process, is included as a separate comment. > > [1] - https://openjdk.java.net/jeps/419 Maurizio Cimadamore has updated the pull request incrementally with one additional commit since the last revision: Rename MemorySegment::ofAddressNative to MemorySegment::ofAddress (which is consistent with other restricted factories in VaList and NativeSymbol) ------------- Changes: - all: https://git.openjdk.java.net/jdk/pull/5907/files - new: https://git.openjdk.java.net/jdk/pull/5907/files/350f1f07..663e72a8 Webrevs: - full: https://webrevs.openjdk.java.net/?repo=jdk&pr=5907&range=19 - incr: https://webrevs.openjdk.java.net/?repo=jdk&pr=5907&range=18-19 Stats: 51 lines in 23 files changed: 0 ins; 3 del; 48 mod Patch: https://git.openjdk.java.net/jdk/pull/5907.diff Fetch: git fetch https://git.openjdk.java.net/jdk pull/5907/head:pull/5907 PR: https://git.openjdk.java.net/jdk/pull/5907 From ihse at openjdk.java.net Fri Nov 5 11:39:09 2021 From: ihse at openjdk.java.net (Magnus Ihse Bursie) Date: Fri, 5 Nov 2021 11:39:09 GMT Subject: RFR: 8276400: openjdk image Jars, Zips and Jmods built from the same source are not reproducible In-Reply-To: <5UDNFhfUqwOsOgcV2wEBWHZKU84d5w6Cc5JZQ-XEITk=.e4431826-178a-4c33-8cf4-b5a6e83016e5@github.com> References: <5UDNFhfUqwOsOgcV2wEBWHZKU84d5w6Cc5JZQ-XEITk=.e4431826-178a-4c33-8cf4-b5a6e83016e5@github.com> Message-ID: <7gOYWLbtg-mpCeFFGgqfrQs4e1QBfuXb7sz7UNkSTws=.90227519-ead9-487b-9e94-6acc15335312@github.com> On Thu, 4 Nov 2021 20:56:45 GMT, Andrew Leonard wrote: > This PR enables reproducible Jars, Jmods and openjdk image zip files (eg.src.zip). > It provides support for SOURCE_DATE_EPOCH for Jar, Jmod and underlying ZipOutputStream's. > It fixes the following keys issues relating to reproducibility: > - Jar and ZipOutputStream are not SOURCE_DATE_EPOCH aware > - Jar and ZipOutputStream now detect SOURCE_DATE_EPOCH environment setting > - Jar and Jmod file content ordering was non-determinsitic > - Fixes to Jar and Jmod main's to ensure sorted classes content ordering > - openjdk image zip file generation used "zip" which is non-determinsitic > - New openjdk build tool "GenerateZip" which produces the final determinsitic zip files as part of the build and also detects SOURCE_DATE_EPOCH > > Signed-off-by: Andrew Leonard I agree with Alan's comment above. First of all, to be absolutely clear, I think this is a very worthy goal, and much overdue. I'll do my best to help get this implemented. But. This PR conflates two different issues. One of them is "changing the zip implementation in the JDK". This will affect all users of the JDK, and as Alan says, this will require a thorough discussion to get it right, including a proper CSR. Otoh, the benefit of doing this will affect *all* Java projects out there in the wild wanting to do reproducible builds, so this has actually a much higher leverage for the reproducible build community as a whole. And, also, it is a prerequisite for issue number two, which is "use the new zip implementation in the build system". Once the former is in place, this is more or less a no-brainer. Most will come automatically, but there are some things to fix. But these require just the approval of developers in the build team; no long discussions or CSRs. (Also, when we get to this point, I believe we should get rid of the "zip" utility and write our own in Java, rather than unzipping and then re-zipping. But we can take that discussion when we get there.) In a way, Alan is right when saying a PR is premature for issue #1. OTOH, my personal opinion is that "working code" is a good start for any discussion, as long as one is prepared to throw it all out and start over, or rewrite it over and over again, like @jaikiran did in the Property storage PR. So my suggestion is that your break out these changes to the zip and jar implementation, open a new PR, and start a discussion around them. ------------- PR: https://git.openjdk.java.net/jdk/pull/6268 From ihse at openjdk.java.net Fri Nov 5 11:45:14 2021 From: ihse at openjdk.java.net (Magnus Ihse Bursie) Date: Fri, 5 Nov 2021 11:45:14 GMT Subject: RFR: 8276400: openjdk image Jars, Zips and Jmods built from the same source are not reproducible In-Reply-To: References: <5UDNFhfUqwOsOgcV2wEBWHZKU84d5w6Cc5JZQ-XEITk=.e4431826-178a-4c33-8cf4-b5a6e83016e5@github.com> Message-ID: On Fri, 5 Nov 2021 11:31:34 GMT, Andrew Leonard wrote: >> This PR enables reproducible Jars, Jmods and openjdk image zip files (eg.src.zip). >> It provides support for SOURCE_DATE_EPOCH for Jar, Jmod and underlying ZipOutputStream's. >> It fixes the following keys issues relating to reproducibility: >> - Jar and ZipOutputStream are not SOURCE_DATE_EPOCH aware >> - Jar and ZipOutputStream now detect SOURCE_DATE_EPOCH environment setting >> - Jar and Jmod file content ordering was non-determinsitic >> - Fixes to Jar and Jmod main's to ensure sorted classes content ordering >> - openjdk image zip file generation used "zip" which is non-determinsitic >> - New openjdk build tool "GenerateZip" which produces the final determinsitic zip files as part of the build and also detects SOURCE_DATE_EPOCH >> >> Signed-off-by: Andrew Leonard > >> I think this is going to require discussion, a PR may be too premature. Is your goal to get the JDK build and run-time images creates with jlink to use the SOURCE_DATE_EPOCH? Are you looking for project builds that use the jar/jmod/etc. tools to use it? Or are you looking to have every library and application that uses the java.util.zip APIs to read it? If it includes the latter, and the patch in the PR suggests you are, then it will require analysis to work through the API spec changes. >> >> One suggestion is to look at JDK-8231640 and PR 5372 [1]. The original proposal was to use SOURCE_DATE_EPOCH. After a lengthy discussion we converged on the system property java.properties.date rather than the env variable. I suspect that much of that discussion applies here. >> >> -Alan >> >> [1] https://github.com/[/pull/5372](https://github.com/openjdk/jdk/pull/5372) > > @AlanBateman Hi Alan, thank you for the comments. So yes, totally agree this has a way to go, I thought creating a PR would kick things off...! So just to clarify the initial objectives which you raise above. My current aim is actually purely an OpenJDK JDK image perspective, building on the work @magicus has been doing with SOURCE_DATE_EPOCH in the build, hence my natural approach to this PR. So from that perspective, it's purely a "build" change. However, the openjdk build obviously uses openjdk Jar, Jmod tooling to build itself, hence the direction I took in this PR. > > Magnus's PR #5372 is useful thank you, I had not discovered that change, as you say the direction there is similar to mine, and I understand the potential pitfalls of adding doPrivileged's etc. So that resolved to a new system property java.properties.date, which makes sense. > > So with that in mind, and given the objective of JDK image reproducibility, this would imply presumably an approach of a new cmd line option to Jar and Jmod tools (eg.--source-date ), since these tools are cmds? > > One of the key changes in this PR is to ZipOutputStream where it sets the ZipEntry.xdostime if not set, to the current time. Now I am thinking given the "objective", if we fix all upstream tooling to ensure they set ZipEntry times, using "--source-date" (or whatever mechanism), then we can then avoid changing ZipOutputStream I believe. JDK tools like jar, jmod generate/update extra files like Manifests, and in fact I think the current jmod implementation writes all its content using the current time. > > So something along these lines: > jar -cf myjar.jar --source-date "" * > jmod create --source-date "" ... my.jmod > The jar, jmod,.. tooling then ensure all ZipEntry times are set to "source-date". > > I chose "source-date" name based on being synonymous with SOURCE_DATE_EPOCH, implying a similar "use case". > > So taking this approach for JDK tooling should achieve the reproducible JDK image objective, without a java API behavior change I am hoping. > Thoughts? > > Regards > Andrew @andrew-m-leonard Just to be clear: changing arguments to the command line utilities is still an API change and will need a CSR. My gut feeling is that using a system property that affect all users of ZipStream is preferable. That way a user could run like `java -Djava.properties.date=$(SOURCE_DATE_EPOCH) -Djava.zipfile.date=$(SOURCE_DATE_EPOCH) ...` (or whatever) and get reproducible behavior in all their code, for places that could otherwise be hard to fix. ------------- PR: https://git.openjdk.java.net/jdk/pull/6268 From ihse at openjdk.java.net Fri Nov 5 11:45:17 2021 From: ihse at openjdk.java.net (Magnus Ihse Bursie) Date: Fri, 5 Nov 2021 11:45:17 GMT Subject: RFR: 8276400: openjdk image Jars, Zips and Jmods built from the same source are not reproducible In-Reply-To: <5UDNFhfUqwOsOgcV2wEBWHZKU84d5w6Cc5JZQ-XEITk=.e4431826-178a-4c33-8cf4-b5a6e83016e5@github.com> References: <5UDNFhfUqwOsOgcV2wEBWHZKU84d5w6Cc5JZQ-XEITk=.e4431826-178a-4c33-8cf4-b5a6e83016e5@github.com> Message-ID: On Thu, 4 Nov 2021 20:56:45 GMT, Andrew Leonard wrote: > This PR enables reproducible Jars, Jmods and openjdk image zip files (eg.src.zip). > It provides support for SOURCE_DATE_EPOCH for Jar, Jmod and underlying ZipOutputStream's. > It fixes the following keys issues relating to reproducibility: > - Jar and ZipOutputStream are not SOURCE_DATE_EPOCH aware > - Jar and ZipOutputStream now detect SOURCE_DATE_EPOCH environment setting > - Jar and Jmod file content ordering was non-determinsitic > - Fixes to Jar and Jmod main's to ensure sorted classes content ordering > - openjdk image zip file generation used "zip" which is non-determinsitic > - New openjdk build tool "GenerateZip" which produces the final determinsitic zip files as part of the build and also detects SOURCE_DATE_EPOCH > > Signed-off-by: Andrew Leonard (And we can easily make sure that we do that from the JDK build) ------------- PR: https://git.openjdk.java.net/jdk/pull/6268 From mcimadamore at openjdk.java.net Fri Nov 5 11:54:14 2021 From: mcimadamore at openjdk.java.net (Maurizio Cimadamore) Date: Fri, 5 Nov 2021 11:54:14 GMT Subject: RFR: 8275063: Implementation of Foreign Function & Memory API (Second incubator) [v12] In-Reply-To: References: Message-ID: On Tue, 2 Nov 2021 15:40:45 GMT, Paul Sandoz wrote: >> Maurizio Cimadamore has updated the pull request incrementally with one additional commit since the last revision: >> >> Tweak javadoc of loaderLookup > > Marked as reviewed by psandoz (Reviewer). I have made some minor API changes (added two methods to `CLinker` to return the upcall and downcall method types, as suggested offline by @PaulSandoz). I've also cleaned up the `CLinker` javadoc, and added a section on safety consideration, streamlined the links in the package-level javadoc and renamed `MemorySegment::ofAddressNative` to simply `MemorySegment::ofAddress` (which is consistent with restricted factories in `NativeSymbol` and `VaList`). javadoc: http://cr.openjdk.java.net/~mcimadamore/JEP-419/v2/javadoc/jdk/incubator/foreign/package-summary.html specdiff: http://cr.openjdk.java.net/~mcimadamore/JEP-419/v2/specdiff_out/overview-summary.html ------------- PR: https://git.openjdk.java.net/jdk/pull/5907 From aleonard at openjdk.java.net Fri Nov 5 12:13:10 2021 From: aleonard at openjdk.java.net (Andrew Leonard) Date: Fri, 5 Nov 2021 12:13:10 GMT Subject: RFR: 8276400: openjdk image Jars, Zips and Jmods built from the same source are not reproducible In-Reply-To: References: <5UDNFhfUqwOsOgcV2wEBWHZKU84d5w6Cc5JZQ-XEITk=.e4431826-178a-4c33-8cf4-b5a6e83016e5@github.com> Message-ID: <06lpIVfltXdvWVSiRgOywDz2YZl7HCGrDSZbYgQMDRw=.12a56f57-45e4-4fad-a1f7-a2692efde752@github.com> On Fri, 5 Nov 2021 11:16:45 GMT, Lance Andersen wrote: >> This PR enables reproducible Jars, Jmods and openjdk image zip files (eg.src.zip). >> It provides support for SOURCE_DATE_EPOCH for Jar, Jmod and underlying ZipOutputStream's. >> It fixes the following keys issues relating to reproducibility: >> - Jar and ZipOutputStream are not SOURCE_DATE_EPOCH aware >> - Jar and ZipOutputStream now detect SOURCE_DATE_EPOCH environment setting >> - Jar and Jmod file content ordering was non-determinsitic >> - Fixes to Jar and Jmod main's to ensure sorted classes content ordering >> - openjdk image zip file generation used "zip" which is non-determinsitic >> - New openjdk build tool "GenerateZip" which produces the final determinsitic zip files as part of the build and also detects SOURCE_DATE_EPOCH >> >> Signed-off-by: Andrew Leonard > > src/jdk.jartool/share/classes/sun/tools/jar/Main.java line 795: > >> 793: // Ensure files list is sorted for reproducible jar content >> 794: Arrays.sort(files); >> 795: > > Have you had a chance to measure the performance with a large number of Zip entries with this change? No I haven't, but my thoughts on this were assuming you had a zip with many 1000s of ZipEntries the file I/O would be far more significant. Also, you will note this is not sorting the whole set, just within each directory, so the sort won't be complex, unless you had 1000s of files in a single directory. The "non-determinism" comes from the File.list() implementation which uses OS file listing, whose order is non-deterministic. ------------- PR: https://git.openjdk.java.net/jdk/pull/6268 From Alan.Bateman at oracle.com Fri Nov 5 12:19:45 2021 From: Alan.Bateman at oracle.com (Alan Bateman) Date: Fri, 5 Nov 2021 12:19:45 +0000 Subject: RFR: 8276400: openjdk image Jars, Zips and Jmods built from the same source are not reproducible In-Reply-To: <7gOYWLbtg-mpCeFFGgqfrQs4e1QBfuXb7sz7UNkSTws=.90227519-ead9-487b-9e94-6acc15335312@github.com> References: <5UDNFhfUqwOsOgcV2wEBWHZKU84d5w6Cc5JZQ-XEITk=.e4431826-178a-4c33-8cf4-b5a6e83016e5@github.com> <7gOYWLbtg-mpCeFFGgqfrQs4e1QBfuXb7sz7UNkSTws=.90227519-ead9-487b-9e94-6acc15335312@github.com> Message-ID: <5e565027-20f6-e1c0-f42b-3c16cd47cf4b@oracle.com> On 05/11/2021 11:39, Magnus Ihse Bursie wrote: > : > I agree with Alan's comment above. First of all, to be absolutely clear, I think this is a very worthy goal, and much overdue. I'll do my best to help get this implemented. > One suggestion is to separate out the issue of ordering of entries (in zip/JAR and JMOD) and whether it would be the default. There may be trade-offs to discuss, also whether it's limited to just new zip/JAR files or updates to existing zip/JARs files. -Alan From aleonard at openjdk.java.net Fri Nov 5 12:27:09 2021 From: aleonard at openjdk.java.net (Andrew Leonard) Date: Fri, 5 Nov 2021 12:27:09 GMT Subject: RFR: 8276400: openjdk image Jars, Zips and Jmods built from the same source are not reproducible In-Reply-To: References: <5UDNFhfUqwOsOgcV2wEBWHZKU84d5w6Cc5JZQ-XEITk=.e4431826-178a-4c33-8cf4-b5a6e83016e5@github.com> Message-ID: <7d_9gJtgcNEfYDH_VkkDrdj-aW6wQPphDNsBObc8e4k=.422c1bc2-c9ef-492c-a45c-55995a2b1965@github.com> On Fri, 5 Nov 2021 11:31:34 GMT, Andrew Leonard wrote: >> This PR enables reproducible Jars, Jmods and openjdk image zip files (eg.src.zip). >> It provides support for SOURCE_DATE_EPOCH for Jar, Jmod and underlying ZipOutputStream's. >> It fixes the following keys issues relating to reproducibility: >> - Jar and ZipOutputStream are not SOURCE_DATE_EPOCH aware >> - Jar and ZipOutputStream now detect SOURCE_DATE_EPOCH environment setting >> - Jar and Jmod file content ordering was non-determinsitic >> - Fixes to Jar and Jmod main's to ensure sorted classes content ordering >> - openjdk image zip file generation used "zip" which is non-determinsitic >> - New openjdk build tool "GenerateZip" which produces the final determinsitic zip files as part of the build and also detects SOURCE_DATE_EPOCH >> >> Signed-off-by: Andrew Leonard > >> I think this is going to require discussion, a PR may be too premature. Is your goal to get the JDK build and run-time images creates with jlink to use the SOURCE_DATE_EPOCH? Are you looking for project builds that use the jar/jmod/etc. tools to use it? Or are you looking to have every library and application that uses the java.util.zip APIs to read it? If it includes the latter, and the patch in the PR suggests you are, then it will require analysis to work through the API spec changes. >> >> One suggestion is to look at JDK-8231640 and PR 5372 [1]. The original proposal was to use SOURCE_DATE_EPOCH. After a lengthy discussion we converged on the system property java.properties.date rather than the env variable. I suspect that much of that discussion applies here. >> >> -Alan >> >> [1] https://github.com/[/pull/5372](https://github.com/openjdk/jdk/pull/5372) > > @AlanBateman Hi Alan, thank you for the comments. So yes, totally agree this has a way to go, I thought creating a PR would kick things off...! So just to clarify the initial objectives which you raise above. My current aim is actually purely an OpenJDK JDK image perspective, building on the work @magicus has been doing with SOURCE_DATE_EPOCH in the build, hence my natural approach to this PR. So from that perspective, it's purely a "build" change. However, the openjdk build obviously uses openjdk Jar, Jmod tooling to build itself, hence the direction I took in this PR. > > Magnus's PR #5372 is useful thank you, I had not discovered that change, as you say the direction there is similar to mine, and I understand the potential pitfalls of adding doPrivileged's etc. So that resolved to a new system property java.properties.date, which makes sense. > > So with that in mind, and given the objective of JDK image reproducibility, this would imply presumably an approach of a new cmd line option to Jar and Jmod tools (eg.--source-date ), since these tools are cmds? > > One of the key changes in this PR is to ZipOutputStream where it sets the ZipEntry.xdostime if not set, to the current time. Now I am thinking given the "objective", if we fix all upstream tooling to ensure they set ZipEntry times, using "--source-date" (or whatever mechanism), then we can then avoid changing ZipOutputStream I believe. JDK tools like jar, jmod generate/update extra files like Manifests, and in fact I think the current jmod implementation writes all its content using the current time. > > So something along these lines: > jar -cf myjar.jar --source-date "" * > jmod create --source-date "" ... my.jmod > The jar, jmod,.. tooling then ensure all ZipEntry times are set to "source-date". > > I chose "source-date" name based on being synonymous with SOURCE_DATE_EPOCH, implying a similar "use case". > > So taking this approach for JDK tooling should achieve the reproducible JDK image objective, without a java API behavior change I am hoping. > Thoughts? > > Regards > Andrew > @andrew-m-leonard Just to be clear: changing arguments to the command line utilities is still an API change and will need a CSR. > > My gut feeling is that using a system property that affect all users of ZipStream is preferable. That way a user could run like `java -Djava.properties.date=$(SOURCE_DATE_EPOCH) -Djava.zipfile.date=$(SOURCE_DATE_EPOCH) ...` (or whatever) and get reproducible behavior in all their code, for places that could otherwise be hard to fix. Thanks @magicus Yes, I understand a CSR is required, I was trying to state there would not be a java.util.zip API behavior change necessarily. Just to state again, and although I need to check this, I think if all JDK tooling ensures ZipEntry's had date set to whatever prior to calling ZipOutputStream.putNextEntry(e), then I think no change to ZipOutputStream would be needed (I think). However, you do elude to the benefit to "others" of changing ZipOutputStream to use (say -Djava.zipfile.date), in that it makes it easier for consumers to make reproducible builds rather than tracking down all the code that calls it in their project. GenerateZip as you say is slightly separate, and bespoke to the openjdk build. I did start by trying to create my own "zip", but that is actually quite involved due to how openjdk build makes zip files, and how it leverages zip functionality for things like --includes & --excludes. It is far easier actually to just construct the final deterministic zip file using this class at the end. But I agree, if someone could write a determinisitic "zip", that would be great please! I will move GenerateZip to a separate PR, and make it take "timestamp" just like CreateSymbols already does. Thanks again Andrew ------------- PR: https://git.openjdk.java.net/jdk/pull/6268 From aleonard at openjdk.java.net Fri Nov 5 12:34:13 2021 From: aleonard at openjdk.java.net (Andrew Leonard) Date: Fri, 5 Nov 2021 12:34:13 GMT Subject: RFR: 8276400: openjdk image Jars, Zips and Jmods built from the same source are not reproducible In-Reply-To: <5UDNFhfUqwOsOgcV2wEBWHZKU84d5w6Cc5JZQ-XEITk=.e4431826-178a-4c33-8cf4-b5a6e83016e5@github.com> References: <5UDNFhfUqwOsOgcV2wEBWHZKU84d5w6Cc5JZQ-XEITk=.e4431826-178a-4c33-8cf4-b5a6e83016e5@github.com> Message-ID: <35zICVlvWHkfc0XXtfZ40b5y0DIruMeYwHXWniMe69A=.2c674398-8d22-457b-99a1-dd7ef2f6f5b7@github.com> On Thu, 4 Nov 2021 20:56:45 GMT, Andrew Leonard wrote: > This PR enables reproducible Jars, Jmods and openjdk image zip files (eg.src.zip). > It provides support for SOURCE_DATE_EPOCH for Jar, Jmod and underlying ZipOutputStream's. > It fixes the following keys issues relating to reproducibility: > - Jar and ZipOutputStream are not SOURCE_DATE_EPOCH aware > - Jar and ZipOutputStream now detect SOURCE_DATE_EPOCH environment setting > - Jar and Jmod file content ordering was non-determinsitic > - Fixes to Jar and Jmod main's to ensure sorted classes content ordering > - openjdk image zip file generation used "zip" which is non-determinsitic > - New openjdk build tool "GenerateZip" which produces the final determinsitic zip files as part of the build and also detects SOURCE_DATE_EPOCH > > Signed-off-by: Andrew Leonard > _Mailing list message from [Alan Bateman](mailto:Alan.Bateman at oracle.com) on [build-dev](mailto:build-dev at mail.openjdk.java.net):_ > > On 05/11/2021 11:39, Magnus Ihse Bursie wrote: > > > : > > I agree with Alan's comment above. First of all, to be absolutely clear, I think this is a very worthy goal, and much overdue. I'll do my best to help get this implemented. > > One suggestion is to separate out the issue of ordering of entries (in zip/JAR and JMOD) and whether it would be the default. There may be trade-offs to discuss, also whether it's limited to just new zip/JAR files or updates to existing zip/JARs files. > > -Alan Yes, makes sense, @LanceAndersen already picked up on that for file sorting. We can discuss in a separate PR. So picking up on what @magicus suggested as well, what I shall do is split this into 3: 1) GenerateZip openjdk build tool 2) Changes to Jar and Jmod main's to ensure file ordering 3) Jar, Jmod and/or ZipOutputStream changes to support a "source-date" specification ------------- PR: https://git.openjdk.java.net/jdk/pull/6268 From aleonard at openjdk.java.net Fri Nov 5 12:43:10 2021 From: aleonard at openjdk.java.net (Andrew Leonard) Date: Fri, 5 Nov 2021 12:43:10 GMT Subject: RFR: 8276400: openjdk image Jars, Zips and Jmods built from the same source are not reproducible In-Reply-To: References: <5UDNFhfUqwOsOgcV2wEBWHZKU84d5w6Cc5JZQ-XEITk=.e4431826-178a-4c33-8cf4-b5a6e83016e5@github.com> Message-ID: On Fri, 5 Nov 2021 11:19:16 GMT, Lance Andersen wrote: >> This PR enables reproducible Jars, Jmods and openjdk image zip files (eg.src.zip). >> It provides support for SOURCE_DATE_EPOCH for Jar, Jmod and underlying ZipOutputStream's. >> It fixes the following keys issues relating to reproducibility: >> - Jar and ZipOutputStream are not SOURCE_DATE_EPOCH aware >> - Jar and ZipOutputStream now detect SOURCE_DATE_EPOCH environment setting >> - Jar and Jmod file content ordering was non-determinsitic >> - Fixes to Jar and Jmod main's to ensure sorted classes content ordering >> - openjdk image zip file generation used "zip" which is non-determinsitic >> - New openjdk build tool "GenerateZip" which produces the final determinsitic zip files as part of the build and also detects SOURCE_DATE_EPOCH >> >> Signed-off-by: Andrew Leonard > > test/jdk/java/util/zip/TestZipSourceDateEpoch.sh line 1: > >> 1: #!/bin/sh > > Unless there is a specific reason to use a shell script, it would be better to convert this to java. We have been trying to reduce the usage of shell scripts I had copied an existing example, I obviously chose a bad example! I can re-write with ProcessBuilder... cheers ------------- PR: https://git.openjdk.java.net/jdk/pull/6268 From ihse at openjdk.java.net Fri Nov 5 13:22:13 2021 From: ihse at openjdk.java.net (Magnus Ihse Bursie) Date: Fri, 5 Nov 2021 13:22:13 GMT Subject: RFR: 8276400: openjdk image Jars, Zips and Jmods built from the same source are not reproducible In-Reply-To: <5UDNFhfUqwOsOgcV2wEBWHZKU84d5w6Cc5JZQ-XEITk=.e4431826-178a-4c33-8cf4-b5a6e83016e5@github.com> References: <5UDNFhfUqwOsOgcV2wEBWHZKU84d5w6Cc5JZQ-XEITk=.e4431826-178a-4c33-8cf4-b5a6e83016e5@github.com> Message-ID: On Thu, 4 Nov 2021 20:56:45 GMT, Andrew Leonard wrote: > This PR enables reproducible Jars, Jmods and openjdk image zip files (eg.src.zip). > It provides support for SOURCE_DATE_EPOCH for Jar, Jmod and underlying ZipOutputStream's. > It fixes the following keys issues relating to reproducibility: > - Jar and ZipOutputStream are not SOURCE_DATE_EPOCH aware > - Jar and ZipOutputStream now detect SOURCE_DATE_EPOCH environment setting > - Jar and Jmod file content ordering was non-determinsitic > - Fixes to Jar and Jmod main's to ensure sorted classes content ordering > - openjdk image zip file generation used "zip" which is non-determinsitic > - New openjdk build tool "GenerateZip" which produces the final determinsitic zip files as part of the build and also detects SOURCE_DATE_EPOCH > > Signed-off-by: Andrew Leonard Sounds like a good plan. As for the build part ("GenerateZip"), this is not as important in the build as it used to be. Back in the days, before Jigsaw, we had a huge rt.jar that needed updating each time an incremental build was done. Hence the elaborate system for updating existing zip files. (Because recompressing the entire rt.jar was prohibitively slow.) Now we instead have jmod/jlink which do not support incremental updates and are almost as slow as before (that's a bit underprioritized, as well...) but due to the modularization, all modules except java.base are fairly quick to link anyway. So the old zip generation is only used for src.zip, afaik. (We might be using it in closed Oracle code as well for some side artifacts, but that is not relevant to the OpenJDK story, and to be honest, I'm not sure anymore.) This has no need to be quick to update incrementally. If it turns out to be too slow, we can move it out of the normal "jdk-image" target and add a "image-srczip" or whatever. So, another way of stating this is that GenerateZip has as its main goal to facilitate creation of src.zip. And most of the logic in CreateZipArchive.gmk can basically just be thrown out. ------------- PR: https://git.openjdk.java.net/jdk/pull/6268 From aleonard at openjdk.java.net Fri Nov 5 13:39:08 2021 From: aleonard at openjdk.java.net (Andrew Leonard) Date: Fri, 5 Nov 2021 13:39:08 GMT Subject: RFR: 8276400: openjdk image Jars, Zips and Jmods built from the same source are not reproducible In-Reply-To: <5UDNFhfUqwOsOgcV2wEBWHZKU84d5w6Cc5JZQ-XEITk=.e4431826-178a-4c33-8cf4-b5a6e83016e5@github.com> References: <5UDNFhfUqwOsOgcV2wEBWHZKU84d5w6Cc5JZQ-XEITk=.e4431826-178a-4c33-8cf4-b5a6e83016e5@github.com> Message-ID: <7rbFC_bx2ucbe6pkZzoMbUj8AMQqS_2PF2o9bkFFTxM=.2969e668-2ff8-4bdd-a0f1-72f18425ae0e@github.com> On Thu, 4 Nov 2021 20:56:45 GMT, Andrew Leonard wrote: > This PR enables reproducible Jars, Jmods and openjdk image zip files (eg.src.zip). > It provides support for SOURCE_DATE_EPOCH for Jar, Jmod and underlying ZipOutputStream's. > It fixes the following keys issues relating to reproducibility: > - Jar and ZipOutputStream are not SOURCE_DATE_EPOCH aware > - Jar and ZipOutputStream now detect SOURCE_DATE_EPOCH environment setting > - Jar and Jmod file content ordering was non-determinsitic > - Fixes to Jar and Jmod main's to ensure sorted classes content ordering > - openjdk image zip file generation used "zip" which is non-determinsitic > - New openjdk build tool "GenerateZip" which produces the final determinsitic zip files as part of the build and also detects SOURCE_DATE_EPOCH > > Signed-off-by: Andrew Leonard Thanks Magnus, You've just reminded me of why I gave up trying to write my own zip, I achieved it, but realized it was slow damn slow, and how efficient and optimized the off the shelf zip is...! Hence, why I let zip still do it's inclusion/exclusion job it does so well, and then just re-generate at the end. Takes about 2-3 seconds longer, so not an issue really. cheers ------------- PR: https://git.openjdk.java.net/jdk/pull/6268 From jvernee at openjdk.java.net Fri Nov 5 14:29:19 2021 From: jvernee at openjdk.java.net (Jorn Vernee) Date: Fri, 5 Nov 2021 14:29:19 GMT Subject: RFR: 8275063: Implementation of Foreign Function & Memory API (Second incubator) [v18] In-Reply-To: <4iHYQMJoHZwfRJCHV9tYB_5t92pjEsgISw_d9_Nt6H8=.1fb75c40-a6ee-498a-9d4e-cf1b6d11b583@github.com> References: <4iHYQMJoHZwfRJCHV9tYB_5t92pjEsgISw_d9_Nt6H8=.1fb75c40-a6ee-498a-9d4e-cf1b6d11b583@github.com> Message-ID: On Fri, 5 Nov 2021 11:06:53 GMT, Maurizio Cimadamore wrote: >> This PR contains the API and implementation changes for JEP-419 [1]. A more detailed description of such changes, to avoid repetitions during the review process, is included as a separate comment. >> >> [1] - https://openjdk.java.net/jeps/419 > > Maurizio Cimadamore has updated the pull request incrementally with one additional commit since the last revision: > > * Add two new CLinker static methods to compute upcall/downcall method types > * Clarify section on CLinker downcall type > * Add section on CLinker safety guarantees src/jdk.incubator.foreign/share/classes/jdk/incubator/foreign/CLinker.java line 65: > 63: *

  • if {@code L} is a {@link ValueLayout} with carrier {@code E} then there are two cases: > 64: *
      > 65: *
    • if {@code L} occurs in a parameter position and {@code E} is {@code NativeAddress.class}, This looks spurious src/jdk.incubator.foreign/share/classes/jdk/incubator/foreign/CLinker.java line 134: > 132: *

      > 133: * Upcall stubs are generally safer to work with, as the linker runtime can validate the type of the target method > 134: * handle against the provided function descriptor and report an error if any mismatch is detected. If the target method But, in the case of upcalls, errors can still occur if the native code casts the pointer to the upcall stub to an incorrect type, e.g. `FunctionDescriptor.ofVoid(ADDRESS, ADDRESS)`, but on the native side cast it to `void (*)(void*)`, meaning the second argument would be garbage on the Java side. i.e. there is still room for a mismatch the same as with downcalls. src/jdk.incubator.foreign/share/classes/jdk/incubator/foreign/CLinker.java line 267: > 265: static MethodType upcallType(FunctionDescriptor functionDescriptor) { > 266: return SharedUtils.inferMethodType(functionDescriptor, true); > 267: } Nice! :) ------------- PR: https://git.openjdk.java.net/jdk/pull/5907 From mcimadamore at openjdk.java.net Fri Nov 5 14:37:23 2021 From: mcimadamore at openjdk.java.net (Maurizio Cimadamore) Date: Fri, 5 Nov 2021 14:37:23 GMT Subject: RFR: 8275063: Implementation of Foreign Function & Memory API (Second incubator) [v18] In-Reply-To: References: <4iHYQMJoHZwfRJCHV9tYB_5t92pjEsgISw_d9_Nt6H8=.1fb75c40-a6ee-498a-9d4e-cf1b6d11b583@github.com> Message-ID: On Fri, 5 Nov 2021 14:25:35 GMT, Jorn Vernee wrote: >> Maurizio Cimadamore has updated the pull request incrementally with one additional commit since the last revision: >> >> * Add two new CLinker static methods to compute upcall/downcall method types >> * Clarify section on CLinker downcall type >> * Add section on CLinker safety guarantees > > src/jdk.incubator.foreign/share/classes/jdk/incubator/foreign/CLinker.java line 134: > >> 132: *

      >> 133: * Upcall stubs are generally safer to work with, as the linker runtime can validate the type of the target method >> 134: * handle against the provided function descriptor and report an error if any mismatch is detected. If the target method > > But, in the case of upcalls, errors can still occur if the native code casts the pointer to the upcall stub to an incorrect type, e.g. `FunctionDescriptor.ofVoid(ADDRESS, ADDRESS)`, but on the native side cast it to `void (*)(void*)`, meaning the second argument would be garbage on the Java side. i.e. there is still room for a mismatch the same as with downcalls. Yes and no. In a downcall, you just don't know what signature the downcall will feature in the native lib. So you pass a function descriptor and you hope it's ok. In the upcall case you _do_ know the signature of the Java upcall code you want to call, so you can validate the descriptor against that. Of course the native code can still cast things around in ways that blow things up, but the two problems seem somewhat different, at least to me. But I can tweak the text a bit. ------------- PR: https://git.openjdk.java.net/jdk/pull/5907 From mcimadamore at openjdk.java.net Fri Nov 5 15:28:45 2021 From: mcimadamore at openjdk.java.net (Maurizio Cimadamore) Date: Fri, 5 Nov 2021 15:28:45 GMT Subject: RFR: 8275063: Implementation of Foreign Function & Memory API (Second incubator) [v21] In-Reply-To: References: Message-ID: <_YLlQk23TfRkCzouXvgHH3Zxktw1sxo1uvae5KsjlFw=.3c4f3aeb-e24f-424b-94f4-04b19f0e834b@github.com> > This PR contains the API and implementation changes for JEP-419 [1]. A more detailed description of such changes, to avoid repetitions during the review process, is included as a separate comment. > > [1] - https://openjdk.java.net/jeps/419 Maurizio Cimadamore has updated the pull request incrementally with one additional commit since the last revision: Clarify safety considerations for upcalls ------------- Changes: - all: https://git.openjdk.java.net/jdk/pull/5907/files - new: https://git.openjdk.java.net/jdk/pull/5907/files/663e72a8..2aa126a9 Webrevs: - full: https://webrevs.openjdk.java.net/?repo=jdk&pr=5907&range=20 - incr: https://webrevs.openjdk.java.net/?repo=jdk&pr=5907&range=19-20 Stats: 5 lines in 1 file changed: 2 ins; 0 del; 3 mod Patch: https://git.openjdk.java.net/jdk/pull/5907.diff Fetch: git fetch https://git.openjdk.java.net/jdk pull/5907/head:pull/5907 PR: https://git.openjdk.java.net/jdk/pull/5907 From aleonard at openjdk.java.net Fri Nov 5 15:39:29 2021 From: aleonard at openjdk.java.net (Andrew Leonard) Date: Fri, 5 Nov 2021 15:39:29 GMT Subject: RFR: 8276654: element-list order is non deterministic Message-ID: Fixes: https://bugs.openjdk.java.net/browse/JDK-8276654 A intermittent problem with the make dependencies means the jdk.javadoc element-list-.txt generation can remove the src defined element|package-list-<7,8,9,10>.txt files. Recreatable by using --with-jobs=1 causing jdk.javadoc "gendata" to always occur after "java" module build dependency. This fix puts a dependency of jdk.javadoc-java on jdk.javadoc-gendata to avoid this. Signed-off-by: Andrew Leonard ------------- Commit messages: - 8276654: element-list order is non deterministic Changes: https://git.openjdk.java.net/jdk/pull/6278/files Webrev: https://webrevs.openjdk.java.net/?repo=jdk&pr=6278&range=00 Issue: https://bugs.openjdk.java.net/browse/JDK-8276654 Stats: 5 lines in 1 file changed: 5 ins; 0 del; 0 mod Patch: https://git.openjdk.java.net/jdk/pull/6278.diff Fetch: git fetch https://git.openjdk.java.net/jdk pull/6278/head:pull/6278 PR: https://git.openjdk.java.net/jdk/pull/6278 From jvernee at openjdk.java.net Fri Nov 5 15:52:16 2021 From: jvernee at openjdk.java.net (Jorn Vernee) Date: Fri, 5 Nov 2021 15:52:16 GMT Subject: RFR: 8275063: Implementation of Foreign Function & Memory API (Second incubator) [v18] In-Reply-To: References: <4iHYQMJoHZwfRJCHV9tYB_5t92pjEsgISw_d9_Nt6H8=.1fb75c40-a6ee-498a-9d4e-cf1b6d11b583@github.com> Message-ID: On Fri, 5 Nov 2021 14:33:44 GMT, Maurizio Cimadamore wrote: >> src/jdk.incubator.foreign/share/classes/jdk/incubator/foreign/CLinker.java line 134: >> >>> 132: *

      >>> 133: * Upcall stubs are generally safer to work with, as the linker runtime can validate the type of the target method >>> 134: * handle against the provided function descriptor and report an error if any mismatch is detected. If the target method >> >> But, in the case of upcalls, errors can still occur if the native code casts the pointer to the upcall stub to an incorrect type, e.g. `FunctionDescriptor.ofVoid(ADDRESS, ADDRESS)`, but on the native side cast it to `void (*)(void*)`, meaning the second argument would be garbage on the Java side. i.e. there is still room for a mismatch the same as with downcalls. > > Yes and no. In a downcall, you just don't know what signature the downcall will feature in the native lib. So you pass a function descriptor and you hope it's ok. In the upcall case you _do_ know the signature of the Java upcall code you want to call, so you can validate the descriptor against that. Of course the native code can still cast things around in ways that blow things up, but the two problems seem somewhat different, at least to me. But I can tweak the text a bit. Ok, thanks. I think of it more like this: in both cases we specify a native type as well as a Java type, both in the form of a FunctionDescriptor, from which we then derive the Java type in the form of a MethodType. If there is a mismatch here with what the native code does we are in trouble, this seems the same for downcalls and upcalls. In both cases we know the Java side for sure, it's the native side we can't validate (they are just flipped around for upcalls). But, for upcalls there is an additional thing that can go wrong: the type of the target MethodHandle we pass could have a mismatch with the type we inferred from the FunctionDescriptor, so there we need to do an extra check. i.e. in a way this seems _less_ safe (though a different kind of safety), than downcalls, since there is an additional way to mess up with the linkage request, although we can catch that case. ------------- PR: https://git.openjdk.java.net/jdk/pull/5907 From mcimadamore at openjdk.java.net Fri Nov 5 16:02:43 2021 From: mcimadamore at openjdk.java.net (Maurizio Cimadamore) Date: Fri, 5 Nov 2021 16:02:43 GMT Subject: RFR: 8275063: Implementation of Foreign Function & Memory API (Second incubator) [v22] In-Reply-To: References: Message-ID: > This PR contains the API and implementation changes for JEP-419 [1]. A more detailed description of such changes, to avoid repetitions during the review process, is included as a separate comment. > > [1] - https://openjdk.java.net/jeps/419 Maurizio Cimadamore has updated the pull request incrementally with one additional commit since the last revision: Further tweak upcall safety considerations ------------- Changes: - all: https://git.openjdk.java.net/jdk/pull/5907/files - new: https://git.openjdk.java.net/jdk/pull/5907/files/2aa126a9..4e3af9f1 Webrevs: - full: https://webrevs.openjdk.java.net/?repo=jdk&pr=5907&range=21 - incr: https://webrevs.openjdk.java.net/?repo=jdk&pr=5907&range=20-21 Stats: 3 lines in 1 file changed: 0 ins; 0 del; 3 mod Patch: https://git.openjdk.java.net/jdk/pull/5907.diff Fetch: git fetch https://git.openjdk.java.net/jdk pull/5907/head:pull/5907 PR: https://git.openjdk.java.net/jdk/pull/5907 From erikj at openjdk.java.net Fri Nov 5 16:18:09 2021 From: erikj at openjdk.java.net (Erik Joelsson) Date: Fri, 5 Nov 2021 16:18:09 GMT Subject: RFR: 8276654: element-list order is non deterministic In-Reply-To: References: Message-ID: <8zSMZC3e7kI-ccGa9yxTnR0NuFaDvMK2N5YdpNY_Uu0=.6d36535b-d881-4ae6-9dbb-0d92a1b1922a@github.com> On Fri, 5 Nov 2021 15:26:40 GMT, Andrew Leonard wrote: > Fixes: https://bugs.openjdk.java.net/browse/JDK-8276654 > > A intermittent problem with the make dependencies means the jdk.javadoc element-list-.txt generation can remove the src defined element|package-list-<7,8,9,10>.txt files. > Recreatable by using --with-jobs=1 causing jdk.javadoc "gendata" to always occur after "java" module build dependency. > This fix puts a dependency of jdk.javadoc-java on jdk.javadoc-gendata to avoid this. > > Signed-off-by: Andrew Leonard Good catch, but wouldn't it be better if we could fix the recipe in jdk.javadoc/Gendata.gmk to only delete the files it intends to create. If I understand what files are created where correctly, you could generate a glob for the rm statement using something like this: $(RM) $(ELEMENT_LISTS_DIR)/element-list-{$(call CommaList,$(call sequence,11,$(VERSION_FEATURE)))}.txt ------------- PR: https://git.openjdk.java.net/jdk/pull/6278 From ihse at openjdk.java.net Fri Nov 5 16:36:26 2021 From: ihse at openjdk.java.net (Magnus Ihse Bursie) Date: Fri, 5 Nov 2021 16:36:26 GMT Subject: RFR: 8276746: Add section on reproducible builds in building.md Message-ID: <4qxomlh4etmgqyTVynmu5SjW1IRhCXVTvckEiPOXrCM=.bdc66132-8299-480c-9276-7c03c54202f5@github.com> Reproducible builds are all the vogue. The JDK has been making great strides in getting there, but still has some way to go. However, to get as close as possible, some special configuration is needed. This has been "tribal knowledge" of a few persons in the build team. It needs to be properly documented to help other users wanting to create reproducible builds of the JDK. ------------- Commit messages: - 8276746: Add section on reproducible builds in building.md Changes: https://git.openjdk.java.net/jdk/pull/6279/files Webrev: https://webrevs.openjdk.java.net/?repo=jdk&pr=6279&range=00 Issue: https://bugs.openjdk.java.net/browse/JDK-8276746 Stats: 89 lines in 2 files changed: 89 ins; 0 del; 0 mod Patch: https://git.openjdk.java.net/jdk/pull/6279.diff Fetch: git fetch https://git.openjdk.java.net/jdk pull/6279/head:pull/6279 PR: https://git.openjdk.java.net/jdk/pull/6279 From ihse at openjdk.java.net Fri Nov 5 16:37:12 2021 From: ihse at openjdk.java.net (Magnus Ihse Bursie) Date: Fri, 5 Nov 2021 16:37:12 GMT Subject: RFR: 8276672: Cannot build hsdis on WSL In-Reply-To: <4lLdxiPsFsB1GfI7iWSgLOXQ-Q_WU4ZZdOXHkRuO0kc=.45eea799-3e3d-4b1c-90dc-a6fa5e3d5b74@github.com> References: <4lLdxiPsFsB1GfI7iWSgLOXQ-Q_WU4ZZdOXHkRuO0kc=.45eea799-3e3d-4b1c-90dc-a6fa5e3d5b74@github.com> Message-ID: On Fri, 5 Nov 2021 03:03:55 GMT, Yasumasa Suenaga wrote: > JDK-8275128 introduced new Makefile for hsdis to integrate it to normal build system, however it does not work on WSL 1 Ubuntu 20.04 and MinGW from Ubuntu. > > Hsdis.gmk has two problems: > > 1. MinGW version is fixed to "9.2.0" > 2. Assumes sysroot for MinGW is located at /usr/$MINGW_BASE/sys-root > > I tested this change on WSL only, so it is appreciate to review on Cygwin. Hi @YaSuenag, Thank you for your contribution. I'll have a closer look and try this out on Cygwin after the weekend. ------------- PR: https://git.openjdk.java.net/jdk/pull/6269 From magnus.ihse.bursie at oracle.com Fri Nov 5 16:38:52 2021 From: magnus.ihse.bursie at oracle.com (Magnus Ihse Bursie) Date: Fri, 5 Nov 2021 17:38:52 +0100 Subject: Does CDS archive generation work for crossbuilds? In-Reply-To: <3360cbca-16b5-f656-4ebe-2a7f40c86265@azul.com> References: <3360cbca-16b5-f656-4ebe-2a7f40c86265@azul.com> Message-ID: <066d1032-aa12-d011-c3c5-2316a8b583d1@oracle.com> On 2021-10-29 21:35, Anton Kozlov wrote: > QEMU user space emulation [1] works pretty fast. On my linux-x86 > laptop native > java -Xshare:dump completes in 0.4 sec, and with qemu-aarch64 it's > about 2.3 > sec. GHA now provides a runnable instruction on how to create the > sysroot [2]. But that time excludes booting up the virtual machine, I presume. Nevertheless, if someone is interested in going down this road to get CDS generation for cross-compilation, I would not object to putting it in the build system. /Magnus > > Regarding problems, different page size for linux-aarch64 first comes > to mind, > but it should be fixed by JDK-8268396: "CDS archive with 4K alignment > unusable > on machines with 64k pages" > > Thanks, > Anton > > [1] https://www.qemu.org/docs/master/user/main.html > [2] > https://github.com/openjdk/jdk/blob/master/.github/workflows/submit.yml#L514 > > > > On 10/29/21 14:35, Magnus Ihse Bursie wrote: >> On 2021-10-28 22:56, Ioi Lam wrote: >>> How reliable would it be to use qemu to run the cross-compiled >>> binaries? Has anyone tried that recently? >> I experimented with qemu and the JDK build some years ago (so not >> really recently). As I remember it, the takeaway was that it probably >> was reliable, but it was slow as h*ll, up to the point of being >> practically unusable. >> >> Add to this the fact that you need to prepare an entire OS image, >> complete with all tools needed to build the JDK... I set up a few >> such images for my own use (I think it was emulating linux-aarch64 on >> x64) to run testing. I had an idea of describing how I did to share >> the knowledge, but in the end it was just too complicated and slow to >> even consider recommending. >> >> /Magnus >>> >>> >>> On 10/23/21 5:48 AM, Thomas St?fe wrote: >>>> Hi Alan, >>>> >>>> On Sat, Oct 23, 2021 at 9:58 AM Alan Bateman >>>> wrote: >>>> >>>>> >>>>> On 23/10/2021 07:57, Thomas St?fe wrote: >>>>>> Hi, >>>>>> >>>>>> when I crossbuild (for linux aarch64, using a devkit, building on >>>>>> linux >>>>>> x64), for some reason I don't >>>>>> get the classes.jsa generated inside the images directory. >>>>>> >>>>>> My configure options: >>>>>> >>>>>> >>>>> --with-devkit=/shared/projects/openjdk/devkits/x86_64-linux-gnu-to-aarch64-linux-gnu >>>>> >>>>>> --openjdk-target=aarch64-linux-gnu >>>>>> --with-boot-jdk=/shared/projects/openjdk/jdks/sapmachine17 >>>>>> >>>>> --with-build-jdk=/shared/projects/openjdk/jdk-jdk/output-release/images/jdk >>>>> >>>>>> --with-gtest=/shared/projects/openjdk/gtest/googletest >>>>>> --with-debug-level=fastdebug >>>>>> >>>>>> The build jdk is a freshly build x64 release VM from the same source >>>>> tree. >>>>>> Am I missing something obvious? Is CDS archive generation even >>>>>> supported >>>>>> for crossbuilds? >>>>> It needs the generate run-time to execute "java -Xshare:dump" so I >>>>> don't >>>>> expect so. hotspot-runtime-dev is probably the place to discuss the >>>>> details. BTW: this came up recently in the context of the jlink >>>>> plugin >>>>> that generates the CDS archive. The plugin needed a check to >>>>> ensure that >>>>> the target platform matched the current platform as it could >>>>> launch the >>>>> target VM to create the dump. >>>>> >>>>> >>>> Thinking for a second, probably it cannot work since we copy binary >>>> structures verbatim to the archive; I guess the chance that they >>>> are binary >>>> compatible between platforms is very small. But it should be easily >>>> rectified by calling Xshare:dump on the target platform. >>>> >>>> Thank you! >>>> >>>> ..Thomas >>>> >>>> >>>>> -Alan >>>>> >>> >> From ihse at openjdk.java.net Fri Nov 5 16:39:13 2021 From: ihse at openjdk.java.net (Magnus Ihse Bursie) Date: Fri, 5 Nov 2021 16:39:13 GMT Subject: RFR: 8276654: element-list order is non deterministic In-Reply-To: <8zSMZC3e7kI-ccGa9yxTnR0NuFaDvMK2N5YdpNY_Uu0=.6d36535b-d881-4ae6-9dbb-0d92a1b1922a@github.com> References: <8zSMZC3e7kI-ccGa9yxTnR0NuFaDvMK2N5YdpNY_Uu0=.6d36535b-d881-4ae6-9dbb-0d92a1b1922a@github.com> Message-ID: On Fri, 5 Nov 2021 16:15:02 GMT, Erik Joelsson wrote: >> Fixes: https://bugs.openjdk.java.net/browse/JDK-8276654 >> >> A intermittent problem with the make dependencies means the jdk.javadoc element-list-.txt generation can remove the src defined element|package-list-<7,8,9,10>.txt files. >> Recreatable by using --with-jobs=1 causing jdk.javadoc "gendata" to always occur after "java" module build dependency. >> This fix puts a dependency of jdk.javadoc-java on jdk.javadoc-gendata to avoid this. >> >> Signed-off-by: Andrew Leonard > > Good catch, but wouldn't it be better if we could fix the recipe in jdk.javadoc/Gendata.gmk to only delete the files it intends to create. If I understand what files are created where correctly, you could generate a glob for the rm statement using something like this: > > $(RM) $(ELEMENT_LISTS_DIR)/element-list-{$(call CommaList,$(call sequence,11,$(VERSION_FEATURE)))}.txt @erikj79 Ah, we have a `sequence` function! I forgot about it. My initial reaction was also to prefer fixing the recipe to not delete non-related files, but I could not find a nice and not too overly complicated way of doing it. I agree that this is a better solution. ------------- PR: https://git.openjdk.java.net/jdk/pull/6278 From erikj at openjdk.java.net Fri Nov 5 16:50:10 2021 From: erikj at openjdk.java.net (Erik Joelsson) Date: Fri, 5 Nov 2021 16:50:10 GMT Subject: RFR: 8276746: Add section on reproducible builds in building.md In-Reply-To: <4qxomlh4etmgqyTVynmu5SjW1IRhCXVTvckEiPOXrCM=.bdc66132-8299-480c-9276-7c03c54202f5@github.com> References: <4qxomlh4etmgqyTVynmu5SjW1IRhCXVTvckEiPOXrCM=.bdc66132-8299-480c-9276-7c03c54202f5@github.com> Message-ID: <8Jeimuw8nhFlKu6lsoapq7jDbl7GFcCYuxC78Hc2W1Y=.09ffebcd-1880-4053-866e-96f07226a8c0@github.com> On Fri, 5 Nov 2021 16:30:26 GMT, Magnus Ihse Bursie wrote: > Reproducible builds are all the vogue. The JDK has been making great strides in getting there, but still has some way to go. However, to get as close as possible, some special configuration is needed. > > This has been "tribal knowledge" of a few persons in the build team. It needs to be properly documented to help other users wanting to create reproducible builds of the JDK. Marked as reviewed by erikj (Reviewer). doc/building.md line 1551: > 1549: > 1550: **Hint:** If your build environment already sets `SOURCE_DATE_EPOCH`, you can > 1551: propagate this using `--with-source-date=$(SOURCE_DATE_EPOCH)`. Shouldn't this example be $SOURCE_DATE_EPOCH, or at least ${SOURCE_DATE_EPOCH} as this would most likely be run in a shell? Or are you trying to send in the make variable reference verbatim to be evaluated in the makefiles? ------------- PR: https://git.openjdk.java.net/jdk/pull/6279 From aleonard at openjdk.java.net Fri Nov 5 17:01:10 2021 From: aleonard at openjdk.java.net (Andrew Leonard) Date: Fri, 5 Nov 2021 17:01:10 GMT Subject: RFR: 8276654: element-list order is non deterministic In-Reply-To: <8zSMZC3e7kI-ccGa9yxTnR0NuFaDvMK2N5YdpNY_Uu0=.6d36535b-d881-4ae6-9dbb-0d92a1b1922a@github.com> References: <8zSMZC3e7kI-ccGa9yxTnR0NuFaDvMK2N5YdpNY_Uu0=.6d36535b-d881-4ae6-9dbb-0d92a1b1922a@github.com> Message-ID: On Fri, 5 Nov 2021 16:15:02 GMT, Erik Joelsson wrote: >> Fixes: https://bugs.openjdk.java.net/browse/JDK-8276654 >> >> A intermittent problem with the make dependencies means the jdk.javadoc element-list-.txt generation can remove the src defined element|package-list-<7,8,9,10>.txt files. >> Recreatable by using --with-jobs=1 causing jdk.javadoc "gendata" to always occur after "java" module build dependency. >> This fix puts a dependency of jdk.javadoc-java on jdk.javadoc-gendata to avoid this. >> >> Signed-off-by: Andrew Leonard > > Good catch, but wouldn't it be better if we could fix the recipe in jdk.javadoc/Gendata.gmk to only delete the files it intends to create. If I understand what files are created where correctly, you could generate a glob for the rm statement using something like this: > > $(RM) $(ELEMENT_LISTS_DIR)/element-list-{$(call CommaList,$(call sequence,11,$(VERSION_FEATURE)))}.txt > @erikj79 Ah, we have a `sequence` function! I forgot about it. My initial reaction was also to prefer fixing the recipe to not delete non-related files, but I could not find a nice and not too overly complicated way of doing it. > > I agree that this is a better solution. @erikj79 Yes, I thought that to start with, but exactly as @magicus said, I couldn't figure and simple/non-complex way in gnu make language to fix it there either! > Good catch, but wouldn't it be better if we could fix the recipe in jdk.javadoc/Gendata.gmk to only delete the files it intends to create. If I understand what files are created where correctly, you could generate a glob for the rm statement using something like this: > > $(RM) $(ELEMENT_LISTS_DIR)/element-list-{$(call CommaList,$(call sequence,11,$(VERSION_FEATURE)))}.txt I'll give that a try, cheers ------------- PR: https://git.openjdk.java.net/jdk/pull/6278 From ihse at openjdk.java.net Fri Nov 5 17:06:12 2021 From: ihse at openjdk.java.net (Magnus Ihse Bursie) Date: Fri, 5 Nov 2021 17:06:12 GMT Subject: RFR: 8276746: Add section on reproducible builds in building.md In-Reply-To: <8Jeimuw8nhFlKu6lsoapq7jDbl7GFcCYuxC78Hc2W1Y=.09ffebcd-1880-4053-866e-96f07226a8c0@github.com> References: <4qxomlh4etmgqyTVynmu5SjW1IRhCXVTvckEiPOXrCM=.bdc66132-8299-480c-9276-7c03c54202f5@github.com> <8Jeimuw8nhFlKu6lsoapq7jDbl7GFcCYuxC78Hc2W1Y=.09ffebcd-1880-4053-866e-96f07226a8c0@github.com> Message-ID: <_r2gtOlh5f3kkpolCvRHJiSRE8QRf7Nt92v21aPM-ww=.491f21ee-0496-49af-8b4a-5e5c3588639c@github.com> On Fri, 5 Nov 2021 16:45:34 GMT, Erik Joelsson wrote: >> Reproducible builds are all the vogue. The JDK has been making great strides in getting there, but still has some way to go. However, to get as close as possible, some special configuration is needed. >> >> This has been "tribal knowledge" of a few persons in the build team. It needs to be properly documented to help other users wanting to create reproducible builds of the JDK. > > doc/building.md line 1551: > >> 1549: >> 1550: **Hint:** If your build environment already sets `SOURCE_DATE_EPOCH`, you can >> 1551: propagate this using `--with-source-date=$(SOURCE_DATE_EPOCH)`. > > Shouldn't this example be $SOURCE_DATE_EPOCH, or at least ${SOURCE_DATE_EPOCH} as this would most likely be run in a shell? Or are you trying to send in the make variable reference verbatim to be evaluated in the makefiles? Yeah, no, you're right. It's environmental damage due to working too long in makefiles. :) I'll fix, thanks for catching it. ------------- PR: https://git.openjdk.java.net/jdk/pull/6279 From sgehwolf at openjdk.java.net Fri Nov 5 17:25:10 2021 From: sgehwolf at openjdk.java.net (Severin Gehwolf) Date: Fri, 5 Nov 2021 17:25:10 GMT Subject: RFR: 8276746: Add section on reproducible builds in building.md In-Reply-To: <4qxomlh4etmgqyTVynmu5SjW1IRhCXVTvckEiPOXrCM=.bdc66132-8299-480c-9276-7c03c54202f5@github.com> References: <4qxomlh4etmgqyTVynmu5SjW1IRhCXVTvckEiPOXrCM=.bdc66132-8299-480c-9276-7c03c54202f5@github.com> Message-ID: On Fri, 5 Nov 2021 16:30:26 GMT, Magnus Ihse Bursie wrote: > Reproducible builds are all the vogue. The JDK has been making great strides in getting there, but still has some way to go. However, to get as close as possible, some special configuration is needed. > > This has been "tribal knowledge" of a few persons in the build team. It needs to be properly documented to help other users wanting to create reproducible builds of the JDK. LGTM. Thanks for documenting it! ------------- Marked as reviewed by sgehwolf (Reviewer). PR: https://git.openjdk.java.net/jdk/pull/6279 From aleonard at openjdk.java.net Fri Nov 5 18:46:26 2021 From: aleonard at openjdk.java.net (Andrew Leonard) Date: Fri, 5 Nov 2021 18:46:26 GMT Subject: RFR: 8276654: element-list order is non deterministic [v2] In-Reply-To: References: Message-ID: > Fixes: https://bugs.openjdk.java.net/browse/JDK-8276654 > > A intermittent problem with the make dependencies means the jdk.javadoc element-list-.txt generation can remove the src defined element|package-list-<7,8,9,10>.txt files. > Recreatable by using --with-jobs=1 causing jdk.javadoc "gendata" to always occur after "java" module build dependency. > This fix puts a dependency of jdk.javadoc-java on jdk.javadoc-gendata to avoid this. > > Signed-off-by: Andrew Leonard Andrew Leonard has refreshed the contents of this pull request, and previous commits have been removed. The incremental views will show differences compared to the previous content of the PR. The pull request contains one new commit since the last revision: 8276654: element-list order is non deterministic Signed-off-by: Andrew Leonard ------------- Changes: - all: https://git.openjdk.java.net/jdk/pull/6278/files - new: https://git.openjdk.java.net/jdk/pull/6278/files/28fbedde..014534cf Webrevs: - full: https://webrevs.openjdk.java.net/?repo=jdk&pr=6278&range=01 - incr: https://webrevs.openjdk.java.net/?repo=jdk&pr=6278&range=00-01 Stats: 6 lines in 2 files changed: 0 ins; 5 del; 1 mod Patch: https://git.openjdk.java.net/jdk/pull/6278.diff Fetch: git fetch https://git.openjdk.java.net/jdk pull/6278/head:pull/6278 PR: https://git.openjdk.java.net/jdk/pull/6278 From aleonard at openjdk.java.net Fri Nov 5 18:46:28 2021 From: aleonard at openjdk.java.net (Andrew Leonard) Date: Fri, 5 Nov 2021 18:46:28 GMT Subject: RFR: 8276654: element-list order is non deterministic In-Reply-To: <8zSMZC3e7kI-ccGa9yxTnR0NuFaDvMK2N5YdpNY_Uu0=.6d36535b-d881-4ae6-9dbb-0d92a1b1922a@github.com> References: <8zSMZC3e7kI-ccGa9yxTnR0NuFaDvMK2N5YdpNY_Uu0=.6d36535b-d881-4ae6-9dbb-0d92a1b1922a@github.com> Message-ID: On Fri, 5 Nov 2021 16:15:02 GMT, Erik Joelsson wrote: >> Fixes: https://bugs.openjdk.java.net/browse/JDK-8276654 >> >> A intermittent problem with the make dependencies means the jdk.javadoc element-list-.txt generation can remove the src defined element|package-list-<7,8,9,10>.txt files. >> Recreatable by using --with-jobs=1 causing jdk.javadoc "gendata" to always occur after "java" module build dependency. >> This fix puts a dependency of jdk.javadoc-java on jdk.javadoc-gendata to avoid this. >> >> Signed-off-by: Andrew Leonard > > Good catch, but wouldn't it be better if we could fix the recipe in jdk.javadoc/Gendata.gmk to only delete the files it intends to create. If I understand what files are created where correctly, you could generate a glob for the rm statement using something like this: > > $(RM) $(ELEMENT_LISTS_DIR)/element-list-{$(call CommaList,$(call sequence,11,$(VERSION_FEATURE)))}.txt @erikj79 yep looks good: ------------- PR: https://git.openjdk.java.net/jdk/pull/6278 From aleonard at openjdk.java.net Fri Nov 5 19:15:43 2021 From: aleonard at openjdk.java.net (Andrew Leonard) Date: Fri, 5 Nov 2021 19:15:43 GMT Subject: Withdrawn: 8276400: openjdk image Jars, Zips and Jmods built from the same source are not reproducible In-Reply-To: <5UDNFhfUqwOsOgcV2wEBWHZKU84d5w6Cc5JZQ-XEITk=.e4431826-178a-4c33-8cf4-b5a6e83016e5@github.com> References: <5UDNFhfUqwOsOgcV2wEBWHZKU84d5w6Cc5JZQ-XEITk=.e4431826-178a-4c33-8cf4-b5a6e83016e5@github.com> Message-ID: On Thu, 4 Nov 2021 20:56:45 GMT, Andrew Leonard wrote: > This PR enables reproducible Jars, Jmods and openjdk image zip files (eg.src.zip). > It provides support for SOURCE_DATE_EPOCH for Jar, Jmod and underlying ZipOutputStream's. > It fixes the following keys issues relating to reproducibility: > - Jar and ZipOutputStream are not SOURCE_DATE_EPOCH aware > - Jar and ZipOutputStream now detect SOURCE_DATE_EPOCH environment setting > - Jar and Jmod file content ordering was non-determinsitic > - Fixes to Jar and Jmod main's to ensure sorted classes content ordering > - openjdk image zip file generation used "zip" which is non-determinsitic > - New openjdk build tool "GenerateZip" which produces the final determinsitic zip files as part of the build and also detects SOURCE_DATE_EPOCH > > Signed-off-by: Andrew Leonard This pull request has been closed without being integrated. ------------- PR: https://git.openjdk.java.net/jdk/pull/6268 From aleonard at openjdk.java.net Fri Nov 5 19:15:43 2021 From: aleonard at openjdk.java.net (Andrew Leonard) Date: Fri, 5 Nov 2021 19:15:43 GMT Subject: RFR: 8276400: openjdk image Jars, Zips and Jmods built from the same source are not reproducible In-Reply-To: <5UDNFhfUqwOsOgcV2wEBWHZKU84d5w6Cc5JZQ-XEITk=.e4431826-178a-4c33-8cf4-b5a6e83016e5@github.com> References: <5UDNFhfUqwOsOgcV2wEBWHZKU84d5w6Cc5JZQ-XEITk=.e4431826-178a-4c33-8cf4-b5a6e83016e5@github.com> Message-ID: On Thu, 4 Nov 2021 20:56:45 GMT, Andrew Leonard wrote: > This PR enables reproducible Jars, Jmods and openjdk image zip files (eg.src.zip). > It provides support for SOURCE_DATE_EPOCH for Jar, Jmod and underlying ZipOutputStream's. > It fixes the following keys issues relating to reproducibility: > - Jar and ZipOutputStream are not SOURCE_DATE_EPOCH aware > - Jar and ZipOutputStream now detect SOURCE_DATE_EPOCH environment setting > - Jar and Jmod file content ordering was non-determinsitic > - Fixes to Jar and Jmod main's to ensure sorted classes content ordering > - openjdk image zip file generation used "zip" which is non-determinsitic > - New openjdk build tool "GenerateZip" which produces the final determinsitic zip files as part of the build and also detects SOURCE_DATE_EPOCH > > Signed-off-by: Andrew Leonard @AlanBateman @magicus thank you both for your guidance. I have now split this bug into the 3 mentioned: - GenerateZip: https://bugs.openjdk.java.net/browse/JDK-8276743 - Jar/Jmod content ordering: https://bugs.openjdk.java.net/browse/JDK-8276764 - Jar/Jmod/ZipOutputStream timestamp option: https://bugs.openjdk.java.net/browse/JDK-8276766 **(requires CSR)** Closing this PR to be replaced with 3 new PRs for the above bugs. ------------- PR: https://git.openjdk.java.net/jdk/pull/6268 From erikj at openjdk.java.net Fri Nov 5 19:22:45 2021 From: erikj at openjdk.java.net (Erik Joelsson) Date: Fri, 5 Nov 2021 19:22:45 GMT Subject: RFR: 8276654: element-list order is non deterministic [v2] In-Reply-To: References: Message-ID: On Fri, 5 Nov 2021 18:46:26 GMT, Andrew Leonard wrote: >> Fixes: https://bugs.openjdk.java.net/browse/JDK-8276654 >> >> A intermittent problem with the make dependencies means the jdk.javadoc element-list-.txt generation can remove the src defined element|package-list-<7,8,9,10>.txt files. >> Recreatable by using --with-jobs=1 causing jdk.javadoc "gendata" to always occur after "java" module build dependency. >> This fix puts a dependency of jdk.javadoc-java on jdk.javadoc-gendata to avoid this. >> >> Signed-off-by: Andrew Leonard > > Andrew Leonard has refreshed the contents of this pull request, and previous commits have been removed. The incremental views will show differences compared to the previous content of the PR. make/modules/jdk.javadoc/Gendata.gmk line 75: > 73: $(call MakeTargetDir) > 74: $(call LogInfo, Creating javadoc element lists) > 75: $(RM) $(ELEMENT_LISTS_DIR)/element-list-{$(call CommaList,$(call sequence,$(GENERATE_SYMBOLS_FROM_JDK_VERSION),$(JDK_SOURCE_TARGET_VERSION)))}.txt Good to see that it worked! I would only wish that you found a way to break up the line. Long lines make future side-by-side reviews and 3-way merges hard. We don't enforce strict 80, but try to stay in some reasonable ballpark in the build files. I think both CommaList and sequence are ok with whitespace in their parameters. Otherwise you could pre-calculate the numbers list in a variable before the recipe. ------------- PR: https://git.openjdk.java.net/jdk/pull/6278 From aleonard at openjdk.java.net Fri Nov 5 19:25:34 2021 From: aleonard at openjdk.java.net (Andrew Leonard) Date: Fri, 5 Nov 2021 19:25:34 GMT Subject: RFR: 8276746: Add section on reproducible builds in building.md In-Reply-To: <4qxomlh4etmgqyTVynmu5SjW1IRhCXVTvckEiPOXrCM=.bdc66132-8299-480c-9276-7c03c54202f5@github.com> References: <4qxomlh4etmgqyTVynmu5SjW1IRhCXVTvckEiPOXrCM=.bdc66132-8299-480c-9276-7c03c54202f5@github.com> Message-ID: On Fri, 5 Nov 2021 16:30:26 GMT, Magnus Ihse Bursie wrote: > Reproducible builds are all the vogue. The JDK has been making great strides in getting there, but still has some way to go. However, to get as close as possible, some special configuration is needed. > > This has been "tribal knowledge" of a few persons in the build team. It needs to be properly documented to help other users wanting to create reproducible builds of the JDK. Looks good Magnus ------------- Marked as reviewed by aleonard (Committer). PR: https://git.openjdk.java.net/jdk/pull/6279 From aleonard at openjdk.java.net Fri Nov 5 19:29:39 2021 From: aleonard at openjdk.java.net (Andrew Leonard) Date: Fri, 5 Nov 2021 19:29:39 GMT Subject: RFR: 8276654: element-list order is non deterministic [v2] In-Reply-To: References: Message-ID: On Fri, 5 Nov 2021 19:20:10 GMT, Erik Joelsson wrote: >> Andrew Leonard has refreshed the contents of this pull request, and previous commits have been removed. The incremental views will show differences compared to the previous content of the PR. > > make/modules/jdk.javadoc/Gendata.gmk line 75: > >> 73: $(call MakeTargetDir) >> 74: $(call LogInfo, Creating javadoc element lists) >> 75: $(RM) $(ELEMENT_LISTS_DIR)/element-list-{$(call CommaList,$(call sequence,$(GENERATE_SYMBOLS_FROM_JDK_VERSION),$(JDK_SOURCE_TARGET_VERSION)))}.txt > > Good to see that it worked! I would only wish that you found a way to break up the line. Long lines make future side-by-side reviews and 3-way merges hard. We don't enforce strict 80, but try to stay in some reasonable ballpark in the build files. > > I think both CommaList and sequence are ok with whitespace in their parameters. Otherwise you could pre-calculate the numbers list in a variable before the recipe. yep, i'll have a look to see what I can do, not helped with existing variables being so long!! (GENERATE_SYMBOLS_FROM_JDK_VERSION) ------------- PR: https://git.openjdk.java.net/jdk/pull/6278 From aleonard at openjdk.java.net Fri Nov 5 19:47:04 2021 From: aleonard at openjdk.java.net (Andrew Leonard) Date: Fri, 5 Nov 2021 19:47:04 GMT Subject: RFR: 8276654: element-list order is non deterministic [v3] In-Reply-To: References: Message-ID: > Fixes: https://bugs.openjdk.java.net/browse/JDK-8276654 > > A intermittent problem with the make dependencies means the jdk.javadoc element-list-.txt generation can remove the src defined element|package-list-<7,8,9,10>.txt files. > Recreatable by using --with-jobs=1 causing jdk.javadoc "gendata" to always occur after "java" module build dependency. > This fix puts a dependency of jdk.javadoc-java on jdk.javadoc-gendata to avoid this. > > Signed-off-by: Andrew Leonard Andrew Leonard has refreshed the contents of this pull request, and previous commits have been removed. The incremental views will show differences compared to the previous content of the PR. The pull request contains one new commit since the last revision: 8276654: element-list order is non deterministic Signed-off-by: Andrew Leonard ------------- Changes: - all: https://git.openjdk.java.net/jdk/pull/6278/files - new: https://git.openjdk.java.net/jdk/pull/6278/files/014534cf..9d7b338c Webrevs: - full: https://webrevs.openjdk.java.net/?repo=jdk&pr=6278&range=02 - incr: https://webrevs.openjdk.java.net/?repo=jdk&pr=6278&range=01-02 Stats: 2 lines in 1 file changed: 1 ins; 0 del; 1 mod Patch: https://git.openjdk.java.net/jdk/pull/6278.diff Fetch: git fetch https://git.openjdk.java.net/jdk pull/6278/head:pull/6278 PR: https://git.openjdk.java.net/jdk/pull/6278 From aleonard at openjdk.java.net Fri Nov 5 19:47:07 2021 From: aleonard at openjdk.java.net (Andrew Leonard) Date: Fri, 5 Nov 2021 19:47:07 GMT Subject: RFR: 8276654: element-list order is non deterministic [v2] In-Reply-To: References: Message-ID: <8DCp2e6Vza2kvsV_MsnTqx2EyMkj_YXgK1JGoarP5Fw=.f24a9010-f56e-4727-bea0-0bd64fdc8480@github.com> On Fri, 5 Nov 2021 19:26:56 GMT, Andrew Leonard wrote: >> make/modules/jdk.javadoc/Gendata.gmk line 75: >> >>> 73: $(call MakeTargetDir) >>> 74: $(call LogInfo, Creating javadoc element lists) >>> 75: $(RM) $(ELEMENT_LISTS_DIR)/element-list-{$(call CommaList,$(call sequence,$(GENERATE_SYMBOLS_FROM_JDK_VERSION),$(JDK_SOURCE_TARGET_VERSION)))}.txt >> >> Good to see that it worked! I would only wish that you found a way to break up the line. Long lines make future side-by-side reviews and 3-way merges hard. We don't enforce strict 80, but try to stay in some reasonable ballpark in the build files. >> >> I think both CommaList and sequence are ok with whitespace in their parameters. Otherwise you could pre-calculate the numbers list in a variable before the recipe. > > yep, i'll have a look to see what I can do, not helped with existing variables being so long!! (GENERATE_SYMBOLS_FROM_JDK_VERSION) yep, split on CommaList seems better and works fine, see revision ------------- PR: https://git.openjdk.java.net/jdk/pull/6278 From ihse at openjdk.java.net Fri Nov 5 21:11:41 2021 From: ihse at openjdk.java.net (Magnus Ihse Bursie) Date: Fri, 5 Nov 2021 21:11:41 GMT Subject: RFR: 8276654: element-list order is non deterministic [v2] In-Reply-To: <8DCp2e6Vza2kvsV_MsnTqx2EyMkj_YXgK1JGoarP5Fw=.f24a9010-f56e-4727-bea0-0bd64fdc8480@github.com> References: <8DCp2e6Vza2kvsV_MsnTqx2EyMkj_YXgK1JGoarP5Fw=.f24a9010-f56e-4727-bea0-0bd64fdc8480@github.com> Message-ID: On Fri, 5 Nov 2021 19:36:34 GMT, Andrew Leonard wrote: >> yep, i'll have a look to see what I can do, not helped with existing variables being so long!! (GENERATE_SYMBOLS_FROM_JDK_VERSION) > > yep, split on CommaList seems better and works fine, see revision Does it work if you have space after the comma in the call to sequence? We try to have a whitespace after comma where possible (and try to make function handle that case). If it does not work, we should really fix sequence... ------------- PR: https://git.openjdk.java.net/jdk/pull/6278 From ihse at openjdk.java.net Fri Nov 5 21:19:13 2021 From: ihse at openjdk.java.net (Magnus Ihse Bursie) Date: Fri, 5 Nov 2021 21:19:13 GMT Subject: RFR: 8276746: Add section on reproducible builds in building.md [v2] In-Reply-To: <4qxomlh4etmgqyTVynmu5SjW1IRhCXVTvckEiPOXrCM=.bdc66132-8299-480c-9276-7c03c54202f5@github.com> References: <4qxomlh4etmgqyTVynmu5SjW1IRhCXVTvckEiPOXrCM=.bdc66132-8299-480c-9276-7c03c54202f5@github.com> Message-ID: <9qgBGFZ6HuRFa-5bDyXV_6IXLAiauD4De6DvHtAM358=.f6041bfc-5ef4-4254-8463-8b2113a6eb16@github.com> > Reproducible builds are all the vogue. The JDK has been making great strides in getting there, but still has some way to go. However, to get as close as possible, some special configuration is needed. > > This has been "tribal knowledge" of a few persons in the build team. It needs to be properly documented to help other users wanting to create reproducible builds of the JDK. Magnus Ihse Bursie has updated the pull request incrementally with one additional commit since the last revision: Use shell syntax, not make ------------- Changes: - all: https://git.openjdk.java.net/jdk/pull/6279/files - new: https://git.openjdk.java.net/jdk/pull/6279/files/cfb154b3..d8698421 Webrevs: - full: https://webrevs.openjdk.java.net/?repo=jdk&pr=6279&range=01 - incr: https://webrevs.openjdk.java.net/?repo=jdk&pr=6279&range=00-01 Stats: 2 lines in 2 files changed: 0 ins; 0 del; 2 mod Patch: https://git.openjdk.java.net/jdk/pull/6279.diff Fetch: git fetch https://git.openjdk.java.net/jdk pull/6279/head:pull/6279 PR: https://git.openjdk.java.net/jdk/pull/6279 From ihse at openjdk.java.net Fri Nov 5 21:24:45 2021 From: ihse at openjdk.java.net (Magnus Ihse Bursie) Date: Fri, 5 Nov 2021 21:24:45 GMT Subject: Integrated: 8276746: Add section on reproducible builds in building.md In-Reply-To: <4qxomlh4etmgqyTVynmu5SjW1IRhCXVTvckEiPOXrCM=.bdc66132-8299-480c-9276-7c03c54202f5@github.com> References: <4qxomlh4etmgqyTVynmu5SjW1IRhCXVTvckEiPOXrCM=.bdc66132-8299-480c-9276-7c03c54202f5@github.com> Message-ID: On Fri, 5 Nov 2021 16:30:26 GMT, Magnus Ihse Bursie wrote: > Reproducible builds are all the vogue. The JDK has been making great strides in getting there, but still has some way to go. However, to get as close as possible, some special configuration is needed. > > This has been "tribal knowledge" of a few persons in the build team. It needs to be properly documented to help other users wanting to create reproducible builds of the JDK. This pull request has now been integrated. Changeset: 59c3dcc7 Author: Magnus Ihse Bursie URL: https://git.openjdk.java.net/jdk/commit/59c3dcc761ac7a9eab1517743cbb77e5526ca6f3 Stats: 89 lines in 2 files changed: 89 ins; 0 del; 0 mod 8276746: Add section on reproducible builds in building.md Reviewed-by: erikj, sgehwolf, aleonard ------------- PR: https://git.openjdk.java.net/jdk/pull/6279 From jjg at openjdk.java.net Sat Nov 6 20:57:36 2021 From: jjg at openjdk.java.net (Jonathan Gibbons) Date: Sat, 6 Nov 2021 20:57:36 GMT Subject: RFR: 8276654: element-list order is non deterministic [v3] In-Reply-To: References: Message-ID: <7KhnvJgG1xTjKrC6S0Sw9HBZtmGHS8cIyNkhN1nN5og=.41822512-c987-40a9-aa4a-077e491b9dd2@github.com> On Fri, 5 Nov 2021 19:47:04 GMT, Andrew Leonard wrote: >> Fixes: https://bugs.openjdk.java.net/browse/JDK-8276654 >> >> A intermittent problem with the make dependencies means the jdk.javadoc element-list-.txt generation can remove the src defined element|package-list-<7,8,9,10>.txt files. >> Recreatable by using --with-jobs=1 causing jdk.javadoc "gendata" to always occur after "java" module build dependency. >> >> Signed-off-by: Andrew Leonard > > Andrew Leonard has refreshed the contents of this pull request, and previous commits have been removed. The incremental views will show differences compared to the previous content of the PR. Can someone please verify that the contents of these files after the fix is the same as the contents of the files for "good" builds before the fix? ------------- PR: https://git.openjdk.java.net/jdk/pull/6278 From aleonard at openjdk.java.net Sun Nov 7 11:58:36 2021 From: aleonard at openjdk.java.net (Andrew Leonard) Date: Sun, 7 Nov 2021 11:58:36 GMT Subject: RFR: 8276654: element-list order is non deterministic [v3] In-Reply-To: <7KhnvJgG1xTjKrC6S0Sw9HBZtmGHS8cIyNkhN1nN5og=.41822512-c987-40a9-aa4a-077e491b9dd2@github.com> References: <7KhnvJgG1xTjKrC6S0Sw9HBZtmGHS8cIyNkhN1nN5og=.41822512-c987-40a9-aa4a-077e491b9dd2@github.com> Message-ID: On Sat, 6 Nov 2021 20:54:27 GMT, Jonathan Gibbons wrote: > Can someone please verify that the contents of these files after the fix is the same as the contents of the files for "good" builds before the fix? @jonathan-gibbons done, all good ------------- PR: https://git.openjdk.java.net/jdk/pull/6278 From vitaly.provodin at jetbrains.com Mon Nov 8 00:36:12 2021 From: vitaly.provodin at jetbrains.com (Vitaly Provodin) Date: Mon, 8 Nov 2021 07:36:12 +0700 Subject: AWT & Swing tests failed: Function frag_col has a deployment target which is incompatible with this OS. In-Reply-To: References: <8A9D3B5B-67A1-4FBE-910B-0CDC00682655@jetbrains.com> <090A5816-8F67-493D-8B00-DA20AC3C4CB5@jetbrains.com> <6E19D881-5C6A-4AF6-8A1A-EFAFAF694D1E@oracle.com> <5b12e5e3-8246-d652-70f1-3d66ed6f4b99@oracle.com> Message-ID: Actually adding the parameter --with-macosx-version-max does not resolve the issue. Accidentaly, the tests were executed on machines, where Xcode 12.2 were installed, and passed. But they failed on machines with Xcode 11.X After removing Xcode at all, the tests stably failed with the "Function frag_col has a deployment target which is incompatible with this OS" diagnostic. In all cases above JDK was built with --with-macosx-version-max=10.12.00 Thanks, Vitaly > On 29 Oct 2021, at 13:33, Vitaly Provodin wrote: > > Erik, > > the option --with-macosx-version-max=10.12.00 resolves the issue. > > thank you, > Vitaly > >> On 28 Oct 2021, at 00:09, erik.joelsson at oracle.com wrote: >> >> As far as I know, this is expected. You need to tell Xcode to compile and link with an older deployment target. I'm a bit fuzzy on the details, but as I remember it, by default, configure only sets up half the required settings for your to produce binaries that will work on Macos versions older than the build environment. At Oracle we also set "--with-macosx-version-max=10.12.00" which I believe is also needed. >> >> No need to file an issue in my opinion. >> >> /Erik >> >> On 2021-10-27 09:55, Philip Race wrote: >>> A bug should be filed but if it is specific to using xcode 12.5 we won't have seen >>> any issue yet at Oracle as all official builds use xcode 12.4. >>> >>> However at some point (JDK 19) we'll likely upgrade and then it'll be an issue. >>> >>> -phil. >>> >>> On 10/27/21 3:02 AM, Jayathirth D V wrote: >>>> Hi Vitaly, >>>> >>>> There is no issue in JBS with this failure. >>>> We had lanai CI jtreg/JCK test run recently on 23rd October and many of those runs were on macOS 10.15.7 and macOS 11.6 systems but I don?t see any compilation error reported. >>>> >>>> Also from below mail I am not able to get whether issue is related to some Xcode version or macOS version. Please clarify. >>>> >>>> Thanks, >>>> Jay >>>> >>>>> On 27-Oct-2021, at 3:04 PM, Vitaly Provodin wrote: >>>>> >>>>> Sorry did not complete the previous mail >>>>> >>>>> >>>>> If JDK is built on Catalina with Xcode 12.2, the tests passed well. >>>>> >>>>> >>>>> >>>>>> On 27 Oct 2021, at 16:32, Vitaly Provodin wrote: >>>>>> >>>>>> Hi all, >>>>>> >>>>>> it was found out that a lot of AWT & Swing tests failed on macOS-x64 <=10.15 with the following message >>>>>> >>>>>> >>>>>> 2021-10-25 05:36:42.170 java[6916:107817] Failed to create pipeline state, error Error Domain=CompilerError Code=1 "Function frag_col has a deployment target which is incompatible with this OS." UserInfo={NSLocalizedDescription=Function frag_col has a deployment target which is incompatible with this OS.} >>>>>> >>>>>> >>>>>> The issue was observed when JDK17, JDK18 was built on BigSur with Xcode12.5 >>>>>> 1. -Dsun.java2d.metal=true & macOS-x64 <=10.15 FAILED >>>>>> 2. -Dsun.java2d.metal=true & macOS-x64 >10.15 PASSED >>>>>> 2. -Dsun.java2d.metal=false & macOS-x64 <=10.15 PASSED >>>>>> 2. -Dsun.java2d.metal=false & macOS-x64 > 10.15 PASSED >>>>>> >>>>>> If JDK is build on Catalina with Xcode12.2 >>>>>> >>>>>> Not sure if this issue is known. Shoiuld it be reported to JBS? >>>>>> >>>>>> Thanks, >>>>>> Vitaly >>>>>> >>> > From ioi.lam at oracle.com Mon Nov 8 00:45:36 2021 From: ioi.lam at oracle.com (Ioi Lam) Date: Sun, 7 Nov 2021 16:45:36 -0800 Subject: Does CDS archive generation work for crossbuilds? In-Reply-To: <066d1032-aa12-d011-c3c5-2316a8b583d1@oracle.com> References: <3360cbca-16b5-f656-4ebe-2a7f40c86265@azul.com> <066d1032-aa12-d011-c3c5-2316a8b583d1@oracle.com> Message-ID: On 11/5/21 9:38 AM, Magnus Ihse Bursie wrote: > On 2021-10-29 21:35, Anton Kozlov wrote: >> QEMU user space emulation [1] works pretty fast. On my linux-x86 >> laptop native >> java -Xshare:dump completes in 0.4 sec, and with qemu-aarch64 it's >> about 2.3 >> sec. GHA now provides a runnable instruction on how to create the >> sysroot [2]. > But that time excludes booting up the virtual machine, I presume. > I think user space emulation doesn't spin up a new virtual machine. It runs on the same kernel as the build host. QEMU does the translation between the the user program and the OS (system calls, signals, etc). So the start-up overhead is pretty minimal. > Nevertheless, if someone is interested in going down this road to get > CDS generation for cross-compilation, I would not object to putting it > in the build system. I filed https://bugs.openjdk.java.net/browse/JDK-8276791 in case there are any volunteers :-) Thanks - Ioi > > /Magnus >> >> Regarding problems, different page size for linux-aarch64 first comes >> to mind, >> but it should be fixed by JDK-8268396: "CDS archive with 4K alignment >> unusable >> on machines with 64k pages" >> >> Thanks, >> Anton >> >> [1] https://www.qemu.org/docs/master/user/main.html >> [2] >> https://github.com/openjdk/jdk/blob/master/.github/workflows/submit.yml#L514 >> >> >> >> On 10/29/21 14:35, Magnus Ihse Bursie wrote: >>> On 2021-10-28 22:56, Ioi Lam wrote: >>>> How reliable would it be to use qemu to run the cross-compiled >>>> binaries? Has anyone tried that recently? >>> I experimented with qemu and the JDK build some years ago (so not >>> really recently). As I remember it, the takeaway was that it >>> probably was reliable, but it was slow as h*ll, up to the point of >>> being practically unusable. >>> >>> Add to this the fact that you need to prepare an entire OS image, >>> complete with all tools needed to build the JDK... I set up a few >>> such images for my own use (I think it was emulating linux-aarch64 >>> on x64) to run testing. I had an idea of describing how I did to >>> share the knowledge, but in the end it was just too complicated and >>> slow to even consider recommending. >>> >>> /Magnus >>>> >>>> >>>> On 10/23/21 5:48 AM, Thomas St?fe wrote: >>>>> Hi Alan, >>>>> >>>>> On Sat, Oct 23, 2021 at 9:58 AM Alan Bateman >>>>> >>>>> wrote: >>>>> >>>>>> >>>>>> On 23/10/2021 07:57, Thomas St?fe wrote: >>>>>>> Hi, >>>>>>> >>>>>>> when I crossbuild (for linux aarch64, using a devkit, building >>>>>>> on linux >>>>>>> x64), for some reason I don't >>>>>>> get the classes.jsa generated inside the images directory. >>>>>>> >>>>>>> My configure options: >>>>>>> >>>>>>> >>>>>> --with-devkit=/shared/projects/openjdk/devkits/x86_64-linux-gnu-to-aarch64-linux-gnu >>>>>> >>>>>>> --openjdk-target=aarch64-linux-gnu >>>>>>> --with-boot-jdk=/shared/projects/openjdk/jdks/sapmachine17 >>>>>>> >>>>>> --with-build-jdk=/shared/projects/openjdk/jdk-jdk/output-release/images/jdk >>>>>> >>>>>>> --with-gtest=/shared/projects/openjdk/gtest/googletest >>>>>>> --with-debug-level=fastdebug >>>>>>> >>>>>>> The build jdk is a freshly build x64 release VM from the same >>>>>>> source >>>>>> tree. >>>>>>> Am I missing something obvious? Is CDS archive generation even >>>>>>> supported >>>>>>> for crossbuilds? >>>>>> It needs the generate run-time to execute "java -Xshare:dump" so >>>>>> I don't >>>>>> expect so. hotspot-runtime-dev is probably the place to discuss the >>>>>> details. BTW: this came up recently in the context of the jlink >>>>>> plugin >>>>>> that generates the CDS archive. The plugin needed a check to >>>>>> ensure that >>>>>> the target platform matched the current platform as it could >>>>>> launch the >>>>>> target VM to create the dump. >>>>>> >>>>>> >>>>> Thinking for a second, probably it cannot work since we copy binary >>>>> structures verbatim to the archive; I guess the chance that they >>>>> are binary >>>>> compatible between platforms is very small. But it should be easily >>>>> rectified by calling Xshare:dump on the target platform. >>>>> >>>>> Thank you! >>>>> >>>>> ..Thomas >>>>> >>>>> >>>>>> -Alan >>>>>> >>>> >>> > From ihse at openjdk.java.net Mon Nov 8 10:27:38 2021 From: ihse at openjdk.java.net (Magnus Ihse Bursie) Date: Mon, 8 Nov 2021 10:27:38 GMT Subject: RFR: 8276654: element-list order is non deterministic [v3] In-Reply-To: References: <7KhnvJgG1xTjKrC6S0Sw9HBZtmGHS8cIyNkhN1nN5og=.41822512-c987-40a9-aa4a-077e491b9dd2@github.com> Message-ID: On Sun, 7 Nov 2021 11:55:19 GMT, Andrew Leonard wrote: >> Can someone please verify that the contents of these files after the fix is the same as the contents of the files for "good" builds before the fix? > >> Can someone please verify that the contents of these files after the fix is the same as the contents of the files for "good" builds before the fix? > > @jonathan-gibbons done, all good @andrew-m-leonard Please refrain from force-pushing to a PR. It makes the evolution of the patch hard to follow for reviewers. I thought I had commented on the commas, but I can't find it now so maybe I failed to press the Comment button. So let me do it again. Can you verify if it is possible to add whitespace after the commas in the "sequence" call? Normally, we use whitespace after all commas (except where impossible, like for built-in make functions). If that does not work, we need to fix the "sequence" method so it accepts whitespace after commas. Otherwise this looks good! ------------- PR: https://git.openjdk.java.net/jdk/pull/6278 From aleonard at openjdk.java.net Mon Nov 8 12:41:01 2021 From: aleonard at openjdk.java.net (Andrew Leonard) Date: Mon, 8 Nov 2021 12:41:01 GMT Subject: RFR: 8276654: element-list order is non deterministic [v4] In-Reply-To: References: Message-ID: <0Xc9YREsjdI5eZNzR6Dos3IC8GSlm92EySYQQCpI3ZI=.0b432dbd-5069-46df-ab12-30a5fc24fa59@github.com> > Fixes: https://bugs.openjdk.java.net/browse/JDK-8276654 > > A intermittent problem with the make dependencies means the jdk.javadoc element-list-.txt generation can remove the src defined element|package-list-<7,8,9,10>.txt files. > Recreatable by using --with-jobs=1 causing jdk.javadoc "gendata" to always occur after "java" module build dependency. > > Signed-off-by: Andrew Leonard Andrew Leonard has updated the pull request incrementally with one additional commit since the last revision: 8276654: element-list order is non deterministic Signed-off-by: Andrew Leonard ------------- Changes: - all: https://git.openjdk.java.net/jdk/pull/6278/files - new: https://git.openjdk.java.net/jdk/pull/6278/files/9d7b338c..a7474d01 Webrevs: - full: https://webrevs.openjdk.java.net/?repo=jdk&pr=6278&range=03 - incr: https://webrevs.openjdk.java.net/?repo=jdk&pr=6278&range=02-03 Stats: 2 lines in 1 file changed: 1 ins; 0 del; 1 mod Patch: https://git.openjdk.java.net/jdk/pull/6278.diff Fetch: git fetch https://git.openjdk.java.net/jdk pull/6278/head:pull/6278 PR: https://git.openjdk.java.net/jdk/pull/6278 From aleonard at openjdk.java.net Mon Nov 8 12:41:01 2021 From: aleonard at openjdk.java.net (Andrew Leonard) Date: Mon, 8 Nov 2021 12:41:01 GMT Subject: RFR: 8276654: element-list order is non deterministic [v3] In-Reply-To: References: <7KhnvJgG1xTjKrC6S0Sw9HBZtmGHS8cIyNkhN1nN5og=.41822512-c987-40a9-aa4a-077e491b9dd2@github.com> Message-ID: On Sun, 7 Nov 2021 11:55:19 GMT, Andrew Leonard wrote: >> Can someone please verify that the contents of these files after the fix is the same as the contents of the files for "good" builds before the fix? > >> Can someone please verify that the contents of these files after the fix is the same as the contents of the files for "good" builds before the fix? > > @jonathan-gibbons done, all good > @andrew-m-leonard Please refrain from force-pushing to a PR. It makes the evolution of the patch hard to follow for reviewers. > > I thought I had commented on the commas, but I can't find it now so maybe I failed to press the Comment button. So let me do it again. > > Can you verify if it is possible to add whitespace after the commas in the "sequence" call? Normally, we use whitespace after all commas (except where impossible, like for built-in make functions). If that does not work, we need to fix the "sequence" method so it accepts whitespace after commas. > > Otherwise this looks good! @magicus Yes splits fine, updated... thanks ------------- PR: https://git.openjdk.java.net/jdk/pull/6278 From erik.joelsson at oracle.com Mon Nov 8 13:30:08 2021 From: erik.joelsson at oracle.com (erik.joelsson at oracle.com) Date: Mon, 8 Nov 2021 05:30:08 -0800 Subject: [External] : Re: AWT & Swing tests failed: Function frag_col has a deployment target which is incompatible with this OS. In-Reply-To: References: <8A9D3B5B-67A1-4FBE-910B-0CDC00682655@jetbrains.com> <090A5816-8F67-493D-8B00-DA20AC3C4CB5@jetbrains.com> <6E19D881-5C6A-4AF6-8A1A-EFAFAF694D1E@oracle.com> <5b12e5e3-8246-d652-70f1-3d66ed6f4b99@oracle.com> Message-ID: In that case, feel free to report a bug. /Erik On 2021-11-07 16:36, Vitaly Provodin wrote: > Actually adding the parameter --with-macosx-version-max does not resolve the issue. Accidentaly, the tests were executed on machines, where Xcode 12.2 were installed, and passed. But they failed on machines with Xcode 11.X > After removing Xcode at all, the tests stably failed with the "Function frag_col has a deployment target which is incompatible with this OS" diagnostic. > In all cases above JDK was built with --with-macosx-version-max=10.12.00 > > Thanks, > Vitaly > > >> On 29 Oct 2021, at 13:33, Vitaly Provodin wrote: >> >> Erik, >> >> the option --with-macosx-version-max=10.12.00 resolves the issue. >> >> thank you, >> Vitaly >> >>> On 28 Oct 2021, at 00:09, erik.joelsson at oracle.com wrote: >>> >>> As far as I know, this is expected. You need to tell Xcode to compile and link with an older deployment target. I'm a bit fuzzy on the details, but as I remember it, by default, configure only sets up half the required settings for your to produce binaries that will work on Macos versions older than the build environment. At Oracle we also set "--with-macosx-version-max=10.12.00" which I believe is also needed. >>> >>> No need to file an issue in my opinion. >>> >>> /Erik >>> >>> On 2021-10-27 09:55, Philip Race wrote: >>>> A bug should be filed but if it is specific to using xcode 12.5 we won't have seen >>>> any issue yet at Oracle as all official builds use xcode 12.4. >>>> >>>> However at some point (JDK 19) we'll likely upgrade and then it'll be an issue. >>>> >>>> -phil. >>>> >>>> On 10/27/21 3:02 AM, Jayathirth D V wrote: >>>>> Hi Vitaly, >>>>> >>>>> There is no issue in JBS with this failure. >>>>> We had lanai CI jtreg/JCK test run recently on 23rd October and many of those runs were on macOS 10.15.7 and macOS 11.6 systems but I don?t see any compilation error reported. >>>>> >>>>> Also from below mail I am not able to get whether issue is related to some Xcode version or macOS version. Please clarify. >>>>> >>>>> Thanks, >>>>> Jay >>>>> >>>>>> On 27-Oct-2021, at 3:04 PM, Vitaly Provodin wrote: >>>>>> >>>>>> Sorry did not complete the previous mail >>>>>> >>>>>> >>>>>> If JDK is built on Catalina with Xcode 12.2, the tests passed well. >>>>>> >>>>>> >>>>>> >>>>>>> On 27 Oct 2021, at 16:32, Vitaly Provodin wrote: >>>>>>> >>>>>>> Hi all, >>>>>>> >>>>>>> it was found out that a lot of AWT & Swing tests failed on macOS-x64 <=10.15 with the following message >>>>>>> >>>>>>> >>>>>>> 2021-10-25 05:36:42.170 java[6916:107817] Failed to create pipeline state, error Error Domain=CompilerError Code=1 "Function frag_col has a deployment target which is incompatible with this OS." UserInfo={NSLocalizedDescription=Function frag_col has a deployment target which is incompatible with this OS.} >>>>>>> >>>>>>> >>>>>>> The issue was observed when JDK17, JDK18 was built on BigSur with Xcode12.5 >>>>>>> 1. -Dsun.java2d.metal=true & macOS-x64 <=10.15 FAILED >>>>>>> 2. -Dsun.java2d.metal=true & macOS-x64 >10.15 PASSED >>>>>>> 2. -Dsun.java2d.metal=false & macOS-x64 <=10.15 PASSED >>>>>>> 2. -Dsun.java2d.metal=false & macOS-x64 > 10.15 PASSED >>>>>>> >>>>>>> If JDK is build on Catalina with Xcode12.2 >>>>>>> >>>>>>> Not sure if this issue is known. Shoiuld it be reported to JBS? >>>>>>> >>>>>>> Thanks, >>>>>>> Vitaly >>>>>>> From ihse at openjdk.java.net Mon Nov 8 13:32:35 2021 From: ihse at openjdk.java.net (Magnus Ihse Bursie) Date: Mon, 8 Nov 2021 13:32:35 GMT Subject: RFR: 8276654: element-list order is non deterministic [v4] In-Reply-To: <0Xc9YREsjdI5eZNzR6Dos3IC8GSlm92EySYQQCpI3ZI=.0b432dbd-5069-46df-ab12-30a5fc24fa59@github.com> References: <0Xc9YREsjdI5eZNzR6Dos3IC8GSlm92EySYQQCpI3ZI=.0b432dbd-5069-46df-ab12-30a5fc24fa59@github.com> Message-ID: On Mon, 8 Nov 2021 12:41:01 GMT, Andrew Leonard wrote: >> Fixes: https://bugs.openjdk.java.net/browse/JDK-8276654 >> >> A intermittent problem with the make dependencies means the jdk.javadoc element-list-.txt generation can remove the src defined element|package-list-<7,8,9,10>.txt files. >> Recreatable by using --with-jobs=1 causing jdk.javadoc "gendata" to always occur after "java" module build dependency. >> >> Signed-off-by: Andrew Leonard > > Andrew Leonard has updated the pull request incrementally with one additional commit since the last revision: > > 8276654: element-list order is non deterministic > > Signed-off-by: Andrew Leonard make/modules/jdk.javadoc/Gendata.gmk line 76: > 74: $(call LogInfo, Creating javadoc element lists) > 75: $(RM) $(ELEMENT_LISTS_DIR)/element-list-{$(call CommaList, \ > 76: $(call sequence,$(GENERATE_SYMBOLS_FROM_JDK_VERSION), \ Suggestion: $(call sequence, $(GENERATE_SYMBOLS_FROM_JDK_VERSION), \ Suggestion: $(call sequence,$(GENERATE_SYMBOLS_FROM_JDK_VERSION), \ ------------- PR: https://git.openjdk.java.net/jdk/pull/6278 From Alan.Bateman at oracle.com Mon Nov 8 13:51:08 2021 From: Alan.Bateman at oracle.com (Alan Bateman) Date: Mon, 8 Nov 2021 13:51:08 +0000 Subject: RFR: 8276400: openjdk image Jars, Zips and Jmods built from the same source are not reproducible In-Reply-To: References: <5UDNFhfUqwOsOgcV2wEBWHZKU84d5w6Cc5JZQ-XEITk=.e4431826-178a-4c33-8cf4-b5a6e83016e5@github.com> Message-ID: <2ffd51f1-c0a4-6bf2-e191-2681292837b0@oracle.com> On 05/11/2021 19:15, Andrew Leonard wrote: > : > @AlanBateman @magicus thank you both for your guidance. I have now split this bug into the 3 mentioned: > - GenerateZip: https://bugs.openjdk.java.net/browse/JDK-8276743 > - Jar/Jmod content ordering: https://bugs.openjdk.java.net/browse/JDK-8276764 > - Jar/Jmod/ZipOutputStream timestamp option: https://bugs.openjdk.java.net/browse/JDK-8276766 **(requires CSR)** > > Closing this PR to be replaced with 3 new PRs for the above bugs. > Thanks this seems a reasonable plan. When you get the 3rd item then we'll need to discuss whether ZipOutputStream.putEntry overrides (or not) the time stamps of entries that have a modification time. -Alan From aleonard at openjdk.java.net Mon Nov 8 14:17:56 2021 From: aleonard at openjdk.java.net (Andrew Leonard) Date: Mon, 8 Nov 2021 14:17:56 GMT Subject: RFR: 8276654: element-list order is non deterministic [v5] In-Reply-To: References: Message-ID: > Fixes: https://bugs.openjdk.java.net/browse/JDK-8276654 > > A intermittent problem with the make dependencies means the jdk.javadoc element-list-.txt generation can remove the src defined element|package-list-<7,8,9,10>.txt files. > Recreatable by using --with-jobs=1 causing jdk.javadoc "gendata" to always occur after "java" module build dependency. > > Signed-off-by: Andrew Leonard Andrew Leonard has updated the pull request incrementally with one additional commit since the last revision: 8276654: element-list order is non deterministic Signed-off-by: Andrew Leonard ------------- Changes: - all: https://git.openjdk.java.net/jdk/pull/6278/files - new: https://git.openjdk.java.net/jdk/pull/6278/files/a7474d01..85056a09 Webrevs: - full: https://webrevs.openjdk.java.net/?repo=jdk&pr=6278&range=04 - incr: https://webrevs.openjdk.java.net/?repo=jdk&pr=6278&range=03-04 Stats: 2 lines in 1 file changed: 0 ins; 0 del; 2 mod Patch: https://git.openjdk.java.net/jdk/pull/6278.diff Fetch: git fetch https://git.openjdk.java.net/jdk pull/6278/head:pull/6278 PR: https://git.openjdk.java.net/jdk/pull/6278 From aleonard at openjdk.java.net Mon Nov 8 14:17:57 2021 From: aleonard at openjdk.java.net (Andrew Leonard) Date: Mon, 8 Nov 2021 14:17:57 GMT Subject: RFR: 8276654: element-list order is non deterministic [v4] In-Reply-To: References: <0Xc9YREsjdI5eZNzR6Dos3IC8GSlm92EySYQQCpI3ZI=.0b432dbd-5069-46df-ab12-30a5fc24fa59@github.com> Message-ID: On Mon, 8 Nov 2021 13:29:57 GMT, Magnus Ihse Bursie wrote: >> Andrew Leonard has updated the pull request incrementally with one additional commit since the last revision: >> >> 8276654: element-list order is non deterministic >> >> Signed-off-by: Andrew Leonard > > make/modules/jdk.javadoc/Gendata.gmk line 76: > >> 74: $(call LogInfo, Creating javadoc element lists) >> 75: $(RM) $(ELEMENT_LISTS_DIR)/element-list-{$(call CommaList, \ >> 76: $(call sequence,$(GENERATE_SYMBOLS_FROM_JDK_VERSION), \ > > Suggestion: > > $(call sequence, $(GENERATE_SYMBOLS_FROM_JDK_VERSION), \ done ------------- PR: https://git.openjdk.java.net/jdk/pull/6278 From ihse at openjdk.java.net Mon Nov 8 18:57:32 2021 From: ihse at openjdk.java.net (Magnus Ihse Bursie) Date: Mon, 8 Nov 2021 18:57:32 GMT Subject: RFR: 8276654: element-list order is non deterministic [v5] In-Reply-To: References: Message-ID: <2mrv4rVyHmt7ucn_qmh8sb15QplpFcc2-SdZT4OkpEk=.aeaaf07b-fdbe-4c3b-89dc-a16e134c01e7@github.com> On Mon, 8 Nov 2021 14:17:56 GMT, Andrew Leonard wrote: >> Fixes: https://bugs.openjdk.java.net/browse/JDK-8276654 >> >> A intermittent problem with the make dependencies means the jdk.javadoc element-list-.txt generation can remove the src defined element|package-list-<7,8,9,10>.txt files. >> Recreatable by using --with-jobs=1 causing jdk.javadoc "gendata" to always occur after "java" module build dependency. >> >> Signed-off-by: Andrew Leonard > > Andrew Leonard has updated the pull request incrementally with one additional commit since the last revision: > > 8276654: element-list order is non deterministic > > Signed-off-by: Andrew Leonard Marked as reviewed by ihse (Reviewer). ------------- PR: https://git.openjdk.java.net/jdk/pull/6278 From aleonard at openjdk.java.net Mon Nov 8 20:40:37 2021 From: aleonard at openjdk.java.net (Andrew Leonard) Date: Mon, 8 Nov 2021 20:40:37 GMT Subject: Integrated: 8276654: element-list order is non deterministic In-Reply-To: References: Message-ID: On Fri, 5 Nov 2021 15:26:40 GMT, Andrew Leonard wrote: > Fixes: https://bugs.openjdk.java.net/browse/JDK-8276654 > > A intermittent problem with the make dependencies means the jdk.javadoc element-list-.txt generation can remove the src defined element|package-list-<7,8,9,10>.txt files. > Recreatable by using --with-jobs=1 causing jdk.javadoc "gendata" to always occur after "java" module build dependency. > > Signed-off-by: Andrew Leonard This pull request has now been integrated. Changeset: 14d66bd4 Author: Andrew Leonard URL: https://git.openjdk.java.net/jdk/commit/14d66bd438dfa1feeafaca39be8f79a91e2968e9 Stats: 3 lines in 1 file changed: 2 ins; 0 del; 1 mod 8276654: element-list order is non deterministic Reviewed-by: ihse ------------- PR: https://git.openjdk.java.net/jdk/pull/6278 From ysuenaga at openjdk.java.net Tue Nov 9 09:36:50 2021 From: ysuenaga at openjdk.java.net (Yasumasa Suenaga) Date: Tue, 9 Nov 2021 09:36:50 GMT Subject: RFR: 8276841: Add support for Visual Studio 2022 Message-ID: Now OpenJDK supports VS 2017 and 2019 for building. VS 2022 has been released in Nov. 2021, so I want to add VS 2022 to build system for Windows. ------------- Commit messages: - 8276841: Add support for Visual Studio 2022 Changes: https://git.openjdk.java.net/jdk/pull/6304/files Webrev: https://webrevs.openjdk.java.net/?repo=jdk&pr=6304&range=00 Issue: https://bugs.openjdk.java.net/browse/JDK-8276841 Stats: 17 lines in 1 file changed: 15 ins; 0 del; 2 mod Patch: https://git.openjdk.java.net/jdk/pull/6304.diff Fetch: git fetch https://git.openjdk.java.net/jdk pull/6304/head:pull/6304 PR: https://git.openjdk.java.net/jdk/pull/6304 From aleonard at openjdk.java.net Tue Nov 9 13:10:55 2021 From: aleonard at openjdk.java.net (Andrew Leonard) Date: Tue, 9 Nov 2021 13:10:55 GMT Subject: RFR: 8276743: Make openjdk build Zip Archive generation "reproducible" Message-ID: <13DpyL4de49GOSlp9DU1co_rkznpvX920O6Z0RsmjbY=.23225459-16de-459d-b361-884d8b1d8d35@github.com> This PR adds a new openjdk build tool GenerateZip, which generates a final "zip" file from an input folder, and creates it in a deterministic way, ensuring ordering and timestamps are set as specified. Using this tool in ZipArchive.gmk will ensure src.zip is then created deterministically. Signed-off-by: Andrew Leonard ------------- Commit messages: - 8276743: Make openjdk build Zip Archive generation "reproducible" Changes: https://git.openjdk.java.net/jdk/pull/6311/files Webrev: https://webrevs.openjdk.java.net/?repo=jdk&pr=6311&range=00 Issue: https://bugs.openjdk.java.net/browse/JDK-8276743 Stats: 303 lines in 3 files changed: 300 ins; 0 del; 3 mod Patch: https://git.openjdk.java.net/jdk/pull/6311.diff Fetch: git fetch https://git.openjdk.java.net/jdk pull/6311/head:pull/6311 PR: https://git.openjdk.java.net/jdk/pull/6311 From erikj at openjdk.java.net Tue Nov 9 13:30:40 2021 From: erikj at openjdk.java.net (Erik Joelsson) Date: Tue, 9 Nov 2021 13:30:40 GMT Subject: RFR: 8276841: Add support for Visual Studio 2022 In-Reply-To: References: Message-ID: <9Jc6wlgEsi7S07_kqnbmjz-GaSThMpGOtqeQEz-yClw=.8d65f68e-2470-4464-be55-6b430263ede1@github.com> On Tue, 9 Nov 2021 06:36:53 GMT, Yasumasa Suenaga wrote: > Now OpenJDK supports VS 2017 and 2019 for building. > VS 2022 has been released in Nov. 2021, so I want to add VS 2022 to build system for Windows. Marked as reviewed by erikj (Reviewer). ------------- PR: https://git.openjdk.java.net/jdk/pull/6304 From ihse at openjdk.java.net Tue Nov 9 13:39:38 2021 From: ihse at openjdk.java.net (Magnus Ihse Bursie) Date: Tue, 9 Nov 2021 13:39:38 GMT Subject: RFR: 8276841: Add support for Visual Studio 2022 In-Reply-To: References: Message-ID: On Tue, 9 Nov 2021 06:36:53 GMT, Yasumasa Suenaga wrote: > Now OpenJDK supports VS 2017 and 2019 for building. > VS 2022 has been released in Nov. 2021, so I want to add VS 2022 to build system for Windows. Marked as reviewed by ihse (Reviewer). ------------- PR: https://git.openjdk.java.net/jdk/pull/6304 From erikj at openjdk.java.net Tue Nov 9 14:07:39 2021 From: erikj at openjdk.java.net (Erik Joelsson) Date: Tue, 9 Nov 2021 14:07:39 GMT Subject: RFR: 8276743: Make openjdk build Zip Archive generation "reproducible" In-Reply-To: <13DpyL4de49GOSlp9DU1co_rkznpvX920O6Z0RsmjbY=.23225459-16de-459d-b361-884d8b1d8d35@github.com> References: <13DpyL4de49GOSlp9DU1co_rkznpvX920O6Z0RsmjbY=.23225459-16de-459d-b361-884d8b1d8d35@github.com> Message-ID: <8SDOXwz8OhlhhqpWHjDbdxj9M1ZQfQcls2aRNt-cukQ=.f177d1e3-b30e-4e8b-ae35-d27606e18a5e@github.com> On Tue, 9 Nov 2021 12:59:17 GMT, Andrew Leonard wrote: > This PR adds a new openjdk build tool GenerateZip, which generates a final "zip" file from an input folder, and creates it in a deterministic way, ensuring ordering and timestamps are set as specified. > > Using this tool in ZipArchive.gmk will ensure src.zip is then created deterministically. > > Signed-off-by: Andrew Leonard I think it would it make sense add a configure parameter to to enable/disable this new functionality. I think developers would want to opt out of extra unnecessary build steps when building and testing locally. Not sure what the default should be. For other reproducible measures, we require active enabling today. Maybe default on for release builds and off for debug builds. make/common/ZipArchive.gmk line 29: > 27: _ZIP_ARCHIVE_GMK := 1 > 28: > 29: include ../ToolsJdk.gmk Should probably add a comment about inclusion of this file now requires an explicit dependency on build-tools in Main.gmk. make/common/ZipArchive.gmk line 178: > 176: (cd $$(SUPPORT_OUTPUTDIR)/ziptmp/$1/files && \ > 177: $(RM) $$@ && \ > 178: $(UNZIP) -q $$(SUPPORT_OUTPUTDIR)/ziptmp/$1/tmp.zip && \ Having to explode the zip here is unfortunate. This means we are creating an almost full copy of the whole src tree in the build directory, something I tried to avoid by leveraging the include/exclude functionality of zip, instead of generating make rules for copying the files I wanted into a source tree to run zip on. This may be a small overhead on Linux, but I'm pretty sure it will be very noticeable on Windows. When reading about your tool at first, I assumed it would read the intermediate zip file directly when rebuilding the zip. I don't think modifying it to do that would be too complicated, basically read and processing ZipEntrys instead of walking the file system. make/jdk/src/classes/build/tools/generatezip/GenerateZip.java line 161: > 159: boolean pathIsDir = fpath.isDirectory(); > 160: > 161: // Keep a sorted Set of files to be processed, so that the Jmod is reproducible Aren't we generating zip files rather than jmod files here? ------------- PR: https://git.openjdk.java.net/jdk/pull/6311 From ihse at openjdk.java.net Tue Nov 9 14:30:39 2021 From: ihse at openjdk.java.net (Magnus Ihse Bursie) Date: Tue, 9 Nov 2021 14:30:39 GMT Subject: RFR: 8276672: Cannot build hsdis on WSL In-Reply-To: <4lLdxiPsFsB1GfI7iWSgLOXQ-Q_WU4ZZdOXHkRuO0kc=.45eea799-3e3d-4b1c-90dc-a6fa5e3d5b74@github.com> References: <4lLdxiPsFsB1GfI7iWSgLOXQ-Q_WU4ZZdOXHkRuO0kc=.45eea799-3e3d-4b1c-90dc-a6fa5e3d5b74@github.com> Message-ID: On Fri, 5 Nov 2021 03:03:55 GMT, Yasumasa Suenaga wrote: > JDK-8275128 introduced new Makefile for hsdis to integrate it to normal build system, however it does not work on WSL 1 Ubuntu 20.04 and MinGW from Ubuntu. > > Hsdis.gmk has two problems: > > 1. MinGW version is fixed to "9.2.0" > 2. Assumes sysroot for MinGW is located at /usr/$MINGW_BASE/sys-root > > I tested this change on WSL only, so it is appreciate to review on Cygwin. Hi @YaSuenag, Unfortunately this patch does not work out-of-the box on Cygwin, at least not on my machine. :-( I found two issues: 1) The libraries are in $(MINGW_SYSROOT)/mingw/lib, not $(MINGW_SYSROOT)/lib. For `-L`, we can of course include both paths just to be safe rather than sorry, but that will not work for dllcrt2.o, so we need to verify which path works. 2) On the machine I'm testing, mingw 11.0.4 is installed, but MINGW_VERSION should need to be just 11. (It's worth noting that my original implementation would not have worked either, in this case.) So here too I think we need to check both possibilities and verify them. Also, there's no need to use eval to assign to variables. You can just use normal assignment :=. I'll try to make this work for cygwin in a branch in my personal fork. I'll let you know when I'm done and then you can test if it works for you. ------------- PR: https://git.openjdk.java.net/jdk/pull/6269 From aleonard at openjdk.java.net Tue Nov 9 14:37:32 2021 From: aleonard at openjdk.java.net (Andrew Leonard) Date: Tue, 9 Nov 2021 14:37:32 GMT Subject: RFR: 8276743: Make openjdk build Zip Archive generation "reproducible" In-Reply-To: <8SDOXwz8OhlhhqpWHjDbdxj9M1ZQfQcls2aRNt-cukQ=.f177d1e3-b30e-4e8b-ae35-d27606e18a5e@github.com> References: <13DpyL4de49GOSlp9DU1co_rkznpvX920O6Z0RsmjbY=.23225459-16de-459d-b361-884d8b1d8d35@github.com> <8SDOXwz8OhlhhqpWHjDbdxj9M1ZQfQcls2aRNt-cukQ=.f177d1e3-b30e-4e8b-ae35-d27606e18a5e@github.com> Message-ID: On Tue, 9 Nov 2021 13:58:24 GMT, Erik Joelsson wrote: >> This PR adds a new openjdk build tool GenerateZip, which generates a final "zip" file from an input folder, and creates it in a deterministic way, ensuring ordering and timestamps are set as specified. >> >> Using this tool in ZipArchive.gmk will ensure src.zip is then created deterministically. >> >> Signed-off-by: Andrew Leonard > > make/common/ZipArchive.gmk line 178: > >> 176: (cd $$(SUPPORT_OUTPUTDIR)/ziptmp/$1/files && \ >> 177: $(RM) $$@ && \ >> 178: $(UNZIP) -q $$(SUPPORT_OUTPUTDIR)/ziptmp/$1/tmp.zip && \ > > Having to explode the zip here is unfortunate. This means we are creating an almost full copy of the whole src tree in the build directory, something I tried to avoid by leveraging the include/exclude functionality of zip, instead of generating make rules for copying the files I wanted into a source tree to run zip on. This may be a small overhead on Linux, but I'm pretty sure it will be very noticeable on Windows. > > When reading about your tool at first, I assumed it would read the intermediate zip file directly when rebuilding the zip. I don't think modifying it to do that would be too complicated, basically read and processing ZipEntrys instead of walking the file system. @erikj79 sure, I can look at doing that > make/jdk/src/classes/build/tools/generatezip/GenerateZip.java line 161: > >> 159: boolean pathIsDir = fpath.isDirectory(); >> 160: >> 161: // Keep a sorted Set of files to be processed, so that the Jmod is reproducible > > Aren't we generating zip files rather than jmod files here? copy&paste error... ------------- PR: https://git.openjdk.java.net/jdk/pull/6311 From aleonard at openjdk.java.net Tue Nov 9 14:40:37 2021 From: aleonard at openjdk.java.net (Andrew Leonard) Date: Tue, 9 Nov 2021 14:40:37 GMT Subject: RFR: 8276743: Make openjdk build Zip Archive generation "reproducible" In-Reply-To: <8SDOXwz8OhlhhqpWHjDbdxj9M1ZQfQcls2aRNt-cukQ=.f177d1e3-b30e-4e8b-ae35-d27606e18a5e@github.com> References: <13DpyL4de49GOSlp9DU1co_rkznpvX920O6Z0RsmjbY=.23225459-16de-459d-b361-884d8b1d8d35@github.com> <8SDOXwz8OhlhhqpWHjDbdxj9M1ZQfQcls2aRNt-cukQ=.f177d1e3-b30e-4e8b-ae35-d27606e18a5e@github.com> Message-ID: On Tue, 9 Nov 2021 14:04:27 GMT, Erik Joelsson wrote: > I think it would it make sense add a configure parameter to to enable/disable this new functionality. I think developers would want to opt out of extra unnecessary build steps when building and testing locally. Not sure what the default should be. For other reproducible measures, we require active enabling today. Maybe default on for release builds and off for debug builds. @erikj79 so I was hoping to not add another configure arg, when the intentions of this should just all be positive, ordered&deterministic. Also @magicus I get the impression from your comments that we don't really want to end up with in order to build reproducibly you have to specify all these options x,y,z,... Maybe we should just have one big flag --with-reproducible-build ? ------------- PR: https://git.openjdk.java.net/jdk/pull/6311 From ysuenaga at openjdk.java.net Tue Nov 9 14:58:40 2021 From: ysuenaga at openjdk.java.net (Yasumasa Suenaga) Date: Tue, 9 Nov 2021 14:58:40 GMT Subject: Integrated: 8276841: Add support for Visual Studio 2022 In-Reply-To: References: Message-ID: On Tue, 9 Nov 2021 06:36:53 GMT, Yasumasa Suenaga wrote: > Now OpenJDK supports VS 2017 and 2019 for building. > VS 2022 has been released in Nov. 2021, so I want to add VS 2022 to build system for Windows. This pull request has now been integrated. Changeset: f65db88b Author: Yasumasa Suenaga URL: https://git.openjdk.java.net/jdk/commit/f65db88b74911e5896d2ff536c4ac97e7f62d98b Stats: 17 lines in 1 file changed: 15 ins; 0 del; 2 mod 8276841: Add support for Visual Studio 2022 Reviewed-by: erikj, ihse ------------- PR: https://git.openjdk.java.net/jdk/pull/6304 From ihse at openjdk.java.net Tue Nov 9 15:03:37 2021 From: ihse at openjdk.java.net (Magnus Ihse Bursie) Date: Tue, 9 Nov 2021 15:03:37 GMT Subject: RFR: 8276743: Make openjdk build Zip Archive generation "reproducible" In-Reply-To: <13DpyL4de49GOSlp9DU1co_rkznpvX920O6Z0RsmjbY=.23225459-16de-459d-b361-884d8b1d8d35@github.com> References: <13DpyL4de49GOSlp9DU1co_rkznpvX920O6Z0RsmjbY=.23225459-16de-459d-b361-884d8b1d8d35@github.com> Message-ID: On Tue, 9 Nov 2021 12:59:17 GMT, Andrew Leonard wrote: > This PR adds a new openjdk build tool GenerateZip, which generates a final "zip" file from an input folder, and creates it in a deterministic way, ensuring ordering and timestamps are set as specified. > > Using this tool in ZipArchive.gmk will ensure src.zip is then created deterministically. > > Signed-off-by: Andrew Leonard We already have an --enable-reproducible-builds. If (and I say if) we need to turn this on/off with a flag, this would to fine. However, as I have said previously in a private discussion with Andrew, I prefer it if we can make reproducible builds so cheap, reliable and uncontroversial so we can always to reproducible builds, and remove that flag. For this case, I think it depends on two things: 1) the extra time to make the zip file reproducible. Some benchmarking on the GenerateZip for src.zip would be good to have, at least for some ballpark figures. 2) When is src.zip built? (I don't remember right now) If it is part of the exploded-image, then it is really time sensitive (and should maybe move out of that target). If it part of the jdk-image, I'd say it is not as sensitive, due to a) this is very much slower anyway, and b) that part is much more parallelizible, so src.zip can be produced while waiting for jlink or whatever. Also, the name is not really a correct description at this point. Maybe a straight `MakeZipReproducible` instead or something like that? And have you verified that running with `zip -X` is not enough? I remember checking into this before, but I don't remember the conclusions. The purpose is to leave out some extra information, such as time stamps, but it might not be enough to guarantee reproducible builds. ------------- PR: https://git.openjdk.java.net/jdk/pull/6311 From aleonard at openjdk.java.net Tue Nov 9 15:03:38 2021 From: aleonard at openjdk.java.net (Andrew Leonard) Date: Tue, 9 Nov 2021 15:03:38 GMT Subject: RFR: 8276743: Make openjdk build Zip Archive generation "reproducible" In-Reply-To: <8SDOXwz8OhlhhqpWHjDbdxj9M1ZQfQcls2aRNt-cukQ=.f177d1e3-b30e-4e8b-ae35-d27606e18a5e@github.com> References: <13DpyL4de49GOSlp9DU1co_rkznpvX920O6Z0RsmjbY=.23225459-16de-459d-b361-884d8b1d8d35@github.com> <8SDOXwz8OhlhhqpWHjDbdxj9M1ZQfQcls2aRNt-cukQ=.f177d1e3-b30e-4e8b-ae35-d27606e18a5e@github.com> Message-ID: On Tue, 9 Nov 2021 13:58:24 GMT, Erik Joelsson wrote: >> This PR adds a new openjdk build tool GenerateZip, which generates a final "zip" file from an input folder, and creates it in a deterministic way, ensuring ordering and timestamps are set as specified. >> >> Using this tool in ZipArchive.gmk will ensure src.zip is then created deterministically. >> >> Signed-off-by: Andrew Leonard > > make/common/ZipArchive.gmk line 178: > >> 176: (cd $$(SUPPORT_OUTPUTDIR)/ziptmp/$1/files && \ >> 177: $(RM) $$@ && \ >> 178: $(UNZIP) -q $$(SUPPORT_OUTPUTDIR)/ziptmp/$1/tmp.zip && \ > > Having to explode the zip here is unfortunate. This means we are creating an almost full copy of the whole src tree in the build directory, something I tried to avoid by leveraging the include/exclude functionality of zip, instead of generating make rules for copying the files I wanted into a source tree to run zip on. This may be a small overhead on Linux, but I'm pretty sure it will be very noticeable on Windows. > > When reading about your tool at first, I assumed it would read the intermediate zip file directly when rebuilding the zip. I don't think modifying it to do that would be too complicated, basically read and processing ZipEntrys instead of walking the file system. @erikj79 so had a bit of a think, and part of the unzipping.. then re-gen'ing is not having to load all the entries into memory. You can't guarantee the order "zip" has created them in, so realistically i'd have to read all the ZipEntry's into memory, then re-write.. which we can do.. src.zip is only 55MB or so, so memory requirements won't be huge given src.zip is the only target here currently. ------------- PR: https://git.openjdk.java.net/jdk/pull/6311 From ihse at openjdk.java.net Tue Nov 9 15:09:40 2021 From: ihse at openjdk.java.net (Magnus Ihse Bursie) Date: Tue, 9 Nov 2021 15:09:40 GMT Subject: RFR: 8276672: Cannot build hsdis on WSL In-Reply-To: <4lLdxiPsFsB1GfI7iWSgLOXQ-Q_WU4ZZdOXHkRuO0kc=.45eea799-3e3d-4b1c-90dc-a6fa5e3d5b74@github.com> References: <4lLdxiPsFsB1GfI7iWSgLOXQ-Q_WU4ZZdOXHkRuO0kc=.45eea799-3e3d-4b1c-90dc-a6fa5e3d5b74@github.com> Message-ID: <0YUNwCXUxPBoqE69W9_YinizmaGWFPpXWHgcka-9iD0=.6b22737f-66f1-47f7-84f5-5c05761e6396@github.com> On Fri, 5 Nov 2021 03:03:55 GMT, Yasumasa Suenaga wrote: > JDK-8275128 introduced new Makefile for hsdis to integrate it to normal build system, however it does not work on WSL 1 Ubuntu 20.04 and MinGW from Ubuntu. > > Hsdis.gmk has two problems: > > 1. MinGW version is fixed to "9.2.0" > 2. Assumes sysroot for MinGW is located at /usr/$MINGW_BASE/sys-root > > I tested this change on WSL only, so it is appreciate to review on Cygwin. Please check out if https://github.com/magicus/jdk/tree/hsdis-on-wsl works OK in your environment. It does work for me on Cygwin. ------------- PR: https://git.openjdk.java.net/jdk/pull/6269 From aleonard at openjdk.java.net Tue Nov 9 15:10:42 2021 From: aleonard at openjdk.java.net (Andrew Leonard) Date: Tue, 9 Nov 2021 15:10:42 GMT Subject: RFR: 8276743: Make openjdk build Zip Archive generation "reproducible" In-Reply-To: References: <13DpyL4de49GOSlp9DU1co_rkznpvX920O6Z0RsmjbY=.23225459-16de-459d-b361-884d8b1d8d35@github.com> Message-ID: On Tue, 9 Nov 2021 15:00:59 GMT, Magnus Ihse Bursie wrote: > And have you verified that running with `zip -X` is not enough? I remember checking into this before, but I don't remember the conclusions. The purpose is to leave out some extra information, such as time stamps, but it might not be enough to guarantee reproducible builds. Correct, I have tried that already, that does remove the extended timestamp information, and as long as the input source is already correctly timestamped it works to a point, that point being the "file ordering" is not-deterministic and I think is dependent on how the OS file queries returns files. From history this is what I tried: https://github.com/andrew-m-leonard/jdk-1/blob/f18b5826b9ae3e7e5750190a669edcfd8bb8b2cc/make/common/ZipArchive.gmk#L166 > Also, the name is not really a correct description at this point. Maybe a straight `MakeZipReproducible` instead or something like that? MakeZipReproducible sounds good. ------------- PR: https://git.openjdk.java.net/jdk/pull/6311 From aleonard at openjdk.java.net Tue Nov 9 15:20:37 2021 From: aleonard at openjdk.java.net (Andrew Leonard) Date: Tue, 9 Nov 2021 15:20:37 GMT Subject: RFR: 8276743: Make openjdk build Zip Archive generation "reproducible" In-Reply-To: References: <13DpyL4de49GOSlp9DU1co_rkznpvX920O6Z0RsmjbY=.23225459-16de-459d-b361-884d8b1d8d35@github.com> Message-ID: On Tue, 9 Nov 2021 14:57:52 GMT, Magnus Ihse Bursie wrote: > We already have an --enable-reproducible-builds. If (and I say if) we need to turn this on/off with a flag, this would to fine. > > However, as I have said previously in a private discussion with Andrew, I prefer it if we can make reproducible builds so cheap, reliable and uncontroversial so we can always to reproducible builds, and remove that flag. > > For this case, I think it depends on two things: > > 1. the extra time to make the zip file reproducible. Some benchmarking on the GenerateZip for src.zip would be good to have, at least for some ballpark figures. > > 2. When is src.zip built? (I don't remember right now) If it is part of the exploded-image, then it is really time sensitive (and should maybe move out of that target). If it part of the jdk-image, I'd say it is not as sensitive, due to > a) this is very much slower anyway, and > b) that part is much more parallelizible, so src.zip can be produced while waiting for jlink or whatever. @magicus 1) I'll do some rough benchmarks 2) "zip-source" is only dependent on "gensrc", ie.all module -gensrc stages must have completed, so in theory it should run in parallel while the linking goes on, a quick scan of a parallel build log seems to confirm that, the "Updating support/src.zip" occurs quite some time before the exploded image linking: 20:34:45 Compiling 31 files for jdk.management.agent 20:34:45 Compiling 16 files for jdk.security.jgss 20:34:45 Compiling 30 files for jdk.security.auth 20:34:45 Creating support/native/java.security.jgss/libj2gss/static/libj2gss.a from 3 file(s) 20:34:45 Updating support/src.zip 20:34:46 Creating support/native/jdk.management.agent/libmanagement_agent/static/libmanagement_agent.a from 1 file(s) 20:34:46 Creating support/native/jdk.security.auth/libjaas/static/libjaas.a from 1 file(s) 20:34:48 Compiling 136 files for jdk.jdeps 20:34:48 Creating support/test/lib-test/jtreg/native/bin/jvm-test-launcher from 1 file(s) 20:34:49 Creating support/modules_libs/java.base/libverify.so from 1 file(s) ......... 20:36:17 Optimizing the exploded image 20:36:18 Creating java.datatransfer.jmod 20:36:18 Creating java.instrument.jmod 20:36:18 Creating java.compiler.jmod ------------- PR: https://git.openjdk.java.net/jdk/pull/6311 From shade at openjdk.java.net Tue Nov 9 16:51:46 2021 From: shade at openjdk.java.net (Aleksey Shipilev) Date: Tue, 9 Nov 2021 16:51:46 GMT Subject: RFR: 8276864: Update boot JDKs to 17.0.1 in GHA Message-ID: Current GHA runs at 17 GA, which misses fixes from 17.0.1. Additional testing: - [x] GHA builds are fine - [ ] GHA tests are fine ------------- Commit messages: - Fix Changes: https://git.openjdk.java.net/jdk/pull/6314/files Webrev: https://webrevs.openjdk.java.net/?repo=jdk&pr=6314&range=00 Issue: https://bugs.openjdk.java.net/browse/JDK-8276864 Stats: 9 lines in 1 file changed: 0 ins; 0 del; 9 mod Patch: https://git.openjdk.java.net/jdk/pull/6314.diff Fetch: git fetch https://git.openjdk.java.net/jdk pull/6314/head:pull/6314 PR: https://git.openjdk.java.net/jdk/pull/6314 From erikj at openjdk.java.net Tue Nov 9 17:28:45 2021 From: erikj at openjdk.java.net (Erik Joelsson) Date: Tue, 9 Nov 2021 17:28:45 GMT Subject: RFR: 8276743: Make openjdk build Zip Archive generation "reproducible" In-Reply-To: References: <13DpyL4de49GOSlp9DU1co_rkznpvX920O6Z0RsmjbY=.23225459-16de-459d-b361-884d8b1d8d35@github.com> <8SDOXwz8OhlhhqpWHjDbdxj9M1ZQfQcls2aRNt-cukQ=.f177d1e3-b30e-4e8b-ae35-d27606e18a5e@github.com> Message-ID: On Tue, 9 Nov 2021 14:55:52 GMT, Andrew Leonard wrote: >> make/common/ZipArchive.gmk line 178: >> >>> 176: (cd $$(SUPPORT_OUTPUTDIR)/ziptmp/$1/files && \ >>> 177: $(RM) $$@ && \ >>> 178: $(UNZIP) -q $$(SUPPORT_OUTPUTDIR)/ziptmp/$1/tmp.zip && \ >> >> Having to explode the zip here is unfortunate. This means we are creating an almost full copy of the whole src tree in the build directory, something I tried to avoid by leveraging the include/exclude functionality of zip, instead of generating make rules for copying the files I wanted into a source tree to run zip on. This may be a small overhead on Linux, but I'm pretty sure it will be very noticeable on Windows. >> >> When reading about your tool at first, I assumed it would read the intermediate zip file directly when rebuilding the zip. I don't think modifying it to do that would be too complicated, basically read and processing ZipEntrys instead of walking the file system. > > @erikj79 so had a bit of a think, and part of the unzipping.. then re-gen'ing is not having to load all the entries into memory. You can't guarantee the order "zip" has created them in, so realistically i'd have to read all the ZipEntry's into memory, then re-write.. which we can do.. src.zip is only 55MB or so, so memory requirements won't be huge given src.zip is the only target here currently. You are already keeping all the filenames in memory for sorting, so reading up the ZipEntry:s isn't that much more data, just some extra metadata for each entry. The actual file contents is not part of the ZipEntry object. When actually copying the files, you can use the ZipFile class to access ZipEntry's in arbitrary order to read their streams as InputStream. ------------- PR: https://git.openjdk.java.net/jdk/pull/6311 From erikj at openjdk.java.net Tue Nov 9 17:32:33 2021 From: erikj at openjdk.java.net (Erik Joelsson) Date: Tue, 9 Nov 2021 17:32:33 GMT Subject: RFR: 8276864: Update boot JDKs to 17.0.1 in GHA In-Reply-To: References: Message-ID: <4dd7WHG6LGkmAhOVD2THt-e65EmyDmysv-axln8nQXA=.c524ec37-882c-4159-83e2-1d996e99b9e9@github.com> On Tue, 9 Nov 2021 14:45:56 GMT, Aleksey Shipilev wrote: > Current GHA runs at 17 GA, which misses fixes from 17.0.1. > > Additional testing: > - [x] GHA builds are fine > - [ ] GHA tests are fine Marked as reviewed by erikj (Reviewer). ------------- PR: https://git.openjdk.java.net/jdk/pull/6314 From erikj at openjdk.java.net Tue Nov 9 17:34:38 2021 From: erikj at openjdk.java.net (Erik Joelsson) Date: Tue, 9 Nov 2021 17:34:38 GMT Subject: RFR: 8276743: Make openjdk build Zip Archive generation "reproducible" In-Reply-To: <13DpyL4de49GOSlp9DU1co_rkznpvX920O6Z0RsmjbY=.23225459-16de-459d-b361-884d8b1d8d35@github.com> References: <13DpyL4de49GOSlp9DU1co_rkznpvX920O6Z0RsmjbY=.23225459-16de-459d-b361-884d8b1d8d35@github.com> Message-ID: On Tue, 9 Nov 2021 12:59:17 GMT, Andrew Leonard wrote: > This PR adds a new openjdk build tool GenerateZip, which generates a final "zip" file from an input folder, and creates it in a deterministic way, ensuring ordering and timestamps are set as specified. > > Using this tool in ZipArchive.gmk will ensure src.zip is then created deterministically. > > Signed-off-by: Andrew Leonard I agree that ideally reproducibility should be on by default, but if there is a cost, then you can be sure OpenJDK developers will be looking for a way to remove it for faster turnaround times. I would propose a specific configure parameter for this specific case, reproducible zip files, that is default on for release builds and off for debug builds (debug builds aren't reproducible by nature) and let the existing meta flag also control the value of this new flag. ------------- PR: https://git.openjdk.java.net/jdk/pull/6311 From erikj at openjdk.java.net Tue Nov 9 17:34:38 2021 From: erikj at openjdk.java.net (Erik Joelsson) Date: Tue, 9 Nov 2021 17:34:38 GMT Subject: RFR: 8276743: Make openjdk build Zip Archive generation "reproducible" In-Reply-To: References: <13DpyL4de49GOSlp9DU1co_rkznpvX920O6Z0RsmjbY=.23225459-16de-459d-b361-884d8b1d8d35@github.com> <8SDOXwz8OhlhhqpWHjDbdxj9M1ZQfQcls2aRNt-cukQ=.f177d1e3-b30e-4e8b-ae35-d27606e18a5e@github.com> Message-ID: On Tue, 9 Nov 2021 17:26:05 GMT, Erik Joelsson wrote: >> @erikj79 so had a bit of a think, and part of the unzipping.. then re-gen'ing is not having to load all the entries into memory. You can't guarantee the order "zip" has created them in, so realistically i'd have to read all the ZipEntry's into memory, then re-write.. which we can do.. src.zip is only 55MB or so, so memory requirements won't be huge given src.zip is the only target here currently. > > You are already keeping all the filenames in memory for sorting, so reading up the ZipEntry:s isn't that much more data, just some extra metadata for each entry. The actual file contents is not part of the ZipEntry object. When actually copying the files, you can use the ZipFile class to access ZipEntry's in arbitrary order to read their streams as InputStream. Actually, you don't even need to save the ZipEntry:s in memory, you can just extract filenames from them on the first pass, sort them, then lookup the entries in ZipFile again on the second lap. :) I don't think that's necessary though. ------------- PR: https://git.openjdk.java.net/jdk/pull/6311 From asarkar at openjdk.java.net Tue Nov 9 20:54:56 2021 From: asarkar at openjdk.java.net (Anirvan Sarkar) Date: Tue, 9 Nov 2021 20:54:56 GMT Subject: RFR: 8276854: Windows GHA builds fail due to broken Cygwin Message-ID: Use version `3.2.0` of `cygwin` instead of latest version `3.3.2` on GitHub Actions. `make` SEGVs on the latest version. Version number format obtained from https://cygwin.com/packages/summary/cygwin.html ------------- Commit messages: - Use cygwin version 3.2.0 on GitHub Actions Changes: https://git.openjdk.java.net/jdk/pull/6319/files Webrev: https://webrevs.openjdk.java.net/?repo=jdk&pr=6319&range=00 Issue: https://bugs.openjdk.java.net/browse/JDK-8276854 Stats: 3 lines in 1 file changed: 0 ins; 0 del; 3 mod Patch: https://git.openjdk.java.net/jdk/pull/6319.diff Fetch: git fetch https://git.openjdk.java.net/jdk pull/6319/head:pull/6319 PR: https://git.openjdk.java.net/jdk/pull/6319 From clanger at openjdk.java.net Tue Nov 9 22:35:33 2021 From: clanger at openjdk.java.net (Christoph Langer) Date: Tue, 9 Nov 2021 22:35:33 GMT Subject: RFR: 8276854: Windows GHA builds fail due to broken Cygwin In-Reply-To: References: Message-ID: On Tue, 9 Nov 2021 20:47:29 GMT, Anirvan Sarkar wrote: > Use version `3.2.0` of `cygwin` instead of latest version `3.3.2` on GitHub Actions. > `make` SEGVs on the latest version. > > Version number format obtained from https://cygwin.com/packages/summary/cygwin.html Cool. Thanks for doing this fix. And thanks to @shipilev for chasing this down. ------------- Marked as reviewed by clanger (Reviewer). PR: https://git.openjdk.java.net/jdk/pull/6319 From ysuenaga at openjdk.java.net Wed Nov 10 00:55:13 2021 From: ysuenaga at openjdk.java.net (Yasumasa Suenaga) Date: Wed, 10 Nov 2021 00:55:13 GMT Subject: RFR: 8276672: Cannot build hsdis on WSL [v2] In-Reply-To: <4lLdxiPsFsB1GfI7iWSgLOXQ-Q_WU4ZZdOXHkRuO0kc=.45eea799-3e3d-4b1c-90dc-a6fa5e3d5b74@github.com> References: <4lLdxiPsFsB1GfI7iWSgLOXQ-Q_WU4ZZdOXHkRuO0kc=.45eea799-3e3d-4b1c-90dc-a6fa5e3d5b74@github.com> Message-ID: <9IXxoV7qReNTavl84nideiEvwLPBQ-T1ar7oZVQWzNQ=.274957db-948f-485d-aea4-a1fd0e5d2e75@github.com> > JDK-8275128 introduced new Makefile for hsdis to integrate it to normal build system, however it does not work on WSL 1 Ubuntu 20.04 and MinGW from Ubuntu. > > Hsdis.gmk has two problems: > > 1. MinGW version is fixed to "9.2.0" > 2. Assumes sysroot for MinGW is located at /usr/$MINGW_BASE/sys-root > > I tested this change on WSL only, so it is appreciate to review on Cygwin. Yasumasa Suenaga has updated the pull request with a new target base due to a merge or a rebase. The incremental webrev excludes the unrelated changes brought in by the merge/rebase. The pull request contains four additional commits since the last revision: - Merge remote-tracking branch 'magicus/hsdis-on-wsl' into JDK-8276672 - Make hsdis building on Windows more general - Merge remote-tracking branch 'upstream/master' into JDK-8276672 - 8276672: Cannot build hsdis on WSL ------------- Changes: - all: https://git.openjdk.java.net/jdk/pull/6269/files - new: https://git.openjdk.java.net/jdk/pull/6269/files/ea5e749b..7fa765cd Webrevs: - full: https://webrevs.openjdk.java.net/?repo=jdk&pr=6269&range=01 - incr: https://webrevs.openjdk.java.net/?repo=jdk&pr=6269&range=00-01 Stats: 397648 lines in 435 files changed: 195125 ins; 194851 del; 7672 mod Patch: https://git.openjdk.java.net/jdk/pull/6269.diff Fetch: git fetch https://git.openjdk.java.net/jdk pull/6269/head:pull/6269 PR: https://git.openjdk.java.net/jdk/pull/6269 From ysuenaga at openjdk.java.net Wed Nov 10 00:55:15 2021 From: ysuenaga at openjdk.java.net (Yasumasa Suenaga) Date: Wed, 10 Nov 2021 00:55:15 GMT Subject: RFR: 8276672: Cannot build hsdis on WSL In-Reply-To: <0YUNwCXUxPBoqE69W9_YinizmaGWFPpXWHgcka-9iD0=.6b22737f-66f1-47f7-84f5-5c05761e6396@github.com> References: <4lLdxiPsFsB1GfI7iWSgLOXQ-Q_WU4ZZdOXHkRuO0kc=.45eea799-3e3d-4b1c-90dc-a6fa5e3d5b74@github.com> <0YUNwCXUxPBoqE69W9_YinizmaGWFPpXWHgcka-9iD0=.6b22737f-66f1-47f7-84f5-5c05761e6396@github.com> Message-ID: On Tue, 9 Nov 2021 15:06:05 GMT, Magnus Ihse Bursie wrote: >> JDK-8275128 introduced new Makefile for hsdis to integrate it to normal build system, however it does not work on WSL 1 Ubuntu 20.04 and MinGW from Ubuntu. >> >> Hsdis.gmk has two problems: >> >> 1. MinGW version is fixed to "9.2.0" >> 2. Assumes sysroot for MinGW is located at /usr/$MINGW_BASE/sys-root >> >> I tested this change on WSL only, so it is appreciate to review on Cygwin. > > Please check out if https://github.com/magicus/jdk/tree/hsdis-on-wsl works OK in your environment. It does work for me on Cygwin. @magicus Thanks ! Your change works fine on my WSL, so I merged it to this PR (and also I added you as a contributor). ------------- PR: https://git.openjdk.java.net/jdk/pull/6269 From ysuenaga at openjdk.java.net Wed Nov 10 01:41:03 2021 From: ysuenaga at openjdk.java.net (Yasumasa Suenaga) Date: Wed, 10 Nov 2021 01:41:03 GMT Subject: RFR: 8276672: Cannot build hsdis on WSL [v3] In-Reply-To: <4lLdxiPsFsB1GfI7iWSgLOXQ-Q_WU4ZZdOXHkRuO0kc=.45eea799-3e3d-4b1c-90dc-a6fa5e3d5b74@github.com> References: <4lLdxiPsFsB1GfI7iWSgLOXQ-Q_WU4ZZdOXHkRuO0kc=.45eea799-3e3d-4b1c-90dc-a6fa5e3d5b74@github.com> Message-ID: > JDK-8275128 introduced new Makefile for hsdis to integrate it to normal build system, however it does not work on WSL 1 Ubuntu 20.04 and MinGW from Ubuntu. > > Hsdis.gmk has two problems: > > 1. MinGW version is fixed to "9.2.0" > 2. Assumes sysroot for MinGW is located at /usr/$MINGW_BASE/sys-root > > I tested this change on WSL only, so it is appreciate to review on Cygwin. Yasumasa Suenaga has updated the pull request incrementally with one additional commit since the last revision: Remove SYSROOT_FLAGS ------------- Changes: - all: https://git.openjdk.java.net/jdk/pull/6269/files - new: https://git.openjdk.java.net/jdk/pull/6269/files/7fa765cd..0eaf388a Webrevs: - full: https://webrevs.openjdk.java.net/?repo=jdk&pr=6269&range=02 - incr: https://webrevs.openjdk.java.net/?repo=jdk&pr=6269&range=01-02 Stats: 1 line in 1 file changed: 0 ins; 1 del; 0 mod Patch: https://git.openjdk.java.net/jdk/pull/6269.diff Fetch: git fetch https://git.openjdk.java.net/jdk pull/6269/head:pull/6269 PR: https://git.openjdk.java.net/jdk/pull/6269 From ysuenaga at openjdk.java.net Wed Nov 10 01:41:09 2021 From: ysuenaga at openjdk.java.net (Yasumasa Suenaga) Date: Wed, 10 Nov 2021 01:41:09 GMT Subject: RFR: 8276672: Cannot build hsdis on WSL [v2] In-Reply-To: <9IXxoV7qReNTavl84nideiEvwLPBQ-T1ar7oZVQWzNQ=.274957db-948f-485d-aea4-a1fd0e5d2e75@github.com> References: <4lLdxiPsFsB1GfI7iWSgLOXQ-Q_WU4ZZdOXHkRuO0kc=.45eea799-3e3d-4b1c-90dc-a6fa5e3d5b74@github.com> <9IXxoV7qReNTavl84nideiEvwLPBQ-T1ar7oZVQWzNQ=.274957db-948f-485d-aea4-a1fd0e5d2e75@github.com> Message-ID: On Wed, 10 Nov 2021 00:55:13 GMT, Yasumasa Suenaga wrote: >> JDK-8275128 introduced new Makefile for hsdis to integrate it to normal build system, however it does not work on WSL 1 Ubuntu 20.04 and MinGW from Ubuntu. >> >> Hsdis.gmk has two problems: >> >> 1. MinGW version is fixed to "9.2.0" >> 2. Assumes sysroot for MinGW is located at /usr/$MINGW_BASE/sys-root >> >> I tested this change on WSL only, so it is appreciate to review on Cygwin. > > Yasumasa Suenaga has updated the pull request with a new target base due to a merge or a rebase. The incremental webrev excludes the unrelated changes brought in by the merge/rebase. The pull request contains four additional commits since the last revision: > > - Merge remote-tracking branch 'magicus/hsdis-on-wsl' into JDK-8276672 > - Make hsdis building on Windows more general > - Merge remote-tracking branch 'upstream/master' into JDK-8276672 > - 8276672: Cannot build hsdis on WSL I removed `SYSROOT_FLAGS` because it is no longer used. ------------- PR: https://git.openjdk.java.net/jdk/pull/6269 From asarkar at openjdk.java.net Wed Nov 10 05:54:43 2021 From: asarkar at openjdk.java.net (Anirvan Sarkar) Date: Wed, 10 Nov 2021 05:54:43 GMT Subject: Integrated: 8276854: Windows GHA builds fail due to broken Cygwin In-Reply-To: References: Message-ID: <7IbeK_Go4pBobwKzD1c3zwelLGHGlEpWvoT5KrcFttE=.46a5b2c4-74f9-4d77-8dd5-b9bf7a3a494d@github.com> On Tue, 9 Nov 2021 20:47:29 GMT, Anirvan Sarkar wrote: > Use version `3.2.0` of `cygwin` instead of latest version `3.3.2` on GitHub Actions. > `make` SEGVs on the latest version. > > Version number format obtained from https://cygwin.com/packages/summary/cygwin.html This pull request has now been integrated. Changeset: 403f3185 Author: Anirvan Sarkar Committer: Christoph Langer URL: https://git.openjdk.java.net/jdk/commit/403f3185f0988dcf6ef4e857d6659533bfa2943f Stats: 3 lines in 1 file changed: 0 ins; 0 del; 3 mod 8276854: Windows GHA builds fail due to broken Cygwin Reviewed-by: clanger ------------- PR: https://git.openjdk.java.net/jdk/pull/6319 From shade at openjdk.java.net Wed Nov 10 07:48:41 2021 From: shade at openjdk.java.net (Aleksey Shipilev) Date: Wed, 10 Nov 2021 07:48:41 GMT Subject: RFR: 8276854: Windows GHA builds fail due to broken Cygwin In-Reply-To: References: Message-ID: On Tue, 9 Nov 2021 20:47:29 GMT, Anirvan Sarkar wrote: > Use version `3.2.0` of `cygwin` instead of latest version `3.3.2` on GitHub Actions. > `make` SEGVs on the latest version. > > Version number format obtained from https://cygwin.com/packages/summary/cygwin.html The patch looks okay. But what just happened here process-wise? Please do *not* take the issue out of assignee's hands without asking first, and definitely do *NOT* push the fix until the original assignee at least performs some sort of handover. Especially don't do that in the span of less than 24 hours, where the assignee might just be asleep. I also suspect the build infra changes require more reviews, anyway. Don't rush it, folks! ------------- PR: https://git.openjdk.java.net/jdk/pull/6319 From stuefe at openjdk.java.net Wed Nov 10 10:07:43 2021 From: stuefe at openjdk.java.net (Thomas Stuefe) Date: Wed, 10 Nov 2021 10:07:43 GMT Subject: RFR: 8276854: Windows GHA builds fail due to broken Cygwin In-Reply-To: References: Message-ID: On Tue, 9 Nov 2021 20:47:29 GMT, Anirvan Sarkar wrote: > Use version `3.2.0` of `cygwin` instead of latest version `3.3.2` on GitHub Actions. > `make` SEGVs on the latest version. > > Version number format obtained from https://cygwin.com/packages/summary/cygwin.html > The patch looks okay. > > But what just happened here process-wise? Please do _not_ take the issue out of assignee's hands without asking first, and definitely do _NOT_ push the fix until the original assignee at least performs some sort of handover. Especially don't do that in the span of less than 24 hours, where the assignee might just be asleep. Yes, that was odd. Please don't reassign JBS issues unless you either talk with the assignee or it has been clearly abandoned. ------------- PR: https://git.openjdk.java.net/jdk/pull/6319 From asarkar at openjdk.java.net Wed Nov 10 10:17:41 2021 From: asarkar at openjdk.java.net (Anirvan Sarkar) Date: Wed, 10 Nov 2021 10:17:41 GMT Subject: RFR: 8276854: Windows GHA builds fail due to broken Cygwin In-Reply-To: References: Message-ID: On Tue, 9 Nov 2021 20:47:29 GMT, Anirvan Sarkar wrote: > Use version `3.2.0` of `cygwin` instead of latest version `3.3.2` on GitHub Actions. > `make` SEGVs on the latest version. > > Version number format obtained from https://cygwin.com/packages/summary/cygwin.html It is my mistake. Sorry for rushing and breaking the etiquette. I will ensure that this does not happen again. ------------- PR: https://git.openjdk.java.net/jdk/pull/6319 From shade at openjdk.java.net Wed Nov 10 10:42:42 2021 From: shade at openjdk.java.net (Aleksey Shipilev) Date: Wed, 10 Nov 2021 10:42:42 GMT Subject: RFR: 8276854: Windows GHA builds fail due to broken Cygwin In-Reply-To: References: Message-ID: <2LLzOBaUmJwZXOG9G6frl_wwaYPQKSphmwiPaSPmFNY=.c6e5e2be-73a1-4b92-8944-0500062ac943@github.com> On Wed, 10 Nov 2021 10:14:36 GMT, Anirvan Sarkar wrote: > It is my mistake. Sorry for rushing and breaking the etiquette. I will ensure that this does not happen again. Good, thanks. No harm done in this particular case, as the patch is definitely sensible. I'll handle the backports... ------------- PR: https://git.openjdk.java.net/jdk/pull/6319 From shade at openjdk.java.net Wed Nov 10 11:24:37 2021 From: shade at openjdk.java.net (Aleksey Shipilev) Date: Wed, 10 Nov 2021 11:24:37 GMT Subject: RFR: 8276864: Update boot JDKs to 17.0.1 in GHA In-Reply-To: References: Message-ID: On Tue, 9 Nov 2021 14:45:56 GMT, Aleksey Shipilev wrote: > Current GHA runs at 17 GA, which misses fixes from 17.0.1. > > Additional testing: > - [x] GHA builds are fine > - [x] GHA tests are fine Thanks, Erik. Anyone else wants/needs to review this? ------------- PR: https://git.openjdk.java.net/jdk/pull/6314 From aleonard at openjdk.java.net Wed Nov 10 11:25:42 2021 From: aleonard at openjdk.java.net (Andrew Leonard) Date: Wed, 10 Nov 2021 11:25:42 GMT Subject: RFR: 8276743: Make openjdk build Zip Archive generation "reproducible" In-Reply-To: References: <13DpyL4de49GOSlp9DU1co_rkznpvX920O6Z0RsmjbY=.23225459-16de-459d-b361-884d8b1d8d35@github.com> <8SDOXwz8OhlhhqpWHjDbdxj9M1ZQfQcls2aRNt-cukQ=.f177d1e3-b30e-4e8b-ae35-d27606e18a5e@github.com> Message-ID: On Tue, 9 Nov 2021 17:31:16 GMT, Erik Joelsson wrote: >> You are already keeping all the filenames in memory for sorting, so reading up the ZipEntry:s isn't that much more data, just some extra metadata for each entry. The actual file contents is not part of the ZipEntry object. When actually copying the files, you can use the ZipFile class to access ZipEntry's in arbitrary order to read their streams as InputStream. > > Actually, you don't even need to save the ZipEntry:s in memory, you can just extract filenames from them on the first pass, sort them, then lookup the entries in ZipFile again on the second lap. :) I don't think that's necessary though. @erikj79 thanks I didn't realize you can do that: "you can use the ZipFile class to access ZipEntry's in arbitrary order" ------------- PR: https://git.openjdk.java.net/jdk/pull/6311 From ihse at openjdk.java.net Wed Nov 10 11:52:39 2021 From: ihse at openjdk.java.net (Magnus Ihse Bursie) Date: Wed, 10 Nov 2021 11:52:39 GMT Subject: RFR: 8276672: Cannot build hsdis on WSL [v3] In-Reply-To: References: <4lLdxiPsFsB1GfI7iWSgLOXQ-Q_WU4ZZdOXHkRuO0kc=.45eea799-3e3d-4b1c-90dc-a6fa5e3d5b74@github.com> Message-ID: <8Cw05IJxaD8nMnb-uPZ9Ulogjm3Ex5lZdlzpDiy8ryA=.f09df9a7-69a9-4cf7-aec2-682eb102af0c@github.com> On Wed, 10 Nov 2021 01:41:03 GMT, Yasumasa Suenaga wrote: >> JDK-8275128 introduced new Makefile for hsdis to integrate it to normal build system, however it does not work on WSL 1 Ubuntu 20.04 and MinGW from Ubuntu. >> >> Hsdis.gmk has two problems: >> >> 1. MinGW version is fixed to "9.2.0" >> 2. Assumes sysroot for MinGW is located at /usr/$MINGW_BASE/sys-root >> >> I tested this change on WSL only, so it is appreciate to review on Cygwin. > > Yasumasa Suenaga has updated the pull request incrementally with one additional commit since the last revision: > > Remove SYSROOT_FLAGS Great! I missed the redundant `SYSROOT_FLAGS`, thanks for taking care of that. ------------- Marked as reviewed by ihse (Reviewer). PR: https://git.openjdk.java.net/jdk/pull/6269 From duke at openjdk.java.net Wed Nov 10 12:39:59 2021 From: duke at openjdk.java.net (Alan Hayward) Date: Wed, 10 Nov 2021 12:39:59 GMT Subject: RFR: 8264130: PAC-RET protection for Linux/AArch64 Message-ID: PAC is an optional feature in AArch64 8.3 and is compulsory in v9. One of its uses is to protect against ROP based attacks. This is done by signing the Link Register whenever it is stored on the stack, and authenticating the value when it is loaded back from the stack. If an attacker were to try to change control flow by editing the stack then the authentication check of the Link Register will fail, causing a segfault when the function returns. On a system with PAC enabled, it is expected that all applications will be compiled with ROP protection. Fedora 33 and upwards already provide this. By compiling for ARMv8.0, GCC and LLVM will only use the set of PAC instructions that exist in the NOP space - on hardware without PAC, these instructions act as NOPs, allowing backward compatibility for negligible performance cost (2 NOPs per non-leaf function). Hardware is currently limited to the Apple M1 MacBooks. All testing has been done within a Fedora Docker image. A run of SpecJVM showed no difference to that of noise - which was surprising. The most important part of this patch is simply compiling using branch protection provided by GCC/LLVM. This protects all C++ code from being used in ROP attacks, removing all static ROP gadgets from use. The remainder of the patch adds ROP protection to runtime generated code, in both stubs and compiled Java code. Attacks here are much harder as ROP gadgets must be found dynamically at runtime. If/when AOT compilation is added to JDK, then all stubs and compiled Java will be susceptible ROP gadgets being found by static analysis and therefore potentially as vulnerable as C++ code. There are a number of places where the VM changes control flow by rewriting the stack or otherwise. I?ve done some analysis as to how these could also be used for attacks (which I didn?t want to post here). These areas can be protected ensuring the pointers to various stubs and entry points are stored in memory as signed pointers. These changes are simple to make (they can be reduced to a type change in common code and a few addition sign/auth calls in the backend), but there a lot of them and the total code change is fairly large. I?m happy to provide a few work in progress patches. In order to match the security benefits of the Apple Arm64e ABI across the whole of JDK, then all the changes mentioned above would be required. ------------- Commit messages: - 8264130: PAC-RET protection for Linux/AArch64 - Add PAC assembly instructions - Add AArch64 ROP protection runtime flag - Build with branch protection Changes: https://git.openjdk.java.net/jdk/pull/6334/files Webrev: https://webrevs.openjdk.java.net/?repo=jdk&pr=6334&range=00 Issue: https://bugs.openjdk.java.net/browse/JDK-8264130 Stats: 1273 lines in 25 files changed: 457 ins; 20 del; 796 mod Patch: https://git.openjdk.java.net/jdk/pull/6334.diff Fetch: git fetch https://git.openjdk.java.net/jdk pull/6334/head:pull/6334 PR: https://git.openjdk.java.net/jdk/pull/6334 From aph at openjdk.java.net Wed Nov 10 13:14:43 2021 From: aph at openjdk.java.net (Andrew Haley) Date: Wed, 10 Nov 2021 13:14:43 GMT Subject: RFR: 8264130: PAC-RET protection for Linux/AArch64 In-Reply-To: References: Message-ID: On Wed, 10 Nov 2021 12:32:53 GMT, Alan Hayward wrote: > PAC is an optional feature in AArch64 8.3 and is compulsory in v9. One > of its uses is to protect against ROP based attacks. This is done by > signing the Link Register whenever it is stored on the stack, and > authenticating the value when it is loaded back from the stack. If an > attacker were to try to change control flow by editing the stack then > the authentication check of the Link Register will fail, causing a > segfault when the function returns. > > On a system with PAC enabled, it is expected that all applications will > be compiled with ROP protection. Fedora 33 and upwards already provide > this. By compiling for ARMv8.0, GCC and LLVM will only use the set of > PAC instructions that exist in the NOP space - on hardware without PAC, > these instructions act as NOPs, allowing backward compatibility for > negligible performance cost (2 NOPs per non-leaf function). > > Hardware is currently limited to the Apple M1 MacBooks. All testing has > been done within a Fedora Docker image. A run of SpecJVM showed no > difference to that of noise - which was surprising. > > The most important part of this patch is simply compiling using branch > protection provided by GCC/LLVM. This protects all C++ code from being > used in ROP attacks, removing all static ROP gadgets from use. > > The remainder of the patch adds ROP protection to runtime generated > code, in both stubs and compiled Java code. Attacks here are much harder > as ROP gadgets must be found dynamically at runtime. If/when AOT > compilation is added to JDK, then all stubs and compiled Java will be > susceptible ROP gadgets being found by static analysis and therefore > potentially as vulnerable as C++ code. > > There are a number of places where the VM changes control flow by > rewriting the stack or otherwise. I?ve done some analysis as to how > these could also be used for attacks (which I didn?t want to post here). > These areas can be protected ensuring the pointers to various stubs and > entry points are stored in memory as signed pointers. These changes are > simple to make (they can be reduced to a type change in common code and > a few addition sign/auth calls in the backend), but there a lot of them > and the total code change is fairly large. I?m happy to provide a few > work in progress patches. > > In order to match the security benefits of the Apple Arm64e ABI across > the whole of JDK, then all the changes mentioned above would be > required. Gosh. This is going to take some time to review, and will need at least two reviewers. ------------- PR: https://git.openjdk.java.net/jdk/pull/6334 From aph at openjdk.java.net Wed Nov 10 13:25:40 2021 From: aph at openjdk.java.net (Andrew Haley) Date: Wed, 10 Nov 2021 13:25:40 GMT Subject: RFR: 8264130: PAC-RET protection for Linux/AArch64 In-Reply-To: References: Message-ID: On Wed, 10 Nov 2021 12:32:53 GMT, Alan Hayward wrote: > PAC is an optional feature in AArch64 8.3 and is compulsory in v9. One > of its uses is to protect against ROP based attacks. This is done by > signing the Link Register whenever it is stored on the stack, and > authenticating the value when it is loaded back from the stack. If an > attacker were to try to change control flow by editing the stack then > the authentication check of the Link Register will fail, causing a > segfault when the function returns. > > On a system with PAC enabled, it is expected that all applications will > be compiled with ROP protection. Fedora 33 and upwards already provide > this. By compiling for ARMv8.0, GCC and LLVM will only use the set of > PAC instructions that exist in the NOP space - on hardware without PAC, > these instructions act as NOPs, allowing backward compatibility for > negligible performance cost (2 NOPs per non-leaf function). > > Hardware is currently limited to the Apple M1 MacBooks. All testing has > been done within a Fedora Docker image. A run of SpecJVM showed no > difference to that of noise - which was surprising. > > The most important part of this patch is simply compiling using branch > protection provided by GCC/LLVM. This protects all C++ code from being > used in ROP attacks, removing all static ROP gadgets from use. > > The remainder of the patch adds ROP protection to runtime generated > code, in both stubs and compiled Java code. Attacks here are much harder > as ROP gadgets must be found dynamically at runtime. If/when AOT > compilation is added to JDK, then all stubs and compiled Java will be > susceptible ROP gadgets being found by static analysis and therefore > potentially as vulnerable as C++ code. > > There are a number of places where the VM changes control flow by > rewriting the stack or otherwise. I?ve done some analysis as to how > these could also be used for attacks (which I didn?t want to post here). > These areas can be protected ensuring the pointers to various stubs and > entry points are stored in memory as signed pointers. These changes are > simple to make (they can be reduced to a type change in common code and > a few addition sign/auth calls in the backend), but there a lot of them > and the total code change is fairly large. I?m happy to provide a few > work in progress patches. > > In order to match the security benefits of the Apple Arm64e ABI across > the whole of JDK, then all the changes mentioned above would be > required. src/hotspot/cpu/aarch64/macroAssembler_aarch64.cpp line 5185: > 5183: // ROP Protection > 5184: > 5185: void MacroAssembler::protect_return_address() { We need proper, full, detailed comments about what these functions do, with reference to primary AArch64 documentation. ------------- PR: https://git.openjdk.java.net/jdk/pull/6334 From erikj at openjdk.java.net Wed Nov 10 13:27:39 2021 From: erikj at openjdk.java.net (Erik Joelsson) Date: Wed, 10 Nov 2021 13:27:39 GMT Subject: RFR: 8276672: Cannot build hsdis on WSL [v3] In-Reply-To: References: <4lLdxiPsFsB1GfI7iWSgLOXQ-Q_WU4ZZdOXHkRuO0kc=.45eea799-3e3d-4b1c-90dc-a6fa5e3d5b74@github.com> Message-ID: On Wed, 10 Nov 2021 01:41:03 GMT, Yasumasa Suenaga wrote: >> JDK-8275128 introduced new Makefile for hsdis to integrate it to normal build system, however it does not work on WSL 1 Ubuntu 20.04 and MinGW from Ubuntu. >> >> Hsdis.gmk has two problems: >> >> 1. MinGW version is fixed to "9.2.0" >> 2. Assumes sysroot for MinGW is located at /usr/$MINGW_BASE/sys-root >> >> I tested this change on WSL only, so it is appreciate to review on Cygwin. > > Yasumasa Suenaga has updated the pull request incrementally with one additional commit since the last revision: > > Remove SYSROOT_FLAGS Marked as reviewed by erikj (Reviewer). ------------- PR: https://git.openjdk.java.net/jdk/pull/6269 From erikj at openjdk.java.net Wed Nov 10 13:34:38 2021 From: erikj at openjdk.java.net (Erik Joelsson) Date: Wed, 10 Nov 2021 13:34:38 GMT Subject: RFR: 8264130: PAC-RET protection for Linux/AArch64 In-Reply-To: References: Message-ID: <9inSsWwjEQnZT_x-9GirjL-Avmycfnyj6yZoqCJ8M4g=.770fc4bb-6731-4eb2-83a7-cf018438ed7e@github.com> On Wed, 10 Nov 2021 12:32:53 GMT, Alan Hayward wrote: > PAC is an optional feature in AArch64 8.3 and is compulsory in v9. One > of its uses is to protect against ROP based attacks. This is done by > signing the Link Register whenever it is stored on the stack, and > authenticating the value when it is loaded back from the stack. If an > attacker were to try to change control flow by editing the stack then > the authentication check of the Link Register will fail, causing a > segfault when the function returns. > > On a system with PAC enabled, it is expected that all applications will > be compiled with ROP protection. Fedora 33 and upwards already provide > this. By compiling for ARMv8.0, GCC and LLVM will only use the set of > PAC instructions that exist in the NOP space - on hardware without PAC, > these instructions act as NOPs, allowing backward compatibility for > negligible performance cost (2 NOPs per non-leaf function). > > Hardware is currently limited to the Apple M1 MacBooks. All testing has > been done within a Fedora Docker image. A run of SpecJVM showed no > difference to that of noise - which was surprising. > > The most important part of this patch is simply compiling using branch > protection provided by GCC/LLVM. This protects all C++ code from being > used in ROP attacks, removing all static ROP gadgets from use. > > The remainder of the patch adds ROP protection to runtime generated > code, in both stubs and compiled Java code. Attacks here are much harder > as ROP gadgets must be found dynamically at runtime. If/when AOT > compilation is added to JDK, then all stubs and compiled Java will be > susceptible ROP gadgets being found by static analysis and therefore > potentially as vulnerable as C++ code. > > There are a number of places where the VM changes control flow by > rewriting the stack or otherwise. I?ve done some analysis as to how > these could also be used for attacks (which I didn?t want to post here). > These areas can be protected ensuring the pointers to various stubs and > entry points are stored in memory as signed pointers. These changes are > simple to make (they can be reduced to a type change in common code and > a few addition sign/auth calls in the backend), but there a lot of them > and the total code change is fairly large. I?m happy to provide a few > work in progress patches. > > In order to match the security benefits of the Apple Arm64e ABI across > the whole of JDK, then all the changes mentioned above would be > required. Build change looks good, but I can't comment on the code changes. ------------- Marked as reviewed by erikj (Reviewer). PR: https://git.openjdk.java.net/jdk/pull/6334 From duke at openjdk.java.net Wed Nov 10 13:34:39 2021 From: duke at openjdk.java.net (Alan Hayward) Date: Wed, 10 Nov 2021 13:34:39 GMT Subject: RFR: 8264130: PAC-RET protection for Linux/AArch64 In-Reply-To: References: Message-ID: <4yyBp8jgXpmK0ZyywX4mrjo5vfTWfy5CHV97fnX4-EE=.2bd02511-67cd-40a4-8fb0-a79b095f4bcd@github.com> On Wed, 10 Nov 2021 13:11:21 GMT, Andrew Haley wrote: > Gosh. This is going to take some time to review, and will need at least two reviewers. Sure. And thanks in advance. ------------- PR: https://git.openjdk.java.net/jdk/pull/6334 From aph at openjdk.java.net Wed Nov 10 13:37:41 2021 From: aph at openjdk.java.net (Andrew Haley) Date: Wed, 10 Nov 2021 13:37:41 GMT Subject: RFR: 8264130: PAC-RET protection for Linux/AArch64 In-Reply-To: References: Message-ID: On Wed, 10 Nov 2021 12:32:53 GMT, Alan Hayward wrote: > PAC is an optional feature in AArch64 8.3 and is compulsory in v9. One > of its uses is to protect against ROP based attacks. This is done by > signing the Link Register whenever it is stored on the stack, and > authenticating the value when it is loaded back from the stack. If an > attacker were to try to change control flow by editing the stack then > the authentication check of the Link Register will fail, causing a > segfault when the function returns. > > On a system with PAC enabled, it is expected that all applications will > be compiled with ROP protection. Fedora 33 and upwards already provide > this. By compiling for ARMv8.0, GCC and LLVM will only use the set of > PAC instructions that exist in the NOP space - on hardware without PAC, > these instructions act as NOPs, allowing backward compatibility for > negligible performance cost (2 NOPs per non-leaf function). > > Hardware is currently limited to the Apple M1 MacBooks. All testing has > been done within a Fedora Docker image. A run of SpecJVM showed no > difference to that of noise - which was surprising. > > The most important part of this patch is simply compiling using branch > protection provided by GCC/LLVM. This protects all C++ code from being > used in ROP attacks, removing all static ROP gadgets from use. > > The remainder of the patch adds ROP protection to runtime generated > code, in both stubs and compiled Java code. Attacks here are much harder > as ROP gadgets must be found dynamically at runtime. If/when AOT > compilation is added to JDK, then all stubs and compiled Java will be > susceptible ROP gadgets being found by static analysis and therefore > potentially as vulnerable as C++ code. > > There are a number of places where the VM changes control flow by > rewriting the stack or otherwise. I?ve done some analysis as to how > these could also be used for attacks (which I didn?t want to post here). > These areas can be protected ensuring the pointers to various stubs and > entry points are stored in memory as signed pointers. These changes are > simple to make (they can be reduced to a type change in common code and > a few addition sign/auth calls in the backend), but there a lot of them > and the total code change is fairly large. I?m happy to provide a few > work in progress patches. > > In order to match the security benefits of the Apple Arm64e ABI across > the whole of JDK, then all the changes mentioned above would be > required. src/hotspot/os_cpu/bsd_aarch64/pauth_bsd_aarch64.inline.hpp line 25: > 23: */ > 24: > 25: #ifndef OS_CPU_BSD_AARCH64_PAUTH_BSD_AARCH64_INLINE_HPP Are these two files different enough to separate them for BSD and Linux? ------------- PR: https://git.openjdk.java.net/jdk/pull/6334 From magnus.ihse.bursie at oracle.com Wed Nov 10 13:49:26 2021 From: magnus.ihse.bursie at oracle.com (Magnus Ihse Bursie) Date: Wed, 10 Nov 2021 14:49:26 +0100 Subject: RFR: 8276854: Windows GHA builds fail due to broken Cygwin In-Reply-To: References: Message-ID: <05cc74f0-f1f8-7cf5-2e84-a907e3e22276@oracle.com> https://openjdk.java.net/census#build On 2021-11-10 08:48, Aleksey Shipilev wrote: > On Tue, 9 Nov 2021 20:47:29 GMT, Anirvan Sarkar wrote: > >> Use version `3.2.0` of `cygwin` instead of latest version `3.3.2` on GitHub Actions. >> `make` SEGVs on the latest version. >> >> Version number format obtained from https://cygwin.com/packages/summary/cygwin.html > The patch looks okay. > > But what just happened here process-wise? Please do *not* take the issue out of assignee's hands without asking first, and definitely do *NOT* push the fix until the original assignee at least performs some sort of handover. Especially don't do that in the span of less than 24 hours, where the assignee might just be asleep. I also suspect the build infra changes require more reviews, anyway. Don't rush it, folks! I would like to add to Aleksey's comment that the ideal is that you get a review from someone in the Build Group (https://openjdk.java.net/census#build). You definitely need to wait 24h if you have no Build Group member reviews., unless the PR is extremely urgent. /Magnus > > ------------- > > PR: https://git.openjdk.java.net/jdk/pull/6319 From clanger at openjdk.java.net Wed Nov 10 14:04:39 2021 From: clanger at openjdk.java.net (Christoph Langer) Date: Wed, 10 Nov 2021 14:04:39 GMT Subject: RFR: 8276854: Windows GHA builds fail due to broken Cygwin In-Reply-To: <2LLzOBaUmJwZXOG9G6frl_wwaYPQKSphmwiPaSPmFNY=.c6e5e2be-73a1-4b92-8944-0500062ac943@github.com> References: <2LLzOBaUmJwZXOG9G6frl_wwaYPQKSphmwiPaSPmFNY=.c6e5e2be-73a1-4b92-8944-0500062ac943@github.com> Message-ID: On Wed, 10 Nov 2021 10:39:04 GMT, Aleksey Shipilev wrote: > It is my mistake. Sorry for rushing and breaking the etiquette. I will ensure that this does not happen again. That was also my mistake (probably even more than Anirvan's) since I sponsored this too early and I should have taken care for all the points mentioned above. Sorry folks! ------------- PR: https://git.openjdk.java.net/jdk/pull/6319 From magnus.ihse.bursie at oracle.com Wed Nov 10 14:06:46 2021 From: magnus.ihse.bursie at oracle.com (Magnus Ihse Bursie) Date: Wed, 10 Nov 2021 15:06:46 +0100 Subject: CFV: New Build Group Member: Aleksey Shipilev Message-ID: <6911ff3e-67d5-0991-d0a0-08a06b1b09b2@oracle.com> I hereby nominate Aleksey Shipilev (shade) to Membership in the Build Group. Aleksey is a long standing member of the OpenJDK Members Group and the Hotspot Group. He is a prolific contributor, and has committed more than 600 changes to the JDK over the years. Lately, Aleksey has made a number of improvement to the build system, which is indicated by the more than 140 commits made to files in the "make" directory since 2018. Votes are due by 24 November 2021, 16:00 CET. Only current Members of the Build Group [2] are eligible to vote on this nomination. Votes must be cast in the open by replying to this mailing list For Lazy Consensus voting instructions, see [3]. /Magnus [1] https://github.com/openjdk/jdk/search?q=author-name%3A%22Aleksey+Shipilev%22&type=commits [2]http://openjdk.java.net/census#build [3]http://openjdk.java.net/groups/#member-vote From magnus.ihse.bursie at oracle.com Wed Nov 10 14:12:04 2021 From: magnus.ihse.bursie at oracle.com (Magnus Ihse Bursie) Date: Wed, 10 Nov 2021 15:12:04 +0100 Subject: CFV: New Build Group Member: Aleksey Shipilev In-Reply-To: <6911ff3e-67d5-0991-d0a0-08a06b1b09b2@oracle.com> References: <6911ff3e-67d5-0991-d0a0-08a06b1b09b2@oracle.com> Message-ID: Vote: yes /Magnus On 2021-11-10 15:06, Magnus Ihse Bursie wrote: > I hereby nominate Aleksey Shipilev (shade) to Membership in the Build > Group. > > Aleksey is a long standing member of the OpenJDK Members Group and the > Hotspot > Group. He is a prolific contributor, and has committed more than 600 > changes to > the JDK over the years. Lately, Aleksey has made a number of > improvement to the > build system, which is indicated by the more than 140 commits made to > files in > the "make" directory since 2018. > > Votes are due by 24 November 2021, 16:00 CET. > > Only current Members of the Build Group [2] are eligible > to vote on this nomination.? Votes must be cast in the open by > replying to this mailing list > > For Lazy Consensus voting instructions, see [3]. > > /Magnus > > [1] > https://github.com/openjdk/jdk/search?q=author-name%3A%22Aleksey+Shipilev%22&type=commits > [2]http://openjdk.java.net/census#build > [3]http://openjdk.java.net/groups/#member-vote > From erik.joelsson at oracle.com Wed Nov 10 14:12:49 2021 From: erik.joelsson at oracle.com (erik.joelsson at oracle.com) Date: Wed, 10 Nov 2021 06:12:49 -0800 Subject: CFV: New Build Group Member: Aleksey Shipilev In-Reply-To: <6911ff3e-67d5-0991-d0a0-08a06b1b09b2@oracle.com> References: <6911ff3e-67d5-0991-d0a0-08a06b1b09b2@oracle.com> Message-ID: Vote: yes /Erik On 2021-11-10 06:06, Magnus Ihse Bursie wrote: > I hereby nominate Aleksey Shipilev (shade) to Membership in the Build > Group. > > Aleksey is a long standing member of the OpenJDK Members Group and the > Hotspot > Group. He is a prolific contributor, and has committed more than 600 > changes to > the JDK over the years. Lately, Aleksey has made a number of > improvement to the > build system, which is indicated by the more than 140 commits made to > files in > the "make" directory since 2018. > > Votes are due by 24 November 2021, 16:00 CET. > > Only current Members of the Build Group [2] are eligible > to vote on this nomination.? Votes must be cast in the open by > replying to this mailing list > > For Lazy Consensus voting instructions, see [3]. > > /Magnus > > [1] > https://github.com/openjdk/jdk/search?q=author-name%3A%22Aleksey+Shipilev%22&type=commits > [2]http://openjdk.java.net/census#build > [3]http://openjdk.java.net/groups/#member-vote > From ihse at openjdk.java.net Wed Nov 10 14:24:39 2021 From: ihse at openjdk.java.net (Magnus Ihse Bursie) Date: Wed, 10 Nov 2021 14:24:39 GMT Subject: RFR: 8276864: Update boot JDKs to 17.0.1 in GHA In-Reply-To: References: Message-ID: On Tue, 9 Nov 2021 14:45:56 GMT, Aleksey Shipilev wrote: > Current GHA runs at 17 GA, which misses fixes from 17.0.1. > > Additional testing: > - [x] GHA builds are fine > - [x] GHA tests are fine Marked as reviewed by ihse (Reviewer). ------------- PR: https://git.openjdk.java.net/jdk/pull/6314 From ihse at openjdk.java.net Wed Nov 10 14:32:37 2021 From: ihse at openjdk.java.net (Magnus Ihse Bursie) Date: Wed, 10 Nov 2021 14:32:37 GMT Subject: RFR: 8276743: Make openjdk build Zip Archive generation "reproducible" In-Reply-To: References: <13DpyL4de49GOSlp9DU1co_rkznpvX920O6Z0RsmjbY=.23225459-16de-459d-b361-884d8b1d8d35@github.com> Message-ID: On Tue, 9 Nov 2021 17:28:39 GMT, Erik Joelsson wrote: >> This PR adds a new openjdk build tool GenerateZip, which generates a final "zip" file from an input folder, and creates it in a deterministic way, ensuring ordering and timestamps are set as specified. >> >> Using this tool in ZipArchive.gmk will ensure src.zip is then created deterministically. >> >> Signed-off-by: Andrew Leonard > > I agree that ideally reproducibility should be on by default, but if there is a cost, then you can be sure OpenJDK developers will be looking for a way to remove it for faster turnaround times. I would propose a specific configure parameter for this specific case, reproducible zip files, that is default on for release builds and off for debug builds (debug builds aren't reproducible by nature) and let the existing meta flag also control the value of this new flag. @erikj79 The flag --enable-reproducible-builds sets ENABLE_REPRODUCIBLE_BUILD in spec.gmk. This is set by our JIB profiles. I propose that we also turn it on for GHA builds. I think that the post-processing of the zip file can be dependent on this variable and that it serves no purpose to introduce a separate variable ENABLE_REPRODUCIBLE_ZIP that is set to the same value as ENABLE_REPRODUCIBLE_BUILD. Do you agree? ------------- PR: https://git.openjdk.java.net/jdk/pull/6311 From ihse at openjdk.java.net Wed Nov 10 14:37:40 2021 From: ihse at openjdk.java.net (Magnus Ihse Bursie) Date: Wed, 10 Nov 2021 14:37:40 GMT Subject: RFR: 8264130: PAC-RET protection for Linux/AArch64 In-Reply-To: References: Message-ID: On Wed, 10 Nov 2021 12:32:53 GMT, Alan Hayward wrote: > PAC is an optional feature in AArch64 8.3 and is compulsory in v9. One > of its uses is to protect against ROP based attacks. This is done by > signing the Link Register whenever it is stored on the stack, and > authenticating the value when it is loaded back from the stack. If an > attacker were to try to change control flow by editing the stack then > the authentication check of the Link Register will fail, causing a > segfault when the function returns. > > On a system with PAC enabled, it is expected that all applications will > be compiled with ROP protection. Fedora 33 and upwards already provide > this. By compiling for ARMv8.0, GCC and LLVM will only use the set of > PAC instructions that exist in the NOP space - on hardware without PAC, > these instructions act as NOPs, allowing backward compatibility for > negligible performance cost (2 NOPs per non-leaf function). > > Hardware is currently limited to the Apple M1 MacBooks. All testing has > been done within a Fedora Docker image. A run of SpecJVM showed no > difference to that of noise - which was surprising. > > The most important part of this patch is simply compiling using branch > protection provided by GCC/LLVM. This protects all C++ code from being > used in ROP attacks, removing all static ROP gadgets from use. > > The remainder of the patch adds ROP protection to runtime generated > code, in both stubs and compiled Java code. Attacks here are much harder > as ROP gadgets must be found dynamically at runtime. If/when AOT > compilation is added to JDK, then all stubs and compiled Java will be > susceptible ROP gadgets being found by static analysis and therefore > potentially as vulnerable as C++ code. > > There are a number of places where the VM changes control flow by > rewriting the stack or otherwise. I?ve done some analysis as to how > these could also be used for attacks (which I didn?t want to post here). > These areas can be protected ensuring the pointers to various stubs and > entry points are stored in memory as signed pointers. These changes are > simple to make (they can be reduced to a type change in common code and > a few addition sign/auth calls in the backend), but there a lot of them > and the total code change is fairly large. I?m happy to provide a few > work in progress patches. > > In order to match the security benefits of the Apple Arm64e ABI across > the whole of JDK, then all the changes mentioned above would be > required. Changes requested by ihse (Reviewer). make/autoconf/flags-cflags.m4 line 899: > 897: elif test "x$TOOLCHAIN_TYPE" = xgcc || test "x$TOOLCHAIN_TYPE" = xclang; then > 898: # Check that the compiler actually supports branch protection. > 899: FLAGS_COMPILER_CHECK_ARGUMENTS(ARGUMENT: [${BRANCH_PROTECTION_FLAG}], This branch misses a AC_MSG_RESULT, which prints the newline. The resulting output will look messy. ------------- PR: https://git.openjdk.java.net/jdk/pull/6334 From ysuenaga at openjdk.java.net Wed Nov 10 14:38:39 2021 From: ysuenaga at openjdk.java.net (Yasumasa Suenaga) Date: Wed, 10 Nov 2021 14:38:39 GMT Subject: Integrated: 8276672: Cannot build hsdis on WSL In-Reply-To: <4lLdxiPsFsB1GfI7iWSgLOXQ-Q_WU4ZZdOXHkRuO0kc=.45eea799-3e3d-4b1c-90dc-a6fa5e3d5b74@github.com> References: <4lLdxiPsFsB1GfI7iWSgLOXQ-Q_WU4ZZdOXHkRuO0kc=.45eea799-3e3d-4b1c-90dc-a6fa5e3d5b74@github.com> Message-ID: <3bpMNMo9XMvMG8uw3X5OPP3qPMoGmrlDckuo-OaL4UY=.1cf5330e-8c61-448c-a557-8cfc2c09154f@github.com> On Fri, 5 Nov 2021 03:03:55 GMT, Yasumasa Suenaga wrote: > JDK-8275128 introduced new Makefile for hsdis to integrate it to normal build system, however it does not work on WSL 1 Ubuntu 20.04 and MinGW from Ubuntu. > > Hsdis.gmk has two problems: > > 1. MinGW version is fixed to "9.2.0" > 2. Assumes sysroot for MinGW is located at /usr/$MINGW_BASE/sys-root > > I tested this change on WSL only, so it is appreciate to review on Cygwin. This pull request has now been integrated. Changeset: 38ec3a16 Author: Yasumasa Suenaga URL: https://git.openjdk.java.net/jdk/commit/38ec3a16d722d740d0b2128c6f6c2d1eab7a7c08 Stats: 34 lines in 1 file changed: 29 ins; 1 del; 4 mod 8276672: Cannot build hsdis on WSL Co-authored-by: Magnus Ihse Bursie Co-authored-by: Yasumasa Suenaga Reviewed-by: ihse, erikj ------------- PR: https://git.openjdk.java.net/jdk/pull/6269 From erikj at openjdk.java.net Wed Nov 10 14:42:37 2021 From: erikj at openjdk.java.net (Erik Joelsson) Date: Wed, 10 Nov 2021 14:42:37 GMT Subject: RFR: 8276743: Make openjdk build Zip Archive generation "reproducible" In-Reply-To: References: <13DpyL4de49GOSlp9DU1co_rkznpvX920O6Z0RsmjbY=.23225459-16de-459d-b361-884d8b1d8d35@github.com> Message-ID: On Wed, 10 Nov 2021 14:29:22 GMT, Magnus Ihse Bursie wrote: > I think that the post-processing of the zip file can be dependent on this variable and that it serves no purpose to introduce a separate variable ENABLE_REPRODUCIBLE_ZIP that is set to the same value as ENABLE_REPRODUCIBLE_BUILD. Do you agree? Sure, that works for me. ------------- PR: https://git.openjdk.java.net/jdk/pull/6311 From shade at openjdk.java.net Wed Nov 10 14:44:40 2021 From: shade at openjdk.java.net (Aleksey Shipilev) Date: Wed, 10 Nov 2021 14:44:40 GMT Subject: RFR: 8276864: Update boot JDKs to 17.0.1 in GHA In-Reply-To: References: Message-ID: <2rzY-kC-UV4uoLyxOSHOgmKXccOYZbXeDs6387CCxPM=.7a156544-9d06-4de7-9777-39384c527242@github.com> On Tue, 9 Nov 2021 14:45:56 GMT, Aleksey Shipilev wrote: > Current GHA runs at 17 GA, which misses fixes from 17.0.1. > > Additional testing: > - [x] GHA builds are fine > - [x] GHA tests are fine Ok, good, thanks. GHA are clean. I am integrating. ------------- PR: https://git.openjdk.java.net/jdk/pull/6314 From shade at openjdk.java.net Wed Nov 10 14:44:40 2021 From: shade at openjdk.java.net (Aleksey Shipilev) Date: Wed, 10 Nov 2021 14:44:40 GMT Subject: Integrated: 8276864: Update boot JDKs to 17.0.1 in GHA In-Reply-To: References: Message-ID: On Tue, 9 Nov 2021 14:45:56 GMT, Aleksey Shipilev wrote: > Current GHA runs at 17 GA, which misses fixes from 17.0.1. > > Additional testing: > - [x] GHA builds are fine > - [x] GHA tests are fine This pull request has now been integrated. Changeset: f561d3c1 Author: Aleksey Shipilev URL: https://git.openjdk.java.net/jdk/commit/f561d3c1942ce901fa77c907839032f76feb6998 Stats: 9 lines in 1 file changed: 0 ins; 0 del; 9 mod 8276864: Update boot JDKs to 17.0.1 in GHA Reviewed-by: erikj, ihse ------------- PR: https://git.openjdk.java.net/jdk/pull/6314 From duke at openjdk.java.net Wed Nov 10 15:04:39 2021 From: duke at openjdk.java.net (Alan Hayward) Date: Wed, 10 Nov 2021 15:04:39 GMT Subject: RFR: 8264130: PAC-RET protection for Linux/AArch64 In-Reply-To: References: Message-ID: On Wed, 10 Nov 2021 13:34:38 GMT, Andrew Haley wrote: >> PAC is an optional feature in AArch64 8.3 and is compulsory in v9. One >> of its uses is to protect against ROP based attacks. This is done by >> signing the Link Register whenever it is stored on the stack, and >> authenticating the value when it is loaded back from the stack. If an >> attacker were to try to change control flow by editing the stack then >> the authentication check of the Link Register will fail, causing a >> segfault when the function returns. >> >> On a system with PAC enabled, it is expected that all applications will >> be compiled with ROP protection. Fedora 33 and upwards already provide >> this. By compiling for ARMv8.0, GCC and LLVM will only use the set of >> PAC instructions that exist in the NOP space - on hardware without PAC, >> these instructions act as NOPs, allowing backward compatibility for >> negligible performance cost (2 NOPs per non-leaf function). >> >> Hardware is currently limited to the Apple M1 MacBooks. All testing has >> been done within a Fedora Docker image. A run of SpecJVM showed no >> difference to that of noise - which was surprising. >> >> The most important part of this patch is simply compiling using branch >> protection provided by GCC/LLVM. This protects all C++ code from being >> used in ROP attacks, removing all static ROP gadgets from use. >> >> The remainder of the patch adds ROP protection to runtime generated >> code, in both stubs and compiled Java code. Attacks here are much harder >> as ROP gadgets must be found dynamically at runtime. If/when AOT >> compilation is added to JDK, then all stubs and compiled Java will be >> susceptible ROP gadgets being found by static analysis and therefore >> potentially as vulnerable as C++ code. >> >> There are a number of places where the VM changes control flow by >> rewriting the stack or otherwise. I?ve done some analysis as to how >> these could also be used for attacks (which I didn?t want to post here). >> These areas can be protected ensuring the pointers to various stubs and >> entry points are stored in memory as signed pointers. These changes are >> simple to make (they can be reduced to a type change in common code and >> a few addition sign/auth calls in the backend), but there a lot of them >> and the total code change is fairly large. I?m happy to provide a few >> work in progress patches. >> >> In order to match the security benefits of the Apple Arm64e ABI across >> the whole of JDK, then all the changes mentioned above would be >> required. > > src/hotspot/os_cpu/bsd_aarch64/pauth_bsd_aarch64.inline.hpp line 25: > >> 23: */ >> 24: >> 25: #ifndef OS_CPU_BSD_AARCH64_PAUTH_BSD_AARCH64_INLINE_HPP > > Are these two files different enough to separate them for BSD and Linux? My motivation was to avoid having any ifdefs - but we need one anyway for the apple ifdef. If I merged the two we would end up with just the contents of the BSD version of the file. There is also the windows version of the file, which for now has empty functions. If PAC in windows is added, that'll either use the same code or maybe Windows will provide an API (like the Apple one). Merging everything would mean windows gains the UseROPProtection check. ------------- PR: https://git.openjdk.java.net/jdk/pull/6334 From volker.simonis at gmail.com Wed Nov 10 15:16:21 2021 From: volker.simonis at gmail.com (Volker Simonis) Date: Wed, 10 Nov 2021 16:16:21 +0100 Subject: CFV: New Build Group Member: Aleksey Shipilev In-Reply-To: <6911ff3e-67d5-0991-d0a0-08a06b1b09b2@oracle.com> References: <6911ff3e-67d5-0991-d0a0-08a06b1b09b2@oracle.com> Message-ID: Vote: yes Magnus Ihse Bursie schrieb am Mi., 10. Nov. 2021, 15:07: > I hereby nominate Aleksey Shipilev (shade) to Membership in the Build > Group. > > Aleksey is a long standing member of the OpenJDK Members Group and the > Hotspot > Group. He is a prolific contributor, and has committed more than 600 > changes to > the JDK over the years. Lately, Aleksey has made a number of improvement > to the > build system, which is indicated by the more than 140 commits made to > files in > the "make" directory since 2018. > > Votes are due by 24 November 2021, 16:00 CET. > > Only current Members of the Build Group [2] are eligible > to vote on this nomination. Votes must be cast in the open by > replying to this mailing list > > For Lazy Consensus voting instructions, see [3]. > > /Magnus > > [1] > https://github.com/openjdk/jdk/search?q=author-name%3A%22Aleksey+Shipilev%22&type=commits > [2]http://openjdk.java.net/census#build > [3]http://openjdk.java.net/groups/#member-vote > > From adinn at openjdk.java.net Wed Nov 10 15:27:41 2021 From: adinn at openjdk.java.net (Andrew Dinn) Date: Wed, 10 Nov 2021 15:27:41 GMT Subject: RFR: 8264130: PAC-RET protection for Linux/AArch64 In-Reply-To: References: Message-ID: On Wed, 10 Nov 2021 12:32:53 GMT, Alan Hayward wrote: > PAC is an optional feature in AArch64 8.3 and is compulsory in v9. One > of its uses is to protect against ROP based attacks. This is done by > signing the Link Register whenever it is stored on the stack, and > authenticating the value when it is loaded back from the stack. If an > attacker were to try to change control flow by editing the stack then > the authentication check of the Link Register will fail, causing a > segfault when the function returns. > > On a system with PAC enabled, it is expected that all applications will > be compiled with ROP protection. Fedora 33 and upwards already provide > this. By compiling for ARMv8.0, GCC and LLVM will only use the set of > PAC instructions that exist in the NOP space - on hardware without PAC, > these instructions act as NOPs, allowing backward compatibility for > negligible performance cost (2 NOPs per non-leaf function). > > Hardware is currently limited to the Apple M1 MacBooks. All testing has > been done within a Fedora Docker image. A run of SpecJVM showed no > difference to that of noise - which was surprising. > > The most important part of this patch is simply compiling using branch > protection provided by GCC/LLVM. This protects all C++ code from being > used in ROP attacks, removing all static ROP gadgets from use. > > The remainder of the patch adds ROP protection to runtime generated > code, in both stubs and compiled Java code. Attacks here are much harder > as ROP gadgets must be found dynamically at runtime. If/when AOT > compilation is added to JDK, then all stubs and compiled Java will be > susceptible ROP gadgets being found by static analysis and therefore > potentially as vulnerable as C++ code. > > There are a number of places where the VM changes control flow by > rewriting the stack or otherwise. I?ve done some analysis as to how > these could also be used for attacks (which I didn?t want to post here). > These areas can be protected ensuring the pointers to various stubs and > entry points are stored in memory as signed pointers. These changes are > simple to make (they can be reduced to a type change in common code and > a few addition sign/auth calls in the backend), but there a lot of them > and the total code change is fairly large. I?m happy to provide a few > work in progress patches. > > In order to match the security benefits of the Apple Arm64e ABI across > the whole of JDK, then all the changes mentioned above would be > required. I am also reviewing this. ------------- PR: https://git.openjdk.java.net/jdk/pull/6334 From duke at openjdk.java.net Wed Nov 10 16:03:34 2021 From: duke at openjdk.java.net (Alan Hayward) Date: Wed, 10 Nov 2021 16:03:34 GMT Subject: RFR: 8264130: PAC-RET protection for Linux/AArch64 In-Reply-To: References: Message-ID: On Wed, 10 Nov 2021 14:34:18 GMT, Magnus Ihse Bursie wrote: >> PAC is an optional feature in AArch64 8.3 and is compulsory in v9. One >> of its uses is to protect against ROP based attacks. This is done by >> signing the Link Register whenever it is stored on the stack, and >> authenticating the value when it is loaded back from the stack. If an >> attacker were to try to change control flow by editing the stack then >> the authentication check of the Link Register will fail, causing a >> segfault when the function returns. >> >> On a system with PAC enabled, it is expected that all applications will >> be compiled with ROP protection. Fedora 33 and upwards already provide >> this. By compiling for ARMv8.0, GCC and LLVM will only use the set of >> PAC instructions that exist in the NOP space - on hardware without PAC, >> these instructions act as NOPs, allowing backward compatibility for >> negligible performance cost (2 NOPs per non-leaf function). >> >> Hardware is currently limited to the Apple M1 MacBooks. All testing has >> been done within a Fedora Docker image. A run of SpecJVM showed no >> difference to that of noise - which was surprising. >> >> The most important part of this patch is simply compiling using branch >> protection provided by GCC/LLVM. This protects all C++ code from being >> used in ROP attacks, removing all static ROP gadgets from use. >> >> The remainder of the patch adds ROP protection to runtime generated >> code, in both stubs and compiled Java code. Attacks here are much harder >> as ROP gadgets must be found dynamically at runtime. If/when AOT >> compilation is added to JDK, then all stubs and compiled Java will be >> susceptible ROP gadgets being found by static analysis and therefore >> potentially as vulnerable as C++ code. >> >> There are a number of places where the VM changes control flow by >> rewriting the stack or otherwise. I?ve done some analysis as to how >> these could also be used for attacks (which I didn?t want to post here). >> These areas can be protected ensuring the pointers to various stubs and >> entry points are stored in memory as signed pointers. These changes are >> simple to make (they can be reduced to a type change in common code and >> a few addition sign/auth calls in the backend), but there a lot of them >> and the total code change is fairly large. I?m happy to provide a few >> work in progress patches. >> >> In order to match the security benefits of the Apple Arm64e ABI across >> the whole of JDK, then all the changes mentioned above would be >> required. > > make/autoconf/flags-cflags.m4 line 899: > >> 897: elif test "x$TOOLCHAIN_TYPE" = xgcc || test "x$TOOLCHAIN_TYPE" = xclang; then >> 898: # Check that the compiler actually supports branch protection. >> 899: FLAGS_COMPILER_CHECK_ARGUMENTS(ARGUMENT: [${BRANCH_PROTECTION_FLAG}], > > This branch misses a AC_MSG_RESULT, which prints the newline. The resulting output will look messy. Looking at this block of code again, I've got far too many outputted lines compared to other features. Removing some means I can simplify the code too, so I'll do that. ------------- PR: https://git.openjdk.java.net/jdk/pull/6334 From tim.bell at oracle.com Wed Nov 10 16:04:20 2021 From: tim.bell at oracle.com (tim.bell at oracle.com) Date: Wed, 10 Nov 2021 08:04:20 -0800 Subject: CFV: New Build Group Member: Aleksey Shipilev In-Reply-To: <6911ff3e-67d5-0991-d0a0-08a06b1b09b2@oracle.com> References: <6911ff3e-67d5-0991-d0a0-08a06b1b09b2@oracle.com> Message-ID: <60b8cda7-0ed5-8673-3836-d4d68333b4cd@oracle.com> Vote: yes Tim On 11/10/21 06:06, Magnus Ihse Bursie wrote: > I hereby nominate Aleksey Shipilev (shade) to Membership in the Build > Group. From jdv at openjdk.java.net Thu Nov 11 06:01:08 2021 From: jdv at openjdk.java.net (Jayathirth D V) Date: Thu, 11 Nov 2021 06:01:08 GMT Subject: RFR: 8276905: Function frag_col has a deployment target which is incompatible with this OS Message-ID: <_OkC1YeToScvoDOBBhAZfQHSao_fjjsnw60prsCsWtE=.6ea0dbc2-0270-4caa-bcfc-baa7cb704365@github.com> When JDK is built on BigSur with Xcode12.5 and used to run jtreg tests in macOS10.15 with Xcode <=12.4 we are getting compilation error. We dont set any deployment target when when we use xcrun to create .air file and this issue looks similar to https://developer.apple.com/forums/thread/105719. Also https://developer.apple.com/forums/thread/105719 states that even if they set app deployment target they need to explicitly set deployment in "xcrun metal". Made same change to use -mmacosx-version-min=10.12.00 in our make file. Whether we should keep 10.12.00 or 10.13/14 is open to discussion for metal pipeline. SwingSet2, J2DDemo & JCK/jtreg runs in CI are green. Also Vitaly(Submitter) has confirmed that patch resolves the issue. ------------- Commit messages: - 8276905: Function frag_col has a deployment target which is incompatible with this OS Changes: https://git.openjdk.java.net/jdk/pull/6346/files Webrev: https://webrevs.openjdk.java.net/?repo=jdk&pr=6346&range=00 Issue: https://bugs.openjdk.java.net/browse/JDK-8276905 Stats: 1 line in 1 file changed: 0 ins; 0 del; 1 mod Patch: https://git.openjdk.java.net/jdk/pull/6346.diff Fetch: git fetch https://git.openjdk.java.net/jdk pull/6346/head:pull/6346 PR: https://git.openjdk.java.net/jdk/pull/6346 From duke at openjdk.java.net Thu Nov 11 08:48:07 2021 From: duke at openjdk.java.net (Alan Hayward) Date: Thu, 11 Nov 2021 08:48:07 GMT Subject: RFR: 8264130: PAC-RET protection for Linux/AArch64 [v2] In-Reply-To: References: Message-ID: > PAC is an optional feature in AArch64 8.3 and is compulsory in v9. One > of its uses is to protect against ROP based attacks. This is done by > signing the Link Register whenever it is stored on the stack, and > authenticating the value when it is loaded back from the stack. If an > attacker were to try to change control flow by editing the stack then > the authentication check of the Link Register will fail, causing a > segfault when the function returns. > > On a system with PAC enabled, it is expected that all applications will > be compiled with ROP protection. Fedora 33 and upwards already provide > this. By compiling for ARMv8.0, GCC and LLVM will only use the set of > PAC instructions that exist in the NOP space - on hardware without PAC, > these instructions act as NOPs, allowing backward compatibility for > negligible performance cost (2 NOPs per non-leaf function). > > Hardware is currently limited to the Apple M1 MacBooks. All testing has > been done within a Fedora Docker image. A run of SpecJVM showed no > difference to that of noise - which was surprising. > > The most important part of this patch is simply compiling using branch > protection provided by GCC/LLVM. This protects all C++ code from being > used in ROP attacks, removing all static ROP gadgets from use. > > The remainder of the patch adds ROP protection to runtime generated > code, in both stubs and compiled Java code. Attacks here are much harder > as ROP gadgets must be found dynamically at runtime. If/when AOT > compilation is added to JDK, then all stubs and compiled Java will be > susceptible ROP gadgets being found by static analysis and therefore > potentially as vulnerable as C++ code. > > There are a number of places where the VM changes control flow by > rewriting the stack or otherwise. I?ve done some analysis as to how > these could also be used for attacks (which I didn?t want to post here). > These areas can be protected ensuring the pointers to various stubs and > entry points are stored in memory as signed pointers. These changes are > simple to make (they can be reduced to a type change in common code and > a few addition sign/auth calls in the backend), but there a lot of them > and the total code change is fairly large. I?m happy to provide a few > work in progress patches. > > In order to match the security benefits of the Apple Arm64e ABI across > the whole of JDK, then all the changes mentioned above would be > required. Alan Hayward has updated the pull request incrementally with one additional commit since the last revision: Simplify branch protection configure check ------------- Changes: - all: https://git.openjdk.java.net/jdk/pull/6334/files - new: https://git.openjdk.java.net/jdk/pull/6334/files/e0e3f666..29471d30 Webrevs: - full: https://webrevs.openjdk.java.net/?repo=jdk&pr=6334&range=01 - incr: https://webrevs.openjdk.java.net/?repo=jdk&pr=6334&range=00-01 Stats: 12 lines in 1 file changed: 0 ins; 6 del; 6 mod Patch: https://git.openjdk.java.net/jdk/pull/6334.diff Fetch: git fetch https://git.openjdk.java.net/jdk pull/6334/head:pull/6334 PR: https://git.openjdk.java.net/jdk/pull/6334 From nradomski at openjdk.java.net Thu Nov 11 10:16:58 2021 From: nradomski at openjdk.java.net (Niklas Radomski) Date: Thu, 11 Nov 2021 10:16:58 GMT Subject: RFR: 8276927: [PPC64] Port shenandoahgc to linux on ppc64le Message-ID: Port the Shenandoah garbage collector (JDK-8241457)[https://bugs.openjdk.java.net/browse/JDK-8241457] to linux on ppc64le. ------------- Commit messages: - Port shenandoahgc to linux on ppc64le Changes: https://git.openjdk.java.net/jdk/pull/6325/files Webrev: https://webrevs.openjdk.java.net/?repo=jdk&pr=6325&range=00 Issue: https://bugs.openjdk.java.net/browse/JDK-8276927 Stats: 1526 lines in 8 files changed: 1524 ins; 0 del; 2 mod Patch: https://git.openjdk.java.net/jdk/pull/6325.diff Fetch: git fetch https://git.openjdk.java.net/jdk pull/6325/head:pull/6325 PR: https://git.openjdk.java.net/jdk/pull/6325 From adinn at openjdk.java.net Thu Nov 11 11:21:37 2021 From: adinn at openjdk.java.net (Andrew Dinn) Date: Thu, 11 Nov 2021 11:21:37 GMT Subject: RFR: 8264130: PAC-RET protection for Linux/AArch64 [v2] In-Reply-To: References: Message-ID: On Wed, 10 Nov 2021 13:22:37 GMT, Andrew Haley wrote: >> Alan Hayward has updated the pull request incrementally with one additional commit since the last revision: >> >> Simplify branch protection configure check > > src/hotspot/cpu/aarch64/macroAssembler_aarch64.cpp line 5185: > >> 5183: // ROP Protection >> 5184: >> 5185: void MacroAssembler::protect_return_address() { > > We need proper, full, detailed comments about what these functions do, with reference to primary AArch64 documentation. As far as the AArch64 docs are concerned the relevant details are provided in ARM-ARM D - The PAC functionality is described in ARM-ARM Section D5.1.5 - Overview of the PAC instructions is provided in section C3.1.9 - Detailed PAC instruction descriptions are provided in C6.2.195 - C6.2.199 n.b. I am specifically referring to my (possibly out of date) copy ARM-DDI 0487D.a (ID103018) which is the Initial v8.4 EAC release from 2018. That said, I agree that a description of how these functions use the underlying PAC support and what, effectively, they achieve via that usage would be necessary. A reference to the relevant sections of the ARM doc in the code would be helpful. ------------- PR: https://git.openjdk.java.net/jdk/pull/6334 From adinn at openjdk.java.net Thu Nov 11 11:37:40 2021 From: adinn at openjdk.java.net (Andrew Dinn) Date: Thu, 11 Nov 2021 11:37:40 GMT Subject: RFR: 8264130: PAC-RET protection for Linux/AArch64 [v2] In-Reply-To: References: Message-ID: On Thu, 11 Nov 2021 11:19:03 GMT, Andrew Dinn wrote: >> src/hotspot/cpu/aarch64/macroAssembler_aarch64.cpp line 5185: >> >>> 5183: // ROP Protection >>> 5184: >>> 5185: void MacroAssembler::protect_return_address() { >> >> We need proper, full, detailed comments about what these functions do, with reference to primary AArch64 documentation. > > As far as the AArch64 docs are concerned the relevant details are provided in ARM-ARM D > > - The PAC functionality is described in ARM-ARM Section D5.1.5 > - Overview of the PAC instructions is provided in section C3.1.9 > - Detailed PAC instruction descriptions are provided in C6.2.195 - C6.2.199 > > n.b. I am specifically referring to my (possibly out of date) copy ARM-DDI 0487D.a (ID103018) which is the Initial v8.4 EAC release from 2018. > > That said, I agree that a description of how these functions use the underlying PAC support and what, effectively, they achieve via that usage would be necessary. A reference to the relevant sections of the ARM doc in the code would be helpful. Correction: Using the most up to date ARM ARM G [ARM DDI 0487G.a (ID011921)] - The PAC functionality is described in ARM-ARM Section D5.1.5 - Overview of the PAC instructions is provided in section C3.1.10 - Detailed PAC instruction descriptions are provided in C6.2.208 - C6.2.212 ------------- PR: https://git.openjdk.java.net/jdk/pull/6334 From duke at openjdk.java.net Thu Nov 11 11:47:36 2021 From: duke at openjdk.java.net (Alan Hayward) Date: Thu, 11 Nov 2021 11:47:36 GMT Subject: RFR: 8264130: PAC-RET protection for Linux/AArch64 [v2] In-Reply-To: References: Message-ID: On Thu, 11 Nov 2021 11:34:09 GMT, Andrew Dinn wrote: >> As far as the AArch64 docs are concerned the relevant details are provided in ARM-ARM D >> >> - The PAC functionality is described in ARM-ARM Section D5.1.5 >> - Overview of the PAC instructions is provided in section C3.1.9 >> - Detailed PAC instruction descriptions are provided in C6.2.195 - C6.2.199 >> >> n.b. I am specifically referring to my (possibly out of date) copy ARM-DDI 0487D.a (ID103018) which is the Initial v8.4 EAC release from 2018. >> >> That said, I agree that a description of how these functions use the underlying PAC support and what, effectively, they achieve via that usage would be necessary. A reference to the relevant sections of the ARM doc in the code would be helpful. > > Correction: > Using the most up to date ARM ARM G [ARM DDI 0487G.a (ID011921)] > > - The PAC functionality is described in ARM-ARM Section D5.1.5 > - Overview of the PAC instructions is provided in section C3.1.10 > - Detailed PAC instruction descriptions are provided in C6.2.208 - C6.2.212 I'm thinking for references to the Arm Arm to use header titles instead of section numbers, as the titles should be more stable. Also probably need some description around the code in the pauth_aarch64.hpp too. But I want to make sure I'm not duplicating comments - maybe the macroassembler comments should point to the pauth_aarch64 comments. It didn't seen common in the code to describe instruction functionality, which is why I didn't add any. Agreed it needs something added though. ------------- PR: https://git.openjdk.java.net/jdk/pull/6334 From aph at openjdk.java.net Thu Nov 11 11:55:34 2021 From: aph at openjdk.java.net (Andrew Haley) Date: Thu, 11 Nov 2021 11:55:34 GMT Subject: RFR: 8264130: PAC-RET protection for Linux/AArch64 [v2] In-Reply-To: References: Message-ID: <_9P-UvEbKu8NkBMq_pPr-_-muZxxHfwa2vV7h9nq6ZQ=.11e2de58-4e9e-4364-bfc9-c17791a19933@github.com> On Thu, 11 Nov 2021 11:44:09 GMT, Alan Hayward wrote: >> Correction: >> Using the most up to date ARM ARM G [ARM DDI 0487G.a (ID011921)] >> >> - The PAC functionality is described in ARM-ARM Section D5.1.5 >> - Overview of the PAC instructions is provided in section C3.1.10 >> - Detailed PAC instruction descriptions are provided in C6.2.208 - C6.2.212 > > I'm thinking for references to the Arm Arm to use header titles instead of section numbers, as the titles should be more stable. > > Also probably need some description around the code in the pauth_aarch64.hpp too. But I want to make sure I'm not duplicating comments - maybe the macroassembler comments should point to the pauth_aarch64 comments. > > It didn't seen common in the code to describe instruction functionality, which is why I didn't add any. Agreed it needs something added though. Yeah. At the definitions of `authenticate_return_address()` et al you can say what you expect in the normal case and what you expect when you've been hacked, along with an overview. I realize that it was a bit tricky to make this work with HotSpot because we're synthesizing return addresses just like hackers do, so a comment where we're patching return addresses would be nice. As long as the instructions are easily findable in the docs that's good. ------------- PR: https://git.openjdk.java.net/jdk/pull/6334 From aph at openjdk.java.net Thu Nov 11 11:59:35 2021 From: aph at openjdk.java.net (Andrew Haley) Date: Thu, 11 Nov 2021 11:59:35 GMT Subject: RFR: 8264130: PAC-RET protection for Linux/AArch64 [v2] In-Reply-To: <_9P-UvEbKu8NkBMq_pPr-_-muZxxHfwa2vV7h9nq6ZQ=.11e2de58-4e9e-4364-bfc9-c17791a19933@github.com> References: <_9P-UvEbKu8NkBMq_pPr-_-muZxxHfwa2vV7h9nq6ZQ=.11e2de58-4e9e-4364-bfc9-c17791a19933@github.com> Message-ID: On Thu, 11 Nov 2021 11:52:46 GMT, Andrew Haley wrote: >> I'm thinking for references to the Arm Arm to use header titles instead of section numbers, as the titles should be more stable. >> >> Also probably need some description around the code in the pauth_aarch64.hpp too. But I want to make sure I'm not duplicating comments - maybe the macroassembler comments should point to the pauth_aarch64 comments. >> >> It didn't seen common in the code to describe instruction functionality, which is why I didn't add any. Agreed it needs something added though. > > Yeah. At the definitions of `authenticate_return_address()` et al you can say what you expect in the normal case and what you expect when you've been hacked, along with an overview. I realize that it was a bit tricky to make this work with HotSpot because we're synthesizing return addresses just like hackers do, so a comment where we're patching return addresses would be nice. > > As long as the instructions are easily findable in the docs that's good. Just to be clear: no, don't describe instructions. describe what the macros do, and when to use them. Imagine that you, the reader can't see the contents of the macro at all, just the name and the comments. ------------- PR: https://git.openjdk.java.net/jdk/pull/6334 From ihse at openjdk.java.net Thu Nov 11 12:05:43 2021 From: ihse at openjdk.java.net (Magnus Ihse Bursie) Date: Thu, 11 Nov 2021 12:05:43 GMT Subject: RFR: 8264130: PAC-RET protection for Linux/AArch64 [v2] In-Reply-To: References: Message-ID: On Thu, 11 Nov 2021 08:48:07 GMT, Alan Hayward wrote: >> PAC is an optional feature in AArch64 8.3 and is compulsory in v9. One >> of its uses is to protect against ROP based attacks. This is done by >> signing the Link Register whenever it is stored on the stack, and >> authenticating the value when it is loaded back from the stack. If an >> attacker were to try to change control flow by editing the stack then >> the authentication check of the Link Register will fail, causing a >> segfault when the function returns. >> >> On a system with PAC enabled, it is expected that all applications will >> be compiled with ROP protection. Fedora 33 and upwards already provide >> this. By compiling for ARMv8.0, GCC and LLVM will only use the set of >> PAC instructions that exist in the NOP space - on hardware without PAC, >> these instructions act as NOPs, allowing backward compatibility for >> negligible performance cost (2 NOPs per non-leaf function). >> >> Hardware is currently limited to the Apple M1 MacBooks. All testing has >> been done within a Fedora Docker image. A run of SpecJVM showed no >> difference to that of noise - which was surprising. >> >> The most important part of this patch is simply compiling using branch >> protection provided by GCC/LLVM. This protects all C++ code from being >> used in ROP attacks, removing all static ROP gadgets from use. >> >> The remainder of the patch adds ROP protection to runtime generated >> code, in both stubs and compiled Java code. Attacks here are much harder >> as ROP gadgets must be found dynamically at runtime. If/when AOT >> compilation is added to JDK, then all stubs and compiled Java will be >> susceptible ROP gadgets being found by static analysis and therefore >> potentially as vulnerable as C++ code. >> >> There are a number of places where the VM changes control flow by >> rewriting the stack or otherwise. I?ve done some analysis as to how >> these could also be used for attacks (which I didn?t want to post here). >> These areas can be protected ensuring the pointers to various stubs and >> entry points are stored in memory as signed pointers. These changes are >> simple to make (they can be reduced to a type change in common code and >> a few addition sign/auth calls in the backend), but there a lot of them >> and the total code change is fairly large. I?m happy to provide a few >> work in progress patches. >> >> In order to match the security benefits of the Apple Arm64e ABI across >> the whole of JDK, then all the changes mentioned above would be >> required. > > Alan Hayward has updated the pull request incrementally with one additional commit since the last revision: > > Simplify branch protection configure check Build changes look much better now, thanks! Build part approved; the actual code changes needs approval from others. ------------- Marked as reviewed by ihse (Reviewer). PR: https://git.openjdk.java.net/jdk/pull/6334 From rkennke at openjdk.java.net Thu Nov 11 12:10:41 2021 From: rkennke at openjdk.java.net (Roman Kennke) Date: Thu, 11 Nov 2021 12:10:41 GMT Subject: RFR: 8276927: [PPC64] Port shenandoahgc to linux on ppc64le In-Reply-To: References: Message-ID: On Wed, 10 Nov 2021 09:00:04 GMT, Niklas Radomski wrote: > Port the Shenandoah garbage collector (JDK-8241457)[https://bugs.openjdk.java.net/browse/JDK-8241457] to linux on ppc64le. Hi Niklas, thanks for this awesome work! I can't really comment on the actual PPC code, so this needs to be reviewed by somebody else. Structurally the change looks correct. I have one comment about the C1 CAS barrier code, but it's minor. Thanks & cheers, Roman src/hotspot/cpu/ppc/gc/shenandoah/c1/shenandoahBarrierSetC1_ppc.cpp line 83: > 81: LIRGenerator* gen = access.gen(); > 82: > 83: if (ShenandoahCASBarrier) { I am not sure, but I almost think we should not even end up in the method with -ShenandoahCASBarrier. If anything, -ShenandoahCASBarrier should result in only calling super to emit regular CAS without any barriers. ------------- Marked as reviewed by rkennke (Reviewer). PR: https://git.openjdk.java.net/jdk/pull/6325 From ihse at openjdk.java.net Thu Nov 11 12:21:32 2021 From: ihse at openjdk.java.net (Magnus Ihse Bursie) Date: Thu, 11 Nov 2021 12:21:32 GMT Subject: RFR: 8276927: [PPC64] Port shenandoahgc to linux on ppc64le In-Reply-To: References: Message-ID: On Wed, 10 Nov 2021 09:00:04 GMT, Niklas Radomski wrote: > Port the Shenandoah garbage collector (JDK-8241457)[https://bugs.openjdk.java.net/browse/JDK-8241457] to linux on ppc64le. Build changes look good. Actual code changes needs to be reviewed by someone more knowledgable about this area. ------------- Marked as reviewed by ihse (Reviewer). PR: https://git.openjdk.java.net/jdk/pull/6325 From mcimadamore at openjdk.java.net Thu Nov 11 13:03:56 2021 From: mcimadamore at openjdk.java.net (Maurizio Cimadamore) Date: Thu, 11 Nov 2021 13:03:56 GMT Subject: RFR: 8275063: Implementation of Foreign Function & Memory API (Second incubator) [v23] In-Reply-To: References: Message-ID: > This PR contains the API and implementation changes for JEP-419 [1]. A more detailed description of such changes, to avoid repetitions during the review process, is included as a separate comment. > > [1] - https://openjdk.java.net/jeps/419 Maurizio Cimadamore has updated the pull request with a new target base due to a merge or a rebase. The pull request now contains 32 commits: - Merge branch 'master' into JEP-419 - Revert removal of upcall MH customization (This change caused spurious VM crashes, so reverting to baseline) - Further tweak upcall safety considerations - Clarify safety considerations for upcalls - Rename MemorySegment::ofAddressNative to MemorySegment::ofAddress (which is consistent with other restricted factories in VaList and NativeSymbol) - Streamline javadoc for package-info - * Add two new CLinker static methods to compute upcall/downcall method types * Clarify section on CLinker downcall type * Add section on CLinker safety guarantees - Fix TestUpcall * reverse() has a bug, as it doesn't tweak parameter types * reverse() is applied to the wrong MH - Make ArenaAllocator impl more flexible in the face of OOME An ArenaAllocator should remain open for business, even if OOME is thrown in case other allocations can fit the arena size. - Simplify ArenaAllocator impl. The arena should respect its boundaries and never allocate more memory than its size specifies. - ... and 22 more: https://git.openjdk.java.net/jdk/compare/aea09677...8c3860f8 ------------- Changes: https://git.openjdk.java.net/jdk/pull/5907/files Webrev: https://webrevs.openjdk.java.net/?repo=jdk&pr=5907&range=22 Stats: 14686 lines in 193 files changed: 6956 ins; 5120 del; 2610 mod Patch: https://git.openjdk.java.net/jdk/pull/5907.diff Fetch: git fetch https://git.openjdk.java.net/jdk pull/5907/head:pull/5907 PR: https://git.openjdk.java.net/jdk/pull/5907 From ihse at openjdk.java.net Thu Nov 11 13:48:40 2021 From: ihse at openjdk.java.net (Magnus Ihse Bursie) Date: Thu, 11 Nov 2021 13:48:40 GMT Subject: RFR: 8275745: Reproducible copyright headers In-Reply-To: References: Message-ID: On Sun, 24 Oct 2021 12:11:57 GMT, Emmanuel Bourg wrote: >> The copyright headers are generated at build time, and the year inserted in the template depends on the current date. This means the headers are not reproducible if the project is built a year later. The year in the headers could be derived from the SOURCE_DATE_EPOCH environment variable to make the build reproducible (this variable is already used in other parts of the build). > > Yes that's fine. I guess this involves setting the `COPYRIGHT_YEAR` variable in `make/autoconf/jdk-options.m4` to a value derived from `SOURCE_DATE_EPOCH` (with `SOURCE_DATE_EPOCH` having the priority over `--with-copyright-year` or the opposite?), and then pick the value of `COPYRIGHT_YEAR` in `CopyrightHeaders.java` and `EquivMapsGenerator.java`, right? @ebourg I have now modified this patch so it uses COPYRIGHT_YEAR, and sets COPYRIGHT_YEAR based on SOURCE_DATE_EPOCH, if it exists. The patch is in a branch in my personal fork, https://github.com/magicus/jdk/tree/reproducible-copyright-year. If you think it looks good, we have two possible ways forward. Either you close this PR, and I open a new targeting the same JBS issue, and credit you as co-author. Or you integrate my branch into this PR, and credit me as co-author. Any of them is OK for me, but I think the former is simpler. ------------- PR: https://git.openjdk.java.net/jdk/pull/1498 From ihse at openjdk.java.net Thu Nov 11 13:56:34 2021 From: ihse at openjdk.java.net (Magnus Ihse Bursie) Date: Thu, 11 Nov 2021 13:56:34 GMT Subject: RFR: 8276905: Function frag_col has a deployment target which is incompatible with this OS In-Reply-To: <_OkC1YeToScvoDOBBhAZfQHSao_fjjsnw60prsCsWtE=.6ea0dbc2-0270-4caa-bcfc-baa7cb704365@github.com> References: <_OkC1YeToScvoDOBBhAZfQHSao_fjjsnw60prsCsWtE=.6ea0dbc2-0270-4caa-bcfc-baa7cb704365@github.com> Message-ID: On Thu, 11 Nov 2021 05:52:18 GMT, Jayathirth D V wrote: > When JDK is built on BigSur with Xcode12.5 and used to run jtreg tests in macOS10.15 with Xcode <=12.4 we are getting compilation error. We dont set any deployment target when when we use xcrun to create .air file and this issue looks similar to https://developer.apple.com/forums/thread/105719. Also https://developer.apple.com/forums/thread/105719 states that even if they set app deployment target they need to explicitly set deployment in "xcrun metal". > > Made same change to use -mmacosx-version-min=10.12.00 in our make file. Whether we should keep 10.12.00 or 10.13/14 is open to discussion for metal pipeline. > > SwingSet2, J2DDemo & JCK/jtreg runs in CI are green. Also Vitaly(Submitter) has confirmed that patch resolves the issue. We should not hard-code version numbers like that. Fortunately for you, there already exists a variable $(MACOSX_VERSION_MIN) that you can use. Also, if you did not create this patch yourself, please make sure to use `/contributor` to give proper credits. Or maybe you mean that Vitaly submitted the bug report, not the patch? ------------- Changes requested by ihse (Reviewer). PR: https://git.openjdk.java.net/jdk/pull/6346 From fweimer at openjdk.java.net Thu Nov 11 14:23:43 2021 From: fweimer at openjdk.java.net (Florian Weimer) Date: Thu, 11 Nov 2021 14:23:43 GMT Subject: RFR: 8264130: PAC-RET protection for Linux/AArch64 [v2] In-Reply-To: References: Message-ID: On Thu, 11 Nov 2021 08:48:07 GMT, Alan Hayward wrote: >> PAC is an optional feature in AArch64 8.3 and is compulsory in v9. One >> of its uses is to protect against ROP based attacks. This is done by >> signing the Link Register whenever it is stored on the stack, and >> authenticating the value when it is loaded back from the stack. If an >> attacker were to try to change control flow by editing the stack then >> the authentication check of the Link Register will fail, causing a >> segfault when the function returns. >> >> On a system with PAC enabled, it is expected that all applications will >> be compiled with ROP protection. Fedora 33 and upwards already provide >> this. By compiling for ARMv8.0, GCC and LLVM will only use the set of >> PAC instructions that exist in the NOP space - on hardware without PAC, >> these instructions act as NOPs, allowing backward compatibility for >> negligible performance cost (2 NOPs per non-leaf function). >> >> Hardware is currently limited to the Apple M1 MacBooks. All testing has >> been done within a Fedora Docker image. A run of SpecJVM showed no >> difference to that of noise - which was surprising. >> >> The most important part of this patch is simply compiling using branch >> protection provided by GCC/LLVM. This protects all C++ code from being >> used in ROP attacks, removing all static ROP gadgets from use. >> >> The remainder of the patch adds ROP protection to runtime generated >> code, in both stubs and compiled Java code. Attacks here are much harder >> as ROP gadgets must be found dynamically at runtime. If/when AOT >> compilation is added to JDK, then all stubs and compiled Java will be >> susceptible ROP gadgets being found by static analysis and therefore >> potentially as vulnerable as C++ code. >> >> There are a number of places where the VM changes control flow by >> rewriting the stack or otherwise. I?ve done some analysis as to how >> these could also be used for attacks (which I didn?t want to post here). >> These areas can be protected ensuring the pointers to various stubs and >> entry points are stored in memory as signed pointers. These changes are >> simple to make (they can be reduced to a type change in common code and >> a few addition sign/auth calls in the backend), but there a lot of them >> and the total code change is fairly large. I?m happy to provide a few >> work in progress patches. >> >> In order to match the security benefits of the Apple Arm64e ABI across >> the whole of JDK, then all the changes mentioned above would be >> required. > > Alan Hayward has updated the pull request incrementally with one additional commit since the last revision: > > Simplify branch protection configure check Is the code still mapped read-write all the time? src/hotspot/cpu/aarch64/globals_aarch64.hpp line 115: > 113: range(-1, 4096) \ > 114: product(bool, UseROPProtection, false, \ > 115: "Use ROP based branch protection") \ The description is not correct. It's protection against certain ROP-based attack techniques. ------------- PR: https://git.openjdk.java.net/jdk/pull/6334 From mdoerr at openjdk.java.net Thu Nov 11 14:34:39 2021 From: mdoerr at openjdk.java.net (Martin Doerr) Date: Thu, 11 Nov 2021 14:34:39 GMT Subject: RFR: 8276927: [PPC64] Port shenandoahgc to linux on ppc64le In-Reply-To: References: Message-ID: On Wed, 10 Nov 2021 09:00:04 GMT, Niklas Radomski wrote: > Port the Shenandoah garbage collector (JDK-8241457)[https://bugs.openjdk.java.net/browse/JDK-8241457] to linux on ppc64le. Nice work! Looks correct. For others: Note that this change already contains feedback from my offline review. src/hotspot/cpu/ppc/gc/shenandoah/shenandoahBarrierSetAssembler_ppc.cpp line 74: > 72: // IU barriers are also employed to avoid resurrection of weak references, > 73: // even if Shenandoah does not operate in incremental update mode. > 74: if (ShenandoahIUBarrier || ShenandoahSATBBarrier) { Sharing the code for IU and SATB sounds like a good idea, but one needs to be careful. `ShenandoahBarrierSetC1::iu_barrier` only works with `ShenandoahIUBarrier`, so this trick can't be used in C1. It's a bit confusing, but I'm ok with this version. At least, I don't have any better suggestion at the moment. ------------- Marked as reviewed by mdoerr (Reviewer). PR: https://git.openjdk.java.net/jdk/pull/6325 From mdoerr at openjdk.java.net Thu Nov 11 14:34:40 2021 From: mdoerr at openjdk.java.net (Martin Doerr) Date: Thu, 11 Nov 2021 14:34:40 GMT Subject: RFR: 8276927: [PPC64] Port shenandoahgc to linux on ppc64le In-Reply-To: References: Message-ID: On Thu, 11 Nov 2021 11:32:49 GMT, Roman Kennke wrote: >> Port the Shenandoah garbage collector (JDK-8241457)[https://bugs.openjdk.java.net/browse/JDK-8241457] to linux on ppc64le. > > src/hotspot/cpu/ppc/gc/shenandoah/c1/shenandoahBarrierSetC1_ppc.cpp line 83: > >> 81: LIRGenerator* gen = access.gen(); >> 82: >> 83: if (ShenandoahCASBarrier) { > > I am not sure, but I almost think we should not even end up in the method with -ShenandoahCASBarrier. If anything, -ShenandoahCASBarrier should result in only calling super to emit regular CAS without any barriers. We hit this case when running `jdk/bin/java -XX:+UseShenandoahGC -XX:ShenandoahGCMode=passive -version`. x86 and aarch64 check for ShenandoahCASBarrier, too. So, looks like these checks are needed and correct. ------------- PR: https://git.openjdk.java.net/jdk/pull/6325 From ihse at openjdk.java.net Thu Nov 11 14:38:49 2021 From: ihse at openjdk.java.net (Magnus Ihse Bursie) Date: Thu, 11 Nov 2021 14:38:49 GMT Subject: RFR: 8277012: Use blessed modifier order in src/utils Message-ID: I ran bin/blessed-modifier-order.sh on source code in src/utils. This scripts verifies that modifiers are in the "blessed" order, and fixes it otherwise. I have manually checked the changes made by the script to make sure they are sound. There are no clear ownership of this code, but I believe it's kind of hotspot-related. ------------- Commit messages: - 8277012: Use blessed modifier order in src/utils Changes: https://git.openjdk.java.net/jdk/pull/6354/files Webrev: https://webrevs.openjdk.java.net/?repo=jdk&pr=6354&range=00 Issue: https://bugs.openjdk.java.net/browse/JDK-8277012 Stats: 25 lines in 10 files changed: 0 ins; 0 del; 25 mod Patch: https://git.openjdk.java.net/jdk/pull/6354.diff Fetch: git fetch https://git.openjdk.java.net/jdk pull/6354/head:pull/6354 PR: https://git.openjdk.java.net/jdk/pull/6354 From adinn at openjdk.java.net Thu Nov 11 14:46:43 2021 From: adinn at openjdk.java.net (Andrew Dinn) Date: Thu, 11 Nov 2021 14:46:43 GMT Subject: RFR: 8264130: PAC-RET protection for Linux/AArch64 [v2] In-Reply-To: References: Message-ID: On Thu, 11 Nov 2021 14:20:20 GMT, Florian Weimer wrote: >> Alan Hayward has updated the pull request incrementally with one additional commit since the last revision: >> >> Simplify branch protection configure check > > src/hotspot/cpu/aarch64/globals_aarch64.hpp line 115: > >> 113: range(-1, 4096) \ >> 114: product(bool, UseROPProtection, false, \ >> 115: "Use ROP based branch protection") \ > > The description is not correct. It's protection against certain ROP-based attack techniques. I don't agree that this is incorrect, at least not for the stated reason. The flag switches on a protection mechanism that guards against ROP attacks. To my reading that does not imply it guards against all such attacks, merely that this is the nature of the protection it offers. The description might still be considered incorrect for an unrelated reason. Its use of the adjectival phrase ROP based constitutes a transferred epithet, conflating the symptom with the medicine. In other words, the protection offered is not ROP based i.e. does not rely on an ROP technique. What it does is protect against ROP attacks. So, I'd suggest rewording to "Enable protection of branches against ROP attacks". Florian, if you want to argue for rewording that to "Enable protection of branches against some categories of ROP attacks" or some other equivalently qualified variant please feel free to make a case. However, I don't think see any need to add that rider, nor any precedent in any of the other short descriptions provided in globals.hpp. ------------- PR: https://git.openjdk.java.net/jdk/pull/6334 From adinn at openjdk.java.net Thu Nov 11 14:56:46 2021 From: adinn at openjdk.java.net (Andrew Dinn) Date: Thu, 11 Nov 2021 14:56:46 GMT Subject: RFR: 8264130: PAC-RET protection for Linux/AArch64 [v2] In-Reply-To: References: Message-ID: On Thu, 11 Nov 2021 14:20:33 GMT, Florian Weimer wrote: > Is the code still mapped read-write all the time? That depends on what code you mean. The JVM code compiled from C++ sources is mapped RO(X) in the text section like any compiled C/C++ code. Protection of that code is covered by the changes to the build system. The runtime generated runtime stubs and Java method code into which this patch may insert the required PAC instructions are written into a code cache in a section which is mapped RW(X) all the time. It would be hard to map even a subset of this code cache RO because generated code includes call and data sites that need to be patched during execution. ------------- PR: https://git.openjdk.java.net/jdk/pull/6334 From fweimer at openjdk.java.net Thu Nov 11 14:56:46 2021 From: fweimer at openjdk.java.net (Florian Weimer) Date: Thu, 11 Nov 2021 14:56:46 GMT Subject: RFR: 8264130: PAC-RET protection for Linux/AArch64 [v2] In-Reply-To: References: Message-ID: On Thu, 11 Nov 2021 14:43:59 GMT, Andrew Dinn wrote: >> src/hotspot/cpu/aarch64/globals_aarch64.hpp line 115: >> >>> 113: range(-1, 4096) \ >>> 114: product(bool, UseROPProtection, false, \ >>> 115: "Use ROP based branch protection") \ >> >> The description is not correct. It's protection against certain ROP-based attack techniques. > > I don't agree that this is incorrect, at least not for the stated reason. The flag switches on a protection mechanism that guards against ROP attacks. To my reading that does not imply it guards against all such attacks, merely that this is the nature of the protection it offers. > > The description might still be considered incorrect for an unrelated reason. Its use of the adjectival phrase ROP based constitutes a transferred epithet, conflating the symptom with the medicine. In other words, the protection offered is not ROP based i.e. does not rely on an ROP technique. What it does is protect against ROP attacks. So, I'd suggest rewording to > > "Enable protection of branches against ROP attacks". > > Florian, if you want to argue for rewording that to "Enable protection of branches against some categories of ROP attacks" or some other equivalently qualified variant please feel free to make a case. However, I don't think see any need to add that rider, nor any precedent in any of the other short descriptions provided in globals.hpp. I did mean the description, not the flag name. ------------- PR: https://git.openjdk.java.net/jdk/pull/6334 From adinn at openjdk.java.net Thu Nov 11 15:02:39 2021 From: adinn at openjdk.java.net (Andrew Dinn) Date: Thu, 11 Nov 2021 15:02:39 GMT Subject: RFR: 8264130: PAC-RET protection for Linux/AArch64 [v2] In-Reply-To: References: Message-ID: On Thu, 11 Nov 2021 14:53:54 GMT, Florian Weimer wrote: >> I don't agree that this is incorrect, at least not for the stated reason. The flag switches on a protection mechanism that guards against ROP attacks. To my reading that does not imply it guards against all such attacks, merely that this is the nature of the protection it offers. >> >> The description might still be considered incorrect for an unrelated reason. Its use of the adjectival phrase ROP based constitutes a transferred epithet, conflating the symptom with the medicine. In other words, the protection offered is not ROP based i.e. does not rely on an ROP technique. What it does is protect against ROP attacks. So, I'd suggest rewording to >> >> "Enable protection of branches against ROP attacks". >> >> Florian, if you want to argue for rewording that to "Enable protection of branches against some categories of ROP attacks" or some other equivalently qualified variant please feel free to make a case. However, I don't think see any need to add that rider, nor any precedent in any of the other short descriptions provided in globals.hpp. > > I did mean the description, not the flag name. Yes, understood. I too was talking about the description even though I introduced my comment by talking about what the flag does. ------------- PR: https://git.openjdk.java.net/jdk/pull/6334 From rkennke at openjdk.java.net Thu Nov 11 15:04:36 2021 From: rkennke at openjdk.java.net (Roman Kennke) Date: Thu, 11 Nov 2021 15:04:36 GMT Subject: RFR: 8276927: [PPC64] Port shenandoahgc to linux on ppc64le In-Reply-To: References: Message-ID: On Thu, 11 Nov 2021 14:30:05 GMT, Martin Doerr wrote: >> src/hotspot/cpu/ppc/gc/shenandoah/c1/shenandoahBarrierSetC1_ppc.cpp line 83: >> >>> 81: LIRGenerator* gen = access.gen(); >>> 82: >>> 83: if (ShenandoahCASBarrier) { >> >> I am not sure, but I almost think we should not even end up in the method with -ShenandoahCASBarrier. If anything, -ShenandoahCASBarrier should result in only calling super to emit regular CAS without any barriers. > > We hit this case when running `jdk/bin/java -XX:+UseShenandoahGC -XX:ShenandoahGCMode=passive -version`. x86 and aarch64 check for ShenandoahCASBarrier, too. So, looks like these checks are needed and correct. Ok then. ------------- PR: https://git.openjdk.java.net/jdk/pull/6325 From duke at openjdk.java.net Thu Nov 11 15:15:59 2021 From: duke at openjdk.java.net (Emmanuel Bourg) Date: Thu, 11 Nov 2021 15:15:59 GMT Subject: RFR: 8275745: Reproducible copyright headers [v2] In-Reply-To: References: Message-ID: > The copyright headers are generated at build time, and the year inserted in the template depends on the current date. This means the headers are not reproducible if the project is built a year later. The year in the headers could be derived from the SOURCE_DATE_EPOCH environment variable to make the build reproducible (this variable is already used in other parts of the build). Emmanuel Bourg has updated the pull request incrementally with two additional commits since the last revision: - Make gensrc code use $COPYRIGHT_YEAR - Set COPYRIGHT_YEAR from SOURCE_DATE_EPOCH if present ------------- Changes: - all: https://git.openjdk.java.net/jdk/pull/1498/files - new: https://git.openjdk.java.net/jdk/pull/1498/files/99161710..9c4efbda Webrevs: - full: https://webrevs.openjdk.java.net/?repo=jdk&pr=1498&range=01 - incr: https://webrevs.openjdk.java.net/?repo=jdk&pr=1498&range=00-01 Stats: 59 lines in 8 files changed: 28 ins; 22 del; 9 mod Patch: https://git.openjdk.java.net/jdk/pull/1498.diff Fetch: git fetch https://git.openjdk.java.net/jdk pull/1498/head:pull/1498 PR: https://git.openjdk.java.net/jdk/pull/1498 From ihse at openjdk.java.net Thu Nov 11 15:16:00 2021 From: ihse at openjdk.java.net (Magnus Ihse Bursie) Date: Thu, 11 Nov 2021 15:16:00 GMT Subject: RFR: 8275745: Reproducible copyright headers [v2] In-Reply-To: References: Message-ID: <4kEip7T4tvb3ztV_yLB8az_H42dYodVOIWlHic1MVzI=.0db6c4c0-264e-4007-9bd4-69e093505531@github.com> On Thu, 11 Nov 2021 15:12:21 GMT, Emmanuel Bourg wrote: >> The copyright headers are generated at build time, and the year inserted in the template depends on the current date. This means the headers are not reproducible if the project is built a year later. The year in the headers could be derived from the SOURCE_DATE_EPOCH environment variable to make the build reproducible (this variable is already used in other parts of the build). > > Emmanuel Bourg has updated the pull request incrementally with two additional commits since the last revision: > > - Make gensrc code use $COPYRIGHT_YEAR > - Set COPYRIGHT_YEAR from SOURCE_DATE_EPOCH if present LGTM but then again I wrote most of it. :) Please await further reviewers before integrating. ------------- Marked as reviewed by ihse (Reviewer). PR: https://git.openjdk.java.net/jdk/pull/1498 From duke at openjdk.java.net Thu Nov 11 15:16:00 2021 From: duke at openjdk.java.net (Emmanuel Bourg) Date: Thu, 11 Nov 2021 15:16:00 GMT Subject: RFR: 8275745: Reproducible copyright headers In-Reply-To: References: Message-ID: On Thu, 11 Nov 2021 13:45:21 GMT, Magnus Ihse Bursie wrote: >> Yes that's fine. I guess this involves setting the `COPYRIGHT_YEAR` variable in `make/autoconf/jdk-options.m4` to a value derived from `SOURCE_DATE_EPOCH` (with `SOURCE_DATE_EPOCH` having the priority over `--with-copyright-year` or the opposite?), and then pick the value of `COPYRIGHT_YEAR` in `CopyrightHeaders.java` and `EquivMapsGenerator.java`, right? > > @ebourg I have now modified this patch so it uses COPYRIGHT_YEAR, and sets COPYRIGHT_YEAR based on SOURCE_DATE_EPOCH, if it exists. The patch is in a branch in my personal fork, https://github.com/magicus/jdk/tree/reproducible-copyright-year. > > If you think it looks good, we have two possible ways forward. Either you close this PR, and I open a new targeting the same JBS issue, and credit you as co-author. Or you integrate my branch into this PR, and credit me as co-author. Any of them is OK for me, but I think the former is simpler. @magicus Thank you! I've integrated your branch ------------- PR: https://git.openjdk.java.net/jdk/pull/1498 From duke at openjdk.java.net Thu Nov 11 15:16:04 2021 From: duke at openjdk.java.net (Emmanuel Bourg) Date: Thu, 11 Nov 2021 15:16:04 GMT Subject: RFR: 8275745: Reproducible copyright headers In-Reply-To: References: Message-ID: <3mIE2DgV8DxgUeNPyefE_IcZh24oouYjehfhot8xNsw=.c6587525-d295-4174-a8b7-f193f88a1cc2@github.com> On Sat, 28 Nov 2020 23:14:35 GMT, Emmanuel Bourg wrote: > The copyright headers are generated at build time, and the year inserted in the template depends on the current date. This means the headers are not reproducible if the project is built a year later. The year in the headers could be derived from the SOURCE_DATE_EPOCH environment variable to make the build reproducible (this variable is already used in other parts of the build). I can rebase the branch with your changes only, my commit just adds noise. ------------- PR: https://git.openjdk.java.net/jdk/pull/1498 From aleonard at openjdk.java.net Thu Nov 11 15:22:11 2021 From: aleonard at openjdk.java.net (Andrew Leonard) Date: Thu, 11 Nov 2021 15:22:11 GMT Subject: RFR: 8276743: Make openjdk build Zip Archive generation "reproducible" [v2] In-Reply-To: <13DpyL4de49GOSlp9DU1co_rkznpvX920O6Z0RsmjbY=.23225459-16de-459d-b361-884d8b1d8d35@github.com> References: <13DpyL4de49GOSlp9DU1co_rkznpvX920O6Z0RsmjbY=.23225459-16de-459d-b361-884d8b1d8d35@github.com> Message-ID: > This PR adds a new openjdk build tool GenerateZip, which generates a final "zip" file from an input folder, and creates it in a deterministic way, ensuring ordering and timestamps are set as specified. > > Using this tool in ZipArchive.gmk will ensure src.zip is then created deterministically. > > Signed-off-by: Andrew Leonard Andrew Leonard has updated the pull request incrementally with one additional commit since the last revision: 8276743: Make openjdk build Zip Archive generation "reproducible" Signed-off-by: Andrew Leonard ------------- Changes: - all: https://git.openjdk.java.net/jdk/pull/6311/files - new: https://git.openjdk.java.net/jdk/pull/6311/files/dcf48d65..f8a816af Webrevs: - full: https://webrevs.openjdk.java.net/?repo=jdk&pr=6311&range=01 - incr: https://webrevs.openjdk.java.net/?repo=jdk&pr=6311&range=00-01 Stats: 544 lines in 5 files changed: 241 ins; 287 del; 16 mod Patch: https://git.openjdk.java.net/jdk/pull/6311.diff Fetch: git fetch https://git.openjdk.java.net/jdk pull/6311/head:pull/6311 PR: https://git.openjdk.java.net/jdk/pull/6311 From aleonard at openjdk.java.net Thu Nov 11 15:22:16 2021 From: aleonard at openjdk.java.net (Andrew Leonard) Date: Thu, 11 Nov 2021 15:22:16 GMT Subject: RFR: 8276743: Make openjdk build Zip Archive generation "reproducible" [v2] In-Reply-To: <8SDOXwz8OhlhhqpWHjDbdxj9M1ZQfQcls2aRNt-cukQ=.f177d1e3-b30e-4e8b-ae35-d27606e18a5e@github.com> References: <13DpyL4de49GOSlp9DU1co_rkznpvX920O6Z0RsmjbY=.23225459-16de-459d-b361-884d8b1d8d35@github.com> <8SDOXwz8OhlhhqpWHjDbdxj9M1ZQfQcls2aRNt-cukQ=.f177d1e3-b30e-4e8b-ae35-d27606e18a5e@github.com> Message-ID: On Tue, 9 Nov 2021 14:00:18 GMT, Erik Joelsson wrote: >> Andrew Leonard has updated the pull request incrementally with one additional commit since the last revision: >> >> 8276743: Make openjdk build Zip Archive generation "reproducible" >> >> Signed-off-by: Andrew Leonard > > make/common/ZipArchive.gmk line 29: > >> 27: _ZIP_ARCHIVE_GMK := 1 >> 28: >> 29: include ../ToolsJdk.gmk > > Should probably add a comment about inclusion of this file now requires an explicit dependency on build-tools in Main.gmk. done ------------- PR: https://git.openjdk.java.net/jdk/pull/6311 From aleonard at openjdk.java.net Thu Nov 11 15:22:17 2021 From: aleonard at openjdk.java.net (Andrew Leonard) Date: Thu, 11 Nov 2021 15:22:17 GMT Subject: RFR: 8276743: Make openjdk build Zip Archive generation "reproducible" [v2] In-Reply-To: References: <13DpyL4de49GOSlp9DU1co_rkznpvX920O6Z0RsmjbY=.23225459-16de-459d-b361-884d8b1d8d35@github.com> <8SDOXwz8OhlhhqpWHjDbdxj9M1ZQfQcls2aRNt-cukQ=.f177d1e3-b30e-4e8b-ae35-d27606e18a5e@github.com> Message-ID: <-wfuroLNSVIn8M6w1bus7MnosA2H-_9ZG7In2y_mdsE=.93d65a49-04f1-45e4-88db-d4134caf2612@github.com> On Wed, 10 Nov 2021 11:22:39 GMT, Andrew Leonard wrote: >> Actually, you don't even need to save the ZipEntry:s in memory, you can just extract filenames from them on the first pass, sort them, then lookup the entries in ZipFile again on the second lap. :) I don't think that's necessary though. > > @erikj79 thanks I didn't realize you can do that: "you can use the ZipFile class to access ZipEntry's in arbitrary order" See new commit ------------- PR: https://git.openjdk.java.net/jdk/pull/6311 From aleonard at openjdk.java.net Thu Nov 11 15:30:19 2021 From: aleonard at openjdk.java.net (Andrew Leonard) Date: Thu, 11 Nov 2021 15:30:19 GMT Subject: RFR: 8276743: Make openjdk build Zip Archive generation "reproducible" [v3] In-Reply-To: <13DpyL4de49GOSlp9DU1co_rkznpvX920O6Z0RsmjbY=.23225459-16de-459d-b361-884d8b1d8d35@github.com> References: <13DpyL4de49GOSlp9DU1co_rkznpvX920O6Z0RsmjbY=.23225459-16de-459d-b361-884d8b1d8d35@github.com> Message-ID: > This PR adds a new openjdk build tool GenerateZip, which generates a final "zip" file from an input folder, and creates it in a deterministic way, ensuring ordering and timestamps are set as specified. > > Using this tool in ZipArchive.gmk will ensure src.zip is then created deterministically. > > Signed-off-by: Andrew Leonard Andrew Leonard has refreshed the contents of this pull request, and previous commits have been removed. The incremental views will show differences compared to the previous content of the PR. The pull request contains one new commit since the last revision: 8276743: Make openjdk build Zip Archive generation "reproducible" Signed-off-by: Andrew Leonard ------------- Changes: - all: https://git.openjdk.java.net/jdk/pull/6311/files - new: https://git.openjdk.java.net/jdk/pull/6311/files/f8a816af..44036af7 Webrevs: - full: https://webrevs.openjdk.java.net/?repo=jdk&pr=6311&range=02 - incr: https://webrevs.openjdk.java.net/?repo=jdk&pr=6311&range=01-02 Stats: 3 lines in 1 file changed: 0 ins; 0 del; 3 mod Patch: https://git.openjdk.java.net/jdk/pull/6311.diff Fetch: git fetch https://git.openjdk.java.net/jdk pull/6311/head:pull/6311 PR: https://git.openjdk.java.net/jdk/pull/6311 From jdv at openjdk.java.net Thu Nov 11 15:32:01 2021 From: jdv at openjdk.java.net (Jayathirth D V) Date: Thu, 11 Nov 2021 15:32:01 GMT Subject: RFR: 8276905: Function frag_col has a deployment target which is incompatible with this OS [v2] In-Reply-To: <_OkC1YeToScvoDOBBhAZfQHSao_fjjsnw60prsCsWtE=.6ea0dbc2-0270-4caa-bcfc-baa7cb704365@github.com> References: <_OkC1YeToScvoDOBBhAZfQHSao_fjjsnw60prsCsWtE=.6ea0dbc2-0270-4caa-bcfc-baa7cb704365@github.com> Message-ID: > When JDK is built on BigSur with Xcode12.5 and used to run jtreg tests in macOS10.15 with Xcode <=12.4 we are getting compilation error. We dont set any deployment target when when we use xcrun to create .air file and this issue looks similar to https://developer.apple.com/forums/thread/105719. Also https://developer.apple.com/forums/thread/105719 states that even if they set app deployment target they need to explicitly set deployment in "xcrun metal". > > Made same change to use -mmacosx-version-min=10.12.00 in our make file. Whether we should keep 10.12.00 or 10.13/14 is open to discussion for metal pipeline. > > SwingSet2, J2DDemo & JCK/jtreg runs in CI are green. Also Vitaly(Submitter) has confirmed that patch resolves the issue. Jayathirth D V has updated the pull request incrementally with one additional commit since the last revision: Use Macro for version ------------- Changes: - all: https://git.openjdk.java.net/jdk/pull/6346/files - new: https://git.openjdk.java.net/jdk/pull/6346/files/a9f521a5..7f999650 Webrevs: - full: https://webrevs.openjdk.java.net/?repo=jdk&pr=6346&range=01 - incr: https://webrevs.openjdk.java.net/?repo=jdk&pr=6346&range=00-01 Stats: 3 lines in 1 file changed: 2 ins; 0 del; 1 mod Patch: https://git.openjdk.java.net/jdk/pull/6346.diff Fetch: git fetch https://git.openjdk.java.net/jdk pull/6346/head:pull/6346 PR: https://git.openjdk.java.net/jdk/pull/6346 From jdv at openjdk.java.net Thu Nov 11 15:32:02 2021 From: jdv at openjdk.java.net (Jayathirth D V) Date: Thu, 11 Nov 2021 15:32:02 GMT Subject: RFR: 8276905: Function frag_col has a deployment target which is incompatible with this OS [v2] In-Reply-To: References: <_OkC1YeToScvoDOBBhAZfQHSao_fjjsnw60prsCsWtE=.6ea0dbc2-0270-4caa-bcfc-baa7cb704365@github.com> Message-ID: On Thu, 11 Nov 2021 13:52:13 GMT, Magnus Ihse Bursie wrote: > We should not hard-code version numbers like that. > > Fortunately for you, there already exists a variable $(MACOSX_VERSION_MIN) that you can use. Thanks for the review i have updated code to use the Macro. ------------- PR: https://git.openjdk.java.net/jdk/pull/6346 From jdv at openjdk.java.net Thu Nov 11 15:32:04 2021 From: jdv at openjdk.java.net (Jayathirth D V) Date: Thu, 11 Nov 2021 15:32:04 GMT Subject: RFR: 8276905: Function frag_col has a deployment target which is incompatible with this OS In-Reply-To: References: <_OkC1YeToScvoDOBBhAZfQHSao_fjjsnw60prsCsWtE=.6ea0dbc2-0270-4caa-bcfc-baa7cb704365@github.com> Message-ID: <_FNKFljLEaG0mIWqtPm7vaUUO_UKLMiR2R6tlsu1KNE=.9dc78629-3b4a-4717-bc34-bf270f93e26e@github.com> On Thu, 11 Nov 2021 13:53:05 GMT, Magnus Ihse Bursie wrote: > Also, if you did not create this patch yourself, please make sure to use `/contributor` to give proper credits. Or maybe you mean that Vitaly submitted the bug report, not the patch? By Submitter i meant submitter of bug in JBS. I was not able to verify the patch in our CI, so i shared the patch with Vitaly so that he can verify the reproducibility of the issue. ------------- PR: https://git.openjdk.java.net/jdk/pull/6346 From duke at openjdk.java.net Thu Nov 11 15:33:33 2021 From: duke at openjdk.java.net (Alan Hayward) Date: Thu, 11 Nov 2021 15:33:33 GMT Subject: RFR: 8264130: PAC-RET protection for Linux/AArch64 [v2] In-Reply-To: References: Message-ID: On Thu, 11 Nov 2021 14:52:54 GMT, Andrew Dinn wrote: > The runtime generated runtime stubs and Java method code into which this patch may insert the required PAC instructions are written into a code cache in a section which is mapped RW(X) all the time. It would be hard to map even a subset of this code cache RO because generated code includes call and data sites that need to be patched during execution. Am I right is saying that for Macos, all generated code is remapped RO before execution? An additional concern I have is that if the globals data was attacked then the UseROPProtection flag could be flipped, and all code after that point would be generated without ROP protection. Marking all the globals data as RO would fix that. Alternatively remove UseROPProtection and then in the macroassembler always generate PAC code, using just the subset of instructions that are NOPs on non-PAC hardware. Or alternatively only generate PAC code based on a #define set at build time. Each option has its own downsides. ------------- PR: https://git.openjdk.java.net/jdk/pull/6334 From duke at openjdk.java.net Thu Nov 11 15:37:09 2021 From: duke at openjdk.java.net (Emmanuel Bourg) Date: Thu, 11 Nov 2021 15:37:09 GMT Subject: RFR: 8275745: Reproducible copyright headers [v3] In-Reply-To: References: Message-ID: <2PURhxGA3r-6JCeIFUuYByG1KaAIF3MUEMBACbHza4c=.ab23d1af-107a-4989-ae34-f51f9273c26b@github.com> > The copyright headers are generated at build time, and the year inserted in the template depends on the current date. This means the headers are not reproducible if the project is built a year later. The year in the headers could be derived from the SOURCE_DATE_EPOCH environment variable to make the build reproducible (this variable is already used in other parts of the build). Emmanuel Bourg has updated the pull request with a new target base due to a merge or a rebase. The incremental webrev excludes the unrelated changes brought in by the merge/rebase. The pull request contains two additional commits since the last revision: - Make gensrc code use $COPYRIGHT_YEAR - Set COPYRIGHT_YEAR from SOURCE_DATE_EPOCH if present ------------- Changes: - all: https://git.openjdk.java.net/jdk/pull/1498/files - new: https://git.openjdk.java.net/jdk/pull/1498/files/9c4efbda..618d28a4 Webrevs: - full: https://webrevs.openjdk.java.net/?repo=jdk&pr=1498&range=02 - incr: https://webrevs.openjdk.java.net/?repo=jdk&pr=1498&range=01-02 Stats: 118724 lines in 2698 files changed: 90310 ins; 15575 del; 12839 mod Patch: https://git.openjdk.java.net/jdk/pull/1498.diff Fetch: git fetch https://git.openjdk.java.net/jdk pull/1498/head:pull/1498 PR: https://git.openjdk.java.net/jdk/pull/1498 From aleonard at openjdk.java.net Thu Nov 11 15:40:48 2021 From: aleonard at openjdk.java.net (Andrew Leonard) Date: Thu, 11 Nov 2021 15:40:48 GMT Subject: RFR: 8276743: Make openjdk build Zip Archive generation "reproducible" In-Reply-To: References: <13DpyL4de49GOSlp9DU1co_rkznpvX920O6Z0RsmjbY=.23225459-16de-459d-b361-884d8b1d8d35@github.com> Message-ID: On Wed, 10 Nov 2021 14:39:09 GMT, Erik Joelsson wrote: >> @erikj79 The flag --enable-reproducible-builds sets ENABLE_REPRODUCIBLE_BUILD in spec.gmk. This is set by our JIB profiles. I propose that we also turn it on for GHA builds. >> >> I think that the post-processing of the zip file can be dependent on this variable and that it serves no purpose to introduce a separate variable ENABLE_REPRODUCIBLE_ZIP that is set to the same value as ENABLE_REPRODUCIBLE_BUILD. Do you agree? > >> I think that the post-processing of the zip file can be dependent on this variable and that it serves no purpose to introduce a separate variable ENABLE_REPRODUCIBLE_ZIP that is set to the same value as ENABLE_REPRODUCIBLE_BUILD. Do you agree? > > Sure, that works for me. @erikj79 @magicus I have just pushed a new commit with the suggested changes, and it works well, thank you for the help I've also done a basic average benchmarking, on my rather slow Ubuntu VM: - Existing src.zip processing : 18 seconds - Additional MakeZipReproducible : 6 seconds ------------- PR: https://git.openjdk.java.net/jdk/pull/6311 From ihse at openjdk.java.net Thu Nov 11 16:25:35 2021 From: ihse at openjdk.java.net (Magnus Ihse Bursie) Date: Thu, 11 Nov 2021 16:25:35 GMT Subject: RFR: 8276905: Function frag_col has a deployment target which is incompatible with this OS [v2] In-Reply-To: References: <_OkC1YeToScvoDOBBhAZfQHSao_fjjsnw60prsCsWtE=.6ea0dbc2-0270-4caa-bcfc-baa7cb704365@github.com> Message-ID: On Thu, 11 Nov 2021 15:32:01 GMT, Jayathirth D V wrote: >> When JDK is built on BigSur with Xcode12.5 and used to run jtreg tests in macOS10.15 with Xcode <=12.4 we are getting compilation error. We dont set any deployment target when when we use xcrun to create .air file and this issue looks similar to https://developer.apple.com/forums/thread/105719. Also https://developer.apple.com/forums/thread/105719 states that even if they set app deployment target they need to explicitly set deployment in "xcrun metal". >> >> Made same change to use -mmacosx-version-min=10.12.00 in our make file. Whether we should keep 10.12.00 or 10.13/14 is open to discussion for metal pipeline. >> >> SwingSet2, J2DDemo & JCK/jtreg runs in CI are green. Also Vitaly(Submitter) has confirmed that patch resolves the issue. > > Jayathirth D V has updated the pull request incrementally with one additional commit since the last revision: > > Use Macro for version Marked as reviewed by ihse (Reviewer). ------------- PR: https://git.openjdk.java.net/jdk/pull/6346 From adinn at openjdk.java.net Thu Nov 11 16:34:33 2021 From: adinn at openjdk.java.net (Andrew Dinn) Date: Thu, 11 Nov 2021 16:34:33 GMT Subject: RFR: 8264130: PAC-RET protection for Linux/AArch64 [v2] In-Reply-To: References: Message-ID: On Thu, 11 Nov 2021 15:30:29 GMT, Alan Hayward wrote: > Am I right is saying that for Macos, all generated code is remapped RO before execution? Ah, no, it seems the code cache is not RWX all the time as far as Java threads are concerned. The Macos/AArch64 code is strategically calling pthread_jit_write_protect_np at Java <-> JVM transition points. That ensures that executable regions are executable but not writable (RX) from a Java thread when running JITted Java code and are writable but not executable (RW) when it calls into JVM code. > An additional concern I have is that if the globals data was attacked then the UseROPProtection flag could be flipped, and all code after that point would be generated without ROP protection. Marking all the globals data as RO would fix that. Alternatively remove UseROPProtection and then in the macroassembler always generate PAC code, using just the subset of instructions that are NOPs on non-PAC hardware. Or alternatively only generate PAC code based on a #define set at build time. Each option has its own downsides. Globals data can legitimately be written during JVM startup (perhaps in some cases also during execution?). So, they cannot simply be marked as RO. I am not sure this concern is really warranted. If an attacker is already able to overwrite UseROPProtection then a concern over the resulting omission of JITted ROP protection seems like attending to the loud banging of the stable door while Shergar has already been diced into stew meat. ------------- PR: https://git.openjdk.java.net/jdk/pull/6334 From mdoerr at openjdk.java.net Thu Nov 11 16:35:32 2021 From: mdoerr at openjdk.java.net (Martin Doerr) Date: Thu, 11 Nov 2021 16:35:32 GMT Subject: RFR: 8276927: [PPC64] Port shenandoahgc to linux on ppc64le In-Reply-To: References: Message-ID: On Wed, 10 Nov 2021 09:00:04 GMT, Niklas Radomski wrote: > Port the Shenandoah garbage collector (JDK-8241457)[https://bugs.openjdk.java.net/browse/JDK-8241457] to linux on ppc64le. src/hotspot/cpu/ppc/gc/shenandoah/shenandoahBarrierSetAssembler_ppc.cpp line 536: > 534: if (!preserve_gp_registers) { __ clobber_volatile_gprs(dst); } > 535: if (!needs_frame) { __ clobber_carg_stack_slots(tmp1); } > 536: #endif This clobber code was certainly good during development and early testing. But is it worth keeping it? Other GCs and other places don't have it any more. So, I'd slightly prefer removal. Feel free to do so if you agree. ------------- PR: https://git.openjdk.java.net/jdk/pull/6325 From aph at openjdk.java.net Thu Nov 11 16:51:39 2021 From: aph at openjdk.java.net (Andrew Haley) Date: Thu, 11 Nov 2021 16:51:39 GMT Subject: RFR: 8264130: PAC-RET protection for Linux/AArch64 [v2] In-Reply-To: References: Message-ID: On Thu, 11 Nov 2021 14:59:32 GMT, Andrew Dinn wrote: >> I did mean the description, not the flag name. > > Yes, understood. I too was talking about the description even though I introduced my comment by talking about what the flag does. `"Protect branches against ROP attacks".` ------------- PR: https://git.openjdk.java.net/jdk/pull/6334 From aph at openjdk.java.net Thu Nov 11 18:10:35 2021 From: aph at openjdk.java.net (Andrew Haley) Date: Thu, 11 Nov 2021 18:10:35 GMT Subject: RFR: 8264130: PAC-RET protection for Linux/AArch64 [v2] In-Reply-To: References: Message-ID: On Thu, 11 Nov 2021 16:31:41 GMT, Andrew Dinn wrote: > > Am I right is saying that for Macos, all generated code is remapped RO before execution? > > Ah, no, it seems the code cache is not RWX all the time as far as Java threads are concerned. The Macos/AArch64 code is strategically calling pthread_jit_write_protect_np at Java <-> JVM transition points. And this requires magic kernel support. I did mention it to a kernel engineer who wasn't very impressed, but I think it's pretty cool. ------------- PR: https://git.openjdk.java.net/jdk/pull/6334 From fweimer at openjdk.java.net Thu Nov 11 18:18:41 2021 From: fweimer at openjdk.java.net (Florian Weimer) Date: Thu, 11 Nov 2021 18:18:41 GMT Subject: RFR: 8264130: PAC-RET protection for Linux/AArch64 [v2] In-Reply-To: References: Message-ID: <3ViGybkSVRbuD_wN398vEFGxNJfiuS1wA_SdLkGtM18=.86e45177-8525-42dc-b27f-c22a67489108@github.com> On Thu, 11 Nov 2021 18:07:37 GMT, Andrew Haley wrote: > > > Am I right is saying that for Macos, all generated code is remapped RO before execution? > > > > > > Ah, no, it seems the code cache is not RWX all the time as far as Java threads are concerned. The Macos/AArch64 code is strategically calling pthread_jit_write_protect_np at Java <-> JVM transition points. > > And this requires magic kernel support. I did mention it to a kernel engineer who wasn't very impressed, but I think it's pretty cool. It's possible to emulate this to some extent with memory protection keys on POWER and (recent) x86. See `pkey_alloc`. ------------- PR: https://git.openjdk.java.net/jdk/pull/6334 From aleonard at openjdk.java.net Thu Nov 11 19:48:04 2021 From: aleonard at openjdk.java.net (Andrew Leonard) Date: Thu, 11 Nov 2021 19:48:04 GMT Subject: RFR: 8276743: Make openjdk build Zip Archive generation "reproducible" [v4] In-Reply-To: <13DpyL4de49GOSlp9DU1co_rkznpvX920O6Z0RsmjbY=.23225459-16de-459d-b361-884d8b1d8d35@github.com> References: <13DpyL4de49GOSlp9DU1co_rkznpvX920O6Z0RsmjbY=.23225459-16de-459d-b361-884d8b1d8d35@github.com> Message-ID: <8U04aB7AkgQ_RUg0YRVUqiDFqGc7ewC8_g21lzMI4Ug=.08c34b84-6b3f-4a49-a376-c10b5dd1d53c@github.com> > This PR adds a new openjdk build tool GenerateZip, which generates a final "zip" file from an input folder, and creates it in a deterministic way, ensuring ordering and timestamps are set as specified. > > Using this tool in ZipArchive.gmk will ensure src.zip is then created deterministically. > > Signed-off-by: Andrew Leonard Andrew Leonard has refreshed the contents of this pull request, and previous commits have been removed. The incremental views will show differences compared to the previous content of the PR. The pull request contains one new commit since the last revision: 8276743: Make openjdk build Zip Archive generation "reproducible" Signed-off-by: Andrew Leonard ------------- Changes: - all: https://git.openjdk.java.net/jdk/pull/6311/files - new: https://git.openjdk.java.net/jdk/pull/6311/files/44036af7..8d148065 Webrevs: - full: https://webrevs.openjdk.java.net/?repo=jdk&pr=6311&range=03 - incr: https://webrevs.openjdk.java.net/?repo=jdk&pr=6311&range=02-03 Stats: 0 lines in 0 files changed: 0 ins; 0 del; 0 mod Patch: https://git.openjdk.java.net/jdk/pull/6311.diff Fetch: git fetch https://git.openjdk.java.net/jdk pull/6311/head:pull/6311 PR: https://git.openjdk.java.net/jdk/pull/6311 From ihse at openjdk.java.net Thu Nov 11 20:21:39 2021 From: ihse at openjdk.java.net (Magnus Ihse Bursie) Date: Thu, 11 Nov 2021 20:21:39 GMT Subject: RFR: 8276743: Make openjdk build Zip Archive generation "reproducible" [v4] In-Reply-To: <8U04aB7AkgQ_RUg0YRVUqiDFqGc7ewC8_g21lzMI4Ug=.08c34b84-6b3f-4a49-a376-c10b5dd1d53c@github.com> References: <13DpyL4de49GOSlp9DU1co_rkznpvX920O6Z0RsmjbY=.23225459-16de-459d-b361-884d8b1d8d35@github.com> <8U04aB7AkgQ_RUg0YRVUqiDFqGc7ewC8_g21lzMI4Ug=.08c34b84-6b3f-4a49-a376-c10b5dd1d53c@github.com> Message-ID: <2QiV2Jf8TH0QpFEgjTe1a7A0_xDrguLn7NqQc_I4S1M=.d214a28c-9087-499f-8599-e240e23eb211@github.com> On Thu, 11 Nov 2021 19:48:04 GMT, Andrew Leonard wrote: >> This PR adds a new openjdk build tool GenerateZip, which generates a final "zip" file from an input folder, and creates it in a deterministic way, ensuring ordering and timestamps are set as specified. >> >> Using this tool in ZipArchive.gmk will ensure src.zip is then created deterministically. >> >> Signed-off-by: Andrew Leonard > > Andrew Leonard has refreshed the contents of this pull request, and previous commits have been removed. The incremental views will show differences compared to the previous content of the PR. In the future, please refrain from force pushing to a PR. It makes history hard to follow for reviewers, and is generally strongly discouraged. OpenJDK uses the Skara tools which will automatically squash and rebase the commits in the PR. ------------- PR: https://git.openjdk.java.net/jdk/pull/6311 From ihse at openjdk.java.net Thu Nov 11 20:22:36 2021 From: ihse at openjdk.java.net (Magnus Ihse Bursie) Date: Thu, 11 Nov 2021 20:22:36 GMT Subject: RFR: 8275745: Reproducible copyright headers [v3] In-Reply-To: <2PURhxGA3r-6JCeIFUuYByG1KaAIF3MUEMBACbHza4c=.ab23d1af-107a-4989-ae34-f51f9273c26b@github.com> References: <2PURhxGA3r-6JCeIFUuYByG1KaAIF3MUEMBACbHza4c=.ab23d1af-107a-4989-ae34-f51f9273c26b@github.com> Message-ID: On Thu, 11 Nov 2021 15:37:09 GMT, Emmanuel Bourg wrote: >> The copyright headers are generated at build time, and the year inserted in the template depends on the current date. This means the headers are not reproducible if the project is built a year later. The year in the headers could be derived from the SOURCE_DATE_EPOCH environment variable to make the build reproducible (this variable is already used in other parts of the build). > > Emmanuel Bourg has updated the pull request with a new target base due to a merge or a rebase. The pull request now contains two commits: > > - Make gensrc code use $COPYRIGHT_YEAR > - Set COPYRIGHT_YEAR from SOURCE_DATE_EPOCH if present In the future, please refrain from force pushing to a PR. It makes history hard to follow for reviewers, and is generally strongly discouraged. OpenJDK uses the Skara tools which will automatically squash and rebase the commits in the PR. ------------- PR: https://git.openjdk.java.net/jdk/pull/1498 From aleonard at openjdk.java.net Thu Nov 11 20:28:33 2021 From: aleonard at openjdk.java.net (Andrew Leonard) Date: Thu, 11 Nov 2021 20:28:33 GMT Subject: RFR: 8276743: Make openjdk build Zip Archive generation "reproducible" [v4] In-Reply-To: <2QiV2Jf8TH0QpFEgjTe1a7A0_xDrguLn7NqQc_I4S1M=.d214a28c-9087-499f-8599-e240e23eb211@github.com> References: <13DpyL4de49GOSlp9DU1co_rkznpvX920O6Z0RsmjbY=.23225459-16de-459d-b361-884d8b1d8d35@github.com> <8U04aB7AkgQ_RUg0YRVUqiDFqGc7ewC8_g21lzMI4Ug=.08c34b84-6b3f-4a49-a376-c10b5dd1d53c@github.com> <2QiV2Jf8TH0QpFEgjTe1a7A0_xDrguLn7NqQc_I4S1M=.d214a28c-9087-499f-8599-e240e23eb211@github.com> Message-ID: On Thu, 11 Nov 2021 20:18:40 GMT, Magnus Ihse Bursie wrote: > In the future, please refrain from force pushing to a PR. It makes history hard to follow for reviewers, and is generally strongly discouraged. OpenJDK uses the Skara tools which will automatically squash and rebase the commits in the PR. @magicus I needed to cause a re-submit tests due to a macos timeout, and there seems no github Action or PR command to cause that, so I just force pushed, couldn't see any other way.... I presume there is? @magicus is pushing an empty commit or dummy change preferable? ------------- PR: https://git.openjdk.java.net/jdk/pull/6311 From duke at openjdk.java.net Thu Nov 11 21:37:38 2021 From: duke at openjdk.java.net (Emmanuel Bourg) Date: Thu, 11 Nov 2021 21:37:38 GMT Subject: RFR: 8275745: Reproducible copyright headers [v3] In-Reply-To: <2PURhxGA3r-6JCeIFUuYByG1KaAIF3MUEMBACbHza4c=.ab23d1af-107a-4989-ae34-f51f9273c26b@github.com> References: <2PURhxGA3r-6JCeIFUuYByG1KaAIF3MUEMBACbHza4c=.ab23d1af-107a-4989-ae34-f51f9273c26b@github.com> Message-ID: On Thu, 11 Nov 2021 15:37:09 GMT, Emmanuel Bourg wrote: >> The copyright headers are generated at build time, and the year inserted in the template depends on the current date. This means the headers are not reproducible if the project is built a year later. The year in the headers could be derived from the SOURCE_DATE_EPOCH environment variable to make the build reproducible (this variable is already used in other parts of the build). > > Emmanuel Bourg has updated the pull request with a new target base due to a merge or a rebase. The pull request now contains two commits: > > - Make gensrc code use $COPYRIGHT_YEAR > - Set COPYRIGHT_YEAR from SOURCE_DATE_EPOCH if present Thank you for the advice, I'm not familiar with the Skara workflow yet. ------------- PR: https://git.openjdk.java.net/jdk/pull/1498 From prr at openjdk.java.net Fri Nov 12 03:59:35 2021 From: prr at openjdk.java.net (Phil Race) Date: Fri, 12 Nov 2021 03:59:35 GMT Subject: RFR: 8276905: Function frag_col has a deployment target which is incompatible with this OS [v2] In-Reply-To: References: <_OkC1YeToScvoDOBBhAZfQHSao_fjjsnw60prsCsWtE=.6ea0dbc2-0270-4caa-bcfc-baa7cb704365@github.com> Message-ID: On Thu, 11 Nov 2021 15:32:01 GMT, Jayathirth D V wrote: >> When JDK is built on BigSur with Xcode12.5 and used to run jtreg tests in macOS10.15 with Xcode <=12.4 we are getting compilation error. We dont set any deployment target when when we use xcrun to create .air file and this issue looks similar to https://developer.apple.com/forums/thread/105719. Also https://developer.apple.com/forums/thread/105719 states that even if they set app deployment target they need to explicitly set deployment in "xcrun metal". >> >> Made same change to use -mmacosx-version-min=10.12.00 in our make file. Whether we should keep 10.12.00 or 10.13/14 is open to discussion for metal pipeline. >> >> SwingSet2, J2DDemo & JCK/jtreg runs in CI are green. Also Vitaly(Submitter) has confirmed that patch resolves the issue. > > Jayathirth D V has updated the pull request incrementally with one additional commit since the last revision: > > Use Macro for version make/modules/java.desktop/lib/Awt2dLibraries.gmk line 850: > 848: COMMAND := $(METAL) -c -std=osx-metal2.0 \ > 849: -mmacosx-version-min=$(MACOSX_VERSION_MIN) \ > 850: -o $(SHADERS_AIR) $(SHADERS_SRC), \ No .. we decided metal requires macos 10.14 as a minimum. In fact it won't run (by policy) on earlier releases so settting it to what the broader JDK defines as a minimum is not appropriate right now. And I've no idea what restrictions we'd be placing on metal saying it can only use 10.12 features. Nor do I know what the xcode 12.4 used to build JDK 17 defaults to and how much a change to either 10.12 or 10.14 might require in re-testing. So - we should use 10.14 what's the actual impact of that on a current build using xcode 12.4 - does it change anything we use ? ------------- PR: https://git.openjdk.java.net/jdk/pull/6346 From prr at openjdk.java.net Fri Nov 12 04:02:36 2021 From: prr at openjdk.java.net (Phil Race) Date: Fri, 12 Nov 2021 04:02:36 GMT Subject: RFR: 8276905: Function frag_col has a deployment target which is incompatible with this OS [v2] In-Reply-To: References: <_OkC1YeToScvoDOBBhAZfQHSao_fjjsnw60prsCsWtE=.6ea0dbc2-0270-4caa-bcfc-baa7cb704365@github.com> Message-ID: On Fri, 12 Nov 2021 03:56:57 GMT, Phil Race wrote: >> Jayathirth D V has updated the pull request incrementally with one additional commit since the last revision: >> >> Use Macro for version > > make/modules/java.desktop/lib/Awt2dLibraries.gmk line 850: > >> 848: COMMAND := $(METAL) -c -std=osx-metal2.0 \ >> 849: -mmacosx-version-min=$(MACOSX_VERSION_MIN) \ >> 850: -o $(SHADERS_AIR) $(SHADERS_SRC), \ > > No .. we decided metal requires macos 10.14 as a minimum. In fact it won't run (by policy) on earlier releases so settting it to what the broader JDK defines as a minimum is not appropriate right now. > And I've no idea what restrictions we'd be placing on metal saying it can only use 10.12 features. > Nor do I know what the xcode 12.4 used to build JDK 17 defaults to and how much a change to either 10.12 or 10.14 might require in re-testing. > > So - > we should use 10.14 > what's the actual impact of that on a current build using xcode 12.4 - does it change anything we use ? bear in mind "impact" might mean metal can't use some new faster code, not just runtime errors ------------- PR: https://git.openjdk.java.net/jdk/pull/6346 From dholmes at openjdk.java.net Fri Nov 12 06:50:33 2021 From: dholmes at openjdk.java.net (David Holmes) Date: Fri, 12 Nov 2021 06:50:33 GMT Subject: RFR: 8277012: Use blessed modifier order in src/utils In-Reply-To: References: Message-ID: On Thu, 11 Nov 2021 14:32:18 GMT, Magnus Ihse Bursie wrote: > I ran bin/blessed-modifier-order.sh on source code in src/utils. This scripts verifies that modifiers are in the "blessed" order, and fixes it otherwise. I have manually checked the changes made by the script to make sure they are sound. > > There are no clear ownership of this code, but I believe it's kind of hotspot-related. Looks fine. Thanks, David ------------- Marked as reviewed by dholmes (Reviewer). PR: https://git.openjdk.java.net/jdk/pull/6354 From aleonard at openjdk.java.net Fri Nov 12 08:24:13 2021 From: aleonard at openjdk.java.net (Andrew Leonard) Date: Fri, 12 Nov 2021 08:24:13 GMT Subject: RFR: 8276743: Make openjdk build Zip Archive generation "reproducible" [v5] In-Reply-To: <13DpyL4de49GOSlp9DU1co_rkznpvX920O6Z0RsmjbY=.23225459-16de-459d-b361-884d8b1d8d35@github.com> References: <13DpyL4de49GOSlp9DU1co_rkznpvX920O6Z0RsmjbY=.23225459-16de-459d-b361-884d8b1d8d35@github.com> Message-ID: > This PR adds a new openjdk build tool GenerateZip, which generates a final "zip" file from an input folder, and creates it in a deterministic way, ensuring ordering and timestamps are set as specified. > > Using this tool in ZipArchive.gmk will ensure src.zip is then created deterministically. > > Signed-off-by: Andrew Leonard Andrew Leonard has updated the pull request incrementally with one additional commit since the last revision: 8276743: Make openjdk build Zip Archive generation "reproducible" Signed-off-by: Andrew Leonard ------------- Changes: - all: https://git.openjdk.java.net/jdk/pull/6311/files - new: https://git.openjdk.java.net/jdk/pull/6311/files/8d148065..ea477b9c Webrevs: - full: https://webrevs.openjdk.java.net/?repo=jdk&pr=6311&range=04 - incr: https://webrevs.openjdk.java.net/?repo=jdk&pr=6311&range=03-04 Stats: 1 line in 1 file changed: 0 ins; 0 del; 1 mod Patch: https://git.openjdk.java.net/jdk/pull/6311.diff Fetch: git fetch https://git.openjdk.java.net/jdk pull/6311/head:pull/6311 PR: https://git.openjdk.java.net/jdk/pull/6311 From aleonard at openjdk.java.net Fri Nov 12 08:24:16 2021 From: aleonard at openjdk.java.net (Andrew Leonard) Date: Fri, 12 Nov 2021 08:24:16 GMT Subject: RFR: 8276743: Make openjdk build Zip Archive generation "reproducible" [v4] In-Reply-To: <8U04aB7AkgQ_RUg0YRVUqiDFqGc7ewC8_g21lzMI4Ug=.08c34b84-6b3f-4a49-a376-c10b5dd1d53c@github.com> References: <13DpyL4de49GOSlp9DU1co_rkznpvX920O6Z0RsmjbY=.23225459-16de-459d-b361-884d8b1d8d35@github.com> <8U04aB7AkgQ_RUg0YRVUqiDFqGc7ewC8_g21lzMI4Ug=.08c34b84-6b3f-4a49-a376-c10b5dd1d53c@github.com> Message-ID: On Thu, 11 Nov 2021 19:48:04 GMT, Andrew Leonard wrote: >> This PR adds a new openjdk build tool GenerateZip, which generates a final "zip" file from an input folder, and creates it in a deterministic way, ensuring ordering and timestamps are set as specified. >> >> Using this tool in ZipArchive.gmk will ensure src.zip is then created deterministically. >> >> Signed-off-by: Andrew Leonard > > Andrew Leonard has refreshed the contents of this pull request, and previous commits have been removed. The incremental views will show differences compared to the previous content of the PR. Looks like other PRs also getting mac or windows test failures.. ------------- PR: https://git.openjdk.java.net/jdk/pull/6311 From stuefe at openjdk.java.net Fri Nov 12 08:24:34 2021 From: stuefe at openjdk.java.net (Thomas Stuefe) Date: Fri, 12 Nov 2021 08:24:34 GMT Subject: RFR: 8277012: Use blessed modifier order in src/utils In-Reply-To: References: Message-ID: <0dDzCW4HXazTrH_66L0Jrq9MIx2k86QqIgqBNPwh-lg=.f8928984-7a43-4b4a-8a23-245b79c4aa65@github.com> On Thu, 11 Nov 2021 14:32:18 GMT, Magnus Ihse Bursie wrote: > I ran bin/blessed-modifier-order.sh on source code in src/utils. This scripts verifies that modifiers are in the "blessed" order, and fixes it otherwise. I have manually checked the changes made by the script to make sure they are sound. > > There are no clear ownership of this code, but I believe it's kind of hotspot-related. +1 ------------- Marked as reviewed by stuefe (Reviewer). PR: https://git.openjdk.java.net/jdk/pull/6354 From mcimadamore at openjdk.java.net Fri Nov 12 11:16:17 2021 From: mcimadamore at openjdk.java.net (Maurizio Cimadamore) Date: Fri, 12 Nov 2021 11:16:17 GMT Subject: RFR: 8275063: Implementation of Foreign Function & Memory API (Second incubator) [v24] In-Reply-To: References: Message-ID: > This PR contains the API and implementation changes for JEP-419 [1]. A more detailed description of such changes, to avoid repetitions during the review process, is included as a separate comment. > > [1] - https://openjdk.java.net/jeps/419 Maurizio Cimadamore has updated the pull request incrementally with one additional commit since the last revision: Adopt blessed modofier order ------------- Changes: - all: https://git.openjdk.java.net/jdk/pull/5907/files - new: https://git.openjdk.java.net/jdk/pull/5907/files/8c3860f8..79d3d685 Webrevs: - full: https://webrevs.openjdk.java.net/?repo=jdk&pr=5907&range=23 - incr: https://webrevs.openjdk.java.net/?repo=jdk&pr=5907&range=22-23 Stats: 7 lines in 6 files changed: 0 ins; 0 del; 7 mod Patch: https://git.openjdk.java.net/jdk/pull/5907.diff Fetch: git fetch https://git.openjdk.java.net/jdk pull/5907/head:pull/5907 PR: https://git.openjdk.java.net/jdk/pull/5907 From aleonard at openjdk.java.net Fri Nov 12 11:18:34 2021 From: aleonard at openjdk.java.net (Andrew Leonard) Date: Fri, 12 Nov 2021 11:18:34 GMT Subject: RFR: 8276743: Make openjdk build Zip Archive generation "reproducible" In-Reply-To: References: <13DpyL4de49GOSlp9DU1co_rkznpvX920O6Z0RsmjbY=.23225459-16de-459d-b361-884d8b1d8d35@github.com> Message-ID: On Wed, 10 Nov 2021 14:39:09 GMT, Erik Joelsson wrote: >> @erikj79 The flag --enable-reproducible-builds sets ENABLE_REPRODUCIBLE_BUILD in spec.gmk. This is set by our JIB profiles. I propose that we also turn it on for GHA builds. >> >> I think that the post-processing of the zip file can be dependent on this variable and that it serves no purpose to introduce a separate variable ENABLE_REPRODUCIBLE_ZIP that is set to the same value as ENABLE_REPRODUCIBLE_BUILD. Do you agree? > >> I think that the post-processing of the zip file can be dependent on this variable and that it serves no purpose to introduce a separate variable ENABLE_REPRODUCIBLE_ZIP that is set to the same value as ENABLE_REPRODUCIBLE_BUILD. Do you agree? > > Sure, that works for me. @erikj79 all tests pass, ready for re-review please, thanks ------------- PR: https://git.openjdk.java.net/jdk/pull/6311 From ihse at openjdk.java.net Fri Nov 12 14:12:38 2021 From: ihse at openjdk.java.net (Magnus Ihse Bursie) Date: Fri, 12 Nov 2021 14:12:38 GMT Subject: Integrated: 8277012: Use blessed modifier order in src/utils In-Reply-To: References: Message-ID: <4JP7H2okj8RsYPM_9UpEJKDJwjmryFGpfxf1_7vZICI=.020e8c9f-1035-4a5a-b0e8-d47c121833f9@github.com> On Thu, 11 Nov 2021 14:32:18 GMT, Magnus Ihse Bursie wrote: > I ran bin/blessed-modifier-order.sh on source code in src/utils. This scripts verifies that modifiers are in the "blessed" order, and fixes it otherwise. I have manually checked the changes made by the script to make sure they are sound. > > There are no clear ownership of this code, but I believe it's kind of hotspot-related. This pull request has now been integrated. Changeset: c4b44329 Author: Magnus Ihse Bursie URL: https://git.openjdk.java.net/jdk/commit/c4b44329c1d250f790ca82dd419cdf3330da16f5 Stats: 25 lines in 10 files changed: 0 ins; 0 del; 25 mod 8277012: Use blessed modifier order in src/utils Reviewed-by: dholmes, stuefe ------------- PR: https://git.openjdk.java.net/jdk/pull/6354 From ihse at openjdk.java.net Fri Nov 12 14:19:34 2021 From: ihse at openjdk.java.net (Magnus Ihse Bursie) Date: Fri, 12 Nov 2021 14:19:34 GMT Subject: RFR: 8276905: Function frag_col has a deployment target which is incompatible with this OS [v2] In-Reply-To: References: <_OkC1YeToScvoDOBBhAZfQHSao_fjjsnw60prsCsWtE=.6ea0dbc2-0270-4caa-bcfc-baa7cb704365@github.com> Message-ID: On Thu, 11 Nov 2021 15:32:01 GMT, Jayathirth D V wrote: >> When JDK is built on BigSur with Xcode12.5 and used to run jtreg tests in macOS10.15 with Xcode <=12.4 we are getting compilation error. We dont set any deployment target when when we use xcrun to create .air file and this issue looks similar to https://developer.apple.com/forums/thread/105719. Also https://developer.apple.com/forums/thread/105719 states that even if they set app deployment target they need to explicitly set deployment in "xcrun metal". >> >> Made same change to use -mmacosx-version-min=10.12.00 in our make file. Whether we should keep 10.12.00 or 10.13/14 is open to discussion for metal pipeline. >> >> SwingSet2, J2DDemo & JCK/jtreg runs in CI are green. Also Vitaly(Submitter) has confirmed that patch resolves the issue. > > Jayathirth D V has updated the pull request incrementally with one additional commit since the last revision: > > Use Macro for version I'm withdrawing my approval until the issues raised by Phil has been resolved. It sounds like this change need more thorough discussion about what problems you are experience, and what are acceptable solutions to it. ------------- Changes requested by ihse (Reviewer). PR: https://git.openjdk.java.net/jdk/pull/6346 From erikj at openjdk.java.net Fri Nov 12 14:25:42 2021 From: erikj at openjdk.java.net (Erik Joelsson) Date: Fri, 12 Nov 2021 14:25:42 GMT Subject: RFR: 8275745: Reproducible copyright headers [v3] In-Reply-To: <2PURhxGA3r-6JCeIFUuYByG1KaAIF3MUEMBACbHza4c=.ab23d1af-107a-4989-ae34-f51f9273c26b@github.com> References: <2PURhxGA3r-6JCeIFUuYByG1KaAIF3MUEMBACbHza4c=.ab23d1af-107a-4989-ae34-f51f9273c26b@github.com> Message-ID: On Thu, 11 Nov 2021 15:37:09 GMT, Emmanuel Bourg wrote: >> The copyright headers are generated at build time, and the year inserted in the template depends on the current date. This means the headers are not reproducible if the project is built a year later. The year in the headers could be derived from the SOURCE_DATE_EPOCH environment variable to make the build reproducible (this variable is already used in other parts of the build). > > Emmanuel Bourg has updated the pull request with a new target base due to a merge or a rebase. The pull request now contains two commits: > > - Make gensrc code use $COPYRIGHT_YEAR > - Set COPYRIGHT_YEAR from SOURCE_DATE_EPOCH if present Marked as reviewed by erikj (Reviewer). ------------- PR: https://git.openjdk.java.net/jdk/pull/1498 From ihse at openjdk.java.net Fri Nov 12 14:29:33 2021 From: ihse at openjdk.java.net (Magnus Ihse Bursie) Date: Fri, 12 Nov 2021 14:29:33 GMT Subject: RFR: 8276743: Make openjdk build Zip Archive generation "reproducible" In-Reply-To: References: <13DpyL4de49GOSlp9DU1co_rkznpvX920O6Z0RsmjbY=.23225459-16de-459d-b361-884d8b1d8d35@github.com> Message-ID: On Fri, 12 Nov 2021 11:15:30 GMT, Andrew Leonard wrote: >>> I think that the post-processing of the zip file can be dependent on this variable and that it serves no purpose to introduce a separate variable ENABLE_REPRODUCIBLE_ZIP that is set to the same value as ENABLE_REPRODUCIBLE_BUILD. Do you agree? >> >> Sure, that works for me. > > @erikj79 all tests pass, ready for re-review please, thanks @andrew-m-leonard Yes, pushing an empty commit is much better. The Skara tooling will automatically squash all commits in the PR when it is integrated, so it will be fully invisible in the end. But there is a way to trigger re-runs, although I realize is has close to zero visibility (I'll need to blame Github for that :-(). Go to your personal fork of the JDK, select the "Actions" tab, select "Pre-submit tests" in the list to the left, and then press the `Run workflow` button in the cyan `This workflow has a workflow_dispatch event trigger.` field. There you can select branch to run, and also additionally modify the set of platforms run. ------------- PR: https://git.openjdk.java.net/jdk/pull/6311 From erikj at openjdk.java.net Fri Nov 12 14:29:33 2021 From: erikj at openjdk.java.net (Erik Joelsson) Date: Fri, 12 Nov 2021 14:29:33 GMT Subject: RFR: 8276743: Make openjdk build Zip Archive generation "reproducible" In-Reply-To: References: <13DpyL4de49GOSlp9DU1co_rkznpvX920O6Z0RsmjbY=.23225459-16de-459d-b361-884d8b1d8d35@github.com> Message-ID: <4yLuKwwPjRDaLZuMzHjclzjz5gV9H6cQQi8iv1CXH_Y=.b2cb5d67-3f64-4c37-83d7-aaca901d3221@github.com> On Fri, 12 Nov 2021 14:22:09 GMT, Magnus Ihse Bursie wrote: >> @erikj79 all tests pass, ready for re-review please, thanks > > @andrew-m-leonard Yes, pushing an empty commit is much better. The Skara tooling will automatically squash all commits in the PR when it is integrated, so it will be fully invisible in the end. > > But there is a way to trigger re-runs, although I realize is has close to zero visibility (I'll need to blame Github for that :-(). > > Go to your personal fork of the JDK, select the "Actions" tab, select "Pre-submit tests" in the list to the left, and then press the `Run workflow` button in the cyan `This workflow has a workflow_dispatch event trigger.` field. > > There you can select branch to run, and also additionally modify the set of platforms run. > @magicus is pushing an empty commit or dummy change preferable? Yes, I think that's the only good way of re-triggering github actions. While working on a PR, it's better to let history be history so review comments don't get lost. When we finally integrate to mainline, everything will be squashed automatically. ------------- PR: https://git.openjdk.java.net/jdk/pull/6311 From ihse at openjdk.java.net Fri Nov 12 14:29:35 2021 From: ihse at openjdk.java.net (Magnus Ihse Bursie) Date: Fri, 12 Nov 2021 14:29:35 GMT Subject: RFR: 8276743: Make openjdk build Zip Archive generation "reproducible" [v5] In-Reply-To: References: <13DpyL4de49GOSlp9DU1co_rkznpvX920O6Z0RsmjbY=.23225459-16de-459d-b361-884d8b1d8d35@github.com> Message-ID: On Fri, 12 Nov 2021 08:24:13 GMT, Andrew Leonard wrote: >> This PR adds a new openjdk build tool MakeZipReproducible, which if ENABLE_REPRODUCIBLE_BUILD is enabled, generates a final "zip" file in a deterministic way, ensuring ordering and timestamps are set as specified. >> >> Using this tool in ZipArchive.gmk will ensure src.zip is then created deterministically. >> >> Signed-off-by: Andrew Leonard > > Andrew Leonard has updated the pull request incrementally with one additional commit since the last revision: > > 8276743: Make openjdk build Zip Archive generation "reproducible" > > Signed-off-by: Andrew Leonard make/Main.gmk line 511: > 509: MAKEFILE := Docs, \ > 510: TARGET := docs-zip, \ > 511: DEPS := docs-jdk buildtools-jdk, \ Is this really needed? I did not think jrtfs-jar used zip? ------------- PR: https://git.openjdk.java.net/jdk/pull/6311 From ihse at openjdk.java.net Fri Nov 12 14:42:36 2021 From: ihse at openjdk.java.net (Magnus Ihse Bursie) Date: Fri, 12 Nov 2021 14:42:36 GMT Subject: RFR: 8276743: Make openjdk build Zip Archive generation "reproducible" [v5] In-Reply-To: References: <13DpyL4de49GOSlp9DU1co_rkznpvX920O6Z0RsmjbY=.23225459-16de-459d-b361-884d8b1d8d35@github.com> Message-ID: On Fri, 12 Nov 2021 08:24:13 GMT, Andrew Leonard wrote: >> This PR adds a new openjdk build tool MakeZipReproducible, which if ENABLE_REPRODUCIBLE_BUILD is enabled, generates a final "zip" file in a deterministic way, ensuring ordering and timestamps are set as specified. >> >> Using this tool in ZipArchive.gmk will ensure src.zip is then created deterministically. >> >> Signed-off-by: Andrew Leonard > > Andrew Leonard has updated the pull request incrementally with one additional commit since the last revision: > > 8276743: Make openjdk build Zip Archive generation "reproducible" > > Signed-off-by: Andrew Leonard I think it looks good now. Let Erik have a final say also, before integrating. Thank you for your effort! ------------- Marked as reviewed by ihse (Reviewer). PR: https://git.openjdk.java.net/jdk/pull/6311 From erikj at openjdk.java.net Fri Nov 12 14:42:37 2021 From: erikj at openjdk.java.net (Erik Joelsson) Date: Fri, 12 Nov 2021 14:42:37 GMT Subject: RFR: 8276743: Make openjdk build Zip Archive generation "reproducible" [v5] In-Reply-To: References: <13DpyL4de49GOSlp9DU1co_rkznpvX920O6Z0RsmjbY=.23225459-16de-459d-b361-884d8b1d8d35@github.com> Message-ID: <8qNui7Vppsmck7cJdcyXYdJlLxqSXhicCBKk5ADNYYM=.a22525af-a2cd-4e48-892e-3b3d099bf06e@github.com> On Fri, 12 Nov 2021 08:24:13 GMT, Andrew Leonard wrote: >> This PR adds a new openjdk build tool MakeZipReproducible, which if ENABLE_REPRODUCIBLE_BUILD is enabled, generates a final "zip" file in a deterministic way, ensuring ordering and timestamps are set as specified. >> >> Using this tool in ZipArchive.gmk will ensure src.zip is then created deterministically. >> >> Signed-off-by: Andrew Leonard > > Andrew Leonard has updated the pull request incrementally with one additional commit since the last revision: > > 8276743: Make openjdk build Zip Archive generation "reproducible" > > Signed-off-by: Andrew Leonard Looks good. ------------- Marked as reviewed by erikj (Reviewer). PR: https://git.openjdk.java.net/jdk/pull/6311 From ihse at openjdk.java.net Fri Nov 12 14:42:39 2021 From: ihse at openjdk.java.net (Magnus Ihse Bursie) Date: Fri, 12 Nov 2021 14:42:39 GMT Subject: RFR: 8276743: Make openjdk build Zip Archive generation "reproducible" [v5] In-Reply-To: References: <13DpyL4de49GOSlp9DU1co_rkznpvX920O6Z0RsmjbY=.23225459-16de-459d-b361-884d8b1d8d35@github.com> Message-ID: On Fri, 12 Nov 2021 14:24:00 GMT, Magnus Ihse Bursie wrote: >> Andrew Leonard has updated the pull request incrementally with one additional commit since the last revision: >> >> 8276743: Make openjdk build Zip Archive generation "reproducible" >> >> Signed-off-by: Andrew Leonard > > make/Main.gmk line 511: > >> 509: MAKEFILE := Docs, \ >> 510: TARGET := docs-zip, \ >> 511: DEPS := docs-jdk buildtools-jdk, \ > > Is this really needed? I did not think jrtfs-jar used zip? Sorry, I'm mis-reading the lines here. It's of course for docs-zip. Then it's perfectly in order! :-) ------------- PR: https://git.openjdk.java.net/jdk/pull/6311 From erikj at openjdk.java.net Fri Nov 12 14:42:39 2021 From: erikj at openjdk.java.net (Erik Joelsson) Date: Fri, 12 Nov 2021 14:42:39 GMT Subject: RFR: 8276743: Make openjdk build Zip Archive generation "reproducible" [v5] In-Reply-To: References: <13DpyL4de49GOSlp9DU1co_rkznpvX920O6Z0RsmjbY=.23225459-16de-459d-b361-884d8b1d8d35@github.com> Message-ID: On Fri, 12 Nov 2021 14:28:31 GMT, Magnus Ihse Bursie wrote: >> make/Main.gmk line 511: >> >>> 509: MAKEFILE := Docs, \ >>> 510: TARGET := docs-zip, \ >>> 511: DEPS := docs-jdk buildtools-jdk, \ >> >> Is this really needed? I did not think jrtfs-jar used zip? > > Sorry, I'm mis-reading the lines here. It's of course for docs-zip. Then it's perfectly in order! :-) It's using JarArchive.gmk, which is a similar, but different and probably also needs the same treatment. We should probably share more code between them. ------------- PR: https://git.openjdk.java.net/jdk/pull/6311 From aleonard at openjdk.java.net Fri Nov 12 14:42:39 2021 From: aleonard at openjdk.java.net (Andrew Leonard) Date: Fri, 12 Nov 2021 14:42:39 GMT Subject: RFR: 8276743: Make openjdk build Zip Archive generation "reproducible" [v5] In-Reply-To: References: <13DpyL4de49GOSlp9DU1co_rkznpvX920O6Z0RsmjbY=.23225459-16de-459d-b361-884d8b1d8d35@github.com> Message-ID: On Fri, 12 Nov 2021 14:34:50 GMT, Erik Joelsson wrote: >> Sorry, I'm mis-reading the lines here. It's of course for docs-zip. Then it's perfectly in order! :-) > > It's using JarArchive.gmk, which is a similar, but different and probably also needs the same treatment. We should probably share more code between them. I believe so, it's target docs-zip not jrtfs : https://github.com/openjdk/jdk/blob/51a5731d6dc4b6f6feac920a4b8b49c15fd6b34f/make/Docs.gmk#L712 ------------- PR: https://git.openjdk.java.net/jdk/pull/6311 From aleonard at openjdk.java.net Fri Nov 12 14:48:44 2021 From: aleonard at openjdk.java.net (Andrew Leonard) Date: Fri, 12 Nov 2021 14:48:44 GMT Subject: Integrated: 8276743: Make openjdk build Zip Archive generation "reproducible" In-Reply-To: <13DpyL4de49GOSlp9DU1co_rkznpvX920O6Z0RsmjbY=.23225459-16de-459d-b361-884d8b1d8d35@github.com> References: <13DpyL4de49GOSlp9DU1co_rkznpvX920O6Z0RsmjbY=.23225459-16de-459d-b361-884d8b1d8d35@github.com> Message-ID: On Tue, 9 Nov 2021 12:59:17 GMT, Andrew Leonard wrote: > This PR adds a new openjdk build tool MakeZipReproducible, which if ENABLE_REPRODUCIBLE_BUILD is enabled, generates a final "zip" file in a deterministic way, ensuring ordering and timestamps are set as specified. > > Using this tool in ZipArchive.gmk will ensure src.zip is then created deterministically. > > Signed-off-by: Andrew Leonard This pull request has now been integrated. Changeset: aeba6530 Author: Andrew Leonard URL: https://git.openjdk.java.net/jdk/commit/aeba65303479130d9bab74484accad5d7d116a40 Stats: 261 lines in 4 files changed: 254 ins; 0 del; 7 mod 8276743: Make openjdk build Zip Archive generation "reproducible" Reviewed-by: erikj, ihse ------------- PR: https://git.openjdk.java.net/jdk/pull/6311 From erikj at openjdk.java.net Fri Nov 12 14:52:39 2021 From: erikj at openjdk.java.net (Erik Joelsson) Date: Fri, 12 Nov 2021 14:52:39 GMT Subject: RFR: 8276905: Function frag_col has a deployment target which is incompatible with this OS [v2] In-Reply-To: References: <_OkC1YeToScvoDOBBhAZfQHSao_fjjsnw60prsCsWtE=.6ea0dbc2-0270-4caa-bcfc-baa7cb704365@github.com> Message-ID: On Thu, 11 Nov 2021 15:32:01 GMT, Jayathirth D V wrote: >> When JDK is built on BigSur with Xcode12.5 and used to run jtreg tests in macOS10.15 with Xcode <=12.4 we are getting compilation error. We dont set any deployment target when when we use xcrun to create .air file and this issue looks similar to https://developer.apple.com/forums/thread/105719. Also https://developer.apple.com/forums/thread/105719 states that even if they set app deployment target they need to explicitly set deployment in "xcrun metal". >> >> Made same change to use -mmacosx-version-min=10.12.00 in our make file. Whether we should keep 10.12.00 or 10.13/14 is open to discussion for metal pipeline. >> >> SwingSet2, J2DDemo & JCK/jtreg runs in CI are green. Also Vitaly(Submitter) has confirmed that patch resolves the issue. > > Jayathirth D V has updated the pull request incrementally with one additional commit since the last revision: > > Use Macro for version The JDK default of 10.12 is an old and rather conservative choice. I basically picked it as that was the oldest I could get away with at the time. This value is meant to be bumped whenever a need for it arises. Given that the oldest Macos version still getting Apple support is 10.15, I see no issue with raising this to at least 10.14, if that is what any part of the JDK requires. The whole point of defining that value is that it should reflect our requirements. So I propose, bump the number in configure (and jib-profiles.js) and use it in the client makefiles so we fix the problem. ------------- PR: https://git.openjdk.java.net/jdk/pull/6346 From ihse at openjdk.java.net Fri Nov 12 14:58:42 2021 From: ihse at openjdk.java.net (Magnus Ihse Bursie) Date: Fri, 12 Nov 2021 14:58:42 GMT Subject: RFR: 8276905: Function frag_col has a deployment target which is incompatible with this OS [v2] In-Reply-To: References: <_OkC1YeToScvoDOBBhAZfQHSao_fjjsnw60prsCsWtE=.6ea0dbc2-0270-4caa-bcfc-baa7cb704365@github.com> Message-ID: On Thu, 11 Nov 2021 15:32:01 GMT, Jayathirth D V wrote: >> When JDK is built on BigSur with Xcode12.5 and used to run jtreg tests in macOS10.15 with Xcode <=12.4 we are getting compilation error. We dont set any deployment target when when we use xcrun to create .air file and this issue looks similar to https://developer.apple.com/forums/thread/105719. Also https://developer.apple.com/forums/thread/105719 states that even if they set app deployment target they need to explicitly set deployment in "xcrun metal". >> >> Made same change to use -mmacosx-version-min=10.12.00 in our make file. Whether we should keep 10.12.00 or 10.13/14 is open to discussion for metal pipeline. >> >> SwingSet2, J2DDemo & JCK/jtreg runs in CI are green. Also Vitaly(Submitter) has confirmed that patch resolves the issue. > > Jayathirth D V has updated the pull request incrementally with one additional commit since the last revision: > > Use Macro for version If we leave -mmacosx-version-min unspecified, will metal pick another value by default silently? And if so, might we be actually lowering the min version even if specifying 10.14? In any case, it seems that any change here will need thorough testing both on Oracle CI systems and to make sure it solves @jayathirthrao's problem. ------------- PR: https://git.openjdk.java.net/jdk/pull/6346 From duke at openjdk.java.net Fri Nov 12 16:18:04 2021 From: duke at openjdk.java.net (Alan Hayward) Date: Fri, 12 Nov 2021 16:18:04 GMT Subject: RFR: 8264130: PAC-RET protection for Linux/AArch64 [v3] In-Reply-To: References: Message-ID: > PAC is an optional feature in AArch64 8.3 and is compulsory in v9. One > of its uses is to protect against ROP based attacks. This is done by > signing the Link Register whenever it is stored on the stack, and > authenticating the value when it is loaded back from the stack. If an > attacker were to try to change control flow by editing the stack then > the authentication check of the Link Register will fail, causing a > segfault when the function returns. > > On a system with PAC enabled, it is expected that all applications will > be compiled with ROP protection. Fedora 33 and upwards already provide > this. By compiling for ARMv8.0, GCC and LLVM will only use the set of > PAC instructions that exist in the NOP space - on hardware without PAC, > these instructions act as NOPs, allowing backward compatibility for > negligible performance cost (2 NOPs per non-leaf function). > > Hardware is currently limited to the Apple M1 MacBooks. All testing has > been done within a Fedora Docker image. A run of SpecJVM showed no > difference to that of noise - which was surprising. > > The most important part of this patch is simply compiling using branch > protection provided by GCC/LLVM. This protects all C++ code from being > used in ROP attacks, removing all static ROP gadgets from use. > > The remainder of the patch adds ROP protection to runtime generated > code, in both stubs and compiled Java code. Attacks here are much harder > as ROP gadgets must be found dynamically at runtime. If/when AOT > compilation is added to JDK, then all stubs and compiled Java will be > susceptible ROP gadgets being found by static analysis and therefore > potentially as vulnerable as C++ code. > > There are a number of places where the VM changes control flow by > rewriting the stack or otherwise. I?ve done some analysis as to how > these could also be used for attacks (which I didn?t want to post here). > These areas can be protected ensuring the pointers to various stubs and > entry points are stored in memory as signed pointers. These changes are > simple to make (they can be reduced to a type change in common code and > a few addition sign/auth calls in the backend), but there a lot of them > and the total code change is fairly large. I?m happy to provide a few > work in progress patches. > > In order to match the security benefits of the Apple Arm64e ABI across > the whole of JDK, then all the changes mentioned above would be > required. Alan Hayward has updated the pull request incrementally with two additional commits since the last revision: - Document pauth functions && remove OS split - Update UseROPProtection description ------------- Changes: - all: https://git.openjdk.java.net/jdk/pull/6334/files - new: https://git.openjdk.java.net/jdk/pull/6334/files/29471d30..25e62492 Webrevs: - full: https://webrevs.openjdk.java.net/?repo=jdk&pr=6334&range=02 - incr: https://webrevs.openjdk.java.net/?repo=jdk&pr=6334&range=01-02 Stats: 369 lines in 9 files changed: 129 ins; 219 del; 21 mod Patch: https://git.openjdk.java.net/jdk/pull/6334.diff Fetch: git fetch https://git.openjdk.java.net/jdk pull/6334/head:pull/6334 PR: https://git.openjdk.java.net/jdk/pull/6334 From duke at openjdk.java.net Fri Nov 12 16:18:07 2021 From: duke at openjdk.java.net (Alan Hayward) Date: Fri, 12 Nov 2021 16:18:07 GMT Subject: RFR: 8264130: PAC-RET protection for Linux/AArch64 [v2] In-Reply-To: References: Message-ID: On Thu, 11 Nov 2021 08:48:07 GMT, Alan Hayward wrote: >> PAC is an optional feature in AArch64 8.3 and is compulsory in v9. One >> of its uses is to protect against ROP based attacks. This is done by >> signing the Link Register whenever it is stored on the stack, and >> authenticating the value when it is loaded back from the stack. If an >> attacker were to try to change control flow by editing the stack then >> the authentication check of the Link Register will fail, causing a >> segfault when the function returns. >> >> On a system with PAC enabled, it is expected that all applications will >> be compiled with ROP protection. Fedora 33 and upwards already provide >> this. By compiling for ARMv8.0, GCC and LLVM will only use the set of >> PAC instructions that exist in the NOP space - on hardware without PAC, >> these instructions act as NOPs, allowing backward compatibility for >> negligible performance cost (2 NOPs per non-leaf function). >> >> Hardware is currently limited to the Apple M1 MacBooks. All testing has >> been done within a Fedora Docker image. A run of SpecJVM showed no >> difference to that of noise - which was surprising. >> >> The most important part of this patch is simply compiling using branch >> protection provided by GCC/LLVM. This protects all C++ code from being >> used in ROP attacks, removing all static ROP gadgets from use. >> >> The remainder of the patch adds ROP protection to runtime generated >> code, in both stubs and compiled Java code. Attacks here are much harder >> as ROP gadgets must be found dynamically at runtime. If/when AOT >> compilation is added to JDK, then all stubs and compiled Java will be >> susceptible ROP gadgets being found by static analysis and therefore >> potentially as vulnerable as C++ code. >> >> There are a number of places where the VM changes control flow by >> rewriting the stack or otherwise. I?ve done some analysis as to how >> these could also be used for attacks (which I didn?t want to post here). >> These areas can be protected ensuring the pointers to various stubs and >> entry points are stored in memory as signed pointers. These changes are >> simple to make (they can be reduced to a type change in common code and >> a few addition sign/auth calls in the backend), but there a lot of them >> and the total code change is fairly large. I?m happy to provide a few >> work in progress patches. >> >> In order to match the security benefits of the Apple Arm64e ABI across >> the whole of JDK, then all the changes mentioned above would be >> required. > > Alan Hayward has updated the pull request incrementally with one additional commit since the last revision: > > Simplify branch protection configure check *Updated UseROPProtection message *Moved pauth functions into single file *Added comments *Removed superfluous modifier arg from macroassembler funcs ------------- PR: https://git.openjdk.java.net/jdk/pull/6334 From erikj at openjdk.java.net Fri Nov 12 17:21:36 2021 From: erikj at openjdk.java.net (Erik Joelsson) Date: Fri, 12 Nov 2021 17:21:36 GMT Subject: RFR: 8276905: Function frag_col has a deployment target which is incompatible with this OS [v2] In-Reply-To: References: <_OkC1YeToScvoDOBBhAZfQHSao_fjjsnw60prsCsWtE=.6ea0dbc2-0270-4caa-bcfc-baa7cb704365@github.com> Message-ID: On Fri, 12 Nov 2021 14:55:24 GMT, Magnus Ihse Bursie wrote: > If we leave -mmacosx-version-min unspecified, will metal pick another value by default silently? And if so, might we be actually lowering the min version even if specifying 10.14? I'm not sure how Xcode picks the default target, but in my experience, it's some combination of the SDK bundled with Xcode and the current system the build is running on. I would assume this tool behaves the same way. At Oracle we are currently using Xcode 12.4 and Macos 10.15.4, so specifying 10.14 would most likely be a step down from the assumed current default of 10.15. As far as Oracle is concerned, we should be fine with going with 10.15 as that's the oldest version that we will officially support for JDK 18 anyway. I have generally tried to be a bit more conservative with this version requirement though. I agree that testing is required for such a change. ------------- PR: https://git.openjdk.java.net/jdk/pull/6346 From aph at openjdk.java.net Fri Nov 12 17:39:41 2021 From: aph at openjdk.java.net (Andrew Haley) Date: Fri, 12 Nov 2021 17:39:41 GMT Subject: RFR: 8264130: PAC-RET protection for Linux/AArch64 [v3] In-Reply-To: References: Message-ID: On Fri, 12 Nov 2021 16:18:04 GMT, Alan Hayward wrote: >> PAC is an optional feature in AArch64 8.3 and is compulsory in v9. One >> of its uses is to protect against ROP based attacks. This is done by >> signing the Link Register whenever it is stored on the stack, and >> authenticating the value when it is loaded back from the stack. If an >> attacker were to try to change control flow by editing the stack then >> the authentication check of the Link Register will fail, causing a >> segfault when the function returns. >> >> On a system with PAC enabled, it is expected that all applications will >> be compiled with ROP protection. Fedora 33 and upwards already provide >> this. By compiling for ARMv8.0, GCC and LLVM will only use the set of >> PAC instructions that exist in the NOP space - on hardware without PAC, >> these instructions act as NOPs, allowing backward compatibility for >> negligible performance cost (2 NOPs per non-leaf function). >> >> Hardware is currently limited to the Apple M1 MacBooks. All testing has >> been done within a Fedora Docker image. A run of SpecJVM showed no >> difference to that of noise - which was surprising. >> >> The most important part of this patch is simply compiling using branch >> protection provided by GCC/LLVM. This protects all C++ code from being >> used in ROP attacks, removing all static ROP gadgets from use. >> >> The remainder of the patch adds ROP protection to runtime generated >> code, in both stubs and compiled Java code. Attacks here are much harder >> as ROP gadgets must be found dynamically at runtime. If/when AOT >> compilation is added to JDK, then all stubs and compiled Java will be >> susceptible ROP gadgets being found by static analysis and therefore >> potentially as vulnerable as C++ code. >> >> There are a number of places where the VM changes control flow by >> rewriting the stack or otherwise. I?ve done some analysis as to how >> these could also be used for attacks (which I didn?t want to post here). >> These areas can be protected ensuring the pointers to various stubs and >> entry points are stored in memory as signed pointers. These changes are >> simple to make (they can be reduced to a type change in common code and >> a few addition sign/auth calls in the backend), but there a lot of them >> and the total code change is fairly large. I?m happy to provide a few >> work in progress patches. >> >> In order to match the security benefits of the Apple Arm64e ABI across >> the whole of JDK, then all the changes mentioned above would be >> required. > > Alan Hayward has updated the pull request incrementally with two additional commits since the last revision: > > - Document pauth functions && remove OS split > - Update UseROPProtection description src/hotspot/cpu/aarch64/macroAssembler_aarch64.cpp line 5254: > 5252: // Also use before signing to check that the pointer is valid and hasn't already been signed. > 5253: // > 5254: void MacroAssembler::check_return_address(Register return_reg) { This commentary is excellent. Thanks. ------------- PR: https://git.openjdk.java.net/jdk/pull/6334 From prr at openjdk.java.net Fri Nov 12 17:48:33 2021 From: prr at openjdk.java.net (Phil Race) Date: Fri, 12 Nov 2021 17:48:33 GMT Subject: RFR: 8276905: Function frag_col has a deployment target which is incompatible with this OS [v2] In-Reply-To: References: <_OkC1YeToScvoDOBBhAZfQHSao_fjjsnw60prsCsWtE=.6ea0dbc2-0270-4caa-bcfc-baa7cb704365@github.com> Message-ID: On Thu, 11 Nov 2021 15:32:01 GMT, Jayathirth D V wrote: >> When JDK is built on BigSur with Xcode12.5 and used to run jtreg tests in macOS10.15 with Xcode <=12.4 we are getting compilation error. We dont set any deployment target when when we use xcrun to create .air file and this issue looks similar to https://developer.apple.com/forums/thread/105719. Also https://developer.apple.com/forums/thread/105719 states that even if they set app deployment target they need to explicitly set deployment in "xcrun metal". >> >> Made same change to use -mmacosx-version-min=10.12.00 in our make file. Whether we should keep 10.12.00 or 10.13/14 is open to discussion for metal pipeline. >> >> SwingSet2, J2DDemo & JCK/jtreg runs in CI are green. Also Vitaly(Submitter) has confirmed that patch resolves the issue. > > Jayathirth D V has updated the pull request incrementally with one additional commit since the last revision: > > Use Macro for version My understanding is that the flag here affects ONLY the metal compiler - for compiling metal shaders. And if you don't specify -Dsun.java2d.metal=true (since metal is off by default) its a 100% no-op for the rest of the JDK. And already, if you specify Dsun.java2d.metal=true and you are on 10.13 or lower, we do not honour the request so we haven't changed what platforms will work at all if we do it this way. So our effective deployment target for metal is already 10.14 And I also would not be surprised if someone wants to backport this to 17u, in which case a config change would have the effect of making 17u no longer run on macos 12 .. which I guess will happen sometime during the life of the LTS but right now ?? So making it a metal-specific change is what I think we should do FOR NOW and we can have a follow-on fix that aligns both of these .. maybe that is a subsequent JDK 18 fix, or perhaps it should be an early JDK 19 fix ? ------------- PR: https://git.openjdk.java.net/jdk/pull/6346 From erikj at openjdk.java.net Fri Nov 12 18:21:37 2021 From: erikj at openjdk.java.net (Erik Joelsson) Date: Fri, 12 Nov 2021 18:21:37 GMT Subject: RFR: 8276905: Function frag_col has a deployment target which is incompatible with this OS [v2] In-Reply-To: References: <_OkC1YeToScvoDOBBhAZfQHSao_fjjsnw60prsCsWtE=.6ea0dbc2-0270-4caa-bcfc-baa7cb704365@github.com> Message-ID: On Fri, 12 Nov 2021 17:45:09 GMT, Phil Race wrote: > My understanding is that the flag here affects ONLY the metal compiler - for compiling metal shaders. And if you don't specify -Dsun.java2d.metal=true (since metal is off by defau?lt) its a 100% no-op for the rest of the JDK. And already, if you specify Dsun.java2d.metal=true and you are on 10.13 or lower, we do not honour the request so we haven't changed what platforms will work at all if we do it this way. So our effective deployment target for metal is already 10.14 This explanation certainly makes a good case for giving this particular tool invocation special treatment. If we don't want to bump the general deployment target version, then this makes sense. > And I also would not be surprised if someone wants to backport this to 17u, in which case a config change would have the effect of making 17u no longer run on macos 12 .. which I guess will happen sometime during the life of the LTS but right now ?? I would be fine with backporting a general deployment target version change to 10.14 to 17u LTS. Apple are very aggressive with dropping support for older OS versions. We don't really have any obligations to maintain compatibility with legacy versions of macos. We just haven't actively dropped compatibility (which is different from what any particular distributor officially supports) unless there has been some technical limitation before. But, as you have now explained, we already have a runtime guard for this optional feature, so we aren't actually forced to change the global deployment target. > So making it a metal-specific change is what I think we should do FOR NOW and we can have a follow-on fix that aligns both of these .. maybe that is a subsequent JDK 18 fix, or perhaps it should be an early JDK 19 fix ? I'm ok with a metal specific change here, targeting 10.14, but it needs a comment referencing the global value and explaining that there is a runtime guard around this. Also note that on aarch64 the global value is 11.00.00, and in that case we should use the global value. ------------- PR: https://git.openjdk.java.net/jdk/pull/6346 From dcubed at openjdk.java.net Fri Nov 12 18:38:53 2021 From: dcubed at openjdk.java.net (Daniel D.Daugherty) Date: Fri, 12 Nov 2021 18:38:53 GMT Subject: RFR: JDK-8277071 [BACKOUT] JDK-8276743 Make openjdk build Zip Archive generation "reproducible" Message-ID: This reverts commit aeba65303479130d9bab74484accad5d7d116a40. ------------- Commit messages: - JDK-8277071 [BACKOUT] JDK-8276743 Make openjdk build Zip Archive generation "reproducible" Changes: https://git.openjdk.java.net/jdk/pull/6370/files Webrev: https://webrevs.openjdk.java.net/?repo=jdk&pr=6370&range=00 Issue: https://bugs.openjdk.java.net/browse/JDK-8277071 Stats: 261 lines in 4 files changed: 0 ins; 254 del; 7 mod Patch: https://git.openjdk.java.net/jdk/pull/6370.diff Fetch: git fetch https://git.openjdk.java.net/jdk pull/6370/head:pull/6370 PR: https://git.openjdk.java.net/jdk/pull/6370 From erikj at openjdk.java.net Fri Nov 12 18:41:34 2021 From: erikj at openjdk.java.net (Erik Joelsson) Date: Fri, 12 Nov 2021 18:41:34 GMT Subject: RFR: JDK-8277071 [BACKOUT] JDK-8276743 Make openjdk build Zip Archive generation "reproducible" In-Reply-To: References: Message-ID: On Fri, 12 Nov 2021 18:29:01 GMT, Daniel D. Daugherty wrote: > This reverts commit aeba65303479130d9bab74484accad5d7d116a40. Marked as reviewed by erikj (Reviewer). ------------- PR: https://git.openjdk.java.net/jdk/pull/6370 From dcubed at openjdk.java.net Fri Nov 12 18:50:38 2021 From: dcubed at openjdk.java.net (Daniel D.Daugherty) Date: Fri, 12 Nov 2021 18:50:38 GMT Subject: RFR: JDK-8277071 [BACKOUT] JDK-8276743 Make openjdk build Zip Archive generation "reproducible" In-Reply-To: References: Message-ID: <2E0hmXZsWMOwCm5sI7WTVp-E6oU-Wo1eYcWDqykYIGQ=.0575e212-7d25-430b-9ebb-514e3f294ff7@github.com> On Fri, 12 Nov 2021 18:38:42 GMT, Erik Joelsson wrote: >> This reverts commit aeba65303479130d9bab74484accad5d7d116a40. > > Marked as reviewed by erikj (Reviewer). @erikj79 - Thanks for the fast review! ------------- PR: https://git.openjdk.java.net/jdk/pull/6370 From dcubed at openjdk.java.net Fri Nov 12 18:50:39 2021 From: dcubed at openjdk.java.net (Daniel D.Daugherty) Date: Fri, 12 Nov 2021 18:50:39 GMT Subject: Integrated: JDK-8277071 [BACKOUT] JDK-8276743 Make openjdk build Zip Archive generation "reproducible" In-Reply-To: References: Message-ID: On Fri, 12 Nov 2021 18:29:01 GMT, Daniel D. Daugherty wrote: > This reverts commit aeba65303479130d9bab74484accad5d7d116a40. This pull request has now been integrated. Changeset: 74f3e69d Author: Daniel D. Daugherty URL: https://git.openjdk.java.net/jdk/commit/74f3e69dc888685558408e663df5d32cb906991f Stats: 261 lines in 4 files changed: 0 ins; 254 del; 7 mod 8277071: [BACKOUT] JDK-8276743 Make openjdk build Zip Archive generation "reproducible" Reviewed-by: erikj ------------- PR: https://git.openjdk.java.net/jdk/pull/6370 From prr at openjdk.java.net Fri Nov 12 20:30:41 2021 From: prr at openjdk.java.net (Phil Race) Date: Fri, 12 Nov 2021 20:30:41 GMT Subject: RFR: 8276905: Function frag_col has a deployment target which is incompatible with this OS [v2] In-Reply-To: References: <_OkC1YeToScvoDOBBhAZfQHSao_fjjsnw60prsCsWtE=.6ea0dbc2-0270-4caa-bcfc-baa7cb704365@github.com> Message-ID: On Thu, 11 Nov 2021 15:32:01 GMT, Jayathirth D V wrote: >> When JDK is built on BigSur with Xcode12.5 and used to run jtreg tests in macOS10.15 with Xcode <=12.4 we are getting compilation error. We dont set any deployment target when when we use xcrun to create .air file and this issue looks similar to https://developer.apple.com/forums/thread/105719. Also https://developer.apple.com/forums/thread/105719 states that even if they set app deployment target they need to explicitly set deployment in "xcrun metal". >> >> Made same change to use -mmacosx-version-min=10.12.00 in our make file. Whether we should keep 10.12.00 or 10.13/14 is open to discussion for metal pipeline. >> >> SwingSet2, J2DDemo & JCK/jtreg runs in CI are green. Also Vitaly(Submitter) has confirmed that patch resolves the issue. > > Jayathirth D V has updated the pull request incrementally with one additional commit since the last revision: > > Use Macro for version " Also note that on aarch64 the global value is 11.00.00, and in that case we should use the global value." I have no idea what happens if you specify 10.14 to the metal compiler on aarch64 Something else to check although probably the safest option is to make it 11.0 on that architecture ------------- PR: https://git.openjdk.java.net/jdk/pull/6346 From ysuenaga at openjdk.java.net Sat Nov 13 12:40:56 2021 From: ysuenaga at openjdk.java.net (Yasumasa Suenaga) Date: Sat, 13 Nov 2021 12:40:56 GMT Subject: RFR: 8277089: Use system binutils to build hsdis Message-ID: hsdis requires binutils source tree for building. Most of Linux distros provide binutils package. (e.g. binutils-devel from Fedora, binutils-dev from Ubuntu) It would be nice to be able to use them like zlib and lcms. Unfortunately bfdver.h would not be provided because it is not included install files (`make install`) in binutils. So I changed to use `SEC_ELF_OCTETS` macro to detect binutils version because it was introduced at the same time as `bfd_octets_per_byte()`. https://sourceware.org/git/?p=binutils-gdb.git;a=commit;f=bfd/bfd-in2.h;h=618265039f697eab9e72bb58b95fc2d32925df58 Please see [JDK-8244819](https://bugs.openjdk.java.net/browse/JDK-8244819) why we need version check. ------------- Commit messages: - 8277089: Use system binutils to build hsdis Changes: https://git.openjdk.java.net/jdk/pull/6378/files Webrev: https://webrevs.openjdk.java.net/?repo=jdk&pr=6378&range=00 Issue: https://bugs.openjdk.java.net/browse/JDK-8277089 Stats: 24 lines in 3 files changed: 19 ins; 1 del; 4 mod Patch: https://git.openjdk.java.net/jdk/pull/6378.diff Fetch: git fetch https://git.openjdk.java.net/jdk pull/6378/head:pull/6378 PR: https://git.openjdk.java.net/jdk/pull/6378 From jdv at openjdk.java.net Mon Nov 15 05:36:34 2021 From: jdv at openjdk.java.net (Jayathirth D V) Date: Mon, 15 Nov 2021 05:36:34 GMT Subject: RFR: 8276905: Function frag_col has a deployment target which is incompatible with this OS [v2] In-Reply-To: References: <_OkC1YeToScvoDOBBhAZfQHSao_fjjsnw60prsCsWtE=.6ea0dbc2-0270-4caa-bcfc-baa7cb704365@github.com> Message-ID: <5EbmAyaNA_izgnu9JjvBpNlWeyyeqg7RFdzuM1YzhI4=.854b5a74-8d13-4be9-911b-3aa43bc7495f@github.com> On Fri, 12 Nov 2021 03:59:52 GMT, Phil Race wrote: >> make/modules/java.desktop/lib/Awt2dLibraries.gmk line 850: >> >>> 848: COMMAND := $(METAL) -c -std=osx-metal2.0 \ >>> 849: -mmacosx-version-min=$(MACOSX_VERSION_MIN) \ >>> 850: -o $(SHADERS_AIR) $(SHADERS_SRC), \ >> >> No .. we decided metal requires macos 10.14 as a minimum. In fact it won't run (by policy) on earlier releases so settting it to what the broader JDK defines as a minimum is not appropriate right now. >> And I've no idea what restrictions we'd be placing on metal saying it can only use 10.12 features. >> Nor do I know what the xcode 12.4 used to build JDK 17 defaults to and how much a change to either 10.12 or 10.14 might require in re-testing. >> >> So - >> we should use 10.14 >> what's the actual impact of that on a current build using xcode 12.4 - does it change anything we use ? > > bear in mind "impact" might mean metal can't use some new faster code, not just runtime errors Hi Phil, Thanks for the review. I also had doubts using 10.12. And yes as you mentioned we dont use Metal pipeline for versions < macOS10.14. We dont use any Metal API which is specific to >macOS10.14(We had such usage when Lanai was under development but they were removed subsequently). So i dont see any performance impact of making xcrun to use 10.14 as minimum version. Thanks, Jay ------------- PR: https://git.openjdk.java.net/jdk/pull/6346 From jdv at openjdk.java.net Mon Nov 15 05:49:32 2021 From: jdv at openjdk.java.net (Jayathirth D V) Date: Mon, 15 Nov 2021 05:49:32 GMT Subject: RFR: 8276905: Function frag_col has a deployment target which is incompatible with this OS [v2] In-Reply-To: References: <_OkC1YeToScvoDOBBhAZfQHSao_fjjsnw60prsCsWtE=.6ea0dbc2-0270-4caa-bcfc-baa7cb704365@github.com> Message-ID: On Fri, 12 Nov 2021 20:27:32 GMT, Phil Race wrote: > " Also note that on aarch64 the global value is 11.00.00, and in that case we should use the global value." > > I have no idea what happens if you specify 10.14 to the metal compiler on aarch64 Something else to check although probably the safest option is to make it 11.0 on that architecture Sure. I will make it to use 11.0 on aarch64 and 10.14 on x64 and add comment mentioning that we have runtime guard to ignore metal pipeline for <10.14 versions. With current 10.12 minimum version patch there is CI run on aarch64 it has not thrown any error but i am not sure how that works. ------------- PR: https://git.openjdk.java.net/jdk/pull/6346 From jdv at openjdk.java.net Mon Nov 15 07:20:37 2021 From: jdv at openjdk.java.net (Jayathirth D V) Date: Mon, 15 Nov 2021 07:20:37 GMT Subject: RFR: 8276905: Function frag_col has a deployment target which is incompatible with this OS [v2] In-Reply-To: References: <_OkC1YeToScvoDOBBhAZfQHSao_fjjsnw60prsCsWtE=.6ea0dbc2-0270-4caa-bcfc-baa7cb704365@github.com> Message-ID: On Mon, 15 Nov 2021 05:46:30 GMT, Jayathirth D V wrote: > " Also note that on aarch64 the global value is 11.00.00, and in that case we should use the global value." > > I have no idea what happens if you specify 10.14 to the metal compiler on aarch64 Something else to check although probably the safest option is to make it 11.0 on that architecture A question popped up while doing this. My making 11.0 as minimum macosx version on aarch64, are we not restricting metal to be used only on >=11.0 on aarch 64? ------------- PR: https://git.openjdk.java.net/jdk/pull/6346 From duke at openjdk.java.net Mon Nov 15 09:07:11 2021 From: duke at openjdk.java.net (Alan Hayward) Date: Mon, 15 Nov 2021 09:07:11 GMT Subject: RFR: 8264130: PAC-RET protection for Linux/AArch64 [v4] In-Reply-To: References: Message-ID: > PAC is an optional feature in AArch64 8.3 and is compulsory in v9. One > of its uses is to protect against ROP based attacks. This is done by > signing the Link Register whenever it is stored on the stack, and > authenticating the value when it is loaded back from the stack. If an > attacker were to try to change control flow by editing the stack then > the authentication check of the Link Register will fail, causing a > segfault when the function returns. > > On a system with PAC enabled, it is expected that all applications will > be compiled with ROP protection. Fedora 33 and upwards already provide > this. By compiling for ARMv8.0, GCC and LLVM will only use the set of > PAC instructions that exist in the NOP space - on hardware without PAC, > these instructions act as NOPs, allowing backward compatibility for > negligible performance cost (2 NOPs per non-leaf function). > > Hardware is currently limited to the Apple M1 MacBooks. All testing has > been done within a Fedora Docker image. A run of SpecJVM showed no > difference to that of noise - which was surprising. > > The most important part of this patch is simply compiling using branch > protection provided by GCC/LLVM. This protects all C++ code from being > used in ROP attacks, removing all static ROP gadgets from use. > > The remainder of the patch adds ROP protection to runtime generated > code, in both stubs and compiled Java code. Attacks here are much harder > as ROP gadgets must be found dynamically at runtime. If/when AOT > compilation is added to JDK, then all stubs and compiled Java will be > susceptible ROP gadgets being found by static analysis and therefore > potentially as vulnerable as C++ code. > > There are a number of places where the VM changes control flow by > rewriting the stack or otherwise. I?ve done some analysis as to how > these could also be used for attacks (which I didn?t want to post here). > These areas can be protected ensuring the pointers to various stubs and > entry points are stored in memory as signed pointers. These changes are > simple to make (they can be reduced to a type change in common code and > a few addition sign/auth calls in the backend), but there a lot of them > and the total code change is fairly large. I?m happy to provide a few > work in progress patches. > > In order to match the security benefits of the Apple Arm64e ABI across > the whole of JDK, then all the changes mentioned above would be > required. Alan Hayward has updated the pull request with a new target base due to a merge or a rebase. The pull request now contains eight commits: - Merge master - Document pauth functions && remove OS split - Update UseROPProtection description - Simplify branch protection configure check - 8264130: PAC-RET protection for Linux/AArch64 PAC is an optional feature in AArch64 8.3 and is compulsory in v9. One of its uses is to protect against ROP based attacks. This is done by signing the Link Register whenever it is stored on the stack, and authenticating the value when it is loaded back from the stack. If an attacker were to try to change control flow by editing the stack then the authentication check of the Link Register will fail, causing a segfault when the function returns. On a system with PAC enabled, it is expected that all applications will be compiled with ROP protection. Fedora 33 and upwards already provide this. By compiling for ARMv8.0, GCC and LLVM will only use the set of PAC instructions that exist in the NOP space - on hardware without PAC, these instructions act as NOPs, allowing backward compatibility for negligible performance cost (2 NOPs per non-leaf function). Hardware is currently limited to the Apple M1 MacBooks. All testing has been done within a Fedora Docker image. A run of SpecJVM showed no difference to that of noise - which was surprising. The most important part of this patch is simply compiling using branch protection provided by GCC/LLVM. This protects all C++ code from being used in ROP attacks, removing all static ROP gadgets from use. The remainder of the patch adds ROP protection to runtime generated code, in both stubs and compiled Java code. Attacks here are much harder as ROP gadgets must be found dynamically at runtime. If/when AOT compilation is added to JDK, then all stubs and compiled Java will be susceptible ROP gadgets being found by static analysis and therefore potentially as vulnerable as C++ code. There are a number of places where the VM changes control flow by rewriting the stack or otherwise. I?ve done some analysis as to how these could also be used for attacks (which I didn?t want to post here). These areas can be protected ensuring the pointers to various stubs and entry points are stored in memory as signed pointers. These changes are simple to make (they can be reduced to a type change in common code and a few addition sign/auth calls in the backend), but there a lot of them and the total code change is fairly large. I?m happy to provide a few work in progress patches. In order to match the security benefits of the Apple Arm64e ABI across the whole of JDK, then all the changes mentioned above would be required. - Add PAC assembly instructions - Add AArch64 ROP protection runtime flag - Build with branch protection ------------- Changes: https://git.openjdk.java.net/jdk/pull/6334/files Webrev: https://webrevs.openjdk.java.net/?repo=jdk&pr=6334&range=03 Stats: 1436 lines in 25 files changed: 490 ins; 150 del; 796 mod Patch: https://git.openjdk.java.net/jdk/pull/6334.diff Fetch: git fetch https://git.openjdk.java.net/jdk pull/6334/head:pull/6334 PR: https://git.openjdk.java.net/jdk/pull/6334 From aph at openjdk.java.net Mon Nov 15 10:15:40 2021 From: aph at openjdk.java.net (Andrew Haley) Date: Mon, 15 Nov 2021 10:15:40 GMT Subject: RFR: 8264130: PAC-RET protection for Linux/AArch64 [v4] In-Reply-To: References: Message-ID: <9F2FQ7FTjc4Jzjf63x0pKeb2VPMsjcPQ-iQUo_rwCf4=.16d9e002-e6c9-4a4a-922c-ccbdf6e00eab@github.com> On Mon, 15 Nov 2021 09:07:11 GMT, Alan Hayward wrote: >> PAC is an optional feature in AArch64 8.3 and is compulsory in v9. One >> of its uses is to protect against ROP based attacks. This is done by >> signing the Link Register whenever it is stored on the stack, and >> authenticating the value when it is loaded back from the stack. If an >> attacker were to try to change control flow by editing the stack then >> the authentication check of the Link Register will fail, causing a >> segfault when the function returns. >> >> On a system with PAC enabled, it is expected that all applications will >> be compiled with ROP protection. Fedora 33 and upwards already provide >> this. By compiling for ARMv8.0, GCC and LLVM will only use the set of >> PAC instructions that exist in the NOP space - on hardware without PAC, >> these instructions act as NOPs, allowing backward compatibility for >> negligible performance cost (2 NOPs per non-leaf function). >> >> Hardware is currently limited to the Apple M1 MacBooks. All testing has >> been done within a Fedora Docker image. A run of SpecJVM showed no >> difference to that of noise - which was surprising. >> >> The most important part of this patch is simply compiling using branch >> protection provided by GCC/LLVM. This protects all C++ code from being >> used in ROP attacks, removing all static ROP gadgets from use. >> >> The remainder of the patch adds ROP protection to runtime generated >> code, in both stubs and compiled Java code. Attacks here are much harder >> as ROP gadgets must be found dynamically at runtime. If/when AOT >> compilation is added to JDK, then all stubs and compiled Java will be >> susceptible ROP gadgets being found by static analysis and therefore >> potentially as vulnerable as C++ code. >> >> There are a number of places where the VM changes control flow by >> rewriting the stack or otherwise. I?ve done some analysis as to how >> these could also be used for attacks (which I didn?t want to post here). >> These areas can be protected ensuring the pointers to various stubs and >> entry points are stored in memory as signed pointers. These changes are >> simple to make (they can be reduced to a type change in common code and >> a few addition sign/auth calls in the backend), but there a lot of them >> and the total code change is fairly large. I?m happy to provide a few >> work in progress patches. >> >> In order to match the security benefits of the Apple Arm64e ABI across >> the whole of JDK, then all the changes mentioned above would be >> required. > > Alan Hayward has updated the pull request with a new target base due to a merge or a rebase. The pull request now contains eight commits: > > - Merge master > - Document pauth functions && remove OS split > - Update UseROPProtection description > - Simplify branch protection configure check > - 8264130: PAC-RET protection for Linux/AArch64 > > PAC is an optional feature in AArch64 8.3 and is compulsory in v9. One > of its uses is to protect against ROP based attacks. This is done by > signing the Link Register whenever it is stored on the stack, and > authenticating the value when it is loaded back from the stack. If an > attacker were to try to change control flow by editing the stack then > the authentication check of the Link Register will fail, causing a > segfault when the function returns. > > On a system with PAC enabled, it is expected that all applications will > be compiled with ROP protection. Fedora 33 and upwards already provide > this. By compiling for ARMv8.0, GCC and LLVM will only use the set of > PAC instructions that exist in the NOP space - on hardware without PAC, > these instructions act as NOPs, allowing backward compatibility for > negligible performance cost (2 NOPs per non-leaf function). > > Hardware is currently limited to the Apple M1 MacBooks. All testing has > been done within a Fedora Docker image. A run of SpecJVM showed no > difference to that of noise - which was surprising. > > The most important part of this patch is simply compiling using branch > protection provided by GCC/LLVM. This protects all C++ code from being > used in ROP attacks, removing all static ROP gadgets from use. > > The remainder of the patch adds ROP protection to runtime generated > code, in both stubs and compiled Java code. Attacks here are much harder > as ROP gadgets must be found dynamically at runtime. If/when AOT > compilation is added to JDK, then all stubs and compiled Java will be > susceptible ROP gadgets being found by static analysis and therefore > potentially as vulnerable as C++ code. > > There are a number of places where the VM changes control flow by > rewriting the stack or otherwise. I?ve done some analysis as to how > these could also be used for attacks (which I didn?t want to post here). > These areas can be protected ensuring the pointers to various stubs and > entry points are stored in memory as signed pointers. These changes are > simple to make (they can be reduced to a type change in common code and > a few addition sign/auth calls in the backend), but there a lot of them > and the total code change is fairly large. I?m happy to provide a few > work in progress patches. > > In order to match the security benefits of the Apple Arm64e ABI across > the whole of JDK, then all the changes mentioned above would be > required. > - Add PAC assembly instructions > - Add AArch64 ROP protection runtime flag > - Build with branch protection src/hotspot/cpu/aarch64/c1_Runtime1_aarch64.cpp line 452: > 450: > 451: // only r0 is valid at this time, all other registers have been destroyed by the runtime call > 452: __ invalidate_registers(false, true, true, true, true, true); Not so: `lr` is live. ------------- PR: https://git.openjdk.java.net/jdk/pull/6334 From adinn at openjdk.java.net Mon Nov 15 10:15:41 2021 From: adinn at openjdk.java.net (Andrew Dinn) Date: Mon, 15 Nov 2021 10:15:41 GMT Subject: RFR: 8264130: PAC-RET protection for Linux/AArch64 [v4] In-Reply-To: References: Message-ID: <9psGxGDAGJTaAW2jtH3v3A6jsuq4x7aOMXMgJEyeLLI=.21f995de-9192-4483-a378-0a54e3d3745d@github.com> On Mon, 15 Nov 2021 09:07:11 GMT, Alan Hayward wrote: >> PAC is an optional feature in AArch64 8.3 and is compulsory in v9. One >> of its uses is to protect against ROP based attacks. This is done by >> signing the Link Register whenever it is stored on the stack, and >> authenticating the value when it is loaded back from the stack. If an >> attacker were to try to change control flow by editing the stack then >> the authentication check of the Link Register will fail, causing a >> segfault when the function returns. >> >> On a system with PAC enabled, it is expected that all applications will >> be compiled with ROP protection. Fedora 33 and upwards already provide >> this. By compiling for ARMv8.0, GCC and LLVM will only use the set of >> PAC instructions that exist in the NOP space - on hardware without PAC, >> these instructions act as NOPs, allowing backward compatibility for >> negligible performance cost (2 NOPs per non-leaf function). >> >> Hardware is currently limited to the Apple M1 MacBooks. All testing has >> been done within a Fedora Docker image. A run of SpecJVM showed no >> difference to that of noise - which was surprising. >> >> The most important part of this patch is simply compiling using branch >> protection provided by GCC/LLVM. This protects all C++ code from being >> used in ROP attacks, removing all static ROP gadgets from use. >> >> The remainder of the patch adds ROP protection to runtime generated >> code, in both stubs and compiled Java code. Attacks here are much harder >> as ROP gadgets must be found dynamically at runtime. If/when AOT >> compilation is added to JDK, then all stubs and compiled Java will be >> susceptible ROP gadgets being found by static analysis and therefore >> potentially as vulnerable as C++ code. >> >> There are a number of places where the VM changes control flow by >> rewriting the stack or otherwise. I?ve done some analysis as to how >> these could also be used for attacks (which I didn?t want to post here). >> These areas can be protected ensuring the pointers to various stubs and >> entry points are stored in memory as signed pointers. These changes are >> simple to make (they can be reduced to a type change in common code and >> a few addition sign/auth calls in the backend), but there a lot of them >> and the total code change is fairly large. I?m happy to provide a few >> work in progress patches. >> >> In order to match the security benefits of the Apple Arm64e ABI across >> the whole of JDK, then all the changes mentioned above would be >> required. > > Alan Hayward has updated the pull request with a new target base due to a merge or a rebase. The pull request now contains eight commits: > > - Merge master > - Document pauth functions && remove OS split > - Update UseROPProtection description > - Simplify branch protection configure check > - 8264130: PAC-RET protection for Linux/AArch64 > > PAC is an optional feature in AArch64 8.3 and is compulsory in v9. One > of its uses is to protect against ROP based attacks. This is done by > signing the Link Register whenever it is stored on the stack, and > authenticating the value when it is loaded back from the stack. If an > attacker were to try to change control flow by editing the stack then > the authentication check of the Link Register will fail, causing a > segfault when the function returns. > > On a system with PAC enabled, it is expected that all applications will > be compiled with ROP protection. Fedora 33 and upwards already provide > this. By compiling for ARMv8.0, GCC and LLVM will only use the set of > PAC instructions that exist in the NOP space - on hardware without PAC, > these instructions act as NOPs, allowing backward compatibility for > negligible performance cost (2 NOPs per non-leaf function). > > Hardware is currently limited to the Apple M1 MacBooks. All testing has > been done within a Fedora Docker image. A run of SpecJVM showed no > difference to that of noise - which was surprising. > > The most important part of this patch is simply compiling using branch > protection provided by GCC/LLVM. This protects all C++ code from being > used in ROP attacks, removing all static ROP gadgets from use. > > The remainder of the patch adds ROP protection to runtime generated > code, in both stubs and compiled Java code. Attacks here are much harder > as ROP gadgets must be found dynamically at runtime. If/when AOT > compilation is added to JDK, then all stubs and compiled Java will be > susceptible ROP gadgets being found by static analysis and therefore > potentially as vulnerable as C++ code. > > There are a number of places where the VM changes control flow by > rewriting the stack or otherwise. I?ve done some analysis as to how > these could also be used for attacks (which I didn?t want to post here). > These areas can be protected ensuring the pointers to various stubs and > entry points are stored in memory as signed pointers. These changes are > simple to make (they can be reduced to a type change in common code and > a few addition sign/auth calls in the backend), but there a lot of them > and the total code change is fairly large. I?m happy to provide a few > work in progress patches. > > In order to match the security benefits of the Apple Arm64e ABI across > the whole of JDK, then all the changes mentioned above would be > required. > - Add PAC assembly instructions > - Add AArch64 ROP protection runtime flag > - Build with branch protection src/hotspot/cpu/aarch64/pauth_aarch64.hpp line 33: > 31: > 32: // Support for ROP Protection in VM code. > 33: // This is provided by via the AArch64 PAC feature. "by via" should just be "via" ------------- PR: https://git.openjdk.java.net/jdk/pull/6334 From aph at openjdk.java.net Mon Nov 15 10:18:42 2021 From: aph at openjdk.java.net (Andrew Haley) Date: Mon, 15 Nov 2021 10:18:42 GMT Subject: RFR: 8264130: PAC-RET protection for Linux/AArch64 [v4] In-Reply-To: References: Message-ID: <81l6r4GfgLq9L4qhlvi_VWKE46vPqhspX-d7NG6Qux0=.4dbf25ed-4c3f-415b-9ffc-ddaf69211cf2@github.com> On Mon, 15 Nov 2021 09:07:11 GMT, Alan Hayward wrote: >> PAC is an optional feature in AArch64 8.3 and is compulsory in v9. One >> of its uses is to protect against ROP based attacks. This is done by >> signing the Link Register whenever it is stored on the stack, and >> authenticating the value when it is loaded back from the stack. If an >> attacker were to try to change control flow by editing the stack then >> the authentication check of the Link Register will fail, causing a >> segfault when the function returns. >> >> On a system with PAC enabled, it is expected that all applications will >> be compiled with ROP protection. Fedora 33 and upwards already provide >> this. By compiling for ARMv8.0, GCC and LLVM will only use the set of >> PAC instructions that exist in the NOP space - on hardware without PAC, >> these instructions act as NOPs, allowing backward compatibility for >> negligible performance cost (2 NOPs per non-leaf function). >> >> Hardware is currently limited to the Apple M1 MacBooks. All testing has >> been done within a Fedora Docker image. A run of SpecJVM showed no >> difference to that of noise - which was surprising. >> >> The most important part of this patch is simply compiling using branch >> protection provided by GCC/LLVM. This protects all C++ code from being >> used in ROP attacks, removing all static ROP gadgets from use. >> >> The remainder of the patch adds ROP protection to runtime generated >> code, in both stubs and compiled Java code. Attacks here are much harder >> as ROP gadgets must be found dynamically at runtime. If/when AOT >> compilation is added to JDK, then all stubs and compiled Java will be >> susceptible ROP gadgets being found by static analysis and therefore >> potentially as vulnerable as C++ code. >> >> There are a number of places where the VM changes control flow by >> rewriting the stack or otherwise. I?ve done some analysis as to how >> these could also be used for attacks (which I didn?t want to post here). >> These areas can be protected ensuring the pointers to various stubs and >> entry points are stored in memory as signed pointers. These changes are >> simple to make (they can be reduced to a type change in common code and >> a few addition sign/auth calls in the backend), but there a lot of them >> and the total code change is fairly large. I?m happy to provide a few >> work in progress patches. >> >> In order to match the security benefits of the Apple Arm64e ABI across >> the whole of JDK, then all the changes mentioned above would be >> required. > > Alan Hayward has updated the pull request with a new target base due to a merge or a rebase. The pull request now contains eight commits: > > - Merge master > - Document pauth functions && remove OS split > - Update UseROPProtection description > - Simplify branch protection configure check > - 8264130: PAC-RET protection for Linux/AArch64 > > PAC is an optional feature in AArch64 8.3 and is compulsory in v9. One > of its uses is to protect against ROP based attacks. This is done by > signing the Link Register whenever it is stored on the stack, and > authenticating the value when it is loaded back from the stack. If an > attacker were to try to change control flow by editing the stack then > the authentication check of the Link Register will fail, causing a > segfault when the function returns. > > On a system with PAC enabled, it is expected that all applications will > be compiled with ROP protection. Fedora 33 and upwards already provide > this. By compiling for ARMv8.0, GCC and LLVM will only use the set of > PAC instructions that exist in the NOP space - on hardware without PAC, > these instructions act as NOPs, allowing backward compatibility for > negligible performance cost (2 NOPs per non-leaf function). > > Hardware is currently limited to the Apple M1 MacBooks. All testing has > been done within a Fedora Docker image. A run of SpecJVM showed no > difference to that of noise - which was surprising. > > The most important part of this patch is simply compiling using branch > protection provided by GCC/LLVM. This protects all C++ code from being > used in ROP attacks, removing all static ROP gadgets from use. > > The remainder of the patch adds ROP protection to runtime generated > code, in both stubs and compiled Java code. Attacks here are much harder > as ROP gadgets must be found dynamically at runtime. If/when AOT > compilation is added to JDK, then all stubs and compiled Java will be > susceptible ROP gadgets being found by static analysis and therefore > potentially as vulnerable as C++ code. > > There are a number of places where the VM changes control flow by > rewriting the stack or otherwise. I?ve done some analysis as to how > these could also be used for attacks (which I didn?t want to post here). > These areas can be protected ensuring the pointers to various stubs and > entry points are stored in memory as signed pointers. These changes are > simple to make (they can be reduced to a type change in common code and > a few addition sign/auth calls in the backend), but there a lot of them > and the total code change is fairly large. I?m happy to provide a few > work in progress patches. > > In order to match the security benefits of the Apple Arm64e ABI across > the whole of JDK, then all the changes mentioned above would be > required. > - Add PAC assembly instructions > - Add AArch64 ROP protection runtime flag > - Build with branch protection src/hotspot/cpu/aarch64/c1_Runtime1_aarch64.cpp line 452: > 450: // patch the return address, this stub will directly return to the exception handler > 451: __ str(r0, Address(rfp, 1*BytesPerWord)); > 452: Please explain the reason for this change, that leaves `lr` live across `restore_live_registers()`. ------------- PR: https://git.openjdk.java.net/jdk/pull/6334 From aph at openjdk.java.net Mon Nov 15 10:23:38 2021 From: aph at openjdk.java.net (Andrew Haley) Date: Mon, 15 Nov 2021 10:23:38 GMT Subject: RFR: 8264130: PAC-RET protection for Linux/AArch64 [v4] In-Reply-To: References: Message-ID: On Mon, 15 Nov 2021 09:07:11 GMT, Alan Hayward wrote: >> PAC is an optional feature in AArch64 8.3 and is compulsory in v9. One >> of its uses is to protect against ROP based attacks. This is done by >> signing the Link Register whenever it is stored on the stack, and >> authenticating the value when it is loaded back from the stack. If an >> attacker were to try to change control flow by editing the stack then >> the authentication check of the Link Register will fail, causing a >> segfault when the function returns. >> >> On a system with PAC enabled, it is expected that all applications will >> be compiled with ROP protection. Fedora 33 and upwards already provide >> this. By compiling for ARMv8.0, GCC and LLVM will only use the set of >> PAC instructions that exist in the NOP space - on hardware without PAC, >> these instructions act as NOPs, allowing backward compatibility for >> negligible performance cost (2 NOPs per non-leaf function). >> >> Hardware is currently limited to the Apple M1 MacBooks. All testing has >> been done within a Fedora Docker image. A run of SpecJVM showed no >> difference to that of noise - which was surprising. >> >> The most important part of this patch is simply compiling using branch >> protection provided by GCC/LLVM. This protects all C++ code from being >> used in ROP attacks, removing all static ROP gadgets from use. >> >> The remainder of the patch adds ROP protection to runtime generated >> code, in both stubs and compiled Java code. Attacks here are much harder >> as ROP gadgets must be found dynamically at runtime. If/when AOT >> compilation is added to JDK, then all stubs and compiled Java will be >> susceptible ROP gadgets being found by static analysis and therefore >> potentially as vulnerable as C++ code. >> >> There are a number of places where the VM changes control flow by >> rewriting the stack or otherwise. I?ve done some analysis as to how >> these could also be used for attacks (which I didn?t want to post here). >> These areas can be protected ensuring the pointers to various stubs and >> entry points are stored in memory as signed pointers. These changes are >> simple to make (they can be reduced to a type change in common code and >> a few addition sign/auth calls in the backend), but there a lot of them >> and the total code change is fairly large. I?m happy to provide a few >> work in progress patches. >> >> In order to match the security benefits of the Apple Arm64e ABI across >> the whole of JDK, then all the changes mentioned above would be >> required. > > Alan Hayward has updated the pull request with a new target base due to a merge or a rebase. The pull request now contains eight commits: > > - Merge master > - Document pauth functions && remove OS split > - Update UseROPProtection description > - Simplify branch protection configure check > - 8264130: PAC-RET protection for Linux/AArch64 > > PAC is an optional feature in AArch64 8.3 and is compulsory in v9. One > of its uses is to protect against ROP based attacks. This is done by > signing the Link Register whenever it is stored on the stack, and > authenticating the value when it is loaded back from the stack. If an > attacker were to try to change control flow by editing the stack then > the authentication check of the Link Register will fail, causing a > segfault when the function returns. > > On a system with PAC enabled, it is expected that all applications will > be compiled with ROP protection. Fedora 33 and upwards already provide > this. By compiling for ARMv8.0, GCC and LLVM will only use the set of > PAC instructions that exist in the NOP space - on hardware without PAC, > these instructions act as NOPs, allowing backward compatibility for > negligible performance cost (2 NOPs per non-leaf function). > > Hardware is currently limited to the Apple M1 MacBooks. All testing has > been done within a Fedora Docker image. A run of SpecJVM showed no > difference to that of noise - which was surprising. > > The most important part of this patch is simply compiling using branch > protection provided by GCC/LLVM. This protects all C++ code from being > used in ROP attacks, removing all static ROP gadgets from use. > > The remainder of the patch adds ROP protection to runtime generated > code, in both stubs and compiled Java code. Attacks here are much harder > as ROP gadgets must be found dynamically at runtime. If/when AOT > compilation is added to JDK, then all stubs and compiled Java will be > susceptible ROP gadgets being found by static analysis and therefore > potentially as vulnerable as C++ code. > > There are a number of places where the VM changes control flow by > rewriting the stack or otherwise. I?ve done some analysis as to how > these could also be used for attacks (which I didn?t want to post here). > These areas can be protected ensuring the pointers to various stubs and > entry points are stored in memory as signed pointers. These changes are > simple to make (they can be reduced to a type change in common code and > a few addition sign/auth calls in the backend), but there a lot of them > and the total code change is fairly large. I?m happy to provide a few > work in progress patches. > > In order to match the security benefits of the Apple Arm64e ABI across > the whole of JDK, then all the changes mentioned above would be > required. > - Add PAC assembly instructions > - Add AArch64 ROP protection runtime flag > - Build with branch protection src/hotspot/cpu/aarch64/pauth_aarch64.hpp line 132: > 130: // Authenticate or strip a return value. Use for efficiency and only when the safety of the data > 131: // isn't an issue - for example when viewing the stack. > 132: // So, whether this function authenticates or strips the address depends only on debugging? The vague name makes the callers hard to read. ------------- PR: https://git.openjdk.java.net/jdk/pull/6334 From ihse at openjdk.java.net Mon Nov 15 10:26:36 2021 From: ihse at openjdk.java.net (Magnus Ihse Bursie) Date: Mon, 15 Nov 2021 10:26:36 GMT Subject: RFR: 8277089: Use system binutils to build hsdis In-Reply-To: References: Message-ID: On Sat, 13 Nov 2021 08:08:53 GMT, Yasumasa Suenaga wrote: > hsdis requires binutils source tree for building. Most of Linux distros provide binutils package. (e.g. binutils-devel from Fedora, binutils-dev from Ubuntu) > It would be nice to be able to use them like zlib and lcms. > > Unfortunately bfdver.h would not be provided because it is not included install files (`make install`) in binutils. So I changed to use `SEC_ELF_OCTETS` macro to detect binutils version because it was introduced at the same time as `bfd_octets_per_byte()`. > > https://sourceware.org/git/?p=binutils-gdb.git;a=commit;f=bfd/bfd-in2.h;h=618265039f697eab9e72bb58b95fc2d32925df58 > > Please see [JDK-8244819](https://bugs.openjdk.java.net/browse/JDK-8244819) why we need version check. The basic idea is fine. I also think checking for `SEC_ELF_OCTETS` in the source code, instead of the version number, is actually an improvement. The one thing that itches me a bit is what happens when you specify `--with-binutils=system` and a dependent library is not found: AC_CHECK_LIB(iberty, xmalloc, [ HSDIS_LIBS="$HSDIS_LIBS -liberty" ], [ AC_MSG_ERROR([libiberty not found]) ]) Then the build will fail with no clear indication on why. Instead, I'd recommend that you restructure slightly. First check if with_binutils is system. If so, run your lib checks but like this: AC_CHECK_LIB(iberty, xmalloc, [ HSDIS_LIBS="$HSDIS_LIBS -liberty" ], [ bintils_system_error="libiberty not found" ]) Then you go check the value of with_binutils again in the "switch" statement. And you can replace `AC_MSG_CHECKING` outside the switch statement again. If it is system, you check if `bintils_system_error` is non-empty. If so, you fail and explain that this-and-this error prevented system from working. What distributions have you tested this on? ------------- PR: https://git.openjdk.java.net/jdk/pull/6378 From adinn at openjdk.java.net Mon Nov 15 10:31:52 2021 From: adinn at openjdk.java.net (Andrew Dinn) Date: Mon, 15 Nov 2021 10:31:52 GMT Subject: RFR: 8264130: PAC-RET protection for Linux/AArch64 [v4] In-Reply-To: References: Message-ID: <464NS7NEldWHQM0Q8PF_MzPfl9O0CUj9GjbeI_qdjEc=.758f4244-8239-4e5d-bb08-a0dec85c2a06@github.com> On Mon, 15 Nov 2021 09:07:11 GMT, Alan Hayward wrote: >> PAC is an optional feature in AArch64 8.3 and is compulsory in v9. One >> of its uses is to protect against ROP based attacks. This is done by >> signing the Link Register whenever it is stored on the stack, and >> authenticating the value when it is loaded back from the stack. If an >> attacker were to try to change control flow by editing the stack then >> the authentication check of the Link Register will fail, causing a >> segfault when the function returns. >> >> On a system with PAC enabled, it is expected that all applications will >> be compiled with ROP protection. Fedora 33 and upwards already provide >> this. By compiling for ARMv8.0, GCC and LLVM will only use the set of >> PAC instructions that exist in the NOP space - on hardware without PAC, >> these instructions act as NOPs, allowing backward compatibility for >> negligible performance cost (2 NOPs per non-leaf function). >> >> Hardware is currently limited to the Apple M1 MacBooks. All testing has >> been done within a Fedora Docker image. A run of SpecJVM showed no >> difference to that of noise - which was surprising. >> >> The most important part of this patch is simply compiling using branch >> protection provided by GCC/LLVM. This protects all C++ code from being >> used in ROP attacks, removing all static ROP gadgets from use. >> >> The remainder of the patch adds ROP protection to runtime generated >> code, in both stubs and compiled Java code. Attacks here are much harder >> as ROP gadgets must be found dynamically at runtime. If/when AOT >> compilation is added to JDK, then all stubs and compiled Java will be >> susceptible ROP gadgets being found by static analysis and therefore >> potentially as vulnerable as C++ code. >> >> There are a number of places where the VM changes control flow by >> rewriting the stack or otherwise. I?ve done some analysis as to how >> these could also be used for attacks (which I didn?t want to post here). >> These areas can be protected ensuring the pointers to various stubs and >> entry points are stored in memory as signed pointers. These changes are >> simple to make (they can be reduced to a type change in common code and >> a few addition sign/auth calls in the backend), but there a lot of them >> and the total code change is fairly large. I?m happy to provide a few >> work in progress patches. >> >> In order to match the security benefits of the Apple Arm64e ABI across >> the whole of JDK, then all the changes mentioned above would be >> required. > > Alan Hayward has updated the pull request with a new target base due to a merge or a rebase. The pull request now contains eight commits: > > - Merge master > - Document pauth functions && remove OS split > - Update UseROPProtection description > - Simplify branch protection configure check > - 8264130: PAC-RET protection for Linux/AArch64 > > PAC is an optional feature in AArch64 8.3 and is compulsory in v9. One > of its uses is to protect against ROP based attacks. This is done by > signing the Link Register whenever it is stored on the stack, and > authenticating the value when it is loaded back from the stack. If an > attacker were to try to change control flow by editing the stack then > the authentication check of the Link Register will fail, causing a > segfault when the function returns. > > On a system with PAC enabled, it is expected that all applications will > be compiled with ROP protection. Fedora 33 and upwards already provide > this. By compiling for ARMv8.0, GCC and LLVM will only use the set of > PAC instructions that exist in the NOP space - on hardware without PAC, > these instructions act as NOPs, allowing backward compatibility for > negligible performance cost (2 NOPs per non-leaf function). > > Hardware is currently limited to the Apple M1 MacBooks. All testing has > been done within a Fedora Docker image. A run of SpecJVM showed no > difference to that of noise - which was surprising. > > The most important part of this patch is simply compiling using branch > protection provided by GCC/LLVM. This protects all C++ code from being > used in ROP attacks, removing all static ROP gadgets from use. > > The remainder of the patch adds ROP protection to runtime generated > code, in both stubs and compiled Java code. Attacks here are much harder > as ROP gadgets must be found dynamically at runtime. If/when AOT > compilation is added to JDK, then all stubs and compiled Java will be > susceptible ROP gadgets being found by static analysis and therefore > potentially as vulnerable as C++ code. > > There are a number of places where the VM changes control flow by > rewriting the stack or otherwise. I?ve done some analysis as to how > these could also be used for attacks (which I didn?t want to post here). > These areas can be protected ensuring the pointers to various stubs and > entry points are stored in memory as signed pointers. These changes are > simple to make (they can be reduced to a type change in common code and > a few addition sign/auth calls in the backend), but there a lot of them > and the total code change is fairly large. I?m happy to provide a few > work in progress patches. > > In order to match the security benefits of the Apple Arm64e ABI across > the whole of JDK, then all the changes mentioned above would be > required. > - Add PAC assembly instructions > - Add AArch64 ROP protection runtime flag > - Build with branch protection This is much clearer and looks good to push modulo a minor typo I noted in a comment. ------------- Marked as reviewed by adinn (Reviewer). PR: https://git.openjdk.java.net/jdk/pull/6334 From duke at openjdk.java.net Mon Nov 15 10:42:38 2021 From: duke at openjdk.java.net (Alan Hayward) Date: Mon, 15 Nov 2021 10:42:38 GMT Subject: RFR: 8264130: PAC-RET protection for Linux/AArch64 [v4] In-Reply-To: References: Message-ID: On Wed, 10 Nov 2021 15:01:51 GMT, Alan Hayward wrote: >> src/hotspot/os_cpu/bsd_aarch64/pauth_bsd_aarch64.inline.hpp line 25: >> >>> 23: */ >>> 24: >>> 25: #ifndef OS_CPU_BSD_AARCH64_PAUTH_BSD_AARCH64_INLINE_HPP >> >> Are these two files different enough to separate them for BSD and Linux? > > My motivation was to avoid having any ifdefs - but we need one anyway for the apple ifdef. > > If I merged the two we would end up with just the contents of the BSD version of the file. > > There is also the windows version of the file, which for now has empty functions. If PAC in windows is added, that'll either use the same code or maybe Windows will provide an API (like the Apple one). Merging everything would mean windows gains the UseROPProtection check. >Are these two files different enough to separate them for BSD and Linux? Merging these files then broke everything for windows (because the asm function is different). Having a "ifdef apple, elseif windows else" doesn't really make sense, so I'll split the files out again. ------------- PR: https://git.openjdk.java.net/jdk/pull/6334 From duke at openjdk.java.net Mon Nov 15 11:01:36 2021 From: duke at openjdk.java.net (Alan Hayward) Date: Mon, 15 Nov 2021 11:01:36 GMT Subject: RFR: 8264130: PAC-RET protection for Linux/AArch64 [v4] In-Reply-To: References: Message-ID: On Mon, 15 Nov 2021 10:20:15 GMT, Andrew Haley wrote: >whether this function authenticates or strips the address depends only on debugging? Yes. We only need to strip the value, because we're not jumping to the lr value, only viewing it. The interface is different to a strip (as we need to pass in the modifier). How about something like pauth_authenticate_fast() ? or pauth_authenticate_unsafe() ? Alternatively, this function is only called by the functions in Frame, so the frequency of use is probably low enough (compared to the sign/auth every function) that it's not going to cause any performance issues. So, could just replace with calls to pauth_authenticate. I think that might be the best option. ------------- PR: https://git.openjdk.java.net/jdk/pull/6334 From aph at openjdk.java.net Mon Nov 15 11:11:37 2021 From: aph at openjdk.java.net (Andrew Haley) Date: Mon, 15 Nov 2021 11:11:37 GMT Subject: RFR: 8264130: PAC-RET protection for Linux/AArch64 [v4] In-Reply-To: References: Message-ID: On Mon, 15 Nov 2021 10:58:06 GMT, Alan Hayward wrote: >> src/hotspot/cpu/aarch64/pauth_aarch64.hpp line 132: >> >>> 130: // Authenticate or strip a return value. Use for efficiency and only when the safety of the data >>> 131: // isn't an issue - for example when viewing the stack. >>> 132: // >> >> So, whether this function authenticates or strips the address depends only on debugging? The vague name makes the callers hard to read. > >>whether this function authenticates or strips the address depends only on debugging? > > Yes. We only need to strip the value, because we're not jumping to the lr value, only viewing it. > > The interface is different to a strip (as we need to pass in the modifier). > > How about something like pauth_authenticate_fast() ? or pauth_authenticate_unsafe() ? > > Alternatively, this function is only called by the functions in Frame, so the frequency of use is probably low enough (compared to the sign/auth every function) that it's not going to cause any performance issues. So, could just replace with calls to pauth_authenticate. I think that might be the best option. A simple rule here: function names go with what the release version does. So I'd go with the actual purpose, which is `pauth_strip_addr_for_debuginfo()`. That's right, isn't it? You only want this thing for stack traces, logs, etc. ------------- PR: https://git.openjdk.java.net/jdk/pull/6334 From duke at openjdk.java.net Mon Nov 15 11:24:46 2021 From: duke at openjdk.java.net (Alan Hayward) Date: Mon, 15 Nov 2021 11:24:46 GMT Subject: RFR: 8264130: PAC-RET protection for Linux/AArch64 [v4] In-Reply-To: <81l6r4GfgLq9L4qhlvi_VWKE46vPqhspX-d7NG6Qux0=.4dbf25ed-4c3f-415b-9ffc-ddaf69211cf2@github.com> References: <81l6r4GfgLq9L4qhlvi_VWKE46vPqhspX-d7NG6Qux0=.4dbf25ed-4c3f-415b-9ffc-ddaf69211cf2@github.com> Message-ID: On Mon, 15 Nov 2021 10:15:41 GMT, Andrew Haley wrote: >> Alan Hayward has updated the pull request with a new target base due to a merge or a rebase. The pull request now contains eight commits: >> >> - Merge master >> - Document pauth functions && remove OS split >> - Update UseROPProtection description >> - Simplify branch protection configure check >> - 8264130: PAC-RET protection for Linux/AArch64 >> >> PAC is an optional feature in AArch64 8.3 and is compulsory in v9. One >> of its uses is to protect against ROP based attacks. This is done by >> signing the Link Register whenever it is stored on the stack, and >> authenticating the value when it is loaded back from the stack. If an >> attacker were to try to change control flow by editing the stack then >> the authentication check of the Link Register will fail, causing a >> segfault when the function returns. >> >> On a system with PAC enabled, it is expected that all applications will >> be compiled with ROP protection. Fedora 33 and upwards already provide >> this. By compiling for ARMv8.0, GCC and LLVM will only use the set of >> PAC instructions that exist in the NOP space - on hardware without PAC, >> these instructions act as NOPs, allowing backward compatibility for >> negligible performance cost (2 NOPs per non-leaf function). >> >> Hardware is currently limited to the Apple M1 MacBooks. All testing has >> been done within a Fedora Docker image. A run of SpecJVM showed no >> difference to that of noise - which was surprising. >> >> The most important part of this patch is simply compiling using branch >> protection provided by GCC/LLVM. This protects all C++ code from being >> used in ROP attacks, removing all static ROP gadgets from use. >> >> The remainder of the patch adds ROP protection to runtime generated >> code, in both stubs and compiled Java code. Attacks here are much harder >> as ROP gadgets must be found dynamically at runtime. If/when AOT >> compilation is added to JDK, then all stubs and compiled Java will be >> susceptible ROP gadgets being found by static analysis and therefore >> potentially as vulnerable as C++ code. >> >> There are a number of places where the VM changes control flow by >> rewriting the stack or otherwise. I?ve done some analysis as to how >> these could also be used for attacks (which I didn?t want to post here). >> These areas can be protected ensuring the pointers to various stubs and >> entry points are stored in memory as signed pointers. These changes are >> simple to make (they can be reduced to a type change in common code and >> a few addition sign/auth calls in the backend), but there a lot of them >> and the total code change is fairly large. I?m happy to provide a few >> work in progress patches. >> >> In order to match the security benefits of the Apple Arm64e ABI across >> the whole of JDK, then all the changes mentioned above would be >> required. >> - Add PAC assembly instructions >> - Add AArch64 ROP protection runtime flag >> - Build with branch protection > > src/hotspot/cpu/aarch64/c1_Runtime1_aarch64.cpp line 452: > >> 450: // patch the return address, this stub will directly return to the exception handler >> 451: __ str(r0, Address(rfp, 1*BytesPerWord)); >> 452: > > Please explain the reason for this change, that leaves `lr` live across `restore_live_registers()`. In the original code: *save r0 to the lr location on the stack *restore_live_registers *Standard return: remove stack frame, load lr and fp off the stack, jump to lr. With PAC it would now be: *Sign r0 then save it to the lr location on the stack *restore_live_registers *Standard return: remove stack frame, load lr and fp off the stack, auth lr, jump to lr. After reading the code in restore_live_registers, it doesn't touch lr and so seemed odd to have the save to the stack, only to restore it directly afterwards. ------------- PR: https://git.openjdk.java.net/jdk/pull/6334 From aph at openjdk.java.net Mon Nov 15 11:33:41 2021 From: aph at openjdk.java.net (Andrew Haley) Date: Mon, 15 Nov 2021 11:33:41 GMT Subject: RFR: 8264130: PAC-RET protection for Linux/AArch64 [v4] In-Reply-To: References: <81l6r4GfgLq9L4qhlvi_VWKE46vPqhspX-d7NG6Qux0=.4dbf25ed-4c3f-415b-9ffc-ddaf69211cf2@github.com> Message-ID: <1MtnvG48AfLFiyinjPMaT6KJ1MdM15mp2k2UMrryCgk=.7d91bba7-f1a7-4a26-8a4d-e1388a8b88ea@github.com> On Mon, 15 Nov 2021 11:21:37 GMT, Alan Hayward wrote: >> src/hotspot/cpu/aarch64/c1_Runtime1_aarch64.cpp line 452: >> >>> 450: // patch the return address, this stub will directly return to the exception handler >>> 451: __ str(r0, Address(rfp, 1*BytesPerWord)); >>> 452: >> >> Please explain the reason for this change, that leaves `lr` live across `restore_live_registers()`. > > In the original code: > *save r0 to the lr location on the stack > *restore_live_registers > *Standard return: remove stack frame, load lr and fp off the stack, jump to lr. > > With PAC it would now be: > *Sign r0 then save it to the lr location on the stack > *restore_live_registers > *Standard return: remove stack frame, load lr and fp off the stack, auth lr, jump to lr. > > After reading the code in restore_live_registers, it doesn't touch lr and so seemed odd to have the save to the stack, only to restore it directly afterwards. That's an optimization, though. You shouldn't need to read the code in `restore_live_registers()` to see if it's safe to keep the return address in LR: at best it's pathological coupling, in the sense that the correctness of this code depends on the internal details of `restore_live_registers()`. Let's keep LR live ranges as short as possible. ------------- PR: https://git.openjdk.java.net/jdk/pull/6334 From duke at openjdk.java.net Mon Nov 15 11:40:39 2021 From: duke at openjdk.java.net (Alan Hayward) Date: Mon, 15 Nov 2021 11:40:39 GMT Subject: RFR: 8264130: PAC-RET protection for Linux/AArch64 [v4] In-Reply-To: <1MtnvG48AfLFiyinjPMaT6KJ1MdM15mp2k2UMrryCgk=.7d91bba7-f1a7-4a26-8a4d-e1388a8b88ea@github.com> References: <81l6r4GfgLq9L4qhlvi_VWKE46vPqhspX-d7NG6Qux0=.4dbf25ed-4c3f-415b-9ffc-ddaf69211cf2@github.com> <1MtnvG48AfLFiyinjPMaT6KJ1MdM15mp2k2UMrryCgk=.7d91bba7-f1a7-4a26-8a4d-e1388a8b88ea@github.com> Message-ID: On Mon, 15 Nov 2021 11:30:35 GMT, Andrew Haley wrote: >> In the original code: >> *save r0 to the lr location on the stack >> *restore_live_registers >> *Standard return: remove stack frame, load lr and fp off the stack, jump to lr. >> >> With PAC it would now be: >> *Sign r0 then save it to the lr location on the stack >> *restore_live_registers >> *Standard return: remove stack frame, load lr and fp off the stack, auth lr, jump to lr. >> >> After reading the code in restore_live_registers, it doesn't touch lr and so seemed odd to have the save to the stack, only to restore it directly afterwards. > > That's an optimization, though. You shouldn't need to read the code in `restore_live_registers()` to see if it's safe to keep the return address in LR: at best it's pathological coupling, in the sense that the correctness of this code depends on the internal details of `restore_live_registers()`. Let's keep LR live ranges as short as possible. Ok, that's fine, I'll update it (It'll simplify the total code diff too). ------------- PR: https://git.openjdk.java.net/jdk/pull/6334 From adinn at openjdk.java.net Mon Nov 15 11:56:47 2021 From: adinn at openjdk.java.net (Andrew Dinn) Date: Mon, 15 Nov 2021 11:56:47 GMT Subject: RFR: 8264130: PAC-RET protection for Linux/AArch64 [v4] In-Reply-To: References: Message-ID: On Mon, 15 Nov 2021 11:08:57 GMT, Andrew Haley wrote: >>>whether this function authenticates or strips the address depends only on debugging? >> >> Yes. We only need to strip the value, because we're not jumping to the lr value, only viewing it. >> >> The interface is different to a strip (as we need to pass in the modifier). >> >> How about something like pauth_authenticate_fast() ? or pauth_authenticate_unsafe() ? >> >> Alternatively, this function is only called by the functions in Frame, so the frequency of use is probably low enough (compared to the sign/auth every function) that it's not going to cause any performance issues. So, could just replace with calls to pauth_authenticate. I think that might be the best option. > > A simple rule here: function names go with what the release version does. So I'd go with the actual purpose, which is `pauth_strip_addr_for_debuginfo()`. That's right, isn't it? You only want this thing for stack traces, logs, etc. This function is used by the frame code. So, that means it is used for all stack walks which are far from being simply cosmetic/ornamental. The runtime will rely on this for various different types of thread housekeeping. The difference here is that in product mode this simply strips auth bits whereas in debug mode it actually authenticates as it strips to give extra verification. So, your suggested name is quite misleading. Likewise Alan's suggested names is misleading because the primary product operation is to strip not authenticate. How about pauth_strip_verifiable? and a comment saying that it differs from pauth_strip by actually authenticating when debug is enabled. ------------- PR: https://git.openjdk.java.net/jdk/pull/6334 From ysuenaga at openjdk.java.net Mon Nov 15 13:19:56 2021 From: ysuenaga at openjdk.java.net (Yasumasa Suenaga) Date: Mon, 15 Nov 2021 13:19:56 GMT Subject: RFR: 8277089: Use system binutils to build hsdis [v2] In-Reply-To: References: Message-ID: > hsdis requires binutils source tree for building. Most of Linux distros provide binutils package. (e.g. binutils-devel from Fedora, binutils-dev from Ubuntu) > It would be nice to be able to use them like zlib and lcms. > > Unfortunately bfdver.h would not be provided because it is not included install files (`make install`) in binutils. So I changed to use `SEC_ELF_OCTETS` macro to detect binutils version because it was introduced at the same time as `bfd_octets_per_byte()`. > > https://sourceware.org/git/?p=binutils-gdb.git;a=commit;f=bfd/bfd-in2.h;h=618265039f697eab9e72bb58b95fc2d32925df58 > > Please see [JDK-8244819](https://bugs.openjdk.java.net/browse/JDK-8244819) why we need version check. Yasumasa Suenaga has updated the pull request incrementally with one additional commit since the last revision: Refactoring ------------- Changes: - all: https://git.openjdk.java.net/jdk/pull/6378/files - new: https://git.openjdk.java.net/jdk/pull/6378/files/fc99dc17..a43ce9aa Webrevs: - full: https://webrevs.openjdk.java.net/?repo=jdk&pr=6378&range=01 - incr: https://webrevs.openjdk.java.net/?repo=jdk&pr=6378&range=00-01 Stats: 57 lines in 1 file changed: 34 ins; 19 del; 4 mod Patch: https://git.openjdk.java.net/jdk/pull/6378.diff Fetch: git fetch https://git.openjdk.java.net/jdk pull/6378/head:pull/6378 PR: https://git.openjdk.java.net/jdk/pull/6378 From kcr at openjdk.java.net Mon Nov 15 13:31:38 2021 From: kcr at openjdk.java.net (Kevin Rushforth) Date: Mon, 15 Nov 2021 13:31:38 GMT Subject: RFR: 8276905: Function frag_col has a deployment target which is incompatible with this OS [v2] In-Reply-To: References: <_OkC1YeToScvoDOBBhAZfQHSao_fjjsnw60prsCsWtE=.6ea0dbc2-0270-4caa-bcfc-baa7cb704365@github.com> Message-ID: On Thu, 11 Nov 2021 15:32:01 GMT, Jayathirth D V wrote: >> When JDK is built on BigSur with Xcode12.5 and used to run jtreg tests in macOS10.15 with Xcode <=12.4 we are getting compilation error. We dont set any deployment target when when we use xcrun to create .air file and this issue looks similar to https://developer.apple.com/forums/thread/105719. Also https://developer.apple.com/forums/thread/105719 states that even if they set app deployment target they need to explicitly set deployment in "xcrun metal". >> >> Made same change to use -mmacosx-version-min=10.12.00 in our make file. Whether we should keep 10.12.00 or 10.13/14 is open to discussion for metal pipeline. >> >> SwingSet2, J2DDemo & JCK/jtreg runs in CI are green. Also Vitaly(Submitter) has confirmed that patch resolves the issue. > > Jayathirth D V has updated the pull request incrementally with one additional commit since the last revision: > > Use Macro for version The JDK itself has a minimum version of 11.0 on macOS aarch64 systems (because apple only supports it on macOS >= 11), so it probably doesn't matter much. You might wish to file a low-priority follow-up issue to change the runtime check to be consistent. ------------- PR: https://git.openjdk.java.net/jdk/pull/6346 From erikj at openjdk.java.net Mon Nov 15 13:36:35 2021 From: erikj at openjdk.java.net (Erik Joelsson) Date: Mon, 15 Nov 2021 13:36:35 GMT Subject: RFR: 8276905: Function frag_col has a deployment target which is incompatible with this OS [v2] In-Reply-To: References: <_OkC1YeToScvoDOBBhAZfQHSao_fjjsnw60prsCsWtE=.6ea0dbc2-0270-4caa-bcfc-baa7cb704365@github.com> Message-ID: On Mon, 15 Nov 2021 07:17:45 GMT, Jayathirth D V wrote: > A question popped up while doing this. By making 11.0 as minimum macosx version on aarch64, are we not restricting metal to be used only on >=11.0 on aarch 64? Correct, but there are no older versions of Macos on aarch64, so this is what we want. I think the correct behavior here is to use $(MACOSX_VERSION_MIN) on aarch64 and only override this with an explicit 10.14 for x86_64 and explaining why that override is done in a comment. ------------- PR: https://git.openjdk.java.net/jdk/pull/6346 From jdv at openjdk.java.net Mon Nov 15 13:44:11 2021 From: jdv at openjdk.java.net (Jayathirth D V) Date: Mon, 15 Nov 2021 13:44:11 GMT Subject: RFR: 8276905: Function frag_col has a deployment target which is incompatible with this OS [v3] In-Reply-To: <_OkC1YeToScvoDOBBhAZfQHSao_fjjsnw60prsCsWtE=.6ea0dbc2-0270-4caa-bcfc-baa7cb704365@github.com> References: <_OkC1YeToScvoDOBBhAZfQHSao_fjjsnw60prsCsWtE=.6ea0dbc2-0270-4caa-bcfc-baa7cb704365@github.com> Message-ID: > When JDK is built on BigSur with Xcode12.5 and used to run jtreg tests in macOS10.15 with Xcode <=12.4 we are getting compilation error. We dont set any deployment target when when we use xcrun to create .air file and this issue looks similar to https://developer.apple.com/forums/thread/105719. Also https://developer.apple.com/forums/thread/105719 states that even if they set app deployment target they need to explicitly set deployment in "xcrun metal". > > Made same change to use -mmacosx-version-min=10.12.00 in our make file. Whether we should keep 10.12.00 or 10.13/14 is open to discussion for metal pipeline. > > SwingSet2, J2DDemo & JCK/jtreg runs in CI are green. Also Vitaly(Submitter) has confirmed that patch resolves the issue. Jayathirth D V has updated the pull request incrementally with one additional commit since the last revision: Use 10.14 macOS version ------------- Changes: - all: https://git.openjdk.java.net/jdk/pull/6346/files - new: https://git.openjdk.java.net/jdk/pull/6346/files/7f999650..49104160 Webrevs: - full: https://webrevs.openjdk.java.net/?repo=jdk&pr=6346&range=02 - incr: https://webrevs.openjdk.java.net/?repo=jdk&pr=6346&range=01-02 Stats: 13 lines in 1 file changed: 12 ins; 0 del; 1 mod Patch: https://git.openjdk.java.net/jdk/pull/6346.diff Fetch: git fetch https://git.openjdk.java.net/jdk pull/6346/head:pull/6346 PR: https://git.openjdk.java.net/jdk/pull/6346 From ysuenaga at openjdk.java.net Mon Nov 15 13:55:35 2021 From: ysuenaga at openjdk.java.net (Yasumasa Suenaga) Date: Mon, 15 Nov 2021 13:55:35 GMT Subject: RFR: 8277089: Use system binutils to build hsdis In-Reply-To: References: Message-ID: On Mon, 15 Nov 2021 10:23:48 GMT, Magnus Ihse Bursie wrote: >> hsdis requires binutils source tree for building. Most of Linux distros provide binutils package. (e.g. binutils-devel from Fedora, binutils-dev from Ubuntu) >> It would be nice to be able to use them like zlib and lcms. >> >> Unfortunately bfdver.h would not be provided because it is not included install files (`make install`) in binutils. So I changed to use `SEC_ELF_OCTETS` macro to detect binutils version because it was introduced at the same time as `bfd_octets_per_byte()`. >> >> https://sourceware.org/git/?p=binutils-gdb.git;a=commit;f=bfd/bfd-in2.h;h=618265039f697eab9e72bb58b95fc2d32925df58 >> >> Please see [JDK-8244819](https://bugs.openjdk.java.net/browse/JDK-8244819) why we need version check. > > The basic idea is fine. I also think checking for `SEC_ELF_OCTETS` in the source code, instead of the version number, is actually an improvement. > > The one thing that itches me a bit is what happens when you specify `--with-binutils=system` and a dependent library is not found: > > > AC_CHECK_LIB(iberty, xmalloc, [ HSDIS_LIBS="$HSDIS_LIBS -liberty" ], [ AC_MSG_ERROR([libiberty not found]) ]) > > > Then the build will fail with no clear indication on why. Instead, I'd recommend that you restructure slightly. > > First check if with_binutils is system. If so, run your lib checks but like this: > > AC_CHECK_LIB(iberty, xmalloc, [ HSDIS_LIBS="$HSDIS_LIBS -liberty" ], [ bintils_system_error="libiberty not found" ]) > > > Then you go check the value of with_binutils again in the "switch" statement. And you can replace `AC_MSG_CHECKING` outside the switch statement again. If it is system, you check if `bintils_system_error` is non-empty. If so, you fail and explain that this-and-this error prevented system from working. > > What distributions have you tested this on? @magicus Thanks for your comment! I refactored this change. Could you review again? > What distributions have you tested this on? I tested this change on Fedora 35, and WSL Ubuntu 20.04 for Windows. ------------- PR: https://git.openjdk.java.net/jdk/pull/6378 From duke at openjdk.java.net Mon Nov 15 13:59:40 2021 From: duke at openjdk.java.net (Alan Hayward) Date: Mon, 15 Nov 2021 13:59:40 GMT Subject: RFR: 8264130: PAC-RET protection for Linux/AArch64 [v4] In-Reply-To: References: Message-ID: <8L4qvqk-eda9UU1BCA3i4yf3JVaJ0UMJxTDDLCO8XKg=.fc1bad40-8925-480d-a8a3-33f9c7650315@github.com> On Mon, 15 Nov 2021 11:54:09 GMT, Andrew Dinn wrote: > pauth_strip_verifiable That name works for me. ------------- PR: https://git.openjdk.java.net/jdk/pull/6334 From jdv at openjdk.java.net Mon Nov 15 14:00:59 2021 From: jdv at openjdk.java.net (Jayathirth D V) Date: Mon, 15 Nov 2021 14:00:59 GMT Subject: RFR: 8276905: Function frag_col has a deployment target which is incompatible with this OS [v4] In-Reply-To: <_OkC1YeToScvoDOBBhAZfQHSao_fjjsnw60prsCsWtE=.6ea0dbc2-0270-4caa-bcfc-baa7cb704365@github.com> References: <_OkC1YeToScvoDOBBhAZfQHSao_fjjsnw60prsCsWtE=.6ea0dbc2-0270-4caa-bcfc-baa7cb704365@github.com> Message-ID: > When JDK is built on BigSur with Xcode12.5 and used to run jtreg tests in macOS10.15 with Xcode <=12.4 we are getting compilation error. We dont set any deployment target when when we use xcrun to create .air file and this issue looks similar to https://developer.apple.com/forums/thread/105719. Also https://developer.apple.com/forums/thread/105719 states that even if they set app deployment target they need to explicitly set deployment in "xcrun metal". > > Made same change to use -mmacosx-version-min=10.12.00 in our make file. Whether we should keep 10.12.00 or 10.13/14 is open to discussion for metal pipeline. > > SwingSet2, J2DDemo & JCK/jtreg runs in CI are green. Also Vitaly(Submitter) has confirmed that patch resolves the issue. Jayathirth D V has updated the pull request incrementally with one additional commit since the last revision: Use MACOSX_VERSION_MIN for aarch64 ------------- Changes: - all: https://git.openjdk.java.net/jdk/pull/6346/files - new: https://git.openjdk.java.net/jdk/pull/6346/files/49104160..ecea143b Webrevs: - full: https://webrevs.openjdk.java.net/?repo=jdk&pr=6346&range=03 - incr: https://webrevs.openjdk.java.net/?repo=jdk&pr=6346&range=02-03 Stats: 3 lines in 1 file changed: 1 ins; 0 del; 2 mod Patch: https://git.openjdk.java.net/jdk/pull/6346.diff Fetch: git fetch https://git.openjdk.java.net/jdk pull/6346/head:pull/6346 PR: https://git.openjdk.java.net/jdk/pull/6346 From kcr at openjdk.java.net Mon Nov 15 14:01:00 2021 From: kcr at openjdk.java.net (Kevin Rushforth) Date: Mon, 15 Nov 2021 14:01:00 GMT Subject: RFR: 8276905: Function frag_col has a deployment target which is incompatible with this OS [v4] In-Reply-To: References: <_OkC1YeToScvoDOBBhAZfQHSao_fjjsnw60prsCsWtE=.6ea0dbc2-0270-4caa-bcfc-baa7cb704365@github.com> Message-ID: <5vw6AZ4nbyQtt9kHySnfxNqmPXG2Uvd8CM8lPdGgOK0=.5f25a4b1-99fb-40ef-b90e-0786b93be38b@github.com> On Mon, 15 Nov 2021 13:57:25 GMT, Jayathirth D V wrote: >> When JDK is built on BigSur with Xcode12.5 and used to run jtreg tests in macOS10.15 with Xcode <=12.4 we are getting compilation error. We dont set any deployment target when when we use xcrun to create .air file and this issue looks similar to https://developer.apple.com/forums/thread/105719. Also https://developer.apple.com/forums/thread/105719 states that even if they set app deployment target they need to explicitly set deployment in "xcrun metal". >> >> Made same change to use -mmacosx-version-min=10.12.00 in our make file. Whether we should keep 10.12.00 or 10.13/14 is open to discussion for metal pipeline. >> >> SwingSet2, J2DDemo & JCK/jtreg runs in CI are green. Also Vitaly(Submitter) has confirmed that patch resolves the issue. > > Jayathirth D V has updated the pull request incrementally with one additional commit since the last revision: > > Use MACOSX_VERSION_MIN for aarch64 Marked as reviewed by kcr (Author). ------------- PR: https://git.openjdk.java.net/jdk/pull/6346 From jdv at openjdk.java.net Mon Nov 15 14:01:01 2021 From: jdv at openjdk.java.net (Jayathirth D V) Date: Mon, 15 Nov 2021 14:01:01 GMT Subject: RFR: 8276905: Function frag_col has a deployment target which is incompatible with this OS [v2] In-Reply-To: References: <_OkC1YeToScvoDOBBhAZfQHSao_fjjsnw60prsCsWtE=.6ea0dbc2-0270-4caa-bcfc-baa7cb704365@github.com> Message-ID: On Mon, 15 Nov 2021 13:28:30 GMT, Kevin Rushforth wrote: > The JDK itself has a minimum version of 11.0 on macOS aarch64 systems (because apple only supports it on macOS >= 11), so it probably doesn't matter much. You might wish to file a low-priority follow-up issue to change the runtime check to be consistent. Thanks Kevin for the clarification and for also clearing up the follow up question i wanted to ask about runtime check update. ------------- PR: https://git.openjdk.java.net/jdk/pull/6346 From kcr at openjdk.java.net Mon Nov 15 14:01:09 2021 From: kcr at openjdk.java.net (Kevin Rushforth) Date: Mon, 15 Nov 2021 14:01:09 GMT Subject: RFR: 8276905: Function frag_col has a deployment target which is incompatible with this OS [v3] In-Reply-To: References: <_OkC1YeToScvoDOBBhAZfQHSao_fjjsnw60prsCsWtE=.6ea0dbc2-0270-4caa-bcfc-baa7cb704365@github.com> Message-ID: <8292Qt06yac4wltN03MhQPHqtuVvO7uRBUw98Jtq0Cw=.963fb049-0e03-4e19-bc74-7532b8abb146@github.com> On Mon, 15 Nov 2021 13:44:11 GMT, Jayathirth D V wrote: >> When JDK is built on BigSur with Xcode12.5 and used to run jtreg tests in macOS10.15 with Xcode <=12.4 we are getting compilation error. We dont set any deployment target when when we use xcrun to create .air file and this issue looks similar to https://developer.apple.com/forums/thread/105719. Also https://developer.apple.com/forums/thread/105719 states that even if they set app deployment target they need to explicitly set deployment in "xcrun metal". >> >> Made same change to use -mmacosx-version-min=10.12.00 in our make file. Whether we should keep 10.12.00 or 10.13/14 is open to discussion for metal pipeline. >> >> SwingSet2, J2DDemo & JCK/jtreg runs in CI are green. Also Vitaly(Submitter) has confirmed that patch resolves the issue. > > Jayathirth D V has updated the pull request incrementally with one additional commit since the last revision: > > Use 10.14 macOS version make/modules/java.desktop/lib/Awt2dLibraries.gmk line 844: > 842: # for aarch64 is 11.00.00 at the time of introducing MACOSX_METAL_VERSION_MIN. > 843: ifeq ($(OPENJDK_TARGET_CPU_ARCH), xaarch64) > 844: MACOSX_METAL_VERSION_MIN=11.00.00 I like Erik's suggestion of using `MACOSX_VERSION_MIN` for `aarch64`, since it's one less place you need to duplicate a constant. ------------- PR: https://git.openjdk.java.net/jdk/pull/6346 From erikj at openjdk.java.net Mon Nov 15 14:01:01 2021 From: erikj at openjdk.java.net (Erik Joelsson) Date: Mon, 15 Nov 2021 14:01:01 GMT Subject: RFR: 8276905: Function frag_col has a deployment target which is incompatible with this OS [v4] In-Reply-To: References: <_OkC1YeToScvoDOBBhAZfQHSao_fjjsnw60prsCsWtE=.6ea0dbc2-0270-4caa-bcfc-baa7cb704365@github.com> Message-ID: On Mon, 15 Nov 2021 13:57:25 GMT, Jayathirth D V wrote: >> When JDK is built on BigSur with Xcode12.5 and used to run jtreg tests in macOS10.15 with Xcode <=12.4 we are getting compilation error. We dont set any deployment target when when we use xcrun to create .air file and this issue looks similar to https://developer.apple.com/forums/thread/105719. Also https://developer.apple.com/forums/thread/105719 states that even if they set app deployment target they need to explicitly set deployment in "xcrun metal". >> >> Made same change to use -mmacosx-version-min=10.12.00 in our make file. Whether we should keep 10.12.00 or 10.13/14 is open to discussion for metal pipeline. >> >> SwingSet2, J2DDemo & JCK/jtreg runs in CI are green. Also Vitaly(Submitter) has confirmed that patch resolves the issue. > > Jayathirth D V has updated the pull request incrementally with one additional commit since the last revision: > > Use MACOSX_VERSION_MIN for aarch64 Looks good! ------------- Marked as reviewed by erikj (Reviewer). PR: https://git.openjdk.java.net/jdk/pull/6346 From jdv at openjdk.java.net Mon Nov 15 14:01:13 2021 From: jdv at openjdk.java.net (Jayathirth D V) Date: Mon, 15 Nov 2021 14:01:13 GMT Subject: RFR: 8276905: Function frag_col has a deployment target which is incompatible with this OS [v3] In-Reply-To: <8292Qt06yac4wltN03MhQPHqtuVvO7uRBUw98Jtq0Cw=.963fb049-0e03-4e19-bc74-7532b8abb146@github.com> References: <_OkC1YeToScvoDOBBhAZfQHSao_fjjsnw60prsCsWtE=.6ea0dbc2-0270-4caa-bcfc-baa7cb704365@github.com> <8292Qt06yac4wltN03MhQPHqtuVvO7uRBUw98Jtq0Cw=.963fb049-0e03-4e19-bc74-7532b8abb146@github.com> Message-ID: On Mon, 15 Nov 2021 13:46:29 GMT, Kevin Rushforth wrote: >> Jayathirth D V has updated the pull request incrementally with one additional commit since the last revision: >> >> Use 10.14 macOS version > > make/modules/java.desktop/lib/Awt2dLibraries.gmk line 844: > >> 842: # for aarch64 is 11.00.00 at the time of introducing MACOSX_METAL_VERSION_MIN. >> 843: ifeq ($(OPENJDK_TARGET_CPU_ARCH), xaarch64) >> 844: MACOSX_METAL_VERSION_MIN=11.00.00 > > I like Erik's suggestion of using `MACOSX_VERSION_MIN` for `aarch64`, since it's one less place you need to duplicate a constant. Hi Kevin, I noticed the comment after pushing the change. I have updated it. Thanks, Jay ------------- PR: https://git.openjdk.java.net/jdk/pull/6346 From ihse at openjdk.java.net Mon Nov 15 14:34:36 2021 From: ihse at openjdk.java.net (Magnus Ihse Bursie) Date: Mon, 15 Nov 2021 14:34:36 GMT Subject: RFR: 8277089: Use system binutils to build hsdis [v2] In-Reply-To: References: Message-ID: On Mon, 15 Nov 2021 13:19:56 GMT, Yasumasa Suenaga wrote: >> hsdis requires binutils source tree for building. Most of Linux distros provide binutils package. (e.g. binutils-devel from Fedora, binutils-dev from Ubuntu) >> It would be nice to be able to use them like zlib and lcms. >> >> Unfortunately bfdver.h would not be provided because it is not included install files (`make install`) in binutils. So I changed to use `SEC_ELF_OCTETS` macro to detect binutils version because it was introduced at the same time as `bfd_octets_per_byte()`. >> >> https://sourceware.org/git/?p=binutils-gdb.git;a=commit;f=bfd/bfd-in2.h;h=618265039f697eab9e72bb58b95fc2d32925df58 >> >> Please see [JDK-8244819](https://bugs.openjdk.java.net/browse/JDK-8244819) why we need version check. > > Yasumasa Suenaga has updated the pull request incrementally with one additional commit since the last revision: > > Refactoring Looks good to me now ------------- Marked as reviewed by ihse (Reviewer). PR: https://git.openjdk.java.net/jdk/pull/6378 From ihse at openjdk.java.net Mon Nov 15 14:38:37 2021 From: ihse at openjdk.java.net (Magnus Ihse Bursie) Date: Mon, 15 Nov 2021 14:38:37 GMT Subject: RFR: 8276905: Function frag_col has a deployment target which is incompatible with this OS [v4] In-Reply-To: References: <_OkC1YeToScvoDOBBhAZfQHSao_fjjsnw60prsCsWtE=.6ea0dbc2-0270-4caa-bcfc-baa7cb704365@github.com> Message-ID: On Mon, 15 Nov 2021 14:00:59 GMT, Jayathirth D V wrote: >> When JDK is built on BigSur with Xcode12.5 and used to run jtreg tests in macOS10.15 with Xcode <=12.4 we are getting compilation error. We dont set any deployment target when when we use xcrun to create .air file and this issue looks similar to https://developer.apple.com/forums/thread/105719. Also https://developer.apple.com/forums/thread/105719 states that even if they set app deployment target they need to explicitly set deployment in "xcrun metal". >> >> Made same change to use -mmacosx-version-min=10.12.00 in our make file. Whether we should keep 10.12.00 or 10.13/14 is open to discussion for metal pipeline. >> >> SwingSet2, J2DDemo & JCK/jtreg runs in CI are green. Also Vitaly(Submitter) has confirmed that patch resolves the issue. > > Jayathirth D V has updated the pull request incrementally with one additional commit since the last revision: > > Use MACOSX_VERSION_MIN for aarch64 Looks good to me also. Please let Phil have a say as well before integrating. Also please update the title of the PR and the JBS issue (they must be the same) to something about that we're changing metal min version, rather than the obscure error that was the result of not having the min version... ------------- Marked as reviewed by ihse (Reviewer). PR: https://git.openjdk.java.net/jdk/pull/6346 From jdv at openjdk.java.net Mon Nov 15 14:46:42 2021 From: jdv at openjdk.java.net (Jayathirth D V) Date: Mon, 15 Nov 2021 14:46:42 GMT Subject: RFR: 8276905: Use appropriate macosx_version_minimum value while compiling metal shaders [v4] In-Reply-To: References: <_OkC1YeToScvoDOBBhAZfQHSao_fjjsnw60prsCsWtE=.6ea0dbc2-0270-4caa-bcfc-baa7cb704365@github.com> Message-ID: On Mon, 15 Nov 2021 14:33:21 GMT, Magnus Ihse Bursie wrote: > Looks good to me also. Please let Phil have a say as well before integrating. Thanks Magnus for the review. Yes i have already Re-Requested review from Phil on latest patch. Also i am waiting on Vitaly(JBS submitter) to verify the latest patch. ------------- PR: https://git.openjdk.java.net/jdk/pull/6346 From prr at openjdk.java.net Mon Nov 15 17:34:46 2021 From: prr at openjdk.java.net (Phil Race) Date: Mon, 15 Nov 2021 17:34:46 GMT Subject: RFR: 8276905: Use appropriate macosx_version_minimum value while compiling metal shaders [v4] In-Reply-To: References: <_OkC1YeToScvoDOBBhAZfQHSao_fjjsnw60prsCsWtE=.6ea0dbc2-0270-4caa-bcfc-baa7cb704365@github.com> Message-ID: On Mon, 15 Nov 2021 14:00:59 GMT, Jayathirth D V wrote: >> When JDK is built on BigSur with Xcode12.5 and used to run jtreg tests in macOS10.15 with Xcode <=12.4 we are getting compilation error. We dont set any deployment target when when we use xcrun to create .air file and this issue looks similar to https://developer.apple.com/forums/thread/105719. Also https://developer.apple.com/forums/thread/105719 states that even if they set app deployment target they need to explicitly set deployment in "xcrun metal". >> >> Made same change to use -mmacosx-version-min=10.12.00 in our make file. Whether we should keep 10.12.00 or 10.13/14 is open to discussion for metal pipeline. >> >> SwingSet2, J2DDemo & JCK/jtreg runs in CI are green. Also Vitaly(Submitter) has confirmed that patch resolves the issue. > > Jayathirth D V has updated the pull request incrementally with one additional commit since the last revision: > > Use MACOSX_VERSION_MIN for aarch64 This should be the most compatible solution. ------------- Marked as reviewed by prr (Reviewer). PR: https://git.openjdk.java.net/jdk/pull/6346 From duke at openjdk.java.net Tue Nov 16 08:22:22 2021 From: duke at openjdk.java.net (Alan Hayward) Date: Tue, 16 Nov 2021 08:22:22 GMT Subject: RFR: 8264130: PAC-RET protection for Linux/AArch64 [v5] In-Reply-To: References: Message-ID: > PAC is an optional feature in AArch64 8.3 and is compulsory in v9. One > of its uses is to protect against ROP based attacks. This is done by > signing the Link Register whenever it is stored on the stack, and > authenticating the value when it is loaded back from the stack. If an > attacker were to try to change control flow by editing the stack then > the authentication check of the Link Register will fail, causing a > segfault when the function returns. > > On a system with PAC enabled, it is expected that all applications will > be compiled with ROP protection. Fedora 33 and upwards already provide > this. By compiling for ARMv8.0, GCC and LLVM will only use the set of > PAC instructions that exist in the NOP space - on hardware without PAC, > these instructions act as NOPs, allowing backward compatibility for > negligible performance cost (2 NOPs per non-leaf function). > > Hardware is currently limited to the Apple M1 MacBooks. All testing has > been done within a Fedora Docker image. A run of SpecJVM showed no > difference to that of noise - which was surprising. > > The most important part of this patch is simply compiling using branch > protection provided by GCC/LLVM. This protects all C++ code from being > used in ROP attacks, removing all static ROP gadgets from use. > > The remainder of the patch adds ROP protection to runtime generated > code, in both stubs and compiled Java code. Attacks here are much harder > as ROP gadgets must be found dynamically at runtime. If/when AOT > compilation is added to JDK, then all stubs and compiled Java will be > susceptible ROP gadgets being found by static analysis and therefore > potentially as vulnerable as C++ code. > > There are a number of places where the VM changes control flow by > rewriting the stack or otherwise. I?ve done some analysis as to how > these could also be used for attacks (which I didn?t want to post here). > These areas can be protected ensuring the pointers to various stubs and > entry points are stored in memory as signed pointers. These changes are > simple to make (they can be reduced to a type change in common code and > a few addition sign/auth calls in the backend), but there a lot of them > and the total code change is fairly large. I?m happy to provide a few > work in progress patches. > > In order to match the security benefits of the Apple Arm64e ABI across > the whole of JDK, then all the changes mentioned above would be > required. Alan Hayward has updated the pull request incrementally with three additional commits since the last revision: - Rename pauth_authenticate_or_strip_return_address - Fix windows aarch64 by restoring pauth file split - Don't keep LR live across restore_live_registers ------------- Changes: - all: https://git.openjdk.java.net/jdk/pull/6334/files - new: https://git.openjdk.java.net/jdk/pull/6334/files/2c27eb5e..dbd6bda2 Webrevs: - full: https://webrevs.openjdk.java.net/?repo=jdk&pr=6334&range=04 - incr: https://webrevs.openjdk.java.net/?repo=jdk&pr=6334&range=03-04 Stats: 318 lines in 6 files changed: 233 ins; 70 del; 15 mod Patch: https://git.openjdk.java.net/jdk/pull/6334.diff Fetch: git fetch https://git.openjdk.java.net/jdk pull/6334/head:pull/6334 PR: https://git.openjdk.java.net/jdk/pull/6334 From jdv at openjdk.java.net Tue Nov 16 13:22:40 2021 From: jdv at openjdk.java.net (Jayathirth D V) Date: Tue, 16 Nov 2021 13:22:40 GMT Subject: Integrated: 8276905: Use appropriate macosx_version_minimum value while compiling metal shaders In-Reply-To: <_OkC1YeToScvoDOBBhAZfQHSao_fjjsnw60prsCsWtE=.6ea0dbc2-0270-4caa-bcfc-baa7cb704365@github.com> References: <_OkC1YeToScvoDOBBhAZfQHSao_fjjsnw60prsCsWtE=.6ea0dbc2-0270-4caa-bcfc-baa7cb704365@github.com> Message-ID: On Thu, 11 Nov 2021 05:52:18 GMT, Jayathirth D V wrote: > When JDK is built on BigSur with Xcode12.5 and used to run jtreg tests in macOS10.15 with Xcode <=12.4 we are getting compilation error. We dont set any deployment target when when we use xcrun to create .air file and this issue looks similar to https://developer.apple.com/forums/thread/105719. Also https://developer.apple.com/forums/thread/105719 states that even if they set app deployment target they need to explicitly set deployment in "xcrun metal". > > Made same change to use -mmacosx-version-min=10.12.00 in our make file. Whether we should keep 10.12.00 or 10.13/14 is open to discussion for metal pipeline. > > SwingSet2, J2DDemo & JCK/jtreg runs in CI are green. Also Vitaly(Submitter) has confirmed that patch resolves the issue. This pull request has now been integrated. Changeset: 9a9a157a Author: Jayathirth D V URL: https://git.openjdk.java.net/jdk/commit/9a9a157a7d45cbfb016d4427931e1d5345210f7a Stats: 16 lines in 1 file changed: 15 ins; 0 del; 1 mod 8276905: Use appropriate macosx_version_minimum value while compiling metal shaders Reviewed-by: ihse, kcr, erikj, prr ------------- PR: https://git.openjdk.java.net/jdk/pull/6346 From duke at openjdk.java.net Tue Nov 16 14:23:07 2021 From: duke at openjdk.java.net (Alan Hayward) Date: Tue, 16 Nov 2021 14:23:07 GMT Subject: RFR: 8277204: Implementation of JEP 8264130: PAC-RET protection for Linux/AArch64 [v6] In-Reply-To: References: Message-ID: > PAC is an optional feature in AArch64 8.3 and is compulsory in v9. One > of its uses is to protect against ROP based attacks. This is done by > signing the Link Register whenever it is stored on the stack, and > authenticating the value when it is loaded back from the stack. If an > attacker were to try to change control flow by editing the stack then > the authentication check of the Link Register will fail, causing a > segfault when the function returns. > > On a system with PAC enabled, it is expected that all applications will > be compiled with ROP protection. Fedora 33 and upwards already provide > this. By compiling for ARMv8.0, GCC and LLVM will only use the set of > PAC instructions that exist in the NOP space - on hardware without PAC, > these instructions act as NOPs, allowing backward compatibility for > negligible performance cost (2 NOPs per non-leaf function). > > Hardware is currently limited to the Apple M1 MacBooks. All testing has > been done within a Fedora Docker image. A run of SpecJVM showed no > difference to that of noise - which was surprising. > > The most important part of this patch is simply compiling using branch > protection provided by GCC/LLVM. This protects all C++ code from being > used in ROP attacks, removing all static ROP gadgets from use. > > The remainder of the patch adds ROP protection to runtime generated > code, in both stubs and compiled Java code. Attacks here are much harder > as ROP gadgets must be found dynamically at runtime. If/when AOT > compilation is added to JDK, then all stubs and compiled Java will be > susceptible ROP gadgets being found by static analysis and therefore > potentially as vulnerable as C++ code. > > There are a number of places where the VM changes control flow by > rewriting the stack or otherwise. I?ve done some analysis as to how > these could also be used for attacks (which I didn?t want to post here). > These areas can be protected ensuring the pointers to various stubs and > entry points are stored in memory as signed pointers. These changes are > simple to make (they can be reduced to a type change in common code and > a few addition sign/auth calls in the backend), but there a lot of them > and the total code change is fairly large. I?m happy to provide a few > work in progress patches. > > In order to match the security benefits of the Apple Arm64e ABI across > the whole of JDK, then all the changes mentioned above would be > required. Alan Hayward has updated the pull request with a new target base due to a merge or a rebase. The pull request now contains 12 commits: - Merge master - Rename pauth_authenticate_or_strip_return_address - Fix windows aarch64 by restoring pauth file split - Don't keep LR live across restore_live_registers - Merge master - Document pauth functions && remove OS split - Update UseROPProtection description - Simplify branch protection configure check - 8264130: PAC-RET protection for Linux/AArch64 PAC is an optional feature in AArch64 8.3 and is compulsory in v9. One of its uses is to protect against ROP based attacks. This is done by signing the Link Register whenever it is stored on the stack, and authenticating the value when it is loaded back from the stack. If an attacker were to try to change control flow by editing the stack then the authentication check of the Link Register will fail, causing a segfault when the function returns. On a system with PAC enabled, it is expected that all applications will be compiled with ROP protection. Fedora 33 and upwards already provide this. By compiling for ARMv8.0, GCC and LLVM will only use the set of PAC instructions that exist in the NOP space - on hardware without PAC, these instructions act as NOPs, allowing backward compatibility for negligible performance cost (2 NOPs per non-leaf function). Hardware is currently limited to the Apple M1 MacBooks. All testing has been done within a Fedora Docker image. A run of SpecJVM showed no difference to that of noise - which was surprising. The most important part of this patch is simply compiling using branch protection provided by GCC/LLVM. This protects all C++ code from being used in ROP attacks, removing all static ROP gadgets from use. The remainder of the patch adds ROP protection to runtime generated code, in both stubs and compiled Java code. Attacks here are much harder as ROP gadgets must be found dynamically at runtime. If/when AOT compilation is added to JDK, then all stubs and compiled Java will be susceptible ROP gadgets being found by static analysis and therefore potentially as vulnerable as C++ code. There are a number of places where the VM changes control flow by rewriting the stack or otherwise. I?ve done some analysis as to how these could also be used for attacks (which I didn?t want to post here). These areas can be protected ensuring the pointers to various stubs and entry points are stored in memory as signed pointers. These changes are simple to make (they can be reduced to a type change in common code and a few addition sign/auth calls in the backend), but there a lot of them and the total code change is fairly large. I?m happy to provide a few work in progress patches. In order to match the security benefits of the Apple Arm64e ABI across the whole of JDK, then all the changes mentioned above would be required. - Add PAC assembly instructions - ... and 2 more: https://git.openjdk.java.net/jdk/compare/b8d33a2a...deb17a56 ------------- Changes: https://git.openjdk.java.net/jdk/pull/6334/files Webrev: https://webrevs.openjdk.java.net/?repo=jdk&pr=6334&range=05 Stats: 1347 lines in 25 files changed: 521 ins; 18 del; 808 mod Patch: https://git.openjdk.java.net/jdk/pull/6334.diff Fetch: git fetch https://git.openjdk.java.net/jdk pull/6334/head:pull/6334 PR: https://git.openjdk.java.net/jdk/pull/6334 From ysuenaga at openjdk.java.net Tue Nov 16 14:50:40 2021 From: ysuenaga at openjdk.java.net (Yasumasa Suenaga) Date: Tue, 16 Nov 2021 14:50:40 GMT Subject: Integrated: 8277089: Use system binutils to build hsdis In-Reply-To: References: Message-ID: <3KlOI75YzKulauF-zC1WsUCeT1nmgSBAJAqPHAWdP3U=.bcd684d3-2680-47ec-a6aa-3a85846f47a3@github.com> On Sat, 13 Nov 2021 08:08:53 GMT, Yasumasa Suenaga wrote: > hsdis requires binutils source tree for building. Most of Linux distros provide binutils package. (e.g. binutils-devel from Fedora, binutils-dev from Ubuntu) > It would be nice to be able to use them like zlib and lcms. > > Unfortunately bfdver.h would not be provided because it is not included install files (`make install`) in binutils. So I changed to use `SEC_ELF_OCTETS` macro to detect binutils version because it was introduced at the same time as `bfd_octets_per_byte()`. > > https://sourceware.org/git/?p=binutils-gdb.git;a=commit;f=bfd/bfd-in2.h;h=618265039f697eab9e72bb58b95fc2d32925df58 > > Please see [JDK-8244819](https://bugs.openjdk.java.net/browse/JDK-8244819) why we need version check. This pull request has now been integrated. Changeset: d5e47d6b Author: Yasumasa Suenaga URL: https://git.openjdk.java.net/jdk/commit/d5e47d6b84514edde23a8baff8c2274e5b3ca6bb Stats: 61 lines in 3 files changed: 45 ins; 12 del; 4 mod 8277089: Use system binutils to build hsdis Reviewed-by: ihse ------------- PR: https://git.openjdk.java.net/jdk/pull/6378 From jlahoda at openjdk.java.net Tue Nov 16 20:14:08 2021 From: jlahoda at openjdk.java.net (Jan Lahoda) Date: Tue, 16 Nov 2021 20:14:08 GMT Subject: RFR: 8277106: Cannot compile certain sources with --release Message-ID: While working on JDK-8277105[1], it turned out the historical data in `ct.sym` are not quite right in two directions: a) some API classes extend classes from non-exported packages (often from the `java.base` module). Examples include `jdk.jfr.Event` which extends `jdk.internal.event.Event`. As the current `ct.sym` data do not keep classes from non-exported packages, compilation against these classes fails. b) permitted subtypes may include classes from non-exported packages, and these are needed to properly determine validity of casts and pattern matching switches. But these classes are not kept in the `ct.sym`, and hence the behavior of the casts may be incorrect. (Although I am no aware of any cases where an actual problem would arise with the current APIs.) The proposed solution is to keep a certain amount of classes from non-exported packages in the historical record, specifically classes that are either extended by a class in an exported package, or named as a permitted subclass in an exported package. The first occurrence of a) I was able to find was in JDK 11, so this patch also fixes the historical record till JDK 11. [1] https://bugs.openjdk.java.net/browse/JDK-8277105 ------------- Commit messages: - 8277106: Cannot compile certain sources with --release Changes: https://git.openjdk.java.net/jdk/pull/6417/files Webrev: https://webrevs.openjdk.java.net/?repo=jdk&pr=6417&range=00 Issue: https://bugs.openjdk.java.net/browse/JDK-8277106 Stats: 1955 lines in 22 files changed: 1849 ins; 39 del; 67 mod Patch: https://git.openjdk.java.net/jdk/pull/6417.diff Fetch: git fetch https://git.openjdk.java.net/jdk pull/6417/head:pull/6417 PR: https://git.openjdk.java.net/jdk/pull/6417 From ihse at openjdk.java.net Wed Nov 17 13:20:35 2021 From: ihse at openjdk.java.net (Magnus Ihse Bursie) Date: Wed, 17 Nov 2021 13:20:35 GMT Subject: RFR: 8253757: Add LLVM-based backend for hsdis In-Reply-To: References: Message-ID: On Wed, 13 Oct 2021 00:00:22 GMT, Magnus Ihse Bursie wrote: > This patch expands the newly added system for hsdis backends to include LLVM. > > The actual code in hsdis-llvm.cpp is based heavily on the work by @luhenry, as published in the never integrated PR https://github.com/openjdk/jdk/pull/392. (I have basically just ripped out the binutils-based part of it.) > > Unfortunately I have not been able to make this work properly on Windows. With some additional flags I made it compile without complaints, but it caused hotspot to segfault in `LoadLibrary` (!) in `os::dll_load` when I tried to load the library. This is somewhat ironic, since the initial implementation was created by Ludovic for the very purpose of using it on Windows. > > The lack of Windows support in this patch does not mean it is impossible to get it to work, just that I need to co-operate with someone who has more experience of compiling LLVM on Windows, and/or are more eager to get this combination to work. Yeah bot, I'm still working on it. ------------- PR: https://git.openjdk.java.net/jdk/pull/5920 From smarks at openjdk.java.net Thu Nov 18 01:51:07 2021 From: smarks at openjdk.java.net (Stuart Marks) Date: Thu, 18 Nov 2021 01:51:07 GMT Subject: RFR: JDK-8276422 Add command-line option to disable finalization Message-ID: Pretty much what it says. The new option controls a static member in InstanceKlass that's consulted to determine whether the finalization machinery is activated for instances when a class is loaded. A new native method is added so that this state can be queried from Java. This is used to control whether a finalizer thread is created and to disable the `System` and `Runtime::runFinalization` methods. Includes tests for the above. ------------- Commit messages: - extraneous newline - Merge branch 'master' into JDK-8276422-disable-finalization-option - Simplify InvalidFinalizationOption test. - Change InvalidFinalizationOption test to driver mode. - Revert extraneous whitespace change to globals.hpp. - Renaming within the test class itself. - Rename invalid finalization option test. - Add test for invalid finalization option syntax or value. - Add @bug line to JFR finalization event test. - Test that no jdk.FinalizationStatistics events are generated when finalization is disabled - ... and 7 more: https://git.openjdk.java.net/jdk/compare/29e552c0...3836cc94 Changes: https://git.openjdk.java.net/jdk/pull/6442/files Webrev: https://webrevs.openjdk.java.net/?repo=jdk&pr=6442&range=00 Issue: https://bugs.openjdk.java.net/browse/JDK-8276422 Stats: 266 lines in 13 files changed: 249 ins; 0 del; 17 mod Patch: https://git.openjdk.java.net/jdk/pull/6442.diff Fetch: git fetch https://git.openjdk.java.net/jdk pull/6442/head:pull/6442 PR: https://git.openjdk.java.net/jdk/pull/6442 From dholmes at openjdk.java.net Thu Nov 18 01:59:41 2021 From: dholmes at openjdk.java.net (David Holmes) Date: Thu, 18 Nov 2021 01:59:41 GMT Subject: RFR: JDK-8276422 Add command-line option to disable finalization In-Reply-To: References: Message-ID: On Thu, 18 Nov 2021 01:34:36 GMT, Stuart Marks wrote: > Pretty much what it says. The new option controls a static member in InstanceKlass that's consulted to determine whether the finalization machinery is activated for instances when a class is loaded. A new native method is added so that this state can be queried from Java. This is used to control whether a finalizer thread is created and to disable the `System` and `Runtime::runFinalization` methods. Includes tests for the above. Hi Stuart, This all looks fine to me. The hotspot part needs a second reviewer (especially as I contributed a chunk of that code :) ). Thanks, David ------------- Marked as reviewed by dholmes (Reviewer). PR: https://git.openjdk.java.net/jdk/pull/6442 From jpai at openjdk.java.net Thu Nov 18 04:17:43 2021 From: jpai at openjdk.java.net (Jaikiran Pai) Date: Thu, 18 Nov 2021 04:17:43 GMT Subject: RFR: JDK-8276422 Add command-line option to disable finalization In-Reply-To: References: Message-ID: On Thu, 18 Nov 2021 01:34:36 GMT, Stuart Marks wrote: > Pretty much what it says. The new option controls a static member in InstanceKlass that's consulted to determine whether the finalization machinery is activated for instances when a class is loaded. A new native method is added so that this state can be queried from Java. This is used to control whether a finalizer thread is created and to disable the `System` and `Runtime::runFinalization` methods. Includes tests for the above. src/java.base/share/classes/java/lang/ref/Finalizer.java line 195: > 193: > 194: static { > 195: if (Holder.ENABLED) { Hello Stuart, My understanding of the the lazy `Holder` is that it's there to delay the static initialization of the code that's part of the `Holder`. In this case here, the `Holder` is being used right within the `static` block of the `Finalizer` class, that too as the first thing. In this case, is that `Holder` class necessary? ------------- PR: https://git.openjdk.java.net/jdk/pull/6442 From smarks at openjdk.java.net Thu Nov 18 05:22:35 2021 From: smarks at openjdk.java.net (Stuart Marks) Date: Thu, 18 Nov 2021 05:22:35 GMT Subject: RFR: JDK-8276422 Add command-line option to disable finalization In-Reply-To: References: Message-ID: On Thu, 18 Nov 2021 04:13:21 GMT, Jaikiran Pai wrote: >> Pretty much what it says. The new option controls a static member in InstanceKlass that's consulted to determine whether the finalization machinery is activated for instances when a class is loaded. A new native method is added so that this state can be queried from Java. This is used to control whether a finalizer thread is created and to disable the `System` and `Runtime::runFinalization` methods. Includes tests for the above. > > src/java.base/share/classes/java/lang/ref/Finalizer.java line 195: > >> 193: >> 194: static { >> 195: if (Holder.ENABLED) { > > Hello Stuart, > My understanding of the the lazy `Holder` is that it's there to delay the static initialization of the code that's part of the `Holder`. In this case here, the `Holder` is being used right within the `static` block of the `Finalizer` class, that too as the first thing. In this case, is that `Holder` class necessary? Huh, good catch! This was mostly left over from an earlier version of the flag that used system properties, which aren't initialized until after the Finalizer class is initialized. It might be the case that the Holder can be removed at this point, since the finalization-enabled bit is no longer in a system property and is in a native class member that should be available before the VM is started. I say "might" though because this occurs early in system startup, and weird things potentially happen. For example, suppose the first object with a finalizer is created before the Finalizer class is initialized. The VM will perform an upcall to Finalizer::register. An ordinary call to a static method will ensure the class is initialized before proceeding with the call, but this VM upcall is a special case.... I'll have to investigate this some more. ------------- PR: https://git.openjdk.java.net/jdk/pull/6442 From dholmes at openjdk.java.net Thu Nov 18 07:16:42 2021 From: dholmes at openjdk.java.net (David Holmes) Date: Thu, 18 Nov 2021 07:16:42 GMT Subject: RFR: JDK-8276422 Add command-line option to disable finalization In-Reply-To: References: Message-ID: On Thu, 18 Nov 2021 05:20:02 GMT, Stuart Marks wrote: >> src/java.base/share/classes/java/lang/ref/Finalizer.java line 195: >> >>> 193: >>> 194: static { >>> 195: if (Holder.ENABLED) { >> >> Hello Stuart, >> My understanding of the the lazy `Holder` is that it's there to delay the static initialization of the code that's part of the `Holder`. In this case here, the `Holder` is being used right within the `static` block of the `Finalizer` class, that too as the first thing. In this case, is that `Holder` class necessary? > > Huh, good catch! This was mostly left over from an earlier version of the flag that used system properties, which aren't initialized until after the Finalizer class is initialized. > > It might be the case that the Holder can be removed at this point, since the finalization-enabled bit is no longer in a system property and is in a native class member that should be available before the VM is started. > > I say "might" though because this occurs early in system startup, and weird things potentially happen. For example, suppose the first object with a finalizer is created before the Finalizer class is initialized. The VM will perform an upcall to Finalizer::register. An ordinary call to a static method will ensure the class is initialized before proceeding with the call, but this VM upcall is a special case.... I'll have to investigate this some more. @stuart-marks not sure I see how anything is different here compared to the existing logic. The `Finalizer` class is explicitly initialized quite early in the init process, but if a preceding class's initialization created an object with a finalizer then that same upcall would be involved. ------------- PR: https://git.openjdk.java.net/jdk/pull/6442 From kbarrett at openjdk.java.net Thu Nov 18 07:19:37 2021 From: kbarrett at openjdk.java.net (Kim Barrett) Date: Thu, 18 Nov 2021 07:19:37 GMT Subject: RFR: JDK-8276422 Add command-line option to disable finalization In-Reply-To: <6s-4rTTyX8qZpavXbov9l2-H6BU7QLA0c71-K6xSQUM=.54dd6db2-5e78-4292-8958-72cadd762154@github.com> References: <6s-4rTTyX8qZpavXbov9l2-H6BU7QLA0c71-K6xSQUM=.54dd6db2-5e78-4292-8958-72cadd762154@github.com> Message-ID: On Thu, 18 Nov 2021 06:43:01 GMT, Kim Barrett wrote: >> Pretty much what it says. The new option controls a static member in InstanceKlass that's consulted to determine whether the finalization machinery is activated for instances when a class is loaded. A new native method is added so that this state can be queried from Java. This is used to control whether a finalizer thread is created and to disable the `System` and `Runtime::runFinalization` methods. Includes tests for the above. > > src/hotspot/share/prims/jvm.cpp line 694: > >> 692: >> 693: JVM_ENTRY(jboolean, JVM_IsFinalizationEnabled(JNIEnv * env)) >> 694: return InstanceKlass::finalization_enabled() ? JNI_TRUE : JNI_FALSE; > > missing indentation I think this could just be `return InstanceKlass::finalization_enabled();`. There is lots of code in this file and elsewhere that assumes C++ `bool` converts to `jboolean` appropriately. ------------- PR: https://git.openjdk.java.net/jdk/pull/6442 From kbarrett at openjdk.java.net Thu Nov 18 07:19:37 2021 From: kbarrett at openjdk.java.net (Kim Barrett) Date: Thu, 18 Nov 2021 07:19:37 GMT Subject: RFR: JDK-8276422 Add command-line option to disable finalization In-Reply-To: References: Message-ID: <6s-4rTTyX8qZpavXbov9l2-H6BU7QLA0c71-K6xSQUM=.54dd6db2-5e78-4292-8958-72cadd762154@github.com> On Thu, 18 Nov 2021 01:34:36 GMT, Stuart Marks wrote: > Pretty much what it says. The new option controls a static member in InstanceKlass that's consulted to determine whether the finalization machinery is activated for instances when a class is loaded. A new native method is added so that this state can be queried from Java. This is used to control whether a finalizer thread is created and to disable the `System` and `Runtime::runFinalization` methods. Includes tests for the above. I only really reviewed the hotspot changes. There is nothing here to make the various GCs take advantage of finalization being disabled. Is the plan to leave that to followup changes? src/hotspot/share/oops/instanceKlass.hpp line 338: > 336: > 337: // Queries finalization state > 338: static bool finalization_enabled() { return _finalization_enabled; } Predicate functions like this are often named "is_xxx"; that idiom is common in this class. src/hotspot/share/prims/jvm.cpp line 694: > 692: > 693: JVM_ENTRY(jboolean, JVM_IsFinalizationEnabled(JNIEnv * env)) > 694: return InstanceKlass::finalization_enabled() ? JNI_TRUE : JNI_FALSE; missing indentation ------------- Changes requested by kbarrett (Reviewer). PR: https://git.openjdk.java.net/jdk/pull/6442 From shade at openjdk.java.net Thu Nov 18 07:32:43 2021 From: shade at openjdk.java.net (Aleksey Shipilev) Date: Thu, 18 Nov 2021 07:32:43 GMT Subject: RFR: JDK-8276422 Add command-line option to disable finalization In-Reply-To: References: Message-ID: On Thu, 18 Nov 2021 01:34:36 GMT, Stuart Marks wrote: > Pretty much what it says. The new option controls a static member in InstanceKlass that's consulted to determine whether the finalization machinery is activated for instances when a class is loaded. A new native method is added so that this state can be queried from Java. This is used to control whether a finalizer thread is created and to disable the `System` and `Runtime::runFinalization` methods. Includes tests for the above. >From the brief look, it is OK. Minor nits. src/hotspot/share/prims/jvm.cpp line 694: > 692: > 693: JVM_ENTRY(jboolean, JVM_IsFinalizationEnabled(JNIEnv * env)) > 694: return InstanceKlass::finalization_enabled() ? JNI_TRUE : JNI_FALSE; Suggestion: return InstanceKlass::finalization_enabled() ? JNI_TRUE : JNI_FALSE; ------------- PR: https://git.openjdk.java.net/jdk/pull/6442 From shade at openjdk.java.net Thu Nov 18 07:32:44 2021 From: shade at openjdk.java.net (Aleksey Shipilev) Date: Thu, 18 Nov 2021 07:32:44 GMT Subject: RFR: JDK-8276422 Add command-line option to disable finalization In-Reply-To: References: Message-ID: On Thu, 18 Nov 2021 07:13:55 GMT, David Holmes wrote: >> Huh, good catch! This was mostly left over from an earlier version of the flag that used system properties, which aren't initialized until after the Finalizer class is initialized. >> >> It might be the case that the Holder can be removed at this point, since the finalization-enabled bit is no longer in a system property and is in a native class member that should be available before the VM is started. >> >> I say "might" though because this occurs early in system startup, and weird things potentially happen. For example, suppose the first object with a finalizer is created before the Finalizer class is initialized. The VM will perform an upcall to Finalizer::register. An ordinary call to a static method will ensure the class is initialized before proceeding with the call, but this VM upcall is a special case.... I'll have to investigate this some more. > > @stuart-marks not sure I see how anything is different here compared to the existing logic. The `Finalizer` class is explicitly initialized quite early in the init process, but if a preceding class's initialization created an object with a finalizer then that same upcall would be involved. Do we even have to have a flag on Java side? It looks like these calls are only done as the upcalls from VM, so we might just keep the flag on VM side? ------------- PR: https://git.openjdk.java.net/jdk/pull/6442 From dholmes at openjdk.java.net Thu Nov 18 07:43:35 2021 From: dholmes at openjdk.java.net (David Holmes) Date: Thu, 18 Nov 2021 07:43:35 GMT Subject: RFR: JDK-8276422 Add command-line option to disable finalization In-Reply-To: References: Message-ID: On Thu, 18 Nov 2021 07:27:30 GMT, Aleksey Shipilev wrote: >> @stuart-marks not sure I see how anything is different here compared to the existing logic. The `Finalizer` class is explicitly initialized quite early in the init process, but if a preceding class's initialization created an object with a finalizer then that same upcall would be involved. > > Do we even have to have a flag on Java side? It looks like these calls are only done as the upcalls from VM, so we might just keep the flag on VM side? @shipilev not sure what you mean by "a flag on the Java side". The Java code just queries the VM for the finalization enabled/disabled state and uses that to control things. ------------- PR: https://git.openjdk.java.net/jdk/pull/6442 From shade at openjdk.java.net Thu Nov 18 07:46:38 2021 From: shade at openjdk.java.net (Aleksey Shipilev) Date: Thu, 18 Nov 2021 07:46:38 GMT Subject: RFR: JDK-8276422 Add command-line option to disable finalization In-Reply-To: References: Message-ID: On Thu, 18 Nov 2021 07:40:34 GMT, David Holmes wrote: >> Do we even have to have a flag on Java side? It looks like these calls are only done as the upcalls from VM, so we might just keep the flag on VM side? > > @shipilev not sure what you mean by "a flag on the Java side". The Java code just queries the VM for the finalization enabled/disabled state and uses that to control things. Yeah, "flag" is `Holder.ENABLED` here. I mean, are Java methods `registerFinalizer` and `runFinalization` called only by VM? If so, can VM check the whole thing on VM side, without going to Java and asking back from there? ------------- PR: https://git.openjdk.java.net/jdk/pull/6442 From dholmes at openjdk.java.net Thu Nov 18 07:58:39 2021 From: dholmes at openjdk.java.net (David Holmes) Date: Thu, 18 Nov 2021 07:58:39 GMT Subject: RFR: JDK-8276422 Add command-line option to disable finalization In-Reply-To: References: Message-ID: On Thu, 18 Nov 2021 07:44:05 GMT, Aleksey Shipilev wrote: >> @shipilev not sure what you mean by "a flag on the Java side". The Java code just queries the VM for the finalization enabled/disabled state and uses that to control things. > > Yeah, "flag" is `Holder.ENABLED` here. I mean, are Java methods `registerFinalizer` and `runFinalization` called only by VM? If so, can VM check the whole thing on VM side, without going to Java and asking back from there? `registerFinalizer` does not expect to be called and only uses the "flag" as a form of assertion. `runFinalization` is called from Java code. ------------- PR: https://git.openjdk.java.net/jdk/pull/6442 From dholmes at openjdk.java.net Thu Nov 18 07:58:39 2021 From: dholmes at openjdk.java.net (David Holmes) Date: Thu, 18 Nov 2021 07:58:39 GMT Subject: RFR: JDK-8276422 Add command-line option to disable finalization In-Reply-To: <6s-4rTTyX8qZpavXbov9l2-H6BU7QLA0c71-K6xSQUM=.54dd6db2-5e78-4292-8958-72cadd762154@github.com> References: <6s-4rTTyX8qZpavXbov9l2-H6BU7QLA0c71-K6xSQUM=.54dd6db2-5e78-4292-8958-72cadd762154@github.com> Message-ID: On Thu, 18 Nov 2021 07:16:56 GMT, Kim Barrett wrote: > There is nothing here to make the various GCs take advantage of finalization being disabled. Is the plan to leave that to followup changes? @kimbarrett I provided the basic VM parts here. I'm not aware of what specifically a GC might optimise if it knows there can be no finalizers, but that seems like something the GC folk should look to providing as a follow up. Thanks. > src/hotspot/share/oops/instanceKlass.hpp line 338: > >> 336: >> 337: // Queries finalization state >> 338: static bool finalization_enabled() { return _finalization_enabled; } > > Predicate functions like this are often named "is_xxx"; that idiom is common in this class. This was intended as an accessor function, similar to `count()` or `offset()` not a query as-in `is_shared_boot_class()`. As it is a boolean field you could convert it to a query instead. ------------- PR: https://git.openjdk.java.net/jdk/pull/6442 From smarks at openjdk.java.net Thu Nov 18 08:04:07 2021 From: smarks at openjdk.java.net (Stuart Marks) Date: Thu, 18 Nov 2021 08:04:07 GMT Subject: RFR: JDK-8276422 Add command-line option to disable finalization [v2] In-Reply-To: References: Message-ID: > Pretty much what it says. The new option controls a static member in InstanceKlass that's consulted to determine whether the finalization machinery is activated for instances when a class is loaded. A new native method is added so that this state can be queried from Java. This is used to control whether a finalizer thread is created and to disable the `System` and `Runtime::runFinalization` methods. Includes tests for the above. Stuart Marks has updated the pull request incrementally with one additional commit since the last revision: Include instanceKlass.hpp in arguments.cpp ------------- Changes: - all: https://git.openjdk.java.net/jdk/pull/6442/files - new: https://git.openjdk.java.net/jdk/pull/6442/files/3836cc94..911af0b1 Webrevs: - full: https://webrevs.openjdk.java.net/?repo=jdk&pr=6442&range=01 - incr: https://webrevs.openjdk.java.net/?repo=jdk&pr=6442&range=00-01 Stats: 1 line in 1 file changed: 1 ins; 0 del; 0 mod Patch: https://git.openjdk.java.net/jdk/pull/6442.diff Fetch: git fetch https://git.openjdk.java.net/jdk pull/6442/head:pull/6442 PR: https://git.openjdk.java.net/jdk/pull/6442 From alanb at openjdk.java.net Thu Nov 18 08:42:40 2021 From: alanb at openjdk.java.net (Alan Bateman) Date: Thu, 18 Nov 2021 08:42:40 GMT Subject: RFR: JDK-8276422 Add command-line option to disable finalization [v2] In-Reply-To: References: Message-ID: On Thu, 18 Nov 2021 07:44:05 GMT, Aleksey Shipilev wrote: >> @shipilev not sure what you mean by "a flag on the Java side". The Java code just queries the VM for the finalization enabled/disabled state and uses that to control things. > > Yeah, "flag" is `Holder.ENABLED` here. I mean, are Java methods `registerFinalizer` and `runFinalization` called only by VM? If so, can VM check the whole thing on VM side, without going to Java and asking back from there? I think @shipilev asks a good question. This could be done completely in the VM without the changes to j.l.ref.Finalizer. The CLI option is for experimenting, at least in the short term, and should be benign to have the Finalizer thread running, it just won't do anything. ------------- PR: https://git.openjdk.java.net/jdk/pull/6442 From ysuenaga at openjdk.java.net Thu Nov 18 10:16:56 2021 From: ysuenaga at openjdk.java.net (Yasumasa Suenaga) Date: Thu, 18 Nov 2021 10:16:56 GMT Subject: RFR: 8277370: configure script cannot distinguish WSL version Message-ID: configure script distinguish WSL version if it is run on WSL. It is assumed WSL 2 if `/run/WSL` exists. However it exists in spite of WSL 1 on Windows 11 at least. We need to check it with other method. The method to distinguish WSL version has been discussed on https://github.com/microsoft/WSL/issues/4555 , however they do not seem to get consensus about it. `/run/WSL` was introduced in it, but I think `Hyper-V` in `/proc/interrupt` is more robustness because WSL 2 is run on Hyper-V. https://docs.microsoft.com/en-us/windows/wsl/faq#does-wsl-2-use-hyper-v--will-it-be-available-on-windows-10-home- ------------- Commit messages: - 8277370: configure script cannot distinguish WSL version Changes: https://git.openjdk.java.net/jdk/pull/6446/files Webrev: https://webrevs.openjdk.java.net/?repo=jdk&pr=6446&range=00 Issue: https://bugs.openjdk.java.net/browse/JDK-8277370 Stats: 5 lines in 1 file changed: 1 ins; 0 del; 4 mod Patch: https://git.openjdk.java.net/jdk/pull/6446.diff Fetch: git fetch https://git.openjdk.java.net/jdk pull/6446/head:pull/6446 PR: https://git.openjdk.java.net/jdk/pull/6446 From erikj at openjdk.java.net Thu Nov 18 14:06:38 2021 From: erikj at openjdk.java.net (Erik Joelsson) Date: Thu, 18 Nov 2021 14:06:38 GMT Subject: RFR: 8277370: configure script cannot distinguish WSL version In-Reply-To: References: Message-ID: On Thu, 18 Nov 2021 07:17:07 GMT, Yasumasa Suenaga wrote: > configure script distinguish WSL version if it is run on WSL. It is assumed WSL 2 if `/run/WSL` exists. > However it exists in spite of WSL 1 on Windows 11 at least. We need to check it with other method. > > The method to distinguish WSL version has been discussed on https://github.com/microsoft/WSL/issues/4555 , however they do not seem to get consensus about it. `/run/WSL` was introduced in it, but I think `Hyper-V` in `/proc/interrupt` is more robustness because WSL 2 is run on Hyper-V. https://docs.microsoft.com/en-us/windows/wsl/faq#does-wsl-2-use-hyper-v--will-it-be-available-on-windows-10-home- Marked as reviewed by erikj (Reviewer). make/autoconf/basic_windows.m4 line 39: > 37: # distinguishing between WSL1 and WSL2. > 38: # Check whether "Hyper-V" appears in /proc/interrupts because WSL2 runs on Hyper-V. > 39: grep Hyper-V /proc/interrupts > /dev/null 2>&1 You could use the -q switch to grep instead of piping to /dev/null. ------------- PR: https://git.openjdk.java.net/jdk/pull/6446 From nradomski at openjdk.java.net Thu Nov 18 17:22:28 2021 From: nradomski at openjdk.java.net (Niklas Radomski) Date: Thu, 18 Nov 2021 17:22:28 GMT Subject: RFR: 8276927: [PPC64] Port shenandoahgc to linux on ppc64le [v2] In-Reply-To: References: Message-ID: <-yZ-CH3Zdp9FbqRbcKA908KaXU7dWIuzy_SCyMluDr4=.31797995-8447-4902-89bd-693948925e49@github.com> > Port the Shenandoah garbage collector (JDK-8241457)[https://bugs.openjdk.java.net/browse/JDK-8241457] to linux on ppc64le. Niklas Radomski has updated the pull request incrementally with one additional commit since the last revision: Remove debug clobber code ------------- Changes: - all: https://git.openjdk.java.net/jdk/pull/6325/files - new: https://git.openjdk.java.net/jdk/pull/6325/files/1dec8885..c504b66d Webrevs: - full: https://webrevs.openjdk.java.net/?repo=jdk&pr=6325&range=01 - incr: https://webrevs.openjdk.java.net/?repo=jdk&pr=6325&range=00-01 Stats: 5 lines in 1 file changed: 0 ins; 5 del; 0 mod Patch: https://git.openjdk.java.net/jdk/pull/6325.diff Fetch: git fetch https://git.openjdk.java.net/jdk pull/6325/head:pull/6325 PR: https://git.openjdk.java.net/jdk/pull/6325 From mdoerr at openjdk.java.net Thu Nov 18 18:35:43 2021 From: mdoerr at openjdk.java.net (Martin Doerr) Date: Thu, 18 Nov 2021 18:35:43 GMT Subject: RFR: 8276927: [PPC64] Port shenandoahgc to linux on ppc64le [v2] In-Reply-To: <-yZ-CH3Zdp9FbqRbcKA908KaXU7dWIuzy_SCyMluDr4=.31797995-8447-4902-89bd-693948925e49@github.com> References: <-yZ-CH3Zdp9FbqRbcKA908KaXU7dWIuzy_SCyMluDr4=.31797995-8447-4902-89bd-693948925e49@github.com> Message-ID: <6ikSOeIWtJPZbIzHuiiEbSmpT60lFaZgOWejMxyAg80=.378fb020-a2e1-42f9-8e38-0985408f87f1@github.com> On Thu, 18 Nov 2021 17:22:28 GMT, Niklas Radomski wrote: >> Port the Shenandoah garbage collector (JDK-8241457)[https://bugs.openjdk.java.net/browse/JDK-8241457] to linux on ppc64le. > > Niklas Radomski has updated the pull request incrementally with one additional commit since the last revision: > > Remove debug clobber code Thanks for the update! I think it's good to go. ------------- Marked as reviewed by mdoerr (Reviewer). PR: https://git.openjdk.java.net/jdk/pull/6325 From nradomski at openjdk.java.net Thu Nov 18 18:58:39 2021 From: nradomski at openjdk.java.net (Niklas Radomski) Date: Thu, 18 Nov 2021 18:58:39 GMT Subject: RFR: 8276927: [PPC64] Port shenandoahgc to linux on ppc64le [v2] In-Reply-To: <-yZ-CH3Zdp9FbqRbcKA908KaXU7dWIuzy_SCyMluDr4=.31797995-8447-4902-89bd-693948925e49@github.com> References: <-yZ-CH3Zdp9FbqRbcKA908KaXU7dWIuzy_SCyMluDr4=.31797995-8447-4902-89bd-693948925e49@github.com> Message-ID: On Thu, 18 Nov 2021 17:22:28 GMT, Niklas Radomski wrote: >> Port the Shenandoah garbage collector [JDK-8241457](https://bugs.openjdk.java.net/browse/JDK-8241457) to linux on ppc64le. > > Niklas Radomski has updated the pull request incrementally with one additional commit since the last revision: > > Remove debug clobber code Thank you for your reviews! Happy to see that the change has been so well received. ------------- PR: https://git.openjdk.java.net/jdk/pull/6325 From nradomski at openjdk.java.net Thu Nov 18 19:07:46 2021 From: nradomski at openjdk.java.net (Niklas Radomski) Date: Thu, 18 Nov 2021 19:07:46 GMT Subject: Integrated: 8276927: [PPC64] Port shenandoahgc to linux on ppc64le In-Reply-To: References: Message-ID: On Wed, 10 Nov 2021 09:00:04 GMT, Niklas Radomski wrote: > Port the Shenandoah garbage collector [JDK-8241457](https://bugs.openjdk.java.net/browse/JDK-8241457) to linux on ppc64le. This pull request has now been integrated. Changeset: 57eb8647 Author: Niklas Radomski Committer: Martin Doerr URL: https://git.openjdk.java.net/jdk/commit/57eb864765f38185f8db8f1d37681d6cfe2a3c73 Stats: 1521 lines in 8 files changed: 1519 ins; 0 del; 2 mod 8276927: [PPC64] Port shenandoahgc to linux on ppc64le Reviewed-by: rkennke, ihse, mdoerr ------------- PR: https://git.openjdk.java.net/jdk/pull/6325 From ysuenaga at openjdk.java.net Fri Nov 19 00:00:15 2021 From: ysuenaga at openjdk.java.net (Yasumasa Suenaga) Date: Fri, 19 Nov 2021 00:00:15 GMT Subject: RFR: 8277370: configure script cannot distinguish WSL version [v2] In-Reply-To: References: Message-ID: > configure script distinguish WSL version if it is run on WSL. It is assumed WSL 2 if `/run/WSL` exists. > However it exists in spite of WSL 1 on Windows 11 at least. We need to check it with other method. > > The method to distinguish WSL version has been discussed on https://github.com/microsoft/WSL/issues/4555 , however they do not seem to get consensus about it. `/run/WSL` was introduced in it, but I think `Hyper-V` in `/proc/interrupt` is more robustness because WSL 2 is run on Hyper-V. https://docs.microsoft.com/en-us/windows/wsl/faq#does-wsl-2-use-hyper-v--will-it-be-available-on-windows-10-home- Yasumasa Suenaga has updated the pull request incrementally with one additional commit since the last revision: Use $GREP to check /proc/interrupts ------------- Changes: - all: https://git.openjdk.java.net/jdk/pull/6446/files - new: https://git.openjdk.java.net/jdk/pull/6446/files/5f58daa3..13f1f8b9 Webrevs: - full: https://webrevs.openjdk.java.net/?repo=jdk&pr=6446&range=01 - incr: https://webrevs.openjdk.java.net/?repo=jdk&pr=6446&range=00-01 Stats: 1 line in 1 file changed: 0 ins; 0 del; 1 mod Patch: https://git.openjdk.java.net/jdk/pull/6446.diff Fetch: git fetch https://git.openjdk.java.net/jdk pull/6446/head:pull/6446 PR: https://git.openjdk.java.net/jdk/pull/6446 From ysuenaga at openjdk.java.net Fri Nov 19 00:00:19 2021 From: ysuenaga at openjdk.java.net (Yasumasa Suenaga) Date: Fri, 19 Nov 2021 00:00:19 GMT Subject: RFR: 8277370: configure script cannot distinguish WSL version [v2] In-Reply-To: References: Message-ID: <0pGC2QtzlcuLTPhjyqz_3HVlITRZqXuwxoKrjgII-84=.11064c1c-73f6-40d5-874a-6d00c85146fa@github.com> On Thu, 18 Nov 2021 14:03:33 GMT, Erik Joelsson wrote: >> Yasumasa Suenaga has updated the pull request incrementally with one additional commit since the last revision: >> >> Use $GREP to check /proc/interrupts > > make/autoconf/basic_windows.m4 line 39: > >> 37: # distinguishing between WSL1 and WSL2. >> 38: # Check whether "Hyper-V" appears in /proc/interrupts because WSL2 runs on Hyper-V. >> 39: grep Hyper-V /proc/interrupts > /dev/null 2>&1 > > You could use the -q switch to grep instead of piping to /dev/null. Thanks for your review! I updated to add `-q` and to use `$GREP` instead of `grep`. ------------- PR: https://git.openjdk.java.net/jdk/pull/6446 From redestad at openjdk.java.net Fri Nov 19 01:28:55 2021 From: redestad at openjdk.java.net (Claes Redestad) Date: Fri, 19 Nov 2021 01:28:55 GMT Subject: RFR: 8277427: Update jib-profiles to use JMH 1.33 devkit Message-ID: Use most recent devkit. Testing: staged devkit, built and verified micros locally ------------- Commit messages: - Update jib-profiles to use JMH 1.33 devkit Changes: https://git.openjdk.java.net/jdk/pull/6468/files Webrev: https://webrevs.openjdk.java.net/?repo=jdk&pr=6468&range=00 Issue: https://bugs.openjdk.java.net/browse/JDK-8277427 Stats: 1 line in 1 file changed: 0 ins; 0 del; 1 mod Patch: https://git.openjdk.java.net/jdk/pull/6468.diff Fetch: git fetch https://git.openjdk.java.net/jdk pull/6468/head:pull/6468 PR: https://git.openjdk.java.net/jdk/pull/6468 From shade at openjdk.java.net Fri Nov 19 06:35:39 2021 From: shade at openjdk.java.net (Aleksey Shipilev) Date: Fri, 19 Nov 2021 06:35:39 GMT Subject: RFR: 8277427: Update jib-profiles to use JMH 1.33 devkit In-Reply-To: References: Message-ID: On Fri, 19 Nov 2021 01:20:43 GMT, Claes Redestad wrote: > Use most recent devkit. > > Testing: staged devkit, built and verified micros locally Marked as reviewed by shade (Reviewer). ------------- PR: https://git.openjdk.java.net/jdk/pull/6468 From ihse at openjdk.java.net Fri Nov 19 12:33:43 2021 From: ihse at openjdk.java.net (Magnus Ihse Bursie) Date: Fri, 19 Nov 2021 12:33:43 GMT Subject: RFR: 8275745: Reproducible copyright headers [v3] In-Reply-To: References: <2PURhxGA3r-6JCeIFUuYByG1KaAIF3MUEMBACbHza4c=.ab23d1af-107a-4989-ae34-f51f9273c26b@github.com> Message-ID: <5bbBHjOuUcFoh25tl2_Pt_w_s5ONxQUrGddkZI0z_7s=.560c47d8-49e1-4541-9656-7683b166de84@github.com> On Thu, 11 Nov 2021 21:34:58 GMT, Emmanuel Bourg wrote: >> Emmanuel Bourg has updated the pull request with a new target base due to a merge or a rebase. The incremental webrev excludes the unrelated changes brought in by the merge/rebase. The pull request contains two additional commits since the last revision: >> >> - Make gensrc code use $COPYRIGHT_YEAR >> - Set COPYRIGHT_YEAR from SOURCE_DATE_EPOCH if present > > Thank you for the advice, I'm not familiar with the Skara workflow yet. @ebourg To finish integrating this PR, you need to type `/integrate` as a PR comment. This will allow the Skara bots to move the bug on to the state where I can sponsor it. ------------- PR: https://git.openjdk.java.net/jdk/pull/1498 From erikj at openjdk.java.net Fri Nov 19 13:13:42 2021 From: erikj at openjdk.java.net (Erik Joelsson) Date: Fri, 19 Nov 2021 13:13:42 GMT Subject: RFR: 8277370: configure script cannot distinguish WSL version [v2] In-Reply-To: References: Message-ID: On Fri, 19 Nov 2021 00:00:15 GMT, Yasumasa Suenaga wrote: >> configure script distinguish WSL version if it is run on WSL. It is assumed WSL 2 if `/run/WSL` exists. >> However it exists in spite of WSL 1 on Windows 11 at least. We need to check it with other method. >> >> The method to distinguish WSL version has been discussed on https://github.com/microsoft/WSL/issues/4555 , however they do not seem to get consensus about it. `/run/WSL` was introduced in it, but I think `Hyper-V` in `/proc/interrupt` is more robustness because WSL 2 is run on Hyper-V. https://docs.microsoft.com/en-us/windows/wsl/faq#does-wsl-2-use-hyper-v--will-it-be-available-on-windows-10-home- > > Yasumasa Suenaga has updated the pull request incrementally with one additional commit since the last revision: > > Use $GREP to check /proc/interrupts Marked as reviewed by erikj (Reviewer). ------------- PR: https://git.openjdk.java.net/jdk/pull/6446 From erikj at openjdk.java.net Fri Nov 19 13:14:52 2021 From: erikj at openjdk.java.net (Erik Joelsson) Date: Fri, 19 Nov 2021 13:14:52 GMT Subject: RFR: 8277427: Update jib-profiles to use JMH 1.33 devkit In-Reply-To: References: Message-ID: On Fri, 19 Nov 2021 01:20:43 GMT, Claes Redestad wrote: > Use most recent devkit. > > Testing: staged devkit, built and verified micros locally Marked as reviewed by erikj (Reviewer). ------------- PR: https://git.openjdk.java.net/jdk/pull/6468 From duke at openjdk.java.net Fri Nov 19 13:17:44 2021 From: duke at openjdk.java.net (Emmanuel Bourg) Date: Fri, 19 Nov 2021 13:17:44 GMT Subject: RFR: 8275745: Reproducible copyright headers [v3] In-Reply-To: <5bbBHjOuUcFoh25tl2_Pt_w_s5ONxQUrGddkZI0z_7s=.560c47d8-49e1-4541-9656-7683b166de84@github.com> References: <2PURhxGA3r-6JCeIFUuYByG1KaAIF3MUEMBACbHza4c=.ab23d1af-107a-4989-ae34-f51f9273c26b@github.com> <5bbBHjOuUcFoh25tl2_Pt_w_s5ONxQUrGddkZI0z_7s=.560c47d8-49e1-4541-9656-7683b166de84@github.com> Message-ID: <__e0LLw1GW8pETTtiYuWRT0bMF0_S43IXYBoAI2hFws=.3819dd75-c754-4751-b54b-9a3171a07a03@github.com> On Fri, 19 Nov 2021 12:30:41 GMT, Magnus Ihse Bursie wrote: >> Thank you for the advice, I'm not familiar with the Skara workflow yet. > > @ebourg To finish integrating this PR, you need to type `/integrate` as a PR comment. This will allow the Skara bots to move the bug on to the state where I can sponsor it. @magicus Thank you for the hint! ------------- PR: https://git.openjdk.java.net/jdk/pull/1498 From redestad at openjdk.java.net Fri Nov 19 13:29:43 2021 From: redestad at openjdk.java.net (Claes Redestad) Date: Fri, 19 Nov 2021 13:29:43 GMT Subject: RFR: 8277427: Update jib-profiles.js to use JMH 1.33 devkit In-Reply-To: References: Message-ID: <_Usad71LCxC5lao6tl85jlt3dnXgOaQLPX4P7o34kTg=.69b1e494-9c8b-4e2b-9228-a724dc4d2d33@github.com> On Fri, 19 Nov 2021 06:32:20 GMT, Aleksey Shipilev wrote: >> Use most recent devkit. >> >> Testing: staged devkit, built and verified micros locally > > Marked as reviewed by shade (Reviewer). Thanks for reviews, @shipilev and @erikj79 ------------- PR: https://git.openjdk.java.net/jdk/pull/6468 From redestad at openjdk.java.net Fri Nov 19 13:29:44 2021 From: redestad at openjdk.java.net (Claes Redestad) Date: Fri, 19 Nov 2021 13:29:44 GMT Subject: Integrated: 8277427: Update jib-profiles.js to use JMH 1.33 devkit In-Reply-To: References: Message-ID: On Fri, 19 Nov 2021 01:20:43 GMT, Claes Redestad wrote: > Use most recent devkit. > > Testing: staged devkit, built and verified micros locally This pull request has now been integrated. Changeset: b1a1bf4e Author: Claes Redestad URL: https://git.openjdk.java.net/jdk/commit/b1a1bf4e305d6675f8f751aa8fef7013d99466f1 Stats: 1 line in 1 file changed: 0 ins; 0 del; 1 mod 8277427: Update jib-profiles.js to use JMH 1.33 devkit Reviewed-by: shade, erikj ------------- PR: https://git.openjdk.java.net/jdk/pull/6468 From ihse at openjdk.java.net Fri Nov 19 13:59:47 2021 From: ihse at openjdk.java.net (Magnus Ihse Bursie) Date: Fri, 19 Nov 2021 13:59:47 GMT Subject: RFR: 8275745: Reproducible copyright headers [v3] In-Reply-To: <__e0LLw1GW8pETTtiYuWRT0bMF0_S43IXYBoAI2hFws=.3819dd75-c754-4751-b54b-9a3171a07a03@github.com> References: <2PURhxGA3r-6JCeIFUuYByG1KaAIF3MUEMBACbHza4c=.ab23d1af-107a-4989-ae34-f51f9273c26b@github.com> <5bbBHjOuUcFoh25tl2_Pt_w_s5ONxQUrGddkZI0z_7s=.560c47d8-49e1-4541-9656-7683b166de84@github.com> <__e0LLw1GW8pETTtiYuWRT0bMF0_S43IXYBoAI2hFws=.3819dd75-c754-4751-b54b-9a3171a07a03@github.com> Message-ID: <4eUoRMB1H-iLn99HObsJIYKd0ZNwAOFQHR0HmemuNlA=.15905dc2-c7b1-4018-bb0c-8a1d3b40753a@github.com> On Fri, 19 Nov 2021 13:14:56 GMT, Emmanuel Bourg wrote: >> @ebourg To finish integrating this PR, you need to type `/integrate` as a PR comment. This will allow the Skara bots to move the bug on to the state where I can sponsor it. > > @magicus Thank you for the hint! @ebourg Thank you for your contribution! It's been a long and bumpy ride for something that seemed like a simple patch. I hope you have not been scared away from contributing to the JDK in the future. :-) ------------- PR: https://git.openjdk.java.net/jdk/pull/1498 From duke at openjdk.java.net Fri Nov 19 13:59:47 2021 From: duke at openjdk.java.net (Emmanuel Bourg) Date: Fri, 19 Nov 2021 13:59:47 GMT Subject: Integrated: 8275745: Reproducible copyright headers In-Reply-To: References: Message-ID: On Sat, 28 Nov 2020 23:14:35 GMT, Emmanuel Bourg wrote: > The copyright headers are generated at build time, and the year inserted in the template depends on the current date. This means the headers are not reproducible if the project is built a year later. The year in the headers could be derived from the SOURCE_DATE_EPOCH environment variable to make the build reproducible (this variable is already used in other parts of the build). This pull request has now been integrated. Changeset: a0227965 Author: Magnus Ihse Bursie Committer: Magnus Ihse Bursie URL: https://git.openjdk.java.net/jdk/commit/a0227965bb8d1d49718794d67f8a4cfeebc9bec2 Stats: 53 lines in 8 files changed: 33 ins; 11 del; 9 mod 8275745: Reproducible copyright headers Reviewed-by: ihse, erikj ------------- PR: https://git.openjdk.java.net/jdk/pull/1498 From duke at openjdk.java.net Fri Nov 19 14:24:47 2021 From: duke at openjdk.java.net (Emmanuel Bourg) Date: Fri, 19 Nov 2021 14:24:47 GMT Subject: RFR: 8275745: Reproducible copyright headers [v3] In-Reply-To: <2PURhxGA3r-6JCeIFUuYByG1KaAIF3MUEMBACbHza4c=.ab23d1af-107a-4989-ae34-f51f9273c26b@github.com> References: <2PURhxGA3r-6JCeIFUuYByG1KaAIF3MUEMBACbHza4c=.ab23d1af-107a-4989-ae34-f51f9273c26b@github.com> Message-ID: On Thu, 11 Nov 2021 15:37:09 GMT, Emmanuel Bourg wrote: >> The copyright headers are generated at build time, and the year inserted in the template depends on the current date. This means the headers are not reproducible if the project is built a year later. The year in the headers could be derived from the SOURCE_DATE_EPOCH environment variable to make the build reproducible (this variable is already used in other parts of the build). > > Emmanuel Bourg has updated the pull request with a new target base due to a merge or a rebase. The pull request now contains two commits: > > - Make gensrc code use $COPYRIGHT_YEAR > - Set COPYRIGHT_YEAR from SOURCE_DATE_EPOCH if present Thanks again for walking me through the process, I'll try to submit more reproducibility fixes, there are a few more patches pending in Debian: https://salsa.debian.org/openjdk-team/openjdk/-/blob/openjdk-17/debian/patches/reproducible-build-jmod.diff https://salsa.debian.org/openjdk-team/openjdk/-/blob/openjdk-17/debian/patches/reproducible-build-user.diff https://salsa.debian.org/openjdk-team/openjdk/-/blob/openjdk-17/debian/patches/reproducible-character-data.diff https://salsa.debian.org/openjdk-team/openjdk/-/blob/openjdk-17/debian/patches/reproducible-javadoc-timestamp.diff https://salsa.debian.org/openjdk-team/openjdk/-/blob/openjdk-17/debian/patches/reproducible-module-info.diff https://salsa.debian.org/openjdk-team/openjdk/-/blob/openjdk-17/debian/patches/reproducible-properties-timestamp.diff ------------- PR: https://git.openjdk.java.net/jdk/pull/1498 From ysuenaga at openjdk.java.net Fri Nov 19 20:28:13 2021 From: ysuenaga at openjdk.java.net (Yasumasa Suenaga) Date: Fri, 19 Nov 2021 20:28:13 GMT Subject: Integrated: 8277370: configure script cannot distinguish WSL version In-Reply-To: References: Message-ID: On Thu, 18 Nov 2021 07:17:07 GMT, Yasumasa Suenaga wrote: > configure script distinguish WSL version if it is run on WSL. It is assumed WSL 2 if `/run/WSL` exists. > However it exists in spite of WSL 1 on Windows 11 at least. We need to check it with other method. > > The method to distinguish WSL version has been discussed on https://github.com/microsoft/WSL/issues/4555 , however they do not seem to get consensus about it. `/run/WSL` was introduced in it, but I think `Hyper-V` in `/proc/interrupt` is more robustness because WSL 2 is run on Hyper-V. https://docs.microsoft.com/en-us/windows/wsl/faq#does-wsl-2-use-hyper-v--will-it-be-available-on-windows-10-home- This pull request has now been integrated. Changeset: 2d4af225 Author: Yasumasa Suenaga URL: https://git.openjdk.java.net/jdk/commit/2d4af2255feb2eaeca533424f8cba3ec0945d757 Stats: 5 lines in 1 file changed: 1 ins; 0 del; 4 mod 8277370: configure script cannot distinguish WSL version Reviewed-by: erikj ------------- PR: https://git.openjdk.java.net/jdk/pull/6446 From asemenyuk at openjdk.java.net Sat Nov 20 03:34:27 2021 From: asemenyuk at openjdk.java.net (Alexey Semenyuk) Date: Sat, 20 Nov 2021 03:34:27 GMT Subject: RFR: 8277429: Conflicting jpackage static library name Message-ID: 8277429: Conflicting jpackage static library name ------------- Commit messages: - 8277429: Conflicting jpackage static library name Changes: https://git.openjdk.java.net/jdk/pull/6485/files Webrev: https://webrevs.openjdk.java.net/?repo=jdk&pr=6485&range=00 Issue: https://bugs.openjdk.java.net/browse/JDK-8277429 Stats: 3 lines in 2 files changed: 0 ins; 0 del; 3 mod Patch: https://git.openjdk.java.net/jdk/pull/6485.diff Fetch: git fetch https://git.openjdk.java.net/jdk pull/6485/head:pull/6485 PR: https://git.openjdk.java.net/jdk/pull/6485 From almatvee at openjdk.java.net Sat Nov 20 05:32:09 2021 From: almatvee at openjdk.java.net (Alexander Matveev) Date: Sat, 20 Nov 2021 05:32:09 GMT Subject: RFR: 8277429: Conflicting jpackage static library name In-Reply-To: References: Message-ID: <67fGgoeV2WY6-Foi4wi383G-gNNnvo17KRT9jrffRWU=.37001d11-7130-41fe-a16f-06f4ba2060f9@github.com> On Sat, 20 Nov 2021 03:26:51 GMT, Alexey Semenyuk wrote: > 8277429: Conflicting jpackage static library name Marked as reviewed by almatvee (Reviewer). ------------- PR: https://git.openjdk.java.net/jdk/pull/6485 From herrick at openjdk.java.net Sat Nov 20 11:52:06 2021 From: herrick at openjdk.java.net (Andy Herrick) Date: Sat, 20 Nov 2021 11:52:06 GMT Subject: RFR: 8277429: Conflicting jpackage static library name In-Reply-To: References: Message-ID: On Sat, 20 Nov 2021 03:26:51 GMT, Alexey Semenyuk wrote: > 8277429: Conflicting jpackage static library name Marked as reviewed by herrick (Reviewer). ------------- PR: https://git.openjdk.java.net/jdk/pull/6485 From ihse at openjdk.java.net Sat Nov 20 21:44:10 2021 From: ihse at openjdk.java.net (Magnus Ihse Bursie) Date: Sat, 20 Nov 2021 21:44:10 GMT Subject: RFR: 8275745: Reproducible copyright headers [v3] In-Reply-To: References: <2PURhxGA3r-6JCeIFUuYByG1KaAIF3MUEMBACbHza4c=.ab23d1af-107a-4989-ae34-f51f9273c26b@github.com> Message-ID: On Fri, 19 Nov 2021 14:21:59 GMT, Emmanuel Bourg wrote: >> Emmanuel Bourg has updated the pull request with a new target base due to a merge or a rebase. The pull request now contains two commits: >> >> - Make gensrc code use $COPYRIGHT_YEAR >> - Set COPYRIGHT_YEAR from SOURCE_DATE_EPOCH if present > > Thanks again for walking me through the process, I'll try to submit more reproducibility fixes, there are a few more patches pending in Debian: > https://salsa.debian.org/openjdk-team/openjdk/-/blob/openjdk-17/debian/patches/reproducible-build-jmod.diff > https://salsa.debian.org/openjdk-team/openjdk/-/blob/openjdk-17/debian/patches/reproducible-build-user.diff > https://salsa.debian.org/openjdk-team/openjdk/-/blob/openjdk-17/debian/patches/reproducible-character-data.diff > https://salsa.debian.org/openjdk-team/openjdk/-/blob/openjdk-17/debian/patches/reproducible-javadoc-timestamp.diff > https://salsa.debian.org/openjdk-team/openjdk/-/blob/openjdk-17/debian/patches/reproducible-module-info.diff > https://salsa.debian.org/openjdk-team/openjdk/-/blob/openjdk-17/debian/patches/reproducible-properties-timestamp.diff @ebourg You might be interested in watching what @andrew-m-leonard is doing in the area, for example in https://github.com/openjdk/jdk/pull/6395. ------------- PR: https://git.openjdk.java.net/jdk/pull/1498 From jonathan.gibbons at oracle.com Sun Nov 21 15:47:29 2021 From: jonathan.gibbons at oracle.com (Jonathan Gibbons) Date: Sun, 21 Nov 2021 07:47:29 -0800 Subject: RFR: 8275745: Reproducible copyright headers [v3] In-Reply-To: References: <2PURhxGA3r-6JCeIFUuYByG1KaAIF3MUEMBACbHza4c=.ab23d1af-107a-4989-ae34-f51f9273c26b@github.com> Message-ID: <6b990371-e239-5a20-dd76-342a4db8776d@oracle.com> Emmanual, If you're interested, please contact the javadoc-dev at openjdk.java.net list regarding the javadoc fix. While the diff looks promising, it would require a corresponding regression test. -- Jon On 11/19/21 6:24 AM, Emmanuel Bourg wrote: > On Thu, 11 Nov 2021 15:37:09 GMT, Emmanuel Bourg wrote: [snip] > https://salsa.debian.org/openjdk-team/openjdk/-/blob/openjdk-17/debian/patches/reproducible-javadoc-timestamp.diff [snip] > > ------------- > > PR: https://git.openjdk.java.net/jdk/pull/1498 From ngasson at openjdk.java.net Mon Nov 22 01:44:10 2021 From: ngasson at openjdk.java.net (Nick Gasson) Date: Mon, 22 Nov 2021 01:44:10 GMT Subject: RFR: 8277204: Implementation of JEP 8264130: PAC-RET protection for Linux/AArch64 [v6] In-Reply-To: References: Message-ID: <92o4fGbGUNrm39p_vfOJV2cIXw2lzq5CLYPhdlf4hwI=.87d970cc-7f40-4e10-8202-ecc3fd47d8be@github.com> On Tue, 16 Nov 2021 14:23:07 GMT, Alan Hayward wrote: >> PAC is an optional feature in AArch64 8.3 and is compulsory in v9. One >> of its uses is to protect against ROP based attacks. This is done by >> signing the Link Register whenever it is stored on the stack, and >> authenticating the value when it is loaded back from the stack. If an >> attacker were to try to change control flow by editing the stack then >> the authentication check of the Link Register will fail, causing a >> segfault when the function returns. >> >> On a system with PAC enabled, it is expected that all applications will >> be compiled with ROP protection. Fedora 33 and upwards already provide >> this. By compiling for ARMv8.0, GCC and LLVM will only use the set of >> PAC instructions that exist in the NOP space - on hardware without PAC, >> these instructions act as NOPs, allowing backward compatibility for >> negligible performance cost (2 NOPs per non-leaf function). >> >> Hardware is currently limited to the Apple M1 MacBooks. All testing has >> been done within a Fedora Docker image. A run of SpecJVM showed no >> difference to that of noise - which was surprising. >> >> The most important part of this patch is simply compiling using branch >> protection provided by GCC/LLVM. This protects all C++ code from being >> used in ROP attacks, removing all static ROP gadgets from use. >> >> The remainder of the patch adds ROP protection to runtime generated >> code, in both stubs and compiled Java code. Attacks here are much harder >> as ROP gadgets must be found dynamically at runtime. If/when AOT >> compilation is added to JDK, then all stubs and compiled Java will be >> susceptible ROP gadgets being found by static analysis and therefore >> potentially as vulnerable as C++ code. >> >> There are a number of places where the VM changes control flow by >> rewriting the stack or otherwise. I?ve done some analysis as to how >> these could also be used for attacks (which I didn?t want to post here). >> These areas can be protected ensuring the pointers to various stubs and >> entry points are stored in memory as signed pointers. These changes are >> simple to make (they can be reduced to a type change in common code and >> a few addition sign/auth calls in the backend), but there a lot of them >> and the total code change is fairly large. I?m happy to provide a few >> work in progress patches. >> >> In order to match the security benefits of the Apple Arm64e ABI across >> the whole of JDK, then all the changes mentioned above would be >> required. > > Alan Hayward has updated the pull request with a new target base due to a merge or a rebase. The pull request now contains 12 commits: > > - Merge master > - Rename pauth_authenticate_or_strip_return_address > - Fix windows aarch64 by restoring pauth file split > - Don't keep LR live across restore_live_registers > - Merge master > - Document pauth functions && remove OS split > - Update UseROPProtection description > - Simplify branch protection configure check > - 8264130: PAC-RET protection for Linux/AArch64 > > PAC is an optional feature in AArch64 8.3 and is compulsory in v9. One > of its uses is to protect against ROP based attacks. This is done by > signing the Link Register whenever it is stored on the stack, and > authenticating the value when it is loaded back from the stack. If an > attacker were to try to change control flow by editing the stack then > the authentication check of the Link Register will fail, causing a > segfault when the function returns. > > On a system with PAC enabled, it is expected that all applications will > be compiled with ROP protection. Fedora 33 and upwards already provide > this. By compiling for ARMv8.0, GCC and LLVM will only use the set of > PAC instructions that exist in the NOP space - on hardware without PAC, > these instructions act as NOPs, allowing backward compatibility for > negligible performance cost (2 NOPs per non-leaf function). > > Hardware is currently limited to the Apple M1 MacBooks. All testing has > been done within a Fedora Docker image. A run of SpecJVM showed no > difference to that of noise - which was surprising. > > The most important part of this patch is simply compiling using branch > protection provided by GCC/LLVM. This protects all C++ code from being > used in ROP attacks, removing all static ROP gadgets from use. > > The remainder of the patch adds ROP protection to runtime generated > code, in both stubs and compiled Java code. Attacks here are much harder > as ROP gadgets must be found dynamically at runtime. If/when AOT > compilation is added to JDK, then all stubs and compiled Java will be > susceptible ROP gadgets being found by static analysis and therefore > potentially as vulnerable as C++ code. > > There are a number of places where the VM changes control flow by > rewriting the stack or otherwise. I?ve done some analysis as to how > these could also be used for attacks (which I didn?t want to post here). > These areas can be protected ensuring the pointers to various stubs and > entry points are stored in memory as signed pointers. These changes are > simple to make (they can be reduced to a type change in common code and > a few addition sign/auth calls in the backend), but there a lot of them > and the total code change is fairly large. I?m happy to provide a few > work in progress patches. > > In order to match the security benefits of the Apple Arm64e ABI across > the whole of JDK, then all the changes mentioned above would be > required. > - Add PAC assembly instructions > - ... and 2 more: https://git.openjdk.java.net/jdk/compare/b8d33a2a...deb17a56 LGTM and we did extensive jtreg testing internally (tier1 + hotspot_all, jdk_core). ------------- Marked as reviewed by ngasson (Reviewer). PR: https://git.openjdk.java.net/jdk/pull/6334 From ngasson at openjdk.java.net Mon Nov 22 02:03:13 2021 From: ngasson at openjdk.java.net (Nick Gasson) Date: Mon, 22 Nov 2021 02:03:13 GMT Subject: RFR: 8277204: Implementation of JEP 8264130: PAC-RET protection for Linux/AArch64 [v6] In-Reply-To: References: Message-ID: On Tue, 16 Nov 2021 14:23:07 GMT, Alan Hayward wrote: >> PAC is an optional feature in AArch64 8.3 and is compulsory in v9. One >> of its uses is to protect against ROP based attacks. This is done by >> signing the Link Register whenever it is stored on the stack, and >> authenticating the value when it is loaded back from the stack. If an >> attacker were to try to change control flow by editing the stack then >> the authentication check of the Link Register will fail, causing a >> segfault when the function returns. >> >> On a system with PAC enabled, it is expected that all applications will >> be compiled with ROP protection. Fedora 33 and upwards already provide >> this. By compiling for ARMv8.0, GCC and LLVM will only use the set of >> PAC instructions that exist in the NOP space - on hardware without PAC, >> these instructions act as NOPs, allowing backward compatibility for >> negligible performance cost (2 NOPs per non-leaf function). >> >> Hardware is currently limited to the Apple M1 MacBooks. All testing has >> been done within a Fedora Docker image. A run of SpecJVM showed no >> difference to that of noise - which was surprising. >> >> The most important part of this patch is simply compiling using branch >> protection provided by GCC/LLVM. This protects all C++ code from being >> used in ROP attacks, removing all static ROP gadgets from use. >> >> The remainder of the patch adds ROP protection to runtime generated >> code, in both stubs and compiled Java code. Attacks here are much harder >> as ROP gadgets must be found dynamically at runtime. If/when AOT >> compilation is added to JDK, then all stubs and compiled Java will be >> susceptible ROP gadgets being found by static analysis and therefore >> potentially as vulnerable as C++ code. >> >> There are a number of places where the VM changes control flow by >> rewriting the stack or otherwise. I?ve done some analysis as to how >> these could also be used for attacks (which I didn?t want to post here). >> These areas can be protected ensuring the pointers to various stubs and >> entry points are stored in memory as signed pointers. These changes are >> simple to make (they can be reduced to a type change in common code and >> a few addition sign/auth calls in the backend), but there a lot of them >> and the total code change is fairly large. I?m happy to provide a few >> work in progress patches. >> >> In order to match the security benefits of the Apple Arm64e ABI across >> the whole of JDK, then all the changes mentioned above would be >> required. > > Alan Hayward has updated the pull request with a new target base due to a merge or a rebase. The pull request now contains 12 commits: > > - Merge master > - Rename pauth_authenticate_or_strip_return_address > - Fix windows aarch64 by restoring pauth file split > - Don't keep LR live across restore_live_registers > - Merge master > - Document pauth functions && remove OS split > - Update UseROPProtection description > - Simplify branch protection configure check > - 8264130: PAC-RET protection for Linux/AArch64 > > PAC is an optional feature in AArch64 8.3 and is compulsory in v9. One > of its uses is to protect against ROP based attacks. This is done by > signing the Link Register whenever it is stored on the stack, and > authenticating the value when it is loaded back from the stack. If an > attacker were to try to change control flow by editing the stack then > the authentication check of the Link Register will fail, causing a > segfault when the function returns. > > On a system with PAC enabled, it is expected that all applications will > be compiled with ROP protection. Fedora 33 and upwards already provide > this. By compiling for ARMv8.0, GCC and LLVM will only use the set of > PAC instructions that exist in the NOP space - on hardware without PAC, > these instructions act as NOPs, allowing backward compatibility for > negligible performance cost (2 NOPs per non-leaf function). > > Hardware is currently limited to the Apple M1 MacBooks. All testing has > been done within a Fedora Docker image. A run of SpecJVM showed no > difference to that of noise - which was surprising. > > The most important part of this patch is simply compiling using branch > protection provided by GCC/LLVM. This protects all C++ code from being > used in ROP attacks, removing all static ROP gadgets from use. > > The remainder of the patch adds ROP protection to runtime generated > code, in both stubs and compiled Java code. Attacks here are much harder > as ROP gadgets must be found dynamically at runtime. If/when AOT > compilation is added to JDK, then all stubs and compiled Java will be > susceptible ROP gadgets being found by static analysis and therefore > potentially as vulnerable as C++ code. > > There are a number of places where the VM changes control flow by > rewriting the stack or otherwise. I?ve done some analysis as to how > these could also be used for attacks (which I didn?t want to post here). > These areas can be protected ensuring the pointers to various stubs and > entry points are stored in memory as signed pointers. These changes are > simple to make (they can be reduced to a type change in common code and > a few addition sign/auth calls in the backend), but there a lot of them > and the total code change is fairly large. I?m happy to provide a few > work in progress patches. > > In order to match the security benefits of the Apple Arm64e ABI across > the whole of JDK, then all the changes mentioned above would be > required. > - Add PAC assembly instructions > - ... and 2 more: https://git.openjdk.java.net/jdk/compare/b8d33a2a...deb17a56 We're adding a product option `UseROPProtection` which needs a CSR according to https://wiki.openjdk.java.net/display/HotSpot/Hotspot+Command-line+Flags%3A+Kinds%2C+Lifecycle+and+the+CSR+Process ------------- PR: https://git.openjdk.java.net/jdk/pull/6334 From darcy at openjdk.java.net Mon Nov 22 03:24:17 2021 From: darcy at openjdk.java.net (Joe Darcy) Date: Mon, 22 Nov 2021 03:24:17 GMT Subject: RFR: JDK-8273146: Start of release updates for JDK 19 Message-ID: The time to get JDK 19 underway draws nigh, please review this usual set of start-of-release updates, including CSRs for the javac and javax.lang.model updates: JDK-8277512: Add SourceVersion.RELEASE_19 https://bugs.openjdk.java.net/browse/JDK-8277512 JDK-8277514: Add source 19 and target 19 to javac https://bugs.openjdk.java.net/browse/JDK-8277514 Clean local tier 1 test runs for langtools, jdk, and hotspot. ------------- Commit messages: - Add --release 18 information. Good tier1 test results. - JDK-8273146: Start of release updates for JDK 19 Changes: https://git.openjdk.java.net/jdk/pull/6493/files Webrev: https://webrevs.openjdk.java.net/?repo=jdk&pr=6493&range=00 Issue: https://bugs.openjdk.java.net/browse/JDK-8273146 Stats: 3242 lines in 67 files changed: 3202 ins; 0 del; 40 mod Patch: https://git.openjdk.java.net/jdk/pull/6493.diff Fetch: git fetch https://git.openjdk.java.net/jdk pull/6493/head:pull/6493 PR: https://git.openjdk.java.net/jdk/pull/6493 From dholmes at openjdk.java.net Mon Nov 22 04:21:07 2021 From: dholmes at openjdk.java.net (David Holmes) Date: Mon, 22 Nov 2021 04:21:07 GMT Subject: RFR: JDK-8273146: Start of release updates for JDK 19 In-Reply-To: References: Message-ID: On Mon, 22 Nov 2021 03:15:51 GMT, Joe Darcy wrote: > The time to get JDK 19 underway draws nigh, please review this usual set of start-of-release updates, including CSRs for the javac and javax.lang.model updates: > > JDK-8277512: Add SourceVersion.RELEASE_19 > https://bugs.openjdk.java.net/browse/JDK-8277512 > > JDK-8277514: Add source 19 and target 19 to javac > https://bugs.openjdk.java.net/browse/JDK-8277514 > > Clean local tier 1 test runs for langtools, jdk, and hotspot. Hi Joe, I looked at all the non-sym file changes. Only one potential issue spotted. Thanks, David test/langtools/tools/javac/versions/Versions.java line 91: > 89: SEVENTEEN(false, "61.0", "17", Versions::checksrc17), > 90: EIGHTEEN(false, "62.0", "18", Versions::checksrc18), > 91: NINETEEN(false, "63.0", "19", Versions::checksrc18); checkSrc19? ------------- PR: https://git.openjdk.java.net/jdk/pull/6493 From darcy at openjdk.java.net Mon Nov 22 04:30:40 2021 From: darcy at openjdk.java.net (Joe Darcy) Date: Mon, 22 Nov 2021 04:30:40 GMT Subject: RFR: JDK-8273146: Start of release updates for JDK 19 [v2] In-Reply-To: References: Message-ID: <8ZLnKB6sQq4VRcomQVUkCk8QAD_w7dDk3REQYFEWiPk=.4d2b9116-3eab-48a9-bc94-f1df6d0becf2@github.com> On Mon, 22 Nov 2021 04:17:44 GMT, David Holmes wrote: > Hi Joe, > > I looked at all the non-sym file changes. Only one potential issue spotted. > > Thanks, David *sigh* Yep; cut and paste error -- corrected in next push. Well spotted! Thanks David ------------- PR: https://git.openjdk.java.net/jdk/pull/6493 From darcy at openjdk.java.net Mon Nov 22 04:30:38 2021 From: darcy at openjdk.java.net (Joe Darcy) Date: Mon, 22 Nov 2021 04:30:38 GMT Subject: RFR: JDK-8273146: Start of release updates for JDK 19 [v2] In-Reply-To: References: Message-ID: > The time to get JDK 19 underway draws nigh, please review this usual set of start-of-release updates, including CSRs for the javac and javax.lang.model updates: > > JDK-8277512: Add SourceVersion.RELEASE_19 > https://bugs.openjdk.java.net/browse/JDK-8277512 > > JDK-8277514: Add source 19 and target 19 to javac > https://bugs.openjdk.java.net/browse/JDK-8277514 > > Clean local tier 1 test runs for langtools, jdk, and hotspot. Joe Darcy has updated the pull request incrementally with one additional commit since the last revision: Respond to review feedback. ------------- Changes: - all: https://git.openjdk.java.net/jdk/pull/6493/files - new: https://git.openjdk.java.net/jdk/pull/6493/files/5052cd90..4da056ab Webrevs: - full: https://webrevs.openjdk.java.net/?repo=jdk&pr=6493&range=01 - incr: https://webrevs.openjdk.java.net/?repo=jdk&pr=6493&range=00-01 Stats: 1 line in 1 file changed: 0 ins; 0 del; 1 mod Patch: https://git.openjdk.java.net/jdk/pull/6493.diff Fetch: git fetch https://git.openjdk.java.net/jdk pull/6493/head:pull/6493 PR: https://git.openjdk.java.net/jdk/pull/6493 From dholmes at openjdk.java.net Mon Nov 22 04:35:06 2021 From: dholmes at openjdk.java.net (David Holmes) Date: Mon, 22 Nov 2021 04:35:06 GMT Subject: RFR: JDK-8273146: Start of release updates for JDK 19 [v2] In-Reply-To: References: Message-ID: On Mon, 22 Nov 2021 04:30:38 GMT, Joe Darcy wrote: >> The time to get JDK 19 underway draws nigh, please review this usual set of start-of-release updates, including CSRs for the javac and javax.lang.model updates: >> >> JDK-8277512: Add SourceVersion.RELEASE_19 >> https://bugs.openjdk.java.net/browse/JDK-8277512 >> >> JDK-8277514: Add source 19 and target 19 to javac >> https://bugs.openjdk.java.net/browse/JDK-8277514 >> >> Clean local tier 1 test runs for langtools, jdk, and hotspot. > > Joe Darcy has updated the pull request incrementally with one additional commit since the last revision: > > Respond to review feedback. Update looks good. ------------- Marked as reviewed by dholmes (Reviewer). PR: https://git.openjdk.java.net/jdk/pull/6493 From alanb at openjdk.java.net Mon Nov 22 08:54:10 2021 From: alanb at openjdk.java.net (Alan Bateman) Date: Mon, 22 Nov 2021 08:54:10 GMT Subject: RFR: JDK-8273146: Start of release updates for JDK 19 [v2] In-Reply-To: References: Message-ID: <6sXuvGqJ3j3740KAGE51lv-ipw4SqQauMQxOFXeXabc=.617dd6c0-4bde-47ae-903d-859fd59c1339@github.com> On Mon, 22 Nov 2021 04:30:38 GMT, Joe Darcy wrote: >> The time to get JDK 19 underway draws nigh, please review this usual set of start-of-release updates, including CSRs for the javac and javax.lang.model updates: >> >> JDK-8277512: Add SourceVersion.RELEASE_19 >> https://bugs.openjdk.java.net/browse/JDK-8277512 >> >> JDK-8277514: Add source 19 and target 19 to javac >> https://bugs.openjdk.java.net/browse/JDK-8277514 >> >> Clean local tier 1 test runs for langtools, jdk, and hotspot. > > Joe Darcy has updated the pull request incrementally with one additional commit since the last revision: > > Respond to review feedback. It's not possible to manual review the sym files but everything else looks okay. I searched the test tree for any additional tests that might need updating but didn't spot any. ------------- Marked as reviewed by alanb (Reviewer). PR: https://git.openjdk.java.net/jdk/pull/6493 From mcimadamore at openjdk.java.net Mon Nov 22 12:02:47 2021 From: mcimadamore at openjdk.java.net (Maurizio Cimadamore) Date: Mon, 22 Nov 2021 12:02:47 GMT Subject: RFR: 8275063: Implementation of Foreign Function & Memory API (Second incubator) [v25] In-Reply-To: References: Message-ID: > This PR contains the API and implementation changes for JEP-419 [1]. A more detailed description of such changes, to avoid repetitions during the review process, is included as a separate comment. > > [1] - https://openjdk.java.net/jeps/419 Maurizio Cimadamore has updated the pull request incrementally with one additional commit since the last revision: Fix javadoc issues found in CSR review ------------- Changes: - all: https://git.openjdk.java.net/jdk/pull/5907/files - new: https://git.openjdk.java.net/jdk/pull/5907/files/79d3d685..1817975f Webrevs: - full: https://webrevs.openjdk.java.net/?repo=jdk&pr=5907&range=24 - incr: https://webrevs.openjdk.java.net/?repo=jdk&pr=5907&range=23-24 Stats: 10 lines in 4 files changed: 2 ins; 6 del; 2 mod Patch: https://git.openjdk.java.net/jdk/pull/5907.diff Fetch: git fetch https://git.openjdk.java.net/jdk pull/5907/head:pull/5907 PR: https://git.openjdk.java.net/jdk/pull/5907 From mcimadamore at openjdk.java.net Mon Nov 22 12:09:30 2021 From: mcimadamore at openjdk.java.net (Maurizio Cimadamore) Date: Mon, 22 Nov 2021 12:09:30 GMT Subject: RFR: 8275063: Implementation of Foreign Function & Memory API (Second incubator) [v26] In-Reply-To: References: Message-ID: > This PR contains the API and implementation changes for JEP-419 [1]. A more detailed description of such changes, to avoid repetitions during the review process, is included as a separate comment. > > [1] - https://openjdk.java.net/jeps/419 Maurizio Cimadamore has updated the pull request with a new target base due to a merge or a rebase. The pull request now contains 35 commits: - Merge branch 'master' into JEP-419 - Fix javadoc issues found in CSR review - Adopt blessed modofier order - Merge branch 'master' into JEP-419 - Revert removal of upcall MH customization (This change caused spurious VM crashes, so reverting to baseline) - Further tweak upcall safety considerations - Clarify safety considerations for upcalls - Rename MemorySegment::ofAddressNative to MemorySegment::ofAddress (which is consistent with other restricted factories in VaList and NativeSymbol) - Streamline javadoc for package-info - * Add two new CLinker static methods to compute upcall/downcall method types * Clarify section on CLinker downcall type * Add section on CLinker safety guarantees - ... and 25 more: https://git.openjdk.java.net/jdk/compare/d427c79d...29cc6c60 ------------- Changes: https://git.openjdk.java.net/jdk/pull/5907/files Webrev: https://webrevs.openjdk.java.net/?repo=jdk&pr=5907&range=25 Stats: 14700 lines in 193 files changed: 6958 ins; 5126 del; 2616 mod Patch: https://git.openjdk.java.net/jdk/pull/5907.diff Fetch: git fetch https://git.openjdk.java.net/jdk pull/5907/head:pull/5907 PR: https://git.openjdk.java.net/jdk/pull/5907 From erikj at openjdk.java.net Mon Nov 22 14:03:13 2021 From: erikj at openjdk.java.net (Erik Joelsson) Date: Mon, 22 Nov 2021 14:03:13 GMT Subject: RFR: 8277429: Conflicting jpackage static library name In-Reply-To: References: Message-ID: On Sat, 20 Nov 2021 03:26:51 GMT, Alexey Semenyuk wrote: > 8277429: Conflicting jpackage static library name Marked as reviewed by erikj (Reviewer). ------------- PR: https://git.openjdk.java.net/jdk/pull/6485 From erikj at openjdk.java.net Mon Nov 22 14:03:20 2021 From: erikj at openjdk.java.net (Erik Joelsson) Date: Mon, 22 Nov 2021 14:03:20 GMT Subject: RFR: JDK-8273146: Start of release updates for JDK 19 [v2] In-Reply-To: References: Message-ID: On Mon, 22 Nov 2021 04:30:38 GMT, Joe Darcy wrote: >> The time to get JDK 19 underway draws nigh, please review this usual set of start-of-release updates, including CSRs for the javac and javax.lang.model updates: >> >> JDK-8277512: Add SourceVersion.RELEASE_19 >> https://bugs.openjdk.java.net/browse/JDK-8277512 >> >> JDK-8277514: Add source 19 and target 19 to javac >> https://bugs.openjdk.java.net/browse/JDK-8277514 >> >> Clean local tier 1 test runs for langtools, jdk, and hotspot. > > Joe Darcy has updated the pull request incrementally with one additional commit since the last revision: > > Respond to review feedback. Marked as reviewed by erikj (Reviewer). ------------- PR: https://git.openjdk.java.net/jdk/pull/6493 From ihse at openjdk.java.net Mon Nov 22 15:42:17 2021 From: ihse at openjdk.java.net (Magnus Ihse Bursie) Date: Mon, 22 Nov 2021 15:42:17 GMT Subject: RFR: 8277429: Conflicting jpackage static library name In-Reply-To: References: Message-ID: On Sat, 20 Nov 2021 03:26:51 GMT, Alexey Semenyuk wrote: > 8277429: Conflicting jpackage static library name How come BUILD_JPACKAGE_APPLAUNCHEREXE results in a library? It sounds like an executable... That is, I realize this patch fixes the problem as described, I just don't understand why it happens, so maybe we're solving the symptoms and not the problem. ------------- PR: https://git.openjdk.java.net/jdk/pull/6485 From erikj at openjdk.java.net Mon Nov 22 17:16:19 2021 From: erikj at openjdk.java.net (Erik Joelsson) Date: Mon, 22 Nov 2021 17:16:19 GMT Subject: RFR: 8277429: Conflicting jpackage static library name In-Reply-To: References: Message-ID: On Sat, 20 Nov 2021 03:26:51 GMT, Alexey Semenyuk wrote: > 8277429: Conflicting jpackage static library name The problem arises when running the "static-libs" build, where every library and executable is just linked into a static-lib for downstream consumers (Graal). ------------- PR: https://git.openjdk.java.net/jdk/pull/6485 From duke at openjdk.java.net Mon Nov 22 17:35:41 2021 From: duke at openjdk.java.net (Alan Hayward) Date: Mon, 22 Nov 2021 17:35:41 GMT Subject: RFR: 8277204: Implementation of JEP 8264130: PAC-RET protection for Linux/AArch64 [v7] In-Reply-To: References: Message-ID: > PAC is an optional feature in AArch64 8.3 and is compulsory in v9. One > of its uses is to protect against ROP based attacks. This is done by > signing the Link Register whenever it is stored on the stack, and > authenticating the value when it is loaded back from the stack. If an > attacker were to try to change control flow by editing the stack then > the authentication check of the Link Register will fail, causing a > segfault when the function returns. > > On a system with PAC enabled, it is expected that all applications will > be compiled with ROP protection. Fedora 33 and upwards already provide > this. By compiling for ARMv8.0, GCC and LLVM will only use the set of > PAC instructions that exist in the NOP space - on hardware without PAC, > these instructions act as NOPs, allowing backward compatibility for > negligible performance cost (2 NOPs per non-leaf function). > > Hardware is currently limited to the Apple M1 MacBooks. All testing has > been done within a Fedora Docker image. A run of SpecJVM showed no > difference to that of noise - which was surprising. > > The most important part of this patch is simply compiling using branch > protection provided by GCC/LLVM. This protects all C++ code from being > used in ROP attacks, removing all static ROP gadgets from use. > > The remainder of the patch adds ROP protection to runtime generated > code, in both stubs and compiled Java code. Attacks here are much harder > as ROP gadgets must be found dynamically at runtime. If/when AOT > compilation is added to JDK, then all stubs and compiled Java will be > susceptible ROP gadgets being found by static analysis and therefore > potentially as vulnerable as C++ code. > > There are a number of places where the VM changes control flow by > rewriting the stack or otherwise. I?ve done some analysis as to how > these could also be used for attacks (which I didn?t want to post here). > These areas can be protected ensuring the pointers to various stubs and > entry points are stored in memory as signed pointers. These changes are > simple to make (they can be reduced to a type change in common code and > a few addition sign/auth calls in the backend), but there a lot of them > and the total code change is fairly large. I?m happy to provide a few > work in progress patches. > > In order to match the security benefits of the Apple Arm64e ABI across > the whole of JDK, then all the changes mentioned above would be > required. Alan Hayward has updated the pull request with a new target base due to a merge or a rebase. The pull request now contains 13 commits: - Merge master - Merge master - Rename pauth_authenticate_or_strip_return_address - Fix windows aarch64 by restoring pauth file split - Don't keep LR live across restore_live_registers - Merge master - Document pauth functions && remove OS split - Update UseROPProtection description - Simplify branch protection configure check - 8264130: PAC-RET protection for Linux/AArch64 PAC is an optional feature in AArch64 8.3 and is compulsory in v9. One of its uses is to protect against ROP based attacks. This is done by signing the Link Register whenever it is stored on the stack, and authenticating the value when it is loaded back from the stack. If an attacker were to try to change control flow by editing the stack then the authentication check of the Link Register will fail, causing a segfault when the function returns. On a system with PAC enabled, it is expected that all applications will be compiled with ROP protection. Fedora 33 and upwards already provide this. By compiling for ARMv8.0, GCC and LLVM will only use the set of PAC instructions that exist in the NOP space - on hardware without PAC, these instructions act as NOPs, allowing backward compatibility for negligible performance cost (2 NOPs per non-leaf function). Hardware is currently limited to the Apple M1 MacBooks. All testing has been done within a Fedora Docker image. A run of SpecJVM showed no difference to that of noise - which was surprising. The most important part of this patch is simply compiling using branch protection provided by GCC/LLVM. This protects all C++ code from being used in ROP attacks, removing all static ROP gadgets from use. The remainder of the patch adds ROP protection to runtime generated code, in both stubs and compiled Java code. Attacks here are much harder as ROP gadgets must be found dynamically at runtime. If/when AOT compilation is added to JDK, then all stubs and compiled Java will be susceptible ROP gadgets being found by static analysis and therefore potentially as vulnerable as C++ code. There are a number of places where the VM changes control flow by rewriting the stack or otherwise. I?ve done some analysis as to how these could also be used for attacks (which I didn?t want to post here). These areas can be protected ensuring the pointers to various stubs and entry points are stored in memory as signed pointers. These changes are simple to make (they can be reduced to a type change in common code and a few addition sign/auth calls in the backend), but there a lot of them and the total code change is fairly large. I?m happy to provide a few work in progress patches. In order to match the security benefits of the Apple Arm64e ABI across the whole of JDK, then all the changes mentioned above would be required. - ... and 3 more: https://git.openjdk.java.net/jdk/compare/ca31ed53...280abc41 ------------- Changes: https://git.openjdk.java.net/jdk/pull/6334/files Webrev: https://webrevs.openjdk.java.net/?repo=jdk&pr=6334&range=06 Stats: 1381 lines in 25 files changed: 517 ins; 18 del; 846 mod Patch: https://git.openjdk.java.net/jdk/pull/6334.diff Fetch: git fetch https://git.openjdk.java.net/jdk pull/6334/head:pull/6334 PR: https://git.openjdk.java.net/jdk/pull/6334 From duke at openjdk.java.net Mon Nov 22 17:35:45 2021 From: duke at openjdk.java.net (Alan Hayward) Date: Mon, 22 Nov 2021 17:35:45 GMT Subject: RFR: 8277204: Implementation of JEP 8264130: PAC-RET protection for Linux/AArch64 [v6] In-Reply-To: References: Message-ID: On Tue, 16 Nov 2021 14:23:07 GMT, Alan Hayward wrote: >> PAC is an optional feature in AArch64 8.3 and is compulsory in v9. One >> of its uses is to protect against ROP based attacks. This is done by >> signing the Link Register whenever it is stored on the stack, and >> authenticating the value when it is loaded back from the stack. If an >> attacker were to try to change control flow by editing the stack then >> the authentication check of the Link Register will fail, causing a >> segfault when the function returns. >> >> On a system with PAC enabled, it is expected that all applications will >> be compiled with ROP protection. Fedora 33 and upwards already provide >> this. By compiling for ARMv8.0, GCC and LLVM will only use the set of >> PAC instructions that exist in the NOP space - on hardware without PAC, >> these instructions act as NOPs, allowing backward compatibility for >> negligible performance cost (2 NOPs per non-leaf function). >> >> Hardware is currently limited to the Apple M1 MacBooks. All testing has >> been done within a Fedora Docker image. A run of SpecJVM showed no >> difference to that of noise - which was surprising. >> >> The most important part of this patch is simply compiling using branch >> protection provided by GCC/LLVM. This protects all C++ code from being >> used in ROP attacks, removing all static ROP gadgets from use. >> >> The remainder of the patch adds ROP protection to runtime generated >> code, in both stubs and compiled Java code. Attacks here are much harder >> as ROP gadgets must be found dynamically at runtime. If/when AOT >> compilation is added to JDK, then all stubs and compiled Java will be >> susceptible ROP gadgets being found by static analysis and therefore >> potentially as vulnerable as C++ code. >> >> There are a number of places where the VM changes control flow by >> rewriting the stack or otherwise. I?ve done some analysis as to how >> these could also be used for attacks (which I didn?t want to post here). >> These areas can be protected ensuring the pointers to various stubs and >> entry points are stored in memory as signed pointers. These changes are >> simple to make (they can be reduced to a type change in common code and >> a few addition sign/auth calls in the backend), but there a lot of them >> and the total code change is fairly large. I?m happy to provide a few >> work in progress patches. >> >> In order to match the security benefits of the Apple Arm64e ABI across >> the whole of JDK, then all the changes mentioned above would be >> required. > > Alan Hayward has updated the pull request with a new target base due to a merge or a rebase. The pull request now contains 12 commits: > > - Merge master > - Rename pauth_authenticate_or_strip_return_address > - Fix windows aarch64 by restoring pauth file split > - Don't keep LR live across restore_live_registers > - Merge master > - Document pauth functions && remove OS split > - Update UseROPProtection description > - Simplify branch protection configure check > - 8264130: PAC-RET protection for Linux/AArch64 > > PAC is an optional feature in AArch64 8.3 and is compulsory in v9. One > of its uses is to protect against ROP based attacks. This is done by > signing the Link Register whenever it is stored on the stack, and > authenticating the value when it is loaded back from the stack. If an > attacker were to try to change control flow by editing the stack then > the authentication check of the Link Register will fail, causing a > segfault when the function returns. > > On a system with PAC enabled, it is expected that all applications will > be compiled with ROP protection. Fedora 33 and upwards already provide > this. By compiling for ARMv8.0, GCC and LLVM will only use the set of > PAC instructions that exist in the NOP space - on hardware without PAC, > these instructions act as NOPs, allowing backward compatibility for > negligible performance cost (2 NOPs per non-leaf function). > > Hardware is currently limited to the Apple M1 MacBooks. All testing has > been done within a Fedora Docker image. A run of SpecJVM showed no > difference to that of noise - which was surprising. > > The most important part of this patch is simply compiling using branch > protection provided by GCC/LLVM. This protects all C++ code from being > used in ROP attacks, removing all static ROP gadgets from use. > > The remainder of the patch adds ROP protection to runtime generated > code, in both stubs and compiled Java code. Attacks here are much harder > as ROP gadgets must be found dynamically at runtime. If/when AOT > compilation is added to JDK, then all stubs and compiled Java will be > susceptible ROP gadgets being found by static analysis and therefore > potentially as vulnerable as C++ code. > > There are a number of places where the VM changes control flow by > rewriting the stack or otherwise. I?ve done some analysis as to how > these could also be used for attacks (which I didn?t want to post here). > These areas can be protected ensuring the pointers to various stubs and > entry points are stored in memory as signed pointers. These changes are > simple to make (they can be reduced to a type change in common code and > a few addition sign/auth calls in the backend), but there a lot of them > and the total code change is fairly large. I?m happy to provide a few > work in progress patches. > > In order to match the security benefits of the Apple Arm64e ABI across > the whole of JDK, then all the changes mentioned above would be > required. > - Add PAC assembly instructions > - ... and 2 more: https://git.openjdk.java.net/jdk/compare/b8d33a2a...deb17a56 CSR added: https://bugs.openjdk.java.net/browse/JDK-8277543 ------------- PR: https://git.openjdk.java.net/jdk/pull/6334 From iris at openjdk.java.net Mon Nov 22 17:38:19 2021 From: iris at openjdk.java.net (Iris Clark) Date: Mon, 22 Nov 2021 17:38:19 GMT Subject: RFR: JDK-8273146: Start of release updates for JDK 19 [v2] In-Reply-To: References: Message-ID: On Mon, 22 Nov 2021 04:30:38 GMT, Joe Darcy wrote: >> The time to get JDK 19 underway draws nigh, please review this usual set of start-of-release updates, including CSRs for the javac and javax.lang.model updates: >> >> JDK-8277512: Add SourceVersion.RELEASE_19 >> https://bugs.openjdk.java.net/browse/JDK-8277512 >> >> JDK-8277514: Add source 19 and target 19 to javac >> https://bugs.openjdk.java.net/browse/JDK-8277514 >> >> Clean local tier 1 test runs for langtools, jdk, and hotspot. > > Joe Darcy has updated the pull request incrementally with one additional commit since the last revision: > > Respond to review feedback. Marked as reviewed by iris (Reviewer). ------------- PR: https://git.openjdk.java.net/jdk/pull/6493 From asemenyuk at openjdk.java.net Mon Nov 22 18:08:28 2021 From: asemenyuk at openjdk.java.net (Alexey Semenyuk) Date: Mon, 22 Nov 2021 18:08:28 GMT Subject: Integrated: 8277429: Conflicting jpackage static library name In-Reply-To: References: Message-ID: On Sat, 20 Nov 2021 03:26:51 GMT, Alexey Semenyuk wrote: > 8277429: Conflicting jpackage static library name This pull request has now been integrated. Changeset: e3911a85 Author: Alexey Semenyuk URL: https://git.openjdk.java.net/jdk/commit/e3911a8532e9b93ba5e65c613bd79864485db5ce Stats: 3 lines in 2 files changed: 0 ins; 0 del; 3 mod 8277429: Conflicting jpackage static library name Reviewed-by: almatvee, herrick, erikj ------------- PR: https://git.openjdk.java.net/jdk/pull/6485 From mikael at openjdk.java.net Mon Nov 22 21:27:07 2021 From: mikael at openjdk.java.net (Mikael Vidstedt) Date: Mon, 22 Nov 2021 21:27:07 GMT Subject: RFR: JDK-8273146: Start of release updates for JDK 19 [v2] In-Reply-To: References: Message-ID: On Mon, 22 Nov 2021 04:30:38 GMT, Joe Darcy wrote: >> The time to get JDK 19 underway draws nigh, please review this usual set of start-of-release updates, including CSRs for the javac and javax.lang.model updates: >> >> JDK-8277512: Add SourceVersion.RELEASE_19 >> https://bugs.openjdk.java.net/browse/JDK-8277512 >> >> JDK-8277514: Add source 19 and target 19 to javac >> https://bugs.openjdk.java.net/browse/JDK-8277514 >> >> Clean local tier 1 test runs for langtools, jdk, and hotspot. > > Joe Darcy has updated the pull request incrementally with one additional commit since the last revision: > > Respond to review feedback. Marked as reviewed by mikael (Reviewer). ------------- PR: https://git.openjdk.java.net/jdk/pull/6493 From darcy at openjdk.java.net Mon Nov 22 21:55:21 2021 From: darcy at openjdk.java.net (Joe Darcy) Date: Mon, 22 Nov 2021 21:55:21 GMT Subject: RFR: 8275063: Implementation of Foreign Function & Memory API (Second incubator) [v26] In-Reply-To: References: Message-ID: On Mon, 22 Nov 2021 12:09:30 GMT, Maurizio Cimadamore wrote: >> This PR contains the API and implementation changes for JEP-419 [1]. A more detailed description of such changes, to avoid repetitions during the review process, is included as a separate comment. >> >> [1] - https://openjdk.java.net/jeps/419 > > Maurizio Cimadamore has updated the pull request with a new target base due to a merge or a rebase. The pull request now contains 35 commits: > > - Merge branch 'master' into JEP-419 > - Fix javadoc issues found in CSR review > - Adopt blessed modofier order > - Merge branch 'master' into JEP-419 > - Revert removal of upcall MH customization > (This change caused spurious VM crashes, so reverting to baseline) > - Further tweak upcall safety considerations > - Clarify safety considerations for upcalls > - Rename MemorySegment::ofAddressNative to MemorySegment::ofAddress > (which is consistent with other restricted factories in VaList and NativeSymbol) > - Streamline javadoc for package-info > - * Add two new CLinker static methods to compute upcall/downcall method types > * Clarify section on CLinker downcall type > * Add section on CLinker safety guarantees > - ... and 25 more: https://git.openjdk.java.net/jdk/compare/d427c79d...29cc6c60 Marked as reviewed by darcy (Reviewer). ------------- PR: https://git.openjdk.java.net/jdk/pull/5907 From ihse at openjdk.java.net Tue Nov 23 14:32:18 2021 From: ihse at openjdk.java.net (Magnus Ihse Bursie) Date: Tue, 23 Nov 2021 14:32:18 GMT Subject: RFR: 8277429: Conflicting jpackage static library name In-Reply-To: References: Message-ID: On Sat, 20 Nov 2021 03:26:51 GMT, Alexey Semenyuk wrote: > 8277429: Conflicting jpackage static library name Ok, I'm still a bit confused. Are executables converted to static libs (with a main method?) when doing static builds? I thought this affected only dynamic libraries. Are we sure that this functionality is actually needed? That is, should we not instead stop doing static libraries of executables (and either just ignore them, or keep creating executables) when doing static builds? (I'm still not convinced this is the correct resolution, even though the change is pushed) ------------- PR: https://git.openjdk.java.net/jdk/pull/6485 From darcy at openjdk.java.net Tue Nov 23 19:23:40 2021 From: darcy at openjdk.java.net (Joe Darcy) Date: Tue, 23 Nov 2021 19:23:40 GMT Subject: RFR: JDK-8273146: Start of release updates for JDK 19 [v3] In-Reply-To: References: Message-ID: > The time to get JDK 19 underway draws nigh, please review this usual set of start-of-release updates, including CSRs for the javac and javax.lang.model updates: > > JDK-8277512: Add SourceVersion.RELEASE_19 > https://bugs.openjdk.java.net/browse/JDK-8277512 > > JDK-8277514: Add source 19 and target 19 to javac > https://bugs.openjdk.java.net/browse/JDK-8277514 > > Clean local tier 1 test runs for langtools, jdk, and hotspot. Joe Darcy has updated the pull request with a new target base due to a merge or a rebase. The incremental webrev excludes the unrelated changes brought in by the merge/rebase. The pull request contains five additional commits since the last revision: - Remove SharedSpaces options from VMDeprecatedOptions.java. - Merge branch 'master' into JDK-8273146 - Respond to review feedback. - Add --release 18 information. Good tier1 test results. - JDK-8273146: Start of release updates for JDK 19 ------------- Changes: - all: https://git.openjdk.java.net/jdk/pull/6493/files - new: https://git.openjdk.java.net/jdk/pull/6493/files/4da056ab..5fe55cbf Webrevs: - full: https://webrevs.openjdk.java.net/?repo=jdk&pr=6493&range=02 - incr: https://webrevs.openjdk.java.net/?repo=jdk&pr=6493&range=01-02 Stats: 1511 lines in 44 files changed: 1091 ins; 303 del; 117 mod Patch: https://git.openjdk.java.net/jdk/pull/6493.diff Fetch: git fetch https://git.openjdk.java.net/jdk pull/6493/head:pull/6493 PR: https://git.openjdk.java.net/jdk/pull/6493 From dholmes at openjdk.java.net Wed Nov 24 05:35:04 2021 From: dholmes at openjdk.java.net (David Holmes) Date: Wed, 24 Nov 2021 05:35:04 GMT Subject: RFR: JDK-8273146: Start of release updates for JDK 19 [v3] In-Reply-To: References: Message-ID: On Tue, 23 Nov 2021 19:23:40 GMT, Joe Darcy wrote: >> The time to get JDK 19 underway draws nigh, please review this usual set of start-of-release updates, including CSRs for the javac and javax.lang.model updates: >> >> JDK-8277512: Add SourceVersion.RELEASE_19 >> https://bugs.openjdk.java.net/browse/JDK-8277512 >> >> JDK-8277514: Add source 19 and target 19 to javac >> https://bugs.openjdk.java.net/browse/JDK-8277514 >> >> Clean local tier 1 test runs for langtools, jdk, and hotspot. > > Joe Darcy has updated the pull request with a new target base due to a merge or a rebase. The incremental webrev excludes the unrelated changes brought in by the merge/rebase. The pull request contains five additional commits since the last revision: > > - Remove SharedSpaces options from VMDeprecatedOptions.java. > - Merge branch 'master' into JDK-8273146 > - Respond to review feedback. > - Add --release 18 information. Good tier1 test results. > - JDK-8273146: Start of release updates for JDK 19 Update is good. ------------- Marked as reviewed by dholmes (Reviewer). PR: https://git.openjdk.java.net/jdk/pull/6493 From ihse at openjdk.java.net Wed Nov 24 10:40:10 2021 From: ihse at openjdk.java.net (Magnus Ihse Bursie) Date: Wed, 24 Nov 2021 10:40:10 GMT Subject: RFR: JDK-8273146: Start of release updates for JDK 19 [v3] In-Reply-To: References: Message-ID: On Tue, 23 Nov 2021 19:23:40 GMT, Joe Darcy wrote: >> The time to get JDK 19 underway draws nigh, please review this usual set of start-of-release updates, including CSRs for the javac and javax.lang.model updates: >> >> JDK-8277512: Add SourceVersion.RELEASE_19 >> https://bugs.openjdk.java.net/browse/JDK-8277512 >> >> JDK-8277514: Add source 19 and target 19 to javac >> https://bugs.openjdk.java.net/browse/JDK-8277514 >> >> Clean local tier 1 test runs for langtools, jdk, and hotspot. > > Joe Darcy has updated the pull request with a new target base due to a merge or a rebase. The incremental webrev excludes the unrelated changes brought in by the merge/rebase. The pull request contains five additional commits since the last revision: > > - Remove SharedSpaces options from VMDeprecatedOptions.java. > - Merge branch 'master' into JDK-8273146 > - Respond to review feedback. > - Add --release 18 information. Good tier1 test results. > - JDK-8273146: Start of release updates for JDK 19 Build changes look good ------------- Marked as reviewed by ihse (Reviewer). PR: https://git.openjdk.java.net/jdk/pull/6493 From jboes at openjdk.java.net Wed Nov 24 10:43:30 2021 From: jboes at openjdk.java.net (Julia Boes) Date: Wed, 24 Nov 2021 10:43:30 GMT Subject: RFR: 8277460: Add jwebserver tool Message-ID: This change introduces jwebserver, a dedicated JDK tool for the Simple Web Server. A description is provided in a follow-up comment. ------------- Commit messages: - update module summary - Merge branch 'master' into jwebserver - add version option, consolidate tests, update module and package level doc - drop "tool" in man page - Merge branch 'master' into jwebserver - update markdown - update troffl file - merge origin/jwebserver into jwebserver - initial commit - initial commit Changes: https://git.openjdk.java.net/jdk/pull/6497/files Webrev: https://webrevs.openjdk.java.net/?repo=jdk&pr=6497&range=00 Issue: https://bugs.openjdk.java.net/browse/JDK-8277460 Stats: 683 lines in 17 files changed: 603 ins; 22 del; 58 mod Patch: https://git.openjdk.java.net/jdk/pull/6497.diff Fetch: git fetch https://git.openjdk.java.net/jdk pull/6497/head:pull/6497 PR: https://git.openjdk.java.net/jdk/pull/6497 From jboes at openjdk.java.net Wed Nov 24 10:54:08 2021 From: jboes at openjdk.java.net (Julia Boes) Date: Wed, 24 Nov 2021 10:54:08 GMT Subject: RFR: 8277459: Add jwebserver tool In-Reply-To: References: Message-ID: On Mon, 22 Nov 2021 09:43:19 GMT, Julia Boes wrote: > This change introduces jwebserver, a dedicated JDK tool for the Simple Web Server. > > A description is provided in a follow-up comment. This change adds a dedicated command-line tool for the Simple Web Server, to improve convenience and approachability. The new tool `jwebserver` calls the main entry point of the jdk.httpserver module, thus hiding this implementation detail which is not hidden with `java -m jdk.httpserver`. The command-line options are identical, the only addition is support for `-version` / `--version`. A man page for `jwebserver` is added and the module and package level documentation is updated to reflect the new tool. A new internal class JWebServer is added to be able to distinguish whether the server was launched via `jwebserver` or `java -m jdk.httpserver`. The new tests are copies of the existing command-line tests. CSR: https://bugs.openjdk.java.net/browse/JDK-8277460 ------------- PR: https://git.openjdk.java.net/jdk/pull/6497 From matthias.baesken at sap.com Wed Nov 24 11:50:44 2021 From: matthias.baesken at sap.com (Baesken, Matthias) Date: Wed, 24 Nov 2021 11:50:44 +0000 Subject: Xcode version output when devkit is used Message-ID: Hi, I noticed a small issue when using an XCode devkit (--with-devkit pointing to XCode 13.1 in this case.) in the jdk/jdk build. In the configure output, we get this : configure: Xcode major version: 10 -------------------------------- * Toolchain: clang (clang/LLVM from Xcode 10.2.1) * C Compiler: Version 13.0.0 (at /tools/devkits/xcode13_1/Xcode.app/Contents/Developer/Toolchains/XcodeDefault.xctoolchain/usr/bin/clang) * C++ Compiler: Version 13.0.0 (at /tools/devkits/xcode13_1/Xcode.app/Contents/Developer/Toolchains/XcodeDefault.xctoolchain/usr/bin/clang++) -------------------------------- Looks like the output is picking up the XCode 10 from the system for "Xcode major version" and "Toolchain:" , not what the devkit provides. The system has : bash-3.2$ clang --version Apple LLVM version 10.0.1 (clang-1001.0.46.4) Target: x86_64-apple-darwin18.6.0 Thread model: posix InstalledDir: /Applications/Xcode.app/Contents/Developer/Toolchains/XcodeDefault.xctoolchain/usr/bin Should this be adjusted, or is it considered acceptable ? Best regards, Matthias From mcimadamore at openjdk.java.net Wed Nov 24 11:55:11 2021 From: mcimadamore at openjdk.java.net (Maurizio Cimadamore) Date: Wed, 24 Nov 2021 11:55:11 GMT Subject: Integrated: 8275063: Implementation of Foreign Function & Memory API (Second incubator) In-Reply-To: References: Message-ID: On Tue, 12 Oct 2021 11:16:51 GMT, Maurizio Cimadamore wrote: > This PR contains the API and implementation changes for JEP-419 [1]. A more detailed description of such changes, to avoid repetitions during the review process, is included as a separate comment. > > [1] - https://openjdk.java.net/jeps/419 This pull request has now been integrated. Changeset: 96e36071 Author: Maurizio Cimadamore URL: https://git.openjdk.java.net/jdk/commit/96e36071b63b624d56739b014b457ffc48147c4f Stats: 14700 lines in 193 files changed: 6958 ins; 5126 del; 2616 mod 8275063: Implementation of Foreign Function & Memory API (Second incubator) Reviewed-by: erikj, psandoz, jvernee, darcy ------------- PR: https://git.openjdk.java.net/jdk/pull/5907 From dfuchs at openjdk.java.net Wed Nov 24 12:16:05 2021 From: dfuchs at openjdk.java.net (Daniel Fuchs) Date: Wed, 24 Nov 2021 12:16:05 GMT Subject: RFR: 8277459: Add jwebserver tool In-Reply-To: References: Message-ID: On Mon, 22 Nov 2021 09:43:19 GMT, Julia Boes wrote: > This change introduces jwebserver, a dedicated JDK tool for the Simple Web Server. > > A description is provided in a follow-up comment. src/jdk.httpserver/share/classes/com/sun/net/httpserver/SimpleFileServer.java line 110: > 108: * > 109: *

      A simple HTTP file server implementation is provided via the > 110: * {@code jwebserver} tool. Maybe an `@toolGuide jwebserver` javadoc tag could also be added here too, at line 111? src/jdk.httpserver/share/classes/module-info.java line 30: > 28: * for running a minimal HTTP server. > 29: * > 30: *

      The com.sun.net.httpserver package defines a high-level API for building Maybe that should be "

      The {@link com.sun.net.httpserver} package defines ..." src/jdk.httpserver/share/classes/module-info.java line 34: > 32: * simple HTTP-only file server intended for testing, development and debugging > 33: * purposes. A default implementation is provided via the {@code jwebserver} tool > 34: * and the main entry point of the module, which can be invoked with should we say "which can also be invoked ..." ? src/jdk.httpserver/share/classes/sun/net/httpserver/simpleserver/SimpleFileServerImpl.java line 74: > 72: * @param writer the writer to which output should be written > 73: * @param args the command line options > 74: * @param launcher the launcher the server is started from should this be aligned like the other `@param` tags? src/jdk.httpserver/share/classes/sun/net/httpserver/simpleserver/resources/simpleserver.properties line 30: > 28: \ [-o none|info|verbose] [-h to show options] > 29: > 30: usage.jwebserver=\ should usage also show `-version, --version` in both usage strings, and in the usage summary of the man page below as well? ------------- PR: https://git.openjdk.java.net/jdk/pull/6497 From dfuchs at openjdk.java.net Wed Nov 24 12:41:05 2021 From: dfuchs at openjdk.java.net (Daniel Fuchs) Date: Wed, 24 Nov 2021 12:41:05 GMT Subject: RFR: 8277459: Add jwebserver tool In-Reply-To: References: Message-ID: On Mon, 22 Nov 2021 09:43:19 GMT, Julia Boes wrote: > This change introduces jwebserver, a dedicated JDK tool for the Simple Web Server. > > A description is provided in a follow-up comment. src/jdk.httpserver/share/classes/sun/net/httpserver/simpleserver/resources/simpleserver.properties line 45: > 43: -p, --port - Port to listen on. Default: 8000.\n\ > 44: -h, -?, --help - Print this help message.\n\ > 45: -version, --version - Print version information.\n\ Should this be " - Print version information and exit." ------------- PR: https://git.openjdk.java.net/jdk/pull/6497 From ihse at openjdk.java.net Wed Nov 24 12:52:06 2021 From: ihse at openjdk.java.net (Magnus Ihse Bursie) Date: Wed, 24 Nov 2021 12:52:06 GMT Subject: RFR: 8277459: Add jwebserver tool In-Reply-To: References: Message-ID: <-1JNI0nZ12jhXaZqXow4ETeGIfY7HRj6O99TyHnfyUY=.1fb7ee64-e60f-42f2-a491-88a74d5f49f2@github.com> On Mon, 22 Nov 2021 09:43:19 GMT, Julia Boes wrote: > This change introduces jwebserver, a dedicated JDK tool for the Simple Web Server. > > A description is provided in a follow-up comment. Build changes look good. src/jdk.httpserver/share/classes/sun/net/httpserver/simpleserver/JWebServer.java line 60: > 58: public static void main(String... args) { > 59: int ec = SimpleFileServerImpl.start(new PrintWriter(System.out, true, UTF_8), "jwebserver", args); > 60: if (ec != 0) Personally I strongly dislike multiline if-statements without braces. They are an invitation to mistakes. Please reconsider putting this on one line, or adding braces. ------------- PR: https://git.openjdk.java.net/jdk/pull/6497 From magnus.ihse.bursie at oracle.com Wed Nov 24 12:53:07 2021 From: magnus.ihse.bursie at oracle.com (Magnus Ihse Bursie) Date: Wed, 24 Nov 2021 13:53:07 +0100 Subject: Xcode version output when devkit is used In-Reply-To: References: Message-ID: <834fe8e2-9716-de30-5812-94eed1f6a0e3@oracle.com> On 2021-11-24 12:50, Baesken, Matthias wrote: > Hi, I noticed a small issue when using an XCode devkit (--with-devkit pointing to XCode 13.1 in this case.) in the jdk/jdk build. > In the configure output, we get this : > > > configure: Xcode major version: 10 > > -------------------------------- > * Toolchain: clang (clang/LLVM from Xcode 10.2.1) > > * C Compiler: Version 13.0.0 (at /tools/devkits/xcode13_1/Xcode.app/Contents/Developer/Toolchains/XcodeDefault.xctoolchain/usr/bin/clang) > * C++ Compiler: Version 13.0.0 (at /tools/devkits/xcode13_1/Xcode.app/Contents/Developer/Toolchains/XcodeDefault.xctoolchain/usr/bin/clang++) > -------------------------------- > > > Looks like the output is picking up the XCode 10 from the system for "Xcode major version" and "Toolchain:" , not what the devkit provides. > The system has : > > bash-3.2$ clang --version > Apple LLVM version 10.0.1 (clang-1001.0.46.4) > Target: x86_64-apple-darwin18.6.0 > Thread model: posix > InstalledDir: /Applications/Xcode.app/Contents/Developer/Toolchains/XcodeDefault.xctoolchain/usr/bin > > > Should this be adjusted, or is it considered acceptable ? It sounds like a bug that should get fixed. :) Let me just verify that I understand this correctly. Your specified devkit is used to actually compile the JDK, it's just in the configure script output that things look wrong? (I suspect this will also mess up version checking for clang in the makefiles. Thankfully, that kind of checking is rare.) /Magnus From matthias.baesken at sap.com Wed Nov 24 13:25:36 2021 From: matthias.baesken at sap.com (Baesken, Matthias) Date: Wed, 24 Nov 2021 13:25:36 +0000 Subject: Xcode version output when devkit is used In-Reply-To: <834fe8e2-9716-de30-5812-94eed1f6a0e3@oracle.com> References: <834fe8e2-9716-de30-5812-94eed1f6a0e3@oracle.com> Message-ID: >It sounds like a bug that should get fixed. :) > >Let me just verify that I understand this correctly. Your specified >devkit is used to actually compile the JDK, it's just in the configure >script output that things look wrong? (I suspect this will also mess up >version checking for clang in the makefiles. Thankfully, that kind of >checking is rare.) Hi Magnus, from what I see in the build log, the correct clang / clang++ compilers (and related tools) are taken from the 13.1 devkit . Best regards, Matthias From maxim.kartashev at jetbrains.com Wed Nov 24 13:55:32 2021 From: maxim.kartashev at jetbrains.com (Maxim Kartashev) Date: Wed, 24 Nov 2021 16:55:32 +0300 Subject: reproducible builds doc disagrees with code Message-ID: The description of reproducible builds in doc/building.html and doc/building.md suggests to use the --enable-reproducible-builds option, but in reality the option spelling is singular, not plural: --enable-reproducible-build. There's also a reference to --disable-reproducible-builds (again, plural) in the build scripts. I wonder which is correct? make/autoconf/jdk-options.m4 697: UTIL_ARG_ENABLE(NAME: reproducible-build, DEFAULT: $REPRODUCIBLE_BUILD_DEFAULT, 705: AC_MSG_NOTICE([On Windows it is not possible to combine --disable-reproducible-builds]) The doc commit: https://github.com/openjdk/jdk/commit/59c3dcc761ac7a9eab1517743cbb77e5526ca6f3 From jboes at openjdk.java.net Wed Nov 24 14:08:09 2021 From: jboes at openjdk.java.net (Julia Boes) Date: Wed, 24 Nov 2021 14:08:09 GMT Subject: RFR: 8277459: Add jwebserver tool In-Reply-To: References: Message-ID: On Wed, 24 Nov 2021 11:57:16 GMT, Daniel Fuchs wrote: >> This change introduces jwebserver, a dedicated JDK tool for the Simple Web Server. >> >> A description is provided in a follow-up comment. > > src/jdk.httpserver/share/classes/com/sun/net/httpserver/SimpleFileServer.java line 110: > >> 108: * >> 109: *

      A simple HTTP file server implementation is provided via the >> 110: * {@code jwebserver} tool. > > Maybe an `@toolGuide jwebserver` javadoc tag could also be added here too, at line 111? The tag can currently only be added to module and package summaries, if I read the code correctly, see `open/make/jdk/src/classes/build/tools/taglet/ToolGuide.java`. I can check if we can add class-level support at a later point. ------------- PR: https://git.openjdk.java.net/jdk/pull/6497 From youngty1997 at gmail.com Wed Nov 24 16:15:57 2021 From: youngty1997 at gmail.com (Ty Young) Date: Wed, 24 Nov 2021 10:15:57 -0600 Subject: Integrated: 8275063: Implementation of Foreign Function & Memory API (Second incubator) In-Reply-To: References: Message-ID: <14b442fb-0612-2664-6e90-e0cbdd7c0203@gmail.com> Was this code actually reviewed or even used/tested at all? All "Java" ValueLayouts have an 8-bit alignment, even if they are 2-byte, 4-byte, or 8-byte in size: https://github.com/openjdk/jdk/blob/cf7adae6333c7446048ef0364737927337631f63/src/jdk.incubator.foreign/share/classes/jdk/incubator/foreign/ValueLayout.java#L563 https://github.com/openjdk/jdk/blob/cf7adae6333c7446048ef0364737927337631f63/src/jdk.incubator.foreign/share/classes/jdk/incubator/foreign/ValueLayout.java#L573 https://github.com/openjdk/jdk/blob/cf7adae6333c7446048ef0364737927337631f63/src/jdk.incubator.foreign/share/classes/jdk/incubator/foreign/ValueLayout.java#L583 https://github.com/openjdk/jdk/blob/cf7adae6333c7446048ef0364737927337631f63/src/jdk.incubator.foreign/share/classes/jdk/incubator/foreign/ValueLayout.java#L594 https://github.com/openjdk/jdk/blob/cf7adae6333c7446048ef0364737927337631f63/src/jdk.incubator.foreign/share/classes/jdk/incubator/foreign/ValueLayout.java#L604 The doc for ValueLayout says: ?"The layout constants in this class make implicit alignment and byte-ordering assumption: all layout ?constants in this class are byte-aligned" Which is not true in the code itself. All of the docs for the constants aren't correct, either. This results in a struct that *should* have a byte alignment of 8 actually have 1: GroupLayout layout = MemoryLayout.structLayout( ??????????????? ValueLayout.JAVA_INT, ??????????????? MemoryLayout.paddingLayout(32), ??????????????? ValueLayout.JAVA_LONG); System.out.println(layout.byteAlignment()); // prints 1! Git blame shows that these issues were introduced a whole 2 months ago as part of the "beautiful" and "perfect" API changes: https://github.com/openjdk/panama-foreign/blame/c15cb9c11e03c7552d2143273959acac30969db7/src/jdk.incubator.foreign/share/classes/jdk/incubator/foreign/ValueLayout.java#L560 and no one noticed? There should probably be alignment validation checks in place to catch this. The fact that constants are declared at the bottom instead of the top probably didn't help in making this issue more noticeable. While I'm here, is passing ByteOrder.nativeOrder() to every constructor needed? On 11/24/21 5:55 AM, Maurizio Cimadamore wrote: > On Tue, 12 Oct 2021 11:16:51 GMT, Maurizio Cimadamore wrote: > >> This PR contains the API and implementation changes for JEP-419 [1]. A more detailed description of such changes, to avoid repetitions during the review process, is included as a separate comment. >> >> [1] - https://openjdk.java.net/jeps/419 > This pull request has now been integrated. > > Changeset: 96e36071 > Author: Maurizio Cimadamore > URL: https://git.openjdk.java.net/jdk/commit/96e36071b63b624d56739b014b457ffc48147c4f > Stats: 14700 lines in 193 files changed: 6958 ins; 5126 del; 2616 mod > > 8275063: Implementation of Foreign Function & Memory API (Second incubator) > > Reviewed-by: erikj, psandoz, jvernee, darcy > > ------------- > > PR: https://git.openjdk.java.net/jdk/pull/5907 From maurizio.cimadamore at oracle.com Wed Nov 24 16:25:09 2021 From: maurizio.cimadamore at oracle.com (Maurizio Cimadamore) Date: Wed, 24 Nov 2021 16:25:09 +0000 Subject: Integrated: 8275063: Implementation of Foreign Function & Memory API (Second incubator) In-Reply-To: <14b442fb-0612-2664-6e90-e0cbdd7c0203@gmail.com> References: <14b442fb-0612-2664-6e90-e0cbdd7c0203@gmail.com> Message-ID: <40cf3777-85fc-5fce-f210-049afdfa6ed5@oracle.com> I think this discussion should go either on core-libs or on panama-dev. build-dev is not the place for it. I'm addding panama-dev - and then I'll reply from there. Maurizio On 24/11/2021 16:15, Ty Young wrote: > Was this code actually reviewed or even used/tested at all? All "Java" > ValueLayouts have an 8-bit alignment, even if they are 2-byte, 4-byte, > or 8-byte in size: > > > https://github.com/openjdk/jdk/blob/cf7adae6333c7446048ef0364737927337631f63/src/jdk.incubator.foreign/share/classes/jdk/incubator/foreign/ValueLayout.java#L563 > > > https://github.com/openjdk/jdk/blob/cf7adae6333c7446048ef0364737927337631f63/src/jdk.incubator.foreign/share/classes/jdk/incubator/foreign/ValueLayout.java#L573 > > > https://github.com/openjdk/jdk/blob/cf7adae6333c7446048ef0364737927337631f63/src/jdk.incubator.foreign/share/classes/jdk/incubator/foreign/ValueLayout.java#L583 > > > https://github.com/openjdk/jdk/blob/cf7adae6333c7446048ef0364737927337631f63/src/jdk.incubator.foreign/share/classes/jdk/incubator/foreign/ValueLayout.java#L594 > > > https://github.com/openjdk/jdk/blob/cf7adae6333c7446048ef0364737927337631f63/src/jdk.incubator.foreign/share/classes/jdk/incubator/foreign/ValueLayout.java#L604 > > > > The doc for ValueLayout says: > > > ?"The layout constants in this class make implicit alignment and > byte-ordering assumption: all layout > ?constants in this class are byte-aligned" > > > Which is not true in the code itself. All of the docs for the > constants aren't correct, either. This results in a struct that > *should* have a byte alignment of 8 actually have 1: > > > GroupLayout layout = MemoryLayout.structLayout( > ??????????????? ValueLayout.JAVA_INT, > ??????????????? MemoryLayout.paddingLayout(32), > ??????????????? ValueLayout.JAVA_LONG); > > System.out.println(layout.byteAlignment()); // prints 1! > > > Git blame shows that these issues were introduced a whole 2 months ago > as part of the "beautiful" and "perfect" API changes: > > > https://github.com/openjdk/panama-foreign/blame/c15cb9c11e03c7552d2143273959acac30969db7/src/jdk.incubator.foreign/share/classes/jdk/incubator/foreign/ValueLayout.java#L560 > > > > and no one noticed? There should probably be alignment validation > checks in place to catch this. The fact that constants are declared at > the bottom instead of the top probably didn't help in making this > issue more noticeable. > > > While I'm here, is passing ByteOrder.nativeOrder() to every > constructor needed? > > > On 11/24/21 5:55 AM, Maurizio Cimadamore wrote: >> On Tue, 12 Oct 2021 11:16:51 GMT, Maurizio Cimadamore >> wrote: >> >>> This PR contains the API and implementation changes for JEP-419 [1]. >>> A more detailed description of such changes, to avoid repetitions >>> during the review process, is included as a separate comment. >>> >>> [1] - https://openjdk.java.net/jeps/419 >> This pull request has now been integrated. >> >> Changeset: 96e36071 >> Author:??? Maurizio Cimadamore >> URL: >> https://git.openjdk.java.net/jdk/commit/96e36071b63b624d56739b014b457ffc48147c4f >> Stats:???? 14700 lines in 193 files changed: 6958 ins; 5126 del; 2616 >> mod >> >> 8275063: Implementation of Foreign Function & Memory API (Second >> incubator) >> >> Reviewed-by: erikj, psandoz, jvernee, darcy >> >> ------------- >> >> PR: https://git.openjdk.java.net/jdk/pull/5907 From aleonard at openjdk.java.net Wed Nov 24 16:59:20 2021 From: aleonard at openjdk.java.net (Andrew Leonard) Date: Wed, 24 Nov 2021 16:59:20 GMT Subject: RFR: 8277762: Allow configuration of HOTSPOT_BUILD_USER Message-ID: To better allow "reproducible builds", a new configure parameter is added to set the USERNAME build variable, rather than always using the current user: --with-build-user= HOTSPOT_BUILD_USER is then set to a reproducible USERNAME if required. Signed-off-by: Andrew Leonard ------------- Commit messages: - 8277762: Allow configuration of HOTSPOT_BUILD_USER - 8277762: Allow configuration of HOTSPOT_BUILD_USER - 8277762: Allow configuration of HOTSPOT_BUILD_USER Changes: https://git.openjdk.java.net/jdk/pull/6542/files Webrev: https://webrevs.openjdk.java.net/?repo=jdk&pr=6542&range=00 Issue: https://bugs.openjdk.java.net/browse/JDK-8277762 Stats: 9 lines in 1 file changed: 6 ins; 0 del; 3 mod Patch: https://git.openjdk.java.net/jdk/pull/6542.diff Fetch: git fetch https://git.openjdk.java.net/jdk pull/6542/head:pull/6542 PR: https://git.openjdk.java.net/jdk/pull/6542 From jboes at openjdk.java.net Wed Nov 24 17:15:35 2021 From: jboes at openjdk.java.net (Julia Boes) Date: Wed, 24 Nov 2021 17:15:35 GMT Subject: RFR: 8277459: Add jwebserver tool [v2] In-Reply-To: References: Message-ID: > This change introduces jwebserver, a dedicated JDK tool for the Simple Web Server. > > A description is provided in a follow-up comment. Julia Boes has updated the pull request incrementally with one additional commit since the last revision: address PR comments: * update module and package summary * add -version / --version option to usage * fix markdown formatting ------------- Changes: - all: https://git.openjdk.java.net/jdk/pull/6497/files - new: https://git.openjdk.java.net/jdk/pull/6497/files/80b55848..513c5a1a Webrevs: - full: https://webrevs.openjdk.java.net/?repo=jdk&pr=6497&range=01 - incr: https://webrevs.openjdk.java.net/?repo=jdk&pr=6497&range=00-01 Stats: 164 lines in 11 files changed: 5 ins; 94 del; 65 mod Patch: https://git.openjdk.java.net/jdk/pull/6497.diff Fetch: git fetch https://git.openjdk.java.net/jdk pull/6497/head:pull/6497 PR: https://git.openjdk.java.net/jdk/pull/6497 From dfuchs at openjdk.java.net Wed Nov 24 17:22:04 2021 From: dfuchs at openjdk.java.net (Daniel Fuchs) Date: Wed, 24 Nov 2021 17:22:04 GMT Subject: RFR: 8277459: Add jwebserver tool [v2] In-Reply-To: References: Message-ID: On Wed, 24 Nov 2021 17:15:35 GMT, Julia Boes wrote: >> This change introduces jwebserver, a dedicated JDK tool for the Simple Web Server. >> >> A description is provided in a follow-up comment. > > Julia Boes has updated the pull request incrementally with one additional commit since the last revision: > > address PR comments: > * update module and package summary > * add -version / --version option to usage > * fix markdown formatting src/jdk.httpserver/share/classes/module-info.java line 39: > 37: *

      The {@link com.sun.net.httpserver.spi} package specifies a Service Provider > 38: * Interface (SPI) for locating HTTP server implementations based on the > 39: * com.sun.net.httpserver API. Nit: maybe this should be ` * {@code com.sun.net.httpserver} API.` ------------- PR: https://git.openjdk.java.net/jdk/pull/6497 From jboes at openjdk.java.net Wed Nov 24 17:29:40 2021 From: jboes at openjdk.java.net (Julia Boes) Date: Wed, 24 Nov 2021 17:29:40 GMT Subject: RFR: 8277459: Add jwebserver tool [v3] In-Reply-To: References: Message-ID: > This change introduces jwebserver, a dedicated JDK tool for the Simple Web Server. > > A description is provided in a follow-up comment. Julia Boes has updated the pull request incrementally with one additional commit since the last revision: fix whitespace error, add missing code tag ------------- Changes: - all: https://git.openjdk.java.net/jdk/pull/6497/files - new: https://git.openjdk.java.net/jdk/pull/6497/files/513c5a1a..60acea9d Webrevs: - full: https://webrevs.openjdk.java.net/?repo=jdk&pr=6497&range=02 - incr: https://webrevs.openjdk.java.net/?repo=jdk&pr=6497&range=01-02 Stats: 2 lines in 2 files changed: 0 ins; 1 del; 1 mod Patch: https://git.openjdk.java.net/jdk/pull/6497.diff Fetch: git fetch https://git.openjdk.java.net/jdk pull/6497/head:pull/6497 PR: https://git.openjdk.java.net/jdk/pull/6497 From erikj at openjdk.java.net Wed Nov 24 17:39:04 2021 From: erikj at openjdk.java.net (Erik Joelsson) Date: Wed, 24 Nov 2021 17:39:04 GMT Subject: RFR: 8277762: Allow configuration of HOTSPOT_BUILD_USER In-Reply-To: References: Message-ID: On Wed, 24 Nov 2021 16:49:31 GMT, Andrew Leonard wrote: > To better allow "reproducible builds", a new configure parameter is added to set the USERNAME build variable, rather than always using the current user: > --with-build-user= > HOTSPOT_BUILD_USER is then set to a reproducible USERNAME if required. > > Signed-off-by: Andrew Leonard make/autoconf/basic.m4 line 99: > 97: > 98: # Setup username (for use in adhoc version strings etc) > 99: AC_ARG_WITH([build-user], [AS_HELP_STRING([--with-build-user], This variable is currently initiated in a rather weird location given what it's used for. As it used to be a one liner, it didn't seem worth the effort to move it to a more suitable location, but when expanding it with a --with flag I think we should also move this to jdk-version.m4, which is the only place where it's currently used. I think it makes sense to treat this as part of the version variables as that's what it's used for. ------------- PR: https://git.openjdk.java.net/jdk/pull/6542 From aleonard at openjdk.java.net Wed Nov 24 18:46:12 2021 From: aleonard at openjdk.java.net (Andrew Leonard) Date: Wed, 24 Nov 2021 18:46:12 GMT Subject: RFR: 8277762: Allow configuration of HOTSPOT_BUILD_USER In-Reply-To: References: Message-ID: On Wed, 24 Nov 2021 17:36:03 GMT, Erik Joelsson wrote: >> To better allow "reproducible builds", a new configure parameter is added to set the USERNAME build variable, rather than always using the current user: >> --with-build-user= >> HOTSPOT_BUILD_USER is then set to a reproducible USERNAME if required. >> >> Signed-off-by: Andrew Leonard > > make/autoconf/basic.m4 line 99: > >> 97: >> 98: # Setup username (for use in adhoc version strings etc) >> 99: AC_ARG_WITH([build-user], [AS_HELP_STRING([--with-build-user], > > This variable is currently initiated in a rather weird location given what it's used for. As it used to be a one liner, it didn't seem worth the effort to move it to a more suitable location, but when expanding it with a --with flag I think we should also move this to jdk-version.m4, which is the only place where it's currently used. I think it makes sense to treat this as part of the version variables as that's what it's used for. it's also used in make/hotspot/lib/CompileJvm.gmk, but agree moving to jdk-versions.m4 makes sense ------------- PR: https://git.openjdk.java.net/jdk/pull/6542 From erikj at openjdk.java.net Wed Nov 24 19:02:02 2021 From: erikj at openjdk.java.net (Erik Joelsson) Date: Wed, 24 Nov 2021 19:02:02 GMT Subject: RFR: 8277762: Allow configuration of HOTSPOT_BUILD_USER In-Reply-To: References: Message-ID: On Wed, 24 Nov 2021 18:42:35 GMT, Andrew Leonard wrote: >> make/autoconf/basic.m4 line 99: >> >>> 97: >>> 98: # Setup username (for use in adhoc version strings etc) >>> 99: AC_ARG_WITH([build-user], [AS_HELP_STRING([--with-build-user], >> >> This variable is currently initiated in a rather weird location given what it's used for. As it used to be a one liner, it didn't seem worth the effort to move it to a more suitable location, but when expanding it with a --with flag I think we should also move this to jdk-version.m4, which is the only place where it's currently used. I think it makes sense to treat this as part of the version variables as that's what it's used for. > > it's also used in make/hotspot/lib/CompileJvm.gmk, but agree moving to jdk-versions.m4 makes sense Right, I meant the only place it's used within configure. ------------- PR: https://git.openjdk.java.net/jdk/pull/6542 From jboes at openjdk.java.net Thu Nov 25 16:05:18 2021 From: jboes at openjdk.java.net (Julia Boes) Date: Thu, 25 Nov 2021 16:05:18 GMT Subject: RFR: 8277847: Support toolGuide tag in class-level documentation Message-ID: This change adds support for the `@toolGuide` tag in class-level API documentation. (A use case is the jwebserver tool, where the com.sun.net.httpserver.SimpleFileServer class provides API points for extension and customization of the underlying server and its components. Linking to the tool's man page in the class-level documentation would be beneficial.) ------------- Commit messages: - initial commit Changes: https://git.openjdk.java.net/jdk/pull/6566/files Webrev: https://webrevs.openjdk.java.net/?repo=jdk&pr=6566&range=00 Issue: https://bugs.openjdk.java.net/browse/JDK-8277847 Stats: 10 lines in 1 file changed: 7 ins; 1 del; 2 mod Patch: https://git.openjdk.java.net/jdk/pull/6566.diff Fetch: git fetch https://git.openjdk.java.net/jdk pull/6566/head:pull/6566 PR: https://git.openjdk.java.net/jdk/pull/6566 From alanb at openjdk.java.net Fri Nov 26 08:09:05 2021 From: alanb at openjdk.java.net (Alan Bateman) Date: Fri, 26 Nov 2021 08:09:05 GMT Subject: RFR: 8277847: Support toolGuide tag in class-level documentation In-Reply-To: References: Message-ID: On Thu, 25 Nov 2021 15:58:58 GMT, Julia Boes wrote: > This change adds support for the `@toolGuide` tag in class-level API documentation. > > (A use case is the jwebserver tool, where the com.sun.net.httpserver.SimpleFileServer class provides API points for extension and customization of the underlying server and its components. Linking to the tool's man page in the class-level documentation would be beneficial.) Extending this to allow @toolGuide be used in class descriptions is generally useful. ------------- Marked as reviewed by alanb (Reviewer). PR: https://git.openjdk.java.net/jdk/pull/6566 From michaelm at openjdk.java.net Fri Nov 26 11:39:59 2021 From: michaelm at openjdk.java.net (Michael McMahon) Date: Fri, 26 Nov 2021 11:39:59 GMT Subject: RFR: 8277459: Add jwebserver tool [v3] In-Reply-To: References: Message-ID: On Wed, 24 Nov 2021 17:29:40 GMT, Julia Boes wrote: >> This change introduces jwebserver, a dedicated JDK tool for the Simple Web Server. >> >> A description is provided in a follow-up comment. > > Julia Boes has updated the pull request incrementally with one additional commit since the last revision: > > fix whitespace error, add missing code tag LGTM ------------- Marked as reviewed by michaelm (Reviewer). PR: https://git.openjdk.java.net/jdk/pull/6497 From dfuchs at openjdk.java.net Fri Nov 26 11:46:03 2021 From: dfuchs at openjdk.java.net (Daniel Fuchs) Date: Fri, 26 Nov 2021 11:46:03 GMT Subject: RFR: 8277847: Support toolGuide tag in class-level documentation In-Reply-To: References: Message-ID: On Thu, 25 Nov 2021 15:58:58 GMT, Julia Boes wrote: > This change adds support for the `@toolGuide` tag in class-level API documentation. > > (A use case is the jwebserver tool, where the com.sun.net.httpserver.SimpleFileServer class provides API points for extension and customization of the underlying server and its components. Linking to the tool's man page in the class-level documentation would be beneficial.) make/jdk/src/classes/build/tools/taglet/ToolGuide.java line 159: > 157: .toString() > 158: .replace('.', '/') > 159: .replaceAll("[^/]+", ".."); If the class is in a module don't you have to get one step higher to get the root? I am not familiar with this code, so I'm just reasoning by induction here - trying to match with what the case for PACKAGE seems to be doing... ------------- PR: https://git.openjdk.java.net/jdk/pull/6566 From jboes at openjdk.java.net Fri Nov 26 14:34:06 2021 From: jboes at openjdk.java.net (Julia Boes) Date: Fri, 26 Nov 2021 14:34:06 GMT Subject: RFR: 8277847: Support toolGuide tag in class-level documentation In-Reply-To: References: Message-ID: On Fri, 26 Nov 2021 11:43:25 GMT, Daniel Fuchs wrote: >> This change adds support for the `@toolGuide` tag in class-level API documentation. >> >> (A use case is the jwebserver tool, where the com.sun.net.httpserver.SimpleFileServer class provides API points for extension and customization of the underlying server and its components. Linking to the tool's man page in the class-level documentation would be beneficial.) > > make/jdk/src/classes/build/tools/taglet/ToolGuide.java line 159: > >> 157: .toString() >> 158: .replace('.', '/') >> 159: .replaceAll("[^/]+", ".."); > > If the class is in a module don't you have to get one step higher to get the root? > I am not familiar with this code, so I'm just reasoning by induction here - trying to match with what the case for PACKAGE seems to be doing... Same here, I initially applied the same pattern as for PACKAGE but that does not produce the correct path (it includes 1 ".." too much.): String typePart = te.getQualifiedName() .toString() .replace('.', '/') .replaceAll("[^/]+", ".."); return te.getEnclosingElement().getEnclosingElement() != null ? "../" + typePart : typePart; ------------- PR: https://git.openjdk.java.net/jdk/pull/6566 From dfuchs at openjdk.java.net Fri Nov 26 14:58:06 2021 From: dfuchs at openjdk.java.net (Daniel Fuchs) Date: Fri, 26 Nov 2021 14:58:06 GMT Subject: RFR: 8277847: Support toolGuide tag in class-level documentation In-Reply-To: References: Message-ID: On Fri, 26 Nov 2021 14:31:04 GMT, Julia Boes wrote: >> make/jdk/src/classes/build/tools/taglet/ToolGuide.java line 159: >> >>> 157: .toString() >>> 158: .replace('.', '/') >>> 159: .replaceAll("[^/]+", ".."); >> >> If the class is in a module don't you have to get one step higher to get the root? >> I am not familiar with this code, so I'm just reasoning by induction here - trying to match with what the case for PACKAGE seems to be doing... > > Same here, I initially applied the same pattern as for PACKAGE but that does not produce the correct path (it includes 1 ".." too much.): > > String typePart = te.getQualifiedName() > .toString() > .replace('.', '/') > .replaceAll("[^/]+", ".."); > return te.getEnclosingElement().getEnclosingElement() != null > ? "../" + typePart > : typePart; I see - class has a class name which is a file - so `foo.Bar` produces `../..` relative to the directory foo (foo/../..) - so it already has an additional `..` compared to package - where `a.b` would produce ../.. relative to *directory* b which would lead to `b/../..` and not `a/../..` which is what we need. The difference comes from the fact that `Bar` is a file whereas `b` is a directory. ------------- PR: https://git.openjdk.java.net/jdk/pull/6566 From jjg at openjdk.java.net Fri Nov 26 16:15:08 2021 From: jjg at openjdk.java.net (Jonathan Gibbons) Date: Fri, 26 Nov 2021 16:15:08 GMT Subject: RFR: 8277847: Support toolGuide tag in class-level documentation In-Reply-To: References: Message-ID: <_UtIVOdXFLXqj4M30rqvG94CevIakpvhRcItyO_cPaM=.91e3cfb7-9bf1-4f6d-a096-2e7103b2806c@github.com> On Thu, 25 Nov 2021 15:58:58 GMT, Julia Boes wrote: > This change adds support for the `@toolGuide` tag in class-level API documentation. > > (A use case is the jwebserver tool, where the com.sun.net.httpserver.SimpleFileServer class provides API points for extension and customization of the underlying server and its components. Linking to the tool's man page in the class-level documentation would be beneficial.) Marked as reviewed by jjg (Reviewer). ------------- PR: https://git.openjdk.java.net/jdk/pull/6566 From darcy at openjdk.java.net Mon Nov 29 07:14:37 2021 From: darcy at openjdk.java.net (Joe Darcy) Date: Mon, 29 Nov 2021 07:14:37 GMT Subject: RFR: JDK-8273146: Start of release updates for JDK 19 [v4] In-Reply-To: References: Message-ID: > The time to get JDK 19 underway draws nigh, please review this usual set of start-of-release updates, including CSRs for the javac and javax.lang.model updates: > > JDK-8277512: Add SourceVersion.RELEASE_19 > https://bugs.openjdk.java.net/browse/JDK-8277512 > > JDK-8277514: Add source 19 and target 19 to javac > https://bugs.openjdk.java.net/browse/JDK-8277514 > > Clean local tier 1 test runs for langtools, jdk, and hotspot. Joe Darcy has updated the pull request with a new target base due to a merge or a rebase. The incremental webrev excludes the unrelated changes brought in by the merge/rebase. The pull request contains seven additional commits since the last revision: - Merge branch 'master' into JDK-8273146 - Merge branch 'master' into JDK-8273146 - Remove SharedSpaces options from VMDeprecatedOptions.java. - Merge branch 'master' into JDK-8273146 - Respond to review feedback. - Add --release 18 information. Good tier1 test results. - JDK-8273146: Start of release updates for JDK 19 ------------- Changes: - all: https://git.openjdk.java.net/jdk/pull/6493/files - new: https://git.openjdk.java.net/jdk/pull/6493/files/5fe55cbf..5c36a2ef Webrevs: - full: https://webrevs.openjdk.java.net/?repo=jdk&pr=6493&range=03 - incr: https://webrevs.openjdk.java.net/?repo=jdk&pr=6493&range=02-03 Stats: 21630 lines in 395 files changed: 12529 ins; 5912 del; 3189 mod Patch: https://git.openjdk.java.net/jdk/pull/6493.diff Fetch: git fetch https://git.openjdk.java.net/jdk pull/6493/head:pull/6493 PR: https://git.openjdk.java.net/jdk/pull/6493 From jboes at openjdk.java.net Mon Nov 29 09:26:11 2021 From: jboes at openjdk.java.net (Julia Boes) Date: Mon, 29 Nov 2021 09:26:11 GMT Subject: Integrated: 8277847: Support toolGuide tag in class-level documentation In-Reply-To: References: Message-ID: On Thu, 25 Nov 2021 15:58:58 GMT, Julia Boes wrote: > This change adds support for the `@toolGuide` tag in class-level API documentation. > > (A use case is the jwebserver tool, where the com.sun.net.httpserver.SimpleFileServer class provides API points for extension and customization of the underlying server and its components. Linking to the tool's man page in the class-level documentation would be beneficial.) This pull request has now been integrated. Changeset: e3e5908d Author: Julia Boes URL: https://git.openjdk.java.net/jdk/commit/e3e5908d0d385ef908ba9752908aaf28b4b3e4f4 Stats: 10 lines in 1 file changed: 7 ins; 1 del; 2 mod 8277847: Support toolGuide tag in class-level documentation Reviewed-by: alanb, jjg ------------- PR: https://git.openjdk.java.net/jdk/pull/6566 From jboes at openjdk.java.net Mon Nov 29 12:02:36 2021 From: jboes at openjdk.java.net (Julia Boes) Date: Mon, 29 Nov 2021 12:02:36 GMT Subject: RFR: 8277459: Add jwebserver tool [v4] In-Reply-To: References: Message-ID: > This change introduces jwebserver, a dedicated JDK tool for the Simple Web Server. > > A description is provided in a follow-up comment. Julia Boes has updated the pull request with a new target base due to a merge or a rebase. The pull request now contains 14 commits: - add toolGuide tag to SimpleFileServer class-level documentation - Merge branch 'master' into jwebserver - fix whitespace error, add missing code tag - address PR comments: * update module and package summary * add -version / --version option to usage * fix markdown formatting - update module summary - Merge branch 'master' into jwebserver - add version option, consolidate tests, update module and package level doc - drop "tool" in man page - Merge branch 'master' into jwebserver - update markdown - ... and 4 more: https://git.openjdk.java.net/jdk/compare/37de4422...dc1ac2dd ------------- Changes: https://git.openjdk.java.net/jdk/pull/6497/files Webrev: https://webrevs.openjdk.java.net/?repo=jdk&pr=6497&range=03 Stats: 680 lines in 17 files changed: 585 ins; 27 del; 68 mod Patch: https://git.openjdk.java.net/jdk/pull/6497.diff Fetch: git fetch https://git.openjdk.java.net/jdk pull/6497/head:pull/6497 PR: https://git.openjdk.java.net/jdk/pull/6497 From jboes at openjdk.java.net Mon Nov 29 12:06:07 2021 From: jboes at openjdk.java.net (Julia Boes) Date: Mon, 29 Nov 2021 12:06:07 GMT Subject: RFR: 8277459: Add jwebserver tool [v4] In-Reply-To: References: Message-ID: On Wed, 24 Nov 2021 14:05:03 GMT, Julia Boes wrote: >> src/jdk.httpserver/share/classes/com/sun/net/httpserver/SimpleFileServer.java line 110: >> >>> 108: * >>> 109: *

      A simple HTTP file server implementation is provided via the >>> 110: * {@code jwebserver} tool. >> >> Maybe an `@toolGuide jwebserver` javadoc tag could also be added here too, at line 111? > > The tag can currently only be added to module and package summaries, if I read the code correctly, see `open/make/jdk/src/classes/build/tools/taglet/ToolGuide.java`. I can check if we can add class-level support at a later point. Support for `@toolGuide` in class-level documentation was added to the mainline with https://github.com/openjdk/jdk/commit/e3e5908d0d385ef908ba9752908aaf28b4b3e4f4 ------------- PR: https://git.openjdk.java.net/jdk/pull/6497 From dfuchs at openjdk.java.net Mon Nov 29 12:45:06 2021 From: dfuchs at openjdk.java.net (Daniel Fuchs) Date: Mon, 29 Nov 2021 12:45:06 GMT Subject: RFR: 8277459: Add jwebserver tool [v3] In-Reply-To: References: Message-ID: On Wed, 24 Nov 2021 17:29:40 GMT, Julia Boes wrote: >> This change introduces jwebserver, a dedicated JDK tool for the Simple Web Server. >> >> A description is provided in a follow-up comment. > > Julia Boes has updated the pull request incrementally with one additional commit since the last revision: > > fix whitespace error, add missing code tag src/jdk.httpserver/share/classes/sun/net/httpserver/simpleserver/Main.java line 63: > 61: System.exit(ec); > 62: } // otherwise the server has been started successfully and runs in > 63: // another non-daemon thread. I wonder if we should add: // another non-daemon thread, or -h or -version have been passed and // the main thread will simply exit normally. ------------- PR: https://git.openjdk.java.net/jdk/pull/6497 From jboes at openjdk.java.net Mon Nov 29 14:00:37 2021 From: jboes at openjdk.java.net (Julia Boes) Date: Mon, 29 Nov 2021 14:00:37 GMT Subject: RFR: 8277459: Add jwebserver tool [v5] In-Reply-To: References: Message-ID: > This change introduces jwebserver, a dedicated JDK tool for the Simple Web Server. > > A description is provided in a follow-up comment. Julia Boes has updated the pull request incrementally with one additional commit since the last revision: update comment about exit code ------------- Changes: - all: https://git.openjdk.java.net/jdk/pull/6497/files - new: https://git.openjdk.java.net/jdk/pull/6497/files/dc1ac2dd..81499efa Webrevs: - full: https://webrevs.openjdk.java.net/?repo=jdk&pr=6497&range=04 - incr: https://webrevs.openjdk.java.net/?repo=jdk&pr=6497&range=03-04 Stats: 6 lines in 2 files changed: 2 ins; 0 del; 4 mod Patch: https://git.openjdk.java.net/jdk/pull/6497.diff Fetch: git fetch https://git.openjdk.java.net/jdk pull/6497/head:pull/6497 PR: https://git.openjdk.java.net/jdk/pull/6497 From jboes at openjdk.java.net Mon Nov 29 14:00:44 2021 From: jboes at openjdk.java.net (Julia Boes) Date: Mon, 29 Nov 2021 14:00:44 GMT Subject: RFR: 8277459: Add jwebserver tool [v3] In-Reply-To: References: Message-ID: On Thu, 25 Nov 2021 10:09:21 GMT, Daniel Fuchs wrote: >> Julia Boes has updated the pull request incrementally with one additional commit since the last revision: >> >> fix whitespace error, add missing code tag > > src/jdk.httpserver/share/classes/sun/net/httpserver/simpleserver/Main.java line 63: > >> 61: System.exit(ec); >> 62: } // otherwise the server has been started successfully and runs in >> 63: // another non-daemon thread. > > I wonder if we should add: > > > // another non-daemon thread, or -h or -version have been passed and > // the main thread will simply exit normally. Right, good to mention those two cases. ------------- PR: https://git.openjdk.java.net/jdk/pull/6497 From dfuchs at openjdk.java.net Mon Nov 29 14:00:40 2021 From: dfuchs at openjdk.java.net (Daniel Fuchs) Date: Mon, 29 Nov 2021 14:00:40 GMT Subject: RFR: 8277459: Add jwebserver tool [v5] In-Reply-To: References: Message-ID: On Mon, 29 Nov 2021 13:57:13 GMT, Julia Boes wrote: >> This change introduces jwebserver, a dedicated JDK tool for the Simple Web Server. >> >> A description is provided in a follow-up comment. > > Julia Boes has updated the pull request incrementally with one additional commit since the last revision: > > update comment about exit code LGTM! ------------- Marked as reviewed by dfuchs (Reviewer). PR: https://git.openjdk.java.net/jdk/pull/6497 From duke at openjdk.java.net Mon Nov 29 16:26:23 2021 From: duke at openjdk.java.net (Pavel Kharskii) Date: Mon, 29 Nov 2021 16:26:23 GMT Subject: RFR: 8277944: JDK 18 - update GA Release Date Message-ID: 8277944: JDK 18 - update GA Release Date ------------- Commit messages: - 8277944: JDK 18 - update GA Release Date Changes: https://git.openjdk.java.net/jdk/pull/6596/files Webrev: https://webrevs.openjdk.java.net/?repo=jdk&pr=6596&range=00 Issue: https://bugs.openjdk.java.net/browse/JDK-8277944 Stats: 1 line in 1 file changed: 0 ins; 0 del; 1 mod Patch: https://git.openjdk.java.net/jdk/pull/6596.diff Fetch: git fetch https://git.openjdk.java.net/jdk pull/6596/head:pull/6596 PR: https://git.openjdk.java.net/jdk/pull/6596 From coffeys at openjdk.java.net Mon Nov 29 16:41:01 2021 From: coffeys at openjdk.java.net (Sean Coffey) Date: Mon, 29 Nov 2021 16:41:01 GMT Subject: RFR: 8277944: JDK 18 - update GA Release Date In-Reply-To: References: Message-ID: On Mon, 29 Nov 2021 16:19:12 GMT, Pavel Kharskii wrote: > 8277944: JDK 18 - update GA Release Date Marked as reviewed by coffeys (Reviewer). ------------- PR: https://git.openjdk.java.net/jdk/pull/6596 From duke at openjdk.java.net Mon Nov 29 16:58:07 2021 From: duke at openjdk.java.net (Pavel Kharskii) Date: Mon, 29 Nov 2021 16:58:07 GMT Subject: Integrated: 8277944: JDK 18 - update GA Release Date In-Reply-To: References: Message-ID: On Mon, 29 Nov 2021 16:19:12 GMT, Pavel Kharskii wrote: > 8277944: JDK 18 - update GA Release Date This pull request has now been integrated. Changeset: 825e633e Author: Pavel Kharskii Committer: Sean Coffey URL: https://git.openjdk.java.net/jdk/commit/825e633e71ca942fe88c509e7f951ff8903c45cf Stats: 1 line in 1 file changed: 0 ins; 0 del; 1 mod 8277944: JDK 18 - update GA Release Date Reviewed-by: coffeys ------------- PR: https://git.openjdk.java.net/jdk/pull/6596 From jboes at openjdk.java.net Mon Nov 29 17:00:24 2021 From: jboes at openjdk.java.net (Julia Boes) Date: Mon, 29 Nov 2021 17:00:24 GMT Subject: RFR: 8277459: Add jwebserver tool [v6] In-Reply-To: References: Message-ID: > This change introduces jwebserver, a dedicated JDK tool for the Simple Web Server. > > A description is provided in a follow-up comment. Julia Boes has updated the pull request with a new target base due to a merge or a rebase. The pull request now contains 16 commits: - Merge branch 'master' into jwebserver - update comment about exit code - add toolGuide tag to SimpleFileServer class-level documentation - Merge branch 'master' into jwebserver - fix whitespace error, add missing code tag - address PR comments: * update module and package summary * add -version / --version option to usage * fix markdown formatting - update module summary - Merge branch 'master' into jwebserver - add version option, consolidate tests, update module and package level doc - drop "tool" in man page - ... and 6 more: https://git.openjdk.java.net/jdk/compare/825e633e...936c3d4a ------------- Changes: https://git.openjdk.java.net/jdk/pull/6497/files Webrev: https://webrevs.openjdk.java.net/?repo=jdk&pr=6497&range=05 Stats: 682 lines in 17 files changed: 587 ins; 27 del; 68 mod Patch: https://git.openjdk.java.net/jdk/pull/6497.diff Fetch: git fetch https://git.openjdk.java.net/jdk pull/6497/head:pull/6497 PR: https://git.openjdk.java.net/jdk/pull/6497 From darcy at openjdk.java.net Mon Nov 29 18:38:35 2021 From: darcy at openjdk.java.net (Joe Darcy) Date: Mon, 29 Nov 2021 18:38:35 GMT Subject: RFR: JDK-8273146: Start of release updates for JDK 19 [v5] In-Reply-To: References: Message-ID: > The time to get JDK 19 underway draws nigh, please review this usual set of start-of-release updates, including CSRs for the javac and javax.lang.model updates: > > JDK-8277512: Add SourceVersion.RELEASE_19 > https://bugs.openjdk.java.net/browse/JDK-8277512 > > JDK-8277514: Add source 19 and target 19 to javac > https://bugs.openjdk.java.net/browse/JDK-8277514 > > Clean local tier 1 test runs for langtools, jdk, and hotspot. Joe Darcy has updated the pull request incrementally with one additional commit since the last revision: Update symbol information for JDK 18 b25. ------------- Changes: - all: https://git.openjdk.java.net/jdk/pull/6493/files - new: https://git.openjdk.java.net/jdk/pull/6493/files/5c36a2ef..53e46dec Webrevs: - full: https://webrevs.openjdk.java.net/?repo=jdk&pr=6493&range=04 - incr: https://webrevs.openjdk.java.net/?repo=jdk&pr=6493&range=03-04 Stats: 85 lines in 5 files changed: 62 ins; 22 del; 1 mod Patch: https://git.openjdk.java.net/jdk/pull/6493.diff Fetch: git fetch https://git.openjdk.java.net/jdk pull/6493/head:pull/6493 PR: https://git.openjdk.java.net/jdk/pull/6493 From vromero at openjdk.java.net Mon Nov 29 19:13:29 2021 From: vromero at openjdk.java.net (Vicente Romero) Date: Mon, 29 Nov 2021 19:13:29 GMT Subject: RFR: 8275771: JDK source code contains redundant boolean operations in jdk.compiler and langtools Message-ID: Hi, Please review this PR which is basically rewriting some redundant boolean expressions in the compiler. TIA ------------- Commit messages: - 8275771: JDK source code contains redundant boolean operations in jdk.compiler and langtools Changes: https://git.openjdk.java.net/jdk/pull/6599/files Webrev: https://webrevs.openjdk.java.net/?repo=jdk&pr=6599&range=00 Issue: https://bugs.openjdk.java.net/browse/JDK-8275771 Stats: 6 lines in 2 files changed: 0 ins; 0 del; 6 mod Patch: https://git.openjdk.java.net/jdk/pull/6599.diff Fetch: git fetch https://git.openjdk.java.net/jdk pull/6599/head:pull/6599 PR: https://git.openjdk.java.net/jdk/pull/6599 From iris at openjdk.java.net Mon Nov 29 19:53:10 2021 From: iris at openjdk.java.net (Iris Clark) Date: Mon, 29 Nov 2021 19:53:10 GMT Subject: RFR: JDK-8273146: Start of release updates for JDK 19 [v5] In-Reply-To: References: Message-ID: <3IitEJDF2bwb41CY8BDHGMwrAAcU3ZfZKxOpYfOTiGQ=.89646199-abb3-4550-a4c6-edda6df8c32c@github.com> On Mon, 29 Nov 2021 18:38:35 GMT, Joe Darcy wrote: >> The time to get JDK 19 underway draws nigh, please review this usual set of start-of-release updates, including CSRs for the javac and javax.lang.model updates: >> >> JDK-8277512: Add SourceVersion.RELEASE_19 >> https://bugs.openjdk.java.net/browse/JDK-8277512 >> >> JDK-8277514: Add source 19 and target 19 to javac >> https://bugs.openjdk.java.net/browse/JDK-8277514 >> >> Clean local tier 1 test runs for langtools, jdk, and hotspot. > > Joe Darcy has updated the pull request incrementally with one additional commit since the last revision: > > Update symbol information for JDK 18 b25. Marked as reviewed by iris (Reviewer). ------------- PR: https://git.openjdk.java.net/jdk/pull/6493 From darcy at openjdk.java.net Mon Nov 29 19:57:08 2021 From: darcy at openjdk.java.net (Joe Darcy) Date: Mon, 29 Nov 2021 19:57:08 GMT Subject: RFR: 8275771: JDK source code contains redundant boolean operations in jdk.compiler and langtools In-Reply-To: References: Message-ID: On Mon, 29 Nov 2021 19:05:59 GMT, Vicente Romero wrote: > Hi, > > Please review this PR which is basically rewriting some redundant boolean expressions in the compiler. > > TIA make/langtools/tools/compileproperties/CompileProperties.java line 187: > 185: } > 186: if ( ok && contents != null ) { > 187: String tokens[] = (new String(contents)).split("\\s+"); So the intended composite predicate here is thought to be ok == true && contents != null which is equivalent to ok && contents != null. The semantics of the current code are equivalent to just contents != null right? src/jdk.compiler/share/classes/com/sun/tools/javac/comp/Resolve.java line 1316: > 1314: public void visitReference(JCMemberReference tree) { > 1315: if (sRet.hasTag(VOID)) { > 1316: result = true; Isn't the equivalent statement to result &= true just result ? ------------- PR: https://git.openjdk.java.net/jdk/pull/6599 From duke at openjdk.java.net Tue Nov 30 00:12:09 2021 From: duke at openjdk.java.net (PROgrm_JARvis) Date: Tue, 30 Nov 2021 00:12:09 GMT Subject: RFR: 8275771: JDK source code contains redundant boolean operations in jdk.compiler and langtools In-Reply-To: References: Message-ID: On Mon, 29 Nov 2021 19:05:59 GMT, Vicente Romero wrote: > Hi, > > Please review this PR which is basically rewriting some redundant boolean expressions in the compiler. > > TIA Changes requested by JarvisCraft at github.com (no known OpenJDK username). src/jdk.compiler/share/classes/com/sun/tools/javac/comp/Resolve.java line 1338: > 1336: public void visitLambda(JCLambda tree) { > 1337: if (sRet.hasTag(VOID)) { > 1338: result = true; Same as above, the change is changing the semantics: original code keeps the value of `result` while the new one sets it to `true`. ------------- PR: https://git.openjdk.java.net/jdk/pull/6599 From duke at openjdk.java.net Tue Nov 30 00:12:11 2021 From: duke at openjdk.java.net (PROgrm_JARvis) Date: Tue, 30 Nov 2021 00:12:11 GMT Subject: RFR: 8275771: JDK source code contains redundant boolean operations in jdk.compiler and langtools In-Reply-To: References: Message-ID: On Mon, 29 Nov 2021 19:54:02 GMT, Joe Darcy wrote: >> Hi, >> >> Please review this PR which is basically rewriting some redundant boolean expressions in the compiler. >> >> TIA > > make/langtools/tools/compileproperties/CompileProperties.java line 187: > >> 185: } >> 186: if ( ok && contents != null ) { >> 187: String tokens[] = (new String(contents)).split("\\s+"); > > So the intended composite predicate here is thought to be > ok == true && contents != null > which is equivalent to > ok && contents != null. > The semantics of the current code are equivalent to just > contents != null > right? The semantics is definitely changed (considering the first branch was always true originally) but it may be the original implementation being incorrect, > src/jdk.compiler/share/classes/com/sun/tools/javac/comp/Resolve.java line 1316: > >> 1314: public void visitReference(JCMemberReference tree) { >> 1315: if (sRet.hasTag(VOID)) { >> 1316: result = true; > > Isn't the equivalent statement to > result &= true > just > result > ? Agreeing with jddarcy: `a & true === a` so current code just keeps the value of `result` while the new one sets `result` to `true`. ------------- PR: https://git.openjdk.java.net/jdk/pull/6599 From duke at openjdk.java.net Tue Nov 30 09:55:37 2021 From: duke at openjdk.java.net (Maxim Kartashev) Date: Tue, 30 Nov 2021 09:55:37 GMT Subject: RFR: 8277977: Incorrect references to --enable-reproducible-builds in docs Message-ID: <7k02muEGe6J8DNooPbaX3UR6fS3p4bv_9HOUYve-qiY=.fa13aed1-fc4f-4f3f-8bc0-f327f0bdb0d2@github.com> Corrects spelling mistakes in the documentation and one configure script message. ------------- Commit messages: - 8277977: Incorrect references to --enable-reproducible-builds in docs Changes: https://git.openjdk.java.net/jdk/pull/6609/files Webrev: https://webrevs.openjdk.java.net/?repo=jdk&pr=6609&range=00 Issue: https://bugs.openjdk.java.net/browse/JDK-8277977 Stats: 3 lines in 3 files changed: 0 ins; 0 del; 3 mod Patch: https://git.openjdk.java.net/jdk/pull/6609.diff Fetch: git fetch https://git.openjdk.java.net/jdk pull/6609/head:pull/6609 PR: https://git.openjdk.java.net/jdk/pull/6609 From asotona at openjdk.java.net Tue Nov 30 11:03:25 2021 From: asotona at openjdk.java.net (Adam Sotona) Date: Tue, 30 Nov 2021 11:03:25 GMT Subject: RFR: 8264485: build.tools.depend.Depend.toString(byte[]) creates malformed hex strings Message-ID: make/jdk/src/classes/build/tools/depend/Depend.java method toString(byte[]) constructs hex string out of the given byte array. Actual implementation is using custom conversion code, which does not pad byte values <16 with leading zero. Resulting hex string is invalid and for example sequence of bytes 1 and 0 generates the same hex string as a single byte 16. Proposed patch is delegating hex conversion to java.util.HexFormat instead. Thanks, Adam ------------- Commit messages: - 8264485: build.tools.depend.Depend.toString(byte[]) creates malformed hex strings Changes: https://git.openjdk.java.net/jdk/pull/6610/files Webrev: https://webrevs.openjdk.java.net/?repo=jdk&pr=6610&range=00 Issue: https://bugs.openjdk.java.net/browse/JDK-8264485 Stats: 8 lines in 1 file changed: 1 ins; 6 del; 1 mod Patch: https://git.openjdk.java.net/jdk/pull/6610.diff Fetch: git fetch https://git.openjdk.java.net/jdk pull/6610/head:pull/6610 PR: https://git.openjdk.java.net/jdk/pull/6610 From magnus.ihse.bursie at oracle.com Tue Nov 30 11:25:17 2021 From: magnus.ihse.bursie at oracle.com (Magnus Ihse Bursie) Date: Tue, 30 Nov 2021 12:25:17 +0100 Subject: Result: New Build Group Member: Aleksey Shipilev In-Reply-To: <6911ff3e-67d5-0991-d0a0-08a06b1b09b2@oracle.com> References: <6911ff3e-67d5-0991-d0a0-08a06b1b09b2@oracle.com> Message-ID: The vote for Aleksey Shipilev [1] is now closed. Yes: 4 Veto: 0 Abstain: 0 According to the Bylaws definition of Lazy Consensus, this is sufficient to approve the nomination. /Magnus [1] https://mail.openjdk.java.net/pipermail/build-dev/2021-November/032592.html On 2021-11-10 15:06, Magnus Ihse Bursie wrote: > I hereby nominate Aleksey Shipilev (shade) to Membership in the Build > Group. > > Aleksey is a long standing member of the OpenJDK Members Group and the > Hotspot > Group. He is a prolific contributor, and has committed more than 600 > changes to > the JDK over the years. Lately, Aleksey has made a number of > improvement to the > build system, which is indicated by the more than 140 commits made to > files in > the "make" directory since 2018. > > Votes are due by 24 November 2021, 16:00 CET. > > Only current Members of the Build Group [2] are eligible > to vote on this nomination.? Votes must be cast in the open by > replying to this mailing list > > For Lazy Consensus voting instructions, see [3]. > > /Magnus > > [1] > https://github.com/openjdk/jdk/search?q=author-name%3A%22Aleksey+Shipilev%22&type=commits > [2]http://openjdk.java.net/census#build > [3]http://openjdk.java.net/groups/#member-vote > From ihse at openjdk.java.net Tue Nov 30 12:47:12 2021 From: ihse at openjdk.java.net (Magnus Ihse Bursie) Date: Tue, 30 Nov 2021 12:47:12 GMT Subject: RFR: 8277977: Incorrect references to --enable-reproducible-builds in docs In-Reply-To: <7k02muEGe6J8DNooPbaX3UR6fS3p4bv_9HOUYve-qiY=.fa13aed1-fc4f-4f3f-8bc0-f327f0bdb0d2@github.com> References: <7k02muEGe6J8DNooPbaX3UR6fS3p4bv_9HOUYve-qiY=.fa13aed1-fc4f-4f3f-8bc0-f327f0bdb0d2@github.com> Message-ID: On Tue, 30 Nov 2021 09:47:31 GMT, Maxim Kartashev wrote: > Corrects spelling mistakes in the documentation and one configure script message. Thanks for fixing these typos! ------------- Marked as reviewed by ihse (Reviewer). PR: https://git.openjdk.java.net/jdk/pull/6609 From jlahoda at openjdk.java.net Tue Nov 30 12:50:15 2021 From: jlahoda at openjdk.java.net (Jan Lahoda) Date: Tue, 30 Nov 2021 12:50:15 GMT Subject: RFR: 8264485: build.tools.depend.Depend.toString(byte[]) creates malformed hex strings In-Reply-To: References: Message-ID: <5ynIrkuIk6LVgozR72777LTQy56GECHCRg_ZBwtENTM=.dd687d32-859c-4395-9404-9f28211856d3@github.com> On Tue, 30 Nov 2021 10:46:12 GMT, Adam Sotona wrote: > make/jdk/src/classes/build/tools/depend/Depend.java method toString(byte[]) constructs hex string out of the given byte array. > Actual implementation is using custom conversion code, which does not pad byte values <16 with leading zero. > Resulting hex string is invalid and for example sequence of bytes 1 and 0 generates the same hex string as a single byte 16. > > Proposed patch is delegating hex conversion to java.util.HexFormat instead. > > Thanks, > Adam Looks OK. ------------- Marked as reviewed by jlahoda (Reviewer). PR: https://git.openjdk.java.net/jdk/pull/6610 From ihse at openjdk.java.net Tue Nov 30 12:52:14 2021 From: ihse at openjdk.java.net (Magnus Ihse Bursie) Date: Tue, 30 Nov 2021 12:52:14 GMT Subject: RFR: 8277977: Incorrect references to --enable-reproducible-builds in docs In-Reply-To: <7k02muEGe6J8DNooPbaX3UR6fS3p4bv_9HOUYve-qiY=.fa13aed1-fc4f-4f3f-8bc0-f327f0bdb0d2@github.com> References: <7k02muEGe6J8DNooPbaX3UR6fS3p4bv_9HOUYve-qiY=.fa13aed1-fc4f-4f3f-8bc0-f327f0bdb0d2@github.com> Message-ID: <5J2jz7hRLDYAjO0tmjDPVwbAW-ZyyWCnfMKMZ80qRfI=.d60d2475-691c-4159-b284-8783cc8cd92b@github.com> On Tue, 30 Nov 2021 09:47:31 GMT, Maxim Kartashev wrote: > Corrects spelling mistakes in the documentation and one configure script message. @mkartashev I can sponsor this, once you have done `/integrate`. ------------- PR: https://git.openjdk.java.net/jdk/pull/6609 From ihse at openjdk.java.net Tue Nov 30 13:05:14 2021 From: ihse at openjdk.java.net (Magnus Ihse Bursie) Date: Tue, 30 Nov 2021 13:05:14 GMT Subject: RFR: 8274980: Improve adhoc build version strings [v3] In-Reply-To: <4oFQRaUoo0BXptrkkDta-lAH3iapzfCl7UG7LTSfXOA=.dc5fda6f-d012-4ac7-ad48-8de906f398d6@github.com> References: <_-ASswZ8ksJQA9q6BpdLwCZWXfT_Fbq4o6FHgnC9pbY=.bf2fa1fb-aa32-400e-b257-263b2348e563@github.com> <4oFQRaUoo0BXptrkkDta-lAH3iapzfCl7UG7LTSfXOA=.dc5fda6f-d012-4ac7-ad48-8de906f398d6@github.com> Message-ID: On Thu, 28 Oct 2021 17:01:46 GMT, Magnus Ihse Bursie wrote: >> Current adhoc version build strings are not ideal. Some of the problems: >> * A build number of "0" is inserted, which make the version string look like it's an official build, at least when not reading carefully >> * The version string gives little indication on what source code the build was based >> >> Also, an error was discovered in how the build system generates version strings without a build number (since this basically never happened before). A version without a build number, and without a PRE value, but with an OPT value, should have a "+-" separator, according to JEP 223. While this was correctly handled, the "+-" separator was also applied to versions without a build, but with both PRE and OPT. In this case, the "+" should be omitted, according to JEP 223. This bug is fixed by this commit. >> >> Ideally, the adhoc version string should include something along the lines of the output of `git describe`. However, since the version string is set at configure time, not build time, this will almost immediately become misleading. :-( A substitute is proposed in this patch, where the branch name is included in the OPT string (if it's not `master`). >> >> Finally, this patch fixes hotspot so it can properly build without a build number. This was the last blocker for why the build system always required the "0" as build number before. To facilitate interoperability with external tools (like jib) that still sets build number to "0" to really signal "no build number", a build number exactly matching "0" will be interpreted as having no build number. > > Magnus Ihse Bursie has updated the pull request incrementally with one additional commit since the last revision: > > IS_EMPTY define did not work on msvc Give me a bit more time, bot... ------------- PR: https://git.openjdk.java.net/jdk/pull/6152 From duke at openjdk.java.net Tue Nov 30 13:05:18 2021 From: duke at openjdk.java.net (Maxim Kartashev) Date: Tue, 30 Nov 2021 13:05:18 GMT Subject: Integrated: 8277977: Incorrect references to --enable-reproducible-builds in docs In-Reply-To: <7k02muEGe6J8DNooPbaX3UR6fS3p4bv_9HOUYve-qiY=.fa13aed1-fc4f-4f3f-8bc0-f327f0bdb0d2@github.com> References: <7k02muEGe6J8DNooPbaX3UR6fS3p4bv_9HOUYve-qiY=.fa13aed1-fc4f-4f3f-8bc0-f327f0bdb0d2@github.com> Message-ID: <6UbimBMQ_lA8AmAUlSgBjC0LkSdhiuGB-SbLyV2MO3s=.d112ab13-e359-4657-9803-7139e3c73f03@github.com> On Tue, 30 Nov 2021 09:47:31 GMT, Maxim Kartashev wrote: > Corrects spelling mistakes in the documentation and one configure script message. This pull request has now been integrated. Changeset: 01cefc94 Author: Maxim Kartashev Committer: Magnus Ihse Bursie URL: https://git.openjdk.java.net/jdk/commit/01cefc94c766b87d426cf1dec89a8867454faf0e Stats: 3 lines in 3 files changed: 0 ins; 0 del; 3 mod 8277977: Incorrect references to --enable-reproducible-builds in docs Reviewed-by: ihse ------------- PR: https://git.openjdk.java.net/jdk/pull/6609 From ihse at openjdk.java.net Tue Nov 30 13:07:09 2021 From: ihse at openjdk.java.net (Magnus Ihse Bursie) Date: Tue, 30 Nov 2021 13:07:09 GMT Subject: RFR: 8277459: Add jwebserver tool [v6] In-Reply-To: References: Message-ID: On Mon, 29 Nov 2021 17:00:24 GMT, Julia Boes wrote: >> This change introduces jwebserver, a dedicated JDK tool for the Simple Web Server. >> >> A description is provided in a follow-up comment. > > Julia Boes has updated the pull request with a new target base due to a merge or a rebase. The pull request now contains 16 commits: > > - Merge branch 'master' into jwebserver > - update comment about exit code > - add toolGuide tag to SimpleFileServer class-level documentation > - Merge branch 'master' into jwebserver > - fix whitespace error, add missing code tag > - address PR comments: > * update module and package summary > * add -version / --version option to usage > * fix markdown formatting > - update module summary > - Merge branch 'master' into jwebserver > - add version option, consolidate tests, update module and package level doc > - drop "tool" in man page > - ... and 6 more: https://git.openjdk.java.net/jdk/compare/825e633e...936c3d4a Build changes look good ------------- Marked as reviewed by ihse (Reviewer). PR: https://git.openjdk.java.net/jdk/pull/6497 From duke at openjdk.java.net Tue Nov 30 13:14:11 2021 From: duke at openjdk.java.net (Maxim Kartashev) Date: Tue, 30 Nov 2021 13:14:11 GMT Subject: RFR: 8277977: Incorrect references to --enable-reproducible-builds in docs In-Reply-To: <5J2jz7hRLDYAjO0tmjDPVwbAW-ZyyWCnfMKMZ80qRfI=.d60d2475-691c-4159-b284-8783cc8cd92b@github.com> References: <7k02muEGe6J8DNooPbaX3UR6fS3p4bv_9HOUYve-qiY=.fa13aed1-fc4f-4f3f-8bc0-f327f0bdb0d2@github.com> <5J2jz7hRLDYAjO0tmjDPVwbAW-ZyyWCnfMKMZ80qRfI=.d60d2475-691c-4159-b284-8783cc8cd92b@github.com> Message-ID: On Tue, 30 Nov 2021 12:48:48 GMT, Magnus Ihse Bursie wrote: >> Corrects spelling mistakes in the documentation and one configure script message. > > @mkartashev I can sponsor this, once you have done `/integrate`. @magicus Thanks for a quick review! ------------- PR: https://git.openjdk.java.net/jdk/pull/6609 From maxim.kartashev at jetbrains.com Tue Nov 30 13:16:25 2021 From: maxim.kartashev at jetbrains.com (Maxim Kartashev) Date: Tue, 30 Nov 2021 16:16:25 +0300 Subject: reproducible builds doc disagrees with code In-Reply-To: References: Message-ID: For the record, this was fixed as JDK-8277977 . On Wed, Nov 24, 2021 at 4:55 PM Maxim Kartashev < maxim.kartashev at jetbrains.com> wrote: > The description of reproducible builds in doc/building.html and > doc/building.md suggests to use the --enable-reproducible-builds option, > but in reality the option spelling is singular, not plural: > --enable-reproducible-build. There's also a reference to > --disable-reproducible-builds (again, plural) in the build scripts. I > wonder which is correct? > > make/autoconf/jdk-options.m4 > 697: UTIL_ARG_ENABLE(NAME: reproducible-build, DEFAULT: > $REPRODUCIBLE_BUILD_DEFAULT, > 705: AC_MSG_NOTICE([On Windows it is not possible to combine > --disable-reproducible-builds]) > > The doc commit: > https://github.com/openjdk/jdk/commit/59c3dcc761ac7a9eab1517743cbb77e5526ca6f3 > From darcy at openjdk.java.net Tue Nov 30 22:02:28 2021 From: darcy at openjdk.java.net (Joe Darcy) Date: Tue, 30 Nov 2021 22:02:28 GMT Subject: RFR: JDK-8273146: Start of release updates for JDK 19 [v6] In-Reply-To: References: Message-ID: > The time to get JDK 19 underway draws nigh, please review this usual set of start-of-release updates, including CSRs for the javac and javax.lang.model updates: > > JDK-8277512: Add SourceVersion.RELEASE_19 > https://bugs.openjdk.java.net/browse/JDK-8277512 > > JDK-8277514: Add source 19 and target 19 to javac > https://bugs.openjdk.java.net/browse/JDK-8277514 > > Clean local tier 1 test runs for langtools, jdk, and hotspot. Joe Darcy has updated the pull request with a new target base due to a merge or a rebase. The pull request now contains ten commits: - Merge branch 'master' into JDK-8273146 - Update symbol information for JDK 18 b25. - Merge branch 'master' into JDK-8273146 - Merge branch 'master' into JDK-8273146 - Remove SharedSpaces options from VMDeprecatedOptions.java. - Merge branch 'master' into JDK-8273146 - Respond to review feedback. - Add --release 18 information. Good tier1 test results. - JDK-8273146: Start of release updates for JDK 19 ------------- Changes: https://git.openjdk.java.net/jdk/pull/6493/files Webrev: https://webrevs.openjdk.java.net/?repo=jdk&pr=6493&range=05 Stats: 3286 lines in 68 files changed: 3242 ins; 4 del; 40 mod Patch: https://git.openjdk.java.net/jdk/pull/6493.diff Fetch: git fetch https://git.openjdk.java.net/jdk pull/6493/head:pull/6493 PR: https://git.openjdk.java.net/jdk/pull/6493