April 2020 - core-libs-dev

Method references in annotations
by Julian Ruppel 04 Apr '24

04 Apr '24

Hi, being aware that there has been a discussion quite some time ago ( http://mail.openjdk.java.net/pipermail/lambda-dev/2011-August/003768.html) I wanted to come up again with this topic as this feature would be beneficial especially for framework developers who wants to appoint behaviour to annotations. The following snippet shows what I want to vote for. @MyAnnotation(method = MyClass::myMethod)private String myVariable; This would be far more type-safe compared to using string literals to find the method via reflection. - Julian

2 2

Re: [jpackage] Issue with upgrading from javapackager to jpackage on Windows
by James Elliott 29 Oct '20

29 Oct '20

Sorry, tried sending this with the incorrect email personality first. On Mon, 17 Feb 2020 15:32:14 +0100 Daniel Peintner <daniel.peintner(a)gmail.com <mailto:daniel.peintner@gmail.com>> wrote: > The only *working* solution we found was to *manually* uninstall the old > application before starting msi installer. > Any future update with msi installers work fine. But the first update fails! I had been previously manually creating WiX installers (with the help of a script built by one of my Windows-based users), and he discovered the same problem when I migrated to jpackage. It turns out that the jpackage-built WiX installers are missing an upgrade ID. I mentioned that on this list a few weeks ago, but there was not any interest in investigating the issue. So now I use jpackage to just create the application image, and then separately use WiX directly to create an installer that can properly upgrade older installations. You can find the GitHub Actions workflow that drives the process here: https://github.com/Deep-Symmetry/beat-link-trigger/blob/master/.github/work… <https://github.com/Deep-Symmetry/beat-link-trigger/blob/master/.github/work…> The PowerShell script that runs jpackage and WiX is here: https://github.com/Deep-Symmetry/beat-link-trigger/blob/master/.github/scri… <https://github.com/Deep-Symmetry/beat-link-trigger/blob/master/.github/scri…> And the WIX template it uses is here: https://github.com/Deep-Symmetry/beat-link-trigger/blob/master/.github/reso… <https://github.com/Deep-Symmetry/beat-link-trigger/blob/master/.github/reso…> Hopefully this can help you, and perhaps jpackage can be updated to do this on its own as well. Cheers, -James

4 8

RFR 8214761 : Bug in parallel Kahan summation implementation
by Ivan Gerasimov 03 Sep '20

03 Sep '20

Hello! DoubleSummaryStatistics takes advantage of Kahan summation algorithm to reduce the error of the total sum. Internally it maintains a field double sumCompensation, which keeps lower bits (which were rounded off) of the last addition. Note, that the compensation has to be subtracted from the result to add those bits back: 166 private void sumWithCompensation(double value) { 167 double tmp = value - sumCompensation; 168 double velvel = sum + tmp; // Little wolf of rounding error 169 sumCompensation = (velvel - sum) - tmp; 170 sum = velvel; 171 } At the line 169, tmp normally has more lower bits than (velvel - sum). However, when two DoubleSummaryStatistics objects are combined, this compensation part is *added* to the total, which may result in a less accurate result. The same bug is replicated in DoubleStreams. Would you please help review the fix? BUGURL: https://bugs.openjdk.java.net/browse/JDK-8214761 WEBREV: http://cr.openjdk.java.net/~igerasim/8214761/00/webrev/ -- With kind regards, Ivan Gerasimov

3 4

RFR: 8188055: (ref) Add Reference.refersTo predicate
by Kim Barrett 02 Sep '20

02 Sep '20

[Note review on both core-libs and hotspot-gc-dev lists; try not to lose either when replying.] Please review a new function: java.lang.ref.Reference.refersTo. This function is needed to test the referent of a Reference object without artificially extending the lifetime of the referent object, as may happen when calling Reference.get. Some garbage collectors require extending the lifetime of a weak referent when accessed, in order to maintain collector invariants. Lifetime extension may occur with any collector when the Reference is a SoftReference, as calling get indicates recent access. This new function also allows testing the referent of a PhantomReference, which can't be accessed by calling get. The new function uses a native method whose implementation is in the VM so it can use the Access API. It is the intent that this function will be intrinsified by optimizing compilers like C2 or graal, but that hasn't been implemented yet. Bear that in mind before rushing off to change existing uses of Reference.get. CR: https://bugs.openjdk.java.net/browse/JDK-8188055 https://bugs.openjdk.java.net/browse/JDK-8241029 (CSR) Webrev: https://cr.openjdk.java.net/~kbarrett/8188055/open.04/ Testing: mach5 tier1 Locally (linux-x64) verified the new test passes with various garbage collectors.

9 20

RFR: JDK-8236138: Add tests for jmod applications
by Alexey Semenyuk 10 Aug '20

10 Aug '20

Please review fix [2] for jpackage bug [1]. The fix updates jtreg tests by adding test coverage for scenarios when jpackage is used for packaging applications bundled in .jmod files. - Alexey [1] https://bugs.openjdk.java.net/browse/JDK-8236138 [2] http://cr.openjdk.java.net/~asemenyuk/8236138/webrev.00

8 104

[15] 8235792: LineNumberReader.getLineNumber() behavior is inconsistent with respect to EOF
by Brian Burkhalter 20 Jul '20

20 Jul '20

This issue [1] attempts to reinstate the change for [2] which was backed out to fix [3] thereby maintaining longstanding behavior. The proposed change [4] modifies the specification to increment the line number also when EOF is encountered and is not immediately preceded by a line terminator. If this all seems reasonable then a CSR will be filed after the code review. One thing I am uncertain about is the change to skip() at lines 293-295 of LineNumberReader. I am not sure that those lines should be added. Thanks, Brian [1] https://bugs.openjdk.java.net/browse/JDK-8235792 [2] https://bugs.openjdk.java.net/browse/JDK-8230342 [3] https://bugs.openjdk.java.net/browse/JDK-8235668 [4] http://cr.openjdk.java.net/~bpb/8235792/webrev.00/

3 12

RFR JDK-8223347 Integration of Vector API (Incubator): Java API, implementation, and tests
by Paul Sandoz 13 Jul '20

13 Jul '20

Hi, A prior email sent out a request for review of the Vector API in preparation for JEP 338: Vector API (Incubator) [1] to be proposed for target: https://mail.openjdk.java.net/pipermail/core-libs-dev/2020-March/065345.html <https://mail.openjdk.java.net/pipermail/core-libs-dev/2020-March/065345.html> This email expands the review of the API to the Java implementation and Java tests: http://cr.openjdk.java.net/~psandoz/panama/JDK-8223347-vector-api-integrati… <http://cr.openjdk.java.net/~psandoz/panama/JDK-8223347-vector-api-integrati…> http://cr.openjdk.java.net/~psandoz/panama/JDK-8223347-vector-api-integrati… <http://cr.openjdk.java.net/~psandoz/panama/JDK-8223347-vector-api-integrati…> (Further emails will sent for review of general HotSpot changes CPU architecture specific HotSpot changes, x64 and aarch64, and performance tests. For an early peek see the links in the description of JDK-8223347.) — The Vector API provides - the public Vector class with vector operations common to all supported primitive types - public primitive specializations, such as IntVector, with common operations associated with the primitive type - internal concrete specializations for the vector size, such as Int256Vector, for a vector holding 8 ints. Operations generally defer to one of approximately 20 vector intrinsics. Some operations may be composed of other operations. Explicit casts are performed by vector operations to ensure vectors arguments are of the required shape (bit size) and element type. A vector intrinsic is an internal low-level vector operation. The last argument to the intrinsic is fall back behavior in Java, implementing the scalar operation over the number of elements held by the vector. Thus, If the intrinsic is not supported in C2 for the other arguments then the Java implementation is executed (the Java implementation is always executed when running in the interpreter or for C1). The public primitive specializations and the internal concrete implementations are generated from SSP template files, X-Vector.java.template and X-VectorBits.java.template respectively. Overall the implementation approach is quite formulaic and rather repetitive. Once you grok the pattern It should be easier to review if a little laborious. Unit tests are auto-generated by composing templates for each operation into an SSP template file which is then used to generate unit test files for each concrete vector class. The tests are quite extensive and have found many a HotSpot issue in development across a wide range of platforms. Paul. [1] https://bugs.openjdk.java.net/browse/JDK-8201271 <https://bugs.openjdk.java.net/browse/JDK-8201271>

5 14

Re: 8202469 / 8202473: Correct type annotation resolution for class type variables
by Joel Borggrén-Franck 07 Jun '20

07 Jun '20

Hi Rafael, Thanks for reaching out and reminding me of this! I managed to find some time to look into 8202469 during the weekend. By swapping place of the TVars in the test I managed to isolate this without depending on 8202473, take a look at my hacky copy-paste update to your test at http://cr.openjdk.java.net/~jfranck/8202469/ . The comment on lines 269-276 needs to be updated. Perhaps include a table with the start index depending of the type? Also referencing JVMLS 4.4 for the structure of the bound might be instructive for future readers. My current understanding is that there are two problems with the code on (the original) lines 277-287: 1) Type Variables should have index 0 while getting index 1 due to lines 279-280. 2) Bounds that are instances of ParameterizedType always gets index 1 but if the first bound represents a Class type the index should be 0. Does this make sense? Can you make the if-switches positive? I find my original code with the negative tests hard to read and the update doesn't help. Also can you expand the test to cover the different kinds of bounds mentioned in 4.4 and also test Type Variables on methods, I suspect javac treats method (and constructor) tvars similarly but there have been bugs ... TL;DR please update the webrev with: - Split out test and fix for 8202469 - More test coverage of different kinds of bounds - Test for method tvars - See if you can rework the logic to have (mostly) positive tests in if switch Thanks again for looking into this, I'll start looking into 8202473, I think the fix for that one opens up a bigger rework of the code which is why I want to separate the two. cheers /Joel On Sun, Aug 4, 2019 at 10:12 PM Rafael Winterhalter <rafael.wth(a)gmail.com> wrote: > Hi, > > appologies for the delay, I only managed to get my patched up to webrev > today (http://cr.openjdk.java.net/~winterhalter/) It's three patches in > total now, for the third one I could not add a test case, it would require > to compile an annotation property against an enumeration type and load it > against another class where the enumeration was turned into an annotation. > I struggled a bit with jtreg to make it work but I cannot see myself > complete this in a meaningful amount of time. However, I hope that the > patch and the error it solves are straightforward to see. > > I only submitted a patch for 8202469. 8202473 is solved by it. It's two > bugs but they are a symptom of the same oversight. > > I hope this helps you to review it but let me know if you have any > questions or if I should further adjust my patch. > > Best regards, Rafael > > Am Fr., 2. Aug. 2019 um 12:18 Uhr schrieb Rafael Winterhalter < > rafael.wth(a)gmail.com>: > >> Thanks Daniel, >> I did some work on Glassfish a bunch of years ago, I had no idea. >> I will try to split up the changes (I fixed another bug in annotation >> processing yesterday) and upload everything there. >> Best regards, Rafael >> >> Am Fr., 2. Aug. 2019 um 11:59 Uhr schrieb Daniel Fuchs < >> daniel.fuchs(a)oracle.com>: >> >>> Hi Rafael, >>> >>> On 02/08/2019 09:00, Joel Borggrén-Franck wrote: >>> > I can host webrevs for you on cr to make it easier for other >>> reviewers, if >>> > you also send me the patches or webrefs off-list to get around the >>> > attachment stripping I can upload them to cr. >>> >>> I see that you are a JDK project author, so you already own a page >>> on cr (http://cr.openjdk.java.net/~winterhalter/) where you can >>> upload webrevs (e.g. using `scp` as in: >>> $ scp -r webrev-NNNNN.01 winterhalter(a)cr.openjdk.java.net: ) >>> >>> best regards and welcome! >>> >>> -- daniel >>> >>

2 6

RFR 8243491: Implementation of Foreign-Memory Access API (Second Incubator)
by Maurizio Cimadamore 25 May '20

25 May '20

Hi, time has come for another round of foreign memory access API incubation (see JEP 383 [3]). This iteration aims at polishing some of the rough edges of the API, and adds some of the functionalities that developers have been asking for during this first round of incubation. The revised API tightens the thread-confinement constraints (by removing the MemorySegment::acquire method) and instead provides more targeted support for parallel computation via a segment spliterator. The API also adds a way to create a custom native segment; this is, essentially, an unsafe API point, very similar in spirit to the JNI NewDirectByteBuffer functionality [1]. By using this bit of API, power-users will be able to add support, via MemorySegment, to *their own memory sources* (e.g. think of a custom allocator written in C/C++). For now, this API point is called off as "restricted" and a special read-only JDK property will have to be set on the command line for calls to this method to succeed. We are aware there's no precedent for something like this in the Java SE API - but if Project Panama is to remain true about its ultimate goal of replacing bits of JNI code with (low level) Java code, stuff like this has to be *possible*. We anticipate that, at some point, this property will become a true launcher flag, and that the foreign restricted machinery will be integrated more neatly into the module system. A list of the API, implementation and test changes is provided below. If you have any questions, or need more detailed explanations, I (and the rest of the Panama team) will be happy to point at existing discussions, and/or to provide the feedback required. Thanks Maurizio Webrev: http://cr.openjdk.java.net/~mcimadamore/8243491_v1/webrev Javadoc: http://cr.openjdk.java.net/~mcimadamore/8243491_v1/javadoc Specdiff: http://cr.openjdk.java.net/~mcimadamore/8243491_v1/specdiff/overview-summar… CSR: https://bugs.openjdk.java.net/browse/JDK-8243496 API changes =========== * MemorySegment - drop support for acquire() method - in its place now you can obtain a spliterator from a segment, which supports divide-and-conquer - revamped support for views - e.g. isReadOnly - now segments have access modes - added API to do serial confinement hand-off (MemorySegment::withOwnerThread) - added unsafe factory to construct a native segment out of an existing address; this API is "restricted" and only available if the program is executed using the -Dforeign.unsafe=permit flag. - the MemorySegment::mapFromPath now returns a MappedMemorySegment * MappedMemorySegment - small sub-interface which provides extra capabilities for mapped segments (load(), unload() and force()) * MemoryAddress - added distinction between *checked* and *unchecked* addresses; *unchecked* addresses do not have a segment, so they cannot be dereferenced - added NULL memory address (it's an unchecked address) - added factory to construct MemoryAddress from long value (result is also an unchecked address) - added API point to get raw address value (where possible - e.g. if this is not an address pointing to a heap segment) * MemoryLayout - Added support for layout "attributes" - e.g. store metadata inside MemoryLayouts - Added MemoryLayout::isPadding predicate - Added helper function to SequenceLayout to rehape/flatten sequence layouts (a la NDArray [4]) * MemoryHandles - add support for general VarHandle combinators (similar to MH combinators) - add a combinator to turn a long-VH into a MemoryAddress VH (the resulting MemoryAddress is also *unchecked* and cannot be dereferenced) Implementation changes ====================== * add support for VarHandle combinators (e.g. IndirectVH) The idea here is simple: a VarHandle can almost be thought of as a set of method handles (one for each access mode supported by the var handle) that are lazily linked. This gives us a relatively simple idea upon which to build support for custom var handle adapters: we could create a VarHandle by passing an existing var handle and also specify the set of adaptations that should be applied to the method handle for a given access mode in the original var handle. The result is a new VarHandle which might support a different carrier type and more, or less coordinate types. Adding this support was relatively easy - and it only required one low-level surgery of the lambda forms generated for adapted var handle (this is required so that the "right" var handle receiver can be used for dispatching the access mode call). All the new adapters in the MemoryHandles API (which are really defined inside VarHandles) are really just a bunch of MH adapters that are stitched together into a brand new VH. The only caveat is that, we could have a checked exception mismatch: the VarHandle API methods are specified not to throw any checked exception, whereas method handles can throw any throwable. This means that, potentially, calling get() on an adapted VarHandle could result in a checked exception being thrown; to solve this gnarly issue, we decided to scan all the filter functions passed to the VH combinators and look for direct method handles which throw checked exceptions. If such MHs are found (these can be deeply nested, since the MHs can be adapted on their own), adaptation of the target VH fails fast. * More ByteBuffer implementation changes Some more changes to ByteBuffer support were necessary here. First, we have added support for retrieval of "mapped" properties associated with a ByteBuffer (e.g. the file descriptor, etc.). This is crucial if we want to be able to turn an existing byte buffer into the "right kind" of memory segment. Conversely, we also have to allow creation of mapped byte buffers given existing parameters - which is needed when going from (mapped) segment to a buffer. These two pieces together allow us to go from segment to buffer and back w/o losing any information about the underlying memory mapping (which was an issue in the previous implementation). Lastly, to support the new MappedMemorySegment abstraction, all the memory mapped supporting functionalities have been moved into a common helper class so that MappedMemorySegmentImpl can reuse that (e.g. for MappedMemorySegment::force). * Rewritten memory segment hierarchy The old implementation had a monomorphic memory segment class. In this round we aimed at splitting the various implementation classes so that we have a class for heap segments (HeapMemorySegmentImpl), one for native segments (NativeMemorySegmentImpl) and one for memory mapped segments (MappedMemorySegmentImpl, which extends from NativeMemorySegmentImpl). Not much to see here - although one important point is that, by doing this, we have been able to speed up performances quite a bit, since now e.g. native/mapped segments are _guaranteed_ to have a null "base". We have also done few tricks to make sure that the "base" accessor for heap segment is sharply typed and also NPE checked, which allows C2 to speculate more and hoist. With these changes _all_ segment types have comparable performances and hoisting guarantees (unlike in the old implementation). * Add workarounds in MemoryAddressProxy, AbstractMemorySegmentImpl to special case "small segments" so that VM can apply bound check elimination This is another important piece which allows to get very good performances out of indexes memory access var handles; as you might know, the JIT compiler has troubles in optimizing loops where the loop variable is a long [2]. To make up for that, in this round we add an optimization which allows the API to detect whether a segment is *small* or *large*. For small segments, the API realizes that there's no need to perform long computation (e.g. to perform bound checks, or offset additions), so it falls back to integer logic, which in turns allows bound check elimination. * renaming of the various var handle classes to conform to "memory access var handle" terminology This is mostly stylistic, nothing to see here. Tests changes ============= In addition to the tests for the new API changes, we've also added some stress tests for var handle combinators - e.g. there's a flag that can be enabled which turns on some "dummy" var handle adaptations on all var handles created by the runtime. We've used this flag on existing tests to make sure that things work as expected. To sanity test the new memory segment spliterator, we have wired the new segment spliterator with the existing spliterator test harness. We have also added several micro benchmarks for the memory segment API (and made some changes to the build script so that native libraries would be handled correctly). [1] - https://docs.oracle.com/en/java/javase/14/docs/specs/jni/functions.html#new… [2] - https://bugs.openjdk.java.net/browse/JDK-8223051 [3] - https://openjdk.java.net/jeps/383 [4] - https://docs.scipy.org/doc/numpy/reference/generated/numpy.reshape.html#num…

8 52

6202130: Need to handle UTF-8 values and break up lines longer than 72 bytes
by Philipp Kunz 18 May '20

18 May '20

Hi, The specification says, a line break in a manifest can occur beforeor after a Unicode character encoded in UTF-8. > ...> value: SPACE *otherchar newline *continuation> continuation: SPACE *otherchar newline> ...> otherchar: any UTF-8 character except NUL, CR and LF The current implementation breaks manifest lines at 72 bytes regardless ofhow the bytes around the break are part of a sequence of bytes encoding acharacter. Code points may use up to four bytes when encoded in UTF-8.Manifests with line breaks inside of sequences of bytes encoding Unicodecharacters in UTF-8 with more than one bytes not only are invalid UTF-8but also look ugly in text editors.For example, a manifest could look like this: import java.util.jar.Manifest;import java.util.jar.Attributes;import static java.util.jar.Attributes.Name; public class CharacterBrokenDemo1 { public static void main(String[] args) throws Exception{ Manifest mf = new Manifest(); Attributes attrs = mf.getMainAttributes(); attrs.put(Name.MANIFEST_VERSION, "1.0"); attrs.put(new Name("Some-Key"), "Some languages have decorated characters, " + "for example: español"); // or "espa\u00D1ol" mf.write(System.out); }} Above code produces a result as follows with some unexpected question markswhere the encoding is invalid: > Manifest-Version: 1.0> Some-Key: Some languages have decorated characters, for example: espa?> ?ol This is of course an example written with actual question marks to get a validtext for this message. The trick here is that "Some-Key" to "example :espa"amounts to exactly one byte less encoded in UTF-8 than would fit on one linewith the 72 byte limit so that the subsequent character encoded with two bytesgets broken inside of the sequence of two bytes for this character across acontinuation line break. When decoding the resulting bytes from UTF-8 as one whole string, the twoquestion marks will not fit together again even if the line break with thecontinuation space is removed. However, Manifest::read removes the continuationline breaks ("\r\n ") before decoding the manifest header value from UTF-8 andhence can reproduce the original value. Characters encoded in UTF-8 can not only span up to four bytes for one codepoint each, there are also combining characters or classes thereof or combiningdiacritical marks or whatever the appropriate term could be, that combine morethan one code point into what is usually experienced and referred to as acharacter. The term character really gets ambiguous at this point. I wouldn't know whatthe specification actually refers to with that term "character". So rather thandiving in too much specification or any sorts of theory, let's look at anotherexample: import java.util.jar.Manifest;import java.util.jar.Attributes;import static java.util.jar.Attributes.Name; public class DemoCharacterBroken2 { public static void main(String[] args) throws Exception{ Manifest mf = new Manifest(); Attributes attrs = mf.getMainAttributes(); attrs.put(Name.MANIFEST_VERSION, "1.0"); attrs.put(new Name("Some-Key"), " ".repeat(53) + "Angstro\u0308m"); mf.write(System.out); }} which produces console output as follows: > Manifest-Version: 1.0> Some- Key: Angstro> ̈m (In case this does not display well, the diaeresis is on the m on the last line) When the whole Manifest is decoded from UTF-8 as one big single string andcontinuation line breaks are not removed until after UTF-8 decoding the wholemanifest, the diaeresis (umlaut, two points above, u0308) apparently kind ofjumps onto the following letter m because somehow it cannot be combined withthe preceding space. The UTF-8 decoder (all of my editors I tried, not onlyEclipse and its console view, also less, gedit, cat and terminal) somehowtries to fix that but the diaeresis may not necessarily jump back on the "o"where it originally belonged by removing the continuation line break ("\r\n ")after UTF-8 decoding has taken place, at least that did not work for me. Hence, ideally combining diacritical marks should better not be separated fromwhatever they combine with when breaking manifest lines onto a continuationline. Such combinations, however, seem to be unlimited in terms of number ofcode points combining into the same "experienced" character. I was able tofind combinations that not only exceed the limit of 72 bytes per line but alsoexceed the line buffer size of 512 bytes in Manifest::read. These may be ratheruncommon but still possible to my own surprise. Next consideration would then be to remove that limit of 512 bytes per manifestline but exceeding it would make such manifests incompatible with previousManifest::read implementations and is not really an immediately availableoption at the moment. As a compromise, those characters including combining diacritical marks whichcombine only so many code points as that their binarily encoded form in UTF-8remains within a limit of 71 bytes can be written without an interruptingcontinuation line break, which applies to most cases, but not all. I guess thisshould suit practically and realistically to be expected values well. Another possibility would be to allow for characters that are combinations ofmultiple Unicode code points to be kept together in their encoded form in UTF-8up to 512 bytes line length limit when reading minus a space and a line breakamounting to 509 bytes, but that would still not make manifests be representedas valid Unicode in all corner cases and I guess would not probably make a realimprovement in practice over a limit of 71 bytes. Attached is a patch that tries to implement what was described above using aBreakIterator. While it works from a functional point of view, this might beless desirable performance-wise. Alternatively could be considered to do withoutthe BreakIterator and only keep Unicode code points together by not placingline breaks before a continuation byte, which however would not addresscombining diacritical marks as in the second example above. The jar file specification does not explicitly state that manifest should bevalid UTF-8, and they were not always, but it also does not state otherwise,leaving an impression that manifests could be (mis)taken for UTF-8 encodedstrings, which they also are in many or most cases and which has been confusedmany times. At the moment, the only case where a valid manifest is not also avalid UTF-8 encoded string is when a sequence of bytes encoding the samecharacter happens to be interrupted with a continuation line break. To the bestof my knowledge, all other valid manifests are also valid UTF-8 encoded strings. It would be nice, if manifests could be viewed and manipulated with all Unicodecapable editors and not only be parsed correctly with Manifest::read. Any opinions? Would someone sponsor this patch? Regards,Philipp https://docs.oracle.com/en/java/javase/13/docs/specs/jar/jar.html#specifica…

4 16